Ignitec infuses industry standards and leading technology capabilities to solve complex problems and deliver value with increased quality and lower performance risks. Our solutions combine top technology personnel, the latest cutting-edge technology, and Agile approaches to bring innovative ideas to life. We do not seek to meet expectation, we continuously strive to exceed them.
We have received our MBE Certification from NMSDC as a certified Minority Small Business Enterprise. We take pride in the MBE certification and partner with organizations to meet their Minority (D&I) Small Business goals. We are also a certified Minority Business Enterprise by the USPAACC, which recently awarded Ignitec The FAST 50 Asian American Business Award in 2022. We are also DBE certified by the Virginia Department of SBSD.
Job Title: Sr. Data Engineer
Work Status: US Citizen, Green Card, H1B eligible
Location: 100% onsite in Denver, CO
Overview:
Seeking a Data Engineer to join the Data Lake Team that can independently lead data engineering projects and strive to proactively improve process efficiency, making recommendations for process and system improvements where applicable. The Data Engineer role will be responsible for not only understanding data pipelines but, event streaming applications, and how to build systems that handle massive amounts of data while making it consumable by other application teams, users and data scientists.
The role will be assisting in the design and architecture of highly scalable, fault tolerant infrastructure capable of processing millions of operations per minute coming from millions of TVs, efficiently store petabytes of data and provide fast insights from the data. Modernizing Big Data ecosystem, monitor data quality for cleanliness and fix discrepancies. Ensure data accuracy through validation tasks, perform root cause analysis and implement solutions for data prep and cleanliness, review data at all granular/aggregate levels, and versioning.
Qualifications:
- BS or MS in Computer Science or similar relevant field
- Experienced engineering with 5+ years of experience
- Passion for big data structures
- Possess strong organizational and analytical skills related to working with structured and unstructured data operations
- Experience implementing and maintaining high performance / high availability data structures
- Comfortable operating within cloud based eco systems
Requirements:
- Experience or knowledge of relational SQL and NoSQL databases, including Postgres and Cassandra.
- Strong understanding of in-memory processing and data formats (Avro, Parquet, Json etc.)
- Experience or knowledge of AWS cloud services: EC2, MSK, S3, RDS, SNS, SQS
- Experience or knowledge of stream-processing systems: i.e., Storm, Spark-Structured-Streaming, Kafka consumers.
- Experience or knowledge of object-oriented/object function scripting languages: i.e., Python, Java, Scala, R, SQL.
- Experience or knowledge of data pipeline and workflow management tools: i.e., AWS Data Pipeline, Apache Airflow, Argo.
- Experience or knowledge of big data tools: i.e., Hadoop, Spark, Kafka.
- Experience or knowledge of software engineering tools/practices: i.e., Github, VSCode, CI/CD
- Hands-on experience in designing and maintaining data schema life-cycles.