Sr Data Engineers

CCC Intelligent Solutions, Inc seeks Sr. Data Engineers

for various & unanticipated worksites throughout the U.S (HQ: Chicago, IL) to develop large scale end to end data pipeline applications, covering multiple data sources spread across data center & AWS cloud.

Required skills:

1. Hands-on experience with programming using Python & PySpark; Hadoop; HDFS, map-reduce, YARN, AWS EMR; Hive, HBase, parquet, ORC, Spark SQL, Sqoop, Apache Hudi; orchestrating ETL pipelines involving data sourcing, transformations & publishing using Apache Airflow; performance tuning applications in distributed computing environments designing & developing data pipeline applications with Apache Kafka;
2. Advanced SQL for data profiling & data validation;
3. Unix commands & scripting;
4. Performing root cause analysis on internal & external data & processes to identify opportunities for improvement;
5. JIRA, Gitlab, Subversion; Agile; development of metadata-driven & fully parameterized data processing tools; AWS.

Telecommuting permitted. Apply online: Req ID: 12890