We are currently seeking a Senior GCP Data Engineer to join our technology-driven team.
Job Title: GCP Data Engineer
Location: Irving, TX
Client: Prodapt
Project Duration: 12 months Contract
Interview: Three rounds of Video interview
Job Description:
We are seeking a highly motivated and experienced Senior GCP Data Engineer to join our growing team in Irving, TX. In this role, you will be responsible for designing, building, and maintaining cutting-edge data pipelines and solutions on the Google Cloud Platform (GCP). You will play a crucial role in enabling data-driven decision-making and powering innovative products and services.
Responsibilities:
Design, develop, and deploy robust and scalable data pipelines using GCP services such as BigQuery, Dataflow, Cloud Composer (Airflow), Pub/Sub, DataProc, and related technologies.
Experience in building high-performing data processing frameworks leveraging Google Cloud Platform.
Experience in building data pipelines supporting both batch and real-time streams to enable data collection, storage, processing, transformation and aggregation.
Familiarity with the Technology stack available in the industry for data management, data ingestion, capture, processing and curation
ETL development experience with a strong SQL background, analysing huge data sets, trends and issues, and creating structured outputs.
Understand ETL application design, data sources, data targets, relationships, and business rules.
Process and transform large datasets from various sources, including real-time & batch streaming data.
Optimize data pipelines for performance, cost-efficiency, and reliability.
Build and maintain data warehousing solutions on GCP, ensuring data quality and accessibility.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver impactful solutions.
Qualifications:
15+ years of experience in data engineering.
Strong proficiency in SQL and data modeling.
Hands-on experience with big data technologies such as Hadoop, Spark, and Kafka.
Proficiency in Python or Java for data processing and pipeline development.
Experience with data warehousing and ETL processes.
Experience with CI/CD pipelines and infrastructure-as-code tools (e.g., Terraform).
Experience with data visualization tools (e.g., Looker, Tableau).