ETL Architect

Experience: 12+

Duration: Long Term

Role & Responsibilities

• 12+ years of IT experience in Data Engineering, Data Quality, Data Migrations, Data Architecture, Data Lake formation, and Data Analytics.
• 5+ years of hands-on solid experience with AWS services like S3, EMR, VPC, EC2, IAM, EBS, RDS, Glue, Lambda, Lake Formation, etc.
• Must have produced architecture documents for small to large solution implementations.
• In-depth understanding of Spark Architecture including Spark Code, Spark SQL, DataFrames, Spark Streaming, Spark MLiB, etc. Experience handling very high-volume streaming data in various formats like JSON, XML, AVRO, Snappy, etc.
• Good exposure to Kafka for designing future capacity planning, partition planning, read and write.
• Must have worked with Big Data and possess good knowledge of MapReduce and Spark.
• Must have extensive working exposure to different types of databases like RDBMS, NoSQL, Columnar, Document, distributed databases, cloud databases, and in-memory databases.