Location : ,
Title: Data Engineering Lead
Skills (EXPERT/ADVANCED/NONE):
Develop and optimize complex SQL queries for data extraction, transformation, and loading (ETL) processes.
Implement SQL procedures, with a focus on Slowly Changing Dimension (SCD) implementation.
Expertise in data warehousing principles with a deep understanding of facts and dimensions.
Familiarity with Data Vault architecture for scalable and flexible data modelling.
Strong Python development experience to design data pipelines in Glue.
Proven experience in designing and implementing SCD type 1 and 2
Proficient in working with the Snowflake data platform, ensuring the design and implementation of scalable data storage and processing solutions.
Familiar with AWS S3 Datalake and AWS Glue
Implement and manage data pipelines using Apache Airflow for scheduling, monitoring, and orchestrating ETL processes.
Description:
• Programming and CI/CD Skills:
o Utilize Python for data engineering tasks, automation, and scripting.
o Familiar with CI/CD framework
• Coalesce Data Transformation Tool (Good to have):
o Showcase expertise in using the Coalesce data transformation tool for efficient and scalable data processing and transformation.