Location : ,
Azure Data Engineer
Job Description:
Pyspark and Python are VERY important.
Must Have Qualifications:
• 5+ years of experience in building data and analytics platform focused on Azure data and analytics solutions
• 5+ years of programming experience in Python/Pyspark
• Good experience in Azure services such as Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake
• Knowledge of data modeling and data warehousing concepts
• Experience with cloud-based data migration
• ETL tools (e.g., SSIS, Azure Data Factory, Power BI Dataflow)
• Database management (MSSQL, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse)
• Understanding of capital markets within Fixed Income
• Microsoft Fabric experience
Day-to-Day Responsibilities:
• Design and implement data storage systems using Azure services including Azure SQL Database, Azure Data Lake Storage, and Azure Synapse
• Data Integration and ETL (Extract, Transform, Load).
• Develop and implement data integration processes using Azure Data Factory
• Extract data from various sources, transform it, and load it into data warehouses or data lakes
• Utilize big data technologies such as Apache Spark
• Create data processing workflows and pipelines to support data analytics and machine learning applications
• Monitor and optimize data pipelines and database performance to ensure data processing efficiency
• Build interfaces for supporting evolving and new applications and accommodating new data sources and types of data