|
Description:
|
|
Sr Data Engineer (Hybrid, 3 days onsite)
Client will be requesting onsite Interviews - make sure candidates can commit to completing this
Must Haves:
- 7+ years' experience in python programming experience in complex data structures as well as data pipeline development.
- 5+ years' experience in python libraires Airflow, Pandas, PySpark, Redis, SQL (or similar libraries).
- 5+ Strong SQL programming experience in Mid/Advance functions like Aggregate Functions (SUM, AVG), Conditional Functions (CASE WHEN ,NULLIF), Mathematical Functions (ROUND, ABS), Ranking Functions (RANK) and Windowing Functions etc.
- Strong Proficiency in Python (Programming Language), Apache Airflow and Google Cloud Data Services.
Overview:
- Perform and oversee data loading operations
- Optimize data for extraction and reporting use
- Manage complicated databases by performing suitable database management functions
- Build, maintain, monitor, and orchestrate workflows or data pipelines
- Ensure the high performance of data retrieval processes
Qualifications:
- 7+ years' experience in python programming experience in complex data structures as well as data pipeline development.
- 5+ years' experience in python libraires Airflow, Pandas, PySpark, Redis, SQL (or similar libraries).
- 5+ Strong SQL programming experience in Mid/Advance functions like Aggregate Functions (SUM, AVG), Conditional Functions (CASE WHEN ,NULLIF), Mathematical Functions (ROUND, ABS), Ranking Functions (RANK) and Windowing Functions etc.
- Strong Proficiency in Python (Programming Language), Apache Airflow and Google Cloud Data Services.
- Nice to have: 3+ years' experience in Dataflow, dataproc, Cloud Composer, Big Query, Cloud Storage, GKE, etc.
- Nice to have: Alternative to Google Cloud, data pipeline development using python for 3+ years for any other cloud platform can be considered.
|