Drag

Data Engineer

Location : ,

Job Description

 

Data Engineer 

Assigned To

 

Job Description

The primary focus of this role will be to support internal organizations in implementing and delivering their strategy and results. The Data Engineer is responsible for building, managing, and optimizing reusable enterprise data pipelines effectively and in a timely manner through the development lifecycle to be used by internal consumers, such as business/data analysts and data scientists. The engineer would use both technical and analytical skills to understand and solve business problems using available resources and current technology stack, while following data governance and data security guidelines as communicated by RCG in writing.

Required Skills

1. Create, build, and maintain data pipelines from disparate sources that meet functional / non-functional business requirements
2. Create, maintain and reuse existing ETL processes, employing a variety of data integration and data preparation tools
3. Experienced in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional and new data integration technologies (such as ETL, ELT, data replication, change data captures, message-oriented data movement, API design, stream data integration and data virtualization)
5. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing pipelines for greater scalability, etc.
6. Work with stakeholders including Product, Data and Business teams to assist with data-related technical issues and support their data needs
7. Create datasets for operational reports, key performance indicators/metrics, or other insights into current organizational activities
8. Create datasets for analytics and data science to provide the ability to uncover the answers to major questions that help organizations make objective decisions and/or gain a competitive edge
9. Write, debug and implement moderately complex queries involving multiple tables or databases across platform(s)
10. Experience with streaming technologies (Kafka, Pubsub, Kinesis) and log-based architectures and experience writing batch and stream processing jobs (i.e. Apache Beam, Google Cloud DataFlow, Apache Spark, Apache Storm)
11. Create and maintain technical design documentation
12. Participate in requirements gathering
13. Collaborate with the Enterprise Architecture team to ensure alignment on data standards and processes
14. Highly analytical and interested in data visualization
15. Work with data and analytics experts to strive for greater functionality in data systems
16. Position requires on-call and off-hours support

Required Education

Minimum level of education and experience required to perform this job:
1. Bachelor of Science in Computer Science, Information Technology or equivalent
2. 3+ years of experience in a data/cloud engineering role
3. 3+ years of experience working and creating datasets for a data warehouse
4. Clear understanding of data modeling patterns (relational and dimensional)
5. 3+ years of experience with ETL development tools, Informatica or Azure Data Factory (ADF) preferred
6. 3+ years of cloud experience, Azure preferred
7. Working knowledge and experience in using best practices in designing, building, and managing data pipelines that require data transformations as well as metadata and workload management
8. Working knowledge and experience in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional and new data integration technologies (such as ETL, ELT, data replication, change data captures, message-oriented data movement, API design, stream data integration and data virtualization)
9. Working knowledge and experience in performing root cause analysis on internal and external data and processes to identify issues and opportunities for improvement
10. Experience supporting and working with cross-functional teams in a dynamic environment
11. Demonstrate a certain degree of creativity with analytical and problem-solving skills
12. Display curiosity in understanding the data for the specific area of responsibility
Specific knowledge and skills necessary to perform this job related to the Essential Duties and Responsibilities identified above. Technical capabilities in:
1. Ability to read and write SQL, PL/SQL, and/or T-SQL
1. Hands on experience with scripting languages, such as Python, R, Java, Scala, etc.
2. Knowledge of data storage concepts, such as databases, data warehouses, and data lakes
a. Relational SQL databases, such as Oracle and SQL Server
b. NoSQL databases are a plus
3. Proven ability to collaborate with technical peers
4. Capable of working independently and as part of a team
5. Experience with and understanding of continuous integration and continuous deployment practices
6. Agile

Required Years of Experience

2+