Sr Data Engineer

Job Description

Senior Data Engineer

Your Day-to-Day:

Design and develop highly scalable and reliable data engineering pipelines to process large volumes of data across many data sources in the cloud

Identify, design and implement internal process improvements by automating manual processes and optimizing data delivery

Develop and promote best practices in data engineering

Be part of the on call rotation supporting our SLA's

Key Projects:

Develop data pipelines for processing data from internal and external sources

Migration of on-premise ETL processes to Python and Airflow

Develop a process to capture different data points required for a reliable data quality system

Qualifications:

What you bring to the team:

You have –

Bachelor's Degree in Computer Science or a related discipline

Minimum of 4+ years of applicable engineering experience

Strong proficiency in Python

Ability to write complex SQL to perform common types of analysis and aggregations

Strong knowledge of Application Programming Interfaces

Proven ability to clearly communicate complex solutions

Experience working on large scale data migration projects

Exceptional written and verbal communication skills

You might also have –

Experience with Cloud platforms

Experience with services like Google Dataflow , Google BigQuery, Pubsub, Apache airflow or similar

Experience with Docker containerization

Familiar with a NoSQL database such as MongoDB

Familiar with version control systems (Git and Bitbucket)

Can perform statistical analysis using tools such as R, Numpy/SciPy with Python

Familiar with Atlassian products Jira and Confluence

Experience with a Python web framework such as Django or Flask

Familiar with Apache Beam

Jobcode: Reference SBJ-gw91pp-18-190-153-51-42 in your application.