Principal Data Engineer

Job Description

We are looking for a Staff Data engineer to wrangle data and build data pipelines from several data sources and transform this data into valuable & insightful information. The engineer will contribute to end-to-end platform design for our cloud architecture and work multi-functionally with Operations, data science and the business segments to build batch/real-time data solutions. In this role, the work will influence the company's data consumers and analysts.

Responsibilities:
• Serve as a Technical lead and key coding contributor to the team
• Lead teams and projects and mentor junior team members
• Design, develop and implement large-scale, high-volume, high-performance data lake and data warehouse.
• Build data pipelines to consume data from various data sources and create a unified/enterprise data model for analytics and reporting.
• Capture functional requirements and translate them into robust, scalable, operable solutions that work well within the overall data architecture.
• Participate and provide feedback on architecture reviews, development processes, etc.
• Participate in technology decisions and investments, architecture, tools and process

Basic Qualifications:
• Deep experience of relational, analytical, columnar databases (7+ years)
• Working experience with Hadoop/YARN, Spark, Hive, Streaming technologies (3+ years)
• Experience with workflow orchestration tools like Airflow (2+ years)
• Programming in Python or Scala. (3+ years)
• Experience with developing and deploying in a cloud native environment preferably AWS cloud. (3+ years)
• Proven strength in SQL, data modeling, ETL development, and data warehousing.
• Design solutions for structured and semi-structured data
• Experience with Git, Continuous Integration and related tools
• Good unit testing and integration testing skills
• Experience delivering in a team environment using Agile framework

Preferred Qualifications:
• Experience with Apache Kafka, AWS Kinesis, or Google Pub/Sub
• Experience with BI tools like Tableau etc.

Required Education
• Bachelor's degree – Computer Science or equivalent

Jobcode: Reference SBJ-d22545-18-220-66-151-42 in your application.