Job Description

• Design, develop, and maintain scalable and efficient data pipelines, ETL processes, and data integration solutions.
• Collaborate with cross-functional teams to gather data requirements, translate them into technical specifications, and develop data models.
• Implement and maintain CI/CD pipelines for automating the deployment and testing of data solutions.
• Optimize and tune data workflows and processes to ensure high performance and reliability.
• Monitor and troubleshoot data-related issues, perform root cause analysis, and implement corrective actions.
• Maintain documentation of data infrastructure, processes, and workflows.
• Stay up-to-date with industry trends and emerging technologies in data engineering and cloud computing.

Primary Skills

• Proficiency in at least one of the cloud technologies: Azure, AWS, or GCP.
• Minimum 4 years of relevant experience in data engineering, including strong experience with SQL, Python and PySpark.

Secondary Skills

• Experience in any job scheduling tool e.g. Control M, Autosys, Airflow Luigi would be a big plus
• Experience in complex ETL mappings, CI CD pipelines, dev ops and deployment tools e.g. Docker, Jenkins etc. would be a big plus