Your Role

We are seeking a highly skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have a deep understanding of data engineering principles, data technologies, and a proven track record of designing and building complex data pipelines. This role requires strong expertise in SQL, various data formats, Python, and JavaScript to support our data-driven decision-making processes and enhance our data infrastructure.
• Architect and maintain scalable data pipelines using various programming technologies.
• Use SQL to query, transform, and process data across relational and NoSQL databases.
• Integrate data from APIs, flat files, and streaming sources for consistency and quality.
• Implement real-time data processing using Kafka or Solace.
• Manage data storage in systems and warehouses, optimizing for performance.
• Design data models and apply techniques like partitioning and indexing for efficiency.
• Handle multiple data formats (CSV, JSON, Parquet) and manage unstructured data.
• Utilize Python and JavaScript (Node.js) for data processing, automation, and ETL development.
• Leverage Microsoft technologies, Apache Spark, and Airflow for distributed computing.
• Implement DevOps tools (Jenkins, Git, Docker) for CI/CD and monitor pipeline performance.

Your Profile

• Bachelor’s or Master’s degree in Computer Science, Data Science, IT, or related field.
• 8+ years of experience in data engineering, architecture, or related roles.
• Proven track record in building and maintaining large-scale, complex data pipelines.
• Expertise in managing data infrastructure in high-volume environments.
• Strong foundation in designing and optimizing data systems and workflows.