Data Engineer – Build Scalable Data Pipelines

About the role

We are looking for a Data Engineer to design, develop, and maintain high‑performance data pipelines that support large‑scale analytics. The role involves working with high‑volume data, modern orchestration frameworks, and big‑data technologies in a hybrid environment (2 days onsite per week).

Key responsibilities

• Build and maintain robust data pipelines using Python.
• Develop, optimise, and monitor ETL workflows.
• Write complex SQL queries using Presto and Spark SQL.
• Process large datasets with Apache Spark.
• Ensure data quality, reliability, and observability.
• Collaborate with Data Science, Product, and Analytics teams.

Required profile

• Strong expertise in Python and SQL.
• Experience with data pipeline/orchestration tools such as Airflow‑like platforms (e.g., Dataswarm).
• Hands‑on experience with Spark and big‑data processing.
• Good understanding of data‑warehousing concepts and large‑scale environments.
• Familiarity with data catalog, lineage, and monitoring frameworks.

Required skills

• Python
• SQL (Presto, Spark SQL)
• Apache Spark
• Airflow‑like orchestration (Dataswarm)
• Hive
• Data catalog and lineage tools