Senior Data Engineer – Python & Databricks (Hybrid, London)

About the role

We are partnering with an international bank to hire a Senior Data Engineer for a 6‑month fixed‑term contract. The role is based in London with a hybrid schedule (three days on‑site, two days remote) and focuses on building robust data pipelines using Python and Databricks.

Key responsibilities

• Design, develop and optimise scalable data pipelines on Databricks using Python, Spark and Delta Lake.
• Create and maintain ETL/ELT workflows, ensuring high data quality and governance.
• Build and evolve Lakehouse data models and architectures to support analytics and machine‑learning initiatives.
• Optimise large‑scale data processing workloads and monitor platform reliability.
• Mentor junior engineers and champion best engineering practices.

Required profile

• 10+ years of experience in data engineering.
• Strong expertise in Python and Databricks (Spark/PySpark, Delta Lake).
• Proven track record building scalable ETL pipelines and working with SQL.
• Experience with cloud platforms such as AWS, Azure or GCP.
• Familiarity with modern data warehouses (Snowflake, Redshift, BigQuery) and workflow orchestration tools (Airflow or Databricks Workflows).

Required skills

• Python
• Databricks
• Spark / PySpark
• Delta Lake
• SQL
• AWS, Azure, GCP
• Snowflake, Redshift, BigQuery
• Airflow
• Databricks Workflows
• Kafka
• Docker
• Kubernetes
• CI/CD