Senior Data Engineer

About the role

We are seeking a Senior Data Engineer to join a large‑scale data transformation programme for a leading data‑driven organisation in London. The role will focus on modernising a complex data estate, supporting critical analytics, and enabling future AI and data‑science capabilities.

Key responsibilities

• Design, build and operate high‑volume streaming and batch pipelines in a modern cloud environment.
• Lead migration projects, consolidating real‑time analytics platforms onto a scalable architecture.
• Enhance and optimise the lake‑house environment, including schema management, data contracts, partitioning strategy and performance tuning.
• Develop production‑grade data foundations that support analytics, machine learning and data‑science initiatives.
• Establish engineering best practices such as infrastructure‑as‑code, CI/CD, observability and code quality.
• Provide technical leadership and mentorship to the wider data engineering team.

Required profile

• Proven experience owning and operating production data infrastructure at significant scale.
• Track record delivering large‑scale streaming solutions using technologies such as Kafka, Flink, Spark Streaming or Dataflow.
• Successful delivery of end‑to‑end migrations of live production systems.
• Strong knowledge of cloud platforms (GCP or AWS) and distributed data architectures.
• Hands‑on experience with Kubernetes and infrastructure‑as‑code approaches.
• Ability to collaborate effectively with data scientists, analysts and business stakeholders.

Required skills

• Kafka
• Flink
• Spark Streaming
• Dataflow
• Kubernetes
• Infrastructure as Code (e.g., Terraform)
• GCP
• AWS
• ClickHouse
• Apache Iceberg
• BigQuery
• dbt
• Elasticsearch
• Druid

What we offer

• Competitive daily rate of £550‑£650.
• Hybrid working model – two days per week onsite in London.
• Opportunity to lead high‑impact data platform projects within a large, data‑centric organisation.