Lead Data Engineer – Databricks (Hybrid, London)

About the role

We are partnering with a fast‑growing technology business that is heavily investing in its cloud data platform and AI capabilities. This senior, hands‑on leadership position will own the architecture and development of a modern Databricks environment while mentoring a team of Data Engineers.

Key responsibilities

• Lead the design, development and optimisation of a Databricks‑based data platform.
• Design enterprise‑scale Lakehouse architectures across cloud platforms.
• Define data modelling, governance, security and quality standards.
• Mentor and lead a team of Data Engineers.
• Collaborate with Data Scientists, Analysts and Product teams to support analytics, machine learning and AI initiatives.
• Drive platform performance, scalability and cloud cost optimisation.

Required profile

• Proven experience as a Senior or Lead Data Engineer.
• Deep expertise in Databricks, Spark, Delta Lake, Python and SQL.
• Experience designing and scaling cloud‑based data platforms.
• Strong knowledge of ETL/ELT, data modelling and distributed processing.
• Hands‑on experience with Azure, AWS or GCP environments.
• Previous team leadership or mentoring experience.
• Strong stakeholder management and communication skills.

Required skills

• Databricks
• Apache Spark
• Delta Lake
• Python
• SQL
• Azure
• AWS
• GCP

What we offer

• Salary up to £110,000 per year.
• Hybrid working model.
• Technical ownership of a modern Databricks platform.
• Opportunity to influence architecture, tooling and engineering standards.
• Leadership responsibilities with a growing engineering team.
• Exposure to AI, machine learning and large‑scale data initiatives.