Databricks Platform Engineer

About the role

As a Databricks Platform Engineer you will design and deliver scalable Lakehouse solutions on AWS and/or Azure for our clients. You will work closely with client stakeholders to translate business needs into secure, high‑performance data platforms.

Key responsibilities

• Architect end‑to‑end Databricks platforms, including ingestion, Delta Lake storage, processing and consumption layers.
• Build and configure cloud infrastructure using Terraform and Declarative Automation Bundles.
• Establish secure environments with VNet/VPC, Private Link, IAM/Entra ID, Unity Catalog and fine‑grained access controls.
• Define environment strategies (dev/test/prod), CI/CD pipelines and release processes for Databricks deployments.
• Implement monitoring, logging, cost optimisation and performance tuning across the platform.
• Design data pipelines with Delta Live Tables, Auto Loader and Databricks Workflows for batch and streaming workloads.
• Lead technical workshops, architecture sessions and proof‑of‑concept builds with clients.
• Produce technical documentation, runbooks and deployment guides, and provide knowledge transfer and training.

Required profile

• Proven experience designing and implementing Databricks Lakehouse platforms on AWS or Azure.
• Strong client‑facing skills with the ability to lead workshops and translate requirements into architecture.
• Deep understanding of cloud networking, identity, data governance and security best practices.
• Ability to deliver robust, cost‑effective solutions that meet regulatory standards such as GDPR and ISO 27001.

Required skills

• Databricks
• AWS
• Azure
• Terraform
• Declarative Automation Bundles (DABs)
• VNet / VPC, Private Link
• IAM / Entra ID
• Unity Catalog
• Delta Lake, Delta Live Tables, Auto Loader, Databricks Workflows
• CI/CD pipelines
• Monitoring, logging, cost optimisation, performance tuning