Principal Azure Data Engineer (Databricks)
Capco · UK - London
Job description
About the role
Capco is looking for a Principal Azure Data Engineer (Databricks) in London, working in a hybrid setting. You will architect and lead the delivery of enterprise-grade data solutions using the Databricks platform within Azure, helping clients in financial services modernise their data capabilities. You will also champion AI-enabled ways of working with approved AI and automation tools.
Key responsibilities
- Architect and lead end-to-end delivery of Databricks-based data platforms across Azure environments.
- Design secure, scalable data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog and Azure-native services, applying AI-enabled engineering approaches where appropriate.
- Establish and promote engineering best practices across the data lifecycle, with strong governance, quality and operational excellence.
- Partner with clients to define technical strategy, data architecture and governance frameworks aligned with business objectives.
- Mentor engineering teams, build internal capability and encourage the responsible use of approved AI tools and automation.
Required profile
- Proven expertise with Databricks, including Unity Catalog, Delta Lake, orchestration and enterprise-scale Azure deployments.
- Experience building production-grade data platforms.
- Deep understanding of Data Lakehouse architecture, data modelling, security and GDPR-compliant design.
- Experience using AI-enabled tools responsibly, with human oversight and governance.
Required skills
- Python and PySpark, and distributed data processing frameworks.
- CI/CD pipelines with Azure DevOps, Jenkins, GitHub Actions or equivalent.
- Bonus points: client-facing consulting and commercial experience, mentoring experience, Scala or Java, knowledge of PII handling and financial services regulation, and technical community involvement.
What we offer
- Discretionary bonus, pension, health, life and critical illness insurance.
- Mental health support and family-friendly leave, including maternity, adoption, shared parental and bereavement leave.
- Eight complimentary backup care sessions and five weeks of annual leave.
- Minimum 40 hours of training per year and a Business Coach from day one.
- Wellhub (Gympass), travel insurance, Tastecard, season ticket loans, Cycle to Work and dental insurance.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in the United Kingdom.
Salaries by job title
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 3 days ago
Expires 1 month from now
11 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Capco
UK - London