Data Engineer (Azure)

About the role

The Data Engineer will design, build and operate data pipelines within an Azure‑based analytics and lake‑house environment for a long‑standing mutual society serving South Wales and the South West of England. Working hands‑on, you will implement reliable, secure and well‑governed data solutions that feed Power BI reporting and regulated finance use cases.

Key responsibilities

• Develop and maintain data pipelines using Azure Databricks (Python, PySpark, SQL) across Bronze, Silver and Gold layers with Delta Lake.
• Support and operate Azure Databricks and Azure Synapse platforms, monitoring performance and configuring resources.
• Build Gold‑layer datasets optimized for Power BI and analytics consumption.
• Resolve production incidents, maintain operational run‑books and ensure data security, access control and governance in a regulated environment.
• Maintain accurate documentation, data lineage and contribute to CI/CD pipelines and code reviews.

Required profile

• Proven experience as a Data Engineer with hands‑on Azure Databricks, Python, PySpark and SQL.
• Understanding of medallion architecture and Delta Lake.
• Experience supporting Azure data platforms in production.
• Desirable experience with Azure Synapse Analytics, Azure Data Factory and background in financial services or other regulated sectors.

Required skills

• Azure Databricks
• Python
• PySpark
• SQL
• Delta Lake
• Azure Synapse Analytics
• Azure Data Factory