Data Engineer – Kafka, Databricks, Big Data – £60k‑£71k + 10% Bonus

About the role

We are partnering with a market‑leading technology business in the smart housing sector that processes billions of sensor readings each month. The team is looking for a mid‑level Data Engineer to help build the next generation of scalable, real‑time data solutions.

Key responsibilities

• Design, build and optimise large‑scale data pipelines using Python, SQL and PySpark.
• Process billions of IoT data points and develop streaming solutions.
• Collaborate with Data Scientists and Software Engineers on ML and AI initiatives.
• Contribute to architecture decisions and engineering best practices.
• Participate in greenfield projects leveraging modern technologies.

Required profile

• Strong commercial experience as a Data Engineer.
• Advanced Python and SQL proficiency.
• Extensive experience with PySpark and scalable pipeline development.
• Good understanding of data modelling and database design.
• Excellent analytical, problem‑solving and stakeholder communication skills.
• Degree in Computer Science, Mathematics, Statistics or related field (advantageous).

Required skills

• Python
• SQL
• PySpark
• Databricks (desirable)
• Apache Kafka (desirable)
• Real‑time streaming technologies
• AWS services
• ClickHouse
• Infrastructure as Code
• CI/CD practices

What we offer

• Salary £60,000‑£71,000 per year plus 10% bonus.
• 33 days holiday, pension and health benefits.
• Hybrid working (3 days onsite, 2 days remote).
• Career progression in a modern tech stack environment.