Lead Data Engineer – Kafka & Kinesis

About the role

This is a senior, hands‑on position leading the design and delivery of a greenfield data platform for an AI‑driven property‑management product. You will own the end‑to‑end data architecture and work closely with AI, backend and product teams.

Key responsibilities

• Design and build the company’s data architecture from the ground up.
• Create scalable batch and real‑time data pipelines.
• Develop data ingestion, storage, and transformation frameworks.
• Support AI and machine‑learning infrastructure, including vector‑search systems.
• Implement data quality, monitoring and observability practices.
• Collaborate with AI, backend engineering and product teams.
• Optimise performance, scalability and reliability of large datasets.
• Influence long‑term technical strategy and lay foundations for the data function.

Required profile

• Strong experience building scalable distributed data systems using Python.
• Proven ability to design data architectures and develop batch and streaming pipelines.
• Hands‑on experience with relational databases such as PostgreSQL.
• Familiarity with NoSQL and modern vector databases (e.g., Qdrant, Milvus, pgvector).

Required skills

• Python
• Kafka
• Kinesis
• PostgreSQL
• Qdrant
• Milvus
• pgvector

What we offer

• Salary up to £100,000 per year, depending on experience.