Data Engineer

About the role

We are partnering with a consulting client to expand their data capability. The role focuses on designing, building, and operating scalable data pipelines and streaming solutions using modern cloud and DevOps practices.

Key responsibilities

• Develop, optimise and maintain data pipelines in Java and Python, with exposure to Flink, Kafka or Spark.
• Implement real‑time streaming and event‑driven integrations using AWS managed services.
• Contribute to data modelling, architecture standards, data quality, lineage, governance and security.
• Deploy and manage data applications via CI/CD pipelines and Infrastructure‑as‑Code.
• Work with containerisation and orchestration platforms to support cloud environments.

Required profile

• Proven experience in pipeline development, architecture and deployment.
• Strong programming proficiency in Java or Python.
• Hands‑on experience with at least one streaming technology (Kafka, Flink, Spark).
• Understanding of software engineering best practices, testing, design patterns and version control.
• Willing and eligible to undergo UK security clearance.

Required skills

• Java
• Python
• Kafka
• Flink
• Spark
• AWS managed services
• CI/CD tooling
• Infrastructure‑as‑Code (e.g., Terraform)
• Container technologies (Docker, Kubernetes)
• Git