Quality Engineer – AI Platform (London)

About the role

EPAM is seeking a Quality Engineer to join its London office and work on an AI‑driven enterprise project. The role focuses on exposing internal tooling to AI agents via the Model Context Protocol (MCP) and ensuring robust quality engineering and compliance across intelligent automation solutions.

Key responsibilities

• Develop automated testing frameworks for MCP servers and related AI systems.
• Design and implement evaluation strategies for LLM accuracy, safety, and reliability.
• Create automated tests using Python, Pytest, and BDD frameworks.
• Integrate quality gates into CI/CD pipelines for continuous assurance.
• Identify and mitigate agentic AI failure modes such as hallucination, latency, and incorrect tool usage.
• Collaborate with engineering, QA, and product teams to define quality metrics and acceptance criteria.
• Participate in Agile ceremonies and align testing practices with sprint goals.
• Prepare detailed reports on quality outcomes and maintain documentation for test cases and evaluation pipelines.

Required profile

• Strong programming experience in Python applied to test automation.
• Proficiency with Pytest and familiarity with BDD frameworks (Behave or Cucumber).
• Knowledge of LLM evaluation methods (RAGAS, DeepEval, or custom pipelines).
• Understanding of common agentic AI issues such as hallucination and tool misuse.
• Experience with Agile delivery and CI/CD integration for quality checks.
• Excellent communication and problem‑solving abilities.

Required skills

• Python
• Pytest
• Behave
• Cucumber
• RAGAS
• DeepEval
• Model Context Protocol (MCP)
• Observability/monitoring tools for AI systems
• API and service integration testing
• Containerized environments and cloud‑native architecture

What we offer

• Employee Stock Purchase Plan (ESPP)
• Life assurance, income protection and critical illness cover
• Private medical insurance and dental care
• Employee Assistance Program
• Competitive group pension plan