Machine Learning Quality Engineer

About the role

LSEG's Corporate Engineering AI (CE AI) team builds and governs shared AI platforms for its internal agentic AI ecosystem, including LSEG AI Assist, the Question Answering Service (QAS) and the Internal MCP Gateway. As a Machine Learning Quality Engineer on the LSEG AI Assist / Internal MCP programme, you will define evaluation frameworks, build automated tests and produce testing evidence for permit-to-build and permit-to-operate approvals.

Key responsibilities

• Define and implement evaluation frameworks for MCP tools and Skills covering correctness, safety and regression impact.
• Build and maintain automated test pipelines for agentic behaviour, including tool invocation and multi-step workflows.
• Evaluate and mitigate agentic failure modes such as hallucination, tool misuse, invalid inputs and latency amplification.
• Produce testing evidence required for Permit to Build (PTB) and Permit to Operate (PTO).
• Partner with ML Engineers to embed testability and evaluation hooks into MCP servers and Skills.
• Help define the long-term quality and governance model for federated MCP contributions across LSEG.

Required profile

• Experience working in regulated or highly governed engineering environments.
• Ability to assess solutions against governance, security and audit expectations.
• Understanding of agentic system risks and failure modes.

Required skills

• Strong Python for test harnesses and automation.
• LLM and RAG evaluation frameworks or custom evaluation pipelines.
• Test automation across unit, integration and regression testing.