Associate Director, Clinical AI Evaluation and Responsible Deployment

About the role

AstraZeneca is building an AI capability for Clinical Development within its Enterprise AI Unit, and is hiring an Associate Director, Clinical AI Evaluation and Responsible Deployment. The role creates the evidence systems that determine whether clinical AI is useful, reliable, safe and ready to scale. It sits in the Applied Clinical AI team and works with clinical stakeholders, Engineering, IT and data teams, alongside the existing Engineering, Product and Clinical Solutions teams. It is a hands-on technical leadership role.

Key responsibilities

• Define and implement the evaluation strategy for clinical AI models and agents across development, release, monitoring and change control.
• Build evaluation harnesses, curated test sets, simulation environments, automated regression suites and analysis pipelines in Python.
• Turn expert judgment from CRAs, medical monitors, clinical scientists, statisticians and operations leaders into task definitions, scoring rubrics, error taxonomies and acceptance thresholds.
• Evaluate complete workflows, including retrieval quality, tool selection, orchestration, source fidelity, abstention, human hand-offs, latency and downstream impact.
• Design approaches for noisy, sparse, delayed or expert-dependent ground truth, including adjudication, inter-rater agreement, challenge sets, prospective studies and post-deployment surveillance.
• Lead failure analysis and red-teaming for hallucination, unsupported claims, automation bias, data leakage, prompt injection, subgroup performance and unsafe workflow behaviour.
• Establish traceability from intended use and user need through requirements, test evidence and release decisions, including revalidation throughout the product lifecycle.
• Align evaluation evidence with GCP, GxP, data integrity, validation and inspection-readiness expectations with Quality, Regulatory, Clinical Operations, Privacy, Security and R&D IT.
• Advise teams on when evidence supports progression from prototype to controlled pilot, broader deployment or regulated use.
• Create reusable evaluation components and standards, communicate findings and residual risks clearly, and mentor scientists and engineers in rigorous experimentation and reproducibility.

Required profile

• PhD in Machine Learning, Computer Science, Statistics, Biostatistics, Biomedical Informatics, Computational Biology or a related quantitative discipline; or an MD, master's degree or equivalent experience with a strong computational record.
• 4 to 7 years of post-PhD (or equivalent) experience evaluating, validating or deploying AI/ML systems in healthcare, life sciences, clinical research or another safety- or evidence-critical environment.
• Experience evaluating LLMs or agentic systems, including retrieval, tool use, structured outputs, multi-step workflows, robustness and human-in-the-loop performance.
• Experience producing audit-ready evidence under GCP and GxP, including model-risk management, audit trails and inspection readiness.
• Ability to work directly with clinical stakeholders to define intended use, harmful failure modes, decision thresholds and acceptable human oversight.
• Desirable: direct experience with clinical trial conduct, medical monitoring, pharmacovigilance or clinical data management; experience with prospective, silent-mode, shadow-mode or human-factors evaluations; familiarity with causal inference, calibration, subgroup analysis, weak supervision, active learning or learning with imperfect labels; experience with LLM evaluation platforms, observability, MLOps/LLMOps and production monitoring; peer-reviewed publications, standards work or regulator/consortium engagement.

Required skills

• Hands-on coding in Python and SQL, including automated evaluation pipelines, large dataset analysis, testing and version-controlled environments.
• Experimental design, statistical inference, uncertainty, error analysis and measurement reliability.
• Excellent communication skills and the judgment to explain technical evidence and residual risk to clinical, quality, regulatory and executive audiences.
• Experience leading cross-functional technical work and making evidence-based go/no-go recommendations.

The role expects in-person working on average a minimum of three days per week, with individual flexibility. The closing date is 12 November 2026.