Member of Technical Staff – North Modelling (Evaluations)

About the role

North is Cohere’s AI workspace platform that lets enterprises use AI securely across real workflows. This role is the voice of North inside modelling, building evaluation systems and feedback loops that ensure model improvements translate into better product outcomes. It is a rigorous applied MLE position focused on measurement, model behavior, and real‑world product quality.

Key responsibilities

• Own the evaluation strategy for North, defining metrics for agent workflows, tool use, and enterprise knowledge work.
• Build high‑quality evaluations from product realities such as user feedback, production failures, privacy‑preserving logs, and internal dogfooding.
• Create systems that continuously turn learning from users and feature teams into fresh evaluations, keeping measurement in sync with product development.
• Act as the voice of North inside modelling, turning eval results and customer data into actionable recommendations for central modelling teams.

Required profile

• Experience improving LLM‑powered, agent‑powered, or AI‑product systems through evaluations, feedback loops, data curation, prompting, model adaptation, or model selection.
• Deep commitment to evaluation as a craft, including representative tasks, precise rubrics, clean data, failure analysis, and regression tracking.
• Strong applied MLE judgment with the ability to reason about model behavior, eval validity, product outcomes, and production trade‑offs.
• Comfort working closely with users and product teams, translating qualitative signals into measurable insights.
• Self‑directed, practical, and motivated by open‑ended problems that require both technical depth and product understanding.
• Passion for making AI genuinely useful in real enterprise settings, not just on abstract benchmarks.

What we offer

• Weekly lunch stipend of $75/£75 (or local equivalent).
• Full health and dental benefits, including a mental‑health budget.
• Retirement savings matching (RRSP, 401K, pension scheme).
• 100% parental leave top‑up for up to six months.
• Annual enrichment benefits for arts, culture, fitness, and workspace improvements.
• Education and learning stipend for conferences, courses, and coaching.
• Six weeks of paid vacation (30 working days).
• Travel budget for office visits and an annual company offsite.
• Remote‑friendly policy with a $500 home‑office stipend and co‑working benefits.