Member of Technical Staff, Data Analysis and Evaluation

About the role

As a Member of Technical Staff in Data Analysis and Evaluation, you will ensure the quality, reliability, and performance of Cohere’s large language models. You will design data‑collection tasks, evaluate dataset quality, and analyse model robustness across diverse scenarios.

Key responsibilities

• Design and oversee data collection tasks, supporting human annotators and guaranteeing data quality.
• Develop statistical methods to evaluate dataset reliability and suitability for training.
• Analyse the generalisability and robustness of ML systems across varied use cases.
• Collaborate with researchers, engineers, and annotators to improve dataset quality and model performance.
• Train and fine‑tune large language models on distributed training infrastructures.
• Conduct experiments to assess model performance and identify improvement areas.

Required profile

• Extremely strong software engineering skills.
• Proven experience designing and conducting data‑collection tasks with human annotators.
• Strong statistical expertise and experience evaluating scientific experiments.
• Hands‑on experience training large language models on distributed systems.
• Excellent communication skills for cross‑functional collaboration.

Required skills

• Python
• PyTorch
• TensorFlow
• JAX

What we offer

• Weekly lunch stipend of $75/£75.
• Full health and dental benefits, including a mental‑health budget.
• RRSP matching, 401K, and pension scheme.
• 100% parental‑leave top‑up for up to six months.
• Six weeks of paid vacation (30 days).
• Education and learning stipend for conferences, courses, and coaching.
• $500 home‑office stipend to set up your workspace.