Senior ML Systems Engineer, Frameworks & Tooling

About the role

Cohere is building frontier‑scale foundation models and needs a senior engineer to own and evolve the training framework that powers these large language models. The role sits at the intersection of large‑scale training, distributed systems and high‑performance computing, giving you autonomy to impact model throughput, reliability and developer productivity.

Key responsibilities

• Build and own the training framework for large‑scale LLM training.
• Design distributed training abstractions such as data, tensor and pipeline parallelism, FSDP/ZeRO strategies, and checkpointing.
• Improve training throughput and stability on multi‑node GPU clusters.
• Develop tooling for monitoring, logging, debugging and developer ergonomics.
• Collaborate with infrastructure teams to ensure cluster, container and hardware configurations support high‑performance training.

Required profile

• Strong engineering experience in large‑scale distributed training or HPC systems.
• Deep familiarity with JAX internals and distributed training libraries or custom kernels.
• Experience with multi‑node orchestration tools such as Slurm, Ray or Kubernetes.
• Proven ability to debug performance issues across CUDA/NCCL, networking, I/O and data pipelines.
• Track record of building tools that increase developer velocity for ML teams.

Required skills

• JAX
• CUDA/NCCL
• Slurm, Ray, Kubernetes
• Docker, Singularity/Apptainer
• Distributed training strategies (FSDP, ZeRO)
• Performance profiling and debugging

What we offer

• Weekly lunch stipend of $75/£75.
• Full health, dental and mental‑health benefits.
• RRSP matching, 401k and pension scheme.
• 100 % parental‑leave top‑up for up to six months.
• Six weeks of paid vacation and travel budget for office visits.
• Home‑office stipend and co‑working benefit.