Inference Engine Engineer – Member of Technical Staff

About the role

Callosum is building the next‑generation AI infrastructure that unifies heterogeneous compute across the full stack. This role focuses on extending inference engines beyond homogeneous GPU clusters to fully hardware‑aware, heterogeneous systems.

Key responsibilities

• Contribute upstream to SGLang and vLLM and maintain internal forks where requirements diverge.
• Improve hardware‑awareness within inference engines so scheduling, memory management, and execution adapt to accelerator capabilities.
• Design and implement bespoke parallelism and disaggregation strategies that exploit heterogeneous hardware.
• Collaborate closely with an Accelerator Systems Software engineer to map engine‑level abstractions onto diverse hardware.

Required profile

• Deep familiarity with the internals of SGLang, vLLM, or comparable inference serving frameworks, including scheduler design, memory management, and execution pipelines.
• Strong background in high‑performance Python and C++/CUDA systems for ML inference.
• Experience designing or implementing parallelism strategies for large‑model serving.
• Understanding of disaggregated serving architectures and associated trade‑offs.
• Proven ability to work effectively in fast‑moving open‑source codebases with evolving APIs.

Required skills

• Python
• C++
• CUDA
• SGLang
• vLLM
• Inference serving frameworks
• Scheduler design
• Memory management
• Parallelism strategies
• Disaggregated serving architectures

What we offer

• Competitive salary, determined by skills and experience.
• Equity & ownership.
• Private healthcare.
• Visa sponsorship and relocation benefits.
• On‑site work in our London office with inclusive workplace culture.