Member of Technical Staff (AI Inference Engineer)

About the role

We are looking for an AI Inference Engineer to join our growing team at Perplexity. The role involves building and running the inference engine behind every Perplexity query, deploying dozens of model architectures at scale while meeting tight latency and cost budgets.

Key responsibilities

• Support new model architectures, including transformer‑based retrieval, text‑generation, and multimodal models across the inference stack.
• Migrate in‑house CUDA kernels to NVIDIA’s CuTe DSL for current and future hardware platforms.
• Develop and maintain a Rust‑native serving runtime to replace Python‑based components.
• Profile and eliminate performance bottlenecks from network ingress through continuous batching and GPU kernel interleaving.
• Build dashboards, alerts, and automated remediation to ensure reliability and observability in production.

Required profile

• Deep experience with GPU programming and performance work (CUDA, Triton, CUTLASS, or similar).
• Strong understanding of modern LLM architectures and ability to deploy them reliably.
• Proven track record building and operating production‑grade distributed systems under real load.
• Comfortable working across Rust, Python, and CUDA/CuTe DSL codebases.
• Self‑directed, able to take problems from research paper to production incident resolution.

Required skills

• Rust
• Python
• CUDA
• CuTe DSL
• Triton, CUTLASS
• PyTorch, JAX, TensorFlow
• NCCL, NVLink, InfiniBand, RDMA
• Kubernetes (GPU scheduling, autoscaling)
• Profiling tools such as Nsight Compute, CUDA‑GDB, PTX/SASS analysis

What we offer

• Competitive base salary (determined by experience and expertise).
• Equity participation as part of the total compensation package.