Skip to Content
GenAI On-site

Performance Engineer, Inference

Job Overview

We are hiring two specialized performance roles - Inference (this posting) and Kernels (companion posting). The Performance Engineering team owns the numbers the rest of the company plans against: how fast we serve, how much it costs, and how much we get out of every GPU.

Key points

  1. You will own Sarvam's production serving path for large distributed models end to end.

  2. You will integrate artifacts from the model and kernel teams into a running multi-node, multi-tenant stack…

  3. You will produce, and defend, the latency and throughput numbers the company plans against, and you will spend significant time in cross-team work with…

  4. Experience serving 100B+ parameter models in production across multi-node tensor, pipeline, or expert parallelism.

  5. Source-level fluency in one of SGLang, vLLM, Dynamo, or TensorRT-LLM - you have modified the scheduler, the KV allocator…

  6. Distributed serving at operating-and-extending depth: a disaggregated prefill-decode stack, distributed KV/cache transfer (Mooncake or equivalent)…

  7. Deep understanding of KV cache internals: block tables, copy-on-write, prefix sharing, and fragmentation.

  8. Working command of TP / PP / EP, NCCL primitives, and how they interact with the scheduler.

Skills

LLMsNode.jsREST APIsMachine Learning

Experience: 5+ years · Job type: Full-time

Similar jobs

More open roles like this one