{"id":93498,"url":"https://alion.io/job/sarvam-ai-performance-engineer-inference","title":"Performance Engineer, Inference","company":{"id":229,"name":"Sarvam AI","domain":"sarvam.ai","url":"https://alion.io/company/sarvam-ai","size_band":"501-1000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"A","score":95,"open_postings":39,"ghost_share":0,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":91,"computed_at":"2026-10-16T05:45:00Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"junior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India","Chennai, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":13500,"max_usd":34000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":12},"experience_years_min":2,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"CUDA","optional":false},{"name":"CUDA Toolkit","optional":false},{"name":"Knowledge Distillation","optional":false},{"name":"KV Cache","optional":false},{"name":"LLM","optional":false},{"name":"Mixture of Experts","optional":false},{"name":"Model Distillation","optional":false},{"name":"NCCL","optional":false},{"name":"SGLang","optional":false},{"name":"SLI/SLO/SLA","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"TPOT","optional":false},{"name":"vLLM","optional":false},{"name":"Multimodal AI","optional":true},{"name":"Speech Recognition","optional":true},{"name":"TensorRT","optional":true},{"name":"Text-to-Speech","optional":true}],"status":"live","first_seen_at":"2026-08-10T06:33:05Z","employer_posted_date":"2026-08-10","last_verified_at":"2026-10-16T18:39:59Z","board_verified":true,"closed_at":null,"days_open":67,"trust":{"level":"ok","repost_count":0,"flags":[],"days_open":66},"description":"Performance Engineer, Inference\nPart of Sarvam's Performance Engineering team. We are hiring two specialized performance roles - Inference (this posting) and Kernels (companion posting). They are a vertical stack: the kernels team authors the µs-level GPU code, and the inference team integrates it into a running serving stack and owns the system-level numbers. If your depth genuinely spans both, apply to either and tell us - but most candidates are strongest in one, and we hire for that depth.\nLocation: [Bengaluru / Chennai / Hybrid / On-site] · Team: Performance Engineering · Level: Senior\nAbout the team\nSarvam serves multiple model families - small and large LLMs, Mixture-of-Experts, Indic ASR & TTS, streaming models and multimodal models - across a multi-node, multi-tenant fleet of Hoppers and Blackwells. The Performance Engineering team owns the numbers the rest of the company plans against: how fast we serve, how much it costs, and how much we get out of every GPU. This team works at the intersection of the serving runtime, the kernel layer, and the SRE org that keeps the fleet alive.\nAbout the role\nYou will own Sarvam's production serving path for large distributed models end to end. You should be source-level fluent in at least one of SGLang, vLLM, NVIDIA Dynamo, or TensorRT-LLM - able to read and modify it where stock behavior does not fit our workloads - and you should operate and extend a distributed-serving stack at depth: disaggregated prefill-decode across nodes, distributed KV/cache transfer, and the routing and scheduling that span them. You will integrate artifacts from the model and kernel teams into a running multi-node, multi-tenant stack, and you will build and train your own speculators - draft models, distillation from the target, acceptance-rate tuning against the live serving distribution - rather than only wiring in stock implementations. You will produce, and defend, the latency and throughput numbers the company plans against, and you will spend significant time in cross-team work with architecture co-design, the kernels team, the model team, and SRE.\nYour scoreboard: TTFT (p50 / p95 / p99), TPOT, throughput, GPU utilization, and cost per million tokens.\nWhat we're looking for\n5+ years in ML systems, with 2+ years on inference serving at production scale. Your record shows concrete outcomes - tokens per day, throughput wins, p99 reductions - rather than \"deployed a model.\"\n\nExperience serving 100B+ parameter models in production across multi-node tensor, pipeline, or expert parallelism.\n\nSource-level fluency in one of SGLang, vLLM, Dynamo, or TensorRT-LLM - you have modified the scheduler, the KV allocator, or the disaggregation path - and reading-level familiarity with the other three.\n\nDistributed serving at operating-and-extending depth: a disaggregated prefill-decode stack, distributed KV/cache transfer (Mooncake or equivalent), and cross-node routing and scheduling. You have run one of these in production and modified it where it didn't fit.\n\nSpeculative decoding as a build-and-train competency: you have trained your own draft models or speculators (EAGLE / DFlash or otherwise), distilled them from a target model, measured and tuned acceptance rate against a real serving distribution, and composed speculation with the rest of the stack - not only integrated a stock implementation.\n\nDeep understanding of KV cache internals: block tables, copy-on-write, prefix sharing, and fragmentation.\n\nWorking command of TP / PP / EP, NCCL primitives, and how they interact with the scheduler.\n\nMulti-tenant serving: model co-location and MIG / MPS isolation.\n\nProfiling fluency with Nsight Systems, framework tracing, and py-spy / perf.\n\nC++ and CUDA at a read-and-modify level.\n\nOn-call ownership of an inference SLO.\n\nStrong pluses\nUpstream contributions to SGLang, vLLM, Dynamo, llm-d, TensorRT-LLM, or LMDeploy on non-trivial code paths.\n\nDirect production experience with Dynamo or llm-d at scale.\n\nHaving operated a forked runtime in production.\n\nPublished or shipped speculator work - a draft model or speculative-decoding technique you trained and measured.\n\nMoE serving at scale, long-context (128K+), multi-model serving, or Indic and multilingual workloads.","description_format":"text","description_chars":4249,"description_truncated":false,"requirements":{"experience_years_min":2,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["LLM & Generative AI","Natural Language Processing","Speech & Audio AI","Foundation Models"],"lifecycle":[{"event":"open","at":"2026-08-10T06:33:05Z"}],"visa":[],"liveness":{"score":60,"band":"ok","label":"Likely open","p_open":1,"p_active":0.799,"p_room":0.75,"age_days":66,"expected_fill_days":91,"reasons":["conf:0","velocity","win:late","comp:junior"],"computed_at":"2026-10-16T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/sarvam-ai-performance-engineer-inference","json_url":"https://alion.io/job/sarvam-ai-performance-engineer-inference.json","meta":{"generated_at":"2026-10-16T22:27:24Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":4083,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":229},"rest":"https://alion.io/mcp/rest/get_company?id=229"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fsarvam-ai-performance-engineer-inference"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fsarvam-ai-performance-engineer-inference"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fsarvam-ai-performance-engineer-inference"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/sarvam-ai-performance-engineer-inference\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fsarvam-ai-performance-engineer-inference"}]}