{"id":1262958,"url":"https://alion.io/job/nscale-staff-ai-product-engineer-2","title":"Staff AI Product Engineer","company":{"id":48155,"name":"Nscale","domain":"nscale.com","url":"https://alion.io/company/nscale","size_band":"501-1000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"A","score":87,"open_postings":23,"ghost_share":0,"stale_share":0.304,"repost_share":0,"time_to_fill_p50_days":63,"computed_at":"2026-10-22T06:10:10Z"}},"role":"Backend","role_family":"Backend","seniority":"staff","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["New York, United States","San Francisco, United States","Houston, United States","Seattle, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":220000,"max":330000,"currency":"USD","period":"year","gross":null,"usd_annual":330000},"salary_estimate":null,"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"CUDA","optional":false},{"name":"CUTLASS","optional":false},{"name":"DeepSpeed","optional":false},{"name":"Fine-tuning","optional":false},{"name":"GRPO","optional":false},{"name":"Kubernetes","optional":false},{"name":"KV Cache","optional":false},{"name":"LLM","optional":false},{"name":"Mixture of Experts","optional":false},{"name":"NVLink","optional":false},{"name":"OpenAI","optional":false},{"name":"Post-training","optional":false},{"name":"PPO","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"Reward Modeling","optional":false},{"name":"ROCm","optional":false},{"name":"SGLang","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"Triton","optional":false},{"name":"TRL","optional":false},{"name":"vLLM","optional":false},{"name":"TensorRT","optional":true},{"name":"Transformers","optional":true}],"status":"live","first_seen_at":"2026-09-15T18:26:49Z","employer_posted_date":"2026-10-21","last_verified_at":"2026-10-22T16:35:55Z","board_verified":true,"closed_at":null,"days_open":37,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":37},"description":"About Nscale\nNscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do.\n\nAbout the Role\nNscale is looking for a Staff AI Engineer (Specialised) to set technical direction for part of the inference platform at the core of our AI cloud: dedicated and serverless inference, bring-your-own-model deployments, and the post-training services that sit alongside them.\nYou’ll be the technical authority for one or more focus areas, working across the teams that build serving, post-training, and platform. Your decisions shape the latency, throughput, and cost of the tokens Nscale serves, and whether we can say with evidence that the models we run are the right ones. You’ll take on open questions, such as how KV cache should move between GPUs, nodes, and storage, or where hand-written kernels beat the open-source defaults, and your answers become the standards others build on.\nWe run state-of-the-art GPU systems, and much of the open-source ecosystem hasn’t caught up with them yet. This role is for people who want to close that gap hands-on while growing the engineers around them.\nHow We Work\nDog years. We move quickly and compress a lot of learning into a short time.\nDon’t let perfect be the enemy of good. Ship, measure, iterate.\nBe relentless. Own the problem end to end and see it through.\nOne team, one mission. Outcomes over process, and no “not my job”.\nFocus Areas\nWe’re hiring for depth. You don’t need all of these; we want people who are exceptional in one or more, and we’re deliberately hiring people with different focus areas:\n KV cache offloading and inference performance: KV cache orchestration across GPU, host memory, and storage; cross-instance cache sharing (LMCache or similar); KV-aware routing; disaggregated prefill/decode; speculative decoding; quantisation (FP8, NVFP4, INT8/4); MoE serving\nGPU kernels and performance: writing and tuning kernels in CUDA, Triton, CUTLASS or ROCm; attention, MoE and GEMM optimisation; multi-GPU and multi-node communication on NVLink-scale systems\nEvals and benchmarking: designing evaluation frameworks, turning customer requirements into custom benchmarks, and producing objective model comparisons we can stand behind\n Post-training and RL: fine-tuning and preference optimisation as a service; RL for LLMs (PPO/GRPO-style, reward modelling, multi-turn and tool-use RL); rollout generation, weight synchronisation, and sharing infrastructure between inference and training\n Serving platform and APIs: serving engines (vLLM, SGLang, TensorRT-LLM), model onboarding, and OpenAI-compatible APIs that other engineers and customers build on\nResponsibilities\nSet technical direction for your focus areas and turn it into work that multiple teams can deliver\nLead the resolution of systemic performance and reliability problems across the serving stack, from kernel bottlenecks to fleet-level capacity and multi-tenant isolation\nOwn the trade-offs between cost, latency, throughput, and model quality in your area, and back them with measurement\nBuild reusable frameworks, benchmarks, and tooling that make other AI engineers at Nscale more effective\nEvaluate emerging serving engines, kernel libraries, RL frameworks, and accelerators, and make clear build/adopt/contribute recommendations\nCoach and grow engineers across teams, and raise engineering quality broadly\nWork with research, product, and infrastructure leadership so the platform tracks customer demand\nRepresent your area in cross-team technical reviews and planning\nRequirements\n8-12 years of engineering experience, with significant depth in production AI systems or ML research at scale\n4+ years of hands-on work with LLMs in at least one of the focus areas above, in production or research\nDeep, demonstrated expertise in at least one of the focus areas above\nEnough working knowledge of the full serving stack to debug across it: API, router, scheduler, engine, kernel, and cluster\nA track record of setting technical direction and creating standards or tools adopted beyond your own team\nStrong Python and PyTorch, with a track record of maintainable, well-tested, production-grade systems\nExperience operating AI workloads in containerised, distributed environments (Kubernetes, large GPU clusters)\nSolid understanding of GPU performance: memory bandwidth, interconnect, and parallelism strategies (data, tensor, pipeline, expert)\nDeep understanding of transformer and LLM architectures and how they behave under production load\nPreferred\nContributions to widely used open-source inference, kernel, or RL projects (vLLM, SGLang, TensorRT-LLM, LMCache, FlashInfer, Triton, verl, OpenRLHF, TRL, DeepSpeed, etc.)\nExperience at an AI lab, hyperscaler AI team, or leading ML infrastructure company\nPublished research or technical writing on inference, kernels, evals, or RL (MLSys, NeurIPS, ICLR, blog posts)\nHands-on custom kernel work (CUDA, Triton, CUTLASS) if kernels aren’t already your focus area\nExperience designing developer-facing APIs and SDKs used by external customers\nExperience with control plane / data plane separation and cell-based architecture for scale-out and blast-radius isolation\nThe range below reflects the base salary for the position. Actual compensation may vary based on job-related factors such as skill set, experience, education, and location. In addition to base salary, this role may be eligible for bonus, equity, and/or commission programs. Nscale may offer a competitive benefits package including medical, dental, vision, flexible paid time off, parental leave, and retirement plan participation.\nSalary Range\n$220,000—$330,000 USD\nFor information on how Nscale handles candidate personal data, please see our Employee & Candidate Privacy Notice: Here.\nNscale does not accept unsolicited candidate submissions from recruitment agencies.","description_format":"text","description_chars":6373,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Parental leave","Retirement plans"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Data Centers & Colocation","AI Compute & Inference"],"lifecycle":[{"event":"open","at":"2026-09-25T21:14:28Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 10","filings_12m":10,"filings_prev_12m":0,"green_card_filings_12m":0,"median_offered_wage_usd":198500,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":3}],"liveness":{"score":78,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.867,"p_room":0.9,"age_days":36,"expected_fill_days":63,"reasons":["conf:0","velocity","win:mid","comp:brand"],"computed_at":"2026-10-22T06:10:10Z"},"pay":{"stated_usd_annual":330000,"is_top_pay":true},"html_url":"https://alion.io/job/nscale-staff-ai-product-engineer-2","json_url":"https://alion.io/job/nscale-staff-ai-product-engineer-2.json","meta":{"generated_at":"2026-10-22T19:50:49Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":3436,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-23T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":48155},"rest":"https://alion.io/mcp/rest/get_company?id=48155"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fnscale-staff-ai-product-engineer-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fnscale-staff-ai-product-engineer-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fnscale-staff-ai-product-engineer-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/nscale-staff-ai-product-engineer-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fnscale-staff-ai-product-engineer-2"}]}