{"id":1534649,"url":"https://alion.io/job/nscale-principal-ai-product-engineer","title":"Principal AI Product Engineer","company":{"id":48155,"name":"Nscale","domain":"nscale.com","url":"https://alion.io/company/nscale","size_band":"501-1000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"A","score":91,"open_postings":28,"ghost_share":0,"stale_share":0.179,"repost_share":0,"time_to_fill_p50_days":68,"computed_at":"2026-10-15T05:46:00Z"}},"role":"Backend","role_family":"Backend","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["New York, United States","Houston, United States","San Francisco, United States","Seattle, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":290000,"max":520000,"currency":"USD","period":"year","gross":null,"usd_annual":520000},"salary_estimate":null,"experience_years_min":10,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"CUDA","optional":false},{"name":"CUDA Toolkit","optional":false},{"name":"DeepSpeed","optional":false},{"name":"DPO","optional":false},{"name":"GRPO","optional":false},{"name":"Knowledge Distillation","optional":false},{"name":"KV Cache","optional":false},{"name":"LLM","optional":false},{"name":"Megatron-LM","optional":false},{"name":"Mixture of Experts","optional":false},{"name":"Model Distillation","optional":false},{"name":"Post-training","optional":false},{"name":"Quantization","optional":false},{"name":"Reward Modeling","optional":false},{"name":"ROCm","optional":false},{"name":"SGLang","optional":false},{"name":"SLI/SLO/SLA","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"Tool Use","optional":false},{"name":"Triton","optional":false},{"name":"TRL","optional":false},{"name":"vLLM","optional":false},{"name":"TensorRT","optional":true},{"name":"Transformers","optional":true}],"status":"live","first_seen_at":"2026-09-30T16:28:16Z","employer_posted_date":"2026-10-15","last_verified_at":"2026-10-15T22:45:57Z","board_verified":true,"closed_at":null,"days_open":15,"trust":{"level":"ok","repost_count":1,"flags":[],"days_open":14},"description":"About Nscale\nNscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do.\nAbout the Role\nNscale is looking for a Principal AI Engineer (Specialised) to lead the inference and post-training pillar of our AI systems engineering organization. You’ll define the multi-year technical roadmap for how models are served, evaluated, and post-trained on Nscale’s GPU cloud, across dedicated and serverless inference and bring-your-own-model deployments. You’ll lead the most consequential architectural programmes in that space and set the engineering standards that 20-50+ engineers build to.\nAs a Principal engineer, you are one of the deepest technical authorities in the company on AI systems. Your decisions set the cost, latency, and reliability at which Nscale serves tokens and runs post-training workloads, and those numbers have to compete with the world’s leading AI infrastructure providers. The problems span the full stack: kernel efficiency on state-of-the-art GPU systems, fleet-level KV cache and serving architecture, the evals that prove model quality, and RL loops where inference and training share hardware. You frame the solutions the organization executes against, including the API contracts customers see.\nHow We Work\nDog years. We move quickly and compress a lot of learning into a short time.\nDon’t let perfect be the enemy of good. Ship, measure, iterate.\nBe relentless. Own the problem end to end and see it through.\n One team, one mission. Outcomes over process, and no “not my job”.\nResponsibilities\nDefine and own the multi-year technical roadmap for Nscale’s inference, evals, and post-training platform, and translate it into architecture that multiple teams can execute against\nLead company-scale architectural initiatives in the pillar, such as next-generation serving (disaggregated prefill/decode, KV cache orchestration across GPU, host, and storage tiers, speculative decoding, multi-tenant scheduling), GPU kernel and model efficiency work (custom kernels, FP8/NVFP4/INT8/4 quantization, sparsity, distillation, MoE serving), evals and benchmarking frameworks, and post-training and RL infrastructure\nEstablish engineering standards adopted across all AI teams: API design and compatibility guarantees, benchmarking and evals methodology, training stability norms, and performance testing practices\nOwn the framework by which cost, latency, throughput, and model quality trade-offs are made and measured across the pillar\nIdentify long-horizon systemic risks early (serving engine and framework bets, accelerator support, capability gaps) and resolve them before they block the organization\nAlign AI engineering, research, product, and infrastructure leadership on multi-team technical strategy; frame technical trade-offs in product and commercial terms\nMentor and develop Staff and Senior AI Engineers, and grow the next generation of inference technical leaders at Nscale\nRepresent Nscale’s technical approach externally: open-source leadership in the frameworks we depend on, publications, conference talks, and partnerships with GPU vendors and AI labs\nRequirements\n10-15 years of engineering experience, with a clear track record of pillar-level impact on production AI systems\n4+ years of hands-on work with LLMs in inference, GPU performance, evals, or post-training and RL, in production or research\nDemonstrated ability to define multi-year technical strategy for complex, multi-team AI systems organizations\nWorld-class depth in production LLM inference, GPU performance, evals, and/or post-training and RL infrastructure, with strong working knowledge across the rest\nDemonstrated ownership of the architecture of a large-scale production inference or training platform\nProven ability to create architectural frameworks and engineering standards adopted across large engineering organizations\nDeep understanding of the hardware/software boundary for AI accelerators: CUDA or ROCm, memory bandwidth and interconnect constraints, and distributed compute paradigms\nStrong history of growing technical leaders (Staff and above) and multiplying technical capability across teams\nExternal recognition in the AI systems community through research, open source, or industry contribution\nPreferred\nPrior experience at a top-tier AI lab or major hyperscaler AI infrastructure team\nMaintainer or core contributor to a foundational inference, kernel, or RL framework (vLLM, SGLang, TensorRT-LLM, LMCache, FlashInfer, Triton, verl, OpenRLHF, TRL, DeepSpeed, Megatron-LM, etc.)\nHands-on depth in RL for LLMs (DPO/GRPO-style methods, reward modelling, multi-turn and tool-use RL) and the interaction between inference and training infrastructure\nExperience defining developer API platforms adopted at scale by external developers\nDeep experience with control plane / data plane architecture and cell-based deployment patterns in large-scale inference infrastructure\nPublished work in AI systems: MLSys, NeurIPS Systems Track, OSDI, EuroSys, SC, or equivalent\nExperience with hardware-software co-design: custom accelerator kernels (CUDA, Triton), compiler-level optimization, AI hardware roadmap engagement\nExperience defining pricing, SLO, and capacity models for a commercial inference product\nThe range below reflects the base salary for the position. Actual compensation may vary based on job-related factors such as skill set, experience, education, and location. In addition to base salary, this role may be eligible for bonus, equity, and/or commission programs. Nscale may offer a competitive benefits package including medical, dental, vision, flexible paid time off, parental leave, and retirement plan participation.\nSalary Range\n$290,000—$520,000 USD\nFor information on how Nscale handles candidate personal data, please see our Employee & Candidate Privacy Notice: Here.\nNscale does not accept unsolicited candidate submissions from recruitment agencies.","description_format":"text","description_chars":6463,"description_truncated":false,"requirements":{"experience_years_min":10,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Parental leave","Retirement plans"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Data Centers & Colocation","AI Compute & Inference"],"lifecycle":[{"event":"open","at":"2026-09-30T18:12:12Z"},{"event":"close","at":"2026-10-09T18:19:11Z"},{"event":"reopen","at":"2026-10-15T00:28:39Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 10","filings_12m":10,"filings_prev_12m":0,"green_card_filings_12m":0,"median_offered_wage_usd":198500,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":3}],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":14,"expected_fill_days":68,"reasons":["conf:1","velocity","win:early","comp:brand"],"computed_at":"2026-10-15T05:46:00Z"},"pay":{"stated_usd_annual":520000,"is_top_pay":true},"html_url":"https://alion.io/job/nscale-principal-ai-product-engineer","json_url":"https://alion.io/job/nscale-principal-ai-product-engineer.json","meta":{"generated_at":"2026-10-16T00:24:50Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":768,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":48155},"rest":"https://alion.io/mcp/rest/get_company?id=48155"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fnscale-principal-ai-product-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fnscale-principal-ai-product-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fnscale-principal-ai-product-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/nscale-principal-ai-product-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fnscale-principal-ai-product-engineer"}]}