{"id":3299855,"url":"https://alion.io/job/netpreme-ai-infrastructure-engineer","title":"AI Infrastructure Engineer","company":{"id":680646,"name":"Netpreme","domain":"netpreme.com","url":"https://alion.io/company/netpreme","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":null},"role":"DevOps","role_family":"DevOps","seniority":"junior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Santa Clara, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":89000,"max_usd":180000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":413},"experience_years_min":2,"visa_sponsorship":true,"relocation_package":true,"has_equity":true,"technologies":[{"name":"Ansible","optional":false},{"name":"Function Calling","optional":false},{"name":"Helm","optional":false},{"name":"Kubernetes","optional":false},{"name":"KV Cache","optional":false},{"name":"Linux","optional":false},{"name":"LLM","optional":false},{"name":"Multimodal AI","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"SGLang","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"TPOT","optional":false},{"name":"vLLM","optional":false},{"name":"ChatGPT","optional":true},{"name":"Claude","optional":true},{"name":"Quantization","optional":true},{"name":"Speculative Decoding","optional":true},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-09-08T18:47:59Z","employer_posted_date":"2026-09-08","last_verified_at":"2026-11-08T00:01:21Z","board_verified":true,"closed_at":null,"days_open":60,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":60},"description":"About Netpreme\nThe biggest bottleneck in AI inference today is memory. Chips can compute faster than they can be fed data, a problem known as the memory wall, and Netpreme is building the solution. Our approach spans the full stack, from a proprietary memory management library down to X-Mem™, our Memory Processing Unit (MPU) silicon. Netpreme was co-founded by researchers in optical networking and ML systems. We’re a small team in Cambridge, MA and Santa Clara, CA, and we’re looking for people who want to own hard problems end to end and help shape where the company goes next.\nAbout the Role\nServing frontier LLMs efficiently at scale is a moving target: every new model and workload reshuffles the tradeoffs across parallelism, batching, KV caching, and scheduling. You’ll own Netpreme’s inference-serving stack end to end, from tuning vLLM, SGLang, and TensorRT-LLM to architecting distributed serving with NVIDIA Dynamo (prefill/decode disaggregation, KV-cache-aware routing). Working directly with our hardware and systems engineers, you’ll build the benchmarks and drive the optimizations that take new models from research to production.\nThis role will be performed onsite from one of our offices in Santa Clara, CA or Boston, MA (relocation assistance available).\nThe Opportunity\nImpact: AI is advancing faster than the infrastructure underneath it. We’re working at that layer, unlocking more capability from every GPU while dramatically improving efficiency. The work you do here compounds across state-of-the-art models, systems, and real-world applications.\n\nTiming: Joining now means real ownership, including equity, and a direct say in product direction. You’ll work from first principles, move quickly from insight to execution, and see your work show up in what we ship, not just in a roadmap.\n\nCulture: We care about rigor, clarity, and impact. Disagreement is welcome when it’s thoughtful, the best idea wins regardless of who it comes from, and you’ll learn something new most weeks.\n\nYour Impact\nDeploy and operate models in production: Run large language and multimodal models using inference engines like vLLM, SGLang, and TensorRT-LLM.\n\nOwn the serving infrastructure: Build and maintain distributed inference systems with NVIDIA Dynamo, Kubernetes, Helm, and Ansible, including upgrades, monitoring, troubleshooting, and reliability.\n\nOptimize performance: Tune parallelism strategies (TP/EP/DP/PP), batching, KV caching, routing, and scheduling to meet latency, throughput, and SLA targets.\n\nBenchmark and measure: Build reproducible benchmarks for latency (TTFT, TPOT, E2E), throughput, concurrency scaling, and system efficiency.\n\nSupport diverse workloads: Enable tool calling, agentic and coding workflows, RAG, and long-context applications.\n\nBring new tech into production: Evaluate new models, frameworks, and optimizations, and partner with hardware, model, and systems engineers to ship improvements.\n\nQualifications\nBS, MS, or PhD in Computer Science, Computer Engineering, or a related field, or equivalent practical experience.\n\n2+ years of engineering experience in LLM inference, ML systems, GPU systems, or related infrastructure.\n\nHands-on experience deploying and optimizing at least one LLM inference engine (e.g., vLLM, SGLang, or TensorRT-LLM).\n\nSolid understanding of LLM inference fundamentals: prefill/decode, batching, KV caching, and latency/throughput tradeoffs.\n\nStrong Python and Linux skills, plus hands-on experience with Kubernetes and containerized deployments.\n\nComfortable benchmarking and troubleshooting systems, and communicating clearly in a small engineering team.\n\nNice to Have\nExperience with Helm, Ansible, or similar deployment and configuration tools.\n\nExperience with distributed inference systems like NVIDIA Dynamo, including parallelism strategies (TP/EP/DP/PP), prefill/decode disaggregation, and KV-cache-aware routing.\n\nExperience with advanced inference optimizations such as prefix caching, speculative decoding, and quantization.\n\nExperience analyzing LLM architectures and tuning deployments across different hardware platforms to meet SLAs.\n\nContributions to vLLM, SGLang, TensorRT-LLM, or related open-source projects.\n\nExperience owning production infrastructure, ideally at an early-stage startup.\n\nWhat We Offer\nCompetitive salary with performance-based bonus and early-stage equity grant\n\n100% employer-paid Health, Dental, and Vision coverage for you and your dependents\n\n401(k) match with immediate vesting, and access to financial advisors to help you reach your financial goals\n\n100% employer-paid Life, Disability, and AD&D insurance, plus a fitness stipend and wellness & mental health perks\n\nGenerous PTO: 20 vacation days, 15 company holidays (including 3 floating days of your choosing)\n\nDaily lunch stipend\n\nEnterprise-level Claude & ChatGPT access with a generous token budget\n\nWell-equipped, sunny offices in Santa Clara, CA & Cambridge, MA with on-site parking and EV charging; on-site fitness center in Santa Clara; gym discounts near our Cambridge office\n\nRelocation assistance available to work in Santa Clara, CA or Boston, MA\n\nVisa sponsorship and relocation assistance to one of our office hubs\n\nA collaborative, continuous-learning environment with smart, dedicated colleagues building the next generation of high-performance computing architecture","description_format":"text","description_chars":5361,"description_truncated":false,"requirements":{"experience_years_min":2,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Equity","Relocation assistance"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":true,"industries":["Servers & Data Center Hardware","Memory & Storage Chips"],"lifecycle":[{"event":"open","at":"2026-10-26T23:19:16Z"}],"visa":[{"country":"US","licensed_sponsor":true,"adverts_offering_sponsorship_24m":6,"sanctioned_now":false,"evidence":"H-1B filings in 12 months: 1","filings_12m":1,"filings_prev_12m":0,"green_card_filings_12m":0,"median_offered_wage_usd":222500,"route":null,"cap_exempt":false,"checked_at":"2026-11-07T07:57:00+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","Alion: sponsorship offered in the employer's own job adverts (24 months)"],"filings_for_role_12m":0}],"liveness":{"score":13,"band":"cold","label":"Long shot","p_open":1,"p_active":0.471,"p_room":0.28,"age_days":59,"expected_fill_days":29,"reasons":["conf:1","win:tail","crowd:junior"],"computed_at":"2026-11-07T05:47:08Z"},"pay":null,"html_url":"https://alion.io/job/netpreme-ai-infrastructure-engineer","json_url":"https://alion.io/job/netpreme-ai-infrastructure-engineer.json","meta":{"generated_at":"2026-11-08T01:16:17Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2755,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-09T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":680646},"rest":"https://alion.io/mcp/rest/get_company?id=680646"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fnetpreme-ai-infrastructure-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fnetpreme-ai-infrastructure-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fnetpreme-ai-infrastructure-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/netpreme-ai-infrastructure-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fnetpreme-ai-infrastructure-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3299855}},{"tool":"search_jobs","arguments":{"company":"netpreme.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3299855},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}