{"id":836305,"url":"https://alion.io/job/elorian-inference-infrastructure-engineer-serving","title":"Inference Infrastructure Engineer, Serving","company":{"id":674578,"name":"Elorian","domain":"elorian.ai","url":"https://alion.io/company/elorian","size_band":"11-50","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":null},"role":"DevOps","role_family":"DevOps","seniority":"middle","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Palo Alto, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":200000,"max":400000,"currency":"USD","period":"year","gross":null,"usd_annual":400000},"salary_estimate":null,"experience_years_min":3,"visa_sponsorship":true,"relocation_package":true,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"CUDA","optional":false},{"name":"KV Cache","optional":false},{"name":"Multimodal AI","optional":false},{"name":"Python","optional":false},{"name":"Quantization","optional":false},{"name":"SGLang","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"Triton","optional":false},{"name":"vLLM","optional":false},{"name":"Gemini","optional":true},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-07-22T17:35:25Z","employer_posted_date":"2026-07-22","last_verified_at":"2026-11-07T19:52:41Z","board_verified":true,"closed_at":null,"days_open":108,"trust":{"level":"ok","repost_count":0,"flags":[],"days_open":107},"description":"About Us\nWe are a well-funded, early-stage AI lab focused on building the next generation of frontier multimodal AI models. Founded by former DeepMind researchers, including Andrew Dai, who was previously a leader on Gemini. Our team currently consists of 20 world-class scientists and engineers. We recently raised $55M in seed funding from Striker Ventures, Menlo Ventures, Altimeter Capital, and NVIDIA. We are tackling some of the hardest problems in artificial intelligence, and we are growing fast.\nAbout the Role\nWe're looking for an infrastructure engineer to design, optimize, and scale the systems that serve our large multimodal models. Your work will make inference faster, more cost-effective, and more reliable, so our teams can focus on advancing model capabilities rather than managing bottlenecks.\nOur focus is on performant, efficient inference, both to power real-world applications and to accelerate research. This role owns the infrastructure that ensures every deployment and evaluation runs smoothly at scale for our visual foundation models.\nWhat You Will Do\nBuild low-latency, high-throughput inference serving systems for our large multimodal models\n\nDesign and implement techniques that improve latency, throughput, and efficiency, including quantization, batching, speculative decoding, and KV cache management\n\nOptimize our codebase and GPU fleet to fully utilize hardware FLOPs, bandwidth, and memory\n\nImplement multi-GPU and multi-node model parallelism for serving (tensor or pipeline parallel)\n\nBuild autoscaling and load balancing for production ML services\n\nEstablish standards for reliability, observability, and reproducibility across the inference stack\n\nCollaborate with researchers to enable high-performance inference for novel architectures\n\nSkills and Qualifications\nMinimum qualifications:\n3+ years of experience building low-latency, high-throughput inference serving systems for large models\n\nStrong knowledge of inference optimization techniques (quantization, batching, speculative decoding, KV cache management)\n\nHands-on experience with serving frameworks such as vLLM, TensorRT-LLM, Triton, or SGLang\n\nExperience with multi-GPU/multi-node model parallelism for serving (tensor or pipeline parallel)\n\nStrong systems programming skills; C++/CUDA a plus alongside Python\n\nExperience with autoscaling and load balancing for production ML services\n\nA track record of GPU cost optimization at scale\n\nPreferred qualifications (strong candidates may have some, not all):\nExperience serving multimodal (vision + language) models\n\nContributions to open-source ML or systems infrastructure projects (e.g., vLLM, SGLang, TensorRT-LLM, Triton)\n\nA bias for action and comfort working across stacks and teams in an early-stage environment\n\nLogistics\nLocation: This role is based on-site in Palo Alto, California.\n\nCompensation: Depending on background, skills, and experience, the expected annual base salary range for this position is $200,000 - $400,000 USD, plus equity and benefits.\n\nVisa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.\n\nBenefits: We offer health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.\n\nElorian AI is an equal opportunity employer. We are committed to building a diverse team and inclusive environment.","description_format":"text","description_chars":3452,"description_truncated":false,"requirements":{"experience_years_min":3,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Parental leave","Unlimited PTO"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":true,"industries":["Multimodal AI","AI Research Labs"],"lifecycle":[{"event":"open","at":"2026-09-12T19:11:52Z"}],"visa":[{"country":"US","licensed_sponsor":true,"adverts_offering_sponsorship_24m":0,"sanctioned_now":false,"evidence":"H-1B filings in 12 months: 3","filings_12m":3,"filings_prev_12m":0,"green_card_filings_12m":0,"median_offered_wage_usd":402500,"route":null,"cap_exempt":false,"checked_at":"2026-11-07T07:57:00+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":1}],"liveness":{"score":51,"band":"ok","label":"Likely open","p_open":1,"p_active":0.85,"p_room":0.6,"age_days":107,"expected_fill_days":131,"reasons":["conf:2","velocity","win:late","crowd:"],"computed_at":"2026-11-07T05:47:08Z"},"pay":{"stated_usd_annual":400000,"is_top_pay":true},"html_url":"https://alion.io/job/elorian-inference-infrastructure-engineer-serving","json_url":"https://alion.io/job/elorian-inference-infrastructure-engineer-serving.json","meta":{"generated_at":"2026-11-07T21:28:19Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":34697,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-08T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":674578},"rest":"https://alion.io/mcp/rest/get_company?id=674578"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Felorian-inference-infrastructure-engineer-serving"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Felorian-inference-infrastructure-engineer-serving"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Felorian-inference-infrastructure-engineer-serving"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/elorian-inference-infrastructure-engineer-serving\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Felorian-inference-infrastructure-engineer-serving"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":836305}},{"tool":"search_jobs","arguments":{"company":"elorian.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":836305},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}