{"id":2577258,"url":"https://alion.io/job/hippocratic-ai-llm-inference-systems-engineer","title":"LLM Inference Systems Engineer","company":{"id":4396,"name":"Hippocratic AI","domain":"hippocraticai.com","url":"https://alion.io/company/hippocratic-ai","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Welcome to the Jungle","truth_index":{"grade":"B","score":75,"open_postings":3,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-19T05:45:15Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Menlo Park, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":157000,"max_usd":343000,"period":"year","method":"role_country_seniority_unknown","sample_n":4027},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"CUDA","optional":false},{"name":"CUDA Toolkit","optional":false},{"name":"InfiniBand","optional":false},{"name":"Kubernetes","optional":false},{"name":"KV Cache","optional":false},{"name":"LLM","optional":false},{"name":"LoRA","optional":false},{"name":"NCCL","optional":false},{"name":"NVLink","optional":false},{"name":"Python","optional":false},{"name":"Quantization","optional":false},{"name":"SGLang","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"Triton","optional":false},{"name":"vLLM","optional":false},{"name":"PEFT","optional":true},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-10-15T00:05:04Z","employer_posted_date":"2026-10-15","last_verified_at":"2026-10-19T22:27:31Z","board_verified":true,"closed_at":null,"days_open":4,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":4},"description":"Join our team as an LLM Inference Systems Engineer, where you will be responsible for building the distributed serving and high-performance networking layer behind our large language models. You will work closely with the Inference team, Model team, Infrastructure team, and Application engineering team. Your tasks will include designing and implementing disaggregated LLM serving architectures, optimizing multi-node inference, profiling end-to-end performance, and diagnosing complex distributed failures. A Master's or PhD is required, along with experience in deploying production workloads on Kubernetes, strong Python skills, and a strong understanding of LLM inference.\nMissions\nConception, implementation, and operation of disaggregated LLM serving architectures, including independent prefill and decode pools, KV-cache transfer, routing, batching, and failure recovery.\nOptimization of multi-node inference across modern GPU systems using various communication paths and frameworks.\nProductionization of serving on Kubernetes with health checks, autoscaling, safe rollouts, metrics, tracing, and actionable diagnostics.\nProfil recherché\n - Clear written and verbal communication across research, infrastructure, and product-facing teams- Master's or PhD Required\n- Experience deploying production workloads on Kubernetes and operating them with clear reliability and observability standards\n- Strong Python skills and working proficiency in C++ or another systems language\n- Ability to profile and debug performance across application, runtime, kernel, network, and infrastructure layers\n- Strong understanding of LLM inference, including tensor/pipeline/data parallelism, continuous batching, KV-cache management, and prefill/decode behavior\n- Production experience building or operating distributed systems, high-performance computing systems, or large-scale ML inference platforms\n- Hands-on experience with GPU communication and networking technologies such as NCCL, NVLink/NVSwitch, InfiniBand, RoCE, RDMA, UCX, or equivalent systems\n- Direct experience with disaggregated serving, KV-cache transfer, NVIDIA Dynamo/NIXL, SGLang, vLLM, or TensorRT-LLM\n- CUDA, Triton, custom kernel, or low-level GPU performance experience\n- Experience with NVLS, SHARP, GPUDirect RDMA, UCX, RDMA congestion control, or GPU-cluster topology optimization\n- Experience with speculative decoding, multi-LoRA serving, quantization, or cache-aware routing\n- Contributions to open-source inference, networking, or distributed-systems projects\n- Experience benchmarking new GPU platforms and turning results into production architecture decisions","description_format":"text","description_chars":2637,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"master","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Conversational AI","AI Agents","Medical AI"],"lifecycle":[{"event":"open","at":"2026-10-15T00:31:03Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 36","filings_12m":36,"filings_prev_12m":6,"green_card_filings_12m":0,"median_offered_wage_usd":235278,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":29}],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.902,"p_room":1,"age_days":4,"expected_fill_days":32,"reasons":["conf:3","velocity","win:early"],"computed_at":"2026-10-19T05:45:15Z"},"pay":null,"html_url":"https://alion.io/job/hippocratic-ai-llm-inference-systems-engineer","json_url":"https://alion.io/job/hippocratic-ai-llm-inference-systems-engineer.json","meta":{"generated_at":"2026-10-19T23:02:54Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":1979,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-20T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4396},"rest":"https://alion.io/mcp/rest/get_company?id=4396"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fhippocratic-ai-llm-inference-systems-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fhippocratic-ai-llm-inference-systems-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fhippocratic-ai-llm-inference-systems-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/hippocratic-ai-llm-inference-systems-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fhippocratic-ai-llm-inference-systems-engineer"}]}