{"id":2577258,"url":"https://alion.io/job/hippocratic-ai-llm-inference-systems-engineer","title":"LLM Inference Systems Engineer","company":{"id":4396,"name":"Hippocratic AI","domain":"hippocraticai.com","url":"https://alion.io/company/hippocratic-ai","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Welcome to the Jungle","truth_index":{"grade":"B","score":80,"open_postings":3,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":20,"computed_at":"2026-10-22T06:10:10Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Menlo Park, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":156000,"max_usd":344000,"period":"year","method":"role_country_seniority_unknown","sample_n":4194},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"CUDA","optional":false},{"name":"InfiniBand","optional":false},{"name":"Kubernetes","optional":false},{"name":"KV Cache","optional":false},{"name":"LLM","optional":false},{"name":"LoRA","optional":false},{"name":"NCCL","optional":false},{"name":"NVLink","optional":false},{"name":"Python","optional":false},{"name":"Quantization","optional":false},{"name":"SGLang","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"Triton","optional":false},{"name":"vLLM","optional":false},{"name":"PEFT","optional":true},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-10-15T00:05:04Z","employer_posted_date":"2026-10-23","last_verified_at":"2026-10-23T00:34:56Z","board_verified":true,"closed_at":null,"days_open":8,"trust":{"level":"ok","repost_count":1,"flags":[],"days_open":8},"description":"Join our team as an LLM Inference Systems Engineer, where you will be responsible for building the distributed serving and high-performance networking layer behind our large language models. You will work closely with the Model, Infrastructure, and Application engineering teams to design, implement, and operate disaggregated LLM serving architectures. Your expertise in optimizing multi-node inference across modern GPU systems and diagnosing complex distributed failures will be crucial to the success of this role. A Master's or PhD is required, along with experience in deploying production workloads on Kubernetes and a strong understanding of LLM inference.\nMissions\nConception, mise en œuvre et exploitation d'architectures de service LLM désagrégées, y compris les pools de pré-remplissage et de décodage indépendants.\nOptimisation de l'inférence multi-nœuds sur des systèmes GPU modernes en utilisant NVLink, NVSwitch, NVLS, InfiniBand, RoCEv2, GPUDirect RDMA, NCCL, UCX.\nProduction de services sur Kubernetes avec des vérifications de santé, une mise à l'échelle automatique, des déploiements sûrs, des métriques, un traçage et des diagnostics exploitables.\nProfil recherché\n - Clear written and verbal communication across research, infrastructure, and product-facing teams- Master's or PhD Required\n- Experience deploying production workloads on Kubernetes and operating them with clear reliability and observability standards\n- Strong Python skills and working proficiency in C++ or another systems language\n- Ability to profile and debug performance across application, runtime, kernel, network, and infrastructure layers\n- Strong understanding of LLM inference, including tensor/pipeline/data parallelism, continuous batching, KV-cache management, and prefill/decode behavior\n- Production experience building or operating distributed systems, high-performance computing systems, or large-scale ML inference platforms\n- Hands-on experience with GPU communication and networking technologies such as NCCL, NVLink/NVSwitch, InfiniBand, RoCE, RDMA, UCX, or equivalent systems\n- Direct experience with disaggregated serving, KV-cache transfer, NVIDIA Dynamo/NIXL, SGLang, vLLM, or TensorRT-LLM\n- CUDA, Triton, custom kernel, or low-level GPU performance experience\n- Experience with NVLS, SHARP, GPUDirect RDMA, UCX, RDMA congestion control, or GPU-cluster topology optimization\n- Experience with speculative decoding, multi-LoRA serving, quantization, or cache-aware routing\n- Contributions to open-source inference, networking, or distributed-systems projects\n- Experience benchmarking new GPU platforms and turning results into production architecture decisions","description_format":"text","description_chars":2675,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"master","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Conversational AI","AI Agents","Medical AI"],"lifecycle":[{"event":"open","at":"2026-10-15T00:31:03Z"},{"event":"close","at":"2026-10-20T19:21:41Z"},{"event":"reopen","at":"2026-10-23T00:34:56Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 36","filings_12m":36,"filings_prev_12m":6,"green_card_filings_12m":0,"median_offered_wage_usd":235278,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":29}],"liveness":{"score":73,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.777,"p_room":0.945,"age_days":8,"expected_fill_days":20,"reasons":["conf:0","win:mid"],"computed_at":"2026-10-23T01:29:16Z"},"pay":null,"html_url":"https://alion.io/job/hippocratic-ai-llm-inference-systems-engineer","json_url":"https://alion.io/job/hippocratic-ai-llm-inference-systems-engineer.json","meta":{"generated_at":"2026-10-23T01:29:16Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2438,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-24T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4396},"rest":"https://alion.io/mcp/rest/get_company?id=4396"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fhippocratic-ai-llm-inference-systems-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fhippocratic-ai-llm-inference-systems-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fhippocratic-ai-llm-inference-systems-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/hippocratic-ai-llm-inference-systems-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fhippocratic-ai-llm-inference-systems-engineer"}]}