LLM Inference Systems Engineer
Confirmed on the employer's own hiring board on Oct 17, 2026. First seen by Alion on Oct 15, 2026. Hippocratic AI scores B on the Alion truth index.
Join our team as an LLM Inference Systems Engineer, where you will be responsible for building the distributed serving and high-performance networking layer behind our large language models. You will work closely with the Inference team, Model team, Infrastructure team, and Application engineering team. Your tasks will include designing and implementing disaggregated LLM serving architectures, optimizing multi-node inference, profiling end-to-end performance, and diagnosing complex distributed failures. A Master's or PhD is required, along with experience in deploying production workloads on Kubernetes, strong Python skills, and a strong understanding of LLM inference.
Missions
- Conception, implementation, and operation of disaggregated LLM serving architectures, including independent prefill and decode pools, KV-cache transfer, routing, batching, and failure recovery.
- Optimization of multi-node inference across modern GPU systems using various communication paths and frameworks.
- Productionization of serving on Kubernetes with health checks, autoscaling, safe rollouts, metrics, tracing, and actionable diagnostics.
Profil recherché
- Clear written and verbal communication across research, infrastructure, and product-facing teams- Master's or PhD Required
- Experience deploying production workloads on Kubernetes and operating them with clear reliability and observability standards
- Strong Python skills and working proficiency in C++ or another systems language
- Ability to profile and debug performance across application, runtime, kernel, network, and infrastructure layers
- Strong understanding of LLM inference, including tensor/pipeline/data parallelism, continuous batching, KV-cache management, and prefill/decode behavior
- Production experience building or operating distributed systems, high-performance computing systems, or large-scale ML inference platforms
- Hands-on experience with GPU communication and networking technologies such as NCCL, NVLink/NVSwitch, InfiniBand, RoCE, RDMA, UCX, or equivalent systems
- Direct experience with disaggregated serving, KV-cache transfer, NVIDIA Dynamo/NIXL, SGLang, vLLM, or TensorRT-LLM
- CUDA, Triton, custom kernel, or low-level GPU performance experience
- Experience with NVLS, SHARP, GPUDirect RDMA, UCX, RDMA congestion control, or GPU-cluster topology optimization
- Experience with speculative decoding, multi-LoRA serving, quantization, or cache-aware routing
- Contributions to open-source inference, networking, or distributed-systems projects
- Experience benchmarking new GPU platforms and turning results into production architecture decisions