Lmsys
Nonprofits & Foundations · AI Evaluation & Observability
News
Miles v0.1: Production-level Post-training
We present Miles v0.1, a full-stack production-ready system for frontier post-training, the successor to our first Miles release. Building upon slime's clean design, Miles optimizes every stage in the...
Read more
Report
Agent-Assisted SGLang Development: An Initial Exploration
SGLang development increasingly goes beyond isolated code changes. The same repository now spans LLM serving, distributed runtime, GPU kernels, diffusion pipelines, model-specific execution paths, and...
Read more
Report
The next generation of speculative decoding: DFlash and Spec V2
Using Modal and Z Lab's DFlash speculative decoding models with SGLang's newly default Spec V2 engine, you can achieve state-of-the-art latencies for LLM inference serving. Our new, jointly-released D...
Read more
Report
No Token Left Behind: Demystifying Token-In-Token-Out in Miles
In agentic RL, a rollout is not a single generation. It is a chain of model calls, tool outputs, harness messages, and resumed generations. Token-In-Token-Out (TITO) is a design principle that address...
Read more
Report
DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles
We are thrilled to announce Day-0 support for DeepSeek-V4 across both inference and RL training. SGLang and Miles form the first open-source stack to serve and train DeepSeek-V4 on launch day - with s...
Read more
Report
Pipeline Parallelism in SGLang: Scaling to Million-Token Contexts and Beyond
We are excited to introduce SGLang's highly optimized Pipeline Parallelism (PP) implementation, specifically engineered to tackle the challenges of ultra-long context inference. By integrating Chunked...
Read more
Report
SpecBundle & SpecForge v0.2: Production-Ready Speculative Decoding Models and Framework
The SpecForge team has collaborated with multiple industry partners - including Ant, Meituan, Nex-AGI, and EigenAI - to release SpecBundle (Phase 1), a collection of production-grade EAGLE-3 model che...
Read more
Report
Mini-SGLang: Efficient Inference Engine in a Nutshell
We're excited to introduce Mini-SGLang, a lightweight yet high-performance inference framework for Large Language Models (LLMs). Derived from the SGLang project, Mini-SGLang is designed to demystify t...
Read more
Report
NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
Thanks to NVIDIA's early access program, we are thrilled to get our hands on the NVIDIA DGX Spark. It's quite an unconventional system, as NVIDIA rarely releases compact, all-in-one machines that bri...
Read more
Report
Show more news