Vllm

The @vllm_project maintainers at @inferact are some of the most cracked engineers in the world. They're building one of the inference engines that powers much of the world's intelligence-and doing so with remarkable dedication, kindness, and hard work.

TAIPEI, Taiwan, Aug. 10, 2026 (GLOBE NEWSWIRE) -- TAIONE Open Source Foundation and Embedded LLM today announced a collaboration to build Taiwan...

From paged attention, continuous batching, prefix caching, specdec, etc. to multi-GPU, multi-node dynamic serving at scale.

We believe Mixture-of-Models is the next-generation model architecture for heterogeneous LLM inference. vLLM Semantic Router makes it executable.

The Inference Engine Guide for K3 Deployment

vLLM delivers day-0 Kimi K3 serving with hybrid KDA prefix caching, DSpark speculative decoding, production-scale disaggregation, and optimized kernels across N

Doubling my GPU count raised throughput 61% - and still lost to a one-line config change. A roofline-guided tour of what actually moves production serving metrics.

vLLM prefill paired with TileRT decode through vLLM V1's connector interface: a specialized, latency-optimized decode engine that coexists with native vLLM deco

We're on a journey to advance and democratize artificial intelligence through open source and open science.

How vLLM Semantic Router turns vllm-sr/auto into a bounded micro-agent runtime for Confidence, Ratings, ReMoM, Fusion, Workflows, and benchmark-shaped collabora