Semianalysis

The @vllm_project maintainers at @inferact are some of the most cracked engineers in the world. They're building one of the inference engines that powers much of the world's intelligence-and doing so with remarkable dedication, kindness, and hard work.

Premium-priced "fast modes" are proving that users will pay more for lower latency and faster tokens, potentially yielding higher gross margins. Frontier AI labs such as OpenAI are therefore evaluating purpose-built inference systems, including Cerebras a

Elon Musk shocked the world, once again, when he announced on SpaceX's first earnings his Gigawatt ambitions for next year. He "conservatively" aims to build & deliver an incremental 6-8GW in 2027 alone, with potential for that number to be well above +10

On Wednesday, August 5th, Google announced a complete overhaul of DeepMind leadership. A quick recap:Demis Hassabis, DeepMind co-founder and former CEO, is no longer involved in day-to-day operations.Jeff Dean, former Google Chief Scientist and Gemini co-

Kimi K3 took the world by storm at its announcement, sweeping leaderboards and establishing itself as the open frontier model. While the community is eager to understand how Kimi K3 works, many have been surprised by the unconventional techniques driving

The Labor Problem and Modularization to the Rescue

vLLM delivers day-0 Kimi K3 serving with hybrid KDA prefix caching, DSpark speculative decoding, production-scale disaggregation, and optimized kernels across N

Agentic Kernel Generation, Improvement in Software Quality, Unstable Internal Development Clusters, Helios MI455X Production Ramp Hell, Up to 105% Discounts from Finance Engineering

Meta Infrastructure has become bloated, with middle managers expending resources on over-engineered technology solutions that lose sight of broader organizational needs.

We cut open the Kirin 9030 and put it under an electron microscope.The smallest metal pitch measures 32.5 nanometers. That is tighter than Intel 18A, their brand new leading edge node.A Chinese fab with no EUV is out-pitching Intel's EUV node by roughly 1