Fergusfinn
fergusfinn.com
Actively hiring
Digital Media
Follow
Overview
News
Salaries
Products
People
Growth
Offices
Financials
News
The case for disaggregated LLM serving
Three checkable conditions under which disaggregated prefill is never worse than aggregated serving, and the reasons it's usually better. Plus an inventory-theory model of traffic drift: cold starts, safety stock, and a map of the shocks a deployment can
Read more
Report
Throughputmaxxing: DeepSeek-V4-Flash on Isambard-AI
Single-node serving throughput for DeepSeek-V4-Flash: baseline, MLA/DP-attention, and MoE kernel choice.
Read more
Report
NVLink, NVSwitch, and all that
Why scale-up links are fast and short, how NVSwitch grew the domain from a board to a rack, and what TPU, UALink, and scale-up Ethernet do differently.
Read more
Report
Reverse-engineering NVIDIA's cuda-checkpoint for faster cold starts
Freezing a live CUDA process to host memory and thawing it again, what the driver does - and doesn't - do to make that work, and how understanding that lets us restore CUDA processes up to 4x faster.
Read more
Report
Width vs. depth: speculating on the margin
Some thinking about how to trade off batching and speculative decoding in a running inference engine.
Read more
Report
What happens when you run a CUDA kernel
Tracing one vector-add kernel from nvcc all the way down to the warps that execute it.
Read more
Report
Adaptive speculative decoding: picking draft lengths at runtime
A follow-on to the economics of speculative decoding, we run the inference lab simulator on MTP & DFlash drafters with real acceptance data, and find out whether adaptively choosing the draft length is worth it.
Read more
Report
InfiniBand, RoCE, and all that
How a technically superior but economically isolated solution slowly lost ground to a good-enough one built on infrastructure everyone already owns.
Read more
Report
UCCL-EP: An expert parallel communications kernel without owning the NIC
How UCCL reimplements the DeepEP kernels for arbitrary hardware by swapping the transport out from under them.
Read more
Report
Anatomy of a high-performance EP kernel
How expert-parallel dispatch and combine kernels work, built up from scratch: the high-throughput shape and the low-latency one.
Read more
Report
Show more news