The Frontier Lab
thefrontierlab.ai
Actively hiring
LLM & Generative AI
Follow
Overview
News
Technologies
Salaries
Products
People
Growth
Financials
Overview
Weekly newsletter on running frontier AI on hardware you control. Model comparisons, configs, benchmarks, honest assessments — by Erik.
News
Speculative Decoding Across My Fleet: Gains From +200% to -80%, and No Single Setting Fits Every Model
Seven current models on the bench, five speculative paths that could actually be measured, one build, one evening. Speculative decoding runs from tripling a model's speed to making it five times slower, the sweet spots spread from depth 3 to depth 5, and
Read more
Report
Quantized KV Cache Helps Six of My Nine Models and Actively Hurts the Two Newest Ones
The usual advice is to quantize your KV cache because it is nearly free. That holds for six of the models I keep on this box, and it is wrong for the two with the newest attention designs, one of which loses 37 percent of its prompt speed.Every few weeks
Read more
Report
I Rebuilt the Box, and the Cliff Was Gone. Then I Spent an Evening Proving Which Patch Did It
Part four of the tuning series. A routine platform upgrade turned into the answer to the question that has run through this whole series. The dense flash-attention collapse that cost me half my prompt processing at long context is not there anymore, and i
Read more
Report
A Reader Fixed Quantized KV Cache on Strix Halo. I Verified It at 262k Context, and His Theory About My Dense Cliff Survived Half Its Test
Part three of the tuning series, and the most community-built one yet. A reader published two fixes that make quantized KV cache fast on this hardware. I verified them on a 128GB box, pushed them to the model's full 262k native context, and then tested hi
Read more
Report
Turning Off the IOMMU Made My Dense Model 37% Faster. Five More Strix Halo Tips, Measured, and One Crashed the GPU
Part two of the tuning series. The kernel parameter I had quietly dismissed turns out to be the biggest system-level lever on this box: +34 to +38% dense prompt processing. The rest of the week's list mostly measured out to nothing, one setting reproducib
Read more
Report
Pro access
Upgrade to see all 15 mentions
Upgrade to a paid plan to read every media mention of this company - funding news, awards, product launches and press releases from all the outlets writing about it.
Every media mention and press release
Funding news, awards and product launches
Fresh coverage from every outlet writing about the company
Upgrade now
Cancel anytime. Secure checkout. Instant activation.