Overview
News
Technologies
Salaries
Products
People
Growth
Financials
Overview
Hi, I’m Changyi Yang . I was reading this post by Su Jianlin when one sentence stopped me:Besides the KV cache, decoding now has another variable — MTP, or speculative decoding, whose idea is to trade compute for speed.
News
Tech Why MLA and MTP Fight Each Other: Attention Through Arithmetic Intensity
Reduce MHA to its arithmetic intensity and you get exactly 1. GQA, MQA and MLA come out just as clean -- head counts and nothing else. That last constant is what puts MLA at odds with MTP.
Read more
Report
The Bitter Lesson of Agentic Collaboration (24h Hackathon Takeaways)
TL;DR : Agent execution comes down to one question - how do you sensibly parallelize a single task across multiple agents? That's the Bitter Lesson playing out at the collaboration layer, and it's what 24 hours at a hackathon made obvious to me. I recentl
Read more
Report
Tech Delta Weight Sync: When RL's Weight Sync Stops Being a Network Problem
TL;DR In LLM RL post-training, every optimizer step on the trainer has to push fresh weights into the inference engine doing rollouts. Under a disaggregated training/inference architecture, this has long been treated as a problem that demands high-bandwid
Read more
Report
Tech How Does SGLang Support LoRA?
I recently started looking into LoRA + RL, and as a side project I dug into how SGLang implements LoRA serving. This post is a concise walkthrough. TL;DR At a high level, SGLang's LoRA support looks like this (source: https://arxiv.org/abs/2311.03285) : i
Read more
Report
Tech How Does SGLang Actually Load Model Weights?
When you spin up an LLM with SGLang, you get this satisfying sequence: progress bars flying, safetensors loading, and then - boom - the model starts generating tokens. But have you ever wondered what's actually happening under the hood? From allocating GP
Read more
Report
Pro access
Upgrade to see all 13 mentions
Upgrade to a paid plan to read every media mention of this company - funding news, awards, product launches and press releases from all the outlets writing about it.
Every media mention and press release
Funding news, awards and product launches
Fresh coverage from every outlet writing about the company
Upgrade now
Cancel anytime. Secure checkout. Instant activation.