Tridao
tridao.me
Actively hiring
Blockchain & Crypto
DAO
Follow
Overview
News
Salaries
Products
People
Growth
Offices
Financials
News
ReplaySSM: Cache SSM Inputs, Not State
Math behind SSMs: the state update and output read out (skip if familiar)
Read more
Report
SonicMoE: A Hardware-Efficient and Software-Extensible Blueprint for Fine-Grained MoEs
.post img{max-width:100%;height:auto}.post blockquote p{margin-top:.2em;margin-bottom:.2em;line-height:1.4}.post blockquote p:first-child{margin-top:0}.post blockquote p:nth-child(2),.post blockquote p:nth-child(3){margin-bottom:1em}.post blockquote stron
Read more
Report
Gram Newton-Schulz: A Fast, Hardware-Aware Newton-Schulz Algorithm for Muon
.post img{max-width:100%;height:auto}.post blockquote p{margin-top:.2em;margin-bottom:.2em;line-height:1.4}.post blockquote p:first-child{margin-top:0}.post blockquote p:nth-child(2),.post blockquote p:nth-child(3){margin-bottom:1em}.post blockquote stron
Read more
Report
Report
Report
FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
Paper Code
Read more
Report
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
Paper Code
Read more
Report
State Space Duality (Mamba-2) Part III - The Algorithm
Part I - The Model Part II - The Theory Part III - The Algorithm Part IV - The Systems
Read more
Report
State Space Duality (Mamba-2) Part I - The Model
Paper Code This series is cross-posted at GoombaLab Part I - The Model Part II - The Theory Part III - The Algorithm Part IV - The Systems Since the release of Mamba 6 months ago, we've been pleasantly surprised by the overwhelming community response. It'
Read more
Report
Report