Koleslaw
koleslaw.ai
Actively hiring
Follow
Overview
News
Technologies
Salaries
Products
People
Growth
Financials
News
AWS reclaimed my only GPU seven times in 21 hours
Spot can take the box behind the product with two minutes of notice. Seven reclaims in 21 hours proved out the six design decisions that make that boring.
Read more
Report
4 hours and 37 minutes of serving nothing
Running a 30B model on spot GPUs cut the bill from $588 to $245 a month. First, spot emptied the fleet for 4 hours and 37 minutes, because an armed 100%-spot ASG is never passive.
Read more
Report
My "4-bit" quant was 6.2 bits per weight
Nemotron's hidden dimension is 2688, which is not divisible by 256. That one fact quietly inflated a 4-bit quant by a third and cost a weekend.
Read more
Report