The shift from H100s to 4090s: "now we can incentivize home computers... eventually mobile phones"
Greg Osuri, Marko Stokic, Michael Heinrich & Luki Song on Can User-Owned AI Compete with Big Tech? (Nebular)
“For the first time we’ve seen a shift in what kind of compute we’ll need — H100s were all the glory a few months ago, but now it’s 4090s. 4090s, turns out, are actually pretty good for inference workloads, and you see gaming machines having quite a lot of 4090s. So the shift from data-center-grade compute to a consumer-grade computer — that’s happening, and that’s incredible news because now we can incentivize home computers… So there’s a big shift happening from data centers to home-grade computers, eventually to mobile phones.” — 00:10:09
Context: On incentive design; he also cites the Nvidia-to-AMD shift (Tinygrad’s first box on Akash with 120GB-memory AMD chips “as a home setup”) and Google DeepMind’s DiLoCo paper implemented by Prime Intellect enabling heterogeneous distributed training. Quote spans to [00:11:36].