Phones, MacBooks, and PS5s joining training runs
"AI Sovereignty: Building the Path to Individual AI with Greg Osuri" (The Outpost Podcast)
“Heterogeneous GPUs over the internet means my phone or my MacBook with the M4 can be part of a training [run]. Even if I push it, even my PS5, I sit in idle, which I play once a year… I can even hack that to make it compatible with [DisTrO]. In fact, US Air Force hacked PS2s back in the day to create a supercomputer.” — 00:33:29
Context: On Nous Research’s announced heterogeneous 15B training run; he adds “imagine how many PS5s are out there that are not being used” and “we got about five years, the space, to really knock it before they build the next nuclear data centers” ([00:34:11]).