Training now works on distributed, heterogeneous networks
AI Data Centers Are Eating the Grid. Is There Another Way? (The People's AI: The Decentralized AI Podcast)
“Today, training workloads are evolving to a point that can work on distributed networks, heterogeneous networks, highly fault tolerant. We’re seeing companies like Pluralis, for example, training fairly large or usable models on fully decentralized networks.” — 00:39:02
Context: Adds that algorithms “can communicate less frequently in bigger batches,” and that Razer (the gaming hardware company) ran a successful April 1st image-generation campaign on Akash home nodes — proof home nodes work at scale for a publicly traded company [00:39:02].