No one has solved federated learning on heterogeneous GPUs
Akash: The Internet Of Compute W/ Founder Greg Osuri ($1 To $1 Million Podcast)
“Mark my words: no one solved federated learning on heterogeneous GPUs… While it’s possible to train over geographically distributed clusters, it’s not practical because it takes much longer.” — 00:15:57
Context: Rebutting io.net’s claim of having solved it; cites Google DeepMind’s DiLoCo paper and teams (Gensyn, Prime Intellect, Flock.io, Nous Research) still working on the problem.