Decentralized training on heterogeneous compute is promising
Erik Voorhees and Greg Osuri: "The Power of Permissionless" - Akash Accelerate '24 (Akash Network)
“I think we’re seeing a lot of progress in the decentralized training space. I’m very very positive, especially a lot of the research papers that are coming out, like DiLoCo and whatnot, that propose training on heterogeneous, heavily distributed compute — so seems to be promising.” — 00:18:47
Context: Greg responding to Voorhees’s claim that efficient decentralized training is the most important unsolved problem in AI. Captions render the paper name as “dialog Co” (DiLoCo).