Single-data-center training caps out in two years; distributed training is the only way out
DeepSeek and the Dominance of Open Source AI | Akash & Prime Intellect: Mined with CoinFund Ep. 19 (CoinFund)
“You look at what technologies can enable model training advancement, and if you look around, only decentralization — distributed training — is the only way out. That’s why, in two years I think we’re going to cap out as to how much you can do in a single data center.” — 00:19:12
Context: After citing the Memphis 250MW data center, planned 1.2GW sites, ~96 US nuclear reactors already at full use, the last reactor taking 14 years/$34B, and the ~8-10K H100 per-data-center ceiling in the US.