Distributed training is in its GPT-3 era, limited only by compute
Scaling Web3 and AI through DePIN with Greg Osuri! (Secret Network)
“We’re still in the GPT-3 era of decentralized distributed training, but it’s progressing very fast, and only limiting factor is the amount of compute you can throw at it… how do we get the compute is through incentivization. So if you can create a viable incentive model, that’s where crypto plays a big major role — if you contribute compute, you somehow have rights in the future earnings of that model.” — 00:18:09
Context: Comparing Prime Intellect’s 16B/32B distributed models to GPT-4.5-era centralized frontier models; crypto incentives are the mechanism to close the gap.