Distributed training hits GPT-3 level by end of year
Solving AI's Energy Crisis with Decentralized Compute, w/ Akash CEO Greg Osuri (The People's AI: The Decentralized AI Podcast)
“The biggest model is 72 billion parameter model. For comparison, GPT-3 was 150 billion parameter model. So we’re very, very close to GPT-3… So if you were to ask me where distributed training is, it’s about pre-ChatGPT-3. And by end of the year, it will be ChatGPT-3 level.” — 00:19:37
Context: Dated benchmark prediction (end of 2025). Cites the sponsor protocol (likely Gensyn; Whisper: “Jensen”) running ~12,000 models training concurrently, plus zero-knowledge gradient verification enabling permissionless nodes [00:18:11].