# Distributed training just broke the co-location requirement

- **Speaker:** Greg Osuri
- **Attribution:** attributed
- **Said:** 2025-05-13
- **Theme:** Decentralized AI — https://www.gregosuri.com/predictions/decentralized-ai/
- **Source:** Ep. 654 The Intersection of AI and Cryptocurrency with Akash Network (CRYPTO 101)
- **Watch at:** 00:26:13 — https://podcasts.apple.com/us/podcast/ep-654-the-intersection-of-ai-and/id1262351840?i=1000708305733
- **Canonical:** https://www.gregosuri.com/predictions/decentralized-ai/2025-05-13-distributed-training-just-broke-the-co-location-re-j2w6/
- **Cite as:** Greg Osuri, "Distributed training just broke the co-location requirement," Ep. 654 The Intersection of AI and Cryptocurrency with Akash Network (CRYPTO 101), 13 May 2025, 00:26:13. https://podcasts.apple.com/us/podcast/ep-654-the-intersection-of-ai-and/id1262351840?i=1000708305733

## Quote — verbatim

> There's enormous amount of research... Google really released a paper called \[DiLoCo\] and a bunch of companies actually implemented this paper and actually proved we can train a 32 billion parameter model fully distributed... you can have a cluster in New York... Singapore... San Francisco and actually contribute to a training run... That was not possible a few years ago.

## Context — site annotation, not spoken

He dates the breakthrough to "about six months ago" (00:25:21) and says if Nous Research's DisTrO approach proves out at 100B parameters "that will turn a lot of heads" (00:27:07).