# The internet itself can be the training cluster

- **Speaker:** Greg Osuri
- **Attribution:** attributed
- **Said:** 2025-11-04
- **Theme:** Decentralized AI — https://www.gregosuri.com/predictions/decentralized-ai/
- **Source:** Sponsored Session: Powering PyTorch: Decentralized Training for an Energy-Hungry Future - Greg Osuri (PyTorch)
- **Watch at:** 00:06:39 — https://www.youtube.com/watch?v=T3nA1nAaLss&t=399s
- **Canonical:** https://www.gregosuri.com/predictions/decentralized-ai/2025-11-04-the-internet-itself-can-be-the-training-cluster-i2uz/
- **Cite as:** Greg Osuri, "The internet itself can be the training cluster," Sponsored Session: Powering PyTorch: Decentralized Training for an Energy-Hungry Future - Greg Osuri (PyTorch), 4 Nov 2025, 00:06:39. https://www.youtube.com/watch?v=T3nA1nAaLss&t=399s

## Quote — verbatim

> An alternative is thousands of nodes — small data centers, campuses, labs, and individual providers cooperating over the internet to train one model. That is not science fiction. We've seen precedent at internet scale: Folding@home volunteers once peaked around 2.4 exaflops, outpacing the top supercomputer at that time. With the right algorithms and verification, we can turn the internet itself into a capable, resilient training fabric for PyTorch jobs.

## Context — site annotation, not spoken

After listing hyperscaler ceilings; he cites DiLoCo (~500x sync reduction) and Nous DisTrO (~857x bandwidth reduction) as the enabling breakthroughs.