# A 15B-parameter LLM was trained over the public internet, matching centralized accuracy

- **Speaker:** Greg Osuri
- **Attribution:** attributed
- **Said:** 2025-11-04
- **Theme:** Decentralized AI — https://www.gregosuri.com/predictions/decentralized-ai/
- **Source:** Sponsored Session: Powering PyTorch: Decentralized Training for an Energy-Hungry Future - Greg Osuri (PyTorch)
- **Watch at:** 00:13:16 — https://www.youtube.com/watch?v=T3nA1nAaLss&t=796s
- **Canonical:** https://www.gregosuri.com/predictions/decentralized-ai/2025-11-04-a-15b-parameter-llm-was-trained-over-the-public-in-5kwu/
- **Cite as:** Greg Osuri, "A 15B-parameter LLM was trained over the public internet, matching centralized accuracy," Sponsored Session: Powering PyTorch: Decentralized Training for an Energy-Hungry Future - Greg Osuri (PyTorch), 4 Nov 2025, 00:13:16. https://www.youtube.com/watch?v=T3nA1nAaLss&t=796s

## Quote — verbatim

> A concrete decentralized training milestone is Nous Research trained a 15 billion parameter LLM over the public internet on volunteer machines and donated instances in about 10 days and about 11,000 steps, matching centralized run accuracy using WAN-aware optimizers. This demonstrates that with the right comms and async strategy, frontier class models can train beyond a single hyperscaler.

## Context — site annotation, not spoken

Empirical case study; he also cites Gensyn's verification protocol (46% overhead, 93% accuracy under 50% churn) as the trust layer for untrusted nodes.