Distributed training takes center stage
🔴LIVE: WLFI In Trouble? Winter Is Over. Brian Armstrong Pushing DC HARD, FTDA then Lighter & Akash (The Rollup)
“On the bright side, distributed training actually is taking the center stage. We saw a new model by a subnet in Bittensor called Covenant, which did a 72 billion parameter model, which is not state-of-the-art but definitely better than what Llama 70 billion parameter model was… it’s not hard to see where you have a distributed grid and that grid contributing to a larger distributed network for AI as a possible solution.” — 02:14:33
Context: Framing decentralized training plus decentralized energy (a distributed grid) as the answer to the data-center energy wall.