A 70B model no longer needs Meta's data centers
From AWS to Akash: Greg Osuri on Building a Decentralized Compute Marketplace (Smart Economy Network)
“Frankly, a 70 billion parameter model can be now trained on a fully distributed network and that’s a big deal… that 70B was trained by Meta in their massive data centers. You don’t need a Meta to produce a 70 billion parameter model… I’m very excited for decentralized training to succeed where your home computers can be leveraged, because your energy cost is going to be a bigger variable than anything.” — 00:42:10
Context: Cites a Bittensor subnet (“Templar”) training a 72B model on a distributed network; he runs Llama 70B at home on consumer hardware.