The larger the model gets, the dumber it gets
Greg Osuri, Marko Stokic, Michael Heinrich & Luki Song on Can User-Owned AI Compete with Big Tech? (Nebular)
“The whole trend of this large big-parameter models being good is actually wrong. If you remember, ChatGPT forgot how to code — expert models are way better at coding than ChatGPT. The larger the model gets, the dumber it gets. It may have more knowledge, but it cannot give you accurate information.” — 00:16:42
Context: Pushing back on Marko Stokic’s claim that decentralized AI is losing the model race; Greg argues small expert models beat monolithic LLMs.