Local Compute

“What would a world look like where you can have 1,000 regions offering general-purpose compute at 15 millisecond network latency for 98 percentile of active internet population?… And what would it look like where we can essentially repurpose compute from homes, right? So are we talking about powerful GPUs, be it in your gaming console, to your cars, to every connected device has some degree of AI capability and it will have AI capability in the future? And how does a connected globe that comprises of these low-power devices look?” — Greg Osuri, 00:18:05

“The only way to solve the problem for now is to look inward instead of looking outward. Instead of expanding our capabilities, let’s look at repurposing what we already purchased. Turns out there’s a lot of GPUs that are not used… we have PlayStations sitting in homes that are maybe used once or twice every year… 4090s that I have right here… at least half the time they’re not used while you sleep.” — Greg Osuri, 00:55:00

“Considering AI is going to be the critical aspect of our lives every day, you have to own the chips. And when you own the chips, you’re not using them most of the time, and if there’s a mechanism for you to share your underutilized time in exchange for some economic reward, you can essentially create a P2P network of folks that own chips and folks that want to use chips… if there’s a device in the home that’s serving some desire of yours and there’s underutilized compute, that should be part of the grid — and ultimately we as users of AI should own the AI. That’s the real vision behind decentralized compute.” — Greg Osuri, 00:08:04

“This device sits in your house and becomes part of the Akash network, by which the rewards that this particular node gains by hosting other workloads through Akash will subsidize the cost for the hardware… they can bring the cost of the hardware device to almost free — free in the sense the customer is paying for electricity.” — Greg Osuri, 00:09:42

“It’s not like one person running 10,000 GPUs — it’s 10,000 people each running one GPU, maybe at their house… something like residential internet… you could totally be able to do this at home.” — Robert Myers, 02:55:00

“There are what, 10 billion devices in the world? Akash can technically get to all 10 billion… there’s no reason why a self-driving car just in idle and charging should not be part of a global computation network.” — Greg Osuri, 01:01:40

“I don’t see a reason why we should stop at just data centers. We have to go into homes — we have enormous amounts of GPU supply sitting in homes, like PlayStations and Xboxes that we barely use, [which have] very powerful GPUs. Imagine what we can do when those GPUs come to market, and Akash — we want to do that in the next two years.” — Greg Osuri, 00:46:28

“If you had to point where maximum availability exists today in terms of resources, it is really in the home computers. Like I have a PlayStation at home that I use once a year… I have two consoles that are just in idle 360 days of the year, and that could be part of a computation network.” — Greg Osuri, 00:11:48

“Metaverse is going to be the future, and Metaverse needs really high-power compute… How do you source compute, high-density compute for a Metaverse that’s rendering in your vision pro? That’s only possible when you can reduce the latency. The way you can reduce the latency is through any central cloud.” — Greg Osuri, 00:21:15

“I think compute is going to be like basic necessities of human beings. If you are relying on compute for everything, for intelligence — which is the scary part — you’re going to need some allocation of compute and you’re going to need some currency to represent the allocation of compute. Akash is the currency of compute.” — Greg Osuri, 00:33:42

“We need to make it extremely easy for a home computer to be part of the network by a non-technical person… We have PlayStations to Xboxes with highly capable GPUs that are not utilized. I have a Mac Studio at home that I use for my desktop — highly capable GPUs that are not used.” — Greg Osuri, 00:44:24

“I agree with Altman that a universal basic compute will be a thing of the future, and those who have compute will have higher chances of surviving what’s coming.” — Greg Osuri, 00:34:25

“Folks in Africa don’t exactly have 20 bucks a month to subscribe to ChatGPT. So there is this whole new revolution happening with ultra-local compute, with expert models that are helping local population — and doing it powered by solar. That’s the kind of impact I like to see, and that’s not going to happen with centralized models, with extractive business models. It’s going to happen with local, it’s going to happen open source, it’s going to happen decentralized.” — Greg Osuri, 00:23:11

“There’s a lot more compute that’s spread across user devices than centralized in a data center. Now with techniques that we have coming up for distributed learning and whatnot, users can own their compute and aggregate them through decentralized technologies — and that’s happening very rapidly.” — Greg Osuri, 00:02:15

“For the first time we’ve seen a shift in what kind of compute we’ll need — H100s were all the glory a few months ago, but now it’s 4090s. 4090s, turns out, are actually pretty good for inference workloads, and you see gaming machines having quite a lot of 4090s. So the shift from data-center-grade compute to a consumer-grade computer — that’s happening, and that’s incredible news because now we can incentivize home computers… So there’s a big shift happening from data centers to home-grade computers, eventually to mobile phones.” — Greg Osuri, 00:10:09

“4090, hands down… you can play games on it and you can train AI — at least infer — it works really well… 4090 has been popping up a lot; now people want more 4090s than H100s, A100s, because a lot of the papers are mentioning 4090s. I think 4090s is going to be one of those under-the-radar [GPUs] that’s going to pop up and become the most important GPUs. I have a bunch of 4090s at home that are making about $250 — shilling the bags, though; I’m shilling my own bags.” — Greg Osuri, 00:22:25

“So I’m building a data center in my house in Texas, with a ‘come and take it’ flag in front of it, and that’s going to be 40 kilowatt, solar powered — it’s going to pay for the house. So I think we have an opportunity now: people are going to use my GPUs that are in my house because you have no other option, really.” — Greg Osuri, 00:10:24

“A machine sitting in a home may not be able to serve a production workload unless, you know, maybe intelligently designed with asynchronous model — but definitely for an inference type of scenario you need that low-latency access to the user, and that can only be delivered by a professionally managed data center.” — Greg Osuri, 00:05:20

“There’s a lot of underutilized compute out there that’s just not coordinated. We have underutilized compute data centers, we have compute in homes, game stations, your home appliances and whatnot… and decentralized systems like a blockchain can be used — and is being used right now, Akash is a primary example — to bring that to people.” — Greg Osuri, 00:00:46

“Distribution of supply — there is enormous amount of supply all the way in data centers, but also in homes, in devices and appliances. We haven’t touched even close to what we can touch when you have truly decentralized systems… The limit — it’s limitless, pretty much, because every device has a downtime. For example, the gaming console in my house is used maybe once a year, and it’s got really powerful chips.” — Greg Osuri, 00:05:42

“There’s a lot of underutilized compute out there that’s just not coordinated. We have underutilized compute data centers, we have compute in homes, game stations, your home appliances and whatnot… and decentralized systems like a blockchain can be used — and is being used right now, Akash is a primary example — to bring that to people.” — Greg Osuri, 00:00:46

“I’m building a little data center to be able to run my own AI… Elon was famously quoted about six months ago talking about how they can leverage GPUs in Teslas. There’s a lot of compute on the edge that’s just not being used.” — Greg Osuri, 00:39:43

“There’s a case to be made that compute will become a necessity in terms of ownership, so people will have compute — own compute… this pattern is not going to be permanent. I think we’re going to go back to the older pattern where we have more power at the edge.” — Greg Osuri, 00:39:01

“With Starlink getting smaller and smaller… we’re actually getting to a point where devices are going to be powered by satellite internet — you’re talking about mobile phones in next couple of years. So if you actually look at the trajectory where the technology is going, you’re going to have direct satellite internet on mobile phones, where on the other side is going to be an AI that’s streaming.” — Greg Osuri, 00:14:33

“It takes about 10 years [to build a nuclear reactor]… so we don’t have 10 years to wait to build the next generation AI. So I think what’s going to happen is there’s going to be a lot of the small data centers — they’re everywhere — that need to be connected, and if DiLoCo or distributed training [is] solved, we can actually use a highly distributed network to develop the next AI model.” — Greg Osuri, 00:42:49

“A lot of GPUs — talk about gaming consoles in our homes, right? My gaming console, I use them once a day. There’s enormous compute in our devices that is just distributed everywhere. That’s why we need a system that can take advantage of this distributed compute and bring them in a federated manner — that’s what Akash is.” — Greg Osuri, 00:07:19

“If you don’t need them to be local, you can now distribute your energy supply from anyone that can contribute the energy supply — it could be a home, it could be a data [center], it could be a garage for all I care. As long as it can participate in a training run, it can essentially contribute. That’s a big innovation.” — Greg Osuri, 00:25:20

“Ideally we want supply to come from our home. We’re not there yet — we’re getting there with decentralized training and whatnot. But from a fault tolerance standpoint, if you can have a home device, be it your PS5 or any PlayStations or Xboxes that just sit in idle — I use my Xbox maybe once a year during Christmas.” — Greg Osuri, 00:30:59