GPU Economics

“If an A100 cost about $11,000, if you’re making a thousand bucks a month, which is about $1.40 an hour… a year amortization of the chip is very profitable… compared to real estate, compared to any other asset class. If you’re getting back your money 100% within a year, I think that’s very profitable.” — Greg Osuri, 00:21:36

“When it comes to growth, Akash is growing at 150% month over month, so obviously I’m telling you there is an incredible need for GPUs and compute out there.” — Greg Osuri, 00:14:29

“If you don’t have $100 million in your bank account and not funded by Amazon, Google or Microsoft, one of the big players, it’s impossible to get high-density compute. That’s just the reality.” — Greg Osuri, 00:16:39

“Similarly, Akash will have pools. A pool one could be A100 with [NVMe] storage, with PCI card, with X amount of memory, X amount of bandwidth, and certain characteristics of a provider that are preset, and a set of tokens can distribute to this pool… in the early days when there are a few participants, they’re going to get significantly outsized rewards.” — Greg Osuri, 00:53:39

“Some of the predictions, Gartner, I believe, made a prediction that about 5% of global GDP will be spent on AI hardware. And what is that hardware? GPUs, right? And GPUs are very hard to build… 80% of GPUs used for AI is built by NVIDIA.” — Greg Osuri, 00:52:43

“About 150 GPUs… but these are high-end GPUs, A100s, high density GPUs, and we have about 600 GPUs coming online every month starting this month… most of them are H100s, A100s and A6000s, and Akash will [be] the only place to get H100s for $2 an hour — which anyone that [has used] H100s knows what that number means — and A100 for 79 cents an hour… H100s starting next week.” — Greg Osuri, 00:14:09

“GPUs are very hard to get on demand. Why? Because big cloud providers control GPU leasing. For example, on Amazon the only way to get advanced GPUs is to know somebody that knows somebody and spend millions of dollars to even get access to the chips. So if you’re [an] AI startup, there’s no way in hell you’ll be able to get started without having millions of dollars of funding.” — Greg Osuri, 00:01:33

“There are free GPUs everywhere… a lot of crypto miners that bought a ton of GPUs during the last cycle that are sitting idle… Foundry Digital, which is owned by DCG, is sitting on massive amounts of A100s and 6000s… There are also companies that are raising money with GPUs as collateral, so GPUs are the new spice.” — Greg Osuri, 00:04:25

“GPUs are very hard to get, and according to Elon Musk, harder than drugs. But there are GPUs available in the wild, and Akash is effectively bringing a lot of that GPU supply to the market for people that need them. Right now Akash, I bet, is the only platform you can get A100 and H100s…” — Greg Osuri, 00:02:12

“We have 3x the amount of daily average workloads from last quarter — right now about 2,000 active leases, that means 2,000 different applications running at the same time… A lot of experimentation is going to happen over the next — maybe we got till 2026 before the supply chain goes away. So we got two years; whoever has the biggest war chest is going to win, that’s the reality.” — Greg Osuri, 00:50:05

“Right now Akash is the only place to get on-demand H100s or A100s, which is, honestly, the spice of life. If you think about, GPUs are going to be probably the most important chipsets for humanity for the… foreseeable future.” — Greg Osuri, 00:08:45

“There is no machine learning without GPUs. GPUs is the new spice, so whoever has the spice can do AI — and we are the spice dealers.” — Greg Osuri, 00:03:03

“Spice was the most valuable commodity before oil — think about it, the whole East India Company, the largest corporation in the world, went after spice in the 1600s… spice was the most valuable commodity before oil, and now it’s back.” — Greg Osuri, 00:09:31

“They will not give it to you if you’re hyperscaler. They’ll only give it to you if you purchase smaller units, because they want more customers, they want more diversified customer base… they’re deprioritizing clouds.” — Greg Osuri, 00:06:29

“Akash should always be the first place to get the most advanced compute, so we’re going to get the Blackwell processors very soon, or GH200s very soon… we’re also looking at ASICs. I want Akash to be the first place to get advanced compute before you can go to the cloud.” — Greg Osuri, 00:32:07

“In 2018 in our white paper we talked about primary use case being machine learning. We anticipated the need for GPUs… the cost of GPUs and cost of compute was only going to go up for machine learning because the data sets are just getting larger and larger… we specified the primary use case for Akash is machine learning — it just took seven years to realize that.” — Greg Osuri, 00:09:03

“Akash has more GPUs available than Amazon right now, availability-wise. I mean they probably have more GPUs on count, but not available.” — Greg Osuri, 00:32:51

“We anticipated the need for GPUs because the data was incredible… the cost of GPUs and cost of compute was only going to go up for machine learning because the data sets are just getting larger and larger. We specified the primary use case for Akash is machine learning — it just took seven years to realize that.” — Greg Osuri, 00:07:15

“We know scaling supply is much easier with incentives than scaling demand… it’s like literally going to a store and seeing racks empty or racks full. I’d rather have a store that has racks full and less customers.” — Greg Osuri, 04:06:58

“The price points are so attractive — you can get sub-$2 H100s on demand, which is unheard of… Akash still remains to be the only general-purpose platform that can offer price points at these levels.” — Greg Osuri, 00:16:32

“People come to Akash to get access to a resource that they can’t get access to. I call it the spice — spice must flow. So fortunately we’re in a position where we don’t need to incentivize further to get adoption.” — Greg Osuri, 04:00:29

“The price points are so attractive — you can get sub-$2 H100s on demand, which is unheard of… an H100 on Akash costs about $1.46; the same thing would cost about $12 on Amazon, if you can get it… In order to get H100s you need to get into these long contracts. No one’s going to give a researcher [who] wants two H100s on demand.” — Greg Osuri, 00:16:30

“In 2018, in our white paper, we talked about the primary use case being machine learning… we anticipated the need for GPUs because the data was incredible, it was very transparent — the cost of GPUs and cost of compute was only going to go up for machine learning, because the data sets are just getting larger and larger… we specified the primary use case for Akash as machine learning. It just took seven years to realize that.” — Greg Osuri, 00:06:31

“Akash will start offering H100s at around $2, which is the cheapest ever you can find. For comparison, Amazon offers H100, if you’re lucky, for $12 an hour. Akash is doing the same exact chip for $2 an hour.” — Greg Osuri, 00:25:22

“There’s clear demand for AI, there’s clear demand for chips, and demand is actually gaining. I think we’ve gotten predicted, for what it’s worth, that 5% of global GDP will [be] spent to compute for AI… by 2030. That’s pretty aggressive.” — Greg Osuri, 00:35:35

“From a product standpoint, AI is not even close to being embedded in our everyday products because it’s not fast enough, because there’s not enough chips… imagine ChatGPT part of Siri… imagine the amount of compute you need when you have to power a Siri [on] billions of phones.” — Greg Osuri, 00:42:56

“If you have to put a prediction, I think supply chain will ease out in next three to five years. I don’t think it’s going to be forever compute shortage.” — Greg Osuri, 00:37:05

“There’s no real challenge to the most valuable company in the world… I don’t know if there’s going to be another contender that’s going to immediately threaten Nvidia’s position in the next five years… Nvidia is going to have monopoly, as sad as it looks.” — Greg Osuri, 00:40:41

“We’re still in very early stages, so I would say the next two months revenue is going to be at least triple or quadruple, based on what we’re seeing in terms of supply. We’re supply-constrained now, more than demand-constrained.” — Greg Osuri, 00:23:57

“There’s obvious crazy demand for these chips, but there [is] also just supply, and turns out the only problem is really coordination — and blockchains are phenomenal coordination networks. In doing so we created the world’s first open source cloud.” — Greg Osuri, 00:02:10

“Akash is the only way you can get on-demand [H]100. There’s nowhere right now that you can get an H100 on demand. You can get H100 if you go long contracts… and it’s very, very expensive. But if you want to get H100 for $2 an hour, or $1.59, you can get [on] Akash an hour instantly.” — Greg Osuri, 00:24:28

“We actually talked about machine learning in the White Paper. There was a whole page on it. We talked about how machine learning is only getting expensive… it was very obvious to us, like, that GPUs being the core for AI will be, and the demand for AI as it’s going up, GPUs will be in short supply.” — Greg Osuri, 00:16:48

“AI compute is predicted to have 5% of global GDP by 2030, I believe — and I believe that’s the case. You’re going to see supply chain ease out… I think next 5 years we’re going to see the supply chain catch up a bit.” — Greg Osuri, 00:29:28

“Nvidia is a king obviously and their profit margins are incredibly high — the actual cost of the chip is not as much what Nvidia sells them for… well, it’s a monopoly. Break the monopoly — you’ve got to get better or cheaper chips from competitors like AMD… George Hotz is rewriting the drivers for AMD to make it usable… the first ever Tiny Box is going to be on Akash. So things like that are coming, and that’s going to reduce the cost significantly.” — Greg Osuri, 00:14:32

“To specifically address a question about the $600 billion hole: if you think about it, Amazon makes about 43% margin on compute on average. So if you remove Amazon, you remove 43% at least.” — Greg Osuri, 00:12:17

“GPUs are nearly impossible to lease on demand. So today if you were to go to hyperscalers… basically they ran out of all the A100 and H100s that you can give, and if they do, they gatekeep them. So they don’t give it to you just because you are paying on demand — you got to pay one year up front and maybe if you’re lucky you’ll get some.” — Greg Osuri, 00:05:03

“It was supply before, but supply is actually getting better… supply is getting better for training — H100s — but not so for inference, like A100s or 4090s; it’s getting tighter… but incentivization can go far — if you give money to people, they will supply. That’s not a big of a problem. But it’s really on the demand side… delivering demand in a non-custodial permissionless manner is very challenging… the majority of the machine learning researchers have no idea what crypto is and don’t really care about crypto — they don’t even own Bitcoin.” — Greg Osuri, 00:14:25

“The way trial wallets work is very simple: you get a trial wallet for every user that comes — that alleviates the need to install a wallet or get some AKT to deploy. So we think that’s going to significantly increase activation rate… In the next couple of months we’re going to see a significant increase in demand for Akash because we can solve this problem. It’s easy to convince anybody to come and look at you when you say you have A100 for 80 cents… but it’s really hard to get a user to use you.” — Greg Osuri, 00:18:02

“We wrote a paper in 2018 describing what Akash Network would look like, and we actually talked about GPUs in 2018, but nobody really paid attention to the GPU shortage then — but it’s very obvious now, seeing the success of ChatGPT.” — Greg Osuri, 00:07:18

“There’s enormous amounts of chips actually available right now, essentially for free, in crypto miners… when companies upgrade their chipsets… the older models are available — there’s a whole supply chain that buys up these older models and they just sit idle… sitting in warehouses and not being used.” — Greg Osuri, 00:05:51

“GPUs today are nearly impossible to lease on demand — on demand in the sense, use for however long you need… if you try and go to Amazon today and try to rent a chip, it’s impossible to get on demand, you have to get into a long contract before they can even talk to you.” — Greg Osuri, 00:02:53

“Who powers these AI machines? You need something called GPUs. How do you get them? You have to go to this cloud provider. And it’s nearly impossible for a company to get these GPUs because they’re scarce. These cloud providers sell the GPUs to the highest bidder. So they literally dictate today how AI should be trained and who should use AI by choosing who they want to give their GPUs.” — Greg Osuri, 00:04:35

“There are about 8.1 million data centers in the world… in these data centers, they only use 15% of the time. That means 85% of the time they’re not being used… [Intuit] during tax season, their compute is used 95% of their capacity or higher… Off tax season, they only use 5% or less.” — Greg Osuri, 00:09:13

“Akash again is the only place to get on-demand H100s… H100 is available for [$1.50], which is unheard of in traditional cloud. Akash is the only place you can get that.” — Greg Osuri, 00:07:17

“Apples to apples, Akash costs about 80% cheaper than Amazon… With GPUs, in fact, H100 on Akash cost[s] [$1.49] whereas the same resource would cost I think $5 on Amazon.” — Greg Osuri, 00:15:29

“Apples to apples, Akash costs about 80% cheaper than Amazon… and the way you’re able to get this is because Akash taps into underutilized resources, whereas [on] Amazon you’re actually building new resources and new data centers. With GPUs, in fact, H100 on Akash cost[s] [$1.49] whereas the same resource would cost I think $5 on Amazon.” — Greg Osuri, 00:15:29

“Most compute on Akash is underutilized compute, not cloud compute… same mechanics, same economics as Airbnb versus Hilton. Hilton, they have to sell you at margin, where Airbnb… they have a room they’re not using, so they’ll take whatever they could get.” — Greg Osuri, 00:35:17

“You just get cheap H100s… which are the most advanced chips for AI for buck forty-nine, which is the lowest in the industry — there’s nothing cheaper than that.” — Greg Osuri, 00:28:50

“Amazon Web Services… alone [is] a trillion dollar business for Amazon, and profit margins are over 60%, and that’s not okay. You can’t have [an] enormously profitable layer and you call it a public utility. For example, the cost if you look at decentralized systems like Akash is 80% cheaper than what you would otherwise pay on Amazon, and that’s because there’s compute available everywhere.” — Greg Osuri, 00:09:32

“We are already seeing a decline in Nvidia’s demand, especially as we start cracking being able to deploy or infer on non-Nvidia chips… With MI300s from AMD you’re able to run Llama 3.1 405B pretty well… there’s a company called Exo Labs which seems very promising — they do cluster inference on even Macs. So as the models are getting open-sourced and as we see a lot of optimizations happening for non-Nvidia chips, I think the dominance of Nvidia is going to gradually go down.” — Greg Osuri, 00:21:49

“As AI gets importance, I don’t think Nvidia is going to stay dominant player in terms of hardware. I think it’s going to get significantly diversified — and has to, because you have one company… I don’t want to live in a digital feudalism where you have one digital lord giving us what we need. Today getting chips is so hard — even the richest people in the world; Elon Musk is famous for saying getting GPUs is harder than getting drugs, even though that’s a low bar in San Francisco.” — Greg Osuri, 00:23:16

“GPUs is the bread and butter for Akash now, considering the demand for GPUs is just increasing without the supply chain improving, and Akash is now the only place to get on-demand high-density GPUs anywhere on the cloud — and that doesn’t seem to be changing anytime soon.” — Greg Osuri, 00:01:33

“If you want to incentivize a certain chipset — A100 for example — the way it works in Akash is every time you reach 50% utilization for a certain chipset, the incentives to double the capacity are released… so that way you have a significantly better and more disciplined distribution curve instead of just a broad ‘incentivize everything’ and get thousands of GPUs that no one really uses.” — Greg Osuri, 00:08:06

“The demand is so high and increasing so fast, the supply is not catching up to any degree of satisfaction. Right now, NVIDIA is backlogged for about 18 months to two years, I believe, for H100s, which is the most widely distributed GPUs from NVIDIA, and even longer for the newer models, the B200s.” — Greg Osuri, 00:03:53

“For something like Llama 3.1 [40]5B… it takes about 24 H100s to do inference… that can serve about 20 to 30 concurrent connections at high precision… to serve Apple’s users at decent performance, you need a billion H100s. And there are about, I believe, about 700,000 H100s right now.” — Greg Osuri, 00:05:15

“Assume what the demand is going to be when we onboard 8 billion users, which is going to happen… imagine one H100 per user. And that gets outdated within a year. So then you have newer, bigger models that are more intelligent, that need bigger chips… I haven’t even touched the surface of the kind of demand that I see for H100.” — Greg Osuri, 00:05:58

“Last year when we did the interview I think the daily spend was $500; today now it’s $5,000 a day — it literally 10x’d over a year… and that’s the kind of growth we want to see at an early stage.” — Greg Osuri, 00:11:36

“There are lots of underutilized GPUs out there, and Akash has a capability to federate them and offer them at a significantly lower price than what you would otherwise pay for a new GPU. Akash is a used GPU market… it’s eBay for GPUs without a physical delivery.” — Greg Osuri, 00:08:08

“Nvidia now uses Akash — I don’t think Nvidia uses any other crypto protocol.” — Greg Osuri, 00:06:42

“If one wants to amortize their chip within two years, I think the price is around $2 an hour, roughly, including the energy cost… so if anything less than $2 an hour, you’re losing money. We have chips for [a] dollar an hour too… that tells you the market is — there’s oversupply of chips now.” — Greg Osuri, 00:21:57

“As we were deploying a lot of these solutions to different data centers across the world, we realized most compute is underutilized — somewhere north of 85% to 90% of the time, data centers have underutilized compute, unused compute, because you need peak capacity to add peak workload.” — Greg Osuri, 00:03:02

“We are growing 1700% year-year growth. I think this month alone we grew by 100% from last month in terms of our earnings… If you do that, one order of magnitude growth, for about five years, we’ll reach 1% market share.” — Greg Osuri, 01:07:35

“After several experiments on Ethereum in 2016, we were convinced that this is possible, so we wrote a paper and published that in 2017 called a decentralized supercloud, and we proposed that there is enormous amounts of underutilized capacity around the world — on an average about 75% of data center capacity that’s active is underutilized, and that’s a conservative estimate; in the reality [it’s] somewhere like 90%.” — Greg Osuri, 00:02:15

“As of this morning our utilization for GPUs is about 92% — it’s highest of any cloud, decentralized or not.” — Greg Osuri, 00:19:26

“There is a whole new market for commoditizing, especially H100s. You should be able to trade compute cycles, you should be able to create futures… Certainly I would say like a year, we’re going to have compute market futures.” — Greg Osuri, 01:05:24

“Nvidia’s dominance is dominant for now, but that’s not long term. Obviously you’re going to see quite a lot of innovation that’s happening in the chip space and the energy and all that — it’s going to challenge Nvidia domination in a very short term.” — Greg Osuri, 00:14:28

“You can get the same amount of compute that you get from AWS for 80% cheaper, and the reason for that is decentralized systems can reach places where centralized systems cannot… closed-off networks, corporate networks, where there is heavily underutilized compute… about 80% or 85% of the time most data centers are not used, because you need to have peak capacity to address peak workloads.” — Greg Osuri, 00:11:41

“Andreessen did a study, a paper called the Trillion Dollar Paradox, where they researched S-1 filings of cloud companies, and turns out half of every dollar you spend on one of these cloud services — be it Netflix or Asana — goes to one of the four cloud providers. What this effectively is, is a cloud tax that we all pay but don’t really realize we’re paying, to this group of unelected individuals or entities.” — Greg Osuri, 00:05:55