GPU Economics

“There are about, as far as I estimate in our own interactions, about 80,000 GPUs that were purchased for about 19 billion dollars to mine Ethereum. 19 billion dollars worth of GPUs are sitting idle post-merge… and Akash with GPU capability will be targeting those GPUs.” — Greg Osuri, 00:20:10

“We’re looking at next three months… we’re expecting next three months, but then all our progress is going to be fully public.” — Greg Osuri, 00:23:02

“It takes about six weeks from Nvidia to get you A100 chips. You can’t even get them on Amazon anymore because Amazon exclusively reserves those chips for high-paying customers. So if you’re a startup company, you want to do AI, it’s impossible to get that on Amazon. So we’re starting to see the system break down… the only way to reverse that is to take the control back from these cloud companies.” — Greg Osuri, 00:19:28

“As we’re sort of like going into GPUs in the next six months for Akash, I think it’ll be amazing how we can further actually maybe even provide security to Bitcoin using some of these high-end chips we’ll be tapping into.” — Greg Osuri, 00:36:01

“GPUs are next. So if you have any GPU-style, proof-of-work-style workloads, I think that’s going to be supported as well very, very soon. So you’ll have a complete node hosting solution.” — Greg Osuri, 00:29:37

“Today we have something called Nvidia Tesla A100s… right now to get Nvidia A100 it takes about five weeks wait time — there is just not enough chips… I think Amazon has about 20,000 of them, Facebook has 10,000, Stability AI has four thousand… if you go to Amazon you can get A100s — unless you pay them a lot of money, they’re not going to give it to you.” — Greg Osuri, 00:33:56

“Very few companies are going to control chips, are going to control one of the most important innovations that we have, AI, and that’s not okay. And the demand is just increasing by 10 folds every six months. So to serve that demand you need to decentralize… and share these resources — there’s no reason Facebook can’t share their resources with Stability AI and vice versa when they’re not using that.” — Greg Osuri, 00:34:39

“It’s going to get even worse because complexity of AIs I believe is doubling every six months… all that means is more complex AI coming in and using a lot of these chips.” — Greg Osuri, 00:25:53

“We know at least we are working with the top, I would say, two of the five data centers that… have kind of GPUs from Ethereum, and we’re working with a few operating systems as well, and Akash at launch will have at least 20,000 GPUs.” — Greg Osuri, 00:41:35

“Even the CEO of TSMC in their latest quarterly results came out and said this is going to increase the chip cost by five times. So all our chips are going to increase in prices, especially GPUs.” — Greg Osuri, 00:27:19

“Oh, it’s going to get worse… there is… two main factors driving the prices [of] GPUs, especially high-end ones… it takes about six weeks to get A100 right now… and the cost about 20 grand per chip.” — Greg Osuri, 00:24:27

“So far we were able to account up to 900,000 GPUs that could potentially come onto Akash that were all left from the Ethereum merge… in this whole year, in 2023, all the supply will be online, and we’ll have Akash AI as a beautiful product to deliver this value to AI [devs], and AI is going to explode in terms of usage.” — Greg Osuri, 00:48:10

“The provider incentives will be used to incentivize high-end GPUs like A100s, H100s. We’re anticipating from our estimations that an A100 will cost you somewhere around 50 cents — compare that to a traditional cloud offering which is about four dollars an hour… If you ask any AI dev and offer them an A100 for 50 cents, they will kidnap you.” — Greg Osuri, 00:20:49

“GPUs are the fuel for AI… the need for GPUs is doubling every 3.5 months as these LLM models get more advanced… GPT-3.5 is 175 billion parameters and GPT-4 is going to be a trillion parameters — that means you need that much of compute.” — Greg Osuri, 00:21:35

“We’re seeing quite a lot of AI applications even though… GPUs [are] like a few months away — people are just so excited about it… we anticipate AI is going to be a lot more prominent [a] deployment for Akash just from [the] cost factor… [LLMs] are really good at running on high-end GPUs like Nvidia A100s… on Amazon they cost about four dollars per hour; on Akash [they’ll] actually cost more like a dollar an hour.” — Greg Osuri, 00:20:39

“One of the things that we will be launching the mainnet [with] is H100s… it’s impossible to get them from anyone, and we are able to secure them because people bought H100s and are holding onto them and they have nowhere to sell them except on Akash. So a decentralized system is actually helping them get liquidity on their dead asset… Akash will be the only place where machine learning devs get H100s.” — Greg Osuri, 00:38:01

“Turns out GPUs are the new oil… if you ask anybody doing any decent amount of training or inference or fine tuning, you’ll notice GPUs are very hard to get… Elon Musk purchased about 10,000 of them a few weeks ago and it’s going to be the thing, right? These, this is the new gold… So we think the future of the way you… the only way you’ll be able to access these GPUs” — Greg Osuri, 00:03:39

“With H100 we end up being the only cloud provider to have H100… it’s no longer just an alternative cloud that’s cheap, it’s now the cloud that can get [you] resources that you can[‘t] get on the traditional cloud, and that’s going to be the future.” — Greg Osuri, 01:13:46

“Companies today that are doing machine learning cannot get these GPUs, the high-end GPUs we call A100s and H100s, on the cloud unless you have a special deal with someone higher up in one of these cloud providers… they selectively choose which company should be successful. OpenAI got a big leg up because of this reason and that’s not fair.” — Greg Osuri, 00:13:17

“Soon I think a real vision is machine learning companies should own their own GPUs, because that’s ultimately where the sovereignty stands… when they’re not using it [they can provide it to Akash].” — Greg Osuri, 00:28:28

“Nvidia is a software company, not a hardware company — that’s my contrarian take. Nvidia’s innovation is really software, the CUDA SDK that they came out [with] 10 years ago. That’s why they’re really good with AI.” — Greg Osuri, 01:04:28

“Supply is not catching up to demand, not even close… the demand is exponentially improving, so there’s no indicator that indicates the cost and supply is [improving] over the next five years. And even if they do, Akash is going to be the secondary market, so it doesn’t matter.” — Greg Osuri, 00:35:29

“I predict GPUs are the new oil. Whoever has GPUs [wins] the AI race. Ultimately OpenAI is better than the rest of the LLMs, arguably, because it has access to GPUs… models are open source, the data for the most part is open.” — Greg Osuri, 00:31:58

“Ultimately AI is only as good as the supply of GPUs it has, right? GPUs [are] the blood for AI. Whoever controls GPUs control[s] the AI.” — Greg Osuri, 00:01:26

“The current centralized infrastructure is crumbling [under] server AI [demand]… on the other hand you have the Ethereum merge that basically unleashed 19 billion dollars worth of GPUs that are idle right now.” — Greg Osuri, 00:03:38

“With the AI wave, it’s going to be, the need for compute is going to go excessively. Imagine I can do 10 times — that means I need 10 times more compute as a human being.” — Greg Osuri, 00:38:42

“We’re coming from a position of strength — we’re only ones with A100, nobody else has… we’re not coming from a position of ‘hey, we are cheaper’… we have something no one else has, no one before it has.” — Greg Osuri, 00:25:03

“The market is going to go [9x] in the next 10 years… it’s estimated to, right under 40 billion dollars, it’s estimated to be 400 billion dollars… in nine years.” — Greg Osuri, 00:50:52

“Chips are the new oil, trust me — whoever has the chips has the power.” — Greg Osuri, 00:49:29

“It’s gonna cost five times more. The cost of GPUs are going to go high while the demand is going to go high, so it’s going to be crazy for the next few years with this geopolitical scenario.” — Greg Osuri, 00:48:45

“The chips are going to be scarce and the demand is exponentially gaining, and I think it’s prime for a secondary market to exist.” — Greg Osuri, 00:52:16

“The kind of demand for AI, considering the complexity of AI models, is doubling every six months. Along with that the demand for these chips is also proportional, and it’s easy to see in the next five to ten years where this market is going — I mean, it’ll be way over trillions of dollars.” — Greg Osuri, 00:13:43

“The simplest way to understand is it’s an open market for compute, so think of it like Airbnb for data centers… there is an enormous amount of underutilized compute capacity all around the world.” — Greg Osuri, 00:07:14

“We’re transitioning Akash from like a cheap cloud to the access cloud, just like how we transitioned with Airbnb from a $20 bed-and-breakfast option to now the only place to get a French village… it is about getting access to the resources that you are denied in other markets.” — Greg Osuri, 00:25:09

“You think about where AI is going: the spend in next five years is expected to be 1.5 trillion dollars — a trillion with a T… and Akash’s market cap is what, 200 million? So if you look at room for growth, there is quite a lot. And again, none of this is financial advice.” — Greg Osuri, 00:58:31

“A secondary market is very important now to solve the supply chain problem, and that’s what Akash is solving. Akash is the world’s first open secondary market for advanced, high-density GPUs like A100s and H100s. And that’s just a start — the user comes to the chips but stays for everything else.” — Greg Osuri, 00:35:16

“These chips are the oil — the GPUs are the oil for AI… for the next two to five years we’re not going to be able to solve supply chain problems… right now it takes about two years to get the advanced chips from Nvidia, we’re talking about H100s.” — Greg Osuri, 00:33:09

“Akash can do 15,000 models, it’s not only one model. So for a comparison, Akash is the most capable GPU cloud out there — there’s nothing that comes close.” — Greg Osuri, 01:54:27

“I don’t see any restrictions of AMD using Akash, but AMD really I think has at least two more years by the time they can make anything competitive to Nvidia at this point.” — Greg Osuri, 02:08:14

“Overclock Labs ordered the first set of boxes… so we’ll be able to put the best AMD chips, when they come out, on Akash Network way faster than in any other cloud.” — Greg Osuri, 02:08:56

“What Nvidia is choosing to do is distribute the chips, fragment the market, so that they have more control over the customer. So the chips are right now distributed to companies like CoreWeave, Lambdas… about 20 of these smaller cloud providers that have direct access to Nvidia.” — Greg Osuri, 01:48:37

“Sam Altman, the OpenAI CEO, said their biggest limitation to scale the company is access to GPUs… OpenAI is on track to make a billion dollars this year, so a billion-dollar company in revenue run rate, with I believe 30 billion dollars in cash, is unable to get GPUs. So you can clearly see there’s a demand for GPUs here.” — Greg Osuri, 01:58:04

“You think about where AI is going, the spend in the next five years is expected to be $1.5 trillion, a trillion with a T, right? That’s $1 trillion is $1,000 billion. And I thought the market cap was, what, $200 million? So if you look at room for growth, there is quite a lot.” — Greg Osuri, 00:58:50

“For the next two to five years, we’re not going to be able to solve the supply chain problems. And on the other hand, what’s happening is as companies get serious about AI, they tend to buy these chips as much as possible because they need these chips to run. So these chips are the oil, the GPUs are the oil for AI. And so right now it takes about two years to get the advanced chips from NVIDIA.” — Greg Osuri, 00:33:02

“You get an apartment for a year and you only go there maybe two months of the year… You can put them on Airbnb. Similarly, Akash is in the Airbnb for high-end [GPUs]. Now you get into these long contracts because you cannot get them on Amazon… and now what do you do with these unused chips? You put them on Akash.” — Greg Osuri, 00:14:51

“Ultimately, every AI needs chips, right? Ultimately, without chips, AI cannot evolve. Chips are the new oil, right? Whoever has the chips has essentially the control of the world, right? So access to these chips to be decentralized is extremely critical. And that’s what Akash is working on.” — Greg Osuri, 00:29:03

“Nvidia alone has 80% market share… Nvidia is fragmenting the market by choice, meaning they are giving priority, lower priority to the bigger clouds and higher priority in the smaller cloud [providers]… Core[W]eave has about 35,000 H100s, way more than Google cloud has. So the tides have turned when it comes to Nvidia H100s and A100s.” — Greg Osuri, 00:12:31

“I saw it [an] interesting stat somewhere that said one percent of global GDP will be used for machine learning very soon. I believe it.” — Greg Osuri, 00:20:13

“Right now it’s not the most optimal setup to leverage a globally distributed clusters of 4090s, but in two years I bet you is going to be the case — and that’s really where we’re heading.” — Greg Osuri, 00:41:51

“Now imagine you paid for something for the whole year which you may or may not know if you have use for it. What do you do? You put them in Akash as a secondary market… if you want on-demand chips, will you come [to] Akash? Because Akash is the only place for secondary markets — essentially an unused GPU market. That’s what the whole thesis is all about. So Akash is in a very unique position to take advantage of this supply crunch.” — Greg Osuri, 00:36:42

“The moment we launched GPUs three weeks ago, all the A100s are sold out.” — Greg Osuri, 00:40:22

“H100s… [have] the highest demand. In fact they’re so hard to get, it takes about two years if you order now. It’s harder than a Rolex.” — Greg Osuri, 00:29:28

“Turns out Nvidia deprioritized the big cloud when it comes to giving them chips, because Nvidia considers this big cloud companies as competition… they prioritize… smaller providers like CoreWeave and DataCrunch and Lambda Labs.” — Greg Osuri, 00:32:22

“I think that’s going to be a trend: I think 80% of our demand is going to be inference and 20% will be fine-tuning and training.” — Greg Osuri, 00:30:22

“Copilot is one of those products I use on almost a daily basis… I pay about $8 per month for GitHub and it costs about $20 per month per user to produce it. So on unit economics they’re taking a loss… How do you optimize it? You have to go for something that’s a lot less expensive, and the way you do that is by distributing your clusters and taking advantage of something like Akash.” — Greg Osuri, 00:15:19

“Every time you hit a prompt in ChatGPT, the time it takes to come back requires one H100 per each concurrent user for that time. Now do the math: 100 million users… that’s talking about 100 million H100s. That’s impossible — they don’t exist.” — Greg Osuri, 00:16:43

“Last check it was north of 90% for one-week time frames, and about 50% utilization cumulative for all resources… Filecoin has less than 1% utilization of their storage network. So now we have an interesting problem where traditionally crypto has been good at oversupply and less demand — here we have the opposite problem: high demand and less supply.” — Greg Osuri, 00:10:14

“Akash is a secondary market — it is like Airbnb… you’ll get an apartment for a year because there’s no other way to get them, and now what do you do the 11 months of the year when you’re not in the apartment? You lease it out.” — Greg Osuri, 00:38:17

“I don’t think we will ever have a way to measure heterogeneous computing… you’re never going to have a way to measure diamonds, for example… that’s why Akash doesn’t have a matching engine, it has a bidding engine — it really comes down to the user to select the provider because the variability is incredibly high.” — Greg Osuri, 00:11:09

“Nvidia consider these cloud [providers] to be competitors… Nvidia is very, very aware and are optimized to remove power from the big guys… they are distributing the chips to smaller providers — be CoreWeave, be Lambda… CoreWeave has about 35,000 H100s, way more than Google ever has.” — Greg Osuri, 00:36:51

“Introducing them to additional type of GPUs will open up doors to, call it, innovation beyond imagination — like what can you do when you get access to a million GPUs that have significantly lower cost than what you would normally pay to either buy them or lease them elsewhere?” — Greg Osuri, 00:25:41

“That’s why all the best applications are always launched on Apple… I think that’s going to be a pattern where the latest and the greatest is going to be launched on NVIDIA. But most adoption is going to happen across different GPUs. It has to, because it’s not sustainable the way it is right now.” — Greg Osuri, 00:59:28

“It’s very important that the supply, the GPUs, the oil for AI is diversified and doesn’t come from a single company. It’s very, very important. NVIDIA has 80% market share, which is crazy… It’s even scarier how the world is like literally dependent or so critical, reliant on a stable Taiwan, right? If there’s a war in Taiwan, it’s going to disrupt so much in terms of AI that is scary.” — Greg Osuri, 00:57:54

“Now Akash has probably more than almost all the DePIN networks in terms of revenue… doing that without any incentives, and our utilization on GPUs — for high-end GPUs like A100s — [is] about 95%, and overall GPU utilization is somewhere around 50%.” — Greg Osuri, 00:18:10

“GPUs launched 90 days ago, and they now account for about 36% of all the revenue that has been ever made in the last two and a half years.” — Greg Osuri, 00:29:43

“For H100s, which is very very high demand, Amazon charges about $12 an hour and Akash charges about $2 an hour — so that’s literally the cost difference.” — Greg Osuri, 00:42:39

“That’ll create incentive for providers to bid lower on the prices, so that means you’re going to find A100s significantly cheaper on Akash than any other network, decentralized or centralized.” — Greg Osuri, 00:19:38

“I think 50% is an ideal sweet spot in terms of utilization rate, wherein the suppliers have enough confidence that they will be able to clear the inventory… tenants have enough confidence that they’ll be able to find compute to scale. That’s the sweet spot — we want to keep the network at 50% and scale accordingly.” — Greg Osuri, 00:17:21

“Akash is four or five times larger in terms of usage when compared to the Render.” — Greg Osuri, 00:23:10

“We have to do everything we can, if you want to keep AI alive, to get more chips in people’s hands. Demand is only increasing… but the chip manufacturing and chip supply chain is nowhere close to catching up with the demand.” — Greg Osuri, 00:35:24

“A million dollar a day, there you go — that’s the big milestone when I will be satisfied that Akash has left the building. Right now we’re maybe 0.3% there. We want to get first 1% and then 10% and then hopefully 100% very soon.” — Greg Osuri, 00:59:20