Local AI

“I’m personally connecting super mini to a drone I have, because the drone is limited in its intelligence… I have a Helium device here I am connecting to super mini, so I can actually enable these dumb IoT devices to have some intelligence at the edge. I’m running lots of models.” — 00:55:15

“Machine learning is one of those use cases where you do not have the data gravity issues like our latency requirements that web applications normally require. So it’s an ideal system to be deployed in the home.” — 00:26:54

“Something like a box like a Supermini, which is in the home, which is… a low trust environment it lives in — the use cases for this is more machine learning.” — 00:15:39

“In [a home] where you don’t have strong internet connection, what can you host? You can host machine learning — you don’t need strong internet connection, but you need a lot of compute and GPU… the idea is to have that machine learning — you put it in your computer to become a supercomputer, I mean put GPUs in, you know, to serve machine learning.” — 00:47:41

“AI is very different from mobile… AI is intelligence, mobile is convenience. If you think what mobile did to the internet… intelligence is going to quadruple if not more, because so far intelligence has been reserved for human beings alone, but now we have a machine that’s augmenting human intelligence.” — 00:14:24

“I want to replace my Siri with essentially an AI that’s sitting in my house and my house only… I want to live in a world where I want guarantees that anything I say and I do lives in my house.” — 00:49:02

“I think both can be done in the house without leaving.” — 00:47:34

“I want to live in a future where my data, all my private data, gets fine-tuned or ingested by an AI that I know is not going to expose or share my data with anyone that I don’t want — especially financial, health data.” — 00:53:22

“Really, I think sovereign AI is going to be key part of AI evolution. We haven’t gotten there yet because [we’re] slowly starting to realize, hey, ChatGPT knows a lot about you and you’re giving it all your data.” — 00:43:57

“I do not want my financial data to leave my house, or my family’s financial data, or my family’s health data… Is there a world where I can have sovereign AI without compromising my privacy and still get the benefits of AI? That’s the world I want to build towards.” — 01:37:53

“Guess what — AI can do recollection way better than a human being. So using AI we can empower doctors: if a doctor is able to see 10 patients a day, they can now see 100 patients a day… The last thing I want is you go to a doctor and that doctor is unable to see you because he or she doesn’t have access to a GPU. That is not okay in my book.” — 00:27:07

“I think the first application we’re going to see that is going to take dominance is healthcare… And we have 7 billion people, right? 8 billion people now. Imagine a GPT for each person, right? That can actually diagnose your lab results… And the amount of GPUs you need for that, we don’t have enough H100s in the world.” — 01:01:00

“About three to four years ago, we launched something called Super Mini… It’s a device, a GPU for the home… AI is going to be one of the most important, like, applications for Akash and for the globe. And it’s very, very important that AI remains sovereign, right, instead of being controlled by a few companies. And ultimately, in order to ensure sovereignty, you need to ensure the fuel that runs AI and that happen to be GPUs.” — 00:13:08

“As AI gets more prominent in our daily lives, it’s no question that AI is going to get personal… to have a sustainable AI you will have to host your own AI. I think the future is going to be: people that use AI own the AI, and the way you own the AI is to own the computer that runs the AI — just like electricity, if you have a sustainable way of producing electricity instead of depending on the grid, you have a lot more control.” — 00:07:19

“Federated learning is a mechanism where models can be trained locally without giving access to the data and still be federated at a higher level. Now user owns the data — you’re not giving up the data… Just because a model is running on iPhone, it’s not running on the cloud — it’s still running locally, you still have access and full control of it.” — 00:04:24

“There are autonomous regions in Africa that don’t have physical connectivity, but they’re using Starlink to create remote data centers and using solar-powered sites to have remote data centers that are doing AI… a lot of really critical services now being rendered in local autonomous regions, and we’re looking to connect a lot of those data centers to Akash Network.” — 00:34:40

“The whole trend of this large big-parameter models being good is actually wrong. If you remember, ChatGPT forgot how to code — expert models are way better at coding than ChatGPT. The larger the model gets, the dumber it gets. It may have more knowledge, but it cannot give you accurate information.” — 00:16:42

“I was using Anthropic earlier and Anthropic shut me out because I hit their rate limits. It’s crazy how restrictive hosted AI is — it’s one of the reasons why you want to host your own AI on Akash. Hopefully we’ll get that out very very soon.” — 00:07:35

“Yes, we do have a lot of compute at home that’s very usable, but we haven’t quite cracked the code in distributed training and inference that can leverage various sets of devices.” — 00:07:08

“I’m anxious about the Apple’s AI production and also excited — anxious because a billion phones coming online with AI. I don’t think we have the infrastructure to scale… the globe doesn’t have infrastructure to scale. For context, Nvidia makes about 750,000 H100s a year. We need one H100 per person — that’s about 8 billion H100s to have every second connectivity… and this technology evolves so fast that H100 now will go out of fashion in two years.” — 00:15:13

“If I had a model in my home running on my local compute — unused compute clusters with my wife’s MacBook, my MacBook, my child’s MacBook — I can easily talk to the model and have the privacy guarantees. I think privacy is extremely underrated in the world of massive AI adoption.” — 00:22:22

“There’s nothing that beats a private AI, a private model that doesn’t leave my network… for a health diagnosis I didn’t want to talk to ChatGPT because I know ChatGPT is going to share my data.” — 00:21:40

“It’s my personal opinion: I think it’s going to be a lot of small models that [are] going to come together to essentially create a multi-model mechanism. Small models are inherently good at doing small tasks.” — 00:35:35

“Yesterday when Jensen [Huang]… introduced this like, just supercomputer for home. And if you saw that, right, so they’re literally like, there’s a big movement of [home AI], I would call it, where the AI is not literally on a cloud like today, but rather in home computers, right? And I believe in that vision because I’ve been coding a lot of AI agents these days.” — 00:48:31

“I don’t want that information to leave my home because I don’t trust anything that leaves my home network in terms of privacy, right? So there’s going to be a need for sovereign private AI at home. It’s inevitable because nobody, I’m not comfortable with this stuff leaving my home. I’m pretty sure most people are not comfortable. But if there’s an easy enough solution that you can just buy and plug and play at home, I think people will buy.” — 00:50:34

“I believe that as AI gets more advanced and AI gets more usable, people want privacy, people want to host AIs in the home. That’s how we tap into more distributed grid; that distributed grid will power a decentralized AI… energy bottleneck and the need for privacy are going to be key drivers for decentralization when it comes to compute.” — 00:21:21

“I want the whole house to be automated, every conversation to be recorded, but I would hate that conversation to be stored in a cloud, because I do not trust anything that leaves my home network — as no one should.” — 00:21:49

“Can I have a sovereign AI in the home? I think most people would want an AI in the home as long as it guarantees privacy. And I’m building that… We did a feasibility study: is there any way you can have sovereign AI in a semi-professional data center that takes 30 kilowatts of energy, in the home, that is cost efficient?” — 00:22:32

“I’m building a new house in Texas and I’m having one of the edge data centers in Texas… I want to have AI everywhere — agents everywhere listening to every conversation we have in the house, all sensors… the whole thing should be run by AI. That’s the dream. But there’s so much private information there, I would hate that information leave my home… So I can have local, sovereign AI that is fully private in my home and be able to make money leasing this out on Akash.” — 00:27:00

“Anything that leaves my home, doesn’t matter how secure the cloud or wherever I’m putting this data, I’m never comfortable. So if I want full AI — I’m talking about AI managing every aspect of my home… every aspect of the house is managed by an AI that’s local DeepSeek.” — 00:27:36

“A solar powered server in a rural school can train part of a global model. That’s the dream.” — 00:01:26

“What’s incredible is that these high-powered models are good enough these days to run on consumer hardware — or as I do often, they can run from the comfort of your MacBook Pro.” — 00:04:17

“All the information, everything is recorded and analyzed and fine-tuned locally, using a massive GPU cluster. Nothing leaves the boundary of my home, the network of my home. It’s a fully locked-down network, because a lot of this stuff is private.” — 01:48:09

“The chips to run real-time AI that can respond with low latency to agents, that is expensive… that’s why I think I’m so pro local AI of the future, and leasing the unused compute back to the grid, like an AI grid with Akash. I’m very excited about this new future of shared economy, because everybody wants GPUs if you want AI locally, in your homes.” — 01:51:41

“In terms of training — not there yet. We are able to train, using home computers, about 7 billion parameter models now — still very small — but the models are getting bigger and bigger and more efficient. In terms of training, I’m pretty sure in the next 12 months we’ll see a state-of-the-art model trained over the internet using heterogeneous methods.” — 00:11:36

“For the most part, the challenge in home is going to be bandwidth, to serve reliably. Starlink does about 400 to 500 [Mbps] capacity right now. There could come a day where it could do over a gigabit to 2 gigabit capacity — that’s, I think, when we will start seeing more effective inference at home.” — 00:14:29

“It’s going to be a robotic world, and it’s already starting to happen right now — I mean, we’re seeing with NEO home robots and whatnot. In the world of robotics, the robots themselves are not very powerful because they need a lot of energy to power bigger chips… So there could be a world where you have a home node cluster powering, inferencing the robot for additional more intelligent tasks, where the robot will do most autonomous tasks.” — 00:13:48

“That would be definitely my GPU machine here with the AI loaded on it. A fully trained AI on it, because if your physical needs or protection and your shelter and your food is taken care of, the next important thing is intellectual satisfaction… a fully loaded AI, like a DeepSeek or something that’s coming with me. I will generate infinite amounts of knowledge and read.” — 00:27:22

“The amount of information they have is so private and so intimate. I would never trust the information leaving my cloud, leaving my own network… You just never know what happens to your data the moment it leaves your building.” — 03:15:52

“People are going to start hosting the local models because, well, your privacy and you get significantly lower cost if you can produce your own energy, use that energy for your own models. I think AI at home will become a very common thing in a few years. The cloud is not going to go away, but people that want privacy are going to be using local models.” — 00:35:47

“The idea is to build a model that is local and customized to me. And it’s constantly learning.” — 00:10:04

“It’s only possible if the AI knows everything about me. And I’m no way in hell I’m going to give Anthropic Cloud or OpenAI or Groq or any of these remote models that level of access.” — 00:37:13

“I think local AI with agents especially with cloud bottom models you saw these days, privacy becomes the most important factor. Every time any data that leaves your network is potentially vulnerable to any snooping.” — 00:12:53

“I have solar in my home which basically powers my AI, and I really have zero dependency on the internet at this point. I really like sovereignty and really like freedom and control over my AI. And that’s the future I’m building towards.” — 00:12:09

“My home is entirely run by agents at this point… all my cameras are hooked up to my AI agent. I need local inference because I would never trust cloud on terms of privacy. General rule of thumb is if anything that leaves your network, doesn’t matter how secure it is, it is potentially vulnerable to attack vectors that you have no idea and no control over.” — 00:37:52

“For me… it’s not about crypto, it’s about privacy. I run GPUs at home; all my data that gets generated in my home — camera feeds to sensor feeds to audio feeds — go into the local GPUs. It doesn’t touch the cloud, because I like privacy… I don’t want my private intimate data to be on a cloud because anything that leaves your network is vulnerable, and you can’t really trust anyone with your data except yourself. That’s what I truly believe in.” — 00:32:33

“Nvidia is getting extremely excited about local AI. They’re actually building machines you can buy… they’re called DGX Box. They’re beautiful machines — you can put them in your house and you can cluster them.” — 00:17:57

“So, if you ask me the future of inference is going to be home? It’s going to be local… and it’s dual purpose. You can actually use that to heat your home and you’re using excess capacity anyway.” — 00:37:50