# Serving Apple's users would take a billion H100s — only \~700,000 exist

- **Speaker:** Greg Osuri
- **Attribution:** attributed
- **Said:** 2024-10-02
- **Theme:** GPU Economics — https://www.gregosuri.com/predictions/gpu-economics/
- **Source:** "Will demand for advanced AI chips (GPUs) be 1:1 for every person? We sit down with Greg Osuri to find out." (Block Fuel)
- **Watch at:** 00:05:15 — https://podcasts.apple.com/us/podcast/will-demand-for-advanced-ai-chips-gpus-be-1-1-for-every/id1705468368?i=1000671509211
- **Canonical:** https://www.gregosuri.com/predictions/gpu-economics/2024-10-02-serving-apple-s-users-would-take-a-billion-h100s-o-tra3/
- **Cite as:** Greg Osuri, "Serving Apple's users would take a billion H100s — only \~700,000 exist," "Will demand for advanced AI chips (GPUs) be 1:1 for every person? We sit down with Greg Osuri to find out." (Block Fuel), 2 Oct 2024, 00:05:15. https://podcasts.apple.com/us/podcast/will-demand-for-advanced-ai-chips-gpus-be-1-1-for-every/id1705468368?i=1000671509211

## Quote — verbatim

> For something like Llama 3.1 \[40\]5B... it takes about 24 H100s to do inference... that can serve about 20 to 30 concurrent connections at high precision... to serve Apple's users at decent performance, you need a billion H100s. And there are about, I believe, about 700,000 H100s right now.

## Context — site annotation, not spoken

Back-of-envelope inference math (spans into the 00:05:58 block) prompted by Apple Intelligence onboarding "a billion new devices that'll have AI inference."