# Aggregating GPU demand helps Modal manage capacity

Markdown copy of https://realalpha.app/s/aggregating-gpu-demand-helps-modal-manage-capacity/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Erik Bernhardsson](https://realalpha.app/builders/erik-bernhardsson/) · Erik Bernhardsson is co-founder and CEO of Modal. · Co-founded Modal, the high-performance cloud for AI teams; previously led Spotify's music recommendation and data infrastructure teams.
- Host: [Harry Stebbings](https://realalpha.app/people/harry-stebbings/) · Founder, 20VC · Built a venture firm on his podcast; its third fund closed at $400M.
- Show: 20VC
- Episode: [Modal CEO: Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute](https://realalpha.app/v/1dwyL6Iz_PU/), Nov 2, 2026
- Moment: 17:46 to 21:31 of the episode (4 min), playing on https://realalpha.app/s/aggregating-gpu-demand-helps-modal-manage-capacity/; the episode from there: https://realalpha.app/v/1dwyL6Iz_PU/?t=1066

## Quote

> “You can actually look at the demand in aggregate pretty well and predict based on monthly growth rates, which have been about **30%, 40% month-to-month**”
> — [Erik Bernhardsson](https://realalpha.app/builders/erik-bernhardsson/) at [20:26](https://realalpha.app/v/1dwyL6Iz_PU/?t=1226) · 20VC

## Takeaways

1. Modal works with 25 different neoclouds.
2. Pooling customer workloads makes overall demand more predictable.
3. Demand grew 30–40% month to month over the prior year.

## About this segment

Erik Bernhardsson describes sourcing GPUs from many providers and pooling customers' variable workloads. Modal forecasts demand using monthly growth rates, then buys capacity ahead of three-to-four-month lead times.

From “Modal CEO: Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute”, Nov 2, 2026. [Watch the full episode](https://realalpha.app/v/1dwyL6Iz_PU/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[17:58](https://realalpha.app/v/1dwyL6Iz_PU/?t=1078) What's it like doing that today? Is it as crazy as reports suggest? So it is pretty crazy. I spent some time in March at GTC NVIDIA's big conference in San Jose. I would describe that as like an open air drug market, but GPUs instead of drugs. You're running around, you know, talking to 500 different cloud vendors. You're like, do you have GPUs? Like, please get... And like, you know, in the end, you come out like you do a lot of deals and that's something we've had to develop the muscle.

[18:24](https://realalpha.app/v/1dwyL6Iz_PU/?t=1104) I think part of the value prop of modal is like our customers don't have to think about that. We do the dirty work of doing all the GPU deals under the hood. We work with 25 different NeoClouds or, you know, right now we control like, you know, 0.1% or whatever. I don't even know the number of like the world's compute. But like my like crazy, like, you know, how I think about the long term goal is like as we control more and more of the world's compute, you can aggregate that capacity and aggregate demand on the other side and manage that capacity and demand extremely efficiently. So that to me is like something where I actually think we can do, we can fill a big gap. What does that allow you to do being that aggregation layer?

[18:58](https://realalpha.app/v/1dwyL6Iz_PU/?t=1138) Well, so we built a lot of our infrastructure ourselves. Like we don't use Kubernetes and Docker and things like that. We built our own file system, we built our own container runtime, we built our own scheduler, we built our own basically how we manage the capacity. And that enables us to basically treat like all of the world's compute as one big pool of capacity. And so if you think about the fact that there's, we already have many thousand paying customers and over time it just grows and grows.

[19:20](https://realalpha.app/v/1dwyL6Iz_PU/?t=1160) Each of those customers have like very sort of variable requirements or like their workloads kind of vary over the day and they have maybe bursts. And each individually, they're like very sort of unpredictable. But if you pull all of those customers together, you end up with a very sort of smooth overall demand. And that is something we can manage extremely efficiently. And so one of the valid props of Modal, besides like we can scale up with you, is also you don't even have to do capacity planning.

[19:45](https://realalpha.app/v/1dwyL6Iz_PU/?t=1185) Just come to us, we'll give you 100 GPUs. I think it's so amazing going back to TypeSafe, for instance. Like they, we had talked to them, I think a long time ago before they launched, but they launched overnight on Modal and we weren't even aware of it. They just like scale up and we gave them a ton of GPUs. It just happened automatically. So that magic, I think, is something that people really value with modal.

[20:05](https://realalpha.app/v/1dwyL6Iz_PU/?t=1205) It's like we can give them a lot of GPUs on very short notice. And we can handle that sort of variable. Like if you go to us, you don't have to lock up, you know, like a rectangular block of GPUs over three years. And you have to know the exact number. It's pay as you go. You scale up and down. You pay per GPU second. How do you do GPU buying in terms of forecasting then? Because you don't want to have an insane excess of supply just sitting on your books.

[20:27](https://realalpha.app/v/1dwyL6Iz_PU/?t=1227) Totally. The GPU lead times right now is about three to four months. So we basically do a straight line or exponential line extrapolation, and we look at an aggregate across all our customers. We have a very diversified set of customers. You can actually look at the demand in aggregate pretty well and predict based on monthly growth rates, which have been about 30%, 40% month-to-month for the last 12 months.

[20:46](https://realalpha.app/v/1dwyL6Iz_PU/?t=1246) And you just project that three to four months into the future, you end up with like two to three X. You go out and buy that capacity and make sure to ramp up. And can you just buy that capacity? You can, yeah. I think there's a little bit of a misconception that we're out of capacity. NVIDIA is shipping an insane amount of GPUs every month. I don't know how many hundred thousand Blackwalls are getting installed.

[21:05](https://realalpha.app/v/1dwyL6Iz_PU/?t=1265) Prices are up, for sure. You have to pay more to get the same amount of GPUs. And I don't want to say it's an efficient market, but if you're willing to pay more than other people, you can get a lot of GPUs. Is there a price at which it doesn't make sense for you? Of course there is, but at what point? I don't think so.