Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal

Summary · 5 min read

Erik Bernhardsson spent seven years at Spotify, where he built the first version of its music recommendation system and helped launch features including Related Artists and Discover Weekly. He later served as CTO of Better.com for five years, then founded Modal, an AI infrastructure platform.

Bernhardsson says Modal combines a developer-focused platform with pooled compute capacity, which lets customers scale bursty AI workloads without buying fixed blocks of GPUs. He describes Modal's growth from serverless functions into sandboxes, now about one-third of revenue, and sees fast, snapshot-enabled sandboxes as a way to run agents and tests more efficiently. He is adding GPU capacity aggressively but warns that falling prices and utilization could put long-term commitments at risk. He also discusses security, developer experience, Modal's cloud-first model and possible new uses for AI.

Minute by minute

1. 0:03

   Modal packages AI compute. Bernhardsson describes Modal as a compute and storage platform with an SDK for deploying and scaling AI applications, including inference, sandboxes, training and batch jobs. He says revenue, users and usage had grown at least 50x since his previous Latent Space appearance. He traces Modal's roots to building Spotify's first music recommendation system and seeing how poorly the infrastructure supported AI work; he later spent five years as Better.com's CTO.

2. 4:21

   Stable Diffusion validated demand. Bernhardsson says he began building Modal in 2021, before the generative AI boom, after exploring ideas including web scraping and cloud workflow orchestration. He shifted his focus from scheduling code to executing compute in the cloud. Stable Diffusion brought early demand: monthly revenue rose from $100 in November 2022 to $2,000 the next month and $30,000 in January 2023, mostly from one customer, Mage.

3. 10:39

   GPU infrastructure demands global operations. Bernhardsson says conventional cloud tools were built for steady CPU workloads, so Modal built its own filesystem, snapshotting, scheduler and autoscaler for bursty GPU workloads. Modal treats capacity across regions as one global pool and expanded from hyperscaler spot capacity and one-year reservations to working with almost 25 clouds. He says Modal rents its capacity, often under three-year deals, and weighs the operational burden of owning hardware against the equity or debt needed to finance it.

4. 20:08

   Overbuying GPUs carries tail risk. Bernhardsson says rising GPU prices benefit capacity owners now, but he worries more about a price collapse that could hurt companies locked into expensive supply. He is adding capacity aggressively, but says utilization, prices and resale demand could weaken together, so he ties purchases to expected demand. He cites the Hunt brothers' failed attempt to corner the silver market and says hardware prices tend to fall over the long run.

5. 26:23

   Alternative accelerators are improving. Bernhardsson says AMD GPUs, Google TPUs, Cerebras and Groq are becoming more interesting, and he expects 2027 may be the year non-NVIDIA accelerators gain traction. He describes hybrid inference setups that use NVIDIA for prefill and Cerebras for decode. He says neoclouds can scale by preselling multi-year capacity to large buyers, then using those contracts to raise financing. GPU markets are less standardized than oil, he adds, so treating them as a fully tradable asset class remains uncertain.

6. 32:13

   Functions and sandboxes anchor Modal. Bernhardsson says Modal Functions, launched in 2021, became useful for inference and now account for about two-thirds of revenue. Modal launched Sandboxes in 2023 for running generated code; after a slow start, demand grew with vibe-coding products, background agents and reinforcement-learning workloads. Sandboxes now make up about one-third of revenue, and Bernhardsson says Modal is building a broader set of infrastructure products.

7. 36:22

   Smaller training workloads are underserved. Bernhardsson sees an underserved market in smaller and mid-sized training workloads, including research, speech models, biotech and recommendation systems that may use 64 GPUs rather than 100,000. He also sees opportunity in batch jobs and post-training, including supervised fine-tuning and reinforcement learning. He says open-source models have improved enough that fine-tuning is now optional for many users.

8. 41:27

   Ephemeral sandboxes could speed parallel work. Modal added full virtual machines alongside its earlier gVisor-based sandboxes for workloads that need full Linux compatibility. Bernhardsson sees snapshots, forks and fast resumes as key to starting a sandbox for a command and shutting it down immediately afterward. For CI, he imagines snapshotting setup once and running every test in its own sandbox, reducing the overhead of parallel work.

9. 46:51

   Security needs safer defaults. Bernhardsson says Modal can provide a sandbox for Anthropic's managed agents. He clarifies that the Hugging Face incident involved a vulnerable app running on Modal, not a breach of Modal itself, and says Modal is considering more restrictive defaults. Separately, Modal has started using agents to probe its systems for vulnerabilities and sandbox escapes; Bernhardsson sees that as a potential defensive use of AI.

10. 52:49

    Developer experience still matters to agents. Bernhardsson says good documentation, clear error messages and fast feedback loops help both developers and agents, though agents can make difficult configuration tasks easier. Modal has become more cautious about deprecating features because agents often reproduce old syntax found in training data. The team also tests whether agents recommend Modal for different workloads and works to make its product and pricing information clearer to them.

11. 57:47

    Customer engineers turn gaps into products. Modal grew its forward-deployed engineering team from four people at the start of the year to about 24, using them to tune inference, troubleshoot, support migrations and identify product gaps. Bernhardsson says the long-term aim is to turn that work into self-serve features, and that product managers need deep strategic understanding of infrastructure. He says Modal treats code review as optional for low-risk changes, relying on strong tests and deployment systems, and encourages technical staff to keep building.

12. 1:03:58

    Cloud remains Modal's preferred model. Bernhardsson says he remains a cloud maximalist and sees multi-tenant cloud as the stronger default than on-prem deployment. He views on-prem and bring-your-own-cloud as increasingly outdated security approaches, though Modal might offer BYOC at a high price or use it to access partner capacity. He says the most important near-term use of cash is prepayments for GPU capacity.

13. 1:10:19

    Growth strains culture and capacity. Bernhardsson lists capacity procurement, pricing, product strategy and company culture among Modal's main priorities. The company had recently crossed 200 employees; he says it may reach 250 by year-end and possibly 700 the following year, while aiming to slow from its recent fourfold annualized growth. Modal is hiring systems and GPU-performance engineers, expanding sales and building research capabilities in areas such as post-training, distillation and quantization.

14. 1:17:30

    Flexible capacity fits bursty demand. Bernhardsson says many GPU contracts sell fixed three-year blocks, while customer workloads rise and fall; Modal can often provide 100 GPUs on demand by aggregating demand across customers. He says sandboxes are becoming more ephemeral, with the median Modal sandbox lasting about eight seconds. He also argues that inference providers can shift capacity as model demand changes, and points to computational biology and materials science as areas where AI may advance.

Published Oct 22, 2026.