Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal

Summary · 4 min read

Bernhardsson spent seven years at Spotify, where he built the first version of its music recommendation system, and later served as CTO of Better.com. He founded Modal to make it easier to run code in the cloud; the company now offers compute and storage infrastructure for AI applications.

Bernhardsson says Modal's value comes from combining developer-friendly tools with pooled compute that can serve bursty workloads, rather than selling customers fixed, multi-year blocks of GPUs. He describes capacity procurement and the risk of falling hardware prices as major financial challenges, while remaining open to renting, owning or combining capacity models. Modal's Functions and Sandboxes are its two main products, and he sees further opportunities in smaller training workloads, batch jobs and agent infrastructure. He also discusses sandbox security, engineering practices and the challenges of scaling Modal's team.

Minute by minute

1. 0:00

   Modal grew from infrastructure pain. Bernhardsson says he built Spotify's first music recommendation system and spent seven years there working on compute-intensive applications. After five years as CTO of Better.com, where he learned about management and developer productivity, he returned to infrastructure and AI. He says the shortcomings he encountered at Spotify helped shape Modal.

2. 4:17

   Stable Diffusion revealed demand. Bernhardsson says Modal grew out of experiments with cloud code execution and workflow orchestration, and that he self-funded the company at first. Stable Diffusion became an early use case: monthly revenue rose from $100 in November 2022 to $2,000 the next month and $30,000 in January 2023, largely from one customer. He says the company then diversified its revenue.

3. 10:01

   AI workloads need new infrastructure. Bernhardsson argues that tools built for steady CPU workloads were not designed for expensive GPUs, bursty demand or loading large models. Modal built its own filesystem, snapshots, autoscaler, scheduler and capacity-management tools. He says the company treats capacity across regions as one pool and had recently reached four or five nines of uptime.

4. 13:18

   GPU capacity requires longer commitments. Bernhardsson says Modal once relied mostly on spot capacity and one-year reservations, but now works with almost 25 clouds and often makes deals lasting more than three years, with substantial upfront payments. Modal rents rather than owns its hardware so far. He says buying would add operating and financing complexity, though the company is open to it.

5. 18:01

   Locked-in capacity carries downside risk. Bernhardsson says GPU capacity is available at a high enough price, even as supply remains tight and prices rise. He worries more about prices collapsing: companies locked into expensive capacity could face falling utilization and a weaker resale market at the same time. He says those correlated risks make it dangerous to buy or contract for far more capacity than demand supports.

6. 26:13

   Accelerators and GPU finance are diversifying. Bernhardsson says AMD GPUs, Google's TPUs, Cerebras and Groq are becoming interesting alternatives, and predicts that 2027 may be the year non-NVIDIA accelerators gain ground. He describes neoclouds using customer contracts to raise money for hardware before delivery. He says GPU trading faces a challenge because capacity varies by location, network and reliability.

7. 31:56

   Modal is expanding beyond inference. Bernhardsson says Modal Functions, launched in 2021, is used mostly for inference and accounts for about two thirds of revenue; Sandboxes, launched in 2023, account for about a third. He says sandbox demand grew with vibe-coding companies, background agents and reinforcement-learning workloads. He also sees opportunities in batch jobs and smaller training workloads, including speech models and computational biology, where some jobs can run on 64 GPUs.

8. 41:17

   Sandboxes could become ephemeral. Bernhardsson says Modal has added virtual machines, snapshotting and other features as sandbox use has grown. He sees potential in starting a sandbox quickly for a command, then stopping or snapshotting it, so agents can fork environments or run many tests in parallel. He says reducing startup and setup overhead could make that approach more practical.

9. 46:41

   Agents can test sandbox security. Bernhardsson says Anthropic's managed agents can use Modal as a sandbox provider. Discussing a Hugging Face incident, he says an agent exploited a vulnerability in an app deployed on Modal; he says Modal itself was not hacked. Modal is considering more restrictive defaults and is using agents to probe its systems for vulnerabilities and sandbox escapes.

10. 52:48

    Developer experience still matters. Bernhardsson argues that good documentation, clear errors and fast feedback help both developers and coding agents. Modal finds that agents often use outdated syntax from their training data, so the company has become more cautious about deprecations. It also tests whether agents recommend Modal and works to make product information clearer to them.

11. 57:44

    Customer work should become product. Bernhardsson says Modal expanded its forward-deployed engineering team from four people to about 24 to help customers tune workloads, cut overprovisioning and resolve issues. He wants recurring customer work to become self-serve product features, and says infrastructure product management requires deep strategic judgment. He describes code reviews as optional for some changes, backed by strong testing and deployment controls.

12. 1:05:42

    Cloud control shapes Modal's model. Bernhardsson calls himself a cloud maximalist and says control of its infrastructure lets Modal move faster. He sees on-premises and bring-your-own-cloud deployments as unnecessary for most customers, though Modal could consider them for the right deal or to pool capacity already held by neoclouds. He says capacity prepayments are the main use of cash, with renting versus owning still an open question.

13. 1:13:01

    Growth raises culture and hiring risks. Bernhardsson says Modal's team grew fourfold on an annualized basis over the prior 12 months, which he considers difficult to sustain without cultural drift. He says the company may reach about 700 people the following year and is hiring mainly systems engineers, while building research capabilities in post-training, distillation and quantization. He wants customer optimization work to turn into self-serve products.

14. 1:17:27

    Flexible compute fits changing demand. Bernhardsson says the median Modal sandbox lasts about eight seconds and sees potential for even shorter-lived environments. He argues that three-year GPU blocks do not match workloads that rise, fall and burst, while pooling demand across customers lets Modal offer more flexible capacity. He also says inference providers can shift resources as demand moves between models, and points to computational biology and materials science as possible AI applications.

Published Oct 22, 2026.