Modal’s early sandbox bet became a major revenue line

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 5:21

   for our listeners on the business side. Maybe starting, you know, I think one of the interesting things about the history of modal is, for a while it felt like you publicly said, I think on your first unsupervised learning episode, that you were less focused on these LLM inference calls, specifically like a simple API call for a model. And you kind of recently released a product there. Maybe talk a little bit about the journey and kind of that initial reticence and what made you decide that like now is the right time to really double down on this.

2. 5:53

   Yeah, so we started building this in late 2021, basically, or early, somewhere around that time. And the initial product was this serverless function. And we thought it'd be really useful for a wide range of stuff around like data, AI, machine learning, broadly speaking, kind of like looked at a lot of different use cases. But then like ChatGPT came late 2022 and Stable Diffusion also came around the same time.

3. 6:18

   It turned out like we were like super well positioned to host a number of Stable Diffusion applications. So that was like the first year of growth was like all Stable Diffusion. And then we started like expanding into like adjacent use cases like audio and video and stuff like that. And it was like early teams that, you know, they were deploying JNI inference. They didn't want to deal with down-aligned infrastructure.

4. 6:38

   They wanted to move fast. Like we started aggregating a lot of different GPUs across a lot of different clouds, even back then. We always knew we wanted to be multi-product, but we started with the serverless, like the sort of compute primitives. We felt like it was so general purpose. And we focused a lot on like kind of what I think was like high code use cases, like focus on like machine learning engineers and researchers who want to write code and focus a lot on like being kind of lower in the stack, right?

5. 7:02

   And pretty early, we built also a storage layer, a file system to store data, in particular, container images. Then we launched a sandbox product in 2023. And the reason why we launched it back then, we started seeing people messing around with LMs to generate code. And we figured actually a lot of stuff we built on the infrastructure side is really well suited to run that code. And they're going to need a place to run that in an isolated way.

6. 7:29

   didn't immediately it wasn't like an overnight success but like basically 2025 it kind of blew up because i think models got really good at writing code back then so about a year ago so that was sort of our venture into like kind of becoming like a multi-product vendor like starting about 12 months ago like sandboxes has been in a tier and grown like i don't know a thousand x almost or something like that it's about a third of our revenue right now standalone is a 300 million run rate business uh sandboxes uh so that's been incredible in terms of you know becoming a multi-product vendor we just launched also a training So, sort of, you know, building, trying to lay out, like, all the different building blocks, and, you know, what does that suite of tool looks like, and what are the toolboxes in this, like, new world of, like, AI applications? Like, they're all going to need a bunch of new primitives, and, like, we want to offer all those primitives, and what does that look like?

7. 8:19

   So LM inference was something We were a little bit on the sidelines for a while Until actually I would say the DeepSeq moment So that was about a year and a half ago Like early 2025 When DeepSeq came out Which was like the first Chinese open source model That was like actually really good And then I was like fuck it, we gotta get in here And like we started building up a team To do a lot of inference optimization Models has always been really good at LM inference But we didn't necessarily have a product for it In fact we didn't even have an inference product until very recently.

8. 8:48

   We were just like really good at hosting inference, including LLM inference, but it was like kind of custom. And a lot of the teams that want to run LLM inference a year or two ago, like they had to deploy it themselves or like we would work with them. But we started seeing a lot of demand with the DeepSeek moment over the last 18 months, started seeing a lot of growth. LLM inference is now about a third of our revenue too.

9. 9:08

   So we've seen incredible growth with that. And a few months ago, we shipped a number of LLM specific features Because we think, you know, we kind of laid this like foundational layers, like the storage, the compute, the serverless functions, the file system, etc. I think there's a lot of opportunity to move up in the stack. So we started moving up in the stack now and launched a dedicated inference product.

10. 9:33

    So you can get like a single tenant models, like all the state of the art Chinese models or a bunch of US wants to like one single click or CLI command. You can just deploy on modal as well as a bunch of shared endpoints. You can just call models off the shelf without having to deploy them, fully token-based pricing. And so I think those are the first out of many more products to come slightly higher up in the layer.

11. 9:56

    We think there's a lot of opportunity because we're sitting on this very solid foundation in the compute and the storage.