Modal built a procurement team as GPU supply tightened

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 13:49

   Let's talk a bit about cloud capacity because I think it's actually changed a lot. In the first few years, we got almost all the capacity pretty much from the hyperscalers, spot and on-demand capacity, mostly spot capacity, running in all the different regions using initially AWS, but also GCP and over time Azure. We started renting some capacity from Oracle pretty early because we wanted A10s and A200s and we got really good rates on those.

2. 14:17

   And over time, we added a couple more clouds, like doing reservations. But still up until actually end of last year, like most of our capacity was either spot or one-year reservations. Spot means you're just like renting it right there and then that's it. Exactly, yeah. And it's also preemptible, which we had to deal with. But it sort of works in aggregate because you can sort of just aggregate capacity from many hundreds of regions, basically.

3. 14:41

   So that model worked really well until we started running up again. I mean, first of all, like, modal started growing, like, very fast, and so we needed more capacity. But fundamentally, also, like, the cloud market started changing a lot. So in the last year, basically, now, in order to get the capacity we need, we have to work with a much larger set of clouds. We actually work with almost 25 clouds at this point.

4. 15:03

   And in many cases, in order to get the capacity we want, like, we have to go and do, you know, three-plus-year deals, and you have to pay a meaningful amount of money up front. and you're locking in pricing on a three-year term, right? And so we had to spend the entire last six to nine months kind of rethinking how we get capacity. We had to build a whole procurement desk. We have four guys working full-time, basically.

5. 15:29

   All they're doing is procuring capacity, calling neoclouts, doing deals, vetting the capacity. We've had to build a whole engineering team, basically 10 people just managing bare metal capacity from all over the world and dealing with network issues and bad drives and GPUs falling off the bus and all these things. We don't own capacity. We rent all the capacity so far, you know, on typically these three-year deals.

6. 15:55

   And that's like a big decision. Like, should we buy? Should we, you know, or should we keep renting? I'm actually like not like mega dogmatic about it. I think renting is nice because there's a certain amount of capital efficiency to it. If you're buying, you know, like, I mean, first of all, there's an obvious like operational complexity of it. Like you have to deal with vendors and suppliers and power and cables and all kinds of stuff.

7. 16:20

   But there's also financial complexity to it. Like how do you fund it? Like should you raise equity? And you can raise a billion dollars and go buy hardware for it. That's a very diluted way to do it. You can raise debt. But if you raise debt, you typically want to have like locked in long-term revenue to pay the interest rates. That's also not trivial. all. But, you know, like we look at the economics, we look at the operational complexity, we look at the risks. It's a complex trade-off and we're open to buying. We're sort of, there's other constructs too. Like we're, it's, to get the capacity we need for 2027, it's going to take a lot of work on, you know, going a lot of different avenues to lock it up.

8. 17:03

   So do you have to hire a lot of like finance people to do all this stuff or not really?