Inference providers may capture spend beyond model makers

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 31:34

   I want to move actually to one of the most interesting rounds I thought was Modal closes at a $15.75 billion valuation. Run rate surpasses a billion dollars. It's $750 million Series D led by Excel. I just actually interviewed Eric the other day. He was amazing actually in the studio. How did we think about this round and what it means in terms of the wider inference market? I like it better than CoreWeave.

2. 31:55

   I like adding value to the universe rather than just being some outsourced racks of GPUs. I like this one better, even though, to Rory's point, the competition appears to be infinite and limitless, right? So you better have some cojones in venture growth to do it. But, yeah, I like at least adding value to the universe. The interesting thing about modal is they were always perceived strong by developers, but I think they were like 60 or under 100 million mid-late last year, right?

3. 32:26

   And that level of acceleration is just amazing, right? I mean, it's stunning when you see it. Yeah. So, I mean, the first big picture is, oh, my God, inference is exploding here, right? And yes, they are very developer-friendly. They have a whole bunch of products. So, yes, that's the takeaway. The other thing I was just thinking about it is, you know, how big could this market be? I mean, I think you look at the four or five key inference providers.

4. 32:47

   I mean, you add them all up and maybe they're plus or minus five or six billion dollars in run rate revenue, which is a lot. But if you think about it, they're all serving all the open weight models in a world where right now, and it's not like what like, the sum of the OpenAI and anthropic revenues is probably north of 100, 110 billion right now. So by dollars, it's less than 5%. So it just points to the opportunity here.

5. 33:10

   I mean, going back to Jason's Replit guy, think about Amjad at Replit sitting there going, do I want to give money to Cloud Code and give them a 70% gross margin for this token? Or do I want to buy it on modal and give them a 30% gross margin for this token? Well, if the tokens are roughly the same, you know what you're going to do. So, yeah, it does point to the upside there. I completely agree it does.

6. 33:31

   I'm constantly shocked by like the infinite expanding size of the inference market. Absolutely. No, it's stunning. It's the one thing that I know today that I wish I'd known 12 months ago because I would have done things slightly differently 12 months ago had I been more expansive around my mindset here. Well, so you had to be smarter than me to understand how much this revenue could be captured outside of open an anthropic.

7. 33:51

   A year ago, I wasn't smart enough to really think through this. I didn't think enough of it would be captured outside of it. So I didn't think it was that interesting. Are these inference providers going to be worth three, four, five hundred billion dollars? I would say it's a race between two things. You can put your MBA hat on and say, you know, open weight. Going back to Jason's comments on CoreWeave, all the neoclouds are saying, oh, my God, let me get this straight.

8. 34:12

   We do all the building work. We buy all the compute. We borrow all the capital. And then these bozos come in, buy from us, sell it to developers, and make a shit ton. Why don't we do that? So, you know, you put your MBA hat and you can imagine a lot more competition here, right? And then you say to yourself, I doubt, you know, you get those kind of valuation. On the other hand, when you have this kind of growth rate, nothing else matters.

9. 34:32

   It covers a multitude of sins. My best is their margins have crept up this year, right? They have. Fireworks are mid-30s now, and they'll be hitting 50 pretty soon. Even mid-30s, if it stays there, I mean, the public markets can tolerate that. It's just the multiple you'll ultimately get on all of it, right? There's plenty of folks like that. So it's OK wherever it ends up, right? I mean, it's important for Excel, probably nailing it at $16 billion, but I think it works either way.

10. 34:58

    Do you think the three kind of hyperscalers of inference are base tan, fireworks, and modal? Probably in some simplistic who's the winner, you're probably right. Those are the three biggest. But maybe the statement is this. I don't think it's like the hyperscalers where no one else mattered. Jason, this is a big funding round, but it's too easy for you in the investment committee