Falling compute costs can improve customer economics

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 16:00

   your question. So putting on the CFO hat, help us understand the capital allocation model that you use. Because a lot of businesses over the last decade, two decades that have kind of been these outsized returners have found some unique way to deploy capital at a higher ROC than anyone else. And then you end up plowing all your capital into that higher ROC bucket. What is that for you guys? And how do you think about that portfolio approach to having more of these kind of big returner shots, and is there an engine where that gets better over time? There has to be, because in the end, the durable, high-value companies created in this era, I don't think they're not going to be magical. They're going to look like the great companies of prior eras. They're going to create customer value. Starts with the customer and really helps the customer do something different,

2. 16:51

   better, more revenue, more efficiency, right? Thermo Fisher wants to be able to get patient screening done faster so they get FDA approval faster. That's really important. Like if you have a form of cancer where you have weeks to live, the difference between a breakthrough in four weeks and two weeks can literally be life or death. They also have, I'm going to misquote this, but something like 30,000, 38,000 people in the field selling those amazing, like if you walk into any lab in the country, you'll just see Thermo Fisher plastered all over every device. Those people want to be more efficient going to work. Like the fastest takeoff of Codex within OpenAI right now is actually in our go-to-market team. Our devs are there, but like if you look at the pace of growth kind of month over month, it's all in GTM. So they want more productivity out of their GTM team.

3. 17:43

   And of course, they're doing things in areas like finance, which I get really excited about. So customer value first. From that, now you need to get to a great gross margin. So how do you get to a great gross margin? You're looking at the cost of revenue. The main input is compute. The good news on compute is that there is a massive deflationary curve on cost. From chat GPT 5 to 5.4, I think the deprecation cost was something like 97%.

4. 18:10

   97%. It's kind of an amazing curve. Actually, I'm sliding from 4 to 5.4. It was 97%. But that happened in like two years. That's kind of wowing, right? Even our newest model, if you look at 5.5 that we just released, we're trying to now translate that back to the customer. So we actually raise prices on 5.5 2x. But if you look at what the cost of the customer is, they're probably still getting a break of about 20 to 30% cost reduction per token, because just much more efficient per token. So there's a lot to do in that envelope. And part of making a capital allocation decision is having to, if you make it on today's cost profile, you actually might misprice the outcomes. You have to lean in a little on the cost profile. And then as we think about the bills, yeah, you are having to make, like really, my focus today on compute is what's

5. 19:04

   the compute I can buy for 28 onwards, like that Michigan data center in Saline, I don't think we will be getting compute out of it until probably end of 27, early 28. So that's where you're starting to make your bets. And in fact, where I feel most short of compute right now is starting to look at 30, 31, 32. So you're having to create a business model. Now, the good news is each year goes by, we get more confidence in the build. We're seeing it massively outperform. And so that's giving us more and more confidence. And the market is coming towards us much more. So how are you making the compute need forecast multiple years out, accounting for all of the architectural and model advancements that are happening, where call it value or utility per unit of power is going up and help us understand how you kind of...

6. 20:00

   estimate that given that there's a lot of technology development going on that has a high kind of variance to it. Yeah, yeah. So we do have to make multiple assumptions, both on the compute itself. So we assume right now that compute actually on a per gigawatt is getting more expensive because power is getting more expensive, memory is getting more expensive, and so on. However, the intelligence that we get on the other side out because of the deprecation on the chip side is more than making up for that. So in terms of a per unit sold to a customer, it should actually get a lot less expensive for the customer. No model improvement in that. So that's just the chip. Exactly. That's just the chip itself. We don't try to overestimate on the model side because sometimes like 5.5 is an incredibly good model on the efficiency side. But if you look at

7. 20:47

   something like 5.4, the prior model, it was a really large pre-trained model. It was very expensive. It was actually hard to serve. And sometimes we want to do that really big pre-trained moment. And then we take multiple model turns to be able to kind of drive down on the cost side. I mean, in the near term, like in 26 and 27, I clearly build a model that's bottoms up. So I know what my products are.

8. 21:12

   I have a sense of what the pricing will be. You know, consumer, P times Q. How many wows do I think I have? I can see what the shape of the line is. how many of them will subscribe. Advertising coming in is also still related to how many weekly actives, how many dailies, how many messages, and so on. So you can do actually a pretty good model job in 26 and 27. That said, the shape of the line keeps taking us by surprise to the upside. When you get into the outer years, you're actually looking more at the compute you've bought and almost just doing an algorithm the other way that's saying this amount of compute should equate somewhat to this amount of revenue. I don't know for certain exactly where it will all come from. Like a year ago, I built a model for investors that showed agentic revenue. And the

9. 21:59

   story was, we're going to have this thing. We're going to be in the agentic era. We're going to hand it to a developer. With natural language, they're going to be able to build. And we think they will pay upwards of maybe $2,000 a month for it, which is kind of laughable in hindsight, it, but nobody believed. They were like, I don't even know what she's talking about. There's no way that will happen. And $2,000 a month? Remember when people were losing their minds over ChatGPT Pro being at $200? Like, oh my God, no one will ever pay for that.

10. 22:30

    So why $122 billion? Does it take you to 2031, 2032? How do you get the calculus on the