AI lab profits hinge on compute allocation and demand forecasts

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 58:48

   You've told investors that you plan to be profitable starting in 28. And this is the year where we're potentially getting the country of geniuses of a data center. And this is going to now unlock all this progress in medicine and health and et cetera, et cetera, and new technologies. Wouldn't this be exactly the time where you'd want to reinvest in the business and build bigger countries so they can make more discoveries?

2. 59:15

   So, I mean, profitability is this kind of like weird thing in this field. I don't think in this field profitability is actually a measure of, you know, kind of spending down versus investing in the business. Like, let's just take a model of this. I actually think profitability happens when you underestimated the amount of demand you were going to get and loss happens when you overestimated the amount of demand you were going to get because you're buying the data centers ahead of time.

3. 59:48

   So think about it this way. Ideally, you would like – and again, these are stylized facts. These numbers are not exact. I'm just trying to make a toy model here. Let's say half of your compute is for training and half of your compute is for inference. And the inference has some gross margin that's like more than 50%. And so what that means is that if you were in steady state, you build a data center, if you knew exactly the demand you were getting, you would get a certain amount of revenue.

4. 1:00:21

   Say, I don't know, let's say you pay $100 billion a year for compute. And on $50 billion a year, you support $150 billion of revenue. And the other $50 billion are used for training. So basically you're profitable. You make $50 billion of profit. Those are the economics of the industry today. Or sorry, not today, but like that's where we're projecting forward in a year or two. The only thing that makes that not the case is if you get less demand than $50 billion, then you have more than 50% of your data center for research and you're not profitable.

5. 1:00:59

   So you train stronger models, but you're like not profitable. If you get more demand than you thought, then your research gets squeezed. But, you know, you're kind of able to support more inference and you're more profitable. So it's maybe I'm not explaining it well, but the thing I'm trying to say is you decide the amount of compute first. And then you have some target desire of inference versus versus training.

6. 1:01:26

   But that gets determined by demand. It doesn't get determined by you. What I'm hearing is the reason you're predicting profit is that you are systematically underinvesting in compute, right? Because if you actually like – No, no, no. I'm saying it's hard to predict. So these things about 2028 and when it will happen, that's our attempt to do the best we can with investors. All of this stuff is really uncertain because of the cone of uncertainty.

7. 1:01:50

   Like we could be profitable in 2026 if the revenue grows fast enough. And then if we overestimate or underestimate the next year, that could swing wildly. Like what I'm trying to get is you have a model in your head of like the business invests, invests, invests, invests, gets scale and kind of then becomes profitable. There's a single point at which things turn around. I don't think the economics of this industry work that way.

8. 1:02:19

   I see. So if I'm understanding correctly, you're saying because of the discrepancy between the amount of compute we should have gotten and the amount of compute we got, we were like sort of forced to make profit. But that doesn't mean we're going to continue making profit. We're going to like reinvest the money because, well, now AI has made so much progress and we want the bigger country of geniuses.

9. 1:02:37

   And so then back into revenue is high, but losses are also high. If we predict, if every year we predict exactly what the demand is going to be, we'll be profitable every year. Because spending 50% of your compute on research, roughly, plus a gross margin that's higher than 50% and correct demand prediction leads to profit. That's the profitable business model that I think is kind of like there, but like obscured by these like building ahead in prediction errors.

10. 1:03:13

    I guess you're treating the 50% as a sort of like, you know, just like a given constant. Whereas, in fact, if the AI progress is fast and you can increase the progress by scaling up more, you should just have more than 50% and not make profit. Here's what I'll say. You might want to scale up it more. You might want to scale it up more. But, you know, remember the log returns to scale, right? If 70% would get you a very little bit of a smaller model through a factor of 1.4x, right?

11. 1:03:41

    Like that extra $20 billion is – each dollar there is worth much less to you because the log linear setup. And so you might find that it's better to invest that $20 billion in serving inference or in hiring engineers who are kind of better at what they're doing. So the reason I said 50%, that's not exactly our target. It's not exactly going to be 50%. It will probably vary over time. What I'm saying is the log linear return, what it leads to is you spend of order one fraction of the business, right?

12. 1:04:21

    Like not 5%, not 95%. And then you get diminishing returns because of the log scale. I'm like convincing Dario to believe in AI progress or something. But like, okay, you don't invest in research because it has diminishing returns, but you invest in the other things you mentioned. Again, we're talking about diminishing returns after you're spending $50 billion a year, right? This is a point I'm sure you would make, but diminishing returns on a genius could be quite high.

13. 1:04:52

    And more generally, what is profit in a market economy? Profit is basically saying other companies in the market can do more things with this money than I can. Put aside Anthropic. I don't want to give information about Anthropic is why I'm giving these stylized numbers. But like, let's just derive the equilibrium of the industry, right? I think – so why doesn't everyone spend 100% of their, you know, 100% of their compute on training and not serve any customers, right?

14. 1:05:23

    It's because if they didn't get any revenue, they couldn't raise money. They couldn't do compute deals. They couldn't buy more compute the next year. So there's going to be an equilibrium where every company spends less than 100% on training and certainly less than 100% on inference. It should be clear why you don't just serve the current models and never train another model because then you don't have any demand because you'll fall behind.

15. 1:05:48

    So there's some equilibrium. It's not going to be 10%. It's not going to be 90%. Let's just say as a stylized fact, it's 50%. That's what I'm getting at. And I think we're going to be in a position where that equilibrium of how much you spend on training is less than the gross margins that you're able to get on compute. And so the underlying economics are profitable. The problem is you have this hellish demand prediction problem when you're buying the next year of compute. And you might guess under and be very profitable but have no compute for research. Or you might guess over and you are not profitable and you have all the compute for research in the world.

16. 1:06:33

    does that make sense just as a dynamic model of the industry maybe stepping back i'm like uh i i'm not saying i i think the country of geniuses is going to come in two years and therefore you