Each model can make money while the company spends at a loss

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 29:51

   So is it the is what's happening that these models are just so expensive for Anthropic to run that it's hitting it's a wall of its own? Again, I think you're I think you're making assumptions here. That's why I'm asking the CEO. Yeah. You know, you know, the way I think about it is we think about the models in terms of how much value are they creating. Right. So as the models get better and better, I think about how much value they create.

2. 30:16

   And there's a separate question about how the value is distributed between those who make the model, those who make the chips and, you know, those who make those who make the the the the the the the underlying applications. So again, without being too specific, I think there are some assumptions in your question that are not necessarily correct. Can you tell me which ones? So I'll say this. I do expect the price of providing a given level of intelligence to go down.

3. 30:48

   I expect the price of providing the frontier of intelligence, which will provide kind of increasing economic value. that might go up or it might go down my guess is it probably stays about where it is but again the value that's created you know goes way up so to you to two years from now my guess is that we'll have models that cost of the same order magnitude that they cost today, except they'll be much more capable of doing work, much more autonomously, much more broadly than they are capable of today. One of the things that Amjad mentioned was he thinks that the bigger models are not as intensive to run or more intensive to run given their size because of the architecture and some of these techniques that we talked about, that they're lighting up only certain sections of the model. So his idea, I'm hopefully conveying

4. 31:35

   this truthfully, is that Anthropic can run these models without too much bulk on the back end, but is still keeping those prices where they are. And I think the line that I'm going to draw there is maybe that to get to software margins, there were some reports that Anthropic is slightly below software gross margins. You're going to have to charge a little bit more for these models. So, yeah, again, I think larger models cost more to run than smaller models.

5. 32:06

   You know, I think the technique you're referring to is maybe mixture of experts or something like that. So whether you're models or mixture of experts or not, like mixture of experts is like a way to run models more cheaply. They have a given number of parameters. It's a way to train models. But if you're not using that technique, then larger models that don't use that technique cost more to run than smaller models that don't use that technique.

6. 32:28

   And if you're using that technique, larger models that use that technique cost more to run than smaller models that are using that technique. So I think that's sort of a distortion of the situation. Basically, I'm just guessing and I'm trying to find out what the truth is from you. Yeah, look, so I, you know, in terms of like the cost of the models, like one thing you'd be surprised by, people, you know, people kind of impute this thing to like, oh, man, it's going to be really hard to get the margins from like X percent to Y percent.

7. 33:02

   We make improvements all the time that make the models like 50 percent more efficient than they are before. we are just at the beginning of optimizing inference. Inference has improved a huge amount from where it was a couple years ago to where it is now. That's why the prices are coming down. And then how long is it going to take to be profitable? Because I think the loss is going to be like $3 billion this year. So I would distinguish different things.

8. 33:29

   There's the cost of running the model, right? So for every dollar the model makes, It costs a certain amount that. is actually already fairly profitable. There are separate things. There's the cost of paying people and buildings. That is actually not that large in the scheme of things. The big cost is the cost of training the next model. And I think this idea of the companies losing money and not being profitable, it's a little bit misleading. And you start to understand it better when you look at scaling laws so as a thought exercise these numbers are not exact or even close for anthropic let's imagine that in 2023 you train a model that costs 100 million dollars um and then in 2024 you deploy the 2023 model it makes 200 million dollars in revenue but you spend a billion dollars to train um you know to train a new model in 2024 so and then you know and then in 2025 the billion

9. 34:28

   dollar model makes two billion in revenue and you spend 10 billion to train the next model So the company every year is unprofitable. It lost $800 million in 2024. And then in 2025, it lost $8 billion. So, you know, this looks like a hugely unprofitable enterprise. But if instead, I think in terms of is each model profitable, right? Think of each model as a venture. I invested $100 million in the model.

10. 34:52

    And then I got $200 million out of the model in the next year. So that model had 50% margins and made me $100 million. The next year, the company invested a billion dollars and made $2 billion in – or sorry, the next model, the company invested a billion dollars and made – So are your models profitable? So every model is profitable. They're all profitable. But the company is unprofitable every year.

11. 35:17

    I'm not – this is a style. I'm not like claiming these numbers for Anthropic or claiming these facts for – but this general – Is it true for Anthropic? This general dynamic is, in general terms, the explanation for what is going on. And so at any time, if the models stopped getting better or if a company stopped investing in the next model, you would have probably a viable business with the existing models.

12. 35:49

    But everyone is investing in the next model. And so eventually, it'll get to some scale. But the fact that we're spending more to invest in the next model suggests that the scale of the business is going to be larger the next year than it was the year before. Now, of course, what could happen is like... The models stop getting better and there's this kind of one-time cost that's like a boondoggle and we spend a bunch of money, but then the companies, the industry will kind of return to this plateau, to this level of profitability, or the exponential can keep going. So I think that's a long-winded way to say, I don't think it's really the right way