AI training costs matter more than serving customers

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 32:30

   Yeah. With the bubble discussion, it's important to separate the claims that this technology is highly capable and getting more capable and could become very, very capable. That doesn't mean it will be economically lucrative, lucrative, at least at the beginning. And many past technological revolutions did create bubbles. There was an overcorrect or inflation and then the bubble bursts and there's still a transformative technology that's left behind. But people just get over overly excited and railways and the internet and two examples there yeah right like the dot-com bubble is the you know famous bubble and besides a housing bubble and with that you know telecom companies overbuilt um internet companies were over uh valued but then now the internet and tech is valued way way more than the dot-com bubble ever was and these are robust

2. 33:32

   valuations because these companies are the most successful companies in history. They have billions of users, incredible profit margins. And those companies are the ones that are financing the obsoleting project. And so that's like one important way it's less of a bubble is that this is coming out of profits of these companies that each have very, very successful businesses from like Google search to like Amazon web services to meta, Instagram, et cetera. And so, you know, the dot-com bubble was financed largely with debt. And when it burst, it left, I think, $2 trillion in today's dollars of debt behind. Whereas this, you know, there's some debt involved, but it's much, much less. I think the... Do you expect that to continue, by the way?

3. 34:21

   So at some point, the so-called hyperscalers... we can't finance this anymore. And so where does the money come from for the next training run in the future? I think you'll see more debt being taken on going forward. And it's not that the companies have fully run out of like cashflow to use, but in at least like Meta's case, they just don't want to put on their balance sheet. So they create these special debt vehicles so they can kind of like hide it, which, you know, that's concerning.

4. 34:50

   And that's like a bubbly type move. And you have companies like Oracle, which actually do have a lot of debt. um and are in like a tough financial spot um i think the key mistake that is made by the bubble proponents is around whether it's profitable to serve individual ai models to customers so everyone knows that these companies are unprofitable like famously so open ai is planning to burn 130 billion dollars before it becomes cash flow positive and profitable um for reference Uber burned like 30 something billion before it turned its first profit. And that was the previous record holder. So it's like world historic levels of burning money. But that doesn't actually mean that it's unprofitable to serve a customer. And what's happening here is that it's profitable to go and spend money on using Claude per token. The subscriptions are

5. 35:51

   on average profitable as well, even though some power users might be costing more than they're paying. The average user isn't doing that. What's happening is that it's so expensive to train the next generation of models that the companies are using all of the profits from their existing products, throwing that into training, and they still need more capital. And so then they raise from venture and from big tech investors. And the hope is that you just grow your revenue fast enough that you can exceed the growth rate of the training costs. And, you know, this is plausible if you have, you know, trillion dollars of revenue and your training costs are like in the hundreds of billions of dollars a year, then it could work. And that's like what Anthropic and Opening Eye are projecting will happen. And it sounds crazy until you consider the fact that these

6. 36:44

   These were the fastest growing consumer products of all time, the fastest revenue growth of all time. Anthropics percentage growth is like just truly unprecedented at the scale it's operating, going from like $5 billion at the start of the year to like 60 something billion annualized in a number of months. And so it's been more than 10xing year over year and is now poised to be the biggest IPO ever at two trillion or more dollars um and so but anthropic like they were profitable for one quarter kind of if you squint depending on how you assess these things but they're actually not planning to be profitable yet because they think they'll keep getting more value from training bigger and better models um that's very much part of the plan right so so So...

7. 37:33

   Spending money on training the next model is necessary for these companies' valuations to make sense. Otherwise, if they were to, say, stop training and then just serve the models, the other companies' open source would probably catch up at some point and you would be in a situation in which they couldn't make as much money from serving their model as they could before. Yes. And these companies could be profitable tomorrow if they just stopped training new models. They just don't think that's a good idea. And I think two risks for the industry.