AI compute demand grows as models use more tokens

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 57:18

   So the thing that I'm – a year ago, we were all sitting around and talking about the ups and downs of the overbuild in 2000. around the internet. And yet, when I look at the trajectory that we're on, I saw Mike Intratore on CNBC today, and he said, listen, we're still underestimating the amount of demand that's out there in the world. And when he says it, or when you say it, when Jensen says it, in some ways, people would argue it's self-serving. Of course, you guys are going to say that. That's your business. You're going to tell everybody your business is great, but you're known as a very sober guy who tells it like it is. And what I want you to do is reflect a little bit on the comparison between this and the period in early 2000 when we did get overbuilt, right? And as the saying goes, every shortage ends up in a glut.

2. 58:15

   Why are we not near that point yet today in this market? Well, I mean, you guys as students of the market can go back and, you know, sort of review what the multiples were on earnings and cash flow, you know, around that time. We're nowhere near that for the most part. But if we go back to the underlying activity here, it's all about the tokens, right? And as we go from basic queries to test time compute to deep reasoning to agents and multi-agent systems, the number of tokens just explodes.

3. 59:03

   And what are we talking about in tokens? When we're talking about tokens, we're talking about knowledge. Exactly. And I don't know about you, but I'm using these tools like 50 times a day as my thought partner. To solve problems and, you know, quelch my curiosity and, you know, my usage is skyrocketing and, you know, often multiple models and it's going out there and querying all these websites, doing calculations for me and helping me solve problems, you know, faster than I ever could in the past.

4. 59:49

   And, you know, this, I think it's just at the beginning, right? And the substrate for all of this, of course, is compute and data, right? So we love that at Dell Technologies because that's what we do. And so there's just a ton of growth here. I think it will also be highly distributed. I think it will be, you know, it'll occur in devices, it'll occur in the edge, it'll occur, you know, in all sorts of places.

5. 1:00:21

   And it does feel like we're still a lot closer to the beginning here. What can you share about on-prem AI deployments, Michael? Are you seeing anything interesting there? Yeah, so we, in the last year, you know, delivered a little over 3,000 of these Dell AI factories. And, you know, those are increasingly to enterprise and commercial customers that want to bring the AI to their data, not the data to the AI.

6. 1:00:57

   And, you know, there's just a ton of data that is still on-prem and being generated on-prem. And it turns out, you know, these large models are fantastic, but you don't always need the largest model to solve every problem. A lot of the corporate use cases are perfectly done with smaller models and open source models. And so you see this enormous proliferation and a hugging face of models of all shapes and sizes, tons of cascading innovations.

7. 1:01:31

   And so I think this is going to be highly distributed, and we're definitely seeing growth in on-prem. And colos are also a big deal, you know, because many customers don't want to have the data center themselves. And so they'll put it in a co-location facility. And we've also adopted the consumption type model so you can pay on a usage type basis.