AI demand is concentrated, but AWS won’t displace core customers

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 20:53

   I'm curious how you think about AI demand right now. offline we were talking about how you see it as a barbell and maybe you can sort of situate or contextualize sort of the state of ai demand at the moment well i mean the first uh you know the first context piece is what i said earlier which is just i can't believe how how big and fast ai has grown you know it's you know a few hundred times faster than the beginning of aws in the first three years 25 billion dollar run rate for us just in in ai revenue in aws in the first three years. So it's really large, and yet it really is barbelled if you look at how demand is. And what I would say is on one end of the barbell are the AI labs who are consuming gobs and gobs of compute, unlike anything I've seen. And then a few runaway apps, ChatGPT, CloudCode, things like

2. 21:47

   that. On the other end of the barbell are enterprises who are largely using AI for cost avoidance and productivity. So automating customer service or fraud or business process automation. In the middle of a barbell are the current production workloads of enterprises. Many of which are using AWS and using inference today but I would say the overwhelming majority of which still aren't in any pervasive way. And that I think is going to be the largest absolute segment of AI usage.

3. 22:26

   And I think you're going to see that. It's hard for me to predict the rate at which that'll happen. It's happening to a greater extent every year. But I think if you zoom out five years, five to seven years, and you look at what it'll be like, and I don't think it's going to wait for five to seven years. It never happens in one fell swoop. But if you just zoom out five to seven years, I think that quite a few, if not most, of the enterprise applications that we use and that exist today are going to be using inference in a pervasive way.

4. 22:57

   And so there's just, there is a lot of growth in front of us. And how do you think about it? Because there's kind of this impending data center chip memory everything shortage that's upon us in really 2027 just because, you know, everybody's building now, but it's not going to be ready necessarily by the end of the year and demand's not slowing down at all. And so when faced with, how do you think about it when faced with like the big labs being willing to pay maybe anything or almost anything for power, cooling, chips, you know, kind of the environment, and then you've got these kind of all been around a long time customers who are paying their regular rates, you know, to run their businesses, is there, like, any business pressure to go, well, maybe.

5. 24:00

   we should buy out the old guys and sell them to the new guys because they'll pay us whatever. And the old guys, you know, I can't double their prices. That wouldn't be fair. Or, like, how do you think about that? Because it's such a weird thing in that you've got people you've had in there for 20 years doing a thing that's kind of, you know, as you say, probably going to inference. And then you've got these new guys who are, like, insatiable.

6. 24:24

   They'll buy in. Yeah, it's a good question. It's, you know, we don't think of it quite the way you're talking about, but I can completely understand. If you think about there's only so much data center and power today, there's only so much memory, and I don't think that that shortage ends in 2027. I think, you know, I believe, you know, one of the reasons, coming back to the capital investment, why I think we can invest a lot more and it wouldn't be a bad use of capital is that, you know, 26 is obviously at this point sold out.

7. 24:57

   27 is largely sold out. We always leave a certain amount for on demand in our inference service bedrock or EC2, but largely sold out, all the big chunks. A good chunk of 28 is sold out. Like, there's so much demand that, you know, it is, and you can't, we have, if you think about the AI business and what we call the core business, which think of core as non-AI, they're both growing at really rapid rates right now.

8. 25:24

   And, you know, if you have only a certain amount of memory, you have to decide how much the memory you want to, you know, you want to allocate to HBM, you know, to the AI accelerators versus to DRAM, you know, into, you know, your core. So, these are real tradeoffs you have to make. But I would say that where we start on this issue is being customer focused. And for us, we have deep relationships and partnerships with our customers in AWS.

9. 25:56

   And in the same way during the pandemic, unlike our competitors who I think tried to figure out how to optimize within quarters to drive revenue as much as possible in a world where companies were going to cost optimize, we took a different approach. We, you know, we proactively had our field go out and help customers figure out how they can cost optimize, even though we knew it would diminish our revenue short and medium term.

10. 26:23

    But, you know, we're going to help our customers be successful because our very strong belief is if we help customers over a long period of time, our business will come along with it. And that's the same thing here on the core. I guess you could do calculations and decide that you have customers, because of the scarcity and their willingness to pay, would pay more to squeeze out your core customers, but they're running their businesses on top of AWS.

11. 26:48

    They're running all their core applications. We're not going to squeeze our customers, and we're going to make sure that they can scale. And then our job is to try to figure out how to have as much capacity as we can to allow not just our core customers, but all of our customers who want to use us on the AI side and people want to use us on the AI side to be able to do so and do so in a way where, you know, it's sustainably economically for us as well.

12. 27:18

    And so, you know, I wish I could tell you that I thought that the supply constraints would be just a 2027 issue. I think we'll see them in different components for several years. I think that we're probably putting more capacity in than anybody else because we are very passionate about continuing to be the significant leader in the infrastructure and AI space like we've been. But as I said, we're spending $220 billion or so this year and we could spend a lot more and it wouldn't be a bad use of capital and both medium to long term.

13. 27:54

    And I think we still probably wouldn't satisfy all the demand. There is just an incredible demand right now.