AWS reserves GPU capacity to support startups

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 16:58

   the capacity needs in these small companies that don't have a lot of credit and whatnot? Well, yeah, it's a great question. And a couple of these things make it more challenging too. One is the CapEx expense needed to go deploy kind of all of the compute that everyone needs right now is massive, right? And so we are, I think... So your CapEx this year is what, $200 or something? $220 billion for 26.

2. 17:26

   At one point I saw that it's a pretty, I mean, that is a larger expense than we've ever had, maybe any company has ever had in a single year. And we don't anticipate slowing down anytime soon because the demand is just massive. And so at some point you're limited by how fast can you build data centers, how fast can you deploy capital, how fast can you get memory and chips and all of those kinds of things.

3. 17:48

   And so all of those are different points, constraints that were, you know, whether it's power, data centers, capital, memory, chips, they're all kind of constraints at various times. Construction people that build buildings, like construction people are at a premium today. And so all of those, we work really hard to make sure to happen. And then with the capacity that we are able to deploy, which is still a huge amount and not enough, we know, we really think intentionally about what is that allocation strategy.

4. 18:15

   And so we're great partners with the large frontier labs, to the Anthropics and OpenAIs and Meta and other large customers. And so those folks are really good customers of ours. Yep. And we want to make sure that we invest in them. We have large enterprise customers, whether it's Salesforce's or JPMC's or other large companies that have demand for fewer numbers of GPUs or accelerators. Sometimes they want Tranium, sometimes they want NVIDIA GPUs.

5. 18:43

   But we want to make sure that we can support them as well. And we're very intentional about how we make sure that we have capacity for startups. And so what we do is we actually do allocate and we basically say, okay, we're going to keep, you're right, we could sell every single GPU or AI accelerator we had to probably just the big frontier labs and call it a day. We choose not to do that because we actually want to keep growing the full ecosystem.

6. 19:07

   They get a large number, but we want to keep supporting a broader set of customers because we actually think that both the whole ecosystem will be more healthy for us. There's some depreciation, but it's also just, we know these are going to be big companies over time. And so we try to support them. I saw recently that we say, you know, yes, in some way, shape, or form to something like 60% of the requests we eventually get.

7. 19:30

   Sometimes it's a little bit later, sometimes it's in a different region, sometimes it's a slightly different configuration than the customers are looking for. But we really try to lean in and try to allocate as much as we can. And every single startup that you have wants more. Yeah, exactly. and so we're working hard to try to make sure that we have all of those but you know it's hard like that's you know as a lot of people say it's a good problem to have yes it is but it's a problem nonetheless and we're you know we continue to look at you know we recently announced we're going to be buying you know 2 million NVIDIA GPUs over the next couple of years like we're landing a massive amount of capacity and 2 million 2 million So, you know, it's a lot.

8. 20:14

   And it's over the next couple of years. And who knows if that's enough? I have no idea, but at some point, again, we're limited by other components as well. But it is. We're very intentional about keeping capacity available for the startups. And I know it's painful to not have enough, but we keep pushing. When did your CapEx cross, I mean, pure AWS, I mean, Amazon was a bigger company, crossed like, let's call it even 1 billion or 10 billion a year.