# AWS CEO Matt Garman on building a $1 trillion business Markdown copy of https://realalpha.app/v/1C_MErgk2Z0/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy. Matt Garman says AI is accelerating cloud adoption and leaves inference far below its eventual scale. - Show: Sources - Hosts: [Alex Heath](https://realalpha.app/people/alex-heath/) - Guest: [Matt Garman](https://realalpha.app/builders/matt-garman/) is CEO of AWS. - Published: Nov 10, 2026 - Length: 53 min ## The best minutes - [41:38](https://realalpha.app/v/1C_MErgk2Z0/?t=2498) [Mathematical proofs could make AI agents safer](https://realalpha.app/s/mathematical-proofs-could-make-ai-agents-safer/) (4 min) - [14:07](https://realalpha.app/v/1C_MErgk2Z0/?t=847) [Compute boom risks depend on how providers finance and sell capacity](https://realalpha.app/s/compute-boom-risks-depend-on-how-providers-finance-and-sell/) (3 min) - [2:21](https://realalpha.app/v/1C_MErgk2Z0/?t=141) [AI is accelerating the move of core workloads to the cloud](https://realalpha.app/s/ai-is-accelerating-the-move-of-core-workloads-to-the-cloud/) (3 min) ## Also worth watching - [38:02](https://realalpha.app/v/1C_MErgk2Z0/?t=2282) [AWS reserves compute for startups not yet formed](https://realalpha.app/s/aws-reserves-compute-for-startups-not-yet-formed/) (2 min) - [16:59](https://realalpha.app/v/1C_MErgk2Z0/?t=1019) [A 50% drop in inference costs can drive tenfold usage](https://realalpha.app/s/a-50-drop-in-inference-costs-can-drive-tenfold-usage/) (3 min) - [9:29](https://realalpha.app/v/1C_MErgk2Z0/?t=569) [AWS expects capacity shortages and stages its spending](https://realalpha.app/s/aws-expects-capacity-shortages-and-stages-its-spending/) (5 min) - [19:59](https://realalpha.app/v/1C_MErgk2Z0/?t=1199) [AWS says local benefits are key to data center support](https://realalpha.app/s/aws-says-local-benefits-are-key-to-data-center-support/) (5 min) - [28:22](https://realalpha.app/v/1C_MErgk2Z0/?t=1702) [Garman sees deserts as likelier data center sites than space](https://realalpha.app/s/garman-sees-deserts-as-likelier-data-center-sites-than-space/) (3 min) Skip the sponsor reads, the inference-provider and stack discussion, the Trainium sales explanation, the Nova reset, and the open-weights debate. ## Full analysis: Matt Garman doesn't think inference is even 1% of its eventual scale - Garman says he does not think inference is even 1% of its eventual scale. - His 50%-cheaper, tenfold-usage observation supports demand, not AWS's $1 trillion ambition. - Budget capacity against workload evidence, then test migration, serving costs and customer concentration. Its 7 sections, what it means for you and what to watch are on [the page](https://realalpha.app/v/1C_MErgk2Z0/#analysis), free with an account: - [2:24](https://realalpha.app/v/1C_MErgk2Z0/?t=144) AI-led migration gives AWS a growth path, not a forecast - [6:56](https://realalpha.app/v/1C_MErgk2Z0/?t=416) Agents make data access valuable, not migration automatic - [18:50](https://realalpha.app/v/1C_MErgk2Z0/?t=1130) Cheaper inference could expand usage faster than revenue - [14:34](https://realalpha.app/v/1C_MErgk2Z0/?t=874) AWS's broad workloads reduce some concentration risk - [12:49](https://realalpha.app/v/1C_MErgk2Z0/?t=769) Staged spending buys time but leaves supply exposed - [39:10](https://realalpha.app/v/1C_MErgk2Z0/?t=2350) AWS keeps compute for customers not yet formed - [43:09](https://realalpha.app/v/1C_MErgk2Z0/?t=2589) Proofs could widen agent use, but remain unproven ## Summary Garman has spent 20 years at AWS and is its CEO. He describes AWS as serving startups, enterprises and governments, while Amazon’s retail business has also tested AWS services at scale. Garman argues that AI is accelerating cloud migration because agents need access to companies’ core data and systems, reinforcing AWS’s growth. He sees inference as far below its eventual scale and says lower costs lead to much more usage, while compute-boom risks vary with providers’ financing and customer concentration. He expects capacity shortages for the next couple of years but says AWS stages investments and reserves compute for customers that have not yet arrived. He also says data center operators need to work with communities and that mathematical proofs could help keep AI agents within defined rules. ### Minute by minute - [0:00](https://realalpha.app/v/1C_MErgk2Z0/?t=0) **AI expands AWS’s growth opportunity.** Alex Heath asks about Andy Jassy’s projection that AWS could reach $1 trillion in annual revenue within five to ten years. Garman says AWS has seen a large opportunity for 20 years; he cites a $182 billion run rate, 39% growth and six straight quarters of accelerating year-over-year growth. He argues that AI agents are prompting companies to move core workloads into the cloud. - [5:48](https://realalpha.app/v/1C_MErgk2Z0/?t=348) **AI is speeding cloud migrations.** Garman says migrations that once happened slowly are speeding up because AI agents need access to data across a company’s systems, including legacy mainframes. He says agents can help uncover insights and new revenue, and that companies that leave workloads on-premises risk falling behind. AWS sees demand from large enterprises as well as startups, including two-person teams. - [9:29](https://realalpha.app/v/1C_MErgk2Z0/?t=569) **AWS is staging capacity investments.** Garman says AWS is developing cost controls that can cap spending or shift inference to a cheaper model. He says demand is outstripping supply and expects capacity shortages for the next couple of years. AWS commits early to long-lead items such as land and power, then waits to buy servers until it has better demand signals. - [14:07](https://realalpha.app/v/1C_MErgk2Z0/?t=847) **Garman sees compute risk as localized.** Garman argues that compute risk depends on how providers finance capacity and how concentrated their customers and workloads are. He contrasts highly debt-financed providers serving a few large training clusters with AWS’s broader customer base and says most AWS AI workloads are inference. He argues inference is distributed across applications and users, who could switch to another model rather than stop using it. - [16:59](https://realalpha.app/v/1C_MErgk2Z0/?t=1019) **Inference is still below its eventual scale.** Garman says inference is not even 1% of what it could eventually become, and argues that AI is already embedded in services people use without paying for it directly. He says every 50% reduction in inference cost has brought an order of magnitude more usage. He expects spending to fund both new products and efficiency gains. - [19:59](https://realalpha.app/v/1C_MErgk2Z0/?t=1199) **Data centers need community support.** Garman says operators need to work with local officials and residents, demonstrate community benefits and limit local impacts. He says AWS pays for the power it brings to the grid, aims to be water-positive by 2030 and says its largest data centers use less water than 40 U.S. households. He argues the industry should develop clearer programs for responsible community engagement. - [28:22](https://realalpha.app/v/1C_MErgk2Z0/?t=1702) **Deserts look more practical than space.** Garman says data centers in space are unlikely to be practical or economical in the next decade or two, citing launch limits, heat dissipation, radiation and serviceability. He also sees challenges with ocean-based sites, but says deserts look more promising because of solar resources and available space. He notes AWS is installing one GPU every 14 seconds. - [31:32](https://realalpha.app/v/1C_MErgk2Z0/?t=1892) **Inference providers add customer choice.** Garman describes inference providers such as Baseten and Fireworks as services built on cloud platforms that are mostly complementary to AWS, though they can also compete. He says they help customers use open-weight models and sees partners as an important part of AWS’s offering. AWS wants to compete across the stack, from chips and infrastructure to services and applications. - [36:49](https://realalpha.app/v/1C_MErgk2Z0/?t=2209) **Garman explains Trainium’s external sales.** Garman says customers want to use Trainium outside AWS, including in on-premises and sovereign AI environments, and AWS sees a business opportunity in selling the chips there. He says the company will keep investing in Trainium. - [38:02](https://realalpha.app/v/1C_MErgk2Z0/?t=2282) **AWS reserves compute for future customers.** Garman says AWS holds regular meetings to decide how to allocate scarce capacity. The company commits some capacity in advance but keeps some available for enterprises, startups that have not yet formed and Bedrock. He says AWS may offer a different delivery date or region when a customer’s first choice is unavailable. - [40:06](https://realalpha.app/v/1C_MErgk2Z0/?t=2406) **AWS still wants frontier models.** Garman says AWS remains committed to building a first-party frontier model and calls the changes to its Nova team a reset, not a retreat. He says AWS learned it needed the right talent, data and infrastructure in place, and that the team is continuing its work. - [41:38](https://realalpha.app/v/1C_MErgk2Z0/?t=2498) **Formal proofs could constrain AI agents.** Garman says customers need tools to keep agents within guardrails and protect data. He describes AWS’s automated-reasoning research, which uses mathematical proofs to check whether permissions and other rules hold. He says combining that approach with AI could help agents handle tasks, such as insurance claims, while following defined rules; he expects some deployments next year. - [46:31](https://realalpha.app/v/1C_MErgk2Z0/?t=2791) **Open-weight model growth remains uncertain.** Garman says frontier labs’ efforts to make distillation harder could slow the progress of open-weight models, though teams may also build them from first principles. He sees customer control and cost as reasons to use open weights, while noting that frontier labs can produce cheaper versions of their own models. He says AWS wants to support whichever approaches customers choose. - [49:12](https://realalpha.app/v/1C_MErgk2Z0/?t=2952) **AWS gains a demanding customer inside Amazon.** Garman says Amazon gives AWS room to pursue large customer problems and encourages teams to build new services. He describes Amazon’s retail website as a demanding internal customer that encountered scale issues before other customers and pushed AWS to improve. He also cites Aurora as an example of a cloud database the company built in response to customer needs. ## Transcript Sources, hosted by Alex Heath, with Matt Garman. Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there. [0:00](https://realalpha.app/v/1C_MErgk2Z0/?t=0) Matt, it's great to be with you. I feel like what's happening right now at AWS feels unprecedented in a lot of ways. So you're the world's biggest cloud computing company. You're the highest capex spender. You have an estimated, I think, like 44 gigawatts of capacity, which is that's coming under development in the U.S., which is more than Google and Meta combined. And then what just happened was Andy Jassy on the earnings call saying AWS could be a trillion dollar revenue business in the next five to 10 years, which is a huge number, obviously. If my math is correct, I mean, based on the run rate you're already on, that requires roughly 40% compound growth over five years or 18-ish over 10 years. [0:39](https://realalpha.app/v/1C_MErgk2Z0/?t=39) And this is off of a massive, massive base. I think a lot of people when they heard that were like, wow, like what are they seeing? And then I was also looking at the way you all have been talking about the business over the last couple quarters. And I think it was towards the beginning of this year, Andy was saying, we think roughly $600 billion by around 2036. And then in Q2, he floated the trillion. And then in Q3, it felt very strong. It felt like a very strong declarative moment. And I'm curious, maybe we can start here, what happened between Q2 and Q3? Yeah. Well, if I step back, You know, I think it's, we have probably for 20 years thought that the AWS opportunity is just a massive business opportunity and unlike one that we've seen before. [1:31](https://realalpha.app/v/1C_MErgk2Z0/?t=91) And we've been, so I've been with AWS for 20 years and we've been growing that whole time. And I think we, over time, have more and more and more gotten conviction that the business opportunity is just really, really, really large. And it's surprising, but at the point that we're at now, we're at $182 billion run rate business. And so you're over $180 billion and you're growing at 39%. There's not many companies that hit that spot. And as we've seen, we've actually increased our year over year growth rate each of the last six quarters. And so as we see that acceleration, what happens there is we've always had this view that the majority of workloads still live in on-premise worlds. [2:14](https://realalpha.app/v/1C_MErgk2Z0/?t=134) And as big as the cloud has gotten, there's still a lot of data centers that people have. There's a lot of workloads that live on-prem. And there are more workloads that are just being created every single day. And I would say over the last two years, really what we've gotten excited about is this flywheel where customers now see that there is a massive opportunity that AI affords them to gain efficiencies, to build new businesses, to create new customer experiences, to rethink how work gets done, to build faster for customers. And they don't get that benefit if their core workloads aren't in a cloud environment. [2:49](https://realalpha.app/v/1C_MErgk2Z0/?t=169) If they're running in an on-prem database, if they're running in an on-prem mainframe server, if they're in a VMware cluster in their legacy data center, they just have a hard time having agents talk to those pieces, pulling data from all those different data stores. It doesn't work. And so what we've actually seen is customers already saw benefits of the cloud. There's cost improvements. There's agility improvements. There's a reason why the AWS business has been growing great for the last 20 years, [3:15](https://realalpha.app/v/1C_MErgk2Z0/?t=195) **Matt Garman:** but it really accelerated when people realized they weren't going to get the benefit of AI if they didn't get those workloads in the cloud, those core workloads as part of it. [3:23](https://realalpha.app/v/1C_MErgk2Z0/?t=203) So we've actually seen our core business really accelerate over the last couple of years. And the more core workloads get in the cloud, the more they can actually take advantage of AI. And so there's this real positive flywheel effect where actually the revenue growth that we're seeing is significantly accelerating. And I think that's just further bolstered our confidence in the growth of AWS. I think the other things that we're seeing is just what's possible. I think a couple of years ago, you saw incredible content generation. You saw incredible conversational chatbots being made. And then as we moved into worlds of agents where work was really done, we're now just seeing customers really scratch the surface of what's possible, where you have a fully autonomous agent taking advantage of what we see are really powerful models out there that can run longer and longer and do harder and harder things. And they're really starting to think about and see how core parts of their workloads can be automated, improved, and built upon. And that opens up effectively all of the things that enterprises do. And it opens up the pie for everyone. It's not like you're taking from where they used to spend over A and bringing it to [4:33](https://realalpha.app/v/1C_MErgk2Z0/?t=273) It's actually creating new businesses, creating new revenue streams for our customers. And they're seeing cloud as a way to get to those places. And so we see that growth across tens of thousands of different startups, enterprises, large companies. And so when we see that growth where we see companies just really getting, scratching the surface of the benefit that they're getting, and we're seeing this flywheel effect, it gives us a lot of confidence that there's a huge amount of business going forward for us. for us. And, you know, it's far from certain. We've got to do a lot of execution. There's a lot of invention. There's a lot of things that we need to do between here and there. But the opportunity and the space for us to go after and help customers go and grow and take advantage of these technologies is bigger than it's ever been. This episode is brought to you by Granola, the AI notepad for people in back-to-back meetings. It works everywhere you do and lets you focus on what matters. Try it at granola.ai slash sources and use the code sources for three months off. This episode is also brought to you by Mercury, AI native banking that's loved by more than 300,000 entrepreneurs, including me. Visit mercury.com to learn more. [5:41](https://realalpha.app/v/1C_MErgk2Z0/?t=341) Mercury is a fintech, not a bank. Check the show notes for details. This episode is brought to you by Jira by Atlassian, where teams get the context, visibility, and control to move work forward. Try it free at jira.com. That's J-I-R-A dot com. AI is causing companies to come from, you know, on-prem to the cloud. That's right. That's driving a lot of this growth. Absolutely. Interesting. I hadn't thought about that. I mean, you've been saying and the company's been saying for so long, like the world, we underestimate how much of data is still on-prem. And for people who don't know, just means like it's not in a cloud. [6:15](https://realalpha.app/v/1C_MErgk2Z0/?t=375) It's somewhere local. What you all are seeing is people who have resisted the cloud are going towards it because of AI. We'll use this as just take something like an analytics workload, okay? An analytics workload, traditionally, a customer might say, great, I've got a data warehouse, it's running in my data center. I could move it to the cloud and take advantage of Amazon's analytics capabilities or move it to a Snowflake or a Databricks or something like that. That would be great. I could do that. And a lot of people did. But they're also like, yeah, but I'm still able to do the analytics here and it's working and maybe I won't move it or maybe I'll do it next year or two years when I can break the project work. And we saw those migrations happening, by the way. But they're slow and measured and scheduled out. Now we see people saying, okay, actually, I can ask an agent to go across all of my sources of data. And not just can I do my analytics faster or cheaper, but with new technologies that are available in a cloud world, say something like a DuckDB data engine that's working across all of my different data sources, I can actually have an agent go and do all of that. [7:17](https://realalpha.app/v/1C_MErgk2Z0/?t=437) And when I have an agent do that, it has to have all of the data sources accessible, they have to be available to the agent. It has to know where all the data is, but I actually get very new insights and actually can build new businesses, unlock new sources of revenue, new source of profitability when I can do that, except one of my source of data is locked up in the mainframe. And so it's just, it draws all of it in because they kind of can start to see where that benefit is. And I think there's a real risk that companies that don't make that move, they're getting passed by by the ones that are in the cloud. How much of the growth is big customers wanting to spend more versus the long tail? [7:54](https://realalpha.app/v/1C_MErgk2Z0/?t=474) Well, look, big customers are big, so they spend lots. So a lot of it is that. But I will say that the long tail is growing bigger. It really is across the board, actually. And as we think about the long tail, I think it's both, you know, it's not a lot of our growth is driven by like, you know, dentist's office, like not really kind of real, real small SMBs. Those are mostly served by a lot of our partners that actually run their software and services on top of AWS. But our customers, we have a really broad swath of whether it's, you know, global top 5,000 enterprises. Actually, just this week, I was in the Bay Area meeting a bunch of startups that many of them started less than six months ago. [8:35](https://realalpha.app/v/1C_MErgk2Z0/?t=515) And they're really thinking about how do they use AWS to accelerate their growth, accelerate their technology development, accelerate their route to market. And our customers love having access to that capability as well. And so it's across the board. Sometimes it's the very largest banks, the very largest healthcare companies, the very largest governments in the world. And sometimes it's a two-person startup that has an idea and just wants to build. And we're seeing growth from all of those segments. In fact, it's never been easier to build than it is today to be a two-person startup and go try something and actually get it to scale. And I do not think I've found a two-person startup that goes and tries to find data center space and rack a server to start get going, right? Like it just doesn't make any sense. [9:15](https://realalpha.app/v/1C_MErgk2Z0/?t=555) And so that's been true for a long time, but now they can use AI to really further amplify the impact that they're going to have and how fast that they can grow. And we're seeing those new companies scale really rapidly when they get that product market fit. And AWS and the cloud gives them a chance to cycle until they get that product fit. Something that does feel new though about this moment, I agree with you, but when you scale right now, especially if you're AI heavy and you're doing a lot of inference, you can, I think it's Linux fireworks, but I've heard the concept of like, you can scale into bankruptcy. [9:47](https://realalpha.app/v/1C_MErgk2Z0/?t=587) There's a sense of like, you know, you can't scale because of the compute, which you have most of. So I'm curious how you think about that and how you advise, you know, companies and startups you meet with that are facing that issue. Because I think it's a really real issue. It is a very real issue. and actually we're thinking deeply and actually investing in more robust cost management tools for companies so that they can do exactly that. Customers want tools that on the range from, you know, shut my account down when it hits this dollar amount. Like, I don't care. It's okay. Like, I'll figure it out, but I don't want to spend more than this to alert me when I hit this dollar amount to, if it's an inference, maybe ratchet my usage from an expensive model to a cheaper model And we're trying to think about what are all of the tools and controls that we can give customers so that they can actually feel confident about that cost management. [10:40](https://realalpha.app/v/1C_MErgk2Z0/?t=640) Even larger customers, by the way, it's not necessarily the small customers that are worried about spending $10,000 and not being able to afford it. But even large companies are saying, you know what, I may want this particular person to be able to spend largely uncontrolled. But, you know, Alex, on your personal account, you know, it's $100 and then I shut it down or whatever it is. And so really giving those fine-grained controls so that companies of all sizes have that, I think, is incredibly important. I think from a capacity point of view, it is a real concern. And it's part of why we're spending so much on CapEx and investing in this infrastructure. [11:16](https://realalpha.app/v/1C_MErgk2Z0/?t=676) We're of this fortunate position. I think a lot of people are worried about things like concentration risk when you're spending capital and things like that. for us, we spread that usage across, you know, we have tens of hundreds of thousands, millions of customers that are all using AWS. And so when I see all of, you know, a lot of that comes from a broad swath of companies, that's where we feel confident that we can go and put down a lot of capital and really grow the amount of compute that's available for customers, because we get really good signals as to where they're all growing, what it looks like, what demand is going to look like next year, what it's going to look like the year after that. And we have good signals and many of our larger customers will commit to five, sometimes even eight, nine, 10 year commitments on their spend with us. But most of them are in the kind of, you know, three, four or five, six year commitments, but that gives us really good visibility as to what their growth is going to look like. And oftentimes those customers, frankly, will outspend their commitments, but it's more a baseline, but it gives us a good sense of where that growth is going to be. [12:11](https://realalpha.app/v/1C_MErgk2Z0/?t=731) And unfortunately, you know, well, fortunately or unfortunately, but right now the demand is far out stripping the supply. And so even spending as much capex as we are investing as much as we [12:20](https://realalpha.app/v/1C_MErgk2Z0/?t=740) **Matt Garman:** are ahead in capacity, I anticipate that we'll be short of capacity for the next couple of years. [12:26](https://realalpha.app/v/1C_MErgk2Z0/?t=746) Can you see into 2030? Some, you know, I think you can see trends. I can't personally, otherwise, you know, you're an alchemist. Yeah. I'll do some sports betting or something, but we get signals, right? And there's, you know, how we think about it. Actually, it a little bit goes back to the kind of the supply chain roots of Amazon, where we really think about how do we manage our full supply chain? And then at AWS, we really think about how do I spend money at the last possible minute so that I have as much possible information as I have while giving myself option value to hit the upside, right? Because I'd like to, the optimal is I would have exactly as much capacity as I need. [13:05](https://realalpha.app/v/1C_MErgk2Z0/?t=785) You're never going to be perfect. You don't want to overspend and you don't want to wildly underspend in that capex. And so what we do is you think about what are those long lead time components that you have to go spend on? You have to buy land, you have to acquire power contracts, you have to, you know, get permitting, things like that. Then as you get closer to like landing a data center, you might have to buy networking equipment, you might have to buy generators, you might have to buy some of those other pieces. So if you're going all the way up to 2030, I might be buying lands today that I plan to use in 2030. We're not buying servers, you know, we're not buying all of those things. And so I can wait for a lot of the capex expenses that I would need to support my 2030 proposed growth until much closer to 2030. And so I don't have to have perfect views into 2030. You can have directions on where they go. And if we're wrong, probably resell land. And it's, you know, worst case you lost a little bit on appreciation or not, but mostly it's a durable asset that you can resell. And so we think about our investments kind of staged that way. And it takes some of the risk out of there as we think about it, [14:04](https://realalpha.app/v/1C_MErgk2Z0/?t=844) but it requires a lot of long-term planning and thinking and watching kind of the various variables that you have in there. Everyone in the markets feels like is looking for reasons to doubt the compute trade, and you've seen it with CoreWeave, you've seen it with the neoclouds in particular and the debt markets. I'd be curious to hear what you make of the moment wherein you had, I think, kind of alluded to it earlier. But you all are in a different spot, obviously, but I do think there's systemic risk a little bit that everyone's trying to figure out, like, how bad is it? Yeah. I don't think it's systemic risk, actually. I do think that the risk is localized because the bets that people are making are materially different. And everybody has to go kind of make their own bets. I think when you've completely debt financed at a relatively high rate, you have a much smaller error bars as to like when your cash flows are going to land. You also have much smaller error bars when you have high concentration of customers and concentration of workloads. [15:02](https://realalpha.app/v/1C_MErgk2Z0/?t=902) And so if you go and look at the workloads that are in many of the NeoClouds, they're really large training clusters for the most part. [15:10](https://realalpha.app/v/1C_MErgk2Z0/?t=910) **Matt Garman:** And those really large training clusters are largely being bought by a small number of customers. [15:16](https://realalpha.app/v/1C_MErgk2Z0/?t=916) And they're all, you know, debt financed at relatively high rates. That is not the business that we're in. We, you know, we both have data centers and computers, but other than that, like the business is actually quite different. Most of our business is spread across the millions of enterprises and startups and small, medium-sized businesses and governments that are running on top of us. And much of that business is core compute. It's compute clusters. It's HPC workloads. It's websites. It's backup storage. It's databases. It's, you know, a lot of that kind of workload. And then the AI workload, the majority of our AI workloads are inference today. [15:52](https://realalpha.app/v/1C_MErgk2Z0/?t=952) Yeah. And so that inference is effectively just, if you really look at inference today, it is just another building block as part of an application, right? Your applications all just have AI embedded in them. It might be AI that's helping somebody serve better ads. It might be AI that's helping you, you know, auto-suggest something in your email. It's, you know, there's a whole bunch of things that it might be doing. It might be AI that's helping traders figure out, like, what their, you know, like, stock market trade is going to be. But it's just part of the application that they're using. And so that is a very distributed workload. [16:23](https://realalpha.app/v/1C_MErgk2Z0/?t=983) And so even if the usage is concentrated on Anthropic and OpenAI and a handful of model providers, the actual end users are thousands and thousands and thousands and thousands of folks out there running inference for us. And so the systemic risk of like one or two companies not doing well, even if it's one of those model providers, it'll fall over to another. Like people aren't going to stop doing inference. They'll find a different model to use. One might be an OpenOids model. It might be open. We're going to have computers. You can put it on those computers. And so for us, that risk is very different. [16:51](https://realalpha.app/v/1C_MErgk2Z0/?t=1011) You all have very large investments in OpenAI and Anthropic, and they're huge partners, obviously. We want them to do well. You want them to do well, of course. I think everyone's with like on pins and needles about Anthropic IPO. But I'm curious about the training versus inference workloads you're seeing, because you have such great visibility. Do you expect training to be a smaller piece of the pie going forward? And it feels early on inference, but I think everyone's trying to figure out how big is inference going to actually be? I mean, you see stats like 2% of Americans have even paid for AI or, you know, companies are barely using it. [17:24](https://realalpha.app/v/1C_MErgk2Z0/?t=1044) And yet inference is exploding. It's like, where does that tap out? And then what does that mean for the training piece of the pie? Yeah. I think, by the way, that stat just seems a little bit misleading. It may be the 2% of the people have paid directly for model usage. But I bet a higher percentage of people like have a Netflix account. Sure. And it recommends a movie to them. Sure. So they actually have used it. So it's like, it is like it's yes and no. Like, it's like how many, what percentage of, you know, Americans have paid for a database? Like probably a really small percentage, but I bet they all like use it on a daily basis anytime they log in somewhere. And so, you know, I think that's a misleading stat into how integrated it is into every single application. People are going to the Amazon website every single day. And when they search for a product, AI is kind of embedded in how they're searching. [18:07](https://realalpha.app/v/1C_MErgk2Z0/?t=1087) Sure. They don't pay for it, but they kind of paid for it. And so from that perspective, I think I think inference is a huge business. It's massive. And you've seen in some of the anthropic filings kind of what their revenue looks like. So it's a very good and large business. I think training is a good business too. It's just very concentrated. And so we'll continue to support training clusters. We do support training clusters today. But it's the minority of our business for the most part just because the main source of our business we want to do is support the millions of customers we're building and growing on top of us. [18:38](https://realalpha.app/v/1C_MErgk2Z0/?t=1118) How early is inference, do you think? Super early. I think you're almost doing a disservice to say that it's at the 1% of what it's going to be. Not even 1%. I don't think it's even 1%. I think there is a massive runway. [18:53](https://realalpha.app/v/1C_MErgk2Z0/?t=1133) **Matt Garman:** And look, every time we see inference get 50% cheaper, we see an order of magnitude more usage. [19:00](https://realalpha.app/v/1C_MErgk2Z0/?t=1140) And so people kind of wring their hands. They're like, look how cheap it's getting. And yet the revenue is higher every single year. I think that we're still very early on that path. and it's why we also feel good about the investment we're making with regards to infrastructure, because I just think there is so much more that can and will be done. Do you think about that spend that's driving that inference? Is it coming out of existing budgets, or is it net new? Like, where is that capital? I mean, if you're talking like potentially trillions of dollars in inference that's happening, I'm just like, where is that coming from? [19:29](https://realalpha.app/v/1C_MErgk2Z0/?t=1169) Yeah. I think it's all those things. I think that sometimes it's creating new value. Sometimes it's coming out of other budgets. And I do think that there's no one answer to that. I think it's a lot of those things. Sometimes it's allowing you to create products and services and capabilities much faster than you could before, which drives revenue. We'll call it like new sources of revenue. Sometimes it's driving efficiencies where it's supplementing. I don't need to necessarily hire a new person. I can get twice as much work done with the person that I, the thing that I have. And I think both of those things are true. [20:03](https://realalpha.app/v/1C_MErgk2Z0/?t=1203) What do you make of where the country is on data centers, especially given the midterms and what happened in Texas where governor campaigned on pausing doled outs, one. And I know you guys have your approach where you do a lot of renewable energy, you're very conservative with water, but at the same time, I just think it's such a lightning rod issue. And it seems like it's going to get worse before it gets better, maybe? Yeah, I think, look, there's a couple of things here. And I think it's incumbent on us to help educate folks as to that all data center producers are not created equal. And that's true for lots of industries, but today it's largely painted with one brush. And I think that there are a handful of us in the industry that spend a lot of time thinking about how do we make sure that we are great citizens and great community members as part of this. And we learned this early on with some of our fulfillment centers and other things like that. And we learned with data centers, we need to work with local politicians. We need to work with community members. [20:59](https://realalpha.app/v/1C_MErgk2Z0/?t=1259) We need to show that we're bringing high paying jobs in communities. We need to make sure that we have low impact on the communities that we operate in. [21:07](https://realalpha.app/v/1C_MErgk2Z0/?t=1267) **Matt Garman:** Our data centers, we bring power and pay for all the power we bring to the grid. [21:13](https://realalpha.app/v/1C_MErgk2Z0/?t=1273) We oftentimes, in many of the communities we work in, actually reduce the tax load by thousands of dollars per person in the communities that we operate in. As you think about the environmental impact, we continue where the single largest creator of new carbon zero energy sources over the last five years in the world. and we continue to bring more carbon zero energy to the grid every single year, huge investments on that front. We were very early in thinking about how we use about water. Actually, we have a commitment that by 2030 will be water positive, where we bring more water to every community that we operate in than we use. [21:48](https://realalpha.app/v/1C_MErgk2Z0/?t=1308) And even our data centers, our largest data centers today use less water than 40 US households. It's like a block, like a block of households. It's actually a relatively small amount of water. I saw a stat that like golf courses use more water. By far. My golf friends don't like that I use that stat. But yes. But yeah, I mean, there's actually, I mean, there are a lot of things out there that use way more water than our data centers. There are ways to run data centers that use lots of water, so we don't do that. And not everybody takes that path that we do. And I think that it's brought a bad name to people that proudly stand up and say, you know, I go against regulations. [22:27](https://realalpha.app/v/1C_MErgk2Z0/?t=1347) I don't even listen. I don't get permits. I just build data centers anyway. They kind of like proudly go against working with the community. That is just not how we do it. Unfortunately, it's kind of brought this bad name to data centers. And I think it's something we're going to have to work through. But I also think that many of the folks, and even you mentioned the midterms, we work quite closely with many of the folks that are both Democrats and Republicans. Like, and we have, and we will continue to do that. We've done that for five different administrations. And they appreciate it. Like, they will work with them. [23:00](https://realalpha.app/v/1C_MErgk2Z0/?t=1380) We don't always agree on all topics, but we work their debate there and figure out how there's win-win scenarios with those things. And we'll continue to do that. So I feel very good about where we are. And I'm excited. And congratulations to everybody who won in the elections. And we feel good about what we've brought to communities and where we'll continue to go. I think technology is so important to our economy today that we can't just turn it off and decide to not have it. I don't think that that would be good for our country. Some people want that. they want that they think they do they think they do but then if you ask them like you're going to give up your netflix subscription they're like well no not that and be like okay are you going to stop like instagramming your friends like well okay not that you're going to stop asking amazon to deliver stuff to you no okay not that either but those things all require data centers like it's like it's like there's a weird disconnect there and so a lot of the things that that that um people want i think is i think they want responsible they don't want the kind of technology leaks to tell them what to do but they do like it when people build good products for [23:59](https://realalpha.app/v/1C_MErgk2Z0/?t=1439) that they want to use. And our view is that there is a great way to do both of those things. You don't have to do that. So when you woke up the next day after the results and you saw everything, was there anything you thought about like, man, what do we do? Like, what do we do as an industry? I mean, you've talked about what Amazon does, but like, does the industry as a whole need to do more? Is there anything the industry can do? I do think, and we've been working with several of the governors in the states that we operate at and thinking about how do we help codify some of the stuff that we actually do and some of the things that they want to do into more structured programs that everybody has to follow. And I think that's one of the things that we need to do to both clearly highlight why it's good to have a data center in your community, because it actually brings a tax base, it brings high paying jobs, and why it's important for the country. Because the alternative is technology will still progress. We would go there, you know, it would happen somewhere else. And I think that would be bad for the security of our country, would be bad for the economy of our country, but the world is not going to stop. And so, [25:03](https://realalpha.app/v/1C_MErgk2Z0/?t=1503) kind of thinking about how do you balance what's great for communities, great for people, great for our country, and good and reasonable for companies, so that they're still incentivized to go work on these problems. And I am very convinced that there is a path there. Framer is the AI website builder that powers the Sources podcast website at podcast.sources.news. Framer brings AI agents into the same canvas where your website is designed, managed, and published so you can move faster without giving up your taste or control. I use Framer to make the Sources podcast website be the destination for everywhere you can find the show, plus you can also see recent issues of my newsletter. [25:44](https://realalpha.app/v/1C_MErgk2Z0/?t=1544) Start building with agents for free today at framer.com slash sources for 30% off a Framer Pro annual plan. Framer.com slash sources. Rules and restrictions may apply. I spend a lot of time context switching between meetings, often with no time to process one before the next starts. Thankfully, Granola runs in the background the whole time. It's an easy-to-use AI notepad for meetings that works everywhere, even on phone calls. I use Granola to recall what was said in meetings and create helpful summaries. I use it every day to stay on top of what I need to get done with my team. It connects to my email and suggests follow-ups for me to quickly review and send, saving me valuable time. [26:22](https://realalpha.app/v/1C_MErgk2Z0/?t=1582) Granola isn't just a core part of my workflow. It's basically my second brain. Try granola at granola.ai slash sources and use the promo code sources for three months off. Mercury is a modern take on banking built for startups like mine. When I decided to start my media business, Mercury was by far the most straightforward, full-featured banking solution for me to set up quickly. The interface is intuitive and simple, saving me valuable time every day. I use Mercury to track my spending, bills, and invoicing. I love that I can delegate permissions to my team so they can keep things running for me in exactly the way I want them to. My favorite part is how forward-looking Mercury is with AI. Legacy banks are stuck in the past, but Mercury is built for how modern software works today. I use its built-in command assistant to analyze cash flow and help me move money. And Mercury also connects to other AI tools like ChatGPT and Cloud. I use this feature all the time and the folks at Mercury actually let me know that I'm one of the top users of it. [27:20](https://realalpha.app/v/1C_MErgk2Z0/?t=1640) So trust me, it's finally easy to get real-time financial data about your business wherever you need it. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not a FDIC-insured bank, banking services provided through Choice Financial Group and Column N.A., members FDIC. As you move faster and bring in more AI agents, keeping tabs on what's actually happening gets harder. Who's working on that? What did that agent just change? Are we still on track for our goals? That's why teams rely on JIRA for visibility across their work, whether it's done by people or agents. [27:54](https://realalpha.app/v/1C_MErgk2Z0/?t=1674) JIRA is powered by Atlassian's Teamwork Graph, which connects work across JIRA, your most used tools and apps, and your agents. That means your agents share the same context your team has, like the goal behind the work, the decision made yesterday, and the project plan, so they work faster and more accurately. You get oversight, track agent progress right alongside human work, spot blockers sooner, and connect your work to goals. You don't have to choose between shipping fast and staying in control. I want to get your reaction to like three areas that people are talking about putting data centers right now. So, there's space, ocean, and desert. People are excited about these to varying degrees. [28:32](https://realalpha.app/v/1C_MErgk2Z0/?t=1712) Maybe we can start with space. What do you think about data centers in space? [28:36](https://realalpha.app/v/1C_MErgk2Z0/?t=1716) **Matt Garman:** I think that it is highly unlikely to be economically feasible or practical in the next decade or two. [28:44](https://realalpha.app/v/1C_MErgk2Z0/?t=1724) Decade or two? Yeah. Okay. So, that's a no on space then. I mean, here's the thing is, it sounds good. You get free energy and it's up there somewhere. there. That's great. Number one, launches are still relatively infrequent and expensive. You're always trying to fix that, but yes. Okay. Yeah. But to your point. Still waiting. Yeah. Like it's a great, I appreciate it. But one person's not going to fix it. And still, I don't know, there's, we as a world don't do one launch a day today, the whole world. And the data center, we're landing in a data center today, one GPU every 14 seconds. Wow. So we don't do a launch a day for the world. And that's just us doing one GPU. [29:28](https://realalpha.app/v/1C_MErgk2Z0/?t=1768) You know, there's obviously like, so let's say one launch. And by the way, a rack of GPUs, you know, might weigh a ton. So I don't even know if you can stick that on a launch vehicle or how many you can get up there. You know, people talk about gigawatt data centers and megawatts of racks and whatever, like most solar panels and things drive kilowatts, not gigawatts. So you need these gigantic things that are up there. They're still, they're really hot, And everyone's like, space is cold. Space also doesn't have air. Actually, dissipating heat in space is actually quite hard. And so that's actually, it's just a really hard environment. [29:59](https://realalpha.app/v/1C_MErgk2Z0/?t=1799) There's radiation everywhere, so you have to build things. It's going to be way more expensive to build servers. From a serviceability point of view, by the way, these are all technically solvable problems. Yeah, it's going to take a while. And it's just going to take a while. Yeah. Like, I'm not saying any one of these things that I can highlight are not impossible. Yeah. And I just think that it's way far out. OK. Oceans, though? Oceans, there's some of those same problems. I think you get cooling, which is good. You have things floating in the water and moving. It's a little harder. So I find oceans to be also probably impractical. [30:32](https://realalpha.app/v/1C_MErgk2Z0/?t=1832) I think deserts of those three that you mentioned are by far the most likely. You have relatively good environmental areas. You actually have heat sinks. You have solar areas that you can do it. I think deserts are very interesting and potentially. Also, when you go to liquid cooling, actually cooling the data center becomes actually a little bit of an easier problem in some dimensions with regards to energy usage at least. So I think ocean or deserts could be a interesting place particularly because, yeah, like you can also like lay a cable. Like it turns out that like actually the data transfer and stuff is pretty useful. And there's no people there, which may be good for the climate we're in. Well, yeah, I mean, look, yes, although then there's no one to benefit from kind of an increased tax base or anything like that too. [31:20](https://realalpha.app/v/1C_MErgk2Z0/?t=1880) So like there are, it turns out people like living in towns. They like where they have high paying jobs. Like there's a bunch of things that like having those things nearby, it just means the high paying jobs aren't going to be in your town. The tax base is not going to be in your town. Like there's a lot of those things that are not great. Yeah. That people kind of say like, Ooh, actually that do want that. I'm really curious what you make of all of these inference providers that are popping up. So, Base 10, Fireworks, Modal, FAL, the list goes on. And they're servicing a very important growing part of, I guess you could call it cloud stack. [31:54](https://realalpha.app/v/1C_MErgk2Z0/?t=1914) Very AI native, very model heavy tuning. And then there's a whole post-training inference world that's developing. When you look at that, and you all obviously have that as well, what do you make of that? In your mind, is that a new kind of cloud market that's forming or is it just a moment in time? Like how do you think about that? I don't really view it a cloud offering. I think it's more like a kind of application or services that are built on top of the cloud that are both complimentary and in some ways competitive, but mostly complimentary to what we built. And so think about in the database side, like MongoDB building on top of AWS, or think in the analytics side of like a Snowflake or a Databricks building on top of AWS, Um, uh, inference providers building, you know, on top of AWS, you know, that, that is net positive to the customers. [32:43](https://realalpha.app/v/1C_MErgk2Z0/?t=1963) They want it in their, they want it in AWS. They want to use that inference with the rest of their AWS stack. It's, it's complimentary. Um, it may be, you know, on some dimensions competitive, but, but, but we've operated in that world for a really long time where many of our products both compete and support customers. And we've built AWS from the very beginning, knowing that partners building on top of us are going to be an incredibly important part of what we offer to our end customers and have always been a huge part of what we go and build. And our view is if partners are successful building on top of us, then we're going to be successful. [33:17](https://realalpha.app/v/1C_MErgk2Z0/?t=1997) And it gives customers more choice. I kind of put them more in that bucket. I think they do a great job. They do a great job of helping customers use open-ways models, which are oftentimes hard to use, hard to optimize. You have to be kind of an expert. They do a great job of helping with that, where customers think they may have interesting data or interesting ways to use open source or open weights models, either to make them cheaper or different capabilities. And I think those folks all, and they all have different kind of slices as to how they help, but they help customers and we like working with them. [33:52](https://realalpha.app/v/1C_MErgk2Z0/?t=2032) They're a great part of our partner ecosystem. And I guess part of what I'm getting at there is how much of the AWS advantage do you think about being the software layer versus the bare metal? And especially in a world where an agent can write the integration and recreate things, and essentially, like, you could really see a world in the near future where agents are picking clouds and doing everything on behalf of their companies. And in that world, like, what level of the stack do you need to own? Have you been thinking about that? Yeah, look, we think it's important for us to play in all of the layers of the stack, just to be super clear. [34:25](https://realalpha.app/v/1C_MErgk2Z0/?t=2065) And so I think all of those are incredibly important for us. I think at the very lowest layers of the stack, there's just not that many, you know, we often think there's like, there's three basically right now. And I don't know that that probably will continue to be roughly the same number. There's economies of scale there. Yeah. And then as you kind of move up, I think it's a funnel that kind of keeps going that way. And then you get to end user applications where there's thousands and thousands, there's bigger and smaller ones, but there's lots of those up there. And we like to play at all of those layers of the stack. [34:50](https://realalpha.app/v/1C_MErgk2Z0/?t=2090) We like to play at the infrastructure layer. We like to make our own chips. make our own networking gear, like all of those kind of things at the very lowest layers. We have services, we have managed databases, we have managed storage offerings, we have managed analytics services. And then with the top layer, we have applications as well, where we have applications for customer service with Connect, we have applications for worker productivity with Quick and applications for developers with Kuro, and we try to compete at all those layers of the stack. And the inference, and as you go up, there are more people that also build on the lower layers of the stack that we have and then compete at that layer. [35:25](https://realalpha.app/v/1C_MErgk2Z0/?t=2125) A lot of these inference players are buying their own chips. And so they're more in this middle layer, right? Where they might buy from AWS and build a service on top of that and run that. I think at the lowest level, you know, we kind of don't really partner with many people other than, you know, hardware providers and things like that. Because, you know, we're kind of the base layer of that thing. But then as you go up, it's more a competition environment where it's great. Like we encourage that. And so I think it's important for us to be at all of those layers of the stack. I think we want to help customers run inference directly on us. [35:52](https://realalpha.app/v/1C_MErgk2Z0/?t=2152) We want to, and for other folks that are building inference engines, whether it's OpenAI, whether it's Anthropic, whether it's Fireworks or Base 10 Optimize and OpenWeights model, whether it's the OpenWeights models directly, whether it's our own model, whether it's, you know, all of those different things, we want to make sure those are all capable, you know, available to our customers. and you know like in the fullness of time we're not there yet but we'd love to have all of those available in bedrock um it's it's not technology-wise able to do that yet but we'd love to make them all available if they want to be there um otherwise if they don't want to be there that's okay too but we want to help it make it as easy as possible for our customers to use um whatever um price performance capability makes the most sense for their application because even if we don't win the upper layer stack our view is if it's built on aws will win but we think we can do a really good job of offering kind of a bunch of those capabilities because we can control more of those pieces. [36:48](https://realalpha.app/v/1C_MErgk2Z0/?t=2208) And we think, you know, we have good teams to build services that are valuable to our customers. Speaking of that lower part of the stack, I'm very interested in your all's decision to start selling Trainium chips externally. I haven't really heard you talk about that. Why are you doing that? I mean, Google's starting to sell TPUs externally. Is this just like pricing power and like, you know, keeping a market that's kind of like revolving around you? Like what's the reason for doing that? Well, look, we've made a huge investment in Trainium and it's gone really well. Like it's, you know, and people tell us that they want to be able to use Trainium in more places. [37:21](https://realalpha.app/v/1C_MErgk2Z0/?t=2241) And, um, but why not make them have to use it in AWS? There are only so many, like, we're not going to build every single data center that exists in the world. Sometimes they're going to be on-prem, whether it's in a country somewhere, whether it's inside of a government somewhere, whether it's, um, you know, there's like a sovereign AI component. It could be, you know, there's, there's lots of places. And if you look at like everywhere that NVIDIA sells chips today. They sell a lot of their chips to, you know, us and Google and Microsoft, and they sell a lot of their chips to other places. And like I said, I think only, you know, 85% of workloads are still on-prem. [37:52](https://realalpha.app/v/1C_MErgk2Z0/?t=2272) There's still a lot of on-prem locations. And customers have told us like, hey, we'd love to run a lot of our workloads inside of the cloud on Trainium, but we may want some of it in our own data center. And we think there's a really good business there and we're excited about it. And so it's one that we will be investing in and doing more and more over time. How do you decide compute allocation across the fleet now when you've got to deal with startups, the guys in the garage who want to build something, massive training clusters with huge equity partnerships you have with large labs? How do you balance that? [38:26](https://realalpha.app/v/1C_MErgk2Z0/?t=2306) It's hard. We spend a lot of time doing it. I wish I had some magic- Do you have daily allocation meetings? It doesn't need to be daily, but we have regular allocation meetings where we actively debate how we think about where we can. I'd love to say yes to everyone, but yes, there's a finite amount of capacity. And as I said, we're adding a ton every day. So oftentimes we're able to say yes. Sometimes we have to go back to customers and say like, I know you want it in two weeks, but it's going to be in four weeks. Or I know that you want it in the East Coast, but it's going to be in the West Coast. [38:59](https://realalpha.app/v/1C_MErgk2Z0/?t=2339) Or it's going to be internationally or it's going to be in India or wherever you have capability or wherever we have capacity. And sometimes they'll say, yes, that works for me. And sometimes no. And then you kind of go back and iterate on some of those. We're very intentional about making sure that we have a diverse customer base. And so I don't want to sell out 100% of my capacity to just the large labs or the large technology companies. I want to be able to say yes if an enterprise comes along and says, hey, I need 10 GPUs. And I think that's part of the AWS value proposition. And so we very intentionally set aside, same for startups, by the way. [39:31](https://realalpha.app/v/1C_MErgk2Z0/?t=2371) There's going to be some startup. I could choose to sell out every single accelerator that I have that I know is landing in 2027 and just be done with it. We choose not to do that for exactly these reasons. We want to make sure that I can say yes to a bunch of enterprises that come in. [39:46](https://realalpha.app/v/1C_MErgk2Z0/?t=2386) **Matt Garman:** We want to be able to say yes to a new startup that hasn't even formed yet that we'll have capacity for. [39:53](https://realalpha.app/v/1C_MErgk2Z0/?t=2393) And so we do, we kind of have a portfolio where we, you know, we do commit a good chunk of our capacity well ahead and commit it for long term. And we keep some behind so that we can make sure that we can say yes to a bunch of those folks. And then we, of course, save a bunch of capacity for Bedrock so that we can support as much of that as possible. Last time you and I talked, we talked a little bit about Nova. This was earlier in the year. And you were all's frontier model ambitions. And you were very clear, like, no, we need to be at the frontier. I want us to be at the frontier. And then since then, there's been some changes. [40:23](https://realalpha.app/v/1C_MErgk2Z0/?t=2423) some, I can't, I haven't been able to quite tell. I wanted to ask you like what's going on. Cause there were some layoffs and Nova models being sunsetted or put on ice or something. And I'm curious, like, is this, you all deciding that no, like we don't need to have models at the frontier in house or what's the, what's the strategy? No, far from it. We're, we're just as invested in that strategy as we've ever been. And I would just say, you know, stay tuned. We're quite excited about it. It was more of a reset. It was more of a reset. That's right. That's right. I think we identified some opportunities to learn from some of the approaches we had taken before. [40:53](https://realalpha.app/v/1C_MErgk2Z0/?t=2453) But it is just as big of an investment, just as important from our long-term strategy perspective. We think that customers are going to want a first-party frontier model from AWS, and we're very invested in going in that direction. Anything on what you learned in terms of what's kind of driving this reset that you can be more specific on? No, I think we just, you know, I think we did a reset and make sure that we had kind of the right talent in place, some of the learnings with regards to making sure that we had kind of some of the right data pieces in place and the right infrastructure in place and learning a little bit about how to do some of those things at scale. [41:31](https://realalpha.app/v/1C_MErgk2Z0/?t=2491) And I think we've made some good learnings there and the team is cranking away and doing really well right now. How are you feeling about the safety piece of agents right now? And, you know, what customers are saying about, I'm worried about stuff going rogue. And then also just companies, you know, banks are sending out emails to customers saying, you know, don't let your Muse, you know, log in, blah, blah, blah. Like there's this, you all blocked Muse actually. I mean, there's this crazy moment where these agents are starting to traverse things and companies are going, what do I do? What have your conversations been like on that? [42:02](https://realalpha.app/v/1C_MErgk2Z0/?t=2522) Yeah. Internally with the team and customers. Yeah, I think those are two different things. I think one is when we talk to our customers that are building agents, I think that very clearly we have to help them with some more tools to help on building agent safety and ways to manage and control agents. And I think at some point, you know, agents and AI models are just tools to get work done. And we've got to do a good job of helping our customers make it easy for those tools to stay within the guardrails and do the work that they want it to do without, you know, too sharp of edges so that they, you know, they don't cause harm to their companies or leak data or have some sort of, you know, big kind of security disaster. [42:46](https://realalpha.app/v/1C_MErgk2Z0/?t=2566) And so that's on us to co-invent what those look like. I actually think one of the big technology unlocks that I feel really excited about is it's largely called neurosymbolic AI, but effectively it's this field where actually Amazon has been probably the single biggest investment that we make internally in research is around what we call automated reasoning. [43:09](https://realalpha.app/v/1C_MErgk2Z0/?t=2589) **Matt Garman:** And it's applying mathematical proofs to computer science problems to prove things to be true. [43:19](https://realalpha.app/v/1C_MErgk2Z0/?t=2599) And so a tangible example of that is when you go and create a new permission, an IAM permissions model in AWS, you run it through IAM Access Analyzer and it basically looks at the whole possibility of all of the permissions that you have inside of your org and it can tell you, is this gonna conflict with some other permission, right? Like this one says you should have access, this one doesn't and it tells you those conflict, you better go figure that out or we won't accept it. And so if you just brute force try to go figure those things out, it's like more computers than the earth has to figure out that calculation. But through mathematical proof, you can actually do it relatively fast and get to 100% proof that these things- Verifiability. [43:57](https://realalpha.app/v/1C_MErgk2Z0/?t=2637) That's right. X plus Y equals Y plus X. If you just brute force try to prove that, you of course can't because there's an infinite number of combinations of that thing, but math can prove it in like three steps or whatever it is. And so if you can take that and you can basically say, I want a non-deterministic agent that's going to be able to go out and try to process an insurance claim, right? I want it to go figure things out. Like it's hard. It's got to go out, look at some data sources and try different things. But I want to make sure that it never breaks this rule or this rule or this rule or follows the policies that I have. If you could have an automated reasoning agent that basically helps steer and build guide rails to your your kind of core AI agent that basically could deterministically prove that it would never break invariance. I think you could actually build a set of tools there by combining automated reasoning with traditional kind of AI systems that could give provably correct agents that would kind of follow rules. And so that is an area that we're investing a lot in and it's showing a ton of promise and it's still early, but tools like that can be really powerful. And I think [45:07](https://realalpha.app/v/1C_MErgk2Z0/?t=2707) it's one of the things that could unlock where you can imagine an insurance company saying, wow, I would love to just in a fully automated way, let my agent process insurance claims. Now it's like people could submit insurance claims. They get, you get an answer back in 10 seconds. Like it would be game changing as opposed to like wait a month to figure out if, you know, you get a claim from your insurance agent because they'd be comfortable sending those things off in a fully automated way. It's where, you know, actually the experience gets better for customers. There's probably more business that they can do, but today they're not comfortable because you actually have to have a human in the loop. That's one of the areas where I think if we can really unblock that, and it's one of the areas we're spending a lot of time thinking about, it has game-changing. And it's one of those places where if you do unlock that, the usage suddenly goes through the roof even more. And it's why I think we're just at those early stages of what's possible What do you think about the timeline of deployment on that? Depends. I think there'll be some next year. Okay. And we're going to be keep iterating it and growing it over time. So this is like the [46:02](https://realalpha.app/v/1C_MErgk2Z0/?t=2762) next big bet, it sounds like. It's one of them. Yeah. It's one of the ways that I think you can add determinism to, without taking away the benefits of the non-deterministic AI models that we have. And that could be the answer, yeah, to a lot of these safety issues that people have. That's right. That's right. You can imagine a, you know, you can imagine if you could, if you could define a rule that tells an agent, you will never try to escalate your privileges. Yeah. All of a sudden, it doesn't matter if you have like a magic box that it could escalate its privileges, it won't try. When you look out into the future and you're thinking about the mix of like open versus closed tokens. How early do you think we are on open weights and open token volume? Do you foresee that the majority of that inference in the future will be open or do you think it's always going to be pretty balanced? Like, have you thought much about that? Yeah, I don't know. I think there's a lot of great use cases for open weights. I do think that a lot of the frontier labs are doing a lot more work to lock down their models to make distillation harder. And so I think there's some chance that the trajectory of how fast open weights models keep progressing will slow down, [47:11](https://realalpha.app/v/1C_MErgk2Z0/?t=2831) because it depends on how successful they are at doing that. What happens with China and yeah. Yeah, I mean, look, there's no secret that almost all of those open weights models were because they were able to distill from frontier models. And so to the extent that others are able, the frontier labs are able to block some of that either by hiding reasoning tokens or encrypting or whatever it is, that you may see some of that slow down. You may, you may not, like there may just be others. There's also plenty of folks that are working from first principles on building their own open weights models. And so I expect that others, both in China and around the world, you here in the US, other places, and to expect that those will come up and still be quite powerful because there's a lot of use cases that I think people want want to really control the weights. [47:57](https://realalpha.app/v/1C_MErgk2Z0/?t=2877) They just want to control, you know, either they want to do some things like mix in my own data, whether, you know, through either fine tuning or post training or whatever it is. Although, you know, it remains to be seen that if the frontier models keep progressing, how valuable that tuning is over time. The other one is of course cost. You've already seen some of the frontier model folks respond by offering like much, you know, turns out they can distill their own models much better than anyone else can kind of try to secretly distill their model. And so they can create a small, cheaper version of their own model and offer it for cheaper. You saw both OpenAI and Anthropik do that the last couple of months here. [48:39](https://realalpha.app/v/1C_MErgk2Z0/?t=2919) And so that's a very interesting market dynamic that I think we'll try to see play out, because I don't think they're just going to seed 90% of the tokens to open weights models. They're incentivized. They've already made the fixed cost investment. They can just go make their own model and sell a very powerful distilled model that they own at a cheaper rate. So there'll be some interesting market dynamics to see what happens there. I don't know. The way I think about it is we want to make sure that we support whoever's successful and whatever works out there. And hopefully as long as they're all running on bedrock, and we make them available and useful for customers that they do well. [49:14](https://realalpha.app/v/1C_MErgk2Z0/?t=2954) Okay, last question. AWS generates about 23% of Amazon's revenue, but I think like 63% of its operating income. Amazon is a lot of things. It's a lot of companies within a company. But I'm curious about AWS. I mean, you've been running it for and involved since the beginning. What is it like being the CEO of AWS inside Amazon these days? Like for people who don't understand like there's this massive thing, but it's also part of a bigger thing. Like, what is that like? Well, I don't know. It's just kind of what I do. It's your life. It's just what I do. I don't know. I've been doing this for decades. [49:50](https://realalpha.app/v/1C_MErgk2Z0/?t=2990) No, it's fun. Amazon is a fantastic company to get to work through. I will say one of the things and one of the reasons I've, you know, just really enjoyed working here for the last 20 years is that it is a company that lets you go build. It is a company that encourages you to think big, that go after like a massive opportunity and go build and make it happen. And some really ambitious things like no one would have thought 20 years ago that an online bookseller would be one of the largest technology companies in the world, powering governments and banks. But when we pitched it, Jeff's like, that's awesome. It's a great opportunity. It's a good one to go after. And we empower our teams to think big about what they can go do. And no one would have expected that Amazon would be one of the, I think we're the largest database company in the world. Those are just things that you wouldn't have expected, except we took a really smart team and said, go invent a new cloud database. [50:46](https://realalpha.app/v/1C_MErgk2Z0/?t=3046) And they came up with Aurora and it was really popular. And here we are. And so that is a really fun environment to operate in. It's a great place to work where every day you can come and just say, like, what are the customer problems that I can try to solve today? And I'm going to go after the very biggest customer problems that startups have, that governments have, that regulated industries have, that my own companies have. And operating inside of Amazon is great because you kind of have some built-in customers who can use your services from AWS. And so I feel incredibly lucky that I have one of the largest applications in the world in the retail website. [51:22](https://realalpha.app/v/1C_MErgk2Z0/?t=3082) You know, when we were growing up and starting AWS, I mean, how fortunate are you to have like one of the largest, most complicated technology applications in the world banging on your services and telling you, I need this, I need this, this doesn't work, make this better, this doesn't scale. They hit scale before every one of our customers did, as you might expect for many things that we do. and so it's a fun place to invent and sometimes it's crazy but I like it. Matt, thank you. Yeah, thank you. Thank you for having me. Granola is the best AI notepad I've tried. It works everywhere on a video or phone call, in person or an Apple Watch. [52:03](https://realalpha.app/v/1C_MErgk2Z0/?t=3123) Try it now at granola.ai slash sources and use the promo code sources at checkout for three months off. Banking should feel like modern software. Get everything you need in one place. Visit mercury.com to learn more and apply online in minutes. Mercury's a fintech, not a bank. Check the show notes for details. Jira by Atlassian gives teams the context, visibility, and control to build with clarity and ship with confidence. Try it free at jira.com. That's J-I-R-A dot com. Thank you.