Free with a sign-in.
The best 10 min of 53
AWS CEO Matt Garman on building a $1 trillion business
Matt Garman says AI is accelerating cloud adoption and leaves inference far below its eventual scale.
Sources · Alex Heath
- Matt Garman is CEO of AWS.
The best minutes
- 41:38Mathematical proofs could make AI agents safer4 min
- 14:07Compute boom risks depend on how providers finance and sell capacity3 min
- 2:21AI is accelerating the move of core workloads to the cloud3 min
Also worth watching
- 38:02AWS reserves compute for startups not yet formed2 min
- 16:59A 50% drop in inference costs can drive tenfold usage3 min
- 9:29AWS expects capacity shortages and stages its spending5 min
- 19:59AWS says local benefits are key to data center support5 min
- 28:22Garman sees deserts as likelier data center sites than space3 min
Skip the sponsor reads, the inference-provider and stack discussion, the Trainium sales explanation, the Nova reset, and the open-weights debate.
Full analysis · 6 min read
Real Alpha · Nov 10
Matt Garman doesn't think inference is even 1% of its eventual scale
- Garman says he does not think inference is even 1% of its eventual scale.
- His 50%-cheaper, tenfold-usage observation supports demand, not AWS's $1 trillion ambition.
- Budget capacity against workload evidence, then test migration, serving costs and customer concentration.
Free with an account
Read the rest of the analysis
7 sections, the 5 other voices it tests Garman against, and what it means for you. Every episode we cut gets one.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Signed up before? The same buttons sign you in. The episode, its moments and the transcript stay free.
AI-led migration gives AWS a growth path, not a forecast
Alex Heath opened with Andy Jassy's $1 trillion AWS revenue ambition within five to 10 years. Heath's rough arithmetic put required compound growth at about 40% over five years, or 18-ish over 10.
Garman cited a $182 billion run rate and six straight quarters of accelerating year-over-year growth. Heath supplied the 39% growth figure while Garman was answering, not as Garman's opening claim.
The business case is that AI agents need access to core data, making cloud migration more urgent. For AWS, migration can add infrastructure demand before companies know which AI products will endure.
But the episode supplies no bridge from migration to $1 trillion. It gives no workload migration rate, AWS share assumption or annual revenue path. As AWS's CEO, Garman is making a supplier's growth case for the cloud infrastructure and services AWS sells.
Our read
The thesis has a credible mechanism but not a quantified bridge to the target. Treat Jassy's $1 trillion figure as an ambition, not a forecast demonstrated by this interview.
Agents make data access valuable, not migration automatic
Garman's strongest mechanism is data access. Agents need to reach across sources, and a mainframe or local database can interrupt that workflow.
For example, a company can leave an analytics warehouse on premises while its existing reports still work. An agent joining warehouse data with other records raises the value of making those sources accessible.
That does not mean every workload must move into one provider's cloud. The episode does not distinguish migration from connecting, exposing or governing data in place.
In October, Jean-Denis Greze, Town's co-founder and CEO, described staged access on the a16z Podcast. Town starts with email and calendar. That is a staged-access example, not evidence that workloads must migrate.
Our read
Count systems made accessible and workloads moved separately. Greze's example is a useful boundary, because wider access and cloud relocation are distinct decisions.
Cheaper inference could expand usage faster than revenue
Garman says he does not think inference has reached even 1% of its eventual scale. He also says each time AWS sees inference get 50% cheaper, usage rises an order of magnitude.
That observation helps explain the investment case, but does not establish revenue growth. If a customer's unit price halves as usage rises tenfold, our arithmetic gives fivefold spend. That assumes the same workload and full price pass-through.
But Garman gives no time horizon, sample or workload mix for the pattern. A rising usage curve therefore says little by itself about provider margins or AWS revenue.
In October, IBM CEO Arvind Krishna discussed open-weight models on All-In. He said companies can match models to tasks, with much enterprise work possible at far lower cost. That is a source of price pressure, not evidence that total inference demand stops growing.
“Every time we see inference get 50% cheaper, we see an order of magnitude more usage.”
Our read
The cost response is a demand hypothesis, not a complete forecast. Finance teams should test how lower serving costs change their own usage, spend and model mix.
AWS's broad workloads reduce some concentration risk
Garman says compute risk is localized, not systemic, because providers differ in financing and customer concentration. He contrasts debt-funded training clusters with AWS's broad base of workloads.
He says millions of customers use AWS, while most AWS AI workloads are inference embedded in applications. For example, an email suggestion serves an end user through ordinary application traffic.
Still, distributed end use does not remove provider concentration. Garman says inference use centers on Anthropic, OpenAI and a handful of model providers, while end users number in the thousands.
Andy Jassy described demand as a barbell on the a16z Podcast in October. Large labs and apps lead while broader enterprise adoption emerges. On MTS in October, SF Compute CEO Evan Conrad described financing and customer concentration as linked GPU-cloud risks.
Andy Jassy · a16z Podcast · Oct 23AI demand is concentrated, but AWS won’t displace core customers
Evan Conrad · MTS · Oct 28GPU clouds face correlated customer and financing risk
Our read
AWS's broad workloads and distributed inference provide a reason for lower customer concentration risk. But the episode does not quantify AWS's exposure to individual model suppliers or financed buildouts.
Staged spending buys time but leaves supply exposed
Garman says AWS stages spending, buying land, securing power and permits before committing to servers. That sequence preserves a chance to delay equipment costs until demand is clearer.
He says most large-customer commitments span three to six years, while some reach eight to 10. Commitments provide a baseline, he says, but customers often spend beyond it.
Even so, Garman expects demand to outstrip supply for a couple more years. Staging limits how early AWS buys some equipment, but does not erase power, permitting or demand risk.
Gil Luria, a D.A. Davidson managing director, discussed Microsoft on Prof G Markets in October. He described long-term data-center commitments paired with short-term neocloud capacity. That makes financing and cash-flow limits part of the capacity plan.
Our read
AWS's approach buys timing flexibility, not perfect forecasting. Separate land and power exposure from later equipment commitments when building an internal downside case.
AWS keeps compute for customers not yet formed
Garman says AWS deliberately keeps part of its future accelerator capacity uncommitted. He says AWS could sell every accelerator expected in 2027, but chooses not to.
The reserve is meant to leave room for enterprises and startups that have not yet formed. AWS also keeps some capacity for Bedrock.
For instance, a customer may receive GPUs later than requested or in another region. Those substitutions show scarcity is not only aggregate supply, but also timing and location.
The episode gives no reserve size or utilization rate. That omission matters because idle reserved compute carries an opportunity cost, while selling out can squeeze new demand.
Our read
A portfolio reserve is concrete policy, not proof of efficient allocation. Track fill rates, delays, regional substitutions and unused capacity against the revenue forgone.
Proofs could widen agent use, but remain unproven
Garman's agent-safety proposal also bears on the growth story. He says mathematical proofs might keep flexible agents inside explicit rules.
For example, AWS's IAM Access Analyzer checks whether permissions conflict. Garman described automated reasoning as a way to prove defined properties without brute-forcing every case.
He imagines an agent processing an insurance claim while never violating specified rules. That could make automated handling acceptable where a human currently reviews each action.
But a proof can enforce only rules that teams define and encode. The episode offers no deployment outcomes, and Garman says some applications may arrive next year.
So this is a possible adoption unlock, not present-day inference demand. Buyers should test what is provable, what remains discretionary, and who owns exceptions.
Our read
Formal reasoning could make agent use practical in sensitive workflows, expanding demand if it works. The interview describes a promising approach, not a demonstrated safety result or deployment record.
What it means for you
- Map agent data access. List the systems an agent needs for one workflow, then record which remain on premises or unavailable. That tests whether AI is creating a real migration requirement.
- Model inference costs. Run a scenario where serving costs fall by 50%, then estimate how usage and total spend would change in your workload. Garman's observation is not a substitute for your own data.
- Separate capex stages. Review land, power and equipment commitments separately, including when each becomes difficult to reverse. Compare that timing with customer commitments and cash-flow limits.
- Track capacity access. Measure how often teams receive requested compute in the requested region and timeframe. Ask whether reserved capacity helps smaller projects without delaying critical workloads.
What to watch
- Whether AI projects produce completed migrations and workload growth, rather than pilots.
- Whether inference costs fall alongside measured usage and customer spend, not just token volume.
- Whether some agents with mathematical guardrails appear next year, as Garman said.
Send this to the CFO and infrastructure lead modeling AI capacity.
Watch the minute where Garman says he does not think inference is even 1%.
Written by Real Alpha from the episode's transcript and the other moments it links, then checked against the transcript. Quotes are the speaker's exact words.
Minute by minute
- 0:00AI expands AWS’s growth opportunity
Alex Heath asks about Andy Jassy’s projection that AWS could reach $1 trillion in annual revenue within five to ten years. Garman says AWS has seen a large opportunity for 20 years; he cites a $182 billion run rate, 39% growth and six straight quarters of accelerating year-over-year growth. He argues that AI agents are prompting companies to move core workloads into the cloud.
- 5:48AI is speeding cloud migrations
Garman says migrations that once happened slowly are speeding up because AI agents need access to data across a company’s systems, including legacy mainframes. He says agents can help uncover insights and new revenue, and that companies that leave workloads on-premises risk falling behind. AWS sees demand from large enterprises as well as startups, including two-person teams.
- 9:29AWS is staging capacity investments
Garman says AWS is developing cost controls that can cap spending or shift inference to a cheaper model. He says demand is outstripping supply and expects capacity shortages for the next couple of years. AWS commits early to long-lead items such as land and power, then waits to buy servers until it has better demand signals.
- 14:07Garman sees compute risk as localized
Garman argues that compute risk depends on how providers finance capacity and how concentrated their customers and workloads are. He contrasts highly debt-financed providers serving a few large training clusters with AWS’s broader customer base and says most AWS AI workloads are inference. He argues inference is distributed across applications and users, who could switch to another model rather than stop using it.
- 16:59Inference is still below its eventual scale
Garman says inference is not even 1% of what it could eventually become, and argues that AI is already embedded in services people use without paying for it directly. He says every 50% reduction in inference cost has brought an order of magnitude more usage. He expects spending to fund both new products and efficiency gains.
- 19:59Data centers need community support
Garman says operators need to work with local officials and residents, demonstrate community benefits and limit local impacts. He says AWS pays for the power it brings to the grid, aims to be water-positive by 2030 and says its largest data centers use less water than 40 U.S. households. He argues the industry should develop clearer programs for responsible community engagement.
- 28:22Deserts look more practical than space
Garman says data centers in space are unlikely to be practical or economical in the next decade or two, citing launch limits, heat dissipation, radiation and serviceability. He also sees challenges with ocean-based sites, but says deserts look more promising because of solar resources and available space. He notes AWS is installing one GPU every 14 seconds.
- 31:32Inference providers add customer choice
Garman describes inference providers such as Baseten and Fireworks as services built on cloud platforms that are mostly complementary to AWS, though they can also compete. He says they help customers use open-weight models and sees partners as an important part of AWS’s offering. AWS wants to compete across the stack, from chips and infrastructure to services and applications.
- 36:49Garman explains Trainium’s external sales
Garman says customers want to use Trainium outside AWS, including in on-premises and sovereign AI environments, and AWS sees a business opportunity in selling the chips there. He says the company will keep investing in Trainium.
- 38:02AWS reserves compute for future customers
Garman says AWS holds regular meetings to decide how to allocate scarce capacity. The company commits some capacity in advance but keeps some available for enterprises, startups that have not yet formed and Bedrock. He says AWS may offer a different delivery date or region when a customer’s first choice is unavailable.
- 40:06AWS still wants frontier models
Garman says AWS remains committed to building a first-party frontier model and calls the changes to its Nova team a reset, not a retreat. He says AWS learned it needed the right talent, data and infrastructure in place, and that the team is continuing its work.
- 41:38Formal proofs could constrain AI agents
Garman says customers need tools to keep agents within guardrails and protect data. He describes AWS’s automated-reasoning research, which uses mathematical proofs to check whether permissions and other rules hold. He says combining that approach with AI could help agents handle tasks, such as insurance claims, while following defined rules; he expects some deployments next year.
- 46:31Open-weight model growth remains uncertain
Garman says frontier labs’ efforts to make distillation harder could slow the progress of open-weight models, though teams may also build them from first principles. He sees customer control and cost as reasons to use open weights, while noting that frontier labs can produce cheaper versions of their own models. He says AWS wants to support whichever approaches customers choose.
- 49:12AWS gains a demanding customer inside Amazon
Garman says Amazon gives AWS room to pursue large customer problems and encourages teams to build new services. He describes Amazon’s retail website as a demanding internal customer that encountered scale issues before other customers and pushed AWS to improve. He also cites Aurora as an example of a cloud database the company built in response to customer needs.
Published Nov 10, 2026.
Transcript · 48 min read
Sources, hosted by Alex Heath, with Matt Garman.
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
Matt, it's great to be with you. I feel like what's happening right now at AWS feels unprecedented in a lot of ways. So you're the world's biggest cloud computing company. You're the highest capex spender. You have an estimated, I think, like 44 gigawatts of capacity, which is that's coming under development in the U.S., which is more than Google and Meta combined. And then what just happened was Andy Jassy on the earnings call saying AWS could be a trillion dollar revenue business in the next five to 10 years, which is a huge number, obviously. If my math is correct, I mean, based on the run rate you're already on, that requires roughly 40% compound growth over five years or 18-ish over 10 years.
And this is off of a massive, massive base. I think a lot of people when they heard that were like, wow, like what are they seeing? And then I was also looking at the way you all have been talking about the business over the last couple quarters. And I think it was towards the beginning of this year, Andy was saying, we think roughly $600 billion by around 2036. And then in Q2, he floated the trillion. And then in Q3, it felt very strong. It felt like a very strong declarative moment. And I'm curious, maybe we can start here, what happened between Q2 and Q3? Yeah. Well, if I step back, You know, I think it's, we have probably for 20 years thought that the AWS opportunity is just a massive business opportunity and unlike one that we've seen before.
And we've been, so I've been with AWS for 20 years and we've been growing that whole time. And I think we, over time, have more and more and more gotten conviction that the business opportunity is just really, really, really large. And it's surprising, but at the point that we're at now, we're at $182 billion run rate business. And so you're over $180 billion and you're growing at 39%. There's not many companies that hit that spot. And as we've seen, we've actually increased our year over year growth rate each of the last six quarters. And so as we see that acceleration, what happens there is we've always had this view that the majority of workloads still live in on-premise worlds.
Read the whole transcript
And as big as the cloud has gotten, there's still a lot of data centers that people have. There's a lot of workloads that live on-prem. And there are more workloads that are just being created every single day.
“they weren't going to get the benefit of AI if they didn't get those workloads in the cloud”
these technologies is bigger than it's ever been. This episode is brought to you by Granola, the AI notepad for people in back-to-back meetings. It works everywhere you do and lets you focus on what matters. Try it at granola.ai slash sources and use the code sources for three months off. This episode is also brought to you by Mercury, AI native banking that's loved by more than 300,000 entrepreneurs, including me. Visit mercury.com to learn more.
Mercury is a fintech, not a bank. Check the show notes for details. This episode is brought to you by Jira by Atlassian, where teams get the context, visibility, and control to move work forward. Try it free at jira.com. That's J-I-R-A dot com. AI is causing companies to come from, you know, on-prem to the cloud. That's right. That's driving a lot of this growth. Absolutely. Interesting. I hadn't thought about that. I mean, you've been saying and the company's been saying for so long, like the world, we underestimate how much of data is still on-prem. And for people who don't know, just means like it's not in a cloud.
It's somewhere local. What you all are seeing is people who have resisted the cloud are going towards it because of AI. We'll use this as just take something like an analytics workload, okay? An analytics workload, traditionally, a customer might say, great, I've got a data warehouse, it's running in my data center. I could move it to the cloud and take advantage of Amazon's analytics capabilities or move it to a Snowflake or a Databricks or something like that. That would be great. I could do that. And a lot of people did. But they're also like, yeah, but I'm still able to do the analytics here and it's working and maybe I won't move it or maybe I'll do it next year or two years when I can break the project work. And we saw those migrations happening, by the way. But they're slow and measured and scheduled out. Now we see people saying, okay, actually, I can ask an agent to go across all of my sources of data. And not just can I do my analytics faster or cheaper, but with new technologies that are available in a cloud world, say something like a DuckDB data engine that's working across all of my different data sources, I can actually have an agent go and do all of that.
And when I have an agent do that, it has to have all of the data sources accessible, they have to be available to the agent. It has to know where all the data is, but I actually get very new insights and actually can build new businesses, unlock new sources of revenue, new source of profitability when I can do that, except one of my source of data is locked up in the mainframe. And so it's just, it draws all of it in because they kind of can start to see where that benefit is. And I think there's a real risk that companies that don't make that move, they're getting passed by by the ones that are in the cloud. How much of the growth is big customers wanting to spend more versus the long tail?
Well, look, big customers are big, so they spend lots. So a lot of it is that. But I will say that the long tail is growing bigger. It really is across the board, actually. And as we think about the long tail, I think it's both, you know, it's not a lot of our growth is driven by like, you know, dentist's office, like not really kind of real, real small SMBs. Those are mostly served by a lot of our partners that actually run their software and services on top of AWS. But our customers, we have a really broad swath of whether it's, you know, global top 5,000 enterprises. Actually, just this week, I was in the Bay Area meeting a bunch of startups that many of them started less than six months ago.
And they're really thinking about how do they use AWS to accelerate their growth, accelerate their technology development, accelerate their route to market. And our customers love having access to that capability as well. And so it's across the board. Sometimes it's the very largest banks, the very largest healthcare companies, the very largest governments in the world. And sometimes it's a two-person startup that has an idea and just wants to build. And we're seeing growth from all of those segments. In fact, it's never been easier to build than it is today to be a two-person startup and go try something and actually get it to scale. And I do not think I've found a two-person startup that goes and tries to find data center space and rack a server to start get going, right? Like it just doesn't make any sense.
And so that's been true for a long time, but now they can use AI to really further amplify the impact that they're going to have and how fast that they can grow. And we're seeing those new companies scale really rapidly when they get that product market fit. And AWS and the cloud gives them a chance to cycle until they get that product fit.
“I anticipate that we'll be short of capacity for the next couple of years.”
“And those really large training clusters are largely being bought by a small number of customers.”
and they're huge partners, obviously. We want them to do well. You want them to do well, of course. I think everyone's with like on pins and needles about Anthropic IPO.
“every time we see inference get 50% cheaper, we see an order of magnitude more usage.”
“Our data centers, we bring power and pay for all the power we bring to the grid.”
Framer is the AI website builder that powers the Sources podcast website at podcast.sources.news. Framer brings AI agents into the same canvas where your website is designed, managed, and published so you can move faster without giving up your taste or control. I use Framer to make the Sources podcast website be the destination for everywhere you can find the show, plus you can also see recent issues of my newsletter.
Start building with agents for free today at framer.com slash sources for 30% off a Framer Pro annual plan. Framer.com slash sources. Rules and restrictions may apply. I spend a lot of time context switching between meetings, often with no time to process one before the next starts. Thankfully, Granola runs in the background the whole time. It's an easy-to-use AI notepad for meetings that works everywhere, even on phone calls. I use Granola to recall what was said in meetings and create helpful summaries. I use it every day to stay on top of what I need to get done with my team. It connects to my email and suggests follow-ups for me to quickly review and send, saving me valuable time.
Granola isn't just a core part of my workflow. It's basically my second brain. Try granola at granola.ai slash sources and use the promo code sources for three months off. Mercury is a modern take on banking built for startups like mine. When I decided to start my media business, Mercury was by far the most straightforward, full-featured banking solution for me to set up quickly. The interface is intuitive and simple, saving me valuable time every day. I use Mercury to track my spending, bills, and invoicing. I love that I can delegate permissions to my team so they can keep things running for me in exactly the way I want them to. My favorite part is how forward-looking Mercury is with AI. Legacy banks are stuck in the past, but Mercury is built for how modern software works today. I use its built-in command assistant to analyze cash flow and help me move money. And Mercury also connects to other AI tools like ChatGPT and Cloud. I use this feature all the time and the folks at Mercury actually let me know that I'm one of the top users of it.
So trust me, it's finally easy to get real-time financial data about your business wherever you need it. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not a FDIC-insured bank, banking services provided through Choice Financial Group and Column N.A., members FDIC. As you move faster and bring in more AI agents, keeping tabs on what's actually happening gets harder. Who's working on that? What did that agent just change? Are we still on track for our goals? That's why teams rely on JIRA for visibility across their work, whether it's done by people or agents.
JIRA is powered by Atlassian's Teamwork Graph, which connects work across JIRA, your most used tools and apps, and your agents. That means your agents share the same context your team has, like the goal behind the work, the decision made yesterday, and the project plan, so they work faster and more accurately. You get oversight, track agent progress right alongside human work, spot blockers sooner, and connect your work to goals. You don't have to choose between shipping fast and staying in control. I want to get your reaction to like three areas that people are talking about putting data centers
“I think that it is highly unlikely to be economically feasible or practical in the next decade or two.”
So, Base 10, Fireworks, Modal, FAL, the list goes on. And they're servicing a very important growing part of, I guess you could call it cloud stack.
Very AI native, very model heavy tuning. And then there's a whole post-training inference world that's developing. When you look at that, and you all obviously have that as well, what do you make of that? In your mind, is that a new kind of cloud market that's forming or is it just a moment in time? Like how do you think about that? I don't really view it a cloud offering. I think it's more like a kind of application or services that are built on top of the cloud that are both complimentary and in some ways competitive, but mostly complimentary to what we built. And so think about in the database side, like MongoDB building on top of AWS, or think in the analytics side of like a Snowflake or a Databricks building on top of AWS, Um, uh, inference providers building, you know, on top of AWS, you know, that, that is net positive to the customers.
They want it in their, they want it in AWS. They want to use that inference with the rest of their AWS stack. It's, it's complimentary. Um, it may be, you know, on some dimensions competitive, but, but, but we've operated in that world for a really long time where many of our products both compete and support customers. And we've built AWS from the very beginning, knowing that partners building on top of us are going to be an incredibly important part of what we offer to our end customers and have always been a huge part of what we go and build. And our view is if partners are successful building on top of us, then we're going to be successful.
And it gives customers more choice. I kind of put them more in that bucket. I think they do a great job. They do a great job of helping customers use open-ways models, which are oftentimes hard to use, hard to optimize. You have to be kind of an expert. They do a great job of helping with that, where customers think they may have interesting data or interesting ways to use open source or open weights models, either to make them cheaper or different capabilities. And I think those folks all, and they all have different kind of slices as to how they help, but they help customers and we like working with them.
They're a great part of our partner ecosystem. And I guess part of what I'm getting at there is how much of the AWS advantage do you think about being the software layer versus the bare metal? And especially in a world where an agent can write the integration and recreate things, and essentially, like, you could really see a world in the near future where agents are picking clouds and doing everything on behalf of their companies. And in that world, like, what level of the stack do you need to own? Have you been thinking about that? Yeah, look, we think it's important for us to play in all of the layers of the stack, just to be super clear.
And so I think all of those are incredibly important for us. I think at the very lowest layers of the stack, there's just not that many, you know, we often think there's like, there's three basically right now. And I don't know that that probably will continue to be roughly the same number. There's economies of scale there. Yeah. And then as you kind of move up, I think it's a funnel that kind of keeps going that way. And then you get to end user applications where there's thousands and thousands, there's bigger and smaller ones, but there's lots of those up there. And we like to play at all of those layers of the stack.
We like to play at the infrastructure layer. We like to make our own chips. make our own networking gear, like all of those kind of things at the very lowest layers. We have services, we have managed databases, we have managed storage offerings, we have managed analytics services. And then with the top layer, we have applications as well, where we have applications for customer service with Connect, we have applications for worker productivity with Quick and applications for developers with Kuro, and we try to compete at all those layers of the stack. And the inference, and as you go up, there are more people that also build on the lower layers of the stack that we have and then compete at that layer.
A lot of these inference players are buying their own chips. And so they're more in this middle layer, right? Where they might buy from AWS and build a service on top of that and run that. I think at the lowest level, you know, we kind of don't really partner with many people other than, you know, hardware providers and things like that. Because, you know, we're kind of the base layer of that thing. But then as you go up, it's more a competition environment where it's great. Like we encourage that. And so I think it's important for us to be at all of those layers of the stack. I think we want to help customers run inference directly on us.
We want to, and for other folks that are building inference engines, whether it's OpenAI, whether it's Anthropic, whether it's Fireworks or Base 10 Optimize and OpenWeights model, whether it's the OpenWeights models directly, whether it's our own model, whether it's, you know, all of those different things, we want to make sure those are all capable, you know, available to our customers. and you know like in the fullness of time we're not there yet but we'd love to have all of those available in bedrock um it's it's not technology-wise able to do that yet but we'd love to make them all available if they want to be there um otherwise if they don't want to be there that's okay too but we want to help it make it as easy as possible for our customers to use um whatever um price performance capability makes the most sense for their application because even if we don't win the upper layer stack our view is if it's built on aws will win but we think we can do a really good job of offering kind of a bunch of those capabilities because we can control more of those pieces.
And we think, you know, we have good teams to build services that are valuable to our customers. Speaking of that lower part of the stack, I'm very interested in your all's decision to start selling Trainium chips externally. I haven't really heard you talk about that. Why are you doing that? I mean, Google's starting to sell TPUs externally. Is this just like pricing power and like, you know, keeping a market that's kind of like revolving around you? Like what's the reason for doing that? Well, look, we've made a huge investment in Trainium and it's gone really well. Like it's, you know, and people tell us that they want to be able to use Trainium in more places.
And, um, but why not make them have to use it in AWS? There are only so many, like, we're not going to build every single data center that exists in the world. Sometimes they're going to be on-prem, whether it's in a country somewhere, whether it's inside of a government somewhere, whether it's, um, you know, there's like a sovereign AI component. It could be, you know, there's, there's lots of places. And if you look at like everywhere that NVIDIA sells chips today. They sell a lot of their chips to, you know, us and Google and Microsoft, and they sell a lot of their chips to other places. And like I said, I think only, you know, 85% of workloads are still on-prem.
There's still a lot of on-prem locations. And customers have told us like, hey, we'd love to run a lot of our workloads inside of the cloud on Trainium, but we may want some of it in our own data center. And we think there's a really good business there and we're excited about it.
“We want to be able to say yes to a new startup that hasn't even formed yet that we'll have capacity for.”
Last time you and I talked, we talked a little bit about Nova. This was earlier in the year. And you were all's frontier model ambitions. And you were very clear, like, no, we need to be at the frontier. I want us to be at the frontier. And then since then, there's been some changes.
some, I can't, I haven't been able to quite tell. I wanted to ask you like what's going on. Cause there were some layoffs and Nova models being sunsetted or put on ice or something. And I'm curious, like, is this, you all deciding that no, like we don't need to have models at the frontier in house or what's the, what's the strategy? No, far from it. We're, we're just as invested in that strategy as we've ever been. And I would just say, you know, stay tuned. We're quite excited about it. It was more of a reset. It was more of a reset. That's right. That's right. I think we identified some opportunities to learn from some of the approaches we had taken before.
But it is just as big of an investment, just as important from our long-term strategy perspective. We think that customers are going to want a first-party frontier model from AWS, and we're very invested in going in that direction. Anything on what you learned in terms of what's kind of driving this reset that you can be more specific on? No, I think we just, you know, I think we did a reset and make sure that we had kind of the right talent in place, some of the learnings with regards to making sure that we had kind of some of the right data pieces in place and the right infrastructure in place and learning a little bit about how to do some of those things at scale.
And I think we've made some good learnings there and the team is cranking away and doing really well right now. How are you feeling about the safety piece of agents right now?
“it's applying mathematical proofs to computer science problems to prove things to be true.”
year. Okay. And we're going to be keep iterating it and growing it over time. So this is like the
next big bet, it sounds like. It's one of them. Yeah. It's one of the ways that I think you can add determinism to, without taking away the benefits of the non-deterministic AI models that we have. And that could be the answer, yeah, to a lot of these safety issues that people have. That's right. That's right. You can imagine a, you know, you can imagine if you could, if you could define a rule that tells an agent, you will never try to escalate your privileges. Yeah. All of a sudden, it doesn't matter if you have like a magic box that it could escalate its privileges, it won't try. When you look out into the future and you're thinking about the mix of like open versus closed tokens. How early do you think we are on open weights and open token volume? Do you foresee that the majority of that inference in the future will be open or do you think it's always going to be pretty balanced? Like, have you thought much about that? Yeah, I don't know. I think there's a lot of great use cases for open weights. I do think that a lot of the frontier labs are doing a lot more work to lock down their models to make distillation harder. And so I think there's some chance that the trajectory of how fast open weights models keep progressing will slow down,
because it depends on how successful they are at doing that. What happens with China and yeah. Yeah, I mean, look, there's no secret that almost all of those open weights models were because they were able to distill from frontier models. And so to the extent that others are able, the frontier labs are able to block some of that either by hiding reasoning tokens or encrypting or whatever it is, that you may see some of that slow down. You may, you may not, like there may just be others. There's also plenty of folks that are working from first principles on building their own open weights models. And so I expect that others, both in China and around the world, you here in the US, other places, and to expect that those will come up and still be quite powerful because there's a lot of use cases that I think people want want to really control the weights.
They just want to control, you know, either they want to do some things like mix in my own data, whether, you know, through either fine tuning or post training or whatever it is. Although, you know, it remains to be seen that if the frontier models keep progressing, how valuable that tuning is over time. The other one is of course cost. You've already seen some of the frontier model folks respond by offering like much, you know, turns out they can distill their own models much better than anyone else can kind of try to secretly distill their model. And so they can create a small, cheaper version of their own model and offer it for cheaper. You saw both OpenAI and Anthropik do that the last couple of months here.
And so that's a very interesting market dynamic that I think we'll try to see play out, because I don't think they're just going to seed 90% of the tokens to open weights models. They're incentivized. They've already made the fixed cost investment. They can just go make their own model and sell a very powerful distilled model that they own at a cheaper rate. So there'll be some interesting market dynamics to see what happens there. I don't know. The way I think about it is we want to make sure that we support whoever's successful and whatever works out there. And hopefully as long as they're all running on bedrock, and we make them available and useful for customers that they do well.
Okay, last question. AWS generates about 23% of Amazon's revenue, but I think like 63% of its operating income. Amazon is a lot of things. It's a lot of companies within a company. But I'm curious about AWS. I mean, you've been running it for and involved since the beginning. What is it like being the CEO of AWS inside Amazon these days? Like for people who don't understand like there's this massive thing, but it's also part of a bigger thing. Like, what is that like? Well, I don't know. It's just kind of what I do. It's your life. It's just what I do. I don't know. I've been doing this for decades.
No, it's fun. Amazon is a fantastic company to get to work through. I will say one of the things and one of the reasons I've, you know, just really enjoyed working here for the last 20 years is that it is a company that lets you go build. It is a company that encourages you to think big, that go after like a massive opportunity and go build and make it happen. And some really ambitious things like no one would have thought 20 years ago that an online bookseller would be one of the largest technology companies in the world, powering governments and banks. But when we pitched it, Jeff's like, that's awesome. It's a great opportunity. It's a good one to go after. And we empower our teams to think big about what they can go do. And no one would have expected that Amazon would be one of the, I think we're the largest database company in the world. Those are just things that you wouldn't have expected, except we took a really smart team and said, go invent a new cloud database.
And they came up with Aurora and it was really popular. And here we are. And so that is a really fun environment to operate in. It's a great place to work where every day you can come and just say, like, what are the customer problems that I can try to solve today? And I'm going to go after the very biggest customer problems that startups have, that governments have, that regulated industries have, that my own companies have. And operating inside of Amazon is great because you kind of have some built-in customers who can use your services from AWS. And so I feel incredibly lucky that I have one of the largest applications in the world in the retail website.
You know, when we were growing up and starting AWS, I mean, how fortunate are you to have like one of the largest, most complicated technology applications in the world banging on your services and telling you, I need this, I need this, this doesn't work, make this better, this doesn't scale. They hit scale before every one of our customers did, as you might expect for many things that we do. and so it's a fun place to invent and sometimes it's crazy but I like it. Matt, thank you. Yeah, thank you. Thank you for having me. Granola is the best AI notepad I've tried. It works everywhere on a video or phone call, in person or an Apple Watch.
Try it now at granola.ai slash sources and use the promo code sources at checkout for three months off. Banking should feel like modern software. Get everything you need in one place. Visit mercury.com to learn more and apply online in minutes. Mercury's a fintech, not a bank. Check the show notes for details. Jira by Atlassian gives teams the context, visibility, and control to build with clarity and ship with confidence. Try it free at jira.com. That's J-I-R-A dot com. Thank you.
