Satya Nadella: Whether to Trust Frontier Models | Why Companies Need to Own Their Own Intelligence

Can I ask, what does that trust system look like? Who's the architecture of it? Who's the arbiter of it? It's a great point. So that's why I kind of look at it and say, you leave AI safety outside of this, right? Because that alignment, for example, at least my take is that it's really mostly has to show up in training, right? In fact, maybe even in pre-training. I don't think you can even RL your way to alignment. You have to truly get alignment to be something that truly gets embedded in the weights right at the very beginning of it. And so that's the model itself. Hopefully, at some point, we'll get aligned.

So if you leave AI safety, then the trust architecture is going to be just, you know, what a company like Microsoft does or all the cyber folks do. So what we have done, for example, is we build these two things, right? Agent 365 and perception, right? What does it do? It gives every agent an identity. It then logs all of its activity, all its tool calls, in fact, every model call and the COT unit. In fact, that's where I'm also asking that all the COT better be in English and not in new release. And then you can check using multiple models. It better be tamper-proof, right? You can't have the agent go modify its own logs.

And then on that, you set policy, and then you behaviorally monitor the activity of the agents so they're confirmed to the policy. So that's the architecture. We don't have that architecture today. Is that just symbolic of where we are in the production? No, that architecture exists. So there are two, I think people conflate what I'll call an engineered security, zero trust architectural system for containment and AI safety, right? Because I think people say, well, AI safety is hard and elusive and not yet achieved. So therefore, there must be nothing we can do on security. And I'm not saying that these two things are all got to come together at the same time. In fact, I would just say, hey, we have to deal with the situation as it exists. So let's build a trust system outside, in fact, of the model, and then just be super aggressive in setting policy and monitoring for policy deviation. On the monitoring

dimensions, which is like traffic and dollars. If we look forward, say, 24 months, what percent of traffic goes through open versus closed? It's a great question. I think it depends on

what the frontier looks like, which is I think most things seem to get saturated. And when it gets saturated, open weights or even just cheaper models of even closed weights seems to work. Because especially one of the things is if you can train on your own traces, then you really don't need the latest model to be able to solve that same problem if the problem has not changed, right? So if you have an eval that you want to continuously keep moving forward, then you may want to pay. But even there, by the way, I'm very thrilled about this hydrofusion stuff we now have in GitHub where- I just saw this. Yeah, because the cool thing there is it's a learned model.

Anyway, we are now going into these agent swarms and multiple sub-agents. So I can call a lot of sub-agents that even happen to be local models. And so I think my suspicion is that there will always be a market for Frontier. There will always be some new things. Science may be the next thing that Frontier is being set at. But I think predominantly token usage will be commodity token usage. Does that mean it's open source predominantly? It may be. It may be open weights. Or as I said, it can also be cheap closed weights, right? Because there'll be lots of price competition, let me put it that way, right?

Because if you look at my own usage inside a co-pilot, we use a lot of close source, cheaper models, along with open rate models.

Because when we think about these customers, like, as you said, some of your very big customers, they will need heavy FDE support, much more so than they have done in traditional years in the last 10 years, will we have a very heavy implementation economy to transition existing customers into this? Yeah, it's interesting, right? The implementation economy, like systems integrators have always existed, right? You know, we've depended on implementation economy for lots of other technology diffusion.

So this one, I think, has different contours, right? Because- What do you think is different about this one? It's just, it's the combination of tech and business, if I could say. But it's not classic management consulting, it's not classic systems integration, and it's not classic sort of technology support. It's sort of, like, for example, think about this, right? You want to start, like, when I look at my FDE engagements, right, even take the Unilever example I talked about at the keynote today. It's fascinating. They've built an agentic system to say, hey, we're going to really have order-to-cash automated using an agentic system, plus their trade claims.

And by the way, they started using a frontier model, and then they substitute for, in fact, I think they're using currently an MAI model for all the trade claims because it's sort of a lower cost. And it's, by the way, they control it, right, because they're able to train on their traces. And so we had some FTEs, but a lot of who got deployed, and now their IT team itself is, you know, managing it all. But it started with what's the business? What's the eval? What's the way to conceptualize the agentic architecture? So it's sort of business consulting combined with technical consulting. And I think, yeah, you're right in saying that there will be a new set of skills that will get built in the ecosystem. Today, I think all of us have, quote unquote, our own FDEs, but FDEs will become like, you know, traditional systems integration will become new FDE.

You've mentioned, obviously, MAI. But when we look at a lot of different players, it's kind of this realization that you need to be full stack from chips to models to application. Is that the future? You have to have vertical ownership? Yeah, it all depends on the industry structure, right? You know, I think that I would say today, you know, there'll be, who knows, five, six closed models all competing, maybe five, six, you know, open weight models all competing. And so therefore... At some level, if you're building an application like a Gentic system, you could even have something like what I said, which is a router with even a decision engine built into it, and then you just route to the best model for a given task. So that's a possibility. Of course, there's always the risk that there could be one vertically integrated model and app tier and all the way to the silicon and system. It's unlikely to me, right? I thought at some point that that might be the case, that there will be one. If anything, it's going the other way, which is every day you wake up and you say, wow, there's more competition in every layer of the stack, whether it's in silicon, whether it's in systems to manage the silicon, in the model layer, in the app layer.

And so I think that's a healthy sort of direction of travel, but you never know, right? Because this is one of those things where it all depends on what becomes the dominant design. But my hope is that this becomes a little bit of a horizontal specialization with multiple winners in each layer, and then the ability for people to compose. The thing I didn't see is the depth of each layer. What I know now that I wish I'd known 12 months ago, it's just like the infinite demand for inference. Yeah, that's a great point. And I wish I'd known it 12 months ago, I would have done some very different investing. Speaking of that, we've had like Mustafa Suleyman saying training runs of

fascinating to see how much the demand is elastic for software development, right? That means the IT back.

clogs were huge and none of us even recognized it i was taught that you had to be the top one or two in a market to win in venture we're in you know factory which is fifth and it's winning fifth yeah this breaks all rules of venture a question that i have again slightly naughty not on schedule but it's just like well we see the tipping in terms of capability advancement in alternative functions finance sales marketing that we've seen in coding because coding has been the mother load I think that that's a good point. People talk a lot about, you know, do you have a verifiable reward and, you know, it's easy to do these because a lot of the magic has been RL on top of these pre-trained models. That is what, you know, has helped coding. Can we crack that in knowledge work seems much harder. But again, all the examples I talked about sort of, you know, in contact centers, in the supply chain systems, in these marketing slash or cash to order systems, I think that it's possible and we're seeing it.

Is the demand as elastic as coding? I think it may be science may be as elastic, right? Or engineering in general. Like, by the way, you know, if I look at even what's happening inside, not just in software development, let's take silicon development. That's another thing that's going through a radical shift, right? Your ability to tape out much faster, simulate everything in silico is much better. Your first spin can completely come out. But even just like optimization of maintenance reports, like you said, with Rolls-Royce, which I would never... It's more. It's unbelievable. Like, I mean, it's a thing that they've been dealing with forever, just scheduled maintenance.

And I had not realized that, wow, that's a big optimization problem that you can go now run. So I've watched many, many of your shows, and we're going to do a quick fire round. And people haven't asked some of these questions, which I've always wanted to know, and I think everyone wants to know, but people haven't asked them. What's the exercise routine given the travel schedule, Satya? Well, I mean, you're in good shape. Well, I try to be. I don't know whether it's all well hidden. but you know the one thing i've gotten into the habit you know this has been maybe 25 years even on which is every morning i get up and you know i used to be able to run unfortunately my knees have gone away and so now at least i walk with incline even this morning i was able to get a half hour in and so that's what in spite of any jet lag wherever i am i get hit the gym every day i do have bad knees it's my age as well uh what have you changed your mind on in the last 12 months? It might be, because after all, I was one of the first, obviously, when Dario and Ilya and Sam and Greg and Elon were all, I guess, one happy family, they were really the believers in scaling laws. And that's what I believed in back in 2018, 2019. And I think the biggest change

now I have is I used to always think that alignment will just fall out. I no longer believe that. So So I think this alignment thing may be a harder, different problem. And so, yeah, that's the one thing I've changed my mind on. What looks crazy today, but will be incredibly common and accepted in five years? You know, it might be that thing that we just talked about, this evals, right? What is a firm? I think a firm would be an AI benchmark. Final one for you. There are many things to be excited about over the next 10 years. For you specifically, like for me, it's my mother's got MS. I'm so excited for the drug development that could be capable.

What are you most excited for? I mean, that's a great one, I think. One of the things that I've always been, I would say, been looking forward to is a general purpose technology, even as powerful as this. And people say, oh, this is not just another general purpose technology, whatever this is. I hope this spreads, diffuses throughout the world faster than anything else and throughout all sectors of the economy, unlike anything else. And truly, for once, there is a real shift in the frontier of that global growth that happens, where it's not about just counting winners and losers, but it's kind of like everyone gets lifted.

That's been my dream. In fact, my father was a development economist. It's about convergence growth. When will the Global South get something that just suddenly boosts that? I hope this is it. That's it. This has been such a joy to do. Thank you so much for taking the time. Absolutely. It's a pleasure, Harry.