The alpha, twice a week
That was 7 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
The Pragmatic Engineer · 7 min · Nov 16, 2026
Companies need control of their own AI learning loop
Satya Nadella argues firms should separate their harness, context and models so they can switch providers without losing their learning loop.
The episode’s best minutes
Full analysis- 9:55AI coding shifts engineering toward intent and verification4 min
- 15:32Software factories make codebases easier to learn and operatePro pick 5 min
- 46:03Companies need control of their own AI learning loop7 min
- 5 more moments in the full episode
“you should be able to replace any model and still hill climb on an eval that's your private eval.”
Satya Nadellaat 50:53 · The Pragmatic EngineerTakeaways
- Keep the harness, context, memory and models replaceable.
- Test model changes against private evaluations that reflect your needs.
- Treat agents as insider risks and control their authority.
About this segment
Satya Nadella argues firms should separate their harness, context and models so they can switch providers without losing their learning loop. Agents also need runtime controls because capability does not guarantee reliability.
Satya Nadella is CEO of Microsoft.
Led Microsoft's cloud infrastructure and services transformation, and has been CEO since February 2014.

Gergely Orosz, Host · Host, The Pragmatic Engineer
Spent four years at Uber heading mobile, web and backend teams; shipped part of its payments system.
From “Software engineering after code becomes cheap with Satya Nadella”, Nov 16, 2026. Watch the full episode
Answers “Are AI agents ready for real work?” See all 157 answers
Answers “Is AI dangerous?” See all 147 answers
Thanks. We'll look at and fix what's off.
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 46:09
their company a service. And you're saying that most companies should probably consider owning this system, having replaceable parts, primitives. That's correct. So in fact, what you're observing is correct, which is it's fascinating, right? In fact, I would say if you're a bank, you should check how software companies that are trying to manage in this ecosystem to become frontier are doing it. They're not ceding control. So in some sense, that's what the bank should be doing.
- 46:37
So in there, they will have leverage. So for example, what we have done, what we describe as this IQ layer is we have taken, for example, the harness that we have underneath Copilot and GitHub Copilot, made it a first class thing that you can now go to Foundry as a managed service and you have the harness. So in there, let's say you're building a supply chain system, you can start with our harness just as essentially an open, or you can bring Pi or any other open source harness as well. But you can just have it like as a managed service, right? It's a battle tested harness from Microsoft, which we ourselves are using is just available there. And then you can have all the foundry models, right? So you have a harness, you can use multiple models as endpoints, you can have your context layer that is completely separate, right? Your memory and context is yours then.
- 47:25
So I think now we have a model system. Oh, by the way, the action space, so with all the plugins and skills and connectors, right? All separate from any given model, being only having the visibility to it. So I think a good sort of classic middle tier platform is emerging even for model first development where you're not just captured. And then you can eval everything, right? You can say, hey, I have an eval and my private evals will matter more, right? All these public benchmarks, which by the way is great, but you need to have your own private eval for what is the outcome that matters to you. And then look at in your RLE, the real performance for across all these models and continuously optimize. In fact, when I look at even, you know, even with our customers, They start with a frontier model.
- 48:14
Then they say, well, great, I've got all those traces. Now I'm going to flip to, hey, this is a deterministic workflow. I mean, it's not like I don't need the next big model drop. I want to now cost optimize. I'll drop an MAI flash model, drop it in there, use the traces to fine tune it, and there you have it. So therefore, that's all stuff you can do in Foundry. So that type of work, I think, is what's going to increasingly be true.
- 48:37
We're saying that you're already seeing that a lot of companies have their reinforcement learning environment as well on top of the evals so that they can they can stay in control control pricing and also i guess not leak their business data to some of the ai companies that might want to train their models on it that's right so if you think about it right so the way The simple analogy I have, at least in my head, is people in the past, whether in an insurance company, in a bank, in a healthcare company, said, hey, I'm going to get a relational database.
- 49:09
Great. I'm going to deploy a relational database. On top of it, I may even buy some software that runs on that relational database that comes with its own business logic. But then I'm going to extend it with my own business logic that I write, right? And then I control it. And never at any point is my data in the database going to somebody else. It's not like when the database vendor says, hey, I'm no longer going to give you a database update.
- 49:34
You still can migrate out. That's the question you have to, how are you going to migrate your memory out if your memory is in the embeddings of a given model, when that model company says, you know what, tough luck, I can't do a next update to you. Plus, I guess what we talked about, when we have new primitives come, how are you going to integrate them? That's a great point, too. Because if you're too reliant on a vendor or a series of vendors, you'll have to wait for them to maybe do it.
- 50:02
And I guess it's also easier to build right now. If you have competent engineers, we're good at this. Yeah. So I think that there are two enterprise architectural decisions that I think, interestingly enough, I'm hoping that people are going to start talking. And in fact, to your point, all you have to do is look at a software team today in a software industry. They're all figuring this out to stay independent.
- 50:27
I think this has to become more of the reality in the commercial space. And there are two, how do we avoid this reverse information paradox, which is don't leak your value. You as a firm need to be in charge and control of your own learning loop. What's the architectural approach? How are you making sure your context is separate from the models and you're using multiple models? And your harness is separate from the context and the models, because that, I think, is the only way you can assure yourself that you can replace any model.
- 50:56
In fact, my acid test for that is you should be able to replace any model and still hill climb on an eval that's your private eval. If you can't do that, then you have to ask yourself, how are you a sovereign firm? That means you do not have control of your own output. So that's kind of the first. You might be just sitting out taking time bomb. Yeah, I mean, you're done. I mean, like, this is my point.
- 51:22
And so then the next thing is you now need a trust architecture, right? Which is you kind of have to, in a world where you now know scaling laws work, except with scaling comes new capability, but not alignment. In fact, no mechanistic understanding that hopefully someday there's a science breakthrough. But right now we're getting more and more capable and less and less reliable, which is the most dangerous place to be in.
- 51:48
And so that means you have to separate out architecturally authority from intelligence. If your intelligence has authority and it's unreliable, again, you're taken, right? You have a massive insider risk problem. And so you've got to treat these agents as insider risk. That means you've got to have a trust system. And there are a lot of things, like even this chain of thought, man, like this, the amount of, like this chain of thought has to be not in new release.
- 52:13
It has to be in human language. It needs to be auditable. It needs to be logged such that it's not tampered. It's not just in, you know, pre-training or in, you know, RL that things happen, right? Which is in test time compute, things could happen because they could reward a hack. And so we have some serious amount of safeguards you have to build at runtime, as I describe it, which is the trust architecture that ensures that you can treat models and agents as insider risk and monitor them and have observability and what have you, and then ensure that you