# Mathematical proofs could make AI agents safer

Markdown copy of https://realalpha.app/s/mathematical-proofs-could-make-ai-agents-safer/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Matt Garman](https://realalpha.app/builders/matt-garman/) · Matt Garman is CEO of AWS.
- Host: [Alex Heath](https://realalpha.app/people/alex-heath/) · Founder & host, Sources · Left The Verge to start Sources in 2025; it has made over $1M since, mostly from sponsors.
- Show: Sources
- Episode: [AWS CEO Matt Garman on building a $1 trillion business](https://realalpha.app/v/1C_MErgk2Z0/), Nov 10, 2026
- Moment: 41:38 to 45:59 of the episode (4 min), playing on https://realalpha.app/s/mathematical-proofs-could-make-ai-agents-safer/; the episode from there: https://realalpha.app/v/1C_MErgk2Z0/?t=2498

## Quote

> “it's applying **mathematical proofs** to computer science problems to prove things to be true.”
> — [Matt Garman](https://realalpha.app/builders/matt-garman/) at [43:07](https://realalpha.app/v/1C_MErgk2Z0/?t=2587) · Sources

## Takeaways

1. Automated reasoning can verify permissions and detect conflicts.
2. Proofs could keep agents from breaking specified rules.
3. Safer agents could automate work like processing insurance claims.

## About this segment

Matt Garman explains how formal proofs can keep unpredictable agents within defined rules. That could let companies automate sensitive work without relying on a human to check every action.

From “AWS CEO Matt Garman on building a $1 trillion business”, Nov 10, 2026. [Watch the full episode](https://realalpha.app/v/1C_MErgk2Z0/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[41:42](https://realalpha.app/v/1C_MErgk2Z0/?t=2502) And, you know, what customers are saying about, I'm worried about stuff going rogue. And then also just companies, you know, banks are sending out emails to customers saying, you know, don't let your Muse, you know, log in, blah, blah, blah. Like there's this, you all blocked Muse actually. I mean, there's this crazy moment where these agents are starting to traverse things and companies are going, what do I do?

[41:59](https://realalpha.app/v/1C_MErgk2Z0/?t=2519) What have your conversations been like on that? Yeah. Internally with the team and customers. Yeah, I think those are two different things. I think one is when we talk to our customers that are building agents, I think that very clearly we have to help them with some more tools to help on building agent safety and ways to manage and control agents. And I think at some point, you know, agents and AI models are just tools to get work done.

[42:27](https://realalpha.app/v/1C_MErgk2Z0/?t=2547) And we've got to do a good job of helping our customers make it easy for those tools to stay within the guardrails and do the work that they want it to do without, you know, too sharp of edges so that they, you know, they don't cause harm to their companies or leak data or have some sort of, you know, big kind of security disaster. And so that's on us to co-invent what those look like. I actually think one of the big technology unlocks that I feel really excited about is it's largely called neurosymbolic AI, but effectively it's this field where actually Amazon has been probably the single biggest investment that we make internally in research is around what we call automated reasoning.

[43:09](https://realalpha.app/v/1C_MErgk2Z0/?t=2589) And it's applying mathematical proofs to computer science problems to prove things to be true. And so a tangible example of that is when you go and create a new permission, an IAM permissions model in AWS, you run it through IAM Access Analyzer and it basically looks at the whole possibility of all of the permissions that you have inside of your org and it can tell you, is this gonna conflict with some other permission, right?

[43:36](https://realalpha.app/v/1C_MErgk2Z0/?t=2616) Like this one says you should have access, this one doesn't and it tells you those conflict, you better go figure that out or we won't accept it. And so if you just brute force try to go figure those things out, it's like more computers than the earth has to figure out that calculation. But through mathematical proof, you can actually do it relatively fast and get to 100% proof that these things- Verifiability.

[43:57](https://realalpha.app/v/1C_MErgk2Z0/?t=2637) That's right. X plus Y equals Y plus X. If you just brute force try to prove that, you of course can't because there's an infinite number of combinations of that thing, but math can prove it in like three steps or whatever it is. And so if you can take that and you can basically say, I want a non-deterministic agent that's going to be able to go out and try to process an insurance claim, right? I want it to go figure things out. Like it's hard. It's got to go out, look at some data sources and try different things. But I want to make sure that it never breaks this rule or this rule or this rule or follows the policies that I have. If you could have an automated reasoning agent that basically helps steer and build guide rails to your your kind of core AI agent that basically could deterministically prove that it would never

[44:47](https://realalpha.app/v/1C_MErgk2Z0/?t=2687) break invariance. I think you could actually build a set of tools there by combining automated reasoning with traditional kind of AI systems that could give provably correct agents that would kind of follow rules. And so that is an area that we're investing a lot in and it's showing a ton of promise and it's still early, but tools like that can be really powerful. And I think it's one of the things that could unlock where you can imagine an insurance company saying, wow, I would love to just in a fully automated way, let my agent process insurance claims. Now it's like people could submit insurance claims. They get, you get an answer back in 10 seconds.

[45:21](https://realalpha.app/v/1C_MErgk2Z0/?t=2721) Like it would be game changing as opposed to like wait a month to figure out if, you know, you get a claim from your insurance agent because they'd be comfortable sending those things off in a fully automated way. It's where, you know, actually the experience gets better for customers. There's probably more business that they can do, but today they're not comfortable because you actually have to have a human in the loop. That's one of the areas where I think if we can really unblock that, and it's one of the areas we're spending a lot of time thinking about, it has game-changing. And it's one of those places where if you do unlock that, the usage suddenly goes through the roof even more. And it's why I think we're just at those early stages of what's possible What do you think about the timeline of deployment on that? Depends. I think there'll be some next