# Can Humans Still Control AI-Assisted Kill Chains? (with Katrina Manson)

Markdown copy of https://realalpha.app/v/nPsqoXabzzc/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

Katrina Manson says Maven speeds military targeting, but human oversight, data quality and accountability remain unresolved.

- Show: Future of Life Institute Podcast
- Hosts: [Gus Docker](https://realalpha.app/people/gus-docker/)
- Guest: [Katrina Manson](https://realalpha.app/people/katrina-manson/) is author of Project Maven: A Marine Colonel, His Team, and the Dawn of AI Warfare.
- Published: Sep 25, 2026
- Length: 42 min

## The best minutes

- [8:42](https://realalpha.app/v/nPsqoXabzzc/?t=522) [AI sharply increased the number of targets the military could strike](https://realalpha.app/s/ai-sharply-increased-the-number-of-targets-the-military/) (7 min)
- [17:43](https://realalpha.app/v/nPsqoXabzzc/?t=1063) [Computer vision struggled when Ukraine's battlefield changed](https://realalpha.app/s/computer-vision-struggled-when-ukraines-battlefield-changed/) (3 min)
- [23:30](https://realalpha.app/v/nPsqoXabzzc/?t=1410) [Targeting doctrine is shifting from human decisions to oversight](https://realalpha.app/s/targeting-doctrine-is-shifting-from-human-decisions-to/) (3 min)

## Also worth watching

- [35:53](https://realalpha.app/v/nPsqoXabzzc/?t=2153) [Accurately labeled data matters more than the model](https://realalpha.app/s/accurately-labeled-data-matters-more-than-the-model/) (3 min)
- [27:00](https://realalpha.app/v/nPsqoXabzzc/?t=1620) [Maven was updated 50 times during Ukraine's first months](https://realalpha.app/s/maven-was-updated-50-times-during-ukraines-first-months/) (3 min)
- [29:53](https://realalpha.app/v/nPsqoXabzzc/?t=1793) [AI companies lack visibility into classified military use](https://realalpha.app/s/ai-companies-lack-visibility-into-classified-military-use/) (3 min)
- [15:45](https://realalpha.app/v/nPsqoXabzzc/?t=945) [AI made human judgment the longest step in a targeting cycle](https://realalpha.app/s/ai-made-human-judgment-the-longest-step-in-a-targeting-cycle/) (2 min)

Skip the opening project overview and system tour, future command-system plans, and the closing open questions.

## Full analysis: Katrina Manson says Maven was used to hit 13,000 targets

- Manson said Central Command reported Maven's use in strikes on 13,000 targets.
- That figure shows throughput, not accuracy, civilian risk or better judgment.
- Operators still need time, trustworthy inputs and audit records to check what systems surface.

Its 7 sections, what it means for you and what to watch are on [the page](https://realalpha.app/v/nPsqoXabzzc/#analysis), free with an account:

- [11:12](https://realalpha.app/v/nPsqoXabzzc/?t=672) Maven's reported reach exceeds its public proof
- [14:00](https://realalpha.app/v/nPsqoXabzzc/?t=840) Faster cycles can increase decisions, not judgment
- [16:39](https://realalpha.app/v/nPsqoXabzzc/?t=999) AI made human judgment the slowest step
- [18:17](https://realalpha.app/v/nPsqoXabzzc/?t=1097) Ukraine exposed how quickly recognition can fail
- [24:26](https://realalpha.app/v/nPsqoXabzzc/?t=1466) Doctrine contemplates AI action under human oversight
- [27:59](https://realalpha.app/v/nPsqoXabzzc/?t=1679) Rapid updates do not establish reliability
- [37:45](https://realalpha.app/v/nPsqoXabzzc/?t=2265) Labels and assessment still limit reliability

## Summary

Katrina Manson is the author of Project Maven: A Marine Colonel, His Team, and the Dawn of AI Warfare. Her reporting examined the Pentagon project, Maven Smart System and the military's use of AI in targeting.

Manson traces Project Maven from a 2017 Pentagon effort to apply AI to military data into Maven Smart System, which she describes as linking intelligence, target selection and weapons workflows. She says supporters see faster, better-informed decisions, while critics and some military officials worry that compressed timelines can encourage operators to trust machine outputs and pave the way for automation. Her reporting highlights model failures in unfamiliar battlefield conditions, data-labeling problems and unresolved questions about auditing decisions and civilian harm. She says the key open question is whether digitization gives operators better information to check or simply speeds up targeting.

### Minute by minute

- [1:34](https://realalpha.app/v/nPsqoXabzzc/?t=94) **Maven began as a push for AI.** Manson says Project Maven began at the Pentagon in 2017 to bring AI—especially computer vision and natural language processing—to military data, including drone video from the Middle East and Afghanistan. She says its founders also wanted the Pentagon to treat software as a core military capability and prepare for warfare that could move faster than humans think.
- [3:20](https://realalpha.app/v/nPsqoXabzzc/?t=200) **Maven links data to targeting.** Manson describes Maven Smart System as a digital map that draws on more than 150 data sources to show forces and surface potential targets. Its target workbench can suggest a weapon and send an execution message through the system. She says AI is used in the find and fix stages and may increasingly be used in the finish stage.
- [7:02](https://realalpha.app/v/nPsqoXabzzc/?t=422) **Operational details remain classified.** Manson says operational use happens on classified networks, while some intelligence sits at even higher classification levels. She says early restrictions made it difficult to obtain details about Maven through Freedom of Information Act requests, leaving journalists and the public with an incomplete picture.
- [8:35](https://realalpha.app/v/nPsqoXabzzc/?t=515) **Supporters prize faster targeting.** Manson says supporters wanted troops to retrieve intelligence more easily and to speed up targeting. She cites figures that computer vision raised capacity from under 100 targets a day to 1,000, while LLM-assisted processes were projected to reach 5,000. She says CENTCOM reported that the system was used for the first 1,000 targets in Operation Epic Fury's opening 24 hours and for strikes on 13,000 targets.
- [11:28](https://realalpha.app/v/nPsqoXabzzc/?t=688) **Faster decisions raise oversight risks.** Manson says critics worry that decision-support systems could normalize autonomy and encourage operators to trust screen outputs over their own judgment. Retired Gen. Jack Shanahan warned that operators might use saved time to make more decisions rather than deliberate longer; Manson also flags unreliable outputs and pressure on legal review. She cites NGO-reported hundreds of civilian-casualty reports submitted to CENTCOM that had not yet been adjudicated.
- [15:45](https://realalpha.app/v/nPsqoXabzzc/?t=945) **Human judgment becomes the bottleneck.** At the 18th Airborne Corps, Manson says a six-step decision cycle originally had a human at each step; after AI took over three data-related steps, human judgment became the slowest part. She says the military might see that as a bottleneck, but it is also where the law of armed conflict is considered. Battle-damage assessment remains difficult, including for computer vision trying to identify objects after they are destroyed.
- [17:43](https://realalpha.app/v/nPsqoXabzzc/?t=1063) **Ukraine exposed model limits.** Manson says computer vision initially struggled to identify Russian tanks in Ukraine because models had been trained on desert and jungle imagery; reported accuracy sometimes fell to 30% and in some cases 10%. Retraining on tank images improved results, but mobile missile launchers arranged in semicircles looked like trucks from above, and operators did not know whether they could share that context with vendors. She says US information passed through Maven helped Ukrainians target 267 items in one day.
- [20:44](https://realalpha.app/v/nPsqoXabzzc/?t=1244) **Auditability and human control remain uncertain.** Manson says Maven is supposed to have audit trails, but she cannot verify how well they work and has found errors and outdated or mislabeled entries in military intelligence databases. She says the Joint Targeting 360 doctrine, revised in April 2026, envisions AI initiating actions under human oversight, while acknowledging unresolved moral and legal dilemmas. She says accountability depends on preserving records of what information informed each decision.
- [27:00](https://realalpha.app/v/nPsqoXabzzc/?t=1620) **Palantir gained influence through integration.** Manson says Palantir's close work with operators let it adapt Maven Smart System to their workflows; she reports 50 updates in the first few months of Russia's invasion of Ukraine. After Google employees protested the company's Maven work, Google did not renew its Pentagon contract, and Manson says the department turned to Palantir. She notes that a senior CENTCOM official publicly gave the system a C plus in 2024.
- [30:15](https://realalpha.app/v/nPsqoXabzzc/?t=1815) **Defense vendors contest usage limits.** Manson says Anthropic set red lines against fully autonomous weapons and domestic mass surveillance, while Pentagon officials argued that the department sets the rules for military use. She notes that Claude was included in Maven Smart System, but companies may lack visibility into how their models are used in classified operations. She says the Pentagon is seeking models that respond to defense-related commands.
- [32:38](https://realalpha.app/v/nPsqoXabzzc/?t=1958) **Maven may underpin broader command systems.** Manson says some commanders expect targeting systems to become automated and sees Maven's use in Operation Epic Fury as a chance for the military to learn. She says the Pentagon may extend the system for a possible conflict involving Taiwan and use it as a core part of JADC2, linking intelligence, operations, sensors and weapons. She cautions that connecting sensors to shooters remains difficult.
- [34:57](https://realalpha.app/v/nPsqoXabzzc/?t=2097) **Data quality constrains AI performance.** Manson says iteration depends on access to relevant data and reliable battle-damage assessments, but the Pentagon's data silos hindered Project Maven. She reports that labeling work moved from frustrated military workers, some of whom defaced images, to gig workers and then professional teams; a labeling contract she describes as $800 million includes Maven data. She says a mislabeled image can contaminate the pipeline and undermine model reliability.
- [38:48](https://realalpha.app/v/nPsqoXabzzc/?t=2328) **Minab and agent use remain open questions.** Manson says she wants a clear account of Maven's role in Operation Epic Fury and the strike on a school in Minab, including how the Pentagon assessed any error and what it changed. She also wants to know how LLMs and agents are used in classified workflows and how operators are trained to check model outputs. She says the department is using something like 100,000 agents at the unclassified level, but she does not know how agents are used in classified operations. Her central question is whether digitization improves the information operators can verify or just speeds up existing problems.

## Transcript

Future of Life Institute Podcast, hosted by Gus Docker, with Katrina Manson.

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[0:00](https://realalpha.app/v/nPsqoXabzzc/?t=0) Project Maven was this effort that began at the Pentagon in 2017 to really find a way for the US military to start using AI. When I've seen it demonstrated to me, if you imagine something like Google Earth for war, you are in the right ballpark. So for the people who support this system, this is about speeding up what they call a kill chain, about closing that kill chain as fast as they possibly can. And AI is being used in the find stage, in the fixed stage, and may increasingly be used in the finish stage where actually something gets hit. The scale at which the U.S. is hitting things and having to make decisions, or certainly did through that period of Operation Epic Fury, I think is unprecedented.

[0:42](https://realalpha.app/v/nPsqoXabzzc/?t=42) And an accounting of actually what happened in those decisions will be really important. I would really like to know precisely what happened in the strike against the school in Minab in Iran and to what extent that relied on Maven and to what extent that relied on other tools and where the Pentagon has assessed there was a mistake and what they've done about it. What are the lessons learned from this kind of error and how seriously are they taking it internally and what changes have they made, if any? Welcome to the Future of Life Institute podcast. My name is Gus Docker and I'm here with Katrina Manson.

[1:21](https://realalpha.app/v/nPsqoXabzzc/?t=81) Katrina, welcome to the show. Thanks so much for having me. We are going to discuss Project Maven. Now, there is a massive IT system that's responsible for helping much of the U.S. military make decisions, but it's not widely known. What can you tell us about Project Maven? Well, Project Maven was this effort that began at the Pentagon in 2017 to really find a way for the U.S. military to start using AI, And in this case, particularly focused on computer vision, but not only that, also natural language processing, what has become LLMs, our chatbots that people are using, and find a way to bring AI to the vast amounts of data that the US military was collecting, mostly through drones, through drone video feeds that they were collecting over the Middle East, over Afghanistan.

[2:11](https://realalpha.app/v/nPsqoXabzzc/?t=131) on. But it was always much more than that. What I really tried to figure out in the book was the overriding impulses of the group that started at Project Maven. And at the beginning, they were trying to apply computer vision to this video feed footage, but they were also trying to change the way that America makes war to begin to understand that digital software would be as important in the views of them as hardware, as tanks, that the Pentagon would have to start buying software completely differently, but also that war was going to speed up, speed up faster than humans could think. That was the part that particularly intrigued and interested me as the big debates about AI were beginning to roil in society at large, looking at this group who had tried to lean into AI and change the nature of war and potentially the way or the character of war, I should say, and potentially the way that humans make life and death decisions on the battlefield felt very important to me. And I wanted to know exactly how AI had performed. You mentioned the system. Out of this project came a platform known as Maven Smart System. That started being developed from about 2018, 29 onwards. And today is becoming

[3:31](https://realalpha.app/v/nPsqoXabzzc/?t=211) what's called a program of record inside the Pentagon, meaning that it is a funded platform and it helps, I think the Pentagon would say, with decision making. When I've seen it demonstrated to me, if you imagine something like Google Earth for war, you are in the right ballpark. It's a digital screen. The idea is that you could see everything in one panel that shows you where your forces are, where enemy forces are, and then is able to surface new targets or potential targets through computer vision and other means. More than 150 different data sources feed in to Maven's smart system. It can also link through to targeting workflows. There's an element called target workbench. It can even suggest the right weapon to pair to the target and then send a message through the system to the weapon to go, as the military says, execute against the target to go and fire at it. So it's become this kind of all-encompassing system that really goes from sucking in the data to actually firing against a target. And so for the people who support this system, this is about speeding up what they call a kill chain, about closing that kill chain as fast as they possibly can. And AI is being used in the find stage, in the fixed stage, and may

[4:55](https://realalpha.app/v/nPsqoXabzzc/?t=295) increasingly be used in the finish stage where actually something gets hit. Can you describe when you're using this system as an operator and you're faced with making a decision, how is the system presenting information to you in ways that might push your decision making around? The way it's been described and demonstrated shows a series of suggestions and then you can go into the target workbench. It is a system that is supposed to have a great deal of transparency and auditability attached to it. The company that makes the interface is called Palantir and is also now the prime for the whole contract, which means they, although lots and lots of other companies work on Maven Smart System, Palantir is the key company. They have always talked about the tagging of information, the metadata that goes along with information. So I know it is meant to be a system that has a great deal of metadata audit logs attached to it. But this is a system that is used at a classified level. The way it's been shown to me is at an unclassified level, green. And what I was shown was a series of objects in the sea, and then being able to zoom in on them, try and check what they are. And eventually, you could make a decision to

[6:14](https://realalpha.app/v/nPsqoXabzzc/?t=374) And I'll see you next time. pair a weapon to those objects or just track them if they were friendly items and work out which port they were in. So it's an awareness system, but also potentially a firing system. I think in terms most recently, a senior Pentagon official has demonstrated this system at a Palantir conference in March this year, we can check. And that indicated actually looking at video feed footage and then being able to get that visual on it then zoom out but obviously other types of targets are what are called deliberate targets so these are things that are already in the system and chosen in advance and then the secondary type of target is called a dynamic target that is a target that is either moving or that wasn't already expected to be hit now okay how much do we know about the system and how it works this is a difficult question to answer because you don't know what you don't know, but do you have a sense of how much of this is classified information versus how much is publicly known? Well, anything operational is classified. So the system itself works at the classified level and the US fights its wars on secret networks. I think something that's an interesting tension for me is a lot of intelligence information lives at an even higher

[7:33](https://realalpha.app/v/nPsqoXabzzc/?t=453) classification level than that, top secret or higher. And so getting information into Maven smart system in order to operate on it is also part of the effort that the US military is going through. Obviously, Maven is public. So there's lots of unclassified information out there about it too. The fact that it exists, the fact that Maven smart system is now a program of record, some of the funding for it, the fact that the US Marines are using it and others are training on it. There's a lot of discussion on it. But I think there's a gap. So very early on in Project Maven, it was made impossible to foyer anything on Project Maven to do a Freedom of Information Act request for information about it. So it's not very easy for members of the public, for journalists to learn about the specifics of Maven. And some information is available and some isn't. And as you say, to some extent, you know, I don't know what I don't know, but a huge amount of it is going to come down to operational usage. And that the military very rarely talks about in public. What are the arguments for and against using a system like this? What do the supporters say? And what do opponents say? So for the people who have spent many, many years now, a decade,

[8:51](https://realalpha.app/v/nPsqoXabzzc/?t=531) trying to create the system to breathe it into life, and are incredibly proud of it, for them, I think it comes down to a few different things. One is the idea that US intelligence has always lagged people on the front lines who were actually risking their lives. And so one of the people who led Project Maven, the chief of it, who ran it day to day and really pushed for it, was called Drew Kukor. He was a colonel in the Marines, and he had been frustrated in Afghanistan. by the complete inability of the US military to really be able to check its own information, to know, as one of his colleagues told me, whether they were going to knock on the door and be greeted by cups of tea or machine guns. And so they simply weren't storing their information in a way that they could recall it. And this is a very basic digital filing system, essentially, that he had been working on for years in order simply to access information about the environment they were in. But I think it went much further to this idea that AI was going to speed up kill chains. At the beginning, they were in no way talking about targeting explicitly, often even inside the Pentagon themselves. But once it became Maven's smart system, and this group started

[10:03](https://realalpha.app/v/nPsqoXabzzc/?t=603) working with the 18th Airborne Corps, they really were trying to pair weapons to objects that became targets and hit them. And using this system, they started to get to a much higher rate of being able to hit things, which if you're in the military counts as success. And so there were some exercises that were trying to get to a thousand decisions an hour. When I went to speak to folks at the National Geospatial Intelligence Agency, I learned that with the help of computer vision, before the US military could hit under a hundred targets, I think in a day, and with the help of

[10:42](https://realalpha.app/v/nPsqoXabzzc/?t=642) **Katrina Manson:** with computer vision, they could hit a thousand targets in a day. With the help of large language models as well, because that sped up processes. For example, making the request for a legal

[10:53](https://realalpha.app/v/nPsqoXabzzc/?t=653) judgment or a legal assessment of why something may or may not be legal, they thought they could get to 5,000 targets in a day. And if you look at what the Pentagon has said about using this system in Operation Epic Fury in Iran, they've used this system for those first 1,000 targets in the opening 24 hours. They've used it to hit 13,000 targets. These are numbers that US Central Command itself has put out. So from their perspective, it's become extremely successful way of speeding up the kill chain and being able to hit more at a rapid rate. And they would argue, make sure that the commander is well informed. Now, the arguments against it are plural. They come from many different places, and I found it very interesting exploring those. Obviously, there's a kind of civil society concern from the likes of stop killer robots that it's a stepping stone towards autonomy, meaning ultimately that machines will make decisions. For the people involved in Maven, they always argued that this was a machine assisting a human and the human would make the final decision, but they don't control the future. And many of the commanders I spoke to said they expect that this system will become automated. There's also been a focus on drones,

[12:08](https://realalpha.app/v/nPsqoXabzzc/?t=728) and this system is not about that currently, it's about decision making. So for the US, they're seeking superior decision making to their adversary. For those who are concerned about this, they're worried that looking at a screen means that people are more likely to trust those outputs on screen than doing the thinking or checking back through the data themselves. Do we know? That's true. Do we have research on whether looking at targets on a screen as opposed to, say, more traditional warfare does actually make it easier to kill people? Well, the data is hard to come by. To do a really good scientific evaluation of Aben's smart system, you'd need baselines and you'd need access to that classified data.

[12:50](https://realalpha.app/v/nPsqoXabzzc/?t=770) So I have found it very helpful to speak to military ethicists who are working with some of these groups themselves. So there is a military ethicist at the 18th Airborne Corps who was brought in to help them think through the implications of what they were doing. And one of the things he raised to me was that he was worried that people would trust machine outputs over their own judgment. And the argument was made to me over and again that humans make poor judgments regularly themselves and more information would help, not less. I think the speed is another element that is introducing an element or could introduce an element of risk. I spoke with the former director of Project Maven. He is an Air Force general now retired named Jack Shanahan. And he, of course, had been a real champion for this. He's even on record saying, essentially paraphrasing, I want the Department of Defense to ensure that AI is baked into every weapon we ever make in future. But even he, after my book came out, we were discussing it, and he said on a panel at Stanford that he was worried that type A personalities in the military, instead of taking more time that Maven's smart system gave them to make a better

[14:06](https://realalpha.app/v/nPsqoXabzzc/?t=846) decision, would simply make more decisions in the time they had, and that there might be this rush to hit things rather than consider hitting the right thing. Now, when I speak to people inside the US military, they stress the doctrine, the workflow, the efforts that go into this. But I do detect that not all of these workflow issues are worked out, that AI is unreliable or mistaken results at points at which operators may not be aware. And so really checking that system and understanding how well joined up it is becomes absolutely key to deciding how far to lean into this and understanding how the nature of decision-making changes. So I think also another key element of the decision-making process from commanders is to what extent they do seek legal advice for their decisions. They have to make sure that a target is valid under the law of armed conflict, how much time a lawyer has to advise on that, and how well a lawyer understands those data systems is also being tested in real time, I think, through Operation Epic Fury. The scale at which the US is hitting things and having to make decisions, or certainly did through that period of Operation Epic Fury, I think is unprecedented. And an accounting of actually

[15:25](https://realalpha.app/v/nPsqoXabzzc/?t=925) what happened in those decisions will be really important. I doubt the military will share that publicly. It's not information I have access to. But we do know that there are hundreds probably of Thank you. reports of civilian casualties in Iran that have been submitted to CENTCOM because the NGOs say they have submitted to them, and they haven't yet been adjudicated on. So to what extent is human judgment already a bottleneck? As you mentioned, the system can present thousands of potential targets. So to what extent is the decision making of lawyers and soldiers and so on already a bottleneck here? It's a really interesting question. When I went down to the 18th Airborne Corps, they showed me their decision-making system. It consisted of six steps, each of which at the time was taken by a human. And the shortest element of time was the human decision. That was the quickest thing. But once they added AI into the system, three elements of that decision-making, those steps in that cycle, were given over to AI. This was about data

[16:32](https://realalpha.app/v/nPsqoXabzzc/?t=992) **Katrina Manson:** collection, data analysis, assessment, those kinds of things. And it changed so that the longest part of the cycle ended up being the human judgment, just because everything else got so much quicker. So I think it is at the stage where the decision is taking longer than other parts of

[16:52](https://realalpha.app/v/nPsqoXabzzc/?t=1012) the process. If what I learned at 18th Airborne Corps is being carried through to central command, which I don't know. But yes, from a military standpoint, it might be seen as a bottleneck, but it might also be seen as the most critical part of a cycle where the law of armed conflict is considered. There's also another stage, which is battle damage assessment, which is very heavily dependent on getting data back, which the US has always struggled with. And getting those kinds of things right are very important for knowing whether you waste munitions hitting the same thing again, but also whether you're risking civilian harm. And I understand from my reporting that it has always been difficult to use computer vision to get an accurate sense of whether the mission has been a success, because it's just very hard for computer vision to recognize military objects that have been blown apart. Yeah, maybe you could give some examples here of how the system might fail? Yes. So some of my reporting for the book focused on the work that the US did for Ukraine. And there, there was a team based in Germany that was using computer vision to try and find objects for the Ukrainians to hit Russian enemy objects. And in the first

[18:09](https://realalpha.app/v/nPsqoXabzzc/?t=1089) few days of the war of Russia's invasion of Ukraine, computer vision couldn't recognize

[18:17](https://realalpha.app/v/nPsqoXabzzc/?t=1097) **Katrina Manson:** the Russian tanks. It had been trained on the desert. It had been trained on the jungle in the Philippines. It wasn't used to Ukrainian, European, and increasingly snowy conditions.

[18:28](https://realalpha.app/v/nPsqoXabzzc/?t=1108) And so what they did was they took the algorithms, they took new pictures of the tanks, and they retrained the algorithms on these tanks. And so then the validity, the accuracy of the algorithms started to increase, but in the opening days of the invasion of Ukraine, US algorithms successes really plummeted. I'm told sometimes down to 30% accuracy, in some cases down to 10% accuracy. And that was one of the major findings for me in the book that in the opening days of a war, which is when AI would be hoped to help speed up a war, actually, it's having to adapt to so many new realities, it wasn't a reliable tool.

[19:13](https://realalpha.app/v/nPsqoXabzzc/?t=1153) Now, when they improved the algorithm accuracy by training it against pictures of Russian tanks, those scores started to go back up. And I still found that there were key examples where a mathematician or someone producing an algorithm wasn't sufficiently aware of how a war actually takes place. And the example I came across was that the Russians arranged their TELs, their transporter erector launchers, which are essentially mobile missile launchers, in a semicircle. Now, from above, from a satellite perspective, they really just look like trucks. And so for an algorithm searching for a military object, it would be very difficult to isolate a mobile missile launcher from a truck.

[19:57](https://realalpha.app/v/nPsqoXabzzc/?t=1197) And obviously, the intention is not to hit civilian trucks. and so the algorithm vendors didn't know about this semicircular arrangement the u.s operators did but they didn't know if they were allowed to tell the algorithm vendors so you have a kind of communication problem between humans about how to use these machines that was from the u.s perspective inhibiting their ability to actually be able to use algorithms to recognize these objects of war i found that a really instructive example overall they sped up though and And I learned in the book that on one occasion, the Ukrainians were able to target 267 items in one day because of the information passed to them from the US using MavenSmart system.

[20:47](https://realalpha.app/v/nPsqoXabzzc/?t=1247) When mistakes are made in this system, how easy is it or has it become easier to find out where the mistake was made and ultimately to assign blame? Say, for example, that you have a mistake where a civilian target is bombed as opposed to a military target. It's a really good question. So the broad things I found out are that at the time I sent in the book, there was no explicit mistake to look at. But I spoke to defense officials, former defense officials, who said AI atrocities are possible. atrocities are possible in the era of AI warfare. And when that first AI atrocity occurs, everyone will investigate it and try and understand what went wrong.

[21:41](https://realalpha.app/v/nPsqoXabzzc/?t=1301) Now, the system is supposed to have good auditability. I don't know if it does have good auditability. Something you can tell from what the Pentagon is... pursuing now is bringing AI and agents into the way they go through their intelligence stores. So at the moment, I'm confident that, or I've reported, that there have been errors in intelligence databases in the US military system. We know that because we've seen errors in the past, and I know that from some of my reporting. Things can be mislabeled in this database, things may not be updated in time. And so the idea of bringing AI to the store is that it will surface anomalies and flag people to look at these anomalies. That is work that is underway now, which means it wasn't currently in place, you know, even last year, whereas AI-assisted targeting was in place. So I think in terms of accountability, theoretically, it should be there, but I'm not the one on the inside pulling these numbers.

[22:52](https://realalpha.app/v/nPsqoXabzzc/?t=1372) Yes, I've mentioned human judgment as a bottleneck here for decision-making, and we've just talked about mistakes being made. And the system might be capable, for example, of identifying or giving information about where mistakes were made, and we might know something about to what extent human judgment is the bottleneck. Is there, do you sense political pressure to say, for example, remove human judgment or decrease the bottleneck that's constituted by human judgment and perhaps obfuscate the source of the mistakes in the system? A few things. So I don't sense any commander saying we want to obfuscate mistakes in any way.

[23:40](https://realalpha.app/v/nPsqoXabzzc/?t=1420) And I think if you look at what they say on record, they believe in the law of armed conflict and they want to make sure their systems work. There is constantly in the messages I'm given a focus on the human makes the decision, the commander is responsible. So that is the official policy of the Defense Department. However, I recently was able to review a new document that was revised in April 2026, which is called the Joint Targeting 360. This is the doctrine on how the entire U.S. forces do targeting. And there's a new section in there. There's a new chapter on the future of targeting that looks at the role of AI.

[24:26](https://realalpha.app/v/nPsqoXabzzc/?t=1466) **Katrina Manson:** and it looks to a time for the first time that I've ever seen in print where AI will initiate actions under human oversight rather than with the human making the decision. And it even says

[24:41](https://realalpha.app/v/nPsqoXabzzc/?t=1481) this throws up moral and legal dilemmas and there is a need for unambiguous guidance on that. Now that to me indicates that unambiguous guidance does not yet exist. The Pentagon is currently revising its directive 3000. 2009 on the autonomous weapons. These are weapons that could theoretically select a target, identify a target, select a target, and then strike a target without any human intervention. At the moment, the language is that that can happen following review with appropriate levels of human judgment over the use of force. So that already suggests supervision or oversight rather than actually making a human decision at the point of firing. That's different from Maven's smart system. That one should understand to be something like a drone, but ultimately it could become a digitally connected decision-making system. And so there's real tension there in the system about the extent to which that is going to happen. There is obviously huge ambition to speed up and scale hitting things. I have not come across anyone who is hoping to do that in order to obfuscate information. But I have come across experts who are within the military who argue that while AI should make things more accountable, more transparent, it's really

[26:12](https://realalpha.app/v/nPsqoXabzzc/?t=1572) important how it's implemented. Because if you have commanders or targeters at the edge who can turn on or off the record of information, that information could be lost to history. Now, I don't know how that works, but I know it's really important that if you're going to rely on a system like that, you can audit it, that you can pull the data and find out exactly what decision was made when, based on what information, and whether that information was checked, and whether there's anything automatic in the system that surfaces anomalies, or if you have to go into the system and look for those anomalies. My sense is that that workflow is not yet worked out.

[26:49](https://realalpha.app/v/nPsqoXabzzc/?t=1609) The joint targeting doctrine document that I reviewed is looking to a future where AI will make decisions, but clearly people are expecting that day to come. Can you describe how the system is developed in a partnership between the US government and US tech companies and how they share responsibility and ultimately share power in the system. Are there any tensions between them and how are those tensions resolved? Well, the department contracts with different commercial companies and sets them the requirements and then the companies deliver. So that's the traditional way of doing it. I think what's been so interesting about the way Palantir in particular has worked on Maven Smart System, because they really are the key vendor here is that they have sat with the operators. That has given them tremendous insight into how the workflow actually goes when you are at the edge making these decisions. My understanding is that it's enabled them to tailor the way their software works around the actual process that is

[27:54](https://realalpha.app/v/nPsqoXabzzc/?t=1674) **Katrina Manson:** used and to make updates. For example, in the book, I report that over the course of the first few months of Russia's invasion in Ukraine, Maven's smart system was updated 50 times. Now, that is

[28:06](https://realalpha.app/v/nPsqoXabzzc/?t=1686) because operators and vendors were sitting close, were understanding each other. And each time the system wasn't good enough, it could be updated or molded to the new reality. And this is a system where even senior officials at US Central Command have graded Maven Smart System publicly a C plus. That was in 2024. And so in 2026, Maven Smart System is now what Alex Karp at Palantir calls the backbone of targeting. So there's been tremendous acceleration into this system, even though there are issues still, obviously, to be ironed out with it and the way it's implemented. In terms of power, I think it's an interesting idea you raise. The Pentagon, as far as I understand it, at one point was really quite desperate for anyone to build this technology for them. Project Maven was really controversial. So in 2018, Google workers found out that they were working on this system without their understanding. And several of them protested. There was a multi-thousand signature letter saying, we don't want to work on the business of war. And Google decided not to renew its contract with the Pentagon. And so at that point, the Pentagon went to Palantir and said, will you help us out? And Palantir said, yes.

[29:25](https://realalpha.app/v/nPsqoXabzzc/?t=1765) So for many years now, Palantir has been in this position where they have been able to argue that they, to some extent, saved this effort to deliver AI targeting within the department. And Palantir as a whole is doing very well in this second Trump administration. Funding has become very important for US defense funding to the company and is on the rise. And so certainly you have this interesting relationship between the Pentagon and Palantir. I think I wanted to come back to the guts of your question. My question is inspired by what happened between Anthropic and the Department of War, where there's a disagreement about which organization or who sets the limits of how these systems are used and can be used.

[30:12](https://realalpha.app/v/nPsqoXabzzc/?t=1812) And I'm wondering whether there is a similar debate happening around this system. Well, the Anthropic case is really, really interesting because as Anthropic decided that they had two red lines that they wanted to keep to in the use of their AI, which was AI should not be used in fully autonomous weapon systems and it should not be used for domestic mass surveillance, nobody agreed with what they were actually fighting about because the Pentagon said, well, we don't do mass domestic surveillance and we make the rules. we have the policy. You can't be the ones to say how something is used. And that is felt very fervently by people I've spoken to at the Pentagon. Now, Anthropics is a particularly interesting case study because their tool, Claude, was in Maven's smart system. And when Dario Amodei, the chief executive, was asked, do you have any concerns about that? He, I think, fell back on his argument that these tools should be used to defend the West and that it was a legal use case to put Claude into a Maven smart system in support of AI targeting.

[31:21](https://realalpha.app/v/nPsqoXabzzc/?t=1881) And essentially that the company has to trust that the Pentagon will follow the law, which obviously the Pentagon says it will. And so there's a real lack of line of sight from some of these companies into how their own tools are being used. It's not just me who doesn't know specifically how some of these AI tools are being used in classified environments.

[31:40](https://realalpha.app/v/nPsqoXabzzc/?t=1900) **Katrina Manson:** the companies themselves don't know how their AI is being used in classified operations.

[31:47](https://realalpha.app/v/nPsqoXabzzc/?t=1907) They can try and set guardrails in terms of policy, as Anthropic did. They can try and set guardrails inside the model, as other companies have tried to do. But ultimately, the Pentagon wants to work with models that will respond favorably to defense-related commands, which a normal model might not. So my understanding is they're experimenting with models that will be much more licentious in what they're prepared to do, but also aware that the most powerful models may also end up being the open weight models and trying to find a way to work with them as well. So it's a moment of profound shift. The Pentagon, certainly, certain parts of it made very clear that they don't want companies dictating to them what the law is. That's not the same as labeling Anthropica a supply chain risk, which is this unprecedented move that hadn't been ever done on any US company before.

[32:42](https://realalpha.app/v/nPsqoXabzzc/?t=1962) How do you foresee this system evolving in the future? For me, one of the most important conversations I had was with the commander who said, ultimately, all this stuff will become automated. And I think it's really important to see the way that MavenSmart system was used in the opening day and days of Operation Epic Fury as some thing of a test for the US military, as much as they were pursuing a very serious major combat operation, they are also always trying to improve their own technology. And so whatever it is they've learned about Maven's smart system from that, it will have application for another potential conflict that the Pentagon is always preparing for, which is the possible invasion of Taiwan by China. And the idea that Pacific Command will be using Maven Smart System, developing that, trying to get more and more weapon systems together, more and more AI rifling through the data, trying to get that auditability if it isn't up to scratch. And then I think scaling it, Maven Smart System is now at the heart of this effort to create something called JADC2. This is essentially a command and control system for the entire U.S. military. Anyone who's covered this says this is a sort of a never-never land, something that the U.S. military has been pursuing

[34:05](https://realalpha.app/v/nPsqoXabzzc/?t=2045) but not achieving. They now explicitly have a tool to try and create this MavenSmart system. I think you'll see an effort to bring together intelligence systems and offensive operation systems. I don't know how successful that will be. It will be controversial within the... Pentagon itself, but an effort to use Maven's smart system more and more, not just for individual operations, but for really the global picture that the US has of the world and how it connects up to weapon systems. The whole idea is linking sensors to shooters, which has become an in vogue phrase that is very easy to say, but actually delivering that has been very hard. And Maven's smart system, I think, is at the core of how the Pentagon wants to try and go about achieving it.

[34:54](https://realalpha.app/v/nPsqoXabzzc/?t=2094) So historically, we've seen this connection between war and the speed of technology development with the airplane, for example. Do you think it's fair to say that we're seeing an effect where when the Maven smart system is used in actual war, you get data and you get feedback that you can use to more rapidly improve the system, such that when it's deployed, it improves more quickly. I mean, that's the hope that they want this iterative loop that will suck in data, do something with it, assess whether it worked, and go back through the loop again. For me, it's been really profound that battle damage assessment still is insufficient. So if you don't know if your mission was a success, you have to make a decision if you want to do the same mission again. And AI really should be helping them determine that. And I think that may be something of a gap still, just because it is very hard to do these battle damage assessments. I'm wondering whether getting data from actual usage means that you can improve the system more quickly than, for example, just trying to improve the system outside of getting actual sort of real world data. It should do. Everything depends on implementation. And that depends on how well the teams are working together, whether they've

[36:12](https://realalpha.app/v/nPsqoXabzzc/?t=2172) connected up to the right data stores, whether they have that time for assessment. It was really interesting to me seeing how hard the people at Project Maven struggled to get access to data. The Pentagon is not a place that actually yields its data easily. One of the key fights within Project Maven was over data, and that was between two rival AI centers within the Pentagon. So data really is extremely valuable. And as several people associated with Project Maven told me,

[36:44](https://realalpha.app/v/nPsqoXabzzc/?t=2204) **Katrina Manson:** it is more valuable than the algorithm or the model. It is having access to well-labeled data

[36:50](https://realalpha.app/v/nPsqoXabzzc/?t=2210) that will change whether the US can fight wars better. Better mean more successfully from their point of view, but also within the law of armed conflict. And the examples I came across in my research for the book indicated several struggles in terms of labeling. Initially, they relied on volunteer workforces or kind of commandeered workforces within the military to do the labeling, some of whom got so fed up with what they were doing, they started scrolling expletives across the image they were supposed to label. This felt like detention for many of the workers. They moved to something that involved essentially gig work, and those individuals weren't sufficiently accurate in their labeling.

[37:38](https://realalpha.app/v/nPsqoXabzzc/?t=2258) And then ultimately, you've begun to see a real professionalization of this effort to label data. So there's something like an $800 million contract now for labeling that includes labeling data for Maven, a recognition that if you do not label your data accurately, you are feeding a system that is going to spit out answers or suggestions, not based on accurate data. And I think, you know, the extent to which the Pentagon maintains or pursues testing and evaluation, labeling at a very high level, makes sure that that labeling is of high quality and weeds out mistakes is really critical. It was put to me that just one mislabeled image can poison in the entire pipeline. It can lead to ultimately algorithmic drift, bias, all these kinds of problems that people are aware about with AI. If you have a system whose quote unquote reasoning isn't available to the operator, if you're having to trust that system more and more or expect that system to get it right, and it's being fed faulty data, either in the labeling or the actual data it's drawing on. You have a system that is not going to be reliable. Yeah, yeah. All right. So as a final question here, I would like to know that after having done the research for this book,

[38:58](https://realalpha.app/v/nPsqoXabzzc/?t=2338) what are your biggest open questions about Maven? What would you really like to know that you weren't able to find out? I would really like to know how Maven was used in Operation Epic Fury. I would really like to know the baseline for what the Pentagon considers success. I would really like to know precisely what happened in the strike against the school in Minab in Iran, and to what extent that relied on Maven, and to what extent that relied on other tools, and where the Pentagon has assessed there was a mistake, and what they've done about it. What are the lessons learned from this kind of error, and how seriously are they taking it internally? And what changes have they made, if any? That to me feels like the most urgent question one could possibly ask. And to understand what role AI and MavenSmart system did or didn't play in that should have ramifications for years, if not decades, about the way the US military thinks about using tools like this. Now, I don't know. I've reported on elements of what went wrong in Minab, which is that some information was remarked on that an intelligence analyst in 2019 made remarks about a change of usage in that site at the school, put it into a digital system that

[40:19](https://realalpha.app/v/nPsqoXabzzc/?t=2419) wasn't connected to the main database that the US uses, that feeds its own databases to draw target lists from. So if connectivity is the problem, if an over-reliance on a system output is the problem imagining all these things are fixed in the back end but actually nothing is That also is a very big question for me. And then I would like to know, how are they using large language models inside this system? And are they using agents now? Are they prepared to hand over tasks to agents to run elements of the workflow without humans doing those jobs to act as humans inside that system? And that would be a really big decision.

[41:02](https://realalpha.app/v/nPsqoXabzzc/?t=2462) They are using something like 100,000 agents inside the department at an unclassified level. I'm not aware of how they're using agents at a classified level. They've also recently tested out different AI models, not in Maven's smart system, but in something that could be a proxy for Maven's smart system. And I'd really like to know what they learned. What did they learn about the results of each model, about the prompts that they put into these models? and to what extent are operators being trained on the fallibility of these models and how far to lean in to them. I think for me, the key question is over and again, I'm told that the human makes the decision to fire. You can only make a good decision if you have good information.

[41:46](https://realalpha.app/v/nPsqoXabzzc/?t=2506) And as that targeting cycle gets compressed, I want to know what kind of information an operator is able to check themselves to dig down into. And if that process feels better and more robust than when they were using scraps of paper and swivel chairs and something extremely analog, is digitization fixing these problems or is digitization speeding up these problems? All fascinating questions. Katrina, thanks for talking with me. Thanks. Thanks for having me.