Future of Life Institute Podcast · 3 min · Sep 25, 2026

Accurately labeled data matters more than the model

Katrina Manson · Reporter, Bloomberg

Katrina Manson says labeled data is the scarce asset; early labeling efforts failed, prompting an $800 million contract for professional work.

In this episode

8:42 AI sharply increased the number of targets the military could strike7 min
17:43 Computer vision struggled when Ukraine's battlefield changed3 min
23:30 Targeting doctrine is shifting from human decisions to oversight3 min
35:53Accurately labeled data matters more than the model3 min
27:00 Maven was updated 50 times during Ukraine's first months3 min
29:53 AI companies lack visibility into classified military use3 min
15:45 AI made human judgment the longest step in a targeting cycle2 min

“it is more valuable than the algorithm or the model. It is having access to well-labeled data”

Katrina Mansonat 36:40 · Future of Life Institute Podcast

Takeaways

Early military labelers sometimes sabotaged images they were assigned.
Gig workers did not label data accurately enough.
An $800 million contract reflects the value of professional labeling.

About this segment

Katrina Manson says labeled data is the scarce asset; early labeling efforts failed, prompting an $800 million contract for professional work. Label quality can determine whether model outputs are reliable.

Katrina Manson is author of Project Maven: A Marine Colonel, His Team, and the Dawn of AI Warfare.

Investigated the US military's AI use and chronicled the Pentagon's Project Maven program in her book.

GD
Gus Docker, Host · Host, Future of Life Institute Podcast

From “Can Humans Still Control AI-Assisted Kill Chains? (with Katrina Manson)”, Sep 25, 2026. Watch the full episode

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

35:57
you can improve the system more quickly than, for example, just trying to improve the system outside of getting actual sort of real world data. It should do. Everything depends on implementation. And that depends on how well the teams are working together, whether they've connected up to the right data stores, whether they have that time for assessment. It was really interesting to me seeing how hard the people at Project Maven struggled to get access to data.
36:24
The Pentagon is not a place that actually yields its data easily. One of the key fights within Project Maven was over data, and that was between two rival AI centers within the Pentagon. So data really is extremely valuable. And as several people associated with Project Maven told me, it is more valuable than the algorithm or the model. It is having access to well-labeled data that will change whether the US can fight wars better. Better mean more successfully from their point of view, but also within the law of armed conflict. And the examples I came across in my research for the book indicated several struggles in terms of labeling. Initially, they relied on volunteer workforces or kind of commandeered workforces within the military to do the labeling, some of whom got so fed up with what they were doing, they started scrolling expletives
37:23
across the image they were supposed to label. This felt like detention for many of the workers. They moved to something that involved essentially gig work, and those individuals weren't sufficiently accurate in their labeling. And then ultimately, you've begun to see a real professionalization of this effort to label data. So there's something like an $800 million contract now for labeling that includes labeling data for Maven, a recognition that if you do not label your data accurately, you are feeding a system that is going to spit out answers or suggestions, not based on accurate data. And I think, you know, the extent to which the Pentagon maintains or pursues testing and evaluation, labeling at a very high level, makes sure that that labeling is of high quality and weeds out mistakes is really critical. It was put to me that just one mislabeled image can poison
38:20
in the entire pipeline. It can lead to ultimately algorithmic drift, bias, all these kinds of problems that people are aware about with AI. If you have a system whose quote unquote reasoning isn't available to the operator, if you're having to trust that system more and more or expect that system to get it right, and it's being fed faulty data, either in the labeling or the actual data it's drawing on. You have a system that is not going to be reliable. Yeah, yeah. All right. So