# Accurately labeled data matters more than the model

Markdown copy of https://realalpha.app/s/accurately-labeled-data-matters-more-than-the-model/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Katrina Manson](https://realalpha.app/people/katrina-manson/) · Katrina Manson is author of Project Maven: A Marine Colonel, His Team, and the Dawn of AI Warfare. · Investigated the US military's AI use and chronicled the Pentagon's Project Maven program in her book.
- Host: [Gus Docker](https://realalpha.app/people/gus-docker/) · Host, Future of Life Institute Podcast
- Show: Future of Life Institute Podcast
- Episode: [Can Humans Still Control AI-Assisted Kill Chains? (with Katrina Manson)](https://realalpha.app/v/nPsqoXabzzc/), Sep 25, 2026
- Moment: 35:53 to 38:53 of the episode (3 min), playing on https://realalpha.app/s/accurately-labeled-data-matters-more-than-the-model/; the episode from there: https://realalpha.app/v/nPsqoXabzzc/?t=2153

## Quote

> “it is **more valuable than the algorithm** or the model. It is having access to well-labeled data”
> — [Katrina Manson](https://realalpha.app/people/katrina-manson/) at [36:40](https://realalpha.app/v/nPsqoXabzzc/?t=2200) · Future of Life Institute Podcast

## Takeaways

1. Early military labelers sometimes sabotaged images they were assigned.
2. Gig workers did not label data accurately enough.
3. An $800 million contract reflects the value of professional labeling.

## About this segment

Katrina Manson says labeled data is the scarce asset; early labeling efforts failed, prompting an $800 million contract for professional work. Label quality can determine whether model outputs are reliable.

From “Can Humans Still Control AI-Assisted Kill Chains? (with Katrina Manson)”, Sep 25, 2026. [Watch the full episode](https://realalpha.app/v/nPsqoXabzzc/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[35:57](https://realalpha.app/v/nPsqoXabzzc/?t=2157) you can improve the system more quickly than, for example, just trying to improve the system outside of getting actual sort of real world data. It should do. Everything depends on implementation. And that depends on how well the teams are working together, whether they've connected up to the right data stores, whether they have that time for assessment. It was really interesting to me seeing how hard the people at Project Maven struggled to get access to data.

[36:24](https://realalpha.app/v/nPsqoXabzzc/?t=2184) The Pentagon is not a place that actually yields its data easily. One of the key fights within Project Maven was over data, and that was between two rival AI centers within the Pentagon. So data really is extremely valuable. And as several people associated with Project Maven told me, it is more valuable than the algorithm or the model. It is having access to well-labeled data that will change whether the US can fight wars better. Better mean more successfully from their point of view, but also within the law of armed conflict. And the examples I came across in my research for the book indicated several struggles in terms of labeling. Initially, they relied on volunteer workforces or kind of commandeered workforces within the military to do the labeling, some of whom got so fed up with what they were doing, they started scrolling expletives

[37:23](https://realalpha.app/v/nPsqoXabzzc/?t=2243) across the image they were supposed to label. This felt like detention for many of the workers. They moved to something that involved essentially gig work, and those individuals weren't sufficiently accurate in their labeling. And then ultimately, you've begun to see a real professionalization of this effort to label data. So there's something like an $800 million contract now for labeling that includes labeling data for Maven, a recognition that if you do not label your data accurately, you are feeding a system that is going to spit out answers or suggestions, not based on accurate data. And I think, you know, the extent to which the Pentagon maintains or pursues testing and evaluation, labeling at a very high level, makes sure that that labeling is of high quality and weeds out mistakes is really critical. It was put to me that just one mislabeled image can poison

[38:20](https://realalpha.app/v/nPsqoXabzzc/?t=2300) in the entire pipeline. It can lead to ultimately algorithmic drift, bias, all these kinds of problems that people are aware about with AI. If you have a system whose quote unquote reasoning isn't available to the operator, if you're having to trust that system more and more or expect that system to get it right, and it's being fed faulty data, either in the labeling or the actual data it's drawing on. You have a system that is not going to be reliable. Yeah, yeah. All right. So