Real Alpha
Watch modeSubscribed

MTS · 2 min · Oct 22, 2026

Models cover far less real-world data than their scale suggests

AS

Aidan Smith

Co-founder, Flapping Airplanes

  • Theo JaffeeHost · Co-founder & host, MTS

    One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.

  • Sophia DewHost · Host, MTS

About this segment

Smith estimates that models contain under 1% of the world’s data, citing gaps in DNA, robotics, trading and bespoke know-how. That scarcity helps explain why models struggle with specialized tasks.

From “AI Models Will Tap the Whole Economy in 2 Years | Aidan Smith”, Oct 22, 2026. Watch the full episode

“We think that we have 99% of data in the world in these models, but we really have under 1%, because there's so much individual knowledge”
ASAidan Smithat 18:04 · MTS18:04

Takeaways

  1. He estimates models capture under 1% of all data.
  2. DNA, robotics and trading have small datasets relative to web text.
  3. Company-specific knowledge remains a major gap.
Watch 2:20 from 17:22

In this episode

Full episode
  1. 6:01Whole-brain emulation has two difficult bottlenecks2:36
  2. 11:05More efficient AI could unlock demand across the economy2:34
  3. 20:22Small labs attract talent by offering meaningful ownership2:36
  4. 2:57AI algorithms should be designed around their hardware3:11
  5. 13:32A new AI lab can compete beyond verifiable code2:58

  6. 17:22Models cover far less real-world data than their scale suggests2:20

Up next · a16z PodcastAWS reserves GPU capacity to support startups4:08

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 17:27

    so when you guys launch models are you doing general purpose models are you building them for a specific use case well so the most obvious sort of data efficiency is in data limited regimes like if you were to just go and ask me like i would say there's these places where you just have almost no data and you try and train a conventional model on them and like what it barely works uh for example i'll go off the one from yesterday i think it's kind of cute like DNA. There's actually not that much DNA data that's collected out there. Like robotics, actually. Very, very limited amounts of hours of robotics data in the world.

  2. 17:53

    Like trading, actually. If you think about all the trades that have been made. Tiny, tiny drop in the ocean of data that is all LMs. And so, this being just a tiny, tiny piece of the broader pie of all the data in the world makes you realize, wow, we really are not actually covering the broader data regime. My co-founder, Ben, would like to say that. We think that we have 99% of data in the world in these models, but we really have under 1%, because there's so much individual knowledge, like how you run your show.

  3. 18:15

    Think about all the bespoke knowledge on the data to actually make these things happen or like like one thing that models are really bad at actually is writing interview questions for guests and i tried to try so hard yeah you know i try what is most load-bearing about your company yeah it's like it's a bunch of this shit it's like yeah like if if i asked it like write questions for like aiden smith of flapping airplanes it would come up with questions along the lines of like oh you know so like how do you you know plan to market your product like it's it's very basic it's very rudimentary um and the models have not improved at this at all since, you know, the first time I did a podcast interview probably three years ago.

  4. 18:50

    Yeah, I mean, one natural story you can see is that part of the spikiness means that there actually are some regressions in some places. Yeah. Where, like, the model's entropy has collapsed. RLVR collapses entropy at a large scale. You take these incredibly smart models and you pass a K at them. It's basically the same thing as pass a K. Maybe that's because we're optimized for pass a K and you just had some teleological argument you could make here.

  5. 19:09

    But, like, realistically, the thing that the model's rewarded for is somewhat of a conversion objective. And there's lots of ways you can train models to get around that. And many, many smart people have published lots of research trying to do that. But the current thing that works at scale and that people are really scaling up doesn't really follow that recipe. And I think that it's a big part of the story for the next few years of ML of like how do you maintain the diversity in these models while scaling their intelligence?

  6. 19:27

    To what extent does learning the data distribution of like web text actually teach you to learn the underlying distribution of like reality? You know, this is what like Ilya Sitzkever talked about three or four years ago. He said like basically