Try Pro free

MTS · 5 min · Nov 12, 2026

AI timeline forecasts need measurable real-world outcomes

Ege Erdil argues that AI 2027-style forecasts often lack tangible outcomes that could prove them wrong.

In this episode

Full analysis
  1. 0:59Frontier labs may struggle to sustain high margins5 min
  2. 9:36Long-term compute commitments could put lab equity at risk3 min
  3. 16:20Compute and data constrain how fast AI research can acceleratePro pick 3 min
  4. 21:05AI timeline forecasts need measurable real-world outcomes5 min
  5. 12:43Profitable memory stocks look safer than frontier labs to Erdil3 min
  6. 6:09Low switching costs weaken AI labs’ pricing power3 min
  7. 25:34Erdil sees the METR report as weak evidence of AI acceleration2 min
“it has to be something with like real world sort of physical consequences”
Ege Erdilat 25:23 · MTS
25:23

Takeaways

  1. Abstract capability metrics can be hard to falsify.
  2. Forecasters should state their own testable predictions.
  3. He favors concrete consequences over capability metrics alone.

About this segment

Ege Erdil argues that AI 2027-style forecasts often lack tangible outcomes that could prove them wrong. He says forecasters should define concrete, physical-world tests themselves.

Ege Erdil

Ege Erdil is a research engineer at Google DeepMind; former Mechanize cofounder and Epoch researcher.

Co-authored research on algorithmic progress in language models and computer vision, and studies AI forecasting and automation.

  • Theo Jaffee, Host · Co-founder & host, MTS

    One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.

  • Sophia Dew, Host · Host, MTS

From “Why Frontier AI Labs Could Go Bankrupt | Ege Erdil”, Nov 12, 2026. Watch the full episode

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 21:10

    I, Ege, was right and Daniel Kogutazer was wrong? I would want to know that because it's not clear to me. So today, Matthew, who was one of my co-founders, tweeted out asking people, so what is the thing we have to see in the real world for the people claiming there will be an intelligence explosion in 2027 or 2028 for their views to have been considered falsified? And I don't think if you can look at like AI 2027, you can look at the entire year of 2027. There's almost no like physical real world thing that happens in the entire 207, which is predicted, you know, in the story as a consequence of the internal explosion. The things that are predicted are these abstract metrics, like uplift, like compute multipliers, like whatever. But they're not like something you can actually see in the physical world. And so I am quite confused. Like, I personally don't know what these people are even

  2. 22:10

    predicting about a physical world to me it seems like they're saying look basically nothing happens in the physical world as a result of this until the moments we're all dead and of course if that's your story then of course like we won't be able to settle anything before all of us are dead but you can always say oh it's you know it's going to happen in the future or something and i mean it's maybe slightly uncharitable but like the issue is that i really don't understand how these people like could be considered to have been proven wrong i think they're not most of them are not making any concrete tangible prediction yeah so so i i also don't know like i don't know what observation would discriminate between what we think and what i think and what they think because i don't understand what predictions they're making sure yeah yeah um i

  3. 22:57

    guess i think that the the the prediction of like what will happen before we're all dead that would vindicate the ai 2027 view is like by 2029 or whatever by 2030 there are like robots all over the place doing physical world tasks yeah i think i think there's a decent chance of that happening though so i don't think by 2029 uh i mean there are already robots out there in the world doing physical tasks that already exists yeah yeah like um in industrial use cases and then maybe like unitary robots dancing in china or korea i don't know like the people have like cleaner like vacuum robots in their house yeah yeah vacuum that's like you have to actually define what they're talking about yeah yeah yeah i agree a lot of this stuff is very poorly defined and it's just difficult to define it seems it seems like kind of a fundamental problem more

  4. 23:47

    like um different criteria for bio improvements maybe that's something that's like somewhat physical um then it would be easier to like quantify uplifting capabilities but yeah yeah but i don't When I have to do this, right? Because like, I can try to figure out like, what do I predict? What do I think is likely going to happen? But it's I don't want to do that for their views. Because because I think they should be doing it for their views. Like I think me doing it on behalf of them could easily be misrepresenting what they actually think or whatever. So then they don't have to admit that they were wrong if things don't happen the way I interpreted their predictions. And just say, Oh, that's not actually what we meant or whatever. So I think they are the ones who should say.

  5. 24:30

    like here is what we expect to happen. And it's like, it's not always something that they don't, they don't do. Because like, in fact, Daniel has done this with some things before in the past. Like, for example, he said that we have this bet or he said, in three years, I will be able to, you know, describe some kind of ML experiments to whatever the best model is available at that time. And we'll just go ahead and implement it. And I won't have to like, deal with like the technical details like it will handle technical details i would just describe what the experiment i want to run is and it's just going to be able to run it and i said that's probably not going to happen i said maybe it's going to not happen in maybe like 50 chance that's in like 10 years or something and they thought it would happen in three years and i think that prediction was proven correct

  6. 25:13

    so it did happen in roughly three years um so that is the kind of thing that i would be looking for where like it's very easy to decide if it happened or not um but for the intelligent of exposure to be something that we should care about it has to be something with like real world sort of physical consequences so that's why that's the kind of prediction i would be looking for instead of predictions of products on abstract metrics which i think are less um less meaningful