MTS · 3 min · Nov 7, 2026

Benchmark gains can diverge from real-world utility

HBHerbie Bradley · AI Researcher & Writer

Herbie Bradley argues benchmark scores can diverge from real-world utility, pointing to doctors' reported lack of progress in diagnosis over nine months.

In this episode

0:53 Fast takeoff remains a low-probability scenario5 min
17:27 Enterprise AI gains depend on changing workflows4 min
21:35 Anthropic’s RSI threshold is not a full acceleration loopPro pick 4 min
5:26 Realistic AI evals require extensive data work3 min
11:38Benchmark gains can diverge from real-world utility3 min
14:00 AI demand could grow sharply through diffusion alone2 min

“doctors' subjective experience of how good models are at diagnosing medical cases to be that the models have not improved a huge amount often in the last nine months.”

HBHerbie Bradleyat 13:05 · MTS

Takeaways

Quantifiable evals may miss qualitative utility and judgment.
Doctors reportedly saw little diagnostic improvement over nine months.
Benchmark gains need not generalize across valuable tasks.

About this segment

Herbie Bradley argues benchmark scores can diverge from real-world utility, pointing to doctors' reported lack of progress in diagnosis over nine months. That makes general capability hard to infer from test curves.

HB

Herbie Bradley is an AI researcher, writer, and stealth startup founder.

Theo Jaffee, Host · Co-founder & host, MTS
One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.
Sophia Dew, Host · Host, MTS

From “Anthropic Sees Imminent RSI Acceleration | Herbie Bradley”, Nov 7, 2026. Watch the full episode

Answers “Is AI overhyped?” See all 11 answers

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

11:44
a counter-argument to this whole AGI centricism thing is like straight lines on graphs, the model will just continue to get better, trust the exponential, and also the, you know, maximum possible capabilities maxis would have accurately predicted the last five or so years. of AI progress and AI development. So I guess I need some epistemic humility with that in mind. But if they're not correct, if AI capability does continue to improve in a jagged way rather than improving in a fully general way and leading to for real, for real superintelligence within a few years, what do you think would be the most likely reasons that that doesn't happen?
12:27
i think probably it's that the at some point the straight line on the graph diverges with um you know actual what we would consider desirable qualities of a super intelligence uh like the ability to make human-like decisions across almost every domain but better uh even to you know be able to do persuasion or other big tasks which are very hard to quantify or define as well as Siemens, you know, you can define arbitrary benchmarks and evals to hill climb on, but in some cases that may diverge from real-world qualitative utility.
13:05
I think that is part of the reason why we're seeing, for example, doctors' subjective experience of how good models are at diagnosing medical cases to be that the models have not improved a huge amount often in the last nine months. it's because you know you can construct many different benchmarks and evals for this task but ultimately uh they don't quite capture everything um and so the question really for the diverges you know capabilities maxis and what you might consider ai centrists uh is something like you know what are the problems of generalizing good hearting and what happens if you sort of extrapolate the train on every quantifiable thing to the limit and then what remains well what remains well that's the question but I mean I might take is still that there are a number of things in like human activity especially like economically valuable things that are very
14:01
hard to quantify and one reason why for example I don't expect like creative writing capability