# AI capability gains are hitting practical limits

Markdown copy of https://realalpha.app/s/ai-capability-gains-are-hitting-practical-limits/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Ed Zitron](https://realalpha.app/people/ed-zitron/) · Ed Zitron is a British tech critic, host of Better Offline, and founder and CEO of EZPR. · Hosts Better Offline and writes Where's Your Ed At, a prominent critic of AI industry hype.
- Host: [Steven Bartlett](https://realalpha.app/people/steven-bartlett/) · Host, The Diary Of A CEO
- Show: The Diary Of A CEO
- Episode: [The Man Who Calls BS On AI: They’re LYING About AI, 2027 Is When It All Breaks! | Ed Zitron](https://realalpha.app/v/Lf5oqGOCRCM/), Aug 27, 2026
- Moment: 1:33:36 to 1:40:57 of the episode (7 min), playing on https://realalpha.app/s/ai-capability-gains-are-hitting-practical-limits/; the episode from there: https://realalpha.app/v/Lf5oqGOCRCM/?t=5616

## Quote

> “the promises **do not line up** with the capabilities or the capability improvements.”
> — [Ed Zitron](https://realalpha.app/people/ed-zitron/) at [1:40:01](https://realalpha.app/v/Lf5oqGOCRCM/?t=6001) · The Diary Of A CEO

## Takeaways

1. Benchmark gains do not prove dependable performance on real tasks.
2. He sees limits in turning generated clips into finished films.
3. He argues autonomy needs more than training data alone.

## About this segment

Ed Zitron accepts that models improve on certain tests but says those gains have not translated into reliable, novel work. He points to video generation and autonomy as practical limits.

From “The Man Who Calls BS On AI: They’re LYING About AI, 2027 Is When It All Breaks! | Ed Zitron”, Aug 27, 2026. [Watch the full episode](https://realalpha.app/v/Lf5oqGOCRCM/)

## Transcript

From the transcript The Diary Of A CEO publishes with the episode.

[1:33:48](https://realalpha.app/v/Lf5oqGOCRCM/?t=5628) But on that last point where you say that it's going to get progressively more intelligent and when it does, it will be a danger. Yeah. Would you agree with the statement that artificial intelligence has gotten more intelligent If you measure it based on any sort of measure of intelligence one might use. It's got better on the tests that are rigged for the models. It's got better at tests where you can train for the test.

[1:34:01](https://realalpha.app/v/Lf5oqGOCRCM/?t=5641) Okay, so it's got better at... It's got better at tests that they're intentionally trained for. So if you logged the rate of improvement on a graph, it would look something like... this. Right. You agree? In terms of what it's capable of doing. Oh, there we go. Yeah. Because it's not got new features. You'll notice that outside of OpenAI and Anthropic, when you remove the coding startups, there's basically no successful AI startup company.

[1:34:28](https://realalpha.app/v/Lf5oqGOCRCM/?t=5668) So we agree that it's got better, it's got more capable At doing things. Yeah, fine. Over time, AI's got more capable. If we imagine that trajectory will continue, it will get more capable, then at some point it does cross, you know, this is what they say to me, it crosses human intelligence. And at such time... Will it not start to do some of the jobs that people are doing today? Outside of software engineering.

[1:34:47](https://realalpha.app/v/Lf5oqGOCRCM/?t=5687) Remove software. Because I will concede software engineering. It's got better at that. Outside of software engineering. Yeah. Where? So the chief of staff things, the admin. Okay. So it's got better at admin. Video generation, photo generation. Okay. Text generation, theoretically. Coding. Coding. Right. And then I'd say agentic workflows. What is an agentic workflow? So automated workflows where you're doing the same.

[1:35:09](https://realalpha.app/v/Lf5oqGOCRCM/?t=5709) I mean, a good example is looking at the backend data of the diary of a CEO. Summarizing. Looking at all of the data, ingesting all of it. Going out onto the internet and searching who Ed is. Looking at every interview you've ever done ever. This is summarizing and generating. Making a little model on the things people want to know from Ed. Producing a report, sending that to my inbox. Me getting a 20, 30, 40, 50 page report on Ed before he arrives.

[1:35:32](https://realalpha.app/v/Lf5oqGOCRCM/?t=5732) This is all basically the same thing it's been doing for years, though. It's not really new capabilities. Research, it's search. It's still the same thing. They've had web search for years. They've had report generation for years. But we couldn't generate... high quality videos that are like indistinguishable from cameras see dance and these ones that look like movies i mean they are incredible so i'm saying the point i'm trying to make is that if we imagine that over the last 10 years there has been a rate of improvement in terms of capabilities and output and quality we've seen hallucinations drop we've seen the models get more quote-unquote intelligent get better you know if you didn't give it an iq test it's getting higher scores than it was 10 years ago we agree that there's been an upward motion of improvement.

[1:36:28](https://realalpha.app/v/Lf5oqGOCRCM/?t=5788) This is pretty much how machine learning goes when you feed it more data. Exactly. And you put more compute behind it. So if this continues, what does the future look like? So the rebuttal I was expecting to hear is that it won't continue. And I actually don't think it... I think that there are hard limits that we're going to hit. So you do believe that there's a hard limit somewhere? We've kind of already hit the diminishing returns level because...

[1:36:52](https://realalpha.app/v/Lf5oqGOCRCM/?t=5812) For example, video generation, which is, by the way, far less an American concern anymore. OpenAI has shut down Sora. I think you can still use the API. But nevertheless, look around you with the amount of stuff and the crew you need to get a shot. People think the movies are just shot by shot by shot, and they just magically happen. When you've got a wonderful girlfriend of... first ADs, assistant directors, you've got gaffers, you've got lighters.

[1:37:14](https://realalpha.app/v/Lf5oqGOCRCM/?t=5834) And also, simulating light is insanely difficult. There are so many magical things that happen in creating visual images that, yeah, you could create a one-minute-long thing that might fool someone. How do you practically turn that into a movie? Because that movie, I forget what the name is, there was a movie that claimed it... aired at can it didn't no one it aired in the city of can during the can film festival it was not at the film festival when it comes to the practical creation of actual things at the end of it versus magic tricks the actual practical outcomes are not there the reason i keep coming back to the capabilities thing for the example is yeah they can do better at tests do better you number go up when it comes to can this actually do distinct tasks you can rely on it You can rely on it for summaries.

[1:37:55](https://realalpha.app/v/Lf5oqGOCRCM/?t=5875) You can rely on it for generations. The things it was doing, it's getting linearly-ish better at. But again, there's a ceiling to that. Like, okay, so it gets really good at research. What does that actually mean? You've already kind of got the automation there. What is the next step of that? Because training it to be more autonomous, for example, that's not something that comes from training data.

[1:38:14](https://realalpha.app/v/Lf5oqGOCRCM/?t=5894) That is actually a new, Gary Marcus, a neurosymbolic. You actually need to build a structure around the AI to make it work. And even then, it's... It doesn't fix the... So you're saying that there will become a point where the rate of improvement will plateau... We're already there. ...and stop. We've already hit that diminishing... Gary Marcus said this in 2022 as well. You know there's lots of people listening now that, like, they've had their workflows completely transformed by these tools.

[1:38:33](https://realalpha.app/v/Lf5oqGOCRCM/?t=5913) Have they? There'll be... Yeah, there are. Yeah, the thing is, first of all, every single one of them, did you pay for the tokens? That's the thing. Did you pay for the tokens? And also... How many tokens did you burn? But putting all that aside, what workflows? Because of it, yeah, I did a bunch of web scraping or web searches. I'm just not impressed. Did you make an entire fucking movie? No, you didn't.

[1:38:52](https://realalpha.app/v/Lf5oqGOCRCM/?t=5932) Is it speeding up your coding? Yeah, I believe that. I've heard that from multiple people. But again... How much can you trust this? I think what I was getting at is, you know, in the moment of any technological innovation, people, they extrapolate linearly or they view it as a static state, i.e. they think today is going to look like tomorrow or they think it's going to get better in this sort of straight line.

[1:39:15](https://realalpha.app/v/Lf5oqGOCRCM/?t=5955) But what we end up seeing a lot of the time is this exponential improvement. All of the innovations we're talking about with you, with compute and all that, with fast processes, those are hardware breakthroughs. The hardware breakthrough companies don't seem to be fixing the LLM problems. Despite all the king's horses, all the king's men, we're what? Nine, ten generations of TPUs from Google now. Broadcoms building stuff with OpenAI, their jalapeno chip.

[1:39:38](https://realalpha.app/v/Lf5oqGOCRCM/?t=5978) And yet none of these people can just say, yeah, we're on the path to making this profitable. Because they can't. If we fix the environmental problems and the profitability situation, maybe I'd be more generous with this stuff. But they don't seem to be able to. And you talk about these improvements in capabilities, there's a certain point at which I'm saying, okay, can it do even a tenth of the stuff they're promising?

[1:40:00](https://realalpha.app/v/Lf5oqGOCRCM/?t=6000) Samuel, when the other fucking week was saying, it was going to be in like six months, we'll be like a genie that you can ask wishes for. It's like, motherfuckers never watched Aladdin. What's he talking about? Like... Also, the genie was charming. Anyway, long story short, the promises do not line up with the capabilities or the capability improvements. An exponential improvement in software and software performance is always a result of direct hardware improvement.

[1:40:27](https://realalpha.app/v/Lf5oqGOCRCM/?t=6027) We have all the gifted mathematicians, all the gifted software engineers, all the gifted hardware engineers. And where are we? Trillion plus dollars in with... The future great financial crisis and the world's greatest marketing psyop. I just think in the future, I do think that all of the devices and the computers we use and the physical items in our world will be more intelligent.