# AI task horizons have doubled every three to seven months

Markdown copy of https://realalpha.app/s/ai-task-horizons-have-doubled-every-three-to-seven-months/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Garrison Lovely](https://realalpha.app/people/garrison-lovely/) · Garrison Lovely is author and journalist. · Wrote Obsolete on AI's race to replace human labor and reported on the political economy of artificial intelligence.
- Host: [Gus Docker](https://realalpha.app/people/gus-docker/) · Host, Future of Life Institute Podcast
- Show: Future of Life Institute Podcast
- Episode: [The Trillion-Dollar AI Race to Replace Us (with Garrison Lovely)](https://realalpha.app/v/NdIrGMpWOZw/), Oct 13, 2026
- Moment: 20:40 to 26:10 of the episode (6 min), playing on https://realalpha.app/s/ai-task-horizons-have-doubled-every-three-to-seven-months/; the episode from there: https://realalpha.app/v/NdIrGMpWOZw/?t=1240

## Quote

> “what the meter chart shows is that the time horizon in human terms of tasks the AIs can handle has been growing exponentially, i.e. doubling every like **three to seven months**.”
> — [Garrison Lovely](https://realalpha.app/people/garrison-lovely/) at [22:26](https://realalpha.app/v/NdIrGMpWOZw/?t=1346) · Future of Life Institute Podcast

## Takeaways

1. METR compares capability across model generations despite benchmark saturation.
2. Measured task horizons have doubled every three to seven months.
3. Automated benchmarks can overstate performance on open-ended work.

## About this segment

Garrison Lovely calls METR the least-bad benchmark for comparing models over time and says measured task horizons have doubled every three to seven months. He also cautions that benchmark tasks miss messy work requiring taste and innovation.

From “The Trillion-Dollar AI Race to Replace Us (with Garrison Lovely)”, Oct 13, 2026. [Watch the full episode](https://realalpha.app/v/NdIrGMpWOZw/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[20:53](https://realalpha.app/v/NdIrGMpWOZw/?t=1253) And I want to contrast it with paying attention to sort of economic data, because it's seems to me that there is still this wide discrepancy between any benchmark, including the meter benchmark, and then what we see in unemployment data, in productivity data, in economic growth, and so on. So where should we pay attention? I think the meter chart is the least bad benchmark that we have, if you had to pick only one.

[21:23](https://realalpha.app/v/NdIrGMpWOZw/?t=1283) and the biggest thing it does is it creates something that can be usable over time so benchmarks have this problem where you create a score from like zero to 100 and models would be like really bad at first at the new benchmark and then within a year in many cases of them starting to make progress they would approach the maximum score and the benchmark would saturate And so it's really hard to compare like GPT-3 to...

[21:51](https://realalpha.app/v/NdIrGMpWOZw/?t=1311) GPT-4 to GPT-5. And you sometimes saw journalists describing this as like, oh, like GPT-5 didn't improve on GPT-4 or something. And it's like, wait, what? No, it totally did. It's like incommensurable. And so Meter created this mechanism that lets you compare GPT-2 to GPT-5.6 to mythos um and it could continue doing this it just gets harder to as the tasks that get longer you need to find humans that can actually do like 16 20 40 hours of you know one software task and then grade them and then see if the ais can do it as well this is just very technically difficult um and yeah i think like what the meter chart shows is that the time horizon in human terms of tasks the AIs can handle has been growing exponentially, i.e. doubling every like three to seven months.

[22:53](https://realalpha.app/v/NdIrGMpWOZw/?t=1373) And it's kind of like hard to understand like how crazy this is, but one intuition pump is that from 2019 to 2026, AI progress was like, there was like, it went from like seconds to, I think like eight hours or something like that, seven hours and something. And then from like a few months period in 2026, it went up in absolute terms the same amount to reach 16 plus hours. And so like, yeah, it's just hard to wrap your mind around exponential growth.

[23:31](https://realalpha.app/v/NdIrGMpWOZw/?t=1411) But, you know, COVID was an exponential process that people underrated and that it took over the world. And I think the intuition from people who are worried about this technology is like we're in February or January 2020 and things are just going to get crazier from here. Now, the caveat to all of this is that any benchmark that can be automatically graded is going to be something that AIs are going to be better at because you can hill climb.

[24:01](https://realalpha.app/v/NdIrGMpWOZw/?t=1441) You can basically just have the AI try and do the thing and like see how well it does. isn't. A lot of tasks that we do, probably the most important tasks that humans do, are not of this quality, right? Like schooling doesn't prepare you for real work that well in a lot of ways. And these benchmarks similarly like overrate the capabilities of these models because you're given these like messy open-ended tasks in real jobs all the time. You often have to do something that's never been done before, you know, innovation and scientific research.

[24:36](https://realalpha.app/v/NdIrGMpWOZw/?t=1476) And the types of things that AIs aren't as good at, like this kind of taste, intuition, this creativity, are still things where humans, like at least the best ones, dominate these systems. And you can see this, like if you use them regularly, like I could feed my entire book into cloud code, could read the whole thing in three minutes, give useful feedback, catch things that me and my editor had missed.

[24:59](https://realalpha.app/v/NdIrGMpWOZw/?t=1499) and then you ask it, like, what are some ideas of the cover for the book? And it's 20 of the worst ideas you've ever heard. And they'll say, like, it's very important to cover not do this thing. And then five of the ideas will do that thing. So you're like, who am I talking to? This is like a weird experience. Because if a human read your book, like Goodwill Hunting, like read your whole book and gave you this feedback, he'd probably have some really good cover ideas, or at least like decent ones um and so this is the phenomena where ais can do incredibly incredibly impressive things in encoding and math and so on things that if a human could do those things that human is probably quite smart in general and can also yeah can also perhaps suggest good covers and can also perhaps write a book and so on that's not the case with ais

[25:47](https://realalpha.app/v/NdIrGMpWOZw/?t=1547) And so you see this phenomena where they are just, they can do things that very few people can do in select domains, but still they lack sort of this, they lack generality, basically.