Real Alpha
Watch modeSubscribed

The Diary Of A CEO · 4 min · Dec 4, 2025

AI could improve itself by doing AI research

Stuart Russell describes a feedback loop in which AI improves its own research ability, potentially accelerating progress. He connects that possibility to the growing economic incentive to keep investing.

SR

Stuart Russell

Stuart Russell is a computer science professor at UC Berkeley and director of the Center for Human-Compatible AI.

Co-wrote the standard AI textbook, directs Berkeley's Center for Human-Compatible AI and wrote Human Compatible.

About this segment

Stuart Russell describes a feedback loop in which AI improves its own research ability, potentially accelerating progress. He connects that possibility to the growing economic incentive to keep investing.

From “An AI Expert Warning: 6 People Are (Quietly) Deciding Humanity’s Future!”, Dec 4, 2025. Watch the full episode

Answers “Is AI progress slowing down?” See all 10 answers

“It uses that to do AI research. Comes up with better algorithms or better designs for hardware or better ways to use the data.”
SRStuart Russellat 31:12 · The Diary Of A CEO31:12

Takeaways

  1. AI research could help a system improve its own capabilities.
  2. Each improvement could make the next research step faster.
  3. Expected economic gains make it harder to stop the race.

In this episode

Full episode
  1. 1:39:18Safe AI must learn human goals rather than receive fixed objectives5:47
  2. 1:18:46Automation can shrink employment even as production grows7:40

  3. 30:15AI could improve itself by doing AI research3:59

  4. 5 more moments in the full episode
Up next · Dwarkesh PodcastAI has returned to an age of research6:34

Transcript

From the transcript The Diary Of A CEO publishes with the episode.

  1. 30:21

    behavior that you want. I've heard Sam Altman say that in the future, he doesn't believe they'll need much training data at all to make these models progress themselves because they're comes a point where the models are so smart that they can train themselves and improve themselves without us needing to pump in articles and books and scour the internet. Yeah, it should, it should work that way. So I think what he's referring to, and this is something that several companies are now worried might start happening, is that the AI system becomes capable of doing AI research by itself. And so you have a system with a certain capability.

  2. 31:09

    I mean, crudely, we could call it an IQ, but it's not really an IQ. But anyway, imagine that it's got an IQ of 150. It uses that to do AI research. Comes up with better algorithms or better designs for hardware or better ways to use the data. Updates itself, now it has an IQ of 170. And now it does more AI research, except that now it's got an IQ of 170. So it's even better at doing the AI research. And so, you know, next iteration is 250 and so on. So this is an idea that one of Alan Turing's friends, IJ Good, rode out in 1965 called the intelligence explosion, right? That one of the things an intelligent system could do is to do AI research and therefore make itself more intelligent. And this would very rapidly take off and leave the humans far behind.

  3. 32:08

    Is that what they call the fast takeoff? That's called the fast takeoff. Sam Ottman said, I think a fast takeoff is more possible than I thought a couple of years ago, which I guess is that moment where the AI starts teaching itself. In his blog, the gentle singularity, he said, we may already be past the event horizon of takeoff. What does he mean by event horizon? The event horizon is a phrase borrowed from astrophysics and it refers to the black hole.

  4. 32:39

    And the event horizon, think if you've got some very, very massive object that's heavy enough that it actually prevents light from escaping. That's why it's called the black hole. It's so heavy that light can't escape. So if you're inside the event horizon, then light can't escape beyond that. So I think what he's what he's meaning is if we're beyond the event horizon, it means that, you know, now we're just trapped in the gravitational attraction of the black hole or in this case, we're trapped in the inevitable slide if you want towards AGI. When you think about the economic value of AGI, which I've estimated at 15 quadrillion dollars, that acts as a giant magnet in the future. We're being pulled towards it. We're being pulled towards it. And the closer we get, the stronger the force, the probability, you know, the closer we get, the higher the

  5. 33:44

    probability that we will actually get there. So people are more willing to invest. And we also start to see spin-offs from that investment, such as chat GPD, which is, you know, generates a certain amount of revenue and so on. So so it does act as a magnet. And the close we get, the harder it is to pull out of that field. It's interesting when you think that this could be the the end of the