# Closed-loop AI self-improvement raises oversight questions

Markdown copy of https://realalpha.app/s/closed-loop-ai-self-improvement-raises-oversight-questions/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Demis Hassabis](https://realalpha.app/builders/demis-hassabis/) · Co-founder & Chair, Google DeepMind · Co-founded Google DeepMind, now its Chair, whose breakthroughs include AlphaGo and the protein-folding AlphaFold.
- Host: [Peter Diamandis](https://realalpha.app/people/peter-diamandis/) · Founder, XPRIZE · Co-founded Singularity University and founded Zero-G; co-wrote the books Abundance and Bold.
- Host: [Salim Ismail](https://realalpha.app/people/salim-ismail/) · Founder, Open ExO · Co-wrote Exponential Organizations; was Singularity University's founding executive director; sold Ångströ to Google.
- Host: [Alexander Wissner-Gross](https://realalpha.app/people/alexander-wissner-gross/) · Founder, Reified · Has backed and advised 40+ tech companies, directly and through 021T Capital and Reified; writes The Innermost Loop.
- Show: Moonshots
- Episode: [Jensen’s Departure, NVIDIA’s succession plan, AI doomer response, and Demis talks RSI | #311 (Live)](https://realalpha.app/v/GZrDapFXk0I/), Nov 13, 2026
- Moment: 57:44 to 1:04:14 of the episode (7 min), playing on https://realalpha.app/s/closed-loop-ai-self-improvement-raises-oversight-questions/; the episode from there: https://realalpha.app/v/GZrDapFXk0I/?t=3464

## Quote

> “The next phase, which I think **carries the most risk**, is closed RSI, closed loop”
> — [Demis Hassabis](https://realalpha.app/builders/demis-hassabis/) at [58:25](https://realalpha.app/v/GZrDapFXk0I/?t=3505) · Moonshots

## Takeaways

1. Today’s coding gains still rely on human review.
2. Closed-loop RSI removes that reviewer from the process.
3. The panel disagrees on how quickly labs are approaching it.

## About this segment

Demis Hassabis distinguishes today’s human-supervised coding gains from closed-loop recursive self-improvement. The panel debates whether human oversight can keep pace with automated training.

From “Jensen’s Departure, NVIDIA’s succession plan, AI doomer response, and Demis talks RSI | #311 (Live)”, Nov 13, 2026. [Watch the full episode](https://realalpha.app/v/GZrDapFXk0I/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[57:52](https://realalpha.app/v/GZrDapFXk0I/?t=3472) So we've seen the enormous progress with math and with coding, and coding's obviously become enormously valuable, and everyone is having to do that now. But I do worry about it, because in the limit, so there's different forms of RSI. Right now, we're in the early days of RSI, I would say, where what's happening is, and you're probably all experiencing this as well with your teams, is, you know, it's making a human engineer maybe three, four, five times more productive.

[58:22](https://realalpha.app/v/GZrDapFXk0I/?t=3502) Okay, so, but the human's very much still in the loop and reviewing code and architecting things and writing code still, but it's all just enhanced. The next phase, which I think carries the most risk, is closed RSI, closed loop, where you basically manage to somehow figure out how to do it without the human reviewer process in the whole procedure. And that will obviously be faster, but I think it's a very dangerous way to get to full AGI because we're likely, for two reasons, it's it's sometimes used to be referred to 20 years ago as like an intelligence explosion because you don't quite know uh how you know how do you control keep control of such a system uh such a process and then the end result will likely be quite alien it's going to be like an alien intelligence there's no reason you to think it would be sort of in the similar kind of mind

[59:16](https://realalpha.app/v/GZrDapFXk0I/?t=3556) space that uh humans are in so that will make it very hard for it to understand to up for us to understand it and maybe for it to understand us. So I think it's, you know, we should be very cautious as we get closer towards that closed loop form of RSI. And I think all the AI researchers sort of know that, but there is a sort of race to the, there's this sort of intense, obviously, competitive race to improve your coding systems. So interesting. Is the press Pressure on. Do you think any labs will actually go to closed loop RSI or is that going to be, you know, verboten?

[59:53](https://realalpha.app/v/GZrDapFXk0I/?t=3593) You know, we talked about in the early days of AI and this was in sort of the discussions that Google was putting forward. Make sure your AI doesn't have open access. Make sure it isn't available to everybody. You know, there were a number of constraints that were being discussed and then OpenAI put out chat GPT and then everybody followed suit. So the question is, when the first domino falls and the lab goes to closed-loop RSI, does everybody follow?

[1:00:20](https://realalpha.app/v/GZrDapFXk0I/?t=3620) Salim, your thoughts? I'd like to just highlight one thing that he said and push it down a particular vector, which he used the word alien, right? And I've been saying a few times that I think where we end up is we create an intelligence which is orthogonal and complementary to human intelligence, but not replicative of it. and it gives us huge extended capabilities but it's not trying to duplicate human intelligence because you've heard my rant about what the hell do we even mean by intelligence but i think the ability to do mass stuff do new discovery do radical creativity and operate in a completely different way is inherently really interesting and inherently value adding to the universe and so i think that part i really really find interesting i think we should do a lot more thinking and pondering and talking about that aspect of it I use the analogy of

[1:01:12](https://realalpha.app/v/GZrDapFXk0I/?t=3672) Google's PageRank it's a completely different way of thinking about intelligence is scanning trillions of web pages and making sense of those that's completely orthogonal and complementary not replicative of human intelligence I think that's where I'd like to push this kind of discussion around this Alex yeah maybe just to respond to Saleem I I think the word alien used in reference to super intelligence is probably fundamentally misguided i don't think we've seen a large amount of evidence really any scalable evidence that when we train foundation models that what we're actually training looks fundamentally alien it may be at times difficult to interpret but difficulty of interpretation is not isomorphic to alienness it's the equivalent of say, novel human language. Just because it speaks a different language and it's hard to translate

[1:02:11](https://realalpha.app/v/GZrDapFXk0I/?t=3731) doesn't mean that at its core it's fundamentally alien. It just means that we haven't properly deciphered it yet. And actually, I'm pretty bullish that at the end of the day, when we have our perfected superintelligence architecture, it will look strikingly familiar. It would be familiar to information theorists. At the beginning, Claude Shannon would recognize Claude. It'll look architecturally familiar, and to the extent it has anything that looks like weights, those weights will probably, I think, be easily linearly decodable to the weight equivalents in the human mind. We saw out of Jean-Marie King's group at Meta and others, even with earlier GPT series models, that if you look at fMRI activations in the human brain when people are listening to stories being read to them, and then you train a shallow linear decoder off of the fMRI voxels

[1:03:06](https://realalpha.app/v/GZrDapFXk0I/?t=3786) that you can decode. I think it was originally GPT-2 hidden activations from that to strikingly high correlations. That to me suggests, and that was only just one of the earliest studies, that probably what we'll find at the end of the day, as these models get more advanced, more sophisticated, is the exact opposite of the models being more and more alien. I think they'll actually be more and more familiar to relative to the human mind and to the extent there is any alienness at all it'll be because the universe that we're stuck in with these super intelligences maybe the universe itself has alien aspects and as we discover and more about the outer, the universe that we're both embedded in, maybe that will have alien aspects. But the way these machines, the way they think and look internally, I think they're going

[1:03:56](https://realalpha.app/v/GZrDapFXk0I/?t=3836) to look more and more human-like over time. But let's go back to the question of closed-loop RSI and do labs get there? Everyone's doing it already, including his lab. Google very publicly
