Recursive self-improvement does not require continual learning

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 48:36

   Okay. So now let's go back to recursive self-improvement. Is that a part of continual learning? Is it super continual learning? Where does that fall into it? I would think about them separately to start. There's ways that they're related, but I would think about them separately. And what I mean by that is even if we don't solve continual learning, so AI that can learn new experiences, learn from its experiences and and learn on the job.

2. 49:00

   You could still have recursive self-improvement, which is the idea that you could have one AI model create its successor, so train the next generation of AI, without any humans involved, no humans in the loop at all. The entire research loop is done just by AI. You could do that under the current paradigm, where we still have one training phase, and then you put the AI models out into the world. The way that they're related is they're sort of both paths to creating a super-intelligent AI.

3. 49:27

   As in, you could imagine that instead of recursive self-improvement, you could have a bunch of AI models out there in the world. They're working with different enterprises, they're working with different organizations, and they're picking up all sorts of new skills. And then somehow they're accumulating that knowledge and feeding it back into some more central model. That's a way that models could also get very good very quickly, even if humans are still involved in the loop of training them to some extent.

4. 49:52

   So I want to ask you about, you know, some of the tradeoffs here of having continuing learning, achieving recursive self-improvement, because, I mean, look, we talked with Stephanie yesterday. She came back from ICML, and she was telling us about how AI researchers are getting nervous that, hey, I mean, first it was AI coders that were nervous that AI could do their job. Now researchers are saying, maybe they could do my job as well.

5. 50:20

   So you have a bit of anxiety there. On the other hand, you have the issue of AI adoption and sort of some excitement that, well, maybe there's an opportunity here for continual learning and stuff like that, these advancements to improve adoption in the enterprise. To what extent do you think that achieving continual learning will help improve adoption? And to what extent do you think it might put AI researchers out of a job?

6. 50:49

   I mean, two meaty questions. Definitely. I think AI researchers are right to be a little bit nervous right now about being automated. To your point, coding is just one piece of what a researcher does in their day. They also design experiments. They come up with hypotheses. They run experiments. They interpret the results. They generate new hypotheses. I mean, that's the scientific method, right? Coding is the piece of that that's most automated right now.

7. 51:12

   You saw Anthropic recently announced that Claude is writing 80% of their code. OpenAI recently said that of their research budget, the compute that goes to research internally, the amount spent on coding has increased 100 times over the past six months. So coding is definitely the frontier here, but AI can do more and more of the whole research process. So I think AI researchers are right to be a little bit nervous.

8. 51:36

   Another way to think of this, though, is they'll be able to get a lot more done. They'll be more productive than ever. For every one hour of time that they spend at work,