RSI requires models to build more capable successors

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 8:02

   So we're also going to talk about RSI and Foom. What exactly do you picture when you think of RSI? I think, you know, sort of like how cyber apocalypse means different things to different people. RSI means different things to different people. Where when some people conceive of RSI, they think of like, oh, we're already experiencing RSI because OpenAI and anthropic researchers are like heavily using models to like write experiment code for them or whatever.

2. 8:26

   and it's accelerating the returns to AI R&D to some extent, maybe 2x. Whereas when other people think of RSI, they think of it as like AI fully automates the AI R&D process and this leads to some incredibly discontinuous, like fast takeoff, hyperbolic growth in AI capability that takes us to like for real, for real superintelligence in a very short period of time. Yeah, look, so I think that both of these things are true, and the only question is of definition. So it's a good thing that you're talking to a lawyer because definition is what we do.

3. 9:07

   um i to a certain extent definitions are not particularly important right because the process as you outlined has already started and eventually i think it will be um this automated growth and capabilities that will be very fast um but for me i think i focus on the word self and self improvement right so until the models can improve themselves and in fact significantly improve themselves not just you know like train and a non-model like under supervision of a bigger model you need to be able to train a model that is your successor and to make it more capable than you are until we have that i don't think it's rsi so i think that right now we don't have rsi at least it's not been publicly disclosed and i don't think that the labs have it internally either um i think we will have it when um an ai model at open ai and or anthropic is going to be able

4. 10:20

   capable of fully autonomously not just um building the next model but also coming up with really creative ways to make the new model more capable right like big um uh kind of like ilia suits cover type like incredible brilliant moments and in a r&d that lead to a gigantic jump in capabilities and i think that's coming pretty soon how soon i mean it's hard to say i've always been um i've always thought that the most important thing for rsi is the ability for models to come up with really novel solutions to you know like random like various problems right and so what we saw up until august of this year primarily was models coming up with like brute force approaches to solve various problems like in math um and indeed like when for example i use gpt6 astra i've had a couple of uh i've kind of messed around with a couple of ciphers but were previously

5. 11:35

   unresolved like like uh from like historical like from world war one and um i kind of successfully had astra decrypt a couple of them which was really cool but every one of the results that i achieved with astra was um the model having the key but um it was it was in in In each case, it was a scenario where humans were not able to use the key for whatever reason, like the cipher was missing some characters or some characters were hard to read or or whatever, like those kinds of things.

6. 12:14

   And then Astra could brute force the solution by, like, searching for the entire possible space of possibilities and going, oh, actually, like this approach works. And here's how the cipher should read. um i have personally not been able to get astra to like come up with the key on its own based on some novel cool idea like oh you know like this person lived in modena italy and so this there's a clue here that says tower and it must be the tower that's 55 miles from his house so like let's look at that and oh that's the key you know i mean like it's like stuff like that like really big creative leaps um so all of which to say that the math results recently achieved by open ai's internal model have really updated me because um it does seem like some of these results were like involved really creative thinking um i think terence now said about the quasi reeman solution

7. 13:20

   that it was a combination of three or four existing ideas but arranged in a very unusual way it's right right so it's when you can arrange ideas in a very unusual novel way um you kind of start thinking about creativity but human genius humans exhibit um and i think that generally speaking these kinds of abilities generalize and eventually they get you creativity in AI, R&D, and then that enables the models to unlock significant new advances.

8. 13:54

   I think there's a delta between creativity and math and creativity and other things. I say this just because I think that Rune has said publicly that there's a delta. I think I've seen a tweet or two from him on that, and I believe him at his word. um so it doesn't necessarily mean that like an internal model that the open that open ai currently has is able to already achieve rsi i'm guessing probably still not but it feels like we're really not very far away um because once the models are able to do something you know how it is in 12 months or maybe sooner they'll be really really really really good at it and we'll be able to do it faster and cheaper.

9. 14:41

   So I think we're on the brink. Right.