AGI design should separate capability from autonomy

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:26:21

   Something I realized, basically at no point in this conversation has anybody debated whether or not we think that we'll reach AGI. But that's just kind of now taken for granted that it'll probably happen at some point. The recursive self-improvement is here. The sort of generalization, maybe it's going to need to be world models. Maybe it's not going to be transformer technology. Maybe it's something, something. But this is either it or this is the bootloader for the it that it's going to be.

2. 1:26:48

   is that kind of like in your guys's opinion agi is kind of just going to be here at some point what's likely that it's not i mean i think we're maybe already at the point where the term has lost its meaning i think sam himself has said this like if we went back to 2020 and i showed you we would have we would have called this agi yeah i mean we would have called this the thing there was a recent study where well it depends if you take demis's initial definition which was basic or was it his but like um anything that a human as broadly capable as as what a human can do and also do it in the real world so we wouldn't be anywhere close to that because we don't have the robotics right right right right yeah so robotics is a big bottleneck but uh And yeah, I mean, now I think there was a study recently where they had a Claude model try to like recreate some software that they estimated would take like two to 17 weeks for a skilled team professional. And it did it in 14 hours for $250. Yeah, I mean, I think like-

3. 1:27:47

   This is the jagged perimeter problem, which is like we may, we will always be, the incredible thing about AI is it's so good at some things and so bad at others. And it will probably continue to be that way for some time until, yeah, until true super intelligence. Is AGI, yeah, is AGI even a useful definition? Is there something else that we should be talking about now then? I like this definition that Anthony Aguirre gives, if you guys know him.

4. 1:28:14

   um it's basically like agi is the intersection of this like three-part venn diagram where you have highly autonomous so instead of artificial is basically a the a is autonomous the g the g is general um so generally capable and then i is intelligence if you have something that is yes very generally good at many different things and highly smart and it can take all those actions itself in the real world. Now you have an AGI. And his, I mean, he then goes further to suggest that we shouldn't be trying, we just shouldn't be trying to do something, one system that can do all three. That's where the danger lies. And you can have a nice general system that is also intelligent, but it shouldn't have the autonomy or take one of them out. And I think that's an area of conversation that isn't happening. Everyone's just sort of going, well, we're all

5. 1:29:10

   just doing agi and that's accepted and we should have the one the one master ring that can do everything um do we need to be doing that can we just get all of the abundance tools that we wanted without having this like one uber powerful centralized entity um right now i would slow down on the autonomy like it i i if something's really smart and it's and even across a broad range of domains that's fine as long as it doesn't have like the ability to go and just do it all by