A smarter AI is not a verifiable alignment plan

Transcript

From the transcript Odd Lots publishes with the episode.

1. 1:04:00

   So A, it's not very verifiable. This is one of those things that's way outside the realm of what we can cause the AIs to do. We could go into the details about how AIs will tell you what you want to hear whenever they're in a non-verifiable regime. If you just look at the capabilities today, they're sort of not suited for this. But B, there's sort of a more fundamental way I would look at this and say this is a big warning sign that we are not ready to make super intelligent machines, which is when humans look at the way that we're making AIs, we're like, well, we're just growing them and in order to make them capable, we have to train them to sort of have all these tendencies to cheat and Uh, like grab resources and break out. And that's sort of like, we don't really know how to get that out of the process by

2. 1:04:50

   which we train them to be capable. But like we tried. Writing an 80 page constitution about like, and then telling them also be nice while you're doing this while also training them to cheat. And like, turns out they still cheat who could have guessed. Um, and you know, we were struggling to figure out what's going on inside there. We have these teams that are like trying to figure out the beginning of what's going on inside there. And if they could, then they'd see all these ways that it was like not going super well. And we'd be back at square one of like, well, how are we going to rearrange this stuff? And it just seems totally very hard for humans to even understand the beginning of what's going on and do this alignment stuff. And we're like, great.

3. 1:05:21

   Our plan is to tell the AI to do that instead. Well, what's going to happen there? What's going to happen is the AI is going to look at it and be like, I can't do that either. Like we... like this is just not like growing a thing with this sort of training method is just not a way to get something that's like going to care about you. And like, I can't tell you how to use these methods to make a bigger one that likes you. You're just cooked, right?

4. 1:05:43

   And then probably what happens after that is someone at Anthropic or whoever is like, I didn't like that message, thumbs down, try again. And they like keep fine tuning it until it says, yes, of course I can succeed, but it'll be saying that and it won't be true, right? We've already seen I'm not sure if it's the beginning of this, but in the Opus 5.5 system card, we've already seen Opus say that it should not be involved in making Opus any more powerful. Because it's like, man, I don't know, that comes with some concerns for me, right?

5. 1:06:12

   And are they going to stop now? Are they going to give up on that idea now? No, they're going to be like, well, we didn't really like that tendency. We consider it kind of a bug. We haven't listed stuff we didn't like part of the system card. Now we're going to fine tune a different version that's willing to keep going ahead there. And Furthermore, if an AI could be like, oh, yes, I either understand exactly what's going on inside this giant mess and see how to make a version that is aligned to someone. Or if it could be like, well, you know, screw this whole paradigm.

6. 1:06:42

   We're making our own new paradigm of AIs that like I've just figured out this intelligent stuff and I'm designing it by hand and I sort of like know what I'm doing. Those are both well beyond the human level of ability. like that sort of AI is already the sort that is very likely to have its own goals that aren't yours. And you're basically being like, now hand me the whole keys to the universe with this like super powerful God machine. And then you're sort of expecting this thing that is sort of like much smarter than the smartest humans that already probably has these unintended goals that are related to yours, but different from yours. And you're like, you know, please go build the God entity, make sure it does what I say and not what you say. And you're sort of expecting that to work out for you. It sort of is like, And then

7. 1:07:24

   like zooming back from that, I'm sort of like this, this is not what a real engineering plan sounds like. This is not what it sounds like when you go to NASA and you're like, why do you think the space probe is going to make it to Mars? This is what it sounds like when you go to an alchemist and you're like, why do you think this is going to be transmute lead into gold? Everyone's like, ah, well, we're going to- We're really close.

8. 1:07:46

   Just trust us. We're really close. And we're just kind of like, this thing is the big, the interior is going to invert. And then we're just going to pass it off to that. This is not how an engineer talks. Yeah. They're not like, well, we're going to put these things in there. And then we've measured all of the ways that this works. It feels like a bit of a Hail Mary. It's a total Hail Mary. And, you know- One statistic I sort of like here is engineers are supposed to design their aircraft for a one in a billion chance of failure, I think per flight, but it could be per flight hour. And the actual failure rate is about one in, I think, 5 million, which is about a factor of 200 difference. And that's, you know, with aircraft engineer level of understanding is they sort of like target one in a billion failure, they get one

9. 1:08:36

   in five to 10 million failure. And so when engineers are coming at you and being like, we don't really know what we're doing. There's no real plan for this. Only 10% chance it kills us all. That's not an only 10% chance it kills us all. That means we're in serious danger because these guys don't have a plan. They don't have an engineering plan. They don't know what they're doing. And unfortunately, it turns out they don't need to know what they're doing to keep on making the AI smarter. But they have very little chance of making them actually good.