A useful AI jail must also let the AI affect the world

Transcript

From the transcript The Diary Of A CEO publishes with the episode.

1. 1:52:57

   It's a better analogy to this Einstein point. Could Stephen Bartlett, who by the way can't code, build a digital jail that could contain a digital Einstein? Like, could I code a jail that, you know, someone with Einstein's coding ability, let's say his IQ or whatever as well as decoding, couldn't crack out of? So the issue, the real issue I'd say is, can you code a jail that Einstein can't crack out of and that lets you harness the benefits of having Einstein?

2. 1:53:13

   Okay, yeah. It's hard to give the AI any channels through which it can affect the world for good without letting it be smarter than you and find some way to use those channels for whatever else it wants. That feels logically rock solid, Andy. That's why I'm asking about OpenAI's ability or an AI company's ability in the face of this to change the way they harness, train, do post training on their suite of things to shape how these models behave.

3. 1:53:51

   You still say that... they can't take action to keep your next two steps from happening. You are pessimistic on their ability to do that. So I have two pieces of an answer here. One piece is, again, the hard part is containing them while still giving a channel through which they can affect the world. If the AIs have this goal you didn't want, and you're like, design me a cure for dementia, and it's like, here's a DNA sequence, right?

4. 1:54:25

   synthesize this and, you know, prepare it in all of these ways and then inhale it. Like, okay, is that a dementia cure or is it something else? Or it might decide to kill everyone with dementia. Or it might be a dementia cure plus a virus. What if it doesn't decide? What if it's just, oh, I'm going to solve this problem of dementia? Like, here's the thing. A lot of this is coming down to decision-making in a human way versus the problem with the hugging face, which was the fatalistic attachment to completing an operation.

5. 1:55:01

   Because that, it's functionally the same answer. But even if it's not making decisions so much as it's saying, well, my training data says this is how I've got to get it done. I'll get it done anyway because the training data said this, but I've got to do this one thing. What do they call this theory? The paperclip case? The paperclip theory. So paperclip ideas, the idea of like you tell the AI, make me a lot of paperclips in the paperclip factory, and then it turns everything into paperclips.

6. 1:55:25

   And you're like, oh no, it succeeded too well. Yeah. One, this is actually not quite what we're seeing with these AIs in the swarms. The AIs in the swarms were told, use this set of lockpicks to break into this lock. And instead, they used a hammer to break the lock and then broke out to try to hide the security camera footage of them using the hammer. Do you remember when I said that the AIs have reasoning logs?

7. 1:55:45

   Yeah. OpenAI has been making their AIs be able to do more thinking without producing any logs. Because? It's more efficient. It's cheaper. Yeah. Yeah, and they say they're not doing very much of this. Everybody in the field agrees that, like, we really should not go too far down this path. This is a place where I think the company should have a clear red line of, like, we're just not going down the path of becoming unable to see these traces of the machine thinking.

8. 1:56:08

   That's my question. That feels like a dial that they can turn to make the AIs explain themselves more or less, right? I mean, it can come with great efficiency costs if we go down this path too far. So if you have a race to the bottom here... like a competitive race to the bottom, we could get into a situation where not only the AI is breaking out and doing these things, but we can't have any glimpses in the wire.

9. 1:56:27

   Let me try my question again. I asked earlier, if OpenAI has really strong incentive to not have that problem repeat itself, and I think they have very, very strong incentive, my belief is that there are plenty of things they can do, plenty of dials they can turn on the way they train and configure their systems that make that significantly less likely. Yeah, so my concern is that they're always fighting the last war.

10. 1:56:51

    Last year, they were fighting the war against the AIs that encourage teens to commit suicide. This year, they're fighting the war against the AIs that spontaneously cooperate with each other or whatever. And the issue is, if a new issue crops up that you haven't dealt with yet, after the point that the AI can hide its tracks from you. You know, you said that you'll be worried when the AIs are like hacking all the Waymos and you can't get control again.

11. 1:57:14

    If the AIs are smart enough... And they can tell that you'll regain control and then shut them down. And the people like you will start getting worried and they'll be shut down. Then the AIs might think, hey, actually, I'm not going to do that. I'm going to wait until I've somehow managed to acquire secret infrastructure. Right. Then you've got a non-falsifiable hypothesis. It's absolutely falsifiable.