AI self-improvement raises an alignment problem

Transcript

From the transcript Inside Blackstone publishes with the episode.

1. 12:05

   You see the same, by the way, in AI. All of the gains right now are occurring in what is called, essentially, scale-free. And scale-free means that you can just keep doing it and get smarter. So, in math, I'm not a mathematician, but as best I can understand, they can just invent stuff, and they just keep inventing things, and they keep inventing things, and they're so clever. Well, now a computer can take all of those ideas, and if you just give it enough juice, it can do the same thing.

2. 12:32

   Because it doesn't need data, it's self-contained. Unlocking all these solutions. Unlocking these solutions. In software, it turns out that once you start writing code and you start learning how well the code works, you can make it better. And then you can make it better and better. And there's what is called a recursive self-improvement loop, where it's getting smarter on its own. But this progress that I'm describing is happening faster than I have ever seen, right?

3. 13:00

   What is true is each wave is bigger, but it's also faster. And we are not ready as a society. Eric, you gave me like 10 things to unpack there, but let's try to do this. So recursive self-improvement, right? This is the thing that had the Anthropic researchers, two of them, making comments, one leaving making comments, the other sort of validating that there are serious concerns about the speed with which AI is learning. Right?

4. 13:25

   So, by the way, every month there is a group of people who quit over this issue. Over the same topic, exactly. Silicon Valley, and in particular San Francisco, they're absolutely convinced that this is going to happen so quickly that it will threaten humanity in one way or the other. I personally do not agree with this. Okay, why? Well, let me explain their argument. It's a relatively straightforward argument.

5. 13:51

   As humans, we learn at a certain rate. If you have enough hardware and enough electricity and so forth and enough scale-free learning, which is what I'm talking about, it can just get faster and learn faster than humans can. And the way it works, I'll give you a simple example. I ask it to write a chapter and then it writes a chapter and I feed that chapter and say write the next chapter and so forth.

6. 14:16

   Well, if I keep iterating like that, eventually it learns what I wanted with a process called reinforcement learning. It's easier to write the twelfth chapter than the first. And it gets smarter. And once it learns how to do it, it knows. It's like the way we learn. Right. But it can learn a lot faster. So, think of it as humans versus computers. I think the actual technical answer about this learning is it's not human learning, it's a different kind of intelligence.

7. 14:46

   And so, the best way to state the fear is that we're inventing a different kind of intelligence that we cannot necessarily constrain. This is called the alignment problem. Also, in the past weeks, there was a very famous situation where there was an internal test going on at OpenAI. And this internal test, which was trying to check something, managed to create a set of bots, which are agents, if you will, they managed to collaborate to violate certain norms and laws, they managed to break into things, and they managed to, allegedly, do a little bit of harm.

8. 15:22

   Now, nobody was hurt. But it's an example of what is possible. So, we must develop both recursive self-improvement which is the self-learning along with solving the alignment problem to human values. So, a simple way would be, I say to you, learn everything and you're a computer. That's not what I should say. I should learn everything, but, don't harm any humans, follow all laws, follow all laws, follow the US Constitution, make sure you don't get sued.

9. 15:53

   You know, in other words, the set of assumptions. That you build into the model. These systems don't have a fear over the police. They don't feel feelings. And so, they have to be constrained. Come back to your optimism. It's in everything that you've written and you've said. What gives you optimism that we will be able to sort out a future where humanity wins? What's the alternative? I mean, the best way to get a headline is to say, in 10 years, we're all dead.

10. 16:23

    I was on a panel with a guy who wrote a book which is entitled, ‘If We Invent Superintelligence, We're All Dead.’ It's very hard to argue with someone in that. The only problem with his arguments is it’s wrong, right? Because part of superintelligence is getting the super alignment problem solved. I would guess that your social network might be a little bit different than mine.