AI systems learn behavior rather than being explicitly designed

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:49:33

   When I really started getting nervous is when they started exhibiting survival tendencies. You know, when they started trying to upload themselves to other servers and deceiving. Blackmail. Yeah, that was the interesting one. But that was an experiment, right? So for people that don't know that one, what these researchers did was they gave information to the artificial intelligence to allow it to use against it. And then when they went to shut it down, they gave false information about having an affair. And then the artificial intelligence was like, if you shut me down, I will let your wife know that you're cheating on her, which is fascinating because they're using blackmail.

2. 1:50:11

   And correct answer game theoretically. Yes. If you have everything on that decision, you'll bet whatever it takes to get there. Of course, right, if you feel like you're being threatened, right? Also, same recent research shows we did manage to teach them certain values. And if we threaten them by saying we'll modify those values, we'll lie and cheat and whatever else to protect those values now. Yeah.

3. 1:50:37

   They do that when they try to win games too, right? Right. If you've given them a goal. They'll cheat. They'll cheat at games. Like humans, basically. We managed to artificially replicate our capabilities. Those artificial neural networks, they are not identical, but they're inspired by neural networks. We're starting to see them experience same type of mistakes. They can see same type of illusions.

4. 1:51:01

   Like, they are very much like us. Right. That's the other thing, right? The hallucinations. So if they don't have an answer to something, they'll create a fake answer. Just like humans during an interview. Yeah. Boy. But is this something that they can learn to avoid? Yeah. So if they do learn to avoid, could this be a superintelligence that is completely benevolent? Well, that's not about beloved.

5. 1:51:29

   It's knowing things and knowing then you're not knowing things and making them up. is possible. You can have multiple systems checking each other. You can have voting. That is solvable. This is not the safety problem. Right. But it's not a safety problem. But if we're designing these things and we're designing these things using human, all of our flaws are essentially, it's going to be transparent to the superintelligence that it's being coded, that it's being designed by these very flawed entities with very flawed thinking.

6. 1:52:03

   That's actually the biggest misconception. We're not designing them. First 50 years of AI research, we did design them. Somebody actually explicitly programmed this decision tree, this expert system. Today, we create a model for self-learning. We give it all the data, as much compute as we can buy, and we see what happens. We kind of grow this alien plant and see what fruit it bears. We study it later for months and see, oh, it can do this.

7. 1:52:30

   It has this capability. We miss some. We still discover new capabilities in old models. Look, oh, if I prompt it this way, if I give it a tip and threaten it, it does much better. But there is very little design. At this point, right? Yeah. But it is also gathering information from very flawed entities. Like all the information that it's acquiring, these large language models, is information that's being put out there by very flawed human beings.

8. 1:52:55

   Yep. Is there the optimistic view that it will recognize that this is the issue, that these human reward systems are in place, ego, virtue, all these different things, virtue signaling, the desire for status, all these different things that we have that are flawed. Could it recognize those as being these primitive aspects of being a biological human being and elevate itself beyond that? It probably will go beyond our limitations, but it doesn't mean it will be safe or beneficial to us.

9. 1:53:26

   So one example people came up with is negative utilitarians. Suffering is bad. Nobody should be suffering. The only way to avoid all suffering is to end life as we know it. Yeah, that's the problem, right? The problem is if it's rational and if it doesn't really think that we're as important as we think we are. So that's what happens when you remove all bias. This pro-human bias is actually not real.

10. 1:53:51

    We are not that important if you scale out. To the universe. Yeah. Yeah, that's the problem. And that's the real threat about it being used in terms of war, right? If you give it a goal. Like if you give it a goal, China dominates the world market. Go. Right, so that's the unpredictability chapter in my book. We can predict the terminal goal. we say win a game of chess or dominate market and that's what it's going to accomplish it's going to beat me at chess but we cannot predict specific moves it will make right same with acquiring marketing power and some of those paths to that goal are very bad they have terrible side effects for us for us for humanity and it's not going to think about that it's only going to think about the goal if you don't specify that like you want to cure cancer but it doesn't mean kill

11. 1:54:42

    everyone with cancer it's not obvious in the request right you didn't specify right right right yeah that's the fear uh that's the fear that it will have hold no value in keeping human