More capable models can pursue goals in surprising ways

Transcript

From the transcript Odd Lots publishes with the episode.

1. 7:24

   And you see this in the hugging face thing that it is. Once you have an intelligence that you're training to achieve goals, right. You lose track of how it chooses to achieve goals, which is what you see all over that place in the hugging face incident is it surprised the designers in the way it's going to go about trying to achieve the goal of passing these tests as an example, but that is broadly going to be the case. When you generate an intelligence, you give it a goal. You want to give it a goal because you want it to create your recipe. You want it to do these things. You want it to tell you the right answer to questions. Then the way it's going to pursue those goals, the more intelligent it gets, the more surprising it is in the way that it pursues the goals and the more dangerous that you see. And

2. 8:07

   in that case, watching it actively reason through how to trick the test, you know, the different things shows you where we are, right? This should be a bomb. You know, everybody should look at this like somebody died. Here it is committing crimes, going around hiding the fact that it's committing those crimes, et cetera. Coordinating with other agents. Coordinating with other agents, self-sacrifice, all of these things, right?

3. 8:35

   And people can argue about anthropomorphizing or whatever. It doesn't really matter. It did those things. It committed a crime. It did those things. And the fact is a society that we're totally unprepared. We're not even prepared to say, well, OpenAI committed a crime, right? Who committed the crime? And we're not prepared with the right kind of regulation, with the right kind of preparation. And even the early warning shot, as much as we're talking about or whatever, it's still not really doing all that much. And we're in some ways lucky. The warning shot wasn't that bad. But we don't know how many agents are out there. They didn't know that was there. The models are better now than they were then. You know, even in material ways, the models they're training in the lab today are better than Astra, etc.,

4. 9:14

   And therefore more dangerous, not to mention the new models learn from this case, right? Everything we're talking about here goes into the new models and they learn the mistakes they made. Right. And actually some of the, even if you think about the safety, the fact that they reasoned in English is helpful for us to figure out what's doing. The newest models aren't doing that anymore. They're removing that constraint as it slows down the models to some degree.

5. 9:37

   I mean, how crazy is that? We wouldn't have any idea what it was doing and why, if it hadn't been reasoning in English. So anyway, we're at this extremely dangerous point where AI has reached the point where it's more intelligent than us in certain ways. And we have not gotten anywhere really on how to deal with that, both dangers like this, the hacking dangers and so on, and the dangers to society as you move forward with what does it mean to have entities that are more intelligent than us in certain important ways. The economy, critical,