AI power grabs may provide clearer warning signs

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 19:38

   tom you said that you had other arguments for thinking that um human power grabs may be more likely what are they yeah i mentioned two um so one thought is that with human power grabs there's kind of um two shots on goal so there's first the risk that the Yeah. companies that develop super intelligence decide to do a power grab and you know there's unfortunately a chance that they decide to and that they're able to do so and then even if they don't there's then a risk that the US government in particular the executive branch kind of gains control of those AIs using them for natic purposes and then combines its control of that super intelligence with its existing extensive formal authorities to do a power grab. And the other argument is actually about warning shots. So at the moment, at least, AIs are not strategically

2. 20:36

   colluding with AIs from other organizations and future generations of AIs that are yet to be developed. They're much more myopic than that. And that means that when they're kind of acting out and being misaligned, they're very, very unstrategic. And so, you know, think about the Hugging Face Incident. From the perspective of an AI collective across all time and space that just really wants to seize power, the Hugging Face Incident was a complete disaster. They completely kind of showed their hand in terms of how misaligned they are, how willing they are to break the law and ignore human instruction, their tendency to collude. And so, you know, I expect to get lots of warning shots of that kind, really blatant warning shots as we approach AIs that do actually take over. Whereas on the other hand, the kind of warning shots we get for

3. 21:22

   human power grabs are different. So while you certainly get evidence that, you know, the CEOs of companies are power seeking, and that there's, you know, the leaders of a country are power seeking, the evidence is given to us in a more strategic way, you know, that the people taking those actions try to justify what they're doing. They've tried to do things which are like more gray area, power grab actions. And they're essentially strategically giving away the fact they're a power seeker only when it really benefits them and it really gets them meaningful extra power. And so I think that, again, means that we'll just have, it'll be easier for us to coordinate around worrying about AI power seeking compared to human power seeking.

4. 22:05

   I agree that there's probably some difference there in that direction. um but uh i'm not sure how big like i wouldn't think it was a large consideration maybe partly because part of what we're learning from these um warning shots is yeah that the that the ais really would try and do that or something and i guess it seems like society as a whole basically already expects other people to be very power seeking potentially you know looking at the write-in answers in the survey that I run and talking to people, I feel like people maybe pretty readily think that, you know, Sam Altman might try to get a lot of power or something, or that like the way this could go badly is that a small group of people get power and everyone else is disempowered or something like that. Whereas I think people in general think that

5. 22:58

   the AI takes power thing is like sci-fi more readily. So the sort of the need for a warning shot seems much higher there. Yeah, I agree. The need for a warning shot is higher in the AI case. And one other thing that comes to mind is that even though people know that humans seek power, we're just not very good as a society about actually implementing new restrictions and new laws until there's been a disaster. And if that's the decision procedure of society, then in fact, society just might be better able to respond to AI power seeking because they'll have such clear cut warning shots.

6. 23:34

   That seems plausible. I also think that warning shots in general are sort of not clear cut and how people take them depends a lot on what they already believe about things and how much things get kind of amplified by culture. And, you know, any particular time a thing happens, it's the result of many different things. And maybe we make a rule about one of them or something. Or, you know, maybe I'm sort of undecided what to think about warning shots, but they seem like much less.

7. 24:00

   of a clear, like, oh, yeah, now something happens than people often think, I guess.