A pause needs safeguards against government power grabs

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 51:08

   things that you think that we should be doing to mitigate the risks of human seizing power or like extreme forms of human power concentration so the key mechanism for human seizing power is the misuse of super intelligent AI systems and so there's kind of on a very high level two two things you can do about that one is you choose the alignment target for powerful AI systems such that those systems will not in fact follow instructions that would lead to power grabs and indeed you can actually you know go for an alignment target where systems will you know proactively just like a virtuous you know human employer would do um kind of flag when there might be risks of extreme power concentration flag when there might be individual humans might pose security risks to kind of really stabilize a society in which power is distributed. So, you know, that the first big class of interventions

2. 52:12

   relates to model specs, you know, how should we design the behavior of really powerful AI systems to avoid extreme concentration of power. The second is around transparency. And in particular, it's often hard to agree ahead of time what people should and shouldn't do with their AI system. So it's going to be hard to find a model spec that kind of settles every single dispute about legitimate AI use. You know, what about this type of persuasion? Is that too much super persuasion or is that just legitimate, you know, company lobbying? You know, what about this, you know, use case for cyber activity? Is that, you know, patching a system or is it actually, you know, investigating a vulnerability that you could later exploit? So I think another part of the solution is going to have to be mutual transparency, especially into how people are using

3. 52:58

   powerful AI systems. And the most important part of that is if an AI company or a government actor has access to particularly powerful systems or deploying them in particularly high stakes contexts like national security, military contexts, then I think it's really important that there's transparency into how those systems are being used. All right. And Katja, Same sort of question for you, but for the prospect of misaligned AI taking over, what would you say the top things to do are to mitigate those risks?

4. 53:28

   I think the best thing to do to mitigate a range of risks here, including sort of power concentration among humans, would be to not build AI that's much more powerful than us, at least until we're very solid on being able to align it and being able to get these other things right about what we're aligning it to, etc. And I guess in practice, it seems like, to my knowledge, we're nowhere close currently to being confident that our AIs are aligned.

5. 54:09

   And so I think near term, at least, the thing to do is to work on not building it, which is to say stopping it, pausing it, slowing it, ideally stopping it. yeah yeah tom so do you agree that yeah a pause or something would be helpful uh in the case of preventing extreme power concentration yeah brought in very sympathetic so we can take the case of um company coups and government coups separately so for company coups i think there's a very strong case that slowing down would help um it would allow the rest of the world to understand what these companies are building it would allow multiple companies to kind of and multiple countries to catch up to the frontier um and so i think the case there is very robust um for government power concentration um i think there are ways of doing a pause which would which

6. 55:05

   would help significantly with that as well i also think there are there are kind of hand-fisted ways of doing a pause which could in fact concentrate a lot of power over ai development in a kind of small group of people who are unaccountable and can abuse that power. And so there are, I think there are ways of doing a pause that actually increase the risk of government extreme power concentration. I think this has been something that I would say, you know, the community of people worried about superintelligence hasn't paid enough attention to historically.

7. 55:36

   And that's partly because they haven't had this human power concentration sufficiently on their radar. And I'm excited for people to think about, okay, let's pause, slow down AI, but let's like properly red team the plan for pausing it, assuming a worst case scenario about the government actors that they kind of inherit that new regulatory environment. And I assume that they try and abuse it as much as possible. And how robust can we make it to that attempted abuse?

8. 56:06

   Can you give us some examples of the versions of a pause that might be bad from a human's grabbing power perspective? I mean, this is going to be, you know, really a really toy example, because I haven't thought in detail about this. But, you know, if you just said the president can pass as an executive order that says that now they just get to decide on a case by case basis whether AI companies are allowed to train and deploy new models on the basis of their own personal judgment about risk, then they could essentially just get AI companies to do whatever they wanted by threatening the abuse of their arbitrary power.

9. 56:48

   they could be like look no that that ai system doesn't have the alignment i like try again i'm not allowing you to train or deploy it yet um and um then you know once a company you know comes with comes with the setup that that they like then they could say okay great now this ai system is is you know going to be helpful only for me but for everyone else will have massive guardrails yeah you're allowed to kind of go ahead and keep developing and deploying that system so that that's a setup which i think um you know could could increase the um the risk of government power concentration by contrast with a scenario where that there's an ecosystem of you know third party auditors that aren't you know part of the executive branch they can't be fired um at the whim of the president and whose recommendations are a requirement for

10. 57:37

    blocking a deployment or whose kind of safety approval is a requirement for going ahead with deployment, that would distribute much more the power over AI development. And I also think it would be much better from an alignment perspective, because you just get more open debate, more expertise going into the decision. I would add to that, that like, to the extent that, you know, the people involved maybe don't even know to what extent what they're involved in is human power grab versus humans being manipulated by AI's power grab. That, you know, there are plausibly cases where the executive, you know, does something bad, where it is actually an AI power grab causing it. And so like, if you think the AI is misaligned, then you probably also want to avoid these more, someone has the ability to grab power

11. 58:38

    and run away with it situations from the perspective of avoiding AI power grabs. Right, so the idea there is if we put one person in a position where they can hit the green light on let's deploy everywhere, then that gives the AI now a really easy route to seizing power because they just have to convince this one person that things are safe and that it's in their interest to, you know, hit the green button, then suddenly the air is going to be deployed in the military.

12. 59:07

    It can easily seize power. Yeah. Not having thought about this much at all.