Alignment progress could weaken incentives to honor an AI deal

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:58:10

   as a critical part of or prerequisite for coordination. What about just using leverage in a less threatening way? I mean, we have a certain amount of stored up leverage because we're not letting them have the chips. Of course, there's leakage. I read the other day that maybe hundreds of thousands of advanced AI chips have gotten to China. I mean, there's hundreds of thousands of chips in it in one Elon Musk data center, I think. So that's not necessarily a flood, the floodgates opening. But I would say, if you agree that we have preserved at least some significant leverage by virtue of the chip controls, one can imagine, you know, saying, okay, we will slowly give you access to more and more in exchange for this. Now, I might add, the day may come when China is so freaked out about this, that it's not going to

2. 1:59:08

   take a lot of threat or leverage to get them to realize that, hey, a global positive, I mean, especially if it's true that their companies are using so much distillation that, you know, they can pretty quickly, well, who knows, I guess without some significant training run, you can't catch up quite the way they have. So I'll take that off the table. But anyway, what what about the um the the more the less threatening kind of leverage because remember in in your scenario there will be growing verification so so if the concern is that oh but maybe they'll take these ships and run with them and then then dario's worth night worst nightmare will materialize and we'll all be living in concentration camps um you know presumably If you're imagining a gradual transition to greater verification and you're releasing the chips incrementally as the verification grows, in principle, you can do it that way. Is there a reason you'd rather include more threatening things?

3. 2:00:18

   Well, so ultimately, the idea behind the threats is they are the last refuge of the scoundrel. I mean, they are the last best hope that we have for, you know, like, what if China doesn't believe what we think they believe? What if they see an opportunity in particular? So here's one destabilizing element that just about like no one has thought about, but I think is we tried to bring attention to this in our first newsletter. If you get to the situation where China and the United States have a tense handshake on the basis of what we've just discussed here, and now you imagine, so part of China's adherence to any kind of agreement will possibly rest on their belief in the actual threat model that like loss of control is a real thing.

4. 2:01:09

   for whatever that's worth, the idea of making progress in AI control and AI safety, AI alignment should be pretty concerning. If China makes control in AI alignment, then that may reduce their level of concern. If they believe they've just invented an alignment or control technique, or that we've invented alignment or control technique that moves the risk of loss of control from 80% down to 20%, how low does that percentage have to be before they start to suspect, hmm, America might just renege on this agreement suddenly, try to make a break for it.

5. 2:01:41

   This is a profound source of destabilization that like no one has grappled. For a long time, the idea of alignment has been viewed as this blanket positive. And I mean, I did alignment research for a long time. Ed did too. That's why we did it. We thought that like, what could be better than just, you know, than that? But it does undermine, if China comes to the table in good faith or if some part of their coming to the table comes from that concern, it undermines that kind of bedrock.

6. 2:02:07

   And in that scenario, if for no other reason, you will want to have the ability to threaten, to be able to impose counter-media. Well, I mean, you might also want to grant them the fine-grained transparency that would reassure them, you know, let them assign an intelligence agent to live in Dario's house, whatever. I mean, you know, it's like that, I'm joking about that, but generically, that is the alternative solution is symmetrical, fine-grained transparency, which I personally prefer to threatening a nuclear power. Right. It's just that it's it's always both. Right. Like transparency is necessary. And as is enforcement in in arms control. Right. They're often referred to like these are the two sides of the coin. You have you have verification and then you have actual enforcement. And by enforcement, they mean the the nasty levers of power, because ultimately.

7. 2:02:59

   But you would expect China to do the same thing. They would say, if we if we find out that, you know, you've been doing this sneaky alignment thing that allows you to achieve super intelligence, we get this flight is into that. It's nuclear war. You would expect that threat.