An AI crisis could start with limits on visible compute assets

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:28:34

   So, but anyway, abstracting all that away, it's a trust issue. And I agree with you that from a trust standpoint, it's absolutely symmetrical. Like there's no escaping this. This is not a situation where China's going, well, the Americans are the good guy. Like we're all the good guys in our story and everybody else is always the bad guys, right? And so this in fact is precisely why I turn to the idea of threats, assuming that the U.S. government and the president in particular and all the folks around him who need to buy in, buy in. We're in a situation where we can't trust our counterparty here. We're the ones who are going to act with agency, I'm guessing, in this situation simply because I expect us to be the recipients of this crisis before the Chinese are.

2. 1:29:19

   And so I expect, for all the reasons described, I expect the president to call Xi. And in that moment, I mean, he's going to have to, so there's not going to be any treaty verification technology on the table. It's not going to be ready yet. There's not going to be an ability, for example, to say, listen, you can keep inferencing whatever models you have. In other words, you can keep serving up the models that you've already trained and built, but you're not going to train any more advanced ones.

3. 1:29:44

   This is like this far, no further. Or, you know, you can keep going up to the scale, but beyond that scale, no good, and so on, simply because there's no way to look at a data center from space, which is what we have, basically, taking the thermals or literally just like optical imaging, and to say, hey, well, okay, this is what it's doing, this is what it's not doing. You can't know if someone's going off book to do an illicit training run using just the standard, what are called national technical means, basically just the spy craft and kind of monitoring technologies that we have to enforce other agreements that we have right now.

4. 1:30:20

   But that's going to be the technology stack that we have available in that moment. And so we're going to have to start with a really coarse-grained agreement. It's going to be economically catastrophic. It's genuinely going to sound like something like, look, if I see an AI, if I see heat signatures consistent with a training run that is above a certain scale, so help me God. It's the same thing we do with a lot of, I mean, I guess I can't quite mention the particular technology, but there are cases where we've gone to the Chinese and the Russians and said, it is untenable for us. If you fuck around with this particular thing in our, say, nuclear command control we are in nuclear war like that's the end of it it's untenable to us that you would be doing this and so therefore so we do share and do you think that the that that the logical big first

5. 1:31:07

   step is a pause on big training runs partly well partly because they're so fundamental in you know if you're going to slow things down that's the biggest lever but also because they are relatively conspicuous it's a relatively easy verification problem yep that's right verification is always going to start at the um it's a lot like regulation in the sense that you know when you think about regulating an industry you think about what's the choke point that has um the the most like difficult to hide high capex or high exp kind of high expenditure a point in the value chain you know you're not going to for example you're not going to try to like um regulate coca-cola by going to all the little mom and pop shops and say hey like you know make sure that you check every bottle to ensure there's not like a worm in it. That was like a famous bit of case law from back

6. 1:31:55

   in the God knows when, the early days of these companies. No, it makes sense to go to the chokehold in the value chain is coca-cola they're this big company they're they're taking in all this money and they're doing stuff at scale it makes sense sense for them to uh to be audited or regulated so same thing here you look at what are the big big things that can't be hidden that are really expensive so they can afford the added tax of monitoring and stuff like that um and it is it's the data centers it's the chips it's the power that that's more or less like in that moment of crisis that that is just where you start and um and this is where again the objection very reasonably gets raised. Well, like, look, like you're going to stop trillions of dollars of economic activity. There's no fucking way the president is going to do that. And to that,

7. 1:32:37

   I simply respond, raise your waterline 10 X more, keep raising it until the president literally feels like he is at risk of dying. And then we're going to see what happens. Like this is that, that, that kind of that moment of acute crisis. And so, um, in that situation, again, we've, you know, catastrophically failed to pre-invest in verification technology. As a sort of side anecdote, one of the reasons for that is if you talk to the big verification company, they're tiny, but the biggest of the verification companies that are coming online now, and you ask them, have you spoken to a single person from the U.S. intelligence community to vet or assess your technology, let alone have you talked to the people who will have to review it for kind of admission into our arsenal of verification technology formally when the president does

8. 1:33:23

   this? The answer is no. With extraordinarily rare exceptions, no one has a clue even how to get in touch with these people. That's one of the gaps that we're trying to urgently fill right now is kind of just connecting those dots. But this stuff takes literally months, more realistically years. If you look at New START, the process for approving the actual verification technologies that went into that treaty was a multi-year process. We had to sign off on it. That took couple of years. Russia had to sign off on it. So these are slow moving things at the best of times.

9. 1:33:51

   And so, again, we're back to this very coarse lens. Yes, we're going to have to stop or slow