Uncertainty can make an AI arms race hard to pause

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 17:29

   uh it the the the question that keeps coming up is you know do we take our foot off the gas uh take a you know hit the pause button um Um, and, and think about this a little bit in which case China gets it. Um, or, you know, do we keep, keep our, keep the pedal to the metal, you know? And, uh, the, the, the, so a funny thing about this book about D is that, um, uh, that question, uh, has a lot of, of parallels with what I'm talking about in, in D because, um, I didn't.

2. 18:10

   know that when I started it. So I started this series a long time ago, like 2012, and put it aside for a long time and then kind of got back into it about four years ago, four or five years, I think. And the whole AI revolution has happened during the time I've been working on it. So if If you look at—actually, there was just a big story that broke last week about the German bomb program. And because the big impetus behind the Manhattan Project was obviously the fear that the Germans might have it.

3. 18:56

   Okay, so in the early—eventually this will end up wrapping back around to AI, by the way, but it's going to take me a minute. The reactor that the Germans wanted to make to start working on the bomb was uranium cubes suspended in a vat of heavy water. So the uranium does the fission, and the heavy water is there as a moderator to slow the neutrons down. And so they needed a lot of those two things.

4. 19:34

   And in February of 1940, so it's after the invasion of Poland, but it's before the invasion of Europe, the rest of Europe, I should say, The head of IG Farben, the soon-to-be infamous giant German chemical company, sends a letter to the manager of this plant in North Carolina. Norway is the only place in the world that makes heavy water and says, we want to buy all of your heavy water now. And so this guy gets in touch with his banker, who's a French guy named Jacques Allier. And Allier notifies the deuxième bureau, which is the French intelligence service. And they talk to physicists and the physicists are like um this is this is really frightening this is really important it's hard to explain why but just take our word for it we do not want the germans to get all of the heavy water so um so that happens in february by march

5. 20:43

   Allier is in Oslo, evacuating the existing stock of heavy water out from under the noses of the Germans just right before the Germans invade Norway. And he succeeds in doing that. There's a whole story downstream of that that's in the book. But it's in June then that the Germans do invade westward and they overrun Belgium. And Belgium has the Belgian Congo, which is where uranium comes from. And the first thing they do upon getting control of Belgium is to place an order for basically all of the uranium they can get from the Belgian Congo mines.

6. 21:32

   So if you're a person in the know, in the intelligence services of France or England or the U.S., and you know that they are aggressively moving to get all of the heavy water and they're aggressively moving to get as much uranium as they can get their hands on, then there's only one conclusion that you can safely draw from that. So that's what kicks off the Manhattan Project. Um, and, and just last week they, um, they found a cube, uh, of uranium in some guy's garage somewhere.

7. 22:13

   He was a, I think a Harvard professor who had, uh, uh, it was one of the cubes that the German, uh, nuclear project made during the war that they were going to suspend in the big tank of, of heavy water. Okay. Uh, and I might get to see it in a couple of weeks. but um uh but the point being that um i told you this was going to get back to ai um if if all you know is you know germany trying to corner the market on heavy water and uranium you have no further info then the only thing you can do at that point is launch the manhattan project. And then later on, we find out, now we know that the Germans never came anywhere near making a bomb. Their reactor didn't work. They didn't have enough stuff. They were maybe screwing up the calculations because Heisenberg wasn't a great experimentalist. So it was kind of,

8. 23:14

   it turned out to be unnecessary paranoia, but you can't know that, you know, when you're in middle of the actual war. So that's kind of where we are, I think, vis-a-vis China with the whole AI thing. It's like, okay, sure, if AI is scary, we could take our foot off the gas. But then, what if there's stuff we don't quite understand that's happening in China that would allow them to seize a commanding lead.