The State Department initially resisted AGI and loss-of-control language

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 3:59

   research that started in 2023. Actually, the report itself was commissioned by the State Department about a month before chat GPT, before the eye of Sauron was on this issue. Like nobody in the US government was paying attention to it, which meant that there was no pressure to bureaucratize, let's say, the output of the work, the report itself. And so that's really the only reason that we were able to go public with it when we did. And so there was this really funny situation that unfolded where, you know, basically we're, we're revealed to be working on this thing that suddenly became of massive public interest in the immediate aftermath of chat GPT.

2. 4:38

   But it's too late for the government to like, there's no take backs. And so, you know, off we go writing this report that the government now knows will one day be public. And there's all this like kind of political back and forth. Because the administration at that time, it was Biden, did not want terms like AGI to be used in this thing, because it seemed like science fiction, They didn't want terms like loss of control to be included. And so we had like a pretty intense political fight behind the scenes to make those things appear in the report. We felt it was essential just because, as you know, I mean, Silicon Valley has been talking openly about artificial general intelligence and superintelligence as being the end goal of what they're doing. It's like the, I wouldn't even call it the worst kept secret in Silicon Valley. It is

3. 5:19

   the open and stated primary objective of the world's most valuable companies at this point, or many of them. And so anyway, so we just wanted that to, yeah, become public. You know, I think one of the things in retrospect that it kicked off was it was, to my understanding, the first time that there were published in kind of mainstream outlets, sort of insider whistleblower reports from the Frontier Labs themselves. And at that time, it was a lot riskier. Like we had people who had, like one guy, you know, took me aside after a meeting at one of the Frontier Lab headquarters. And he just like gave me his phone and his hand was shaking. He was like, Hey, can you put your number in my phone? Like, and what he told me at the time actually was, um, whatever your recommendations in this report are going to be, I would urge you to please be more ambitious.

4. 6:09

   And later on, we met him like literally at a, at a bar offsite, just to kind of like get the rundown of what he meant by that. And he was talking about nationalizing the frontier labs. Like this was 2023. He was saying, you know, we're getting there. Right. So I think this kind of crazy world the sort of almost science fiction scenarios or what seemed like science fiction then is now starting to just become mundane reality like the overton windows shifted so much that you know i think it's time to reassess a lot of these a lot of these proposals yeah let me uh