Harris says early funders discouraged government outreach

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 23:01

   that I think is underappreciated about the kind of AI safety ecosystem of the time, there were a very small number of funders, of grantors in that ecosystem. The main one, the Mac daddy of the AI safety ecosystem at the time was open philanthropy um later or now coefficient giving but but back then open phil um and they were they were funded by dustin moskovitz one of the co-founders of facebook um and very uh kind of tied to the effective altruist cause that has become so kind of uh famous infamous nowadays that maybe we can get into i think it's an important part of this But they had this view that if you spoke to anybody in the government, what would happen is that they would hear you if you came to them and said, look, we're on the cusp of building human level AI.

2. 23:53

   We think we've cracked the code. We think the scaling thing has legs. You may not appreciate this, but Silicon Valley is talking about this as if they have. crack the code. And they're investing like exponentially more every year. Like they smell blood right now. So, you know, this is about to become a real national security problem. We do not think we'll be able to control the systems that we build. And we will get rogue AI incidents in all likelihood out of this that may be challenging for the human species at a civilizational level. If you pitch that argument to somebody in government, the open fill people argued at the time, this would trigger a very specific reaction. The US government guys would go, oh, wow, I hear you about the capabilities. That sounds really compelling. It sounds like a

3. 24:33

   really powerful weapon. I'm going to ignore all the warnings about the risk, and we're going to trigger a Manhattan Project for superintelligence that only gets humanity killed faster. Now, this is a very specific, and it was expressed, by the way, with extreme levels of confidence. There was a paranoia about, at the time, doing anything that wasn't, as they put it, robustly good. And everybody ran around, um, sort of like, uh, almost like worshiping at the altar of a small number of folks at the top of the pyramid who were typically the grantors. And, and I think there's a, there's a big psychological issue there where when you get used to doling out money, you don't realize that people start to change their tune when they talk to you to align with your preexisting beliefs. And you sort of fall into a, a well of your own making, um, kind of

4. 25:17

   surround yourself inadvertently. I mean, these are good people, but, but they surrounded themselves. I do think with a lot of folks who kind of reinforced their their pre-existing views. So were they were they literally trying to keep the government people in Washington from knowing kind of almost the central theme of their own discourse? I think that's a fair I think that's a fair, fair description with one little modification, which is there there was at the time and this is just my view.

5. 25:48

   you know there's nobody here like from their end to kind of give their explanation of it so i'll try to be as sort of even-handed as i can but i am biased of course um there was like a small number of folks at the top of that pyramid who were ordained in some sense by the ecosystem as as those who who were were responsible enough to carry the message to government in their own way and in their own time. Those individuals I had many arguments with refused to and strongly discouraged us to use the phrase artificial general intelligence. They were part of, you know, they themselves were putting forward this argument. And, you know, like the case that we were making at the time was like, look, I get that you have this concern, but right now, artificial general intelligence is not a thing that people know about. And so you have an

6. 26:38

   opportunity to get in here and shape the battle space around that terminology before it becomes politicized because it will anything that rises to high enough levels of salience becomes politicized it's that's just a fact of the matter and so this is an opportunity for you to kind of inform folks who are actually going to make decisions ahead of need and um they you know they they warned us against that they actually said look there's also this whole kind of principal agent or prime mover risk where you know whoever is the most irresponsible will do this first please please don't be that guy. And so the last thing I'll say about this, and I think this is kind of like, again, leans back into the libertarian startup founder side of who we are, which at our core is who we are. Our instinct in hearing this was like, okay, most startup ideas are

7. 27:27

   terrible. They feel really good. We feel the truth of our ideas very deeply, and they're often just laughably wrong. They do not survive contact with reality. The kind of startup impulse is to say, well, probably my product and therefore almost my identity if I'm not careful, but really my product is crap. And so I need to put it in front of people as fast as possible in a low-stake setting and then iterate as quickly as possible. And so we thought to ourselves like, okay, well, let's just start talking to people. Let's see if this thesis actually holds up. Let's bother to just like talk to one person who clearly is not empowered to do.

8. 28:00

   anything with this information. And we did. And by and large, much like if you look at polls of the US population right now, and you ask people how they feel about superintelligence, how they feel about rogue AI, they're passing the test with flying colors. They are correct. Superintelligence is likely to be a massive risk to human civilization. And the public's intuition on this is basically right. Just like US government officials positions on this were basically right when they were first introduced to it, at least by us, back in 2020, 2021, that era.

9. 28:29

   And so just by doing this and iterating on it, we very quickly found like, wow, this like massive money cannon that these folks are holding onto is just pointed at a thesis that has not even been tested, or at least that failed our test and that continued to fail as we escalated the sort of rank and like impact of the folks who we're talking to. And so that for us was a bit of a wake up call as to one of these, I think, deep pathologies, unfortunately, in the ecosystem.

10. 28:59

    It's no longer, by the way, the view of of these entities, but it certainly was back then. Yeah. No, I was going to say, I mean, you were prescient in that sense, too, because they are now engaging in the Washington discourse.