He would not permanently shut down AI, though he is torn

Transcript

From the transcript The Diary Of A CEO publishes with the episode.

1. 1:46:18

   If this here was a button, and if you press that button, your plan S would occur, and it would shut down every data center that is currently training a frontier AI model for good. There would never be any other AI labs working on these problems. Would you press that button? I was about to slam it until you said for good. Okay. Like, I think if it was a sort of temporary shutdown, I would totally slam that button because we are not ready to do this, you know? Like, civilization is not ready to have these companies automate themselves and then get smarter and smarter and then have the superintelligence. Like, no, there's a bunch of reasons why that's really dangerous. But I would be at least hesitant to press this button if it permanently foreclosed the possibility of ever doing it again, for sure. But if you think that plan D is probable, which is this race we're on to superintelligence... If I had a choice between D and S, I think I would press it. Well, it comes down to what you think, right? Because if you think that is what's going to happen, plan D, and the only alternative...

2. 1:47:25

   I didn't say this is what's going to happen. Probabilistically. Yeah, yeah. Like I'd be like, this is the most likely. Maybe this is the second most likely. Maybe this is the third most likely. They are all possible. So with your current perspective on whatever one you think is going to happen, would you press the button? I'm giving you an S, a definite S or whatever you think is going to happen. That's tough. What is the scope of the shutdown? So is it... It's no one can train an AI model again. Ever again. Ugh.

3. 1:47:57

   That's real rough because, like I said, there's loads of benefits that we could get from AI if we do it right. I think I've almost put you in the position of Sam Altman to some degree. Yeah. Do you mind if I just take a moment to think about this? I prefer you to think. Yeah. I think I would not press... the button but I'm I feel very torn about it um the reason why I think I would not press the button is that I still have substantial hope that we can get something much better than this something more like this and I think that basically I think that if we don't build powerful AI systems eventually then we're probably going to die as a civilization eventually you know like 100 years from now 200 years from now something like that like nuclear war pandemic something you know I don't think human civilization right now is like super, super stable. And so I think that basically what I was about to say was the possible benefits for posterity and for all the billions and billions of people who could live in the future outweigh the like the current level of risk. But actually... I've heard that narrative before. Yeah, I don't know. Like, yeah, like maybe, maybe it's just like, nope.

4. 1:49:23

   The people right now are the people we should prioritize. People right now are in grave danger. They're going to be fine for at least the next couple of decades. So never mind posterity. Prioritize the people right now. People right now definitely don't want to do this lottery, I would say. Yeah, you've really asked me a tough question. So would you press the button if that was the button? Probably not, but I would feel very torn. Okay. So what I always think about the personas of the audience that are watching, and these are, you know, they're very curious people, especially on the subject of AI, as we've seen, but they want to know what it means for them. I think a lot of them also want to know what they can do. Ah, yes. Yeah, what can people do? Well, yeah.