Safety rules could sharply lower Max’s AI risk estimate

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 28:00

   Of course, we can lower P-Doom a lot. Yeah, so what are some of the non-catastrophic ways that we can lower P-Doom? He was claiming, you know, we've got to have Asilomar enforced, just like we did with bioweapons. And I said, I've had Annie Jacobson on twice, and she said, you know, their country is violating biological warfare constraints from Asilomar right now on this planet. And he seemed to kind of blow that off.

2. 28:25

   But what would blow your P-Doom lower? I mean, the obvious thing is to simply start, if the U.S. government decides to just start treating AI companies like they treat any other company, saying they can only sell dangerous products after they've convincingly demonstrated that they can meet certain safety requirements, now the PDOOM goes way, way, way, way down, right? I mean, suppose you decide you're going to open a sandwich shop there in San Diego, right?

3. 28:56

   And then the health inspector comes in and says, I found 57 rats in your kitchen, Brian. You know, you're not selling any sandwiches. I'd be relieved. What happened to the other 40? I'm joking. Yeah. But you could turn around to this guy from the government and say, you know, don't tell anyone. I'm not going to sell any sandwiches. I'm just going to release an AI girlfriend product for nine-year-olds.

4. 29:19

   another AI that can help terrorists make bioweapons and I'm gonna release this thing called super intelligence which is gonna basically be a new species and I have a good feeling about this that it's gonna be fine the poor guy from the government will have to tell you yeah actually that's all legal it's fine just don't make any sandwiches right that's how absurd how how absurd corporate welfare we bestow on AI companies they're the only ones who can do crazy shit like this and then have it be completely legal what do you think would happen if one of the pharma companies in San Diego released the new medicine and they're like ah we don't need this clinical trial bs we're just going to sell it anyway in safe way they couldn't do that or if someone released a car and we're not we're refusing to

5. 30:10

   do any crash testing and yeah we know that five of the half the vehicles their gas tanks caught fire during the first hour driving but we're going to sell it anyway you can't even sell kids toys without some safety testing right so if but yeah i mean i do so much and i just want to make clear that it's not going to stop ai research either it's just going to mean that certain fringe crazy kinds of ai research like how you make more addictive ai girlfriends for nine-year-olds that would stop because they really can't demonstrate right now that it does more good than harm and the race to super intelligence would have to wait until they've figured out come up with some sort of plan for how they're going to contain it but all the good stuff curing cancer and making better, proving more cool things about physics and so on,

6. 30:56

   would probably be able to continue more or less full speed ahead. Is that part of the motivation for your work with Bernie Sanders that you were mentioning before I hit record?