Powerful AI may need shared safeguards and regulation

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 20:21

   the question is, is there safety issues? And, you know, the safety issues range, You know, we've been in these conversations, anything from various smart people being, you know, as you were referring to earlier, you know, apocalypse doomsayers, you know, whether it's terminator scenarios, whether it's jobs, etc. So what do you think the right way of trying to navigate the development of all of these, what we both agree are amazing future capabilities, medicine, amplification of software engineering, a whole bunch of other things, and navigating the safety?

2. 21:00

   How much of that should be technical work, constitutional AI, et cetera? How much of that should be incentives? How much of that should be regulatory? What are the guideposts that you'd throw out there or principles by which people should think about how to navigate both keeping up our speed in building AI for all the good things and also navigating safety? Well, in the safety, we can break it down in a couple categories.

3. 21:25

   So there's the super disaster Skynet-y case where, like, AI takes over. Soon to be a Netflix film, New York. Yeah, and we, you know, absolutely have to prevent it. It's not something that's going to happen in the short term, but the danger is that we could slide into it as these things take care of more and more of our life. So I think because the downside or recovery from that, because we don't have time travel, is extremely hard, that's when we have to think about, even though it's low probability, sort of like massive nuclear war.

4. 21:58

   We have to do some stuff about it to prevent it because it would be so disastrous. Okay, so that's the kind of mass nuclear war equivalent case that they take over. Then the other case is, you know, North Korean soldiers use it to design a virus, and, you know, a lot of people die. So think of it as in the hands of the wrong people. It's very powerful. One scenario is sort of combined with synthetic biology.

5. 22:28

   Another scenario is... is cracking into other computer systems. So it turns out AI is very good at breaking in, finding bugs in open source things through code analysis or finding bugs through closed things by probing in the way that national security agencies do today, but instead now it's in the hands of terrorists or other people. So again, think of it, those are two specific examples of bad people using AI as a powerful tool to do very bad things.

6. 23:02

   And so there we've got to make sure that the whole industry does tech prevention, that it's hard to do. Now, right now, they're sort of everyone is doing that because they don't want to see these scenarios. But you could imagine that it might need to be regulated over time to insist that all of the sufficiently powerful AI systems protect against these kind of scenarios. And probably some of them will happen, and then we'll set up a protection regimen afterwards to prevent that kind of thing.