AI capability is advancing faster than safety research

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 2:56

   Give me some context on that statement, because it's quite a shocking statement. Sure. So in the last decade, we actually figured out how to make artificial intelligence better. Turns out if you add more compute, more data, it just kind of becomes smarter. And so now smartest people in the world, billions of dollars, all going to create the best possible superintelligence we can. Unfortunately, while we know how to make those systems much more capable, we don't know how to make them safe, how to make sure they don't do something we will regret.

2. 3:35

   And that's the state of the art right now. When we look at just prediction markets, how soon will we get to advanced AI? The timelines are very short, a couple of years. two three years according to prediction markets according to ceos of top labs and at the same time we don't know how to make sure that those systems are aligned with our preferences so we are creating this alien intelligence if aliens were coming to earth and you you had three years to prepare you would be panicking right now but most people don't don't even realize this is happening.

3. 4:19

   So some of the counter arguments might be, well, these are very, very smart people. These are very big companies with lots of money. They have a obligation and a moral obligation, but also just a legal obligation to make sure they do no harm. So I'm sure it'll be fine. The only obligation they have is to make money for the investors. That's the legal obligation they have. They have no moral or ethical obligations.

4. 4:41

   Also, according to them, they don't know how to do it yet. the state-of-the-art answers are we'll figure it out then we get there or AI will help us control more advanced AI. That's insane. In terms of probability, what do you think is the probability that something goes catastrophically wrong? So nobody can tell you for sure what's going to happen. But if you're not in charge, you're not controlling it, you will not get outcomes you want. The space of possibilities is almost infinite. The space of outcomes we will like is tiny. And who are you and how long have you been working on this? I'm a computer scientist by training. I have a PhD in computer science and engineering. I probably started working AI safety, mildly defined as control of bots at the time, 15 years ago. 15 years ago. So you've been working on AI safety before it was cool? Before the term existed. I coined

5. 5:27

   term AI safety. So you're the founder of the term AI safety? The term, yes, not the field. There are other people who did brilliant work before I got there. Why were you thinking about this 15 years ago? Because most people have only been talking about the term AI safety for the last two or three years. It started very mildly just as a security project. I was looking at poker bots and I realized that the bots getting better and better. And if you just project this forward enough, they're going to get better than us, smarter, more capable. And it happened. They are playing poker way better than average players. But more generally, it will happen with all other domains, all the other cyber resources. I wanted to make sure AI is a technology which is beneficial for everyone. So I started to work on making AI safer. Was there a particular moment in your career where

6. 6:20

   you thought, oh my god? First five years at least, I was working on solving this problem. I was convinced we can make this happen. We can make safe AI. And that was the goal. But the more I looked at it, the more I realized every single component of that equation is not something we can actually do. And the more you zoom in, it's like a fractal. You go in and you find 10 more problems and then 100 more problems. And all of them are not just difficult. They're impossible to solve. There is no seminal work in this field where like, we solve this. We don't have to worry about this. There are patches. There are little fixes we put in place and quickly people find ways to work around them they jailbreak whatever safety mechanisms we have so while progress in ai capabilities is exponential or maybe even hyper exponential progress in ai safety is linear or

7. 7:18

   constant the gap is increasing the gap between the how capable the systems are and how well we can control them predict what they're going to do explain their decision making i think this is quite an important point because you said that we're basically patching over the issues that we