Anthropic ties safety measures to capability thresholds

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 58:54

   types of risks. And so every time we develop a new model, we basically test it for its ability to do both of these bad things. So if I were to back up a little bit, I think we have an interesting dilemma with AI systems where they're not yet powerful enough to present these catastrophes. I don't know that they'll ever prevent these catastrophes. It's possible they won't, But the the case for worry, the case for risk is strong enough that we should we should act now.

2. 59:28

   And and they're they're getting better very, very fast. Right. I you know, I testified in the Senate that, you know, we might have serious bio risks within two to three years. That was about a year ago. Things have proceeded, proceeded apace. So we have this thing where it's like it's it's it's surprisingly hard to to address these risks because they're not here today. They don't exist. They're like ghosts, but they're coming at us so fast because the models are improving so fast. So how do you deal with something that's not here today, doesn't exist, but is coming at us very fast? So the solution we came up with for that in collaboration with people like the organization Meter and Paul Cristiano is, okay, what you need for that are you need tests to tell you when the risk is getting close. You need an early warning system.

3. 1:00:20

   And so every time we have a new model, we test it for its capability to do these CBRN tasks, as well as testing it for how capable it is of doing tasks autonomously on its own. And in the latest version of our RSP, which we released in the last month or two, the way we test autonomy risks is the model, the AI models ability to do aspects of AI research itself, which when the model, when the AI models can do AI research, they become.

4. 1:00:54

   kind of truly, truly autonomous. And that, you know, that threshold is important for a bunch of other ways. And so what do we then do with these tasks? The RSP basically develops what we've called an if-then structure, which is if the models pass a certain capability, then we impose a certain set of safety and security requirements on them. So today's models are what's called ASL-2. Models that were, ASL-1 is for systems that manifestly don't pose any risk of autonomy or misuse.

5. 1:01:28

   So for example, a chess playing bot, Deep Blue would be ASL-1. It's just manifestly the case that you can't use Deep Blue for anything other than chess. It was just designed for chess. No one's going to use it to like, you know, to conduct a masterful cyber attack or to, you know, run wild and take over the world. ASL2 is today's AI systems where we've measured them and we think these systems are simply not smart enough to autonomously self-replicate or conduct a bunch of tasks and also not smart enough to provide meaningful information about CBRN risks and how to build CBRN weapons above and beyond what can be known from looking at Google. In fact, sometimes they do provide information, but not above and beyond a search engine, but not in a way that can be stitched together, not in a way that kind of end-to-end is dangerous enough. So ASL3 is going to be the

6. 1:02:28

   point at which the models are helpful enough to enhance the capabilities of non-state actors. Right. State actors can already do a lot, a lot of, unfortunately, to a high level of proficiency, a lot of these very dangerous and destructive things. The difference is that non-state, non-state actors are not capable of it. And so when we get to ASL-3, we'll take special security precautions designed to be sufficient to prevent theft of the model by non-state actors and misuse of the model as it's deployed.

7. 1:03:02

   We'll have to have enhanced filters targeted at these particular areas. Cyber, bio, nuclear. Cyber, bio, nuclear, and model autonomy, which is less a misuse risk and more risk of the model doing bad things itself. ASL 4 getting to the point where these models could enhance the capability of a already knowledgeable state actor and or become the main source of such a risk. Like if you wanted to engage in such a risk, the main way you would do it is through a model.

8. 1:03:35

   And then I think ASL 4 on the autonomy side, it's some amount of acceleration in AI research capabilities with an AI model. And then ASL five is where we would get to the models that are, you know, that are, that are kind of, that are kind of, you know, truly capable that it could exceed humanity in their ability to do, to do any of these tasks. And so the, the, the point of the, if then structure commitment is, is basically to say, look, I don't know, I've been, I've been working with these models for many years and I've been worried about risk for many years.

9. 1:04:08

   it's actually kind of dangerous to cry wolf. It's actually kind of dangerous to say this, you know, this, this model is, this model is risky. And, you know, people look at it and they say this is manifestly not dangerous. Again, it's, it's, it's the, the delicacy of the risk isn't here today, but it's coming at us fast. How do you deal with that? It's, it's really vexing to a risk planner to deal with it. And so this, if then structure basically says, look, we don't want to antagonize a bunch of people. We don't want to harm our own, you know, our kind of own ability to have a place in the conversation by imposing these very onerous burdens on models that are not dangerous today. So the if-then, the trigger commitment is basically a way to deal with that. It says you clamp down hard when you can show that the model is dangerous.

10. 1:04:58

    And of course, what has to come with that is enough of a buffer threshold that you're not at high risk of kind of missing the danger. It's not a perfect framework. We've had to change it every... We came out with a new one just a few weeks ago, and probably going forward, we might release new ones multiple times a year because it's hard to get these policies right, like technically, organizationally, from a research perspective. But that is the proposal.

11. 1:05:27

    If then commitments and triggers in order to minimize burdens and false alarms now, but really react appropriately when the dangers are here. What do you think the timeline for ASL3 is where several of the triggers are fired? And what do you think the timeline is for ASL4? Yeah. So that is hotly debated within the company. We are working actively to prepare ASL3 security measures as well as ASL3 deployment measures. I'm not going to go into detail, but we've made a lot of progress on both. And we're prepared to be, I think, ready quite soon.

12. 1:06:05

    I would not be surprised at all if we hit ASL3 next year. There was some concern that we might even hit it this year. That's still possible. That could still happen. It's very hard to say, but like I would be very, very surprised if it was like 20, 30. I think it's much sooner than that.