Closed-model refusals can leave cyber defenders without recourse

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 14:10

   in each of these formats in turn. So this is called a MOPD or on-policy distillation. Right. So another important area of model capabilities is cyber. And Mr. Alls talked about how ML4 is like the best open-weights defensive cyber model. It talks about how there's a very high refusal rate for malicious cyber requests, but also it's an open-weights model. So people can download the weights, they can obliterate the model, they can make an offensive cyber tool with it.

2. 14:42

   So do you see any tension there? Yeah, maybe to take a step back and again answer why we open-weights models in the first place, because I think this principle is broader rather than just cyber. Like I said, I think that this technology is going to transform every domain. And we fundamentally believe that this sort of technology shouldn't be in the hands of only a few established players, and it really should be accessed by everyone.

3. 15:09

   So that's why we open the weights model in general, and I think the same principle extends to cyber capabilities. So if you look at some of the most malicious attacks, or the most significant malicious attacks that have happened recently, I'm sure we can discuss many high-profile cases, But these have actually been perpetrated not by open-waste models, but by closed-waste models, which have either been jailbroken or, in some cases, during training, escaped their sandbox, etc.

4. 15:34

   So it's actually, in practice, that's how many of the attacks have been perpetrated. And on the other hand, we've got defenders, enterprises trying to shore up their vulnerabilities, their own defenses. We're finding they're having their requests being refused by the same providers. for completely understandable reasons, because in practice, these providers have classifiers under the hood, which say, is this a, is this a malicious?

5. 16:00

   which is cyber request, or is this a regular request? And for the cutting-edge defense capabilities, these defensive requests in practice look quite similar to the offensive ones. So it makes perfect sense that they are not precisely caught by the classifiers, that they're blocked by these providers. But the net effect of that is that people trying to shore up their defenses from malicious attacks, often from closed-waste models, are finding that they have no recourse but to use an open-waste model.

6. 16:30

   And so we think that it's, I mean, we think that it's critical that there are open U8s models of the same strength, of sufficient strength, that people can really scalably defend their infrastructure. I agree, yeah.