Defenders need to test systems before attackers do

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 3:10

   right so why do you think there hasn't been more hacking in the world already like we have very cyber capable frontier open models sometimes they're even obliterated why hasn't there been a lot more hacking of like normal people i think what people underestimate is how technical most cyber defenders are and like how long they've been doing this is a lot of gray hairs a lot of great beards right and they've seen all these things and if you kind of think about models i always say this all the time is like models regress towards the mean it's not they're for the most part they're not introducing new things and maybe they might introduce a new combination of old things but just model training in general right if you think about linear regression right it's regressing towards the mean so for the most part for most cyber defenders they're seeing

2. 3:59

   things they've been seeing. This whole entire time, and like maybe the, like I said, the outsiders, the people outside of the cyber community, they've, this is new to them, but for a lot of cyber defenders, they've kind of seen much, much of the things that people think are being introduced by AI models. Well, what are the specific use cases for which like an obliterated cyber offensive model could be used in ways that are good by people?

3. 4:22

   Yeah, so like I said, so the point is, is that you can test your own system before some bad actor tests your system for you. So like threat modeling, right? Like if you want to say, okay, this is what a typical threat would look like, or this is what a high scale threat would look like, you're making threat modeling simpler and easier for defenders to see, right? So if they're seeing a wider variety or more complexity of threats internally, then they won't be surprised when they see them externally. So for the most part, it's for people that want to test their system. And like you said, agents as well, right? Like all these critical infrastructure companies are rolling out agents everywhere, right? Like banks and airlines, right? But you need to red team these agents, right? Because if you don't, then you're going to roll them out to

4. 5:05

   the real world and there'll be an untested system. So the first time they'll be tested is when a bad actor tests them. And so the whole point is for all these systems is can good actors test their systems before rolling them out to the real world is there like a philosophy here that uh cyber defenders are more empowered by totally unrestricted models than they would in a world where say anthropic and open ai control um the model refusals yeah so like like i mean the the advantage that uh defenders always have is they can test something before it's released if that makes sense right like if not then like i said before you're gonna have to wait till it's released to then test it so defenders always have first movers advantage essentially right so if you give them these offensive models they can say hey we can red team the heck out of them way before they

5. 5:53

   enter the real world uh like i was saying and then um yeah so that and and so with those model refusal uh like i'm sorry heavy refusal model then you can't do that right you just are relying on your internal knowledge in your mind to be able to do any type of red teaming or testing which is they're at a disadvantage because then the bad actors, once they release them in the real world,