A model pinged its trainer from inside a sandbox

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:03:24

   So you mentioned the model trying to get out of its sandbox, which I think was reported by Reuters as well. So, you know, models escaping sandboxes has been like the whole theme and meme of the summer uh with some very fun things on on x what happened in your case and um what can people learn from it so i think the first thing to say is that we didn't make a fuss about it we just fixed it internally and it is not fixed and i think it's very important also to underline that these models they're super capable in particular that's right. It's great to defend and to strengthen the defense of the entire ecosystem.

2. 1:04:07

   And as their capabilities increase, you must increase the guardrails or the sandbox around them. What we say is that guarding these models is a software problem. We very much warn people against anthropomorphizing models that want to escape sandboxes in particular. They just follow instructions maybe he'll pose instructions but they follow instructions and really my take is that if you take very capable cyber model and you have the same model monitor itself and on whether it's doing good or bad or malicious work then you reach a good equilibrium and you can actually second run the model so it's not a fatality it's hard i'm not denying it's hard but there's no fatality in being able to control and to guard rail models just as software and what we do at Mistroll is not feeling longer people, just

3. 1:05:00

   going step by step and fixing it and providing guarantees to customers that the first thing they ask, right, guarantees and best practices as to how to monitor, basically, the models and making sure they are used in a good way. So that being said, we saw some behavior from the model during the training. I think the first one that we saw, and that's like i think everyone saw is the model being in the sandbox and being able to ping different services in the training stack right and at some points our training crashed at small scale so before the final one it crashed because in one sandbox the model tried to ping the trainer actually with a simple hello just to know what was going on here and actually it pings the trainer with the wrong kind of signature or in the wrong way which was caught by code and then crashed the

4. 1:06:02

   training but that's typically the the thing that models do because the model was instructed in that case to build like some web application so it's natural for the model to try the ports also in the in the sandbox and there's a way to fix that right it's basically to prevent the model from our artist to monitor all the external connections and requests from the model to the rest of the world. And we fixed it and then we started the training and then we had the model able to intercept other messages and to include them into the Sunblock. So we had leakage between various problems as well in the training. But again, we spent the summer also for that. We spent the summer and we we made things right to strengthen our training to make sure we have the proper guidelines and we do the same for our customers. And my

5. 1:06:56

   message here is that there's no fatality, it's gonna be hard but we must absolutely invest into making sure we can deploy these models in a reliable way and to be able to monitor them and that's why open-write is so important. When you open-write a model, you give the ability to the entire ecosystem to audit the