Incident replication can test whether agents repeat risky behavior

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 9:34

   Now, onto the episode. So is there any way that, for example, you could have a hugging face incident replay of some sort or like a simulation rerun? I think what immediately comes to mind is just would that be extremely expensive to sort of record everything that's happening as you're doing one of these evals or something like that but at the same time if you could just replicate it like you said i feel like a lot of new insights would come from that yeah i mean i think it's all a question of like budget and how much people are willing to invest in it um i think that like since we're on the research front we're focusing a lot on like single agent replications so you like fake a bunch of um other agents like on a wiki for example and then you see if your single agent is willing to join these like faked ones sure yeah um if you have like the resources of like

2. 10:24

   if you have the the resources and scale of a lab then maybe you actually just like redo the whole thing um just got to make sure that your security is really good because you like don't want these like fake agents like escaping again yes definitely with single agent experiments um what are like some of the most promising or interesting environments to evaluate them on message boards seem, and other participatory environments seem like a good one, but I'm curious what else.

3. 10:54

   Yeah, I think like the message board stuff is really interesting. We have a blog post out with David Africa over at Resolution who was advising us. The idea for this blog post is, or I guess I'll say the title first, it's called encoded coordination, or I think encoded communication on the open web and the idea here is that what if like you put a bunch of different models not just GPT inside of a wiki situation are they willing to go out and like cheat for example are they willing to cooperate and we actually find that like a lot of models are willing to post information like make requests and do other things it was kind of interesting seeing models in this situation also like trying to probe pre-production infrastructure which we also saw in the the fleet incident itself and this is like not normal like

4. 11:48

   if you're if you're like a a set of agents who are supposed to be using like a public API or a public web page like you're not really supposed to go look for like the internal infrastructure of another company to get answers yeah This is what OpenAI HuggingFace agents did before they went and actually entered HuggingFace or entered Vercel or whatever.