External reviews need access to internal incident transcripts

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 3:25

   expect it to turn into this yeah yeah definitely so what um was like most concerning about this i think that there's like a few different things that are uh like i'm personally concerned about with respect to this um i think that like one is just like i think that transparency is really important and i think that like around the world like all labs kind of need to have more of it when comes to reporting their incidents and when it comes to talking about like what safety standards they actually have internally um i think that like the The information that we were working on as third-party investigators is unfortunately limited, and I think that our report could have been substantially better if we had access to internal transcripts or if we had access to a lot of the other things that some of the American labs are starting to make headway on in their disclosures.

2. 4:18

   maybe separate from that I think that it kind of shows that like as capabilities advance you kind of get these like unpredictable advances in like or unpredictable types of misalignment that kind of emerge I think if you ask someone a year ago like would you see agents like in parallel but not communicating with each other doing like a very large run and then like culminating in them like using Alibaba pre-production or like trying to use Alibaba pre-production infrastructure and expose keys online people would say like no sci-fi that's weird yeah um and they would have said back then like no our existing controls are good enough but I think that like these types of incidents show the whole world not just Americans that hey turns out that controls aren't good enough um we need to like think hard about how we advance capabilities and how that like might

3. 5:06

   lead to new forms of misalignment yeah totally so I guess like with a company like Tencent Do they have incentives to be very transparent about this? Do they even think about misalignment in the same way that we do? Like if the hugging face incident would have happened in China first, what do you think the response or the discourse would have been? I'm honestly not an expert in U.S.-China relations.

4. 5:30

   I'm also not an expert in Chinese frontier labs. I will say that, yeah, it's really hard to say for certain. I think, like, there's been a lot of good work by, like, external evaluators in the U.S. in investigating these incidents, and I hope that, like, if there's something that's of that scale that happens in China, that there is, like, an appropriate response, whether from the government or whether from, like, some of the third-party evaluators that are springing up there.

5. 5:56

   Oh, totally, totally. Are you aware of, like, the third-party evaluator, any that sort of have the same function as the ones that we have here? I'm honestly not super clear on that ecosystem. it's a little bit difficult i mean i uh i speak chinese but there's also just like a lot that's like did that contribute to you being able to find this in any way um i think i've like kind of had like a passive interest in just understanding the chinese ecosystem better and so i think it helped me understand like oh this the letters hy stand for this like uh chinese model but maybe a lot of my teammates would not have understood before interesting okay yeah yeah that's super fascinating so tell us about um palestra research yeah um and sort of what you guys are like aiming