AI safety needs groups willing to challenge powerful labs

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 22:35

   There's policy advocacy. There's lab advocacy. There's the decisions that the labs themselves should make. What do you think people concerned about making AI go well should actually do beyond just research? my sense is that uh the larger a group of people uh a lot of the stuff is going to be bottlenecked by groups of people who can trust each other and actually work together um and like share information clearly and and this is not the kind of thing that you see very much of in uh politics for example it's just like a lot of distrust and like people aren't really willing to think very much about uh they they often end up falling in behind the party line um and my sense is that the story of ai over the past few decades has been that if a small group of people who are thinking more clearly about um factual issues like uh you know the path towards building agi

2. 23:42

   a small group of people can actually coordinate extremely well and then end up gaining a lot like growing a lot and gaining a lot of capabilities to influence the world but you really need that foundation of like can we uh like are people actually trying to communicate clearly with each other. People at OpenAI in the early days were actually trying to have serious conversations about AI to a much greater extent than people in mainstream ML or even at DeepMind.

3. 24:11

   Most of the people at DeepMind, I'd say. And that led to massively compounding benefits. OpenAI out-computed DeepMind and then Anthropic out-computed OpenAI. So my sense is that most people should be orienting to who's being honest and who's actually trying to navigate the power dynamics in a way that is much less scared of getting on powerful people's bad side than the field has been historically.

4. 24:45

   and uh you know insofar as they can uh find people doing that then uh then you can start like following a plan but like before that you just don't have a coherent group of people who are going to actually be able to follow a plan yeah i mean like anthropic for example seems like kind of like what you are describing they seem to be good at uh coordinating with one another internally they seem to be very coherent they seem to be um very good at you know accessing and uh accessing power and increasing their own power it kind of seems like this is like this this this meets the criteria that you're describing and they have you know reached the point at which they should start following a plan of sorts no the difficult thing is that the fastest way to get such a group to grow is by coordinating around uh sort of like sacred truths if you will

5. 25:43

   like things you're not really allowed to question or um like things you're not allowed to look at so like historically something like christianity for instance had sacred concepts and uh dogmas actually i should say um my sense is the dogmas at anthropic are involve something like other people will do it anyway um or maybe you know like where the good guys and they're like and then like you have various narrative you have various like pieces of evidence that are marshalled to support where the good guys and like you know if it weren't for us other people would be doing anyway therefore you know it's it's fine to raise a head or it's fine to be less responsible and so on given that anthropic is not really able to accurately evaluate uh what other people would do or what other companies or countries would do then it's very hard for them to plan I think

6. 26:37

   just because they they aren't willing my sense is that Anthropic as an organization is not willing to look at its own influence on the world and for example the fact that I think many people in China or many researchers are actually becoming pretty scared of Anthropic in particular and you know that's the kind of thing that you really need to be able to take into account when trying to make a plan um so yeah my hope for anthropic would be something like it's able to accurately evaluate this this idea of other people would do it anyway you know therefore um we we can get to do what we want and still be the good guys and somehow look past that right