Open models can help make AI safer

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 16:00

   And it really packs a lot of intelligence in a smaller form factor. I'm curious how you all are thinking about safety and alignment. It's the maybe number one topic, not just in AI, but in geopolitics right now, is aligning these models. models. There's all the headlines recently of OpenAI and others having rogue agents going out there and hacking things, and I think people are concerned about the increasing capabilities of these models. How do you approach alignment, especially for an open weight model?

2. 16:28

   There are kind of two things to kind of think about. One is, what are the benefits of doing safety research and protecting models in the open, and then the technical of how do you actually go and bake in safety into a model. As it pertains to open source, traditionally, and let's not even think about models for an hour because there's a very similar thing that happened in software development and with operating systems and internet.

3. 17:06

   The internet protocols are open. Operating systems like Linux and Android are open. And a lot of the strong encryption protocols that gave rise to the whole field of cybersecurity are open. There's the way you test software, you do penetration testing. There's both black and white box penetration testing. And so the notion of openness has been fundamental to securing software over the last decades to the point, right, there's Linus's law, which is that with enough eyeballs, all bugs become shallow.

4. 17:37

   And that's why open source software is actually in many ways considered safer and more trustworthy than closed software because you can just iron out, right, all the kinks. Now, we have to think about artificial intelligence as, right, it's an evolution of software. It's a machine. It's a piece of software. So it's not a god. Not, well, not that I'm aware of. Yeah. Okay, good. I mean, it's no more than Linux or Android is a god.

5. 18:06

   So maybe there's a church. It's a very pragmatic view. Yeah. Maybe there's a church of Linux somewhere and all the power to it. We'll find it. We'll be in the comments. But it's ultimately, it's a very sophisticated machine that has a long tail of vulnerabilities, like any piece of software. And it is impossible to discover and patch up that long tail with high confidence when you have a small number of people who have access to it.

6. 18:33

   So right across the closed labs, there might be 300 safety researchers. And that's great. And I think the intentions are great. But it's kind of like saying, across my whole body, I have 300 white blood cells. Do I trust those 300 white blood cells to find all the pathogens and get rid of the stuff that's, you know, attacking, you know, your body every day. No, I'd rather have as many white blood cells as possible.

7. 18:56

   And I think that, you know, this notion of with enough eyeballs, right, all bugs become shallow, I think it's true to, like, if we think about mechanistically patching, like, all the vulnerabilities in these systems, of which there are many, you need to bring a community in, right? You need to bring, you know, from 300, 300,000, right, people who can investigate, and inspect these systems. And so fundamentally, I think that in the same way that open source software has actually been the safest way to develop software broadly previously, I think the same thing transfers here.

8. 19:32

   There are some things that are new because the software is so powerful, but the broad strokes are still the same. And so at a high level, I'm excited for moving into a world where a large open source community protects us you know, in a way that I think is going to be much more effective than, you know, having a very