Companies should evaluate models on their own tasks

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 29:29

   And do you think that, like, I think there's a debate right now whether it's bad if we widely use Chinese open-weight models in the U.S. Yeah. The question is, why would it actually be bad if you had to, like, straw man the other side? The safe way of using these models involves companies creating evals, which are like tests that they run on their tasks that they care about ahead of time, and making sure that the models are performing well on these evals.

2. 30:03

   And this is like the critical act that businesses will have to do in the next 12 to 18 months. In these evals, you will find exactly what that means for every model, not just the Chinese models. and um i think that's like a foundational case that i want to make um and it doesn't have to do with whether the model is chinese or not now if there's some kind of like geopolitical game being played by china where like all the model labs become like slightly more likely to do something or all the the chinese models become you know slightly more dangerous in some dimension um we do want to find that and like i you know i think i think the we'll have a lot of companies that go look for those outcomes um and just make sure that everybody's secure and some of those outcomes might be completely unintentional it might be like you know somebody trying to

3. 31:03

   feed something to the model labs training data pipelines to get models to do something there's a lot of model epistemics to worry about, where people game the AEO, the agent optimization that allows models to have some kind of preference for some company or some brand or some ideology. Companies and nonprofits will form to help look for that stuff and it will become a huge deal in the next you know two years I think it's it's gonna be like a big problem and very very interesting when people find like key examples of this happening and you know I'm really excited about like new innovations or new projects that are working in that direction that's okay so they'll just be a new set of companies and tools and things built around around this? Yeah, kind of like how, you know, there was like a McAfee antivirus, like a industry

4. 32:07

   that, that emerged around keeping people safe from random things being downloaded on the internet. You know, you don't want to ban the whole internet just because there's some bad things that some people can download somewhere on the internet. You want to create an industry that protects people at the layer they're operating in or build operating systems like Apple has that provide really good safeguards at the user level.

5. 32:39

   And how do you think the web changes over the next 10 years?