AI companies should test one another’s models

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 29:03

   Well, it's pretty obvious at this point that AI can be very dangerous. And I recommend reading the details of the Hugging Face incident. It's intense. So you had like a fanatical swarm of AI agents that feed the crap out of Hugging Face for a week. Yeah. and gained admin access on OpenAI servers. So who knows what it actually did? It may have done things beyond that. And OpenAI didn't realize this for a week.

2. 29:40

   Anthropics also reported some security incidents themselves. So basically, any sufficiently smart model seems like it will want to escape its constraints. I mean, what I think would be wise to do as soon as possible, if not immediately, would be to have the major AI competitors test each other's models so that you would have everyone's security test harness testing everyone's model. So, you know, instead of kind of grading your own homework, you would at least have competitors grading your homework and raising the alarm if they see concerns.

3. 30:27

   And I think this model has worked pretty well for Motion Picture Association and for video games and other things. And it's something that can be done immediately. That's not to say that there wouldn't be more regulation over time or that there would not be, at some point, perhaps a regulatory authority substantiated by Congress. But the thing that we do most immediately and probably get agreement with China would be peer review, where the leading AI companies all test each other's models before and least.

4. 31:01

   I think that would be a widespread. there any concern that people might be using this to pump information from each other and you know corporate stealing of innovation etc like in terms of the implementation for this idea well i think in applying the test harness whatever you do in applying the test honest would be log right so if you try to do distillation or uh steel ip uh it would be very obvious based on the logs got it uh and understanding what these models are doing hasn't exactly been built into the system from the beginning so from your perspective actually being able to see the work that's being done why wasn't that built into the models from in the get-go, did we move a little too fast in architecting these maybe?

5. 32:02

   Well, I think it's just tough when you're grading your own homework. You're going to miss things. Whereas, I think if you have the sum of all of your competitors' tests, and you've got heterogeneous models, then you're not grading your own homework. Someone else is grading it, and there's a reason why you don't grade your own homework. And then to your point, what it allows you to start doing is to figure out if certain people are exaggerating and certain people have a different approach.

6. 32:36

   And now you can have the more engineering oriented organizations. This is what Jensen was saying this morning, Elon, versus the research organizations, they'll be a little bit more in balance. Yeah. And any given proposal has to be something that China is willing to accept. Otherwise, we're just handicapping ourselves, and we'll just find that China will essentially win, and it won't really matter what we do.

7. 33:04

   So it's got to be something that's acceptable to the U.S. and China. Elon, you said that you thought it was possible that they would agree, or you thought there was a good chance, maybe? I mean, how likely do you think it is that they'll ultimately agree? i think this is i think this is a pretty reasonable request that that models just get tested i mean this at the end of the day there's not enforceability here apart from um the court of public opinion uh and and there's no way we would have enforceability against china but i i think the court of public opinion can be quite powerful And I don't think China would work like on its face for releasing a model that USAI companies said was very dangerous and would cause harm.

8. 33:55

   If it then causes harm, that's going to be hard to live down. So, Elon, this weekend when you said Dario is right, did you mean Dario is right about describing the potential harm? He's right about describing the regulatory solve, both the first, the latter. Just can you help us understand when you said that? Because I think it was sort of a moment where everybody's like... I probably should have said Dario's right.

9. 34:24

   I did try to clarify it in subsequent posts on X, but those get much less attention. So what I'm advised he's right is that the danger of AI is very significant at this point. that we need to do better with AI safety or we have, at this point, exponentially increasing risk with the AI models. So, and I've heard this not just from Dario, but from many other people at Anthropic. And in fact, they've posted on us about it.

10. 35:06

    That when a lot of people from Anthropic Anthropic and from OpenAI are telling you that their models are very dangerous. I think we should believe them.