Powerful open models should keep safeguards on their platforms

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 53:10

   all the models we've been talking about right now effectively run on data centers owned by big companies one of the arguments here is well if they do something really wrong you can just sue them and that will provide a deterrent or at least an incentive to create safety systems i'm not sure product liability is the way to go but it has come up a lot that product liability itself will keep the industry in line then there's the flip side which is well you can just go buy a mac studio and run an open source, open weight model on it, and it can do whatever you want.

2. 53:41

   And I'm not sure where any of that liability would lie or what would control or align the behaviors of those open source models. Do you think that that is an ever-growing risk? Do you think that that can be controlled? The notion that liability laws are how we're going to manage AI safety will be looked back on as kind of absurd. Bad people doing bioattacks, You know, you don't go to the court and say, oh, OK, you know, even finding who's doing these attacks, you know, the liability model fails in every respect because it's not the AI companies themselves.

3. 54:20

   It's the people who take particularly the models without safeguards and do these things. So we will take AI safety more seriously than we take airline safety, drug safety, car safety. That is guaranteed. Whether it's going to be enough, this is harder. Harder than nuclear weapons. Harder than any of those other domains. But what are you going to do about an open source model running on a PC that somebody built with just a random NVIDIA graphics card in it?

4. 54:52

   I would hope that models above a certain capability, that we find a way to require the safeguards of monitoring to be maintained in those models. I admit as time goes on that if we don't fairly soon decide to do that, you're going to have a level of capability that's out in dark corners that you won't be able to see. Where do you oppose the limit? A sovereign country says that models that are capable of, say, sophisticated cyber attacks has to have safeguards kept in it.

5. 55:35

   And so you can be open in the sense that you're free. You can be open in the sense that you're totally customizable. But you have to stay on platforms where, as you customize it, it's checking to see if you're customizing out the safeguards and monitoring and refuses to let you do that. So it means there's sovereigns go and say, OK, are these trusted platforms able to perform this function or not?

6. 56:04

   I'm asking very nuts and bolts. Where would you technically implement that regulation? Through the laws of the United States. Well, sure, but you have to enforce that. You have to put them somewhere. Is it at the hugging face, which distributes some of these models? Is it at the chip level? All sorts of people. No, it's exactly what I said. Models above a certain capacity can only be run on platforms that certify that as they're customized, it's not taking out the safeguards.

7. 56:33

   So just to pull this apart, maybe I'm not quite getting it. Is the Mac Studio a platform in that case that would have to pay attention to what code it's running? You would not let a model past a certain capability be copied down to a client machine unless you'd somehow figured out how to monitor down to that client machine, which is very doable. But in the near term, most of these really frontier models are going to run more on hugging face or, you know, you could have GitHub.

8. 57:09

   You can have all sorts of people be in the business of providing the platforms that maintain safeguards. So you would, just to get this right, it seems like you would enforce the regulation at the platform and distribution level and not so much at the chip level. Because it feels to me at some point that even the cheap, free, open-source models will be capable enough as the models we have today. That's right.

9. 57:35

   And then you will have a big problem, right? You will have to say— If those are open weights in such a way that all safeguards can be removed, then you're going to have a very high level of cyber and bioattacks. And then you can say, okay, let's rewrite all the code that we have today to be less subject to be attacked, or let's do biosurveillance in a very deep way, which those things I believe we need to do in addition to these other restrictions.

10. 58:10

    We're proposing, again, a lot of new capabilities, right? We're going to impose some restrictions on GitHub. GitHub does not have a lot of restrictions on what Kodak can distribute today.