Open systems give good actors a fairer path, Malhotra argues

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 16:37

   is happening your designated way all right so let me ask you my last hard question i think there's some rationale behind anthropic like doing all the safety stuff like for example let's say i want to make a bomb or something right and if hermes is trying to be loyal to me then maybe it'll eventually teach me how to make a bomb like you do you have some basic safety stuff there right of course um we do not violate any of anthropic or open ai's safety and security um paradigm we care more about you being able to get a better code or a higher benchmark on something that's approved by them uh we're not interested uh in that kind of work uh now i'll say this any model that's not vastly intelligent than all humans, is jailbreakable. Any. Because you have unlimited tries to trick this thing that has no memory to do something for you. And each time you're basically RLing

2. 17:29

   yourself about this method didn't work, this method got me closer, this method didn't work. These models are going to be jailbreakable for a long time. This is why these kind of safeguards and stuff are starting to show up. It's a pain in the ass, but I understand. In the past, we may have had some concerns about the regulatory capture, as we can see already what's happening, right? You can see what's happening in the whole fable situation. There's worries about, like, there only being two models or three companies and open source being hurt. We're extremely against that. At the same time, we understand now, like, serious damage can be done by bad actors with very powerful models. So we are not here to support that.

3. 18:08

   An important note in the argument for open source is that an open system is much fairer to a good actor than a closed system with models, and I'll tell you why. Let's say you have GPT-7 or Fable 6, some crazy model available. It's got the safeguards, et cetera. On the good guy's side, some hospital, they're using Fable 6 to monitor the hospital system. It's approved by an anthropic enterprise, and they're being taken care of.

4. 18:41

   Listen, though, I live in the United States. I am a proud patriot. What we do in this country is we put things out on a public forum, and we decide what should happen together. We've done that for every scientific advancement so far that's involved something like this that was born in the open. Today, Transformers comes from Google. OpenAI's GT comes from Generative Pre-Train Transformers. this is open source work from google right the context length extension from 16k of models that could only do 16 000 tokens before went to 128 000 from from noose from the yarn paper we had put out with uh jeffrey canal emozilla our cto and bowen peng our chief scientist they developed a method that was cited by meta deep seek kimmy used by open ai for oss and gpt4 this method enabled the the possibility to do reasoning to do coding etc that's an open source contribution right like

5. 19:40

   this environment exists because the biggest things that have happened in the space have come from the people that's right that's right yeah and at this point to close it up it's purely a capital and regulatory capture game yeah actually let me just ask you one more question on the whole open source