OpenAI weighs safety gains before sharing research

Transcript

From the transcript Odd Lots publishes with the episode.

1. 27:21

   One more sort of question specifically on some of the safety and alignment stuff. Your chief scientist at OpenAI wrote a very interesting blog post or piece, An Alien Mind, Jacob Pachocki, I hope I'm- Jakub Pachocki. Jakub Pachocki wrote a very interesting piece. And there was one line in there that I thought was interesting. And you talk about the release of Asterisk 6. And he said, we invest heavily along the spectrum of approaches spanned by these directions.

2. 27:47

   We also see meaningful progress. GPT-6 is the first model that benefits- from some important advancements we've been working on in a long time and is significantly better aligned than 5.6 Sol. Should the labs share their alignment work? I mean, again, if we're talking about, I understand your competitors and capability gains, et cetera. On questions of safety, if a lab makes a big breakthrough on the alignment question, such that it can develop advanced AI fast in a safer way, Should that be diffused across the rest of the industry or should whatever this, quote, important advancement that you made to make GPT-6 Astra more aligned, is that something that should be kept in house?

3. 28:34

   So I think that there's a lot of specific circumstances to answer that question for any specific advance, because the thing that to me can be a category error is thinking about things as it is a safety thing or it is a capability. Sure. And in fact, you want there to be as little separation as possible between the two. You want as much intertwining. You want to have techniques that are by their very nature safer. And so just as you make more capability progress, you also get the safety progress. And so I think that the way I'd look at it is the more things are a pure alignment thing, more that they're a pure safety thing, the more that they are something that are just pure good, that you're just like, yes, everyone should obviously know this and use it, then yes, share it more. The more that things are just about this

4. 29:17

   will just make the model more capable, won't make it safer anymore. Maybe there's much less sort of reason to share for those reasons. And so I think you need to look at any specific advance along all these axes and all these factors. And I think that there are some areas where we have actually been extremely at the forefront with sharing and talking about it. Chain of thought monitorability is a good example of that, where the moment that we and we were really the ones to pioneer this paradigm of reasoning and chains of thought. The moment we saw that we were instantly thinking we need to preserve this for as long as possible. So when we released a product, we did not actually display the chain of thought, even though it would have been very helpful.

5. 29:56

   People love reading that stuff. It's super interesting. It's super helpful. But it's just clear we're going to immediately feel like we have to optimize it and we're going to have to remove that legibility. We started talking about this. We helped spearhead a paper that was putting forth this position on that you cannot optimize these things. And again, we've shown a lot of the way and led a lot of the way on various techniques. And so I think it comes down to the specifics.