# Open source makes safety tools a one-way door

Markdown copy of https://realalpha.app/s/open-source-makes-safety-tools-a-one-way-door/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Camille François](https://realalpha.app/people/camille-francois/) · Camille François is a founding president of ROOST and professor at Columbia University.
- Host: [Reid Hoffman](https://realalpha.app/people/reid-hoffman/) · Co-founder, Manas AI · Co-founded LinkedIn; one of OpenAI's first investors; Greylock partner.
- Host: Aria Finger
- Show: Possible
- Episode: [Building safety tools no one can take away with ROOST’s Camille François](https://realalpha.app/v/RQpx17UAK9M/), Oct 7, 2026
- Moment: 11:10 to 13:36 of the episode (2 min), playing on https://realalpha.app/s/open-source-makes-safety-tools-a-one-way-door/; the episode from there: https://realalpha.app/v/RQpx17UAK9M/?t=670

## Quote

> “Open source is **a one-way door**, right? So when OpenAI says we are actually going to share this model under an Apache 2 license”
> — [Camille François](https://realalpha.app/people/camille-francois/) at [12:13](https://realalpha.app/v/RQpx17UAK9M/?t=733) · Possible

## Takeaways

1. Apache 2 licensing means the company cannot pull back the released model.
2. Wikimedia adopted the shared moderation model.
3. A hackathon let others test and challenge it.

## About this segment

Camille François describes OpenAI releasing a moderation model under Apache 2, then seeing it adopted and tested. Open licensing lets others keep using it even if the company changes course.

From “Building safety tools no one can take away with ROOST’s Camille François”, Oct 7, 2026. [Watch the full episode](https://realalpha.app/v/RQpx17UAK9M/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[11:15](https://realalpha.app/v/RQpx17UAK9M/?t=675) I think some of those leaders are building this new open source foundations with us. So last year, for instance, we did a big effort with OpenAI. Our team worked closely with their scientific and engineering team to take one of the core piece of their safety stack, a model that was doing a lot of moderation behind ChatGPT and then Sora, and to actually share it with everybody, to put it out in open weights.

[11:44](https://realalpha.app/v/RQpx17UAK9M/?t=704) Now, that's significant for a number of reasons. But the first one is prior to that, it's not like you didn't see big companies share safety innovation. But a lot of that sharing looks like, oh, we're going to make this available to everybody. Usually that lasts for the duration of a really good marketing campaign. And then, of course, like, you know, priorities change and then people move on. And so things are not maintained.

[12:08](https://realalpha.app/v/RQpx17UAK9M/?t=728) People are struggling to get access. Things sort of fall behind. And then, of course, at any moment, they can pull the rug, right? They can say, actually, we changed our mind. But open source isn't that. Open source is a one-way door, right? So when OpenAI says we are actually going to share this model under an Apache 2 license and we're going to put it on hugging phase, that means if tomorrow they change their mind, they can't get it back.

[12:32](https://realalpha.app/v/RQpx17UAK9M/?t=752) And so people know they can build on it, right? So we did that. And they made that statement at the time saying that they believe everybody should be free to study, modify, and use critical safety techniques. It's kind of fun to see OpenAI quoting Richard Stallman. But I love that. And after that, we also did a really nice hackathon working with both OpenAI and HuggingFace. So we invited people saying, OK, look, this model has been shared.

[12:58](https://realalpha.app/v/RQpx17UAK9M/?t=778) Come test it. And so we saw people who are really exciting to immediately use the model. For instance, today, this model is used by Wikimedia, right? We also keep in mind that large, important tech nonprofits also deserve safety tools. But we also had a bunch of people come and break the model, which was really fun to see. I remember there's a team who I think was building recipes for meth and ecstasy in Latin.

[13:24](https://realalpha.app/v/RQpx17UAK9M/?t=804) So, yeah, that's sort of like what we, you know, we see some of those large lab being like, OK, yes, we think we want, you know, we want to tackle safety a bit differently now.