The alpha, twice a week
That was 3 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Possible · 3 min · Oct 7, 2026
Adding safety to AI apps is still a multi-step process
Camille François says removing child sexual abuse material from AI training data requires several separate approvals and tools. Roost wants to make those safeguards easier for small builders to add.
Camille François is a founding president of ROOST and professor at Columbia University.
Reid Hoffman, Host · Co-founder, Manas AI
Co-founded LinkedIn; one of OpenAI's first investors; Greylock partner.
- AF
Aria Finger, Host
About this segment
Camille François says removing child sexual abuse material from AI training data requires several separate approvals and tools. Roost wants to make those safeguards easier for small builders to add.
From “Building safety tools no one can take away with ROOST’s Camille François”, Oct 7, 2026. Watch the full episode
Thanks. We'll look at and fix what's off.
“And you have to go and then register with, you know, the authorities on one hand and then go ask for a hash bank”
Takeaways
- Cleaning AI training data requires multiple separate safety steps.
- Small builders may lack access to these tools today.
- Roost wants safety checks to be easy to add at launch.
In this episode
Full episode- 17:02Discord turned a vendor shutdown into shared safety infrastructure2:22
- 24:58Safety teams need tools for investigation and review, not just detection3:25
- 51:42A two-person app used open-source tools to report 19 cases2:59
- 8:17Social media was a detour from tech’s open foundations3:00
- 11:10Open source makes safety tools a one-way door2:26
- 29:42Shared safety tools can help startups compete4:51
- 35:41Shared tools can be open without exposing every detection rule2:45
46:28Adding safety to AI apps is still a multi-step process2:50
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 46:33
So coming into the AI age, AI is making it just easier and cheaper for folks to watch apps, chatbots, everything. And sometimes it's small teams, sometimes it's one-person teams, and they could have no background in safety whatsoever. However, how can these small teams make things safe right from launch? I have a lot of thoughts on this. I have personal thoughts, professional thoughts. I suppose I should start by acknowledging that I am myself a compulsive vibe coder.
- 47:04
Great. I've coded a number of apps, including like family apps. And I am excited by the fact that people are able to use AI to bring their own vision of something into life faster and faster. Our big vision at ROOST is that we need to bridge the gap between how easy it is to come up with a new app, a new service, and how hard it is to actually do some of the important steps of making it secure. I think where we feel that the most right now is on child sexual exploitation.
- 47:43
We have a lot of builders who are coming in and saying, we want to do this. We want to do this right. And we're very excited when that happens. We're like, OK, we will take you through those steps. But the reality, for instance, of removing child sexual abuse material from a foundational data set that's then going to be used to train an AI model is that right now it is a hero's journey. And you have to go and then register with, you know, the authorities on one hand and then go ask for a hash bank and then go and implement, for instance, HMA to use this hash bank.
- 48:16
And then you have to go and ask for a novel CSTEM classifier and then implement it into something else. And so a lot of our focus is how can we make that process much easier? the overall vision is that one day it's going to be as easy to launch an amazing website and app than to just type ROOST at safety. Totally. And it will just do it. Or even, you know, by default, those tools will tell you, hey, so fun that you've added a field in your app that allows you to upload, say, kids' books into an app.
- 48:54
Also, you may want to run a moderation layer here. And here's why and here's how. And we'll make it so easy for you. So we want to make that much easier. There's a lot of work to do to do this. And we're excited to work again with others to go in and tackle that gap. So you started with a partnership with OpenAI and the GPT-OSS Safeguard.