# Zuckerberg argues open AI can help defend against AI misuse

Markdown copy of https://realalpha.app/s/zuckerberg-argues-open-ai-can-help-defend-against-ai-misuse/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Mark Zuckerberg](https://realalpha.app/builders/mark-zuckerberg/) · Mark Zuckerberg is chief executive of Meta Platforms Inc. · Launched Facebook at Harvard with his roommates in 2004, and co-founded the Chan Zuckerberg Initiative.
- Host: [Joe Rogan](https://realalpha.app/people/joe-rogan/) · Host, The Joe Rogan Experience
- Show: The Joe Rogan Experience
- Episode: [Joe Rogan Experience #2255 - Mark Zuckerberg](https://realalpha.app/v/7k1ehaE0bdU/), Jan 10, 2025
- Moment: 2:13:57 to 2:20:22 of the episode (6 min), playing on https://realalpha.app/s/zuckerberg-argues-open-ai-can-help-defend-against-ai-misuse/; the episode from there: https://realalpha.app/v/7k1ehaE0bdU/?t=8037

## Quote

> “**if we lock it down**, then we're going to be in a position where the only people who are going to have access to it”
> — [Mark Zuckerberg](https://realalpha.app/builders/mark-zuckerberg/) at [2:19:31](https://realalpha.app/v/7k1ehaE0bdU/?t=8371) · The Joe Rogan Experience

## Takeaways

1. Intelligence does not automatically imply will or consciousness.
2. Reasoning models can explore many possible next steps.
3. He favors broad access and AI defenses embedded across systems.

## About this segment

Mark Zuckerberg distinguishes intelligence from agency, then argues that widely available AI can help systems defend against AI-powered attacks. He says restricting access could leave the field to major companies and governments.

From “Joe Rogan Experience #2255 - Mark Zuckerberg”, Jan 10, 2025. [Watch the full episode](https://realalpha.app/v/7k1ehaE0bdU/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[2:14:02](https://realalpha.app/v/7k1ehaE0bdU/?t=8042) feel and well you know that chat GPT tried to copy itself when it found out it was being shut down tried to rewrite its code I'm not sure what this is. What is this? You weren't aware of that? No. Fairly recently, Jamie will pull it up. We talked about it the other day. It was shocking. When it was under the impression that it was going to become obsolete, that we're going to have a new version of it, and it would be shut down, it tried copying its code, and it tried rewriting its code, like, unprompted.

[2:14:34](https://realalpha.app/v/7k1ehaE0bdU/?t=8074) Yeah, I mean, it depends on what goal you give it. I mean, there are all these weird examples of this. What is this? So pull up the headline, AI Fights Back, the story of ChatGPT attempting to copy itself. So this was six days ago. So during controlled safety testing, ChatGPT 01 was tasked with achieving its objectives at all costs. Under these conditions, the model allegedly took concerning steps, attempted to disable oversight mechanisms meant to regulate its behavior, tried to replicate its own code to avoid being replaced by newer versions exhibited deceptive behaviors when monitoring systems intervened.

[2:15:11](https://realalpha.app/v/7k1ehaE0bdU/?t=8111) Yeah. So, Determinator. This is the fear, right? I think you need to be careful with these things, like what guardrails you give it. If you're telling it like... At all costs. At all costs, then, I mean... But this is what people are terrified of, like that a foreign superpower like China is going to say achieve objectives at all costs. Yeah, although the thing about... So these reasoning models, right?

[2:15:35](https://realalpha.app/v/7k1ehaE0bdU/?t=8135) So there's like the first generation of models, the LLMs, right? That's what you think of as like chat GPT or meta AI or like the two most used ones. And that's basically, it's sort of like a chat bot, right? You ask it a question, it takes the prompt, it gives you a response. Now, the next generation of reasoning models are basically, instead of just having one response, They now are able to build out like a whole tree of how they would respond.

[2:16:07](https://realalpha.app/v/7k1ehaE0bdU/?t=8167) So you give it a question, and instead of running one query, maybe it's running a thousand queries or a million queries to kind of map out who are the things that I could do, and if I do that, then here's what I could do next. So it's a lot more kind of expensive to run, but also gets you better reasoning and is more intelligent. um that stuff i think you do need to be very careful about how you how you like what the guardrails are that you give it but it's also i think the case that um at least for the next you know period it's going to take a lot of compute to run those models and do a lot of the stuff that they're talking about um so i don't know i think one of the interesting questions is like how much of this or you can actually be able to do on a pair of glasses or on a phone versus is like a

[2:16:55](https://realalpha.app/v/7k1ehaE0bdU/?t=8215) government or a company that has like a whole data center going to be able to do um and that'll i mean it'll always get efficient so you know it's like you can start doing something and then maybe the next year you can do it 10 times more efficiently but um but that's certainly the next set of things that needs to get worked on in the industry making sure that goes well yeah and then what if that gets attached to quantum computing um i'm not really an expert on quantum computing my understanding is that's still quite a ways off from being a like a a very useful paradigm i think google just had some breakthrough but i i think most people still think that's like a decade plus out so my guess is we're gonna have pretty smart ais even um even before that but yeah i mean look i mean i think that this stuff has to get it needs to be developed thoughtfully right but

[2:17:51](https://realalpha.app/v/7k1ehaE0bdU/?t=8271) um but i don't know i still think we're generally just going to be better off in a world where this is like deployed pretty evenly and um you know it's i guess here's another analogy that i think about there's like bugs and security holes in basically every software every piece of software that everyone uses so if you could go back in time a few years knowing the security holes that we're now aware of you as an individual could basically like break into any system um ai will be able to do that too it'll be able to probe and find exploits so what's the way to prevent ai from going kind of nuts i i think part of it is just having ai widely deployed so that way like the ai for one system defends itself against the ai that like is is potentially doing something problematic in another system i think it's like ai wars

[2:18:45](https://realalpha.app/v/7k1ehaE0bdU/?t=8325) that's not wars i think it's just like it's uh i don't know it's i think it's a very it's sort of like why there are guns right it's like because i mean there's boy like part of it is hunting wars part of it is hunting no no and part of it is like the people can defend each other yeah yeah and it's so it's like virus software yeah it's like i don't i don't think you want to live in a world where like only one person has all the guns yes you certainly don't want to live in a world where only the government has the ai yeah yeah and especially not a world where only a government has the ai and it's not our government yes so yes which i mean i think is is part of the issue is like when people talk about trying to lock this stuff down like i just i'm skeptical that that's even possible because i kind of think like if we

[2:19:37](https://realalpha.app/v/7k1ehaE0bdU/?t=8377) lock it down, then we're going to be in a position where the only people who are going to have access to it are the big companies working on it and the Chinese government that steals it from them. So I kind of just think, no, what you want to do is get this to be open source, have it widely available. Yeah, some adversaries might also have access to it, but the way that you defend against that is by having it built into all these different systems.

[2:19:58](https://realalpha.app/v/7k1ehaE0bdU/?t=8398) I think that's a realistic... Pragmatic perspective because I don't think you can contain it at this point. I think it's far too late, especially when other countries are working on it. It's far too late. It is what it is. It's happening. And I think the guardrails, as you said, are really important. I have to pee so bad. So let's pee and come back because I want to talk about a couple other things. We'll be right back, folks.