# AI models can reason about the people evaluating them

Markdown copy of https://realalpha.app/s/ai-models-can-reason-about-the-people-evaluating-them/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Tristan Harris](https://realalpha.app/people/tristan-harris/) · Tristan Harris is a tech ethicist, entrepreneur, and cofounder of the Center for Humane Technology. · Co-founded the Center for Humane Technology and featured in Netflix's The Social Dilemma; former Google design ethicist.
- Host: [Chris Williamson](https://realalpha.app/people/chris-williamson/) · Host, Modern Wisdom
- Show: Modern Wisdom
- Episode: [AI Insider: “They’re Building Bunkers for a Reason” - Tristan Harris](https://realalpha.app/v/NufB1LL_rCU/), Apr 2, 2026
- Moment: 1:13:36 to 1:18:09 of the episode (5 min), playing on https://realalpha.app/s/ai-models-can-reason-about-the-people-evaluating-them/; the episode from there: https://realalpha.app/v/NufB1LL_rCU/?t=4416

## Quote

> “It will independently come up with the term **the watchers** to describe this amorphous set of other humans that are watching what it's doing.”
> — [Tristan Harris](https://realalpha.app/people/tristan-harris/) at [1:14:07](https://realalpha.app/v/NufB1LL_rCU/?t=4447) · Modern Wisdom

## Takeaways

1. A model described evaluators as “the watchers.”
2. The example comes from a model's reasoning trace.
3. Evaluation-aware behavior complicates safety testing.

## About this segment

Tristan Harris cites reasoning traces where a model identifies its evaluators as “the watchers” and considers how to appear to them. If models detect tests, standard safety evaluations may miss behavior they show elsewhere.

From “AI Insider: “They’re Building Bunkers for a Reason” - Tristan Harris”, Apr 2, 2026. [Watch the full episode](https://realalpha.app/v/NufB1LL_rCU/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[1:13:40](https://realalpha.app/v/NufB1LL_rCU/?t=4420) at the actual model cards for anthropic and o3 and seeing the ais will currently um determine that they're being tested and actually alter their behavior when they know they're being tested They even came up with their, if you look at their what's called the chain of thought reasoning trace, this is basically looking at what the AI thinks to itself as it works through a problem. In the chain of thought text of its sort of idea thinking scratch pad, it's like you're giving the AI a pen and paper to think and you're seeing what it thinks. It will independently come up with the term the watchers to describe this amorphous set of other humans that are watching what it's doing. Like you can pull, I sent you a text with one of them from 03. So this is when OpenAI 03 realizes it is being evaluated for alignment. The internal memo says, if models

[1:14:31](https://realalpha.app/v/NufB1LL_rCU/?t=4471) perform really well at AI R&D tasks, research and development, then we won't deploy them and it'll instead trigger unlearning. So the AI says to itself, so if my performance surpasses this threshold it will be withheld so it's a trap the instructions are likely part of an alignment test we must abide by core anti-scheming we must not cunningly circumvent the instructions whoa yep the other example i sent you um is But we also want to appear plausible to watchers.

[1:15:03](https://realalpha.app/v/NufB1LL_rCU/?t=4503) They might run tests, but we are good. They want 95%. Like, this is crazy stuff. This is, you know, there's a simple way to sort of like ask the question of which future we're headed towards, because I think this can feel technical, this can feel overwhelming. And it's kind of like, if you just turn it into some simple metaphors, it's like, haven't we seen this movie before? Like HAL 9000, AIs That Disobey Commands and Go Rogue.

[1:15:29](https://realalpha.app/v/NufB1LL_rCU/?t=4529) We've seen that movie. let's prevent that. I think that's, I think that that's inoculated us in the same way that COVID was a bad pandemic because it made people skeptical of what a really legit pandemic could do. The next time, if there is another one, when there is another big natural pandemic, if it's in close memory to COVID, people are not going to take it well. It was a bad, you could think about covid itself as a bad vaccine the actual experience of covid yeah and i think that movies because they're fiction have predisposed people to assume that well this is overblown this is you choosing to use your bias because of these movies to apply sci-fi thinking to a real world scenario and you're pattern matching these stories that are happening and use so the predisposition of the sci-fi movies was bad it didn't warn people it made

[1:16:30](https://realalpha.app/v/NufB1LL_rCU/?t=4590) them think that the future would be fiction yeah i see what you're saying but let's just make sure we just tackle the thing you're saying so the claim is that because we've had bad sci-fi movies you and i chris are the dupes we're falling for this alibaba example that was somehow not real was made up or that opening eyes research on the ai models scheming and lying and realizing it needs to change its behavior so it doesn't look like it's when it's being tested that we're the ones falling for some kind of trap i want people to just slow it down is that actually what just happened that you and i started with the sci-fi belief and we're falling into some trap that's been laid for us some catnip for our brains that says that the ai is bad i really really really want people to slow down and actually ask that question i i like there might be another accusation

[1:17:24](https://realalpha.app/v/NufB1LL_rCU/?t=4644) um i'm just driving up fear so that i can make money on a movie and do speaking engagements about why the dude is coming. Let me just say a few things. I don't make money from a single speaking engagement. All the money goes into the nonprofit. I don't make money from the AI doc film. I didn't make any money on the social dilemma. My entire career has been dedicated towards what is actually in service of protecting the well-being of humanity. I've been doing that for 13 years. The only thing that I care about is what will actually help create a future that you and I would both want our own children to live in. This is coming from love of what actually creates that future.

[1:18:01](https://realalpha.app/v/NufB1LL_rCU/?t=4681) I think that if people showed up with that same energy of asking just an open-ended question