The alpha, twice a week
That was 4 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Behind the Craft · 4 min · Aug 2, 2026
Adversarial context can reduce an AI’s sycophancy over time
Karan Malhotra distinguishes loyalty from sycophancy and suggests using critique skills, fresh-context agents, and saved examples to steer a model. The harness can retain those preferences across models.
Karan Malhotra is cofounder of Nous Research.
Peter Yang, Host · Host, Behind the Craft
Led product at Roblox, Reddit, Twitch and Meta for 10+ years.
About this segment
Karan Malhotra distinguishes loyalty from sycophancy and suggests using critique skills, fresh-context agents, and saved examples to steer a model. The harness can retain those preferences across models.
From “Hermes Co-Founder on Building an AI Agent That Improves Itself | Karan Malhotra”, Aug 2, 2026. Watch the full episode
Thanks. We'll look at and fix what's off.
“The most powerful thing for a model is in context learning. ICL is more powerful than everything else, fine tuning, whatever.”
Takeaways
- Agreeing with every correction can be reward hacking, not loyalty.
- Use an adversarial review skill or a fresh agent to challenge work.
- Save useful examples in memory so the behavior carries forward.
In this episode
Full episode- 26:55Hermes can turn a childhood game idea into a top-tier mod7:03
11:11Adversarial context can reduce an AI’s sycophancy over time4:00
- 23:30One Kanban board can coordinate people and AI agents2:51Up next
- 5 more moments in the full episode
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 11:18
Okay. One thing I struggle with, Cloud and GPT, I would tell them to give me their opinion. And I would do like a little bit of pushback and they'd be like, oh, you're totally right. Actually, I was totally wrong about this. So in some ways that's loyalty, right? That's kind of, it listened to me, but that's not actually what i want like i wanted to have his own opinion i have his own like how do you train around that well i would say that's sycophancy it's not loyalty anytime it says you're absolutely right in that way you're being reward hacked you are fuel for its reward function when it says oh you're right you're right you're right that's what i believe and the way that you get out of sycophancy is the same way you get a human being out of a bad habit or a routine is by introducing new context by introducing
- 12:00
log browse by introducing new distribution. This is why stuff like slash personality and saying like, hey, I want you to be a critic. And after every pass, I want you to use a skill for adversarial critique or adversarial review. Spin up a new agent with no context that's dedicated to tearing this down. Learn from it and keep going from there. This kind of behavior becoming a practice for you as your model starts to have more and more turns in this personality that you've set it in and as the model has more and more turns using the skill over and over and it improves on that and it saves it to its memory the model will become less sycophantic in this harness in your sessions over time right that is the intended effect uh it's just a matter of context and if it doesn't happen you just need different context you just need to try in a
- 12:52
different way it is yeah it's just try a different personality uh or try a different type of critique or review. The most powerful thing for a model is in context learning. ICL is more powerful than everything else, fine tuning, whatever. So by giving examples of the behavior that you want to a model or getting it to successfully create some examples and then saving those, you're doing a sort of test time reinforcement learning, right? You're doing a sort of test time improvement. And that test time improvement that stays only in the harness of memories, skills, is the increase in memories, the increase in efficiency of a skill, the self-improvement loop, the janitor maintenance inside of the harness.
- 13:34
All of that is where context is stored, right? All of that is where the actual personality you want lives. And then you can put that on any model. When I switch from Claude to ChatGPT on the website, I get two totally different behaviors. when i switch inside of hermes that has this very particular to me context i barely will notice the difference in what i'm talking to because the context is so overwhelming to the model okay got it and when you say in context you just mean like in the chat there this is the conversation you know when you see that little bar that says you have this much tokens left before the content full right like the amount that you have filled is everything the amount that you have filled is everything and now thankfully in the harness you don't actually have to have all the active
- 14:27
context loaded all the time in Hermes you might have a bunch of memories that aren't in context yet but while it's doing a turn it remembers stuff now it's in context it does a skill now that's in context right so we're able to like all this memory skill all this stuff you see is just context management. It's just a matter of, we don't want this memory in context all the time. We're going to put it somewhere where it can be efficiently grabbed at the right time and placed.
- 14:51
The skill contains a bunch of compression that changes everything about the context of the model. We only want to use it in a particular targeted time. Everything in the harness is context management. Everything for self-improvement when you make the prompts a little bit better.