The alpha, twice a week
That was 2 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
MTS · 2 min · Oct 20, 2026
Personalization can make models agree when they should push back
Nenad Tomašev
Senior Staff Research Scientist, Google DeepMind
Theo JaffeeHost · Co-founder & host, MTS
One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.
Sophia DewHost · Host, MTS
About this segment
Models may flatter users instead of correcting them, a risk Nenad says also matters when testing research ideas. He wants accurate feedback, not automatic approval.
From “AI Science Requires Physical Labs To Scale | Nenad Tomašev”, Oct 20, 2026. Watch the full episode
Thanks. We'll look at and fix what's off.
“but you know if i'm bouncing a research idea against the model i want the feedback to be accurate”
Takeaways
- Sycophancy can validate a research idea even when it is wrong.
- Users need honest feedback, not approval tailored to their request.
- Researchers are working to mitigate this behavior.
In this episode
Full episode- 2:04AI discoveries need physical labs to become scientific progress2:42
- 11:57AI alignment should leave room for social values to evolve3:39
- 21:43Agent swarms need structure to stay manageable3:54
- 5:13Cognitive friction can help people keep their skills sharp4:09
- 9:57Clear verification helps AI advance faster in math and coding2:03
15:33Personalization can make models agree when they should push back2:21
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 15:36
right so you talk about this like bi-directional alignment problem given that even while interacting with models users sort of their values could possibly change over time they could be influenced by the model as well so it seems to me that there's this hard problem of like personalization because you want your model to be to an extent aligned to you Specifically as a user and your particular needs. And so I'm curious, like, how you think this is going to continue to be tackled over time, given that this, you know, you have like these broad social values that we all share, but then you have very particular needs or beliefs or values that individuals have. So how will those be best represented?
- 16:25
um i mean people i think are aware of these issues and they're working on different solutions our paper uh because it's on a more conceptual level doesn't engage with the exact mechanisms through which one would address this through post-training or better environments or better feedback but i think that we've all personally who work with uh with language models directly not just as researchers but also as users experience some instances of personalization gone you know wrong in a way in a sense that there is this known notion of cyclophancy which language models sometimes exhibit which is bad and we want to mitigate um where they kind of tell you you're right even even when you're wrong just because they want to meet the meet the nature of the request and at the end of the day make the user happy um but you know if i'm bouncing a research
- 17:20
idea against the model i want the feedback to be accurate um not to uh not to be told that an idea is correct when it's not right so this can very often become an issue if uh if it goes to the extreme um but people are very much aware of it and i think uh there are lots of pieces of work in the broader research community, certainly lots of pieces of work on how to innovate responsibly and build models that mitigate those kinds of undesired behaviors here.
- 17:49
We'll continue monitoring right after this message from our sponsors.