# Students can practice AI judgment by comparing model answers

Markdown copy of https://realalpha.app/s/students-can-practice-ai-judgment-by-comparing-model-answers/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Shaan Puri](https://realalpha.app/people/shaan-puri/) · Co-host, My First Million · Sold Bebo to Twitch in 2019; co-founded the crypto newsletter Milk Road.
- Host: [Chris Williamson](https://realalpha.app/people/chris-williamson/) · Host, Modern Wisdom
- Show: Modern Wisdom
- Episode: [Crypto Scams, Dopamine Websites, Tasteful Slop & Vape Shop Energy - Rabbit Hole #6](https://realalpha.app/v/n9NhSWUrm38/), Oct 26, 2026
- Moment: 1:27:21 to 1:31:40 of the episode (4 min), playing on https://realalpha.app/s/students-can-practice-ai-judgment-by-comparing-model-answers/; the episode from there: https://realalpha.app/v/n9NhSWUrm38/?t=5241

## Quote

> “you give the student **five LLM outputs** and they have to just tell you which one is the best and why.”
> — [Shaan Puri](https://realalpha.app/people/shaan-puri/) at [1:31:11](https://realalpha.app/v/n9NhSWUrm38/?t=5471) · Modern Wisdom

## Takeaways

1. Recall practice builds learning more than rereading.
2. AI can bypass the effort that helps information stick.
3. Ask students to compare model answers and explain their choice.

## About this segment

The group weighs AI convenience against the retrieval practice that builds memory, then proposes having students rank model answers. That tests judgment instead of trying to police AI use.

From “Crypto Scams, Dopamine Websites, Tasteful Slop & Vape Shop Energy - Rabbit Hole #6”, Oct 26, 2026. [Watch the full episode](https://realalpha.app/v/n9NhSWUrm38/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[1:27:28](https://realalpha.app/v/n9NhSWUrm38/?t=5248) i was at university now i left university 15 years ago. So I was at uni for quite a while. I was at uni for half a decade. And during that time, Facebook was around and had been around since 2006. And I left in like 2011, something like that. But it wasn't compelling. I was on a BlackBerry, I think. The level of distractibility over the last decade and a half has gone up through the roof. And not only that, but now you have to not use the equivalent of the empty process calories cognitively of relying on AI.

[1:27:59](https://realalpha.app/v/n9NhSWUrm38/?t=5279) so even if you can focus on your work there is now always this permanent shortcut that's there which is a simulacrum of you learning but at least based on most of the studies it was that one that came out of harvard somewhere in boston i think that suggests people that students that are using ai retain so much less information etc etc i know that this is this will be triggering all of your techno optimist no no no i was gonna i was gonna build on your argument for a second really yeah i was going to build on it so um uh you remember the old school um brain imaging of black cabs in london people that learned to black cab drive and they have to remember it for about three years structural changes in the brain structurally changes the location of the brain but what's fascinating i always want i would like to fund this is follow-up well not just follow-up

[1:28:46](https://realalpha.app/v/n9NhSWUrm38/?t=5326) but uber drivers in theory are doing as many miles driving around maybe even more um but i i would be willing to bet there's almost no structural change or a tiny percentage comparatively to the black cab drivers. Within that, there's something quite fascinating because it seems to be the big thing for change at the brain level is the risk of failure. And if you ask the AI something and there's no risk of failure, even I think the one thing I'll do now when I'm asking Claude or I'm asking ChatGPT something, is to just ask it, pause, try and think my own answer, and then contrast and compare versus purely flop. I don't know whether risk of failure may be, but friction is pretty much... What's the difference between friction and risk of failure? What do you mean? Something can be difficult and you can

[1:29:37](https://realalpha.app/v/n9NhSWUrm38/?t=5377) still know that it would go well. I think that it's the difficulty, it's the removal of the difficulty. Learning the Ebbinghaus forgetting curve, being able to remember something isn't to do with failure it's to do with friction could you elaborate a little bit more that like doing the rep yes whether it was like if you reread a page yeah but you could like reread the wrong page or something that wouldn't change how much you retain What changes how much you retain is, are you self-testing? Are you closing the book and trying to recall what's on the page? Or are you just reading it over and over and over again?

[1:30:07](https://realalpha.app/v/n9NhSWUrm38/?t=5407) So maybe the failure thing plays into that. Well, I think in that, I think the friction helps, but then if you fail, you're significantly more likely to remember, which makes sense because of the pain that's associated. Motivating force. Failure is friction. This might be a stupid distinction, but the cost of failure, not knowing where you're going as a tax driver is like bankruptcy right you know like the like if you can put yourself in a position where like if i don't know this i can't perform said function but even at university now it's like if you're studying a certain topic like i can produce the materials that i've been asked to produce the vast majority of people are just going to go that route i had a i did this film like earlier this year with a lady who teaches film at a university in hong kong and she's like i can't

[1:30:53](https://realalpha.app/v/n9NhSWUrm38/?t=5453) stop the students just submitting like 80 pages of bullshit coming out that comes out of an LLM and I was like why didn't you just tell them to submit the prompt just cut all of this out and just be like whoever writes the best prompt but they're just gonna they're gonna prompt for the prompt just grade the prompt yeah there's no there's no end to it but they'll still prompt for the prompt I think what you should do is okay grade it with the LLM then you put the you make the student the greater so you give the student five LLM outputs and they have to just tell you which one is the best and why.

[1:31:23](https://realalpha.app/v/n9NhSWUrm38/?t=5483) That's pretty good. Yeah. And then that forces them to at least have taste or a judgment and have to think. Do you guys think there's going to be like brain gyms? Like, you know, there weren't gyms