# Post-training improvements made GPT-5 more reliable

Markdown copy of https://realalpha.app/s/post-training-improvements-made-gpt-5-more-reliable/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Max Schwarzer](https://realalpha.app/people/max-schwarzer/) · Researcher, OpenAI · Led OpenAI's post-training for o1 and o3, and helped develop its o1-preview reasoning model.
- Host: [John Coogan](https://realalpha.app/people/john-coogan/) · Co-host, TBPN · Co-founded Soylent and Lucy; OpenAI bought his show TBPN in April 2026.
- Host: [Jordi Hays](https://realalpha.app/people/jordi-hays/) · Co-host, TBPN · Founded Branded Native; co-founded fintech Capital, which Rho bought.
- Show: TBPN
- Episode: [OpenAI LIVE: Greg Brockman, Mark Chen & More](https://realalpha.app/v/gaImbWPGgtU/), Aug 7, 2025
- Moment: 3:08:28 to 3:13:21 of the episode (5 min), playing on https://realalpha.app/s/post-training-improvements-made-gpt-5-more-reliable/; the episode from there: https://realalpha.app/v/gaImbWPGgtU/?t=11308

## Quote

> “And if you do that, all of a sudden, the model **stops saying things that are false**.”
> — [Max Schwarzer](https://realalpha.app/people/max-schwarzer/) at [3:12:01](https://realalpha.app/v/gaImbWPGgtU/?t=11521) · TBPN

## Takeaways

1. Post-training work produced gains beyond incremental pretraining.
2. Real user behavior helped identify where models failed.
3. Reward functions can penalize false or harmful responses.

## About this segment

OpenAI researcher Max Schwarzer describes rebuilding GPT-5’s post-training stack and targeting real-world failure patterns. He says better reward functions can reduce false claims and improve safety.

From “OpenAI LIVE: Greg Brockman, Mark Chen & More”, Aug 7, 2025. [Watch the full episode](https://realalpha.app/v/gaImbWPGgtU/)

## Transcript

From the transcript TBPN publishes with the episode.

[3:08:37](https://realalpha.app/v/gaImbWPGgtU/?t=11317) Like what can you give us on without, you know, immediately getting poached? What what what can you give us on on the actual innovation that went into GPT five from a post post training perspective? What are the like the the kind of keywords and and paths in the tech tree that we should be digging into over the next few years to understand how this works? You know, I would say the thing

[3:08:56](https://realalpha.app/v/gaImbWPGgtU/?t=11336) that is most impressive to me about g b t five is how much getting all of the details right matters. Mhmm. Like, when I look at g b d five, you know, we had an early version of this thing a while ago that was kind of okay, but clearly did not meet our bar for revolutionary. Mhmm. And we're trying to figure out, you know, why why is that not as good as it should be. And the team basically just went off and did a deep dive over a couple of months of just completely rebuilding the post training stack for this model. And turns out that when you do that, you get what would have taken, you know, another order of magnitude worth of pre training improvements to to produce.

[3:09:35](https://realalpha.app/v/gaImbWPGgtU/?t=11375) How much are you thinking in post training, in research about let's forget the benchmarks and just focus on user satisfaction like NPS score basically or like users user minutes or any of these other the real benchmarks. Yeah. The intangibles about revenue. But yeah. The feeling and the and the joy and the actual value that's delivered, because Studio Ghibli was a delightful moment. It wasn't a benchmark.

[3:10:04](https://realalpha.app/v/gaImbWPGgtU/?t=11404) Yeah. I I think So that was something that we took very seriously for GBT five. Was like, look at what people are actually doing with chat g b t, and look at where the model is failing them. Mhmm. Either in the sense that the model is like, sort of like you said, it's not enjoyable to use. Yeah. And so we we did, I think, make a lot of progress on that. Like Mhmm. G b d five is much more engaging than our previous really smart models. Like o three, I I don't know if you guys talked to o three in the past. It's Yeah. A bit bland.

[3:10:30](https://realalpha.app/v/gaImbWPGgtU/?t=11430) Sure. And g b d five, I think, has a lot more character, is a lot more more interesting. But then also, like, I think for we really care about just actually being accurate. Like, giving if a user is trying to do something economically valuable with our model, we wanna make sure it lands correctly. Yeah. And so what we did there is just like, look at act the actual distributions of what people are doing with our models in the real world, figure out where the models are going wrong, build interventions to target it.

[3:10:58](https://realalpha.app/v/gaImbWPGgtU/?t=11458) And that was where, you know, we got, I think, the most impressive improvements in g d five. Like, o three would just get things wrong and not tell you it wasn't sure it was it was incorrect, and g p d five is much much better about, like, actually being honest when it thinks it might not know.

[3:11:13](https://realalpha.app/v/gaImbWPGgtU/?t=11473) Yeah. How explicit is are all the different pieces of the post training pipeline? Like, you have you have you know, post training. You have stop hallucinating. Give me the real facts. You have make sure the text the the flavor the tone is pleasant. There's so many different things to optimize for. How much of that is, like, try and just blend it all up into one thing versus, like, explicit passes, chunk it out, like, split it up? How how much can you decompose the problem?

[3:11:45](https://realalpha.app/v/gaImbWPGgtU/?t=11505) So, you know, my background is in reinforcement learning, and I think, you know, when you look at something like this, the magic is in the reward function. Right? It's in what you're actually telling the model to be good at. Yeah. And so, fixing things like hallucinations, to a huge extent, is essentially a function of just fixing the reward function. Yeah. Actually making it so that the model is reliably penalized for saying something that's false. And if you do that, all of a sudden, the model stops saying things that are false. Ditto for safety. Right? You know, on the livestream, Sachi talked a bit about the the way we've changed safety for this model.

[3:12:20](https://realalpha.app/v/gaImbWPGgtU/?t=11540) And to a huge extent, it's it's just a function of we are we're actually putting out a paper today on the new safety stack for this model. And the core insight in that paper is just figure out what you actually wanna optimize for, which in our case is helpfulness conditional on not saying something that's actually dangerous or harmful. You know, write that down, figure out what that means as a reward function, then optimize it for it.

[3:12:42](https://realalpha.app/v/gaImbWPGgtU/?t=11562) Mhmm. It's really not magic at all. It's just again, it's it's what I said earlier. You gotta get the details right. You know, if at any part of that process you screw it up, the model will be unusable.

[3:12:51](https://realalpha.app/v/gaImbWPGgtU/?t=11571) What's your current thinking on spiky intelligence? And is there some is there some flywheel that you can get started where you're identifying low points that aren't spiky enough and then you're like almost automatically setting up the infrastructure, the eval to then RL against to create a spike?