Post-training could draw more investment than pre-training

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 44:39

   We touched on it a little bit, but I think that with the advent of agents and products that think and can act and reason, there's going to be this kind of new wave around RL. And I have a deep belief that that will become one of the most important product techniques kind of of the next season or at least the next few seasons. And RL is reinforcement learning. Yes, yes, exactly. Like, I believe we will see, you know, just as much money spent on post-training as we will on pre-training and in the future more on post-training. We talked a little bit about Nathan Lambert's study where his review was that, you know, when a model hits 30 billion parameters, it makes more sense to kind of fine tune and optimize that.

2. 45:20

   you know 50 percent of developers uh according to surveys are now fine-tuning uh and we know fine-tuning is good but like if you actually go through the full loop you can get better results so i think there's there's a bunch there and i think there's a whole new set of infrastructure and platforms and uh companies that will be created that are all around this part of the stack and so um i think it's an exciting time to be in the platform space but it's also an exciting time to be starting companies and be thinking about those problems i want to make sure people truly understand what you're saying here, because not everyone truly understands post-training, pre-training. What's the simplest way to understand the difference there and just why it's such a big deal that investment is moving to post-training?

3. 46:02

   The way that I think about it is, you know, to create a foundation model, it requires a tremendous amount of compute, a tremendous amount of science expertise, as we're seeing who's uh which the cost for scientists or the average value is raising dramatically and i think you know an expertise that we've seen it like isn't everywhere in the world right now and so it's just a big capex investment to do that and with this explosion of models that we talked about in the beginning there's a lot of good models to choose from for different domains and so i think that you just get more leverage economically you get more leverage from a taste perspective of how you actually want to steer a model if you're actually doing reinforcement learning or some sort of fine tuning to actually start to optimize what's off

4. 46:54

   the shelf for some outcome like price, performance, quality. If you think about that, that's not crazy, right? Like, you know, ranking is an age old optimization problem where you don't want to just take what's off the shelf because there's like amazing frameworks and UI and kind of components that, you know, the world is React components that are out there. You still want to tailor the experience to a set of use cases or set of people. I think it's just the same kind of industrial logic. So in practice, what this means is there's like a GPT-5 model. You're saying there's a lot of opportunity and a much more efficient way to spend money, which is take something like that and then train it on additional custom data that you have, whether it's data or just reinforcement learning, maybe even with humans to align it with what you wanted to achieve.

5. 47:41

   Yep, and it could be your own data, it could be data that you buy, it could be synthetic data, it could be, you know, something else. But I think that, I think that we're kind of going to start to see, you know, more and more companies and organizations kind of start to think about how do I adapt a model rather than how do I take something off the shelf as is or invest a bunch of money in building my own models.

6. 48:06

   Yeah, I forget. I know Cursor, when he was on the podcast, he shared that they have a bunch of models that support your experience with Cursor. And over time, they're just going to have their own thing. I forget who it was, Windsurf or one of those guys just uses their own model now. They don't just plug into Claude. I'm much more in the model system camp. Like I believe in model diversity. I think that inexperienced like clod uh like sonnet 4 is awesome for a set of use cases versus gpt5 is different for different use cases i think that there's some tasks where you care about the latency of the model you want you're like cool with the thinking time or you kind of want a quick retrieval and things like that like i i think the the beauty is there's a lot of models that can kind of help you achieve that and so i'm much more in the like model system uh rather than one model

7. 48:56

   to rule them all. Is that the right term? I've also heard ensemble model, ensemble of models. I think about an ensemble of models as a set of multiple models that then you can, you know, fine tune and deploy independently. But, you know, at this point, we're all making up different terminology to define things that we like have deep beliefs on that have like, you know, limited sets of data points because everything is moving so fast.

8. 49:18

   Yeah. With that, we've reached our very exciting lightning round.