Model PMs have to discover what training produces

Transcript

From the transcript Lenny's Podcast publishes with the episode.

1. 2:15

   to pre-training, post-training launch. What's something that you think people maybe misunderstand about how models are created or something that may surprise people about how models are made? Yeah, it's a good question.

2. 2:29

   What I'll say, especially coming from a product management lens is one thing that I think, you know, when you typically talk about product management for anything, including a model, you talk about what are the goals you have for a model, right? What are the capabilities you want to set? What are the outcomes you want to drive? What goals do you want to hit on a set of evals? And often when I talk to customers or even internal users, you know, they'll ask me, okay, what are the goals that you're setting?

3. 2:55

   And they'll sort of expect that we can Really make sure that the model will deliver on those goals in a very traditional product sense. And I think for me, what is often been surprising but also exciting about model development and being a PM for the model is you should do that and you should have an intentional view, but the model isn't predictable. And so it's kind of like, like Snape's potions class, right?

4. 3:23

   So you can put all the ingredients in and you should be thoughtful about what you expect. But you also have to expect that there will be a little bit of magic and that that magic is going to be surprising and that magic is going to be unique. And so your job as a PM is not just to try to shape what goes into the model, but it is also to look at what comes out the other side and really identify where does the model shine, where does it actually look unique in ways that surprise you, and then be able to take that and shape that into something that is amazing for users.

5. 3:55

   It's so interesting that it's just like this code that we have written that we feed data and we need to, like we don't exactly know what's going to happen and how it works. I recently had an interpretability research person on the podcast from Anthropic and we talked a lot about this, this idea that models are grown, not built. And just to kind of reflect back what you're saying, there's these goals that researchers have.

6. 4:17

   For this next model, we're going to get better at writing. We're going to try to be funnier. We're going to try to become better engineers. And you're saying it's unpredictable exactly what ends up at the end. They put in these ideas. They change the data that it's trained on. Is there anything more there about just this kind of like unexpected, I don't know, process of training?

7. 4:36

   You know, of course, there's brilliant research that goes into these models. And so many aspects of it are predictable in the sense that we are consistently monitoring these runs. We have researchers who are tracking them, who are identifying when to intervene, who are seeing different metrics climb up as the run progresses. And so we are making it as much of a science as possible. But I think for me, what is really amazing as a PM is you also need to really understand the vibes of the model on the other side of it, too.

8. 5:04

   So, yes, you can understand that a model has improved on a certain capability. But where does that capability really shine through? How does that showcase that downstream in a product? How do you feel communicating with the model in terms of its behavior and persona? And there's a number of these aspects that we definitely build into training. But also as we understand the model and as we evaluate it and as we test it, we always learn more.

9. 5:26

   We always learn something new, I think, about what the model can do.