Benchmark gains mix targeted training and generalization

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 7:48

   So, so let, let me go into that a little, cause I think it's a, it's a killer point to understand and it is surprising. So I think you're, you're very right to be surprised. In fact, if you look at AI benchmarks, they are all correlated with each other. Even if the benchmark, the scores across models are correlated with each other, even if the benchmarks are in nominally different domains. So maybe you don't expect, you don't find this so surprising if it's like I've got one benchmark that's, you know, graduate chemistry questions and I've got another one that is coding puzzles.

2. 8:25

   And the models that are good on chemistry are also good on software engineering. Maybe they should be correlated largely by the models themselves. Like as the models get better, they're going to get better across these, you know, disparate benchmarks. But this exactly. But that's that's the surprising point. So all that you need to be true to see this linear trend over time is basically every time, you know, Opus 4.5, Opus 4.6, 4.7, GPT 5.2, 5.4.

3. 8:55

   every time these come out, if they get better on all benchmarks at once, then you're going to see this linear trend, this big overtime linear trend. Like that's two sides of the same coin, statistically. Why does this happen? Like this is a huge question. Like you're right to find that suspicious. And I would say there are two explanations that we see. And it's an important question, which of these explanations is more true. And of course, the truth is some mix, but But let me give the two explanations.

4. 9:27

   One is the AI companies are making darn sure that their model gets better at all of these benchmarks with each new generation of model. And the main mechanism you might imagine them doing that if they're like looking around, okay, people care about chemistry, people care about software, people care about operating, but, you know, answering my email. Like, they would collect training data, one form or another, that would help the model learn how to do these tasks.

5. 9:58

   And that's something of a, you know, very manual, human-in-the-loop process of just saying, hey, we're doing product development. Like, they want this feature, they want that feature, our users want all these features. And the way you do that in machine learning is you collect training data and throw it in. I'm oversimplifying, but that's that. sort of like a shallow way to make it happen. So I'll call that shallow.

6. 10:22

   The other one is deep. The deep way to make it happen is if the model general, is if the AI system can generalize very, very well. So yeah, you never trained it on chemistry, but you trained it a lot on math and software engineering and it just got super smart. And with a modicum of chemistry basics, it was able to derive the rest from first principles sort of thing. This is like a deep form of generalization.

7. 10:49

   Both of these happen to some extent, like kind of the revolution with GPT, even GPT-3 around that era. This is like quite a while ago was, oh, if you train it to like predict the next word, then it actually gets better at a wide, wide range of language oriented tasks that people had been trying to handle individually. Suddenly it just got good at everything. And that wasn't because anyone was like, oh, let's have data specifically around sentiment analysis or syntactic structural analysis. It wasn't anything like that. It was just give it this very general thing. But now we're onto this world where you have to train it specifically. Right now, there's a huge industry of collecting all this rich training data.

8. 11:32

   And it's really unclear how much generalization you're getting from in this deep way. And if it's And anyway, that's sort of a big question because that deep generalization means you could really see, if that happened in a rich way where you just didn't have to even be trained on something at all to gain capability in that area, then you might have a radically superhuman AI that would be pretty hard to understand, to bound its capabilities.

9. 12:02

   It's still a little counterintuitive that this would be linear. And maybe part of the question is, like, are we talking about, you know, linear across generations of model releases or linear within some regime, like segmentally linear or? The thing I can say is, we have like a statistical methodology. So benchmark score, like what are we talking about here? Our unit here is a score from zero to 100%. And typically, the way these scores go just historically, empirically, is like they follow like an S curve, like they start out sort of bad, then they get better, they get better pretty rapidly. And then you can only get up to 100%. So then they sort of level off takes them a while to get that last, you know, little chunk, and then they're at 100%.

10. 12:55

    So you've got these, like, sigmoids, is what that S-curve is called. We've just got a statistical methodology for stitching sigmoids together. And what comes out is a linear function. And this looks like this is an artifact of the statistics, to be clear. But this, sure enough, looks very regular in its own statistical analysis terms over across generations of models. Like, absolutely. Like it's been, if anything, it accelerated a little with the development of so-called reasoning models around late 2024, where the models started to get good at math and software engineering and things that require lots of logical reasoning, maybe like a higher linear slope.

11. 13:38

    But this is just like, that's a statistical artifact. You now have to ask, okay, well, if your score is like one model, the score is 150 and the next model score is 152, like what are those two points mean on your benchmark aggregate index? Now, I think part of what I'm asking myself as I hear this described as linear is, does it say that in spite of the model, the Frontier Lab vendor claims that, you know, Fable is like dramatically better than Opus or that GPT-6 is like the step function, you know, improvement over GPT-5.6, you know, linear implies to me that it's kind of incremental and there's really been no step functions. This is a really great question. And yeah, Yeah, we believe this is one of the big uses of this Statsy tool we've got is you can try to find, you know, trend breaks, and you can try to find accelerations. And we mostly don't see that. Yeah, like one way to put it, that is everything just about is on trend, Fable, Astra, whatever is on trend. But the trend is crazy. Like, you got to you got to hold both of these together.