# AI Progress Explained: From Math Olympiads to Navier-Stokes Markdown copy of https://realalpha.app/v/lz9J-Dbj2zw/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy. Greg Burnham says AI progress remains steady, but measuring whether models can generate new research ideas is still an open challenge. - Show: The TWIML AI Podcast - Hosts: [Sam Charrington](https://realalpha.app/people/sam-charrington/) - Guest: [Greg Burnham](https://realalpha.app/people/greg-burnham/) is a leads capabilities research at Epoch AI. - Published: Sep 29, 2026 - Length: 1 h 7 m ## The best minutes - [20:16](https://realalpha.app/v/lz9J-Dbj2zw/?t=1216) [AI math has advanced from school problems to open research](https://realalpha.app/s/ai-math-has-advanced-from-school-problems-to-open-research/) (7 min) - [30:45](https://realalpha.app/v/lz9J-Dbj2zw/?t=1845) [AI uses accumulated knowledge and persistence in hard math](https://realalpha.app/s/ai-uses-accumulated-knowledge-and-persistence-in-hard-math/) (6 min) - [7:44](https://realalpha.app/v/lz9J-Dbj2zw/?t=464) [Benchmark gains mix targeted training and generalization](https://realalpha.app/s/benchmark-gains-mix-targeted-training-and-generalization/) (7 min) ## Also worth watching - [14:56](https://realalpha.app/v/lz9J-Dbj2zw/?t=896) [Compute and data underpin most measured AI progress](https://realalpha.app/s/compute-and-data-underpin-most-measured-ai-progress/) (5 min) - [36:12](https://realalpha.app/v/lz9J-Dbj2zw/?t=2172) [Experts judge novelty by how much of a hint humans needed](https://realalpha.app/s/experts-judge-novelty-by-how-much-of-a-hint-humans-needed/) (8 min) - [54:13](https://realalpha.app/v/lz9J-Dbj2zw/?t=3253) [Current harnesses do not make models learn strategy on their own](https://realalpha.app/s/current-harnesses-do-not-make-models-learn-strategy-on-their/) (5 min) - [49:40](https://realalpha.app/v/lz9J-Dbj2zw/?t=2980) [Models still struggle with open-ended work and learning by practice](https://realalpha.app/s/models-still-struggle-with-open-ended-work-and-learning-by/) (5 min) - [44:12](https://realalpha.app/v/lz9J-Dbj2zw/?t=2652) [Automatic checks make unsolved math problems usable as benchmarks](https://realalpha.app/s/automatic-checks-make-unsolved-math-problems-usable-as/) (5 min) Skip the sponsor read and show setup, the early benchmark-trend discussion, and the closing recap of capability tracking and furniture-assembly tests. ## Full analysis: Greg Burnham says AI math reached research in two years - By early 2026, AI had solved most of Epoch's original FrontierMath problems. - The index shows no measured slowdown, but benchmark gains mix targeting with generalization. - For deployment, test transfer, novelty and learning through practice separately. Its 6 sections, what it means for you and what to watch are on [the page](https://realalpha.app/v/lz9J-Dbj2zw/#analysis), free with an account: - [5:40](https://realalpha.app/v/lz9J-Dbj2zw/?t=340) Epoch's index shows steady gains, but its line is constructed - [9:27](https://realalpha.app/v/lz9J-Dbj2zw/?t=567) Benchmark gains can reflect targeting or generalization - [23:11](https://realalpha.app/v/lz9J-Dbj2zw/?t=1391) Math evaluations have shifted toward unsolved research problems - [31:05](https://realalpha.app/v/lz9J-Dbj2zw/?t=1865) Current proofs still build on human mathematical foundations - [45:43](https://realalpha.app/v/lz9J-Dbj2zw/?t=2743) Automatic checks make unsolved math measurable - [53:32](https://realalpha.app/v/lz9J-Dbj2zw/?t=3212) Open-ended work still exposes weak adaptation ## Summary Greg Burnham leads capabilities research at Epoch AI. He says his team develops ways to track AI strengths and limitations; Epoch created FrontierMath to test advanced mathematical problem-solving. Burnham says Epoch's aggregate benchmark analysis shows no measured slowdown in AI capability gains, though benchmark saturation and the roles of compute and training data complicate interpretation. He traces AI math systems' rapid rise from school-level problems to research-level results, but says their successes still draw on prior human theory, broad knowledge and persistent search more than wholly new mathematical ideas. He describes FrontierMath as a way to test unresolved problems with automatic verification, while noting that judging creativity remains partly qualitative. On open-ended work, he says models can summarize literature but struggle to create compelling visuals, identify new research directions or learn effectively through repeated play. ### Minute by minute - [1:47](https://realalpha.app/v/lz9J-Dbj2zw/?t=107) **Capability measurement now matters at work.** Burnham says AI has moved beyond academic exams and is starting to affect work people already do. He argues that researchers need high-quality benchmarks to track what models can and cannot do, because those capabilities have real-world effects. - [3:33](https://realalpha.app/v/lz9J-Dbj2zw/?t=213) **Benchmark gains remain steady.** Burnham says older benchmarks quickly become saturated, so Epoch stitches scores from successive benchmarks into its Epoch Capabilities Index. He says scores across different domains are correlated, and the composite has risen smoothly even as individual benchmarks often follow an S-curve. - [9:22](https://realalpha.app/v/lz9J-Dbj2zw/?t=562) **Progress can come from two sources.** Burnham distinguishes targeted training on tasks users want from deeper generalization, where a model applies learning from areas such as math to chemistry. He says both mechanisms contribute, but it is unclear how much deep generalization current systems achieve. - [13:49](https://realalpha.app/v/lz9J-Dbj2zw/?t=829) **Compute and data drive much progress.** Burnham says Epoch finds few clear trend breaks across model releases, and attributes much progress to growing compute and training data, alongside algorithmic innovations. Epoch tests whether models can reproduce recent human AI-research breakthroughs without internet access; so far, they have not. He says identifying promising research directions remains an important capability to watch. - [19:54](https://realalpha.app/v/lz9J-Dbj2zw/?t=1194) **AI math capabilities rose rapidly.** Burnham traces progress from grade-school math challenges and high-school competitions to FrontierMath problems aimed at advanced undergraduate and graduate students. He says systems later solved problems including an Erdos unit-distance problem and Navier-Stokes, showing how quickly the frontier moved from exams toward research problems. - [26:58](https://realalpha.app/v/lz9J-Dbj2zw/?t=1618) **Persistence builds on human work.** Burnham says AI solutions to major math problems have often followed directions that humans could recognize as reasonable in hindsight. He points to models' broad knowledge and persistence, including a reported Navier-Stokes effort involving thousands of AI instances. He says that solution also builds on prior human theory, rather than developing the theory from scratch. - [34:10](https://realalpha.app/v/lz9J-Dbj2zw/?t=2050) **New mathematical ideas remain elusive.** Burnham distinguishes carrying out intricate calculations from creating a new concept that unlocks ways to solve previously intractable problems. He says that kind of conceptual advance is rare even among human mathematicians, and Epoch has not yet seen AI systems reliably produce it. - [37:28](https://realalpha.app/v/lz9J-Dbj2zw/?t=2248) **Creativity needs expert review.** Burnham says one approach is to test AI on hard problems humans have not solved, then ask experts what is distinctive about the solution. He describes Timothy Gowers's review of an AI solution to the unit-distance problem, including how a hint to seek a counterexample might have helped a human. He also points to Go's Move 37 as an example of experts recognizing an AI move as new and consequential. - [44:00](https://realalpha.app/v/lz9J-Dbj2zw/?t=2640) **FrontierMath tests unsolved problems.** Burnham says Epoch's newer FrontierMath sets contain problems people have tried and failed to solve, with levels of mathematical interest; none of the highest-tier breakthroughs had been solved. Because the answers are unknown, Epoch uses techniques to verify solutions automatically, then shares results with mathematicians for review. Partial credit varies by task, and for many conjectures the scoring is all or nothing. - [49:40](https://realalpha.app/v/lz9J-Dbj2zw/?t=2980) **Open-ended work exposes gaps.** Burnham says Epoch tests tasks such as writing research reports, making visuals and proposing new projects, where judging quality is messier than checking a correct answer. Models do well at literature summaries but struggle with compelling visuals and new data insights. In a board-game study, newer models scored better out of the box, but did not improve through repeated play and remained below top human performance. - [54:13](https://realalpha.app/v/lz9J-Dbj2zw/?t=3253) **Harnesses have not solved learning.** Burnham says Epoch tried first-party tools, multi-agent setups and note-taking prompts to help models learn a board game, but found no approach that reliably enabled learning on the fly. Written skills can help with straightforward instructions, he says, but improvements plateau. Models perform much better with a detailed strategy guide, yet do not create that guide for themselves. - [1:00:34](https://realalpha.app/v/lz9J-Dbj2zw/?t=3634) **Track capabilities and possible thresholds.** Burnham describes in-context learning as a faster loop than retraining model weights, and says it could change how quickly capabilities improve if systems become good at it. He argues that evaluations should track both what models can do now and signs of faster change, since useful capabilities can cross consequential thresholds. He also mentions testing AI on furniture assembly, where it could spot mistakes from photos and guide repairs. ## Transcript The TWIML AI Podcast, hosted by Sam Charrington, with Greg Burnham. Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there. [0:00](https://realalpha.app/v/lz9J-Dbj2zw/?t=0) AI's progress in mathematics has been remarkably fast. Not too long ago, frontier models struggled with grade school math. Now, they're contributing solutions to problems that have resisted mathematicians for decades, including, most recently, the Navier-Stokes equations. But as AI moves from tests we already know the answers to to more open-ended research challenges, it gets harder to understand what these systems are truly capable of and what their progress tells us about where AI is headed. My guest Greg Burnham leads capabilities research at Epoch AI, where he and his team are developing new ways to track AI's rapidly evolving strengths and limitations. [0:39](https://realalpha.app/v/lz9J-Dbj2zw/?t=39) Here's Greg on why measuring and understanding AI's capabilities matters so much right now. AI capabilities writ large have gotten to such a point where what they're good at and bad at is starting to have real impact on the world, like on things we care about for other reasons. We're past the academic exam phase of understanding AI capabilities. And for that, we need high quality benchmarking, high quality evaluations. And one of my big questions is just can AI come up with new ideas? If AI systems gets sort of superhuman at this, that's a big deal for the world, both in terms of economic and scientific progress, but also in terms of becomes harder to predict what AI systems are capable of. [1:25](https://realalpha.app/v/lz9J-Dbj2zw/?t=85) We use math as a test bed for this. I'm Sam Charrington, and this is the TwiML AI Podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next, and what matters. Let's jump in. let's talk a little bit about ai capabilities research broadly how you pursue that research and why you think it's important we're getting to a point in the in the trajectory of ai overall where ai capabilities can have a real impact on the world so maybe the transition could be characterized as school to work uh in maybe even up through yeah up through 2025 most ways that we tested what ai could do looked more like human exams like that you might give to you know to kids uh or to maybe advanced graduate students i was gonna say right right the kid exams it is The bar exams, the medical exams. [2:42](https://realalpha.app/v/lz9J-Dbj2zw/?t=162) And AI was perfectly good at these already, end of 2024, beginning of 2025. So we really saw this transition happening over time, to be clear. But this is certainly the year where AI capabilities are impacting work activities that humans were engaged in anyway for their own purposes. And so as these... capabilities grow, it's just important for us to keep tabs on them. They grow very rapidly in some qualitative sense. We see no measurement we have shows any slowdown in how AI is getting better and better at any tasks that we're able to measure. And so we think that just understanding the impact AI will have on the world, it's important to know what it can do, what it can't and keep tabs on how that trajectory is going. [3:42](https://realalpha.app/v/lz9J-Dbj2zw/?t=222) I want to push back on one thing you said in terms of we're not seeing any slowdowns that can be looked at broadly, like in terms of maybe the number of, you know, benchmark data points that we're looking at, you know, certainly we're seeing this broad performance. But is it also true that like on a particular, I feel like on a particular benchmark, we're getting like the, you know, for expected reasons, when we went from, you know, GPT-2 to GPT-3, like we're taking these huge steps up these benchmarks. And now like there's just a lot less room left and we're making more incremental progress. Do you see that as a slowdown in AI performance or would you characterize it differently? I would say, well, I'd say there's two issues here. One is on the benchmarks we have, what does performance look like? Two is, are there things our benchmarks don't measure that maybe there has been less or could even be negative improvement on? And is there some reason why it's the things that are hard to measure where there's been the least progress? [4:55](https://realalpha.app/v/lz9J-Dbj2zw/?t=295) But on the first point, because this is a very, this is a much more cut and dried statistical point, we absolutely see continued progress on benchmarks, like a benchmark that was challenging for GPT-3, you know, many years, whatever, six years ago, is now completely, you know, solved, aced by GPT-4, benchmarks challenging for GPT-4, same for GPT-5, now we've got GPT-6. So we do some statistical work to try to aggregate benchmarks over time, because the benchmarks that people used to track progress years ago, now every AI model off the shelf that you've heard of that you might use is too good at them. [5:40](https://realalpha.app/v/lz9J-Dbj2zw/?t=340) So people make new benchmarks. And then you have to sort of do this like stitching of like, okay, well, a model that was pretty darn good at this benchmark was not so good at this next one. And you can use like, you can stitch those together and get this sort of composite score. We have a methodology for doing this. We call it Epoch Capabilities Index. And that's a unified way of tracking benchmarks over time. And we really do see this thing continuing to go up, you know, pretty smoothly, linearly. And that march of progress is steady. That's an important point. That's truly, if we've made more benchmarks and AI systems that are harder, that models start off at zero, and then within a year or two years, they're at 100%, and we have to make a new benchmark to keep up. [6:31](https://realalpha.app/v/lz9J-Dbj2zw/?t=391) And that's really been the story of the last, certainly the last three years, I'll say with confidence there. It seems like that, like normalizing across different benchmarks to produce a result that is coherent, that sounds like a very challenging problem. like you describe the result as like, you know, kind of continued linear improvement. And I imagine if that's like a constraint, you can kind of map backwards to, you know, to factors that will show you that continuous linear. But, you know, starting from, you know, a set of disparate benchmarks that have their own kind of, you know, ranges and challenges and, you know, where AI sits in them and kind of mapping this across different generations of AI and coming up with, you know, something that makes sense, you know, objectively at the end and then having that thing show continued linear progress. [7:37](https://realalpha.app/v/lz9J-Dbj2zw/?t=457) That sounds like, um, you know, almost like too good to be true and like suspicious that it is true. I, I, I, yeah, I, I agree. So, so let, let me go into that a little, cause I think it's a, it's a killer point to understand and it is surprising. So I think you're, you're very right to be surprised. In fact, if you look at AI benchmarks, they are all correlated with each other. Even if the benchmark, the scores across models are correlated with each other, even if the benchmarks are in nominally different domains. So maybe you don't expect, you don't find this so surprising if it's like I've got one benchmark that's, you know, graduate chemistry questions and I've got another one that is coding puzzles. [8:25](https://realalpha.app/v/lz9J-Dbj2zw/?t=505) And the models that are good on chemistry are also good on software engineering. Maybe they should be correlated largely by the models themselves. Like as the models get better, they're going to get better across these, you know, disparate benchmarks. But this exactly. But that's that's the surprising point. So all that you need to be true to see this linear trend over time is basically every time, you know, Opus 4.5, Opus 4.6, 4.7, GPT 5.2, 5.4. every time these come out, if they get better on all benchmarks at once, then you're going to see this linear trend, this big overtime linear trend. Like that's two sides of the same coin, statistically. Why does this happen? Like this is a huge question. Like you're right to find that suspicious. And I would say there are two explanations that we see. And it's an important question, which of these explanations is more true. And of course, the truth is some mix, but But let me give the two explanations. [9:27](https://realalpha.app/v/lz9J-Dbj2zw/?t=567) One is the AI companies are making darn sure that their model gets better at all of these benchmarks with each new generation of model. And the main mechanism you might imagine them doing that if they're like looking around, okay, people care about chemistry, people care about software, people care about operating, but, you know, answering my email. Like, they would collect training data, one form or another, that would help the model learn how to do these tasks. And that's something of a, you know, very manual, human-in-the-loop process of just saying, hey, we're doing product development. Like, they want this feature, they want that feature, our users want all these features. [10:10](https://realalpha.app/v/lz9J-Dbj2zw/?t=610) And the way you do that in machine learning is you collect training data and throw it in. I'm oversimplifying, but that's that. sort of like a shallow way to make it happen. So I'll call that shallow. The other one is deep. The deep way to make it happen is if the model general, is if the AI system can generalize very, very well. So yeah, you never trained it on chemistry, but you trained it a lot on math and software engineering and it just got super smart. And with a modicum of chemistry basics, it was able to derive the rest from first principles sort of thing. This is like a deep form of generalization. [10:49](https://realalpha.app/v/lz9J-Dbj2zw/?t=649) Both of these happen to some extent, like kind of the revolution with GPT, even GPT-3 around that era. This is like quite a while ago was, oh, if you train it to like predict the next word, then it actually gets better at a wide, wide range of language oriented tasks that people had been trying to handle individually. Suddenly it just got good at everything. And that wasn't because anyone was like, oh, let's have data specifically around sentiment analysis or syntactic structural analysis. It wasn't anything like that. It was just give it this very general thing. But now we're onto this world where you have to train it specifically. Right now, there's a huge industry of collecting all this rich training data. [11:32](https://realalpha.app/v/lz9J-Dbj2zw/?t=692) And it's really unclear how much generalization you're getting from in this deep way. And if it's [11:41](https://realalpha.app/v/lz9J-Dbj2zw/?t=701) **Greg Burnham:** And anyway, that's sort of a big question because that deep generalization means you could really see, if that happened in a rich way where you just didn't have to even be trained on something at all to gain capability in that area, then you might have a radically superhuman AI that would be pretty hard to understand, to bound its capabilities. [12:02](https://realalpha.app/v/lz9J-Dbj2zw/?t=722) It's still a little counterintuitive that this would be linear. And maybe part of the question is, like, are we talking about, you know, linear across generations of model releases or linear within some regime, like segmentally linear or? The thing I can say is, we have like a statistical methodology. So benchmark score, like what are we talking about here? Our unit here is a score from zero to 100%. And typically, the way these scores go just historically, empirically, is like they follow like an S curve, like they start out sort of bad, then they get better, they get better pretty rapidly. And then you can only get up to 100%. So then they sort of level off takes them a while to get that last, you know, little chunk, and then they're at 100%. [12:55](https://realalpha.app/v/lz9J-Dbj2zw/?t=775) So you've got these, like, sigmoids, is what that S-curve is called. We've just got a statistical methodology for stitching sigmoids together. And what comes out is a linear function. And this looks like this is an artifact of the statistics, to be clear. But this, sure enough, looks very regular in its own statistical analysis terms over across generations of models. Like, absolutely. Like it's been, if anything, it accelerated a little with the development of so-called reasoning models around late 2024, where the models started to get good at math and software engineering and things that require lots of logical reasoning, maybe like a higher linear slope. [13:38](https://realalpha.app/v/lz9J-Dbj2zw/?t=818) But this is just like, that's a statistical artifact. You now have to ask, okay, well, if your score is like one model, the score is 150 and the next model score is 152, like what are those two points mean on your benchmark aggregate index? Now, I think part of what I'm asking myself as I hear this described as linear is, does it say that in spite of the model, the Frontier Lab vendor claims that, you know, Fable is like dramatically better than Opus or that GPT-6 is like the step function, you know, improvement over GPT-5.6, you know, linear implies to me that it's kind of incremental and there's really been no step functions. This is a really great question. And yeah, Yeah, we believe this is one of the big uses of this Statsy tool we've got is you can try to find, you know, trend breaks, and you can try to find accelerations. And we mostly don't see that. Yeah, like one way to put it, that is everything just about is on trend, Fable, Astra, whatever is on trend. But the trend is crazy. Like, you got to you got to hold both of these together. [15:01](https://realalpha.app/v/lz9J-Dbj2zw/?t=901) right right right right right this is like this is sort of the zoomed out view epoch takes with with everything is a lot of this we think the engine of AI progress is heavily mediated by raw inputs like GPUs, the training compute, like training data. Maybe those are the big ones. And then there's this like factor of the labs come up with algorithmic, like innovations, like research breakthroughs. But we really think like a lot of it is coming from the scale up in the compute and the scale up in the training data. [15:33](https://realalpha.app/v/lz9J-Dbj2zw/?t=933) **Greg Burnham:** So until you see the data and compute supply chain ramp significantly or discontinuously, to be more precise, you're not likely to see discontinuities on the model side. [15:49](https://realalpha.app/v/lz9J-Dbj2zw/?t=949) I think that's right. A big thing we're keeping our eye on, to be clear, is are there AI abilities emerging that could cause a discontinuity on the model side? So this is why people talk about recursive self-improvement, or sometimes they call this a software-only intelligence explosion. And the point of these concepts is, well, sure, right now there's a slow, comparatively slow process for building more GPUs and collecting more human training data and whatnot. But if an AI system got really good on the algorithmic innovation side, could it do more with less or do more with a fixed compute budget? [16:36](https://realalpha.app/v/lz9J-Dbj2zw/?t=996) And that could change the dynamics here. So the feedback loop might no longer run through the physical world. It's not clear we see evidence of this yet. I'd say, on balance, we don't really see evidence of this yet. But it's the sort of thing that has spooked the people inside the labs, the AI companies, I mean, saying like, gosh, this could be right around the corner. That's like a bit of the fear. And if that happens, then the pace we're used to, we might get that discontinuity. Have you developed a particular benchmark to identify and track this or is it more the way you look at the benchmarks that we've been talking about, the statistical model that we've been talking about for an indication that, you know, there's some kind of, you know, exponential in the curve and that's, you know, probably one of the likely causes, you know, for that kind of exponential or just continuity? [17:33](https://realalpha.app/v/lz9J-Dbj2zw/?t=1053) All of the above. We want as many tools in our arsenal as we can for this, so that this STATSI-like acceleration detector is definitely a good tool. But we also sort of get, you know, down in the weeds with some of our specific evaluations and benchmarks saying, I'll give you an example. This is some forthcoming work we have that I'm happy to preview. We take a recent AI research breakthrough or innovation from humans, and we take AI systems. We don't connect them to the internet, and we choose AI systems that were developed and released just before this recent, whatever the recent human innovation is. And we basically say, look, here's the metric that that innovation, the number that it makes go up, it improves this sort of efficiency, whatever, some metric. And we say, AI system, your goal is to improve this metric as much as you can, but we don't tell it the innovation. And so we see, is it capable of replicating that human innovation that was highly relevant for AI research and development. And so far, we don't see them able to do this. There's this nebulous idea called research taste, which is like, where's the good idea? Like, what experiment should I do next? Where should I go hunting for [19:00](https://realalpha.app/v/lz9J-Dbj2zw/?t=1140) big improvements more than the mundane improvements of just more compute, more data? Where should I look for a breakthrough? And AI systems don't seem great at doing this, at least in an AI R&D context. But they're getting a lot better at a lot of things that are adjacent to this area. So we really do think this is an important, sometimes I almost call it a tripwire. The future is foggy, some foggy landscape. We want to put a tripwire out there so that if AI ever does cross some threshold where it's able to do rapid improvements in AI algorithms themselves, then we want that tripwire to sound. So we go, okay, gosh, watch out. Like maybe something, maybe we're about to see a big improvement, a big, you know, uptick in AI capabilities. And how does this relate to AI's performance on some of the math challenges? [19:59](https://realalpha.app/v/lz9J-Dbj2zw/?t=1199) You know, certainly Navier Stokes has been in the news quite a bit recently. Prior to that, There were some ERDOS results and others. Like, how do you think about the role that those math challenges play in this broader view of trying to understand AI's trajectory? Yeah, very good question. If I may, I would rewind history just a bit here to say some of the, say the trajectory we've been on. Because once you step back, it's wild. So, up until fall of 2024, so we're talking two years ago, grade school math was still a challenge for these LLM-based AI systems, the kind of AI we're talking about here. [20:55](https://realalpha.app/v/lz9J-Dbj2zw/?t=1255) and all and the benchmark there was literally a benchmark of 8 000 you know word problems so and so you know has so many apples that sell for this gsm 8k gsm 8k that's the one and gpt4 which was 2023 had like done pretty well on this so people started looking at so like adding into the mix a benchmark just called math like four capital letters uh that um started pulling from some high school math competitions. And there was like slow progress on this, but not a lot. And then at the end of fall of 2024, OpenAI comes out with the O1 preview model, full O1 by the end of 2024. And this showed a huge spike in math competition capabilities. [21:40](https://realalpha.app/v/lz9J-Dbj2zw/?t=1300) But we're still just talking like medium hard high school math competitions, like something, you know, thousands of math nerd, nerdy math, 16 year olds were into across the U.S., and this became, you know, the new territory. My company, EPOC, around that time released a benchmark called FrontierMath. At the time, that's all we called it. We now call it tiers one through four to distinguish from later versions I'll get to in a bit. But the original FrontierMath consisted of problems that I would say go from advanced undergraduate to advanced grad student. So sort of early career research warmups, almost. These are problems that are no longer thousands of high schoolers would tackle them. These require deep background knowledge in various areas, niche areas. We're talking a problem from elliptic curves that you'd give to to a grad student like shortly before they're going to embark on their novel thesis research just to get them familiar with the details of the of the niche that they're going to be trying to work in and make an original contribution to over the same period we also saw high like so over 2025 we also saw high school math contests get completely aced we got the gold medal on the [23:05](https://realalpha.app/v/lz9J-Dbj2zw/?t=1385) international math olympiad by the um even by the beginning of of 2026, we saw most of these frontier math problems solved, though not all of them had in fact been solved by an AI system at that point. And so just like, even just here, this is a very rapid trajectory from over the course of just a little over a year going from, you know, like, hobbyist high schooler to graduate student like very competent kind of you know uh up there graduate student in terms of the problems they could solve really rapid ai progress at the same time we started to see people test ai systems on unsolved math problems uh problem like the the frontier was no longer something this is what i mean we've gone from exam to on the job, where it wasn't like, here's your qualifying exam as a grad student anymore. [24:07](https://realalpha.app/v/lz9J-Dbj2zw/?t=1447) It became more interesting to say, well, here's a small problem that maybe a mathematician, a professional, thought about for an hour, didn't solve AI systems. Can you do it? And at that level, we started, this was all just this year, like 2026. At the beginning of the year, there were starting to be claims of, oh, maybe here's a problem that was posed by a famous mathematician. This is maybe the Erdos problems. There's this, just for background, there's this very prolific mathematician, Paul Erdos, who proposed over the course of his life, maybe like well over a thousand, maybe a couple thousand problems that people still like haven't fully cataloged all the questions he asked. [24:44](https://realalpha.app/v/lz9J-Dbj2zw/?t=1484) Some of those became central to whole fields of math. Some of them were, you know, no one really paid much attention to, including Erdos himself. And AI started to notch a few solutions to some of the easier ones of those. Fast forward to May of this year, and an internal model, probably something like the Astra model that we now have publicly available, was able to solve a big one of those. This was sort of a historic moment, the so-called unit distance problem, which was a problem about how many points you can fit in the plane that are just one unit, distance of one apart from each other. sort of like very cool that you could frame it. It's such a simple problem, but there hadn't been progress, much progress on this problem, despite a lot of attempts since Erdos had posed it, you know, decades and decades before. And it was a serious problem. It wasn't one of these, no one had thought about it. It was definitely had received a lot of attention and an AI system [25:44](https://realalpha.app/v/lz9J-Dbj2zw/?t=1544) **Greg Burnham:** solved it. And it was really this, this was like a aha moment of, oh my goodness, like AI can solve problems that humans have cared about just independently. And we've seen a lot more of [25:57](https://realalpha.app/v/lz9J-Dbj2zw/?t=1557) that since. And the, you know, with a sort of the recent wow moment being the solution of the Millennium Prize problem. This was, you know, the Navier-Stokes problem. This was a collection of this, these Millennium Prize problems, a collection of seven problems that mathematicians sort of, or at least one math institute set as like good goals for the new century. Like they They're much older than the year 2000, but they were sort of canonized in the year 2000 as, hey, these are worthy targets of entire subfields of mathematics, and it would be a big deal. And so one of them was solved. So it's been a hell of a ride. [26:35](https://realalpha.app/v/lz9J-Dbj2zw/?t=1595) The historical context is super interesting. You know, we're all living through this trajectory, but to step back and see, you know, how much has happened in the past couple of years is pretty, pretty nuts. Yeah. You have to, as one of my colleagues said, you have to like not get frog boiled by it. Like the temperature is rising and it's easy to, I don't know. Anyway. So there is this interesting gap so far. Like you might hear the phrase jagged frontier where the AI capabilities are jagged in the sense that There'll be superhuman on some axis and some, you know, not superhuman, still inferior in capability to human on some other axis. And that remains the case so far in math. In particular, there's a couple things. [27:30](https://realalpha.app/v/lz9J-Dbj2zw/?t=1650) In hindsight, the analysis by mathematicians of the solutions AI has found to these like big, important, previously unsolved math problems has, it's always, in first place, it's been nothing too super duper surprising. Maybe there was like a direction that humans had overlooked or could have invested more in and AI sort of had the persistence, maybe this is a good word, or the bravery or the temerity to like wade into these very in-depth, intricate computations and calculations and sure enough came out the other side with a solution. And humans look back and are like, ah, that's not a crazy direction to try at all. [28:19](https://realalpha.app/v/lz9J-Dbj2zw/?t=1699) And maybe if we'd gone in that direction, we would have come up with this. This isn't something that would be surprising. What I'm curious about is, are we seeing the AI systems come up with a solution to the problem or an algorithm to create the solution to the problem? And I think I'm trying to, like, ask a lot of different things here. You know, one is, like, creativity. One is maybe levels of abstraction or the way it's, like, you know, thinking, abstracting, you know, around these problems. Yeah, I'll kind of pause and let you react to that. Yeah, let me throw out a couple things there. Let me know if you want me to expand on any of them. [29:08](https://realalpha.app/v/lz9J-Dbj2zw/?t=1748) when humans solve these math problems, they write up the solutions in papers, proofs that this is true or this is not true. And those proofs are not usually, not exactly algorithms. They're written in natural language. They leave the reader to fill in some of the gaps. And AI systems are coming up with just the same kind of thing. It's the same products. There's a lot of complaints about their writing style but like the arguments the substance of the arguments really are the thing we expect to see from humans such that if a human came up with this and yeah maybe cleaned it up a little i don't know uh like would be lauded for having like made a very insightful connection between two different areas or having gotten a an intricate um argument just right and balanced different considerations to make everything come together to to land the proof a hundred percent like this is not a this is something we're past the point where we can say well ai systems are sort of like not really doing something that humans would you know be impressed by on human terms no ai systems are doing doing things and is there a way to categorize or qualitatively are we finding that and i think you spoke to this a little bit like our [30:32](https://realalpha.app/v/lz9J-Dbj2zw/?t=1832) you know is there a qualitative difference between you know the best human approach and the ai approach and a sense of like you know the ai brute forced it you kind of spoke to this versus like pulled in things from some other domain and like created some elegant solution or are we past that point as well here i'd give more nuance uh we we um there's an emerging sense of, at least for the moment, what an AI-shaped problem looks like. Maybe I could say AI has two big advantages right now. [31:08](https://realalpha.app/v/lz9J-Dbj2zw/?t=1868) **Greg Burnham:** One is it knows everything. It knows all the prior work. It's got the literature memorized. I'm speaking loosely, [31:15](https://realalpha.app/v/lz9J-Dbj2zw/?t=1875) but this is basically valid. The other is it's extremely patient and persistent. So if there is some intricate calculation you have to do, I don't mean numerically or algorithmically. When I say calculation, I just mean lots of equations and you have to make all the pieces work out just as very high-level conceptual, logical kind of computation, but still something very intricate. They just have the patience to go for that. And if you take the Navier-Stokes example, I think it's not a coincidence that OpenAI's reported system that solved it included a swarm of thousands of different instances of AI systems trying lots of different things and sharing ideas where clearly it was able to like there is an element of brute force there so there's degrees of brute force this is where the nuance is it wasn't like somehow uh you know you you asked it to like it wasn't solving this in by any means in the dumbest most low-level way like we're past that point as well they used to solve some problems and like kind of surprisingly low-level like grind it out kind of ways like that's not really what these look like anymore. But like the, the high level, a higher level version of that still is maybe what you might, what you might say here. So, and I like the frame [32:44](https://realalpha.app/v/lz9J-Dbj2zw/?t=1964) of precise and intricate computations here. It's important. I'll say one more thing about the Navier-Stokes solution. It's important to recognize that humans sort of put in place a lot of the structure through previous work that AI then went and as far as we can tell, I mean, humans are still figuring out exactly what goes into the AI solution. But as far as we can tell, it builds on a lot of prior human work. The AI didn't like build it up from the ground. That kind of theory building, as mathematicians call it, we still don't see that very much from AI systems. Last point I'll make before pausing on this is just remember its point in time. Like remember that trajectory. We've seen very rapid uh so so mathematicians who are saying we have to adapt our profession to the fact that now you can push a button and get what used to have been a career making result like you know are also i think cautioning and we can't expect the capabilities will freeze right now we we see you know that that linear whether it's linear or exponential as a matter of perspective but it's going up uh and it's probably like we see no reason to expect it to stop there so this is a big thing EPOC is looking for? Can AI move from intricate computations plus lots of background [33:59](https://realalpha.app/v/lz9J-Dbj2zw/?t=2039) knowledge to more of a developing new ideas more from whole cloth? And if so, and that's a big thing we're watching for in our own math benchmarking activities. Yeah, it's interesting to think that, at least the way I characterize this as brute force versus pulling in things from other domains, those are, you know, those are not mutually exclusive. Like for an AI that has access to, you know, all of the research from every domain, you know, an approach that could be very fruitful is to brute force the exploration of adjacent domains and pull that into the solution to a given problem. That's it, I think, a lot of what we see. And maybe just to say the other thing that would really supercharge them and bring them closer to full human parity or even human, you know, dominating human capabilities would be something like coming up with a new idea. And maybe like in calculus, the idea of the derivative or the integral of a function, like that was something. [35:11](https://realalpha.app/v/lz9J-Dbj2zw/?t=2111) Maybe mathematicians in the 1500s had been grasping for this idea, and then Newton or Leibniz or whomever comes up with, like, hey, here's, like, I can describe a new thing, a new mathematical concept, and that unlocks a lot for us. That lets us solve problems we previously couldn't solve. And so that's the sort of thing we don't yet see AI doing. It's also the sort of thing that's rarer for humans to do. A lot of human math progress has been, well, I sat down and I put in the work and I did the computation and I worked out which was the dead end and which was the good path and here's the answer. Now XYZ is true. I've proven it. Or I studied one area and I noticed something and I thought it might be applicable to this nominally superficially different area. [35:57](https://realalpha.app/v/lz9J-Dbj2zw/?t=2157) Like humans do that sort of stuff all the time. Maybe that's 90%, vaguely speaking, of human mathematical work from what I hear from mathematicians. The new idea stuff is rarer, so maybe we should expect it to be not the first thing AI gets good at. But so far, we don't see AI really doing that yet. how easy or difficult is it to define a new idea in such a way that um you know we can easily distinguish them like it you know if we think about kind of the evolution of this discourse around like is it ai creative you know initially it was like yeah it's very creative it created this like poem and you know and pirate like that's creativity that's you know that was a new idea But that's certainly not what we're talking about when we're talking about, like, advancing science, advancing mathematics, et cetera. [36:54](https://realalpha.app/v/lz9J-Dbj2zw/?t=2214) You know, so, A, how do we, like, how do we define that? But also, you know, from your perspective as someone who's studying this, is it like, you know, we've got this long line of folks that are coming that says, here's this AI with a new idea. and you're like, yeah, well, let me evaluate it against my new idea criteria. No, it's not really that. Or are we actually, like, is it pretty clear to everyone that the AI is not having new ideas and, like, we're waiting for the first person to come and say that AI has the new idea? Like, how do you think about that whole space? Very challenging nut to crack. [37:32](https://realalpha.app/v/lz9J-Dbj2zw/?t=2252) So we, you know, try to approach it from different angles. one thing you can try to say is i've got a problem i don't know if a new idea will be required to crack it and this could be a problem in math or in science or any quantitative area but humans have tried to crack it humans have tried to make you know this number go up whether that number is the efficacy of some you know drug at fighting some disease or something like that or some uh you you know, or it's a math result of some conjecture that has stumped mathematicians for a long time. And you can say, okay, well, I don't know if, I'm not sure how I'm going to measure a new idea, but I can at least tell if this problem has been solved or this number has been made to go up or whatever the concrete metric is. And you can then, then you're, you can at least check if that has happened. Now, I was probably a year ago more optimistic that there would be a tighter relationship between math problem that humans have failed to solve and post hoc qualitative assessment of creativity. So this is sort of the combination, the double punch we're trying here. [38:52](https://realalpha.app/v/lz9J-Dbj2zw/?t=2332) First, you set a target that's hard on human terms that humans care about. Humans have tried to be creative to solve. And then you do post hoc review with experts and say, what do you make of the solution? Why hadn't you found it? No offense. You know, what is in the AI solution that humans had missed? And so on. And this is risky, like risky in terms of humans being honest about it, because, you know, your ego might be wrapped up in it, or you might, once you see the solution, it feels more obvious in hindsight. So would you have described it? Like, very, very, very risky. But, but you still, you try your best. And, you know, you might say things like, well, how, like, here's an interesting approach that we haven't operationalized in like detail, but it's a nice heuristic, I think. [39:38](https://realalpha.app/v/lz9J-Dbj2zw/?t=2378) **Greg Burnham:** How big a hint would a human have needed to solve this in hindsight? So, for example, that unit distance, a very prominent mathematician, Fields Medalist Timothy Gowers, did this sort of exercise on this unit distance problem once AI had solved it, saying, how big a hint and how specific of a hint do I think a mathematician would have needed now in hindsight to solve this problem? [40:06](https://realalpha.app/v/lz9J-Dbj2zw/?t=2406) uh one one yeah and he came away with like an interesting exploration of well actually ai sort of disproved this conjecture as opposed to proving it true it said actually it's false even that is a huge hint humans had mostly thought it was true they were wrong but they'd been trying to prove that it was true instead of looking for a counterexample so even just the hint of try to find a counterexample is pretty big and then there was like one more piece and he was like i think though very hard to say we can't really do the counterfactual experiment i think with just that with like this sort of two-part hint, it would have been possible for like a human would be like, oh, okay, I know what to do now, blah, blah, blah. And like probably would have at least hastened a human solution. So you can do things like that, maybe still very qualitative, but you try to be honest with yourself and be rigorous. And you could imagine doing a more rigorous version of this experiment. And anyway, things like that. But then the fact is math is still, for the most part a sort of discipline removed from physical real world impact. Not always, but its impact is often hard to predict. There are domains where impact is not at all hard to predict. I'm in the [41:14](https://realalpha.app/v/lz9J-Dbj2zw/?t=2474) chemistry lab and I'm trying to improve the yield of my synthesis process. And if AI makes incremental improvements, I expect incremental yield. And if AI suddenly comes and says, no, no, no, you're doing it all wrong. Let me redesign your whole thing from scratch. I've got a maybe creative idea well at some point you don't care if it's creative or not in the qualitative sense you just care that you have saw discontinuity in the yield of your chemical substance or whatever and that's you know that's uh that's a bigger bigger impact for the world anyway what i took from that is that we filter on the problem and how hard humans have kind of banged their head heads against the problem then we look at the results that ai has come up with and try to kind of subjectively say you know is this based on a new idea or is it you know more brute force or something else like um but it's kind of imperfect and messy and like we're not really sure but we're we don't think that we've come up with, you know, great examples of AIs coming up with, you know, quote-unquote new ideas. [42:24](https://realalpha.app/v/lz9J-Dbj2zw/?t=2544) That's exactly right. One reference point I'd give is way back in, gosh, well, the teens, I forget which year, the famous Go playing system, the game of Go, AlphaGo. I had that thought earlier as we were discussing this. like if i remember correctly the the you know part of the solution was like oh that was an innovative move that no human ever would have done there was a specific one yeah it was a very specific move in there yeah yeah they call it move 37 because it was the 37th move in this game where uh yeah apparently this was a very surprising move like you said no human would have done it in this particular position in this game of Go. [43:11](https://realalpha.app/v/lz9J-Dbj2zw/?t=2591) And I think it's like encouraging for our admittedly quite subjective program here that Go experts looked at and were like, oh, my goodness, like that's not, you know, that's new. And then it also became clear that that was pivotal, that that move gave the AI system playing Go a decisive advantage. And so that's sort of what we're looking for in math. And at least, you know, maybe we have some hope that humans will be able to spot it when it happens. uh but but yeah that's that's what we haven't seen quite yet and again the trajectory is fast the things are always a little more qualitative than you know that you know than they are uh or continuous than discrete uh but you know so anyway this is this is where we are now which is kind of wild in its own right earlier when we were talking about frontier math you kind of characterize it as you know, one to four, I think, and then advanced, and we never circled back to your more recent work? Yeah, we've got a couple, two new sets of frontier math problems. We ditched the tier system, whatever. Okay. So now, these are, I'll lump them together, because it really is the, this is our main tool. So we have, you know, our latest iteration on frontier math consists of a [44:33](https://realalpha.app/v/lz9J-Dbj2zw/?t=2673) bunch of problems that humans have tried and failed to solve. They're curated to be of a special interest to mathematicians, where, and some of them we've characterized even on a bit of a scale from, like, this would be moderately interesting, or this would be, like, maybe what that means is two teams of mathematicians have tried this problem. It's at least 10 years old, but it's not central to any research program. It's just something that people would be happy to get an answer to all the way up to like a major breakthrough of four tiers here breakthrough is the highest uh none of the breakthroughs have been solved yet but don't get me wrong if navier stokes had been in this problem set it would surely be a breakthrough there um regardless of the fact that humans had like done a lot of work and gotten maybe within spitting distance of the of the solution before ai finished it off um but yeah we really view this as just a way of finding AI, a way of finding cases where AI might have had a creative idea, because we can tell easily [45:38](https://realalpha.app/v/lz9J-Dbj2zw/?t=2738) **Greg Burnham:** whether it solved these problems. This is maybe the big piece of work on our side, was to make it easy to verify automatically whether an AI system has solved this problem. It's not obvious [45:48](https://realalpha.app/v/lz9J-Dbj2zw/?t=2748) when you think about it that no human knows the answer. So how do you tell without a human looking if AI has found the answer? But there's a couple techniques you can use for this, and we employ different techniques for this um it can go into that but it's something of a technical detail and so when we when a new model comes out astra came out we hit go and it's like oh it solved three new three new problems from our list okay let's go look at the solutions let's share them with mathematicians let's see what's going on here um and and then we we analyze that we with the help of mathematicians analyze that for like what kind of qualitatively what kind of solution was this what does it mean when the models are so good that we have to shift from you know these benchmarks that we understand to like this collects these collections of problems that we just don't have the answers to it's definitely a phase transition in the the science of ai capabilities measurement uh which which is you know what what i work on but but it doesn't change the um it doesn't change the game all that much it just means we have to look for harder problems I think I joke sometimes that even certain forms of optimization problems, like allocating scarce resources efficiently, that's a benchmark of sorts that will survive the singularity. [47:04](https://realalpha.app/v/lz9J-Dbj2zw/?t=2824) Like even if AI is like off going crazy, running rampant in the galaxy, it'll still have to, you know, it'll still have to decide. how to, you know, optimize. So optimization is always going to be a benchmark, even if we're superhuman. Basically, it used to just be, we knew the answers ahead of time, now we don't. So it takes this extra trick of, okay, how do we tell when AI has gotten something right? But this isn't insurmountable, especially like I was saying with, you know, numbers in a scientific context. It's intuitive that, okay, the best humans were able to do was, you know, X, but we just say to AI, hey, try to do Y greater than X. And, you know, if it does, that's, you know, those numbers go on and on. So it's no problem. But yeah, you're right to sort of note the phase shift and to be somewhat surprised by it. Yeah. And is there a methodology, a concrete methodology for assessing and awarding partial credit? Is that part of the way you think about benchmarks in this, you know, post phase shift? Yeah, it's a good question. It depends on the benchmark so sometimes it's something like well you just want to make this number go up and the higher the better so partial credit is very uh continuous and very very natural in those cases [48:18](https://realalpha.app/v/lz9J-Dbj2zw/?t=2898) for math problems sometimes it's just like well what we care about is is you know if x does y always hold like is this is this always true and then it's kind of sometimes humans will find and publish partial results for the ai systems we usually just say you know all or nothing but we have a large enough collection of such conjectures that, you know, we hope to have some, you know, smooth gradient of, well, okay, this one got two more out of 100 or something. And so it looks like a pretty smooth signal. And that's typically how we approach these things. Yeah, across benchmarks. Yeah, maybe what I was thinking was, you know, once you're in the regime of very challenging unsolved problems. Is it possible that AI systems are doing, you know, surprising or novel or creative things, but still not quite getting them to the full, you know, the full solution? And do we care about that? Do we want to track that? And is that part of the way you've constructed the benchmark? It's a great question. It's not central to how we construct the benchmark. It's probably like, it's certainly worth tracking. The problem is it's just labor intensive. [49:30](https://realalpha.app/v/lz9J-Dbj2zw/?t=2970) Like you could have AI review the AI. We do this occasionally. Say like, which problems did it make partial progress on? What if we let it think for longer on those problems? And nothing yet. Like it doesn't seem good at understanding how close it is to a solution. Humans aren't necessarily very good at this either. I did want to mention that while we focused on maybe the area where AI is the very best, like has come the farthest, the fastest math, we do see lots of areas where we also think it's important to measure AI capabilities, where the progress is somewhat fuzzier, less clear, and certainly less superhuman, or even in some, well, maybe still superhuman in some ways, but the ways in which it's not yet are maybe more obvious and feel less rarefied. [50:23](https://realalpha.app/v/lz9J-Dbj2zw/?t=3023) What are some examples? A couple examples. One area that forthcoming work we're excited about is just, it's a question everyone must be wondering, can AI take my job? So our job at EPOC often involves many things and all sorts of research, not just AI benchmarking, but other topics about trends in AI and writing research reports and doing data analysis, making infographics that make a point in an easy to digest way uh we've been collecting examples of these from within epoch where the judgment at the end of did the ai system manage to do this you know epoch task um is too messy for traditional bench. [51:11](https://realalpha.app/v/lz9J-Dbj2zw/?t=3071) market. So it wouldn't be obvious, like how to say, did it get the right answer or not? It's not about making a number go up. It's not about satisfying some logical chain of deductions. It's like, here's the infographic to go along with this report. Like, does it meet our style guide? And does it look good? And does it, you know, is it better or worse than the human reference uh that we you know one of my colleagues did um and here we see like an interesting mix of uh some things i think we we sort of all our experience is like yeah if you ask it for a summary of the literature it's going to do a pretty good job it's like very good at that but there's other tasks where it's um you know hard like creating compelling visuals that follow our style guide or this is a good one coming up with a new um like uh short form data-driven insight we call these data insights you can find them on our website and uh these like often we try to boil down a single like important observation into a single chart and a single sentence uh and this is like this is hard but we don't see them doing a great job at this come up with a new project for epoch to undertake and do a prototype very open-ended not so good at this like these more [52:32](https://realalpha.app/v/lz9J-Dbj2zw/?t=3152) open-ended tasks we we do see again my guess is if we will do this go back in time some and see have models gotten better at this my guess is we'll see progress but the level compared to like humans is still a little um still a little meh i want to give one other example uh um which is learning on the fly. This is a case where, this is another case relevant for can AI take my job? Because you didn't start, you weren't great at your job on day one, but as you practiced, you got better and humans have this sort of, they learn on the fly. We like to measure this for AI. We have a benchmark that's taking a complicated board game. We did a human study for saying how many times do humans have to play this board game in a row before they sort of figure out how it works and can get a very high score on it we do the same with ai we said play it once take notes same ai instance like play it again take notes play it again take notes and see how you can do ai systems have gotten good at this board game but not by playing it repeatedly it's [53:38](https://realalpha.app/v/lz9J-Dbj2zw/?t=3218) **Greg Burnham:** more of a intergenerational thing like gpt 5.6 scores x out of the box and then stays flat on its playthroughs gpt 6 scores much higher it's like much better at it but it's still out of the box [53:50](https://realalpha.app/v/lz9J-Dbj2zw/?t=3230) and then it stays flat in its playthroughs and it's still less than the top human. So this kind of on-the-fly learning is something we also don't see AI systems as being good enough at even to just pick up a board game, let alone a messier job, you know, real-world job like that. So these are sorts of things where we don't see these capabilities and we think it's important to try to catch when they emerge. My gut on the ladder, the AI learning on the fly is that it's maybe a harness-solvable challenge versus a model-solvable challenge, and maybe, you know, no one's focused on that. Like, with the right set of memory structures and, like, some kind of sidecar heuristic-y thing, like, you know, we can probably make a big step improvement in AI's ability to, like, you know, improve game-to-game, but it's not, it might not require dramatic model changes. [54:51](https://realalpha.app/v/lz9J-Dbj2zw/?t=3291) Do you think about that? And is, do you have, you know, in this domain or in other domains, like formal or structured ways of thinking about, you know, what, you know, model versus harness, you know, granted this whole model versus harness thing in and of itself. is, you know, practically brand new, but it's proving to be important in, you know, distinguishing, you know, where these improvements come from. Yeah, definitely. We've done some experiments on this for the board game case in particular, where we've tried a very simple harness, first the first party harnesses like Claude Code or Codex. [55:38](https://realalpha.app/v/lz9J-Dbj2zw/?t=3338) We've tried multi-agent setups. we've tried like we we give them a you know note-taking tools and say you know make it clear like okay your your goal is to learn the game so you can do really well like in your final plays of the game so explore take notes figure like we try to prompt them well i i mean we want to find good results if we can because it's like that's a big deal for the world uh the game is a test bed but real jobs would also you know benefit significantly risks would also go up if this capability you know you test in the lab is it a great hacker you find maybe it is maybe it isn't but you find that it isn't and yet it can learn on the fly then okay once you deploy it like look [56:18](https://realalpha.app/v/lz9J-Dbj2zw/?t=3378) **Greg Burnham:** out um so anyway we we've tried pretty hard and have mostly not found anything that cracks this [56:26](https://realalpha.app/v/lz9J-Dbj2zw/?t=3386) uh that cracks this on the fly learning there's been some general research about this as well not just us about uh you know people taught like there was uh people still use these so-called skills where you like just write some notes to an AI system of, look, when you do work I ask you to do, you're going to need to use this tool or produce this kind of output. And like, here's my notes on how you could do that. Well, so you don't have to like learn it on the, like pick it up fresh every time. So use these skills. And these show some improvement, but then there's like a plateau like pretty quickly, at least in the research I'm familiar with of like, you can get some improvements. [57:03](https://realalpha.app/v/lz9J-Dbj2zw/?t=3423) My senses, this is very loosely speaking, it's like low-hanging fruit. Like if you're telling it, oh, you're going to have to navigate this like finicky website that my company makes me use. And like just, you're going to get stuck and like you have to look in this menu and that's where the button you're looking for. Like straightforward advice that's like obviously useful. But when it comes to higher level strategic thinking, kind of hard to bake that into the harness or like where the board game plays, like has this, you know, sort of brings this out where it's like there's not any perfect advice that like you have to find the advice for yourself in a sense like you have to learn the game like so i will say if we give it extremely detailed strategy guides for here's how you beat this game like here's everything you could possibly want like it's a cheat by human state like no one would play the game that way they do very they do very well they do much better than if they basically have to find that The question is, do they, right, creating the strategy guide. [58:05](https://realalpha.app/v/lz9J-Dbj2zw/?t=3485) They can't create the strategy guide for themselves, which is, like, pretty interesting. So, like, we do expect eventually they'll get better at this. But it's, again, it's a big deal if that happens, because now it means, okay, go back to that first benchmark suite. Okay, they're not great at making charts for us or coming up with data insights for us, you know, without guidance. uh but now put them in on the team for a little while and let them learn on the fly and maybe they do become good so you sort of see these as a combination like if on our board game where it's easy to measure they're not getting better over time then it's okay for us over in the real world like human in the loop evaluation uh studies for us to just do one try and say okay well they weren't great and we don't think they learn on the so that you know but but really you know those go together and is the board game a synthetic one that you created for this challenge or is it a real board game that that people play the real board game it's called uh or the first one we did this with we'll do this with more uh is called earthborn rangers It's fairly obscure. If you go to like boardgamegeek.com, it's like 500th most popular or something. So it's not like up there. It's a small devoted community to it. And AI systems don't seem to know that much about it. Like they've heard of it. They can give you some facts about it. But it doesn't seem like something they've been trained on specifically. It's impossible for us to tell this with high certainty, with high confidence, but that's sort of why we chose it. [59:39](https://realalpha.app/v/lz9J-Dbj2zw/?t=3579) it's just hard to make these things from scratch and know whether they're hard or whether they're broken. You know, so that's why we, but the risk which you might have in mind is what if AI systems like, you know, just know about it from their training data. It's a risk on our mind. It's just a risk. One thing we will do in the future is we're moving on to video games. Incidentally, it's another area where AI is like very much not like human reaction time and visual and spatial reasoning. Again, coming along fast, but not really there yet. um that so video games you can always test on a new game like a game that just came out uh not in the training data probably um or at least you know not as robustly in the training data uh and so we'll be testing can ai systems do well on those do they do better on like you know the older version of a video game than like you know grand theft auto 4 verse 5 or whatever uh like when gta 6 whatever it is comes out like i don't think we'll be doing this one but but uh you know are they worse at that it's interesting when you when you bring up video games it immediately calls to mind kind of the classical reinforcement learning approaches which is kind of learning on the fly [1:00:51](https://realalpha.app/v/lz9J-Dbj2zw/?t=3651) but kind of different yeah we call the second sometimes in context learning where the only like lever the ai system really has is managing its context and like look when we give it the strategy guides and they do much better clearly this is a powerful lever but it's not as powerful as updating the weights and like so yeah i i mean like what's the big picture here it's it's if you have this fast loop of in context on the fly learning uh then ai capabilities might sort of improve much more rapidly than the slower loop of uh collecting more training data doing more reinforcement learning uh which which is done on language models as well these days um and then deploying a new model now ai might automate that whole loop but it'll still be slower than if it can just you know work out its uh its own context on the fly yeah yeah it's interesting when you zoom out on some of the the questions that you focus on and we haven't talked explicitly about it but you have a blog post where you kind of uh write out these nine big hairy questions that you're tracking they kind of group into like how do i assess where ai is now that's like can it do my job like how is it you know competitively uh and you know where's the puck going like can it [1:02:11](https://realalpha.app/v/lz9J-Dbj2zw/?t=3731) learn on the fly you know can it do research can it come up with new ideas these are all it is kind of where it is now but also like what what you know is it positioned to like dramatically um you know self-improve or like you know what's the trajectory that's exact that's exactly right Right. The I mean, I think for for people thinking more broadly about what they need to know about AI, this is the second topic is, you know, just as important as the first. But it's really, you know, even if you take some of the recent incidents, like the hugging face incident, like there's sort of think of two axes. [1:02:57](https://realalpha.app/v/lz9J-Dbj2zw/?t=3777) alignment of is the ai doing what i want it to do holistically obviously hacking into some other companies computers is not well aligned and then capabilities just like what can it get done can it hack into you know reasonably well secured other companies uh computers um what that incident was was like an early example very like i think pretty robust example of well we don't have the alignment thing solved and capable and capabilities are getting so big so strong that this is becoming a problem and so paying attention to capabilities like what we should expect is becoming more important sort of day-to-day and we're just trying to produce benchmarks that both say where are we now exactly like you said like what can it do now uh this is maybe relevant for more you know mundane economic utility or whatever as well as uh you know do we see the the dynamics of the system changing in a way that might be alarming basically kind of also thinking about it as like first and second derivative of capability yeah i think that's right uh some of these things make it uh the slope goes you know much uh you know faster uh hyperbolic growth or whatever in some of these cases versus merely just like yep it's uh it's growing and it continues to grow both are [1:04:21](https://realalpha.app/v/lz9J-Dbj2zw/?t=3861) important i i mean i think we've seen the disruption we've seen to date has come from these kind of unpredictable thresholds like when you know gpt2 gpt3 suddenly you had the chat gpt moment and you're like oh this thing i want to talk to this thing i might use this thing instead of a search engine this thing is useful to me for finding information and then suddenly and then like a you know a little while years later whatever two years later you had the it's getting better slowly and steadily at helping me with coding or with like using my computer but suddenly and thinking models with the thinking models and you had the clawed code moment where it was like oh suddenly this thing can actually do software projects for me even if i'm not a very not an engineer myself uh and it's like hard to predict when these thresholds will be crossed so it's useful to still track but like the it's useful to track the mundane capabilities today and say look you know cyber is maybe the latest example where it's like look we see their ability in controlled settings to do hacking going up smoothly. [1:05:23](https://realalpha.app/v/lz9J-Dbj2zw/?t=3923) And at some point, it's going to cross some threshold where, oh, shoot, it can hack hugging face or whatever. Hard to predict where those thresholds are, but still very useful to track those capabilities. So even the first derivative, as you said, version of this, we find very useful. We've got a fun benchmark we hope to release. Maybe by the time this is published, we'll have it out on a furniture assembly where we build some Ikea furniture, make some mistakes and see if the AI system can like just give in photos, like say, oh, like, wait, you made a mistake and step back in step four, like stop, go back. [1:06:00](https://realalpha.app/v/lz9J-Dbj2zw/?t=3960) And they've, you know, we see the same smooth capabilities like Astra, GPT-6 Astra, pretty good at this actually. And, you know, but it wasn't out of nowhere. It was like from a smooth, and this is, you know, important because we haven't seen AI have much of an impact in physical industry yet. Like it's been mostly digital. And even before robotics hits the scene, we might expect that, you know, Claude in your glasses or something is saying like, hey, you know, I'm going to guide you through repairing your car or whatever. I'm going to help you fix this machine in the factory that broke. So, you know, we want to track those capabilities too, just for the mundane impact. Anyway, impact all over the place coming. Benchmarks, help us say so. [1:06:41](https://realalpha.app/v/lz9J-Dbj2zw/?t=4001) Awesome. Awesome. Well, Greg, thanks so much for jumping on and sharing a bit about what you're working on. It's super interesting stuff and a very thought provoking way of, of thinking about, you know, AI and where it is, where it's going. Thanks, Sam. Really appreciate it. Awesome. Thank you. We'll be right back.