Free with a sign-in.
The best 13 min of 24
Frontier AI Models Fail at Automated Research | Qasim Wani
Qasim Wani argues recursive self-improvement depends on high-quality data and research environments, not just stronger models.
MTS · Theo Jaffee & Sophia Dew
- Qasim Wani is co-founder of Extrapolate and former xAI researcher.
The best minutes
- 1:36Recursive self-improvement means inventing a new lab6 min
- 10:20High-quality data is the bottleneck for recursive improvement4 min
- 18:53Models still struggle to diagnose a failed training run3 min
Also worth watching
Skip the opening setup, the Lovable read, the discussion of experiment selection and RSI risks, and the closing sponsor block.
Summary · 2 min read
Qasim Wani is co-founder of Extrapolate, which builds reinforcement-learning environments where AI agents learn to improve through frontier research. He previously researched at xAI, where he started RSI efforts, and contributed to Semi-Analysis’s GDPX evaluation.
Wani defines recursive self-improvement as an agent’s ability to invent a new lab, not simply automate parts of model training. He says frontier models still struggle to choose valuable experiments, with GDPX showing average accuracy or mean reward of no more than 24%. He argues that progress depends on high-quality data and research environments, pointing to weak synthetic tasks and leaked code patches as evidence of reward-hacking risks. Wani supports automating some lab work but says data-quality auditing remains far from solved.
Minute by minute
- 0:58RSI means more than automating training
Asked whether a reported acceleration in Anthropic’s capabilities scores and R&D uplift signals RSI, Wani says labs define RSI differently. He describes Anthropic’s reported focus as automating pre-training research, but argues that self-improvement involves more than model training.
- 3:02Research is iterative across many benchmarks
Wani says lab research is iterative and researchers spend much of their time examining data. He argues that benchmarks focused on a single task, such as optimizing one model for one benchmark, miss how labs hill-climb across 20 benchmarks at once.
- 5:34Wani defines RSI as inventing a lab
Wani defines RSI as an agent or swarm of agents being able to invent a new lab. He distinguishes that from self-improvement that merely shortens model-development cycles, and says GDPX measures capabilities he sees as important to achieving superintelligence.
- 7:38Models struggle to prioritize experiments
Wani says frontier models scored no more than 24% in average accuracy or mean reward on GDPX. He says models such as Astra, Carbon and Opus do not reliably identify which experiments are most valuable, estimate how long they should take, or use existing logs and research to guide the next bet.
- 9:49Wani sees data as the RSI bottleneck
Asked whether models need full knowledge of a lab to improve it, Wani says his team’s work outside a frontier lab is a deliberate bet. Drawing on his experience at xAI, he argues that RSI is fundamentally a data problem, and says data vendors often lack the expertise to produce frontier-grade samples.
- 12:55Data and environments remain weak
Wani says data is undervalued: he puts the total data industry at no more than $150 billion, compared with model and chip companies worth more than a trillion dollars. He cites a benchmark where the best model passed 49 to 50% of environment-creation tasks, and says agents building synthetic coding tasks accidentally leaked deleted code patches; Xiaomi’s paper released about 21,000 synthetic environments.
- 16:35Wani sees safety work in better data
Wani says he approaches RSI risks as an engineer trying to solve them, alongside the need for better policy and governance. He warns that poor data can create reward-hacking or other harmful behaviors, and says Extrapolate is building safe, aligned datasets for RSI capabilities.
- 18:53Data auditing is not automated
Wani says some lab functions, including starting and monitoring training runs, have been automated, but auditing data quality remains unsolved. In a GDPX task, models received 20 million tokens from about 1,000 rollouts and five hours to diagnose a reward regression; their answers were usually different and incorrect, with the top score around 24%.
Published Nov 6, 2026.
Transcript · 17 min read
MTS, hosted by Theo Jaffee and Sophia Dew, with Qasim Wani.
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
For GDPX, we observed that the average accuracy or the mean reward for all the frontier models was not more than 24 percent, which means that there's still a lot of room to grow for these models to improve on. Part of the reason why these models are not good at it, and we sort of talk about it in our blog post within the GDPX report. one of the big problems with models like Astra or Carbon and Opus is that they don't know what experiments to prioritize. We're live with Qasim Wani, who's the founder of Extrapolate, which builds RL environments, or co-founder of Extrapolate, which builds RL environments where AI agents learn to improve themselves by doing frontier research. And it's also contributed to GDPX, which is Semi-Analysis's new eval, which seems really cool and interesting.
So Kassim, welcome to MTS. Thanks for having me. So where should we start? We were just talking before we went on about the new meter report that says that basically Since October of last year. Yeah. Since October of last year, the rate at which the Anthropic Epoch Capabilities Index benchmark score has been increasing is about double what it was previously and they have other evidence that's non-public uh in addition and then all this combined uh leads them to believe that anthropics models are providing like substantial r d uplift already so do you have any takes on
“can an agent invent the next version of itself”
Read the whole transcript
Well, why can't they do this already? They're not good at it. That's sort of how we... Yeah, but like why? Like where did they come up short? There's so many places. So for GDPX, we observed that the average accuracy or the mean reward... for all the frontier models was not more than 24 percent which means there's still a lot of room to grow for these models to improve on part of the reason why these models are not good at it and we sort of talk about it in our blog post within the gdpx report one of the big problems with models like Astra or Carbon and Opus is that they don't know what experiments to prioritize.
And as a human, as engineers and researchers ourselves, we know that certain experiments are more valuable than other experiments. Now, this is not to say that models don't have any sort of intuition on this. They certainly do. It's just that they're not opinionated enough. opus and sort of the cloud models tend to be more opinionated than others but they still lack that level of they still lack that level of like this is a high taste experiment to run and by doing this experiment here's how long roughly i think it should take to finish and like using that existing logs of information and you know uh background research to make the next bet um and that's basically what we were able to find as like, you know, capability gaps in the frontier models.
I want to shout out our sponsor, Lovable. You know, the app you've been meaning to build, the internal tool, side project, or product you'd otherwise lose a weekend to? With Lovable, you get software you can ship today. That includes an editable code base you can inspect and change, plus two-way GitHub sync. Lovable also handles the backend and infrastructure, including managed Postgres, auth, storage, hosting, payments, and more, so you don't have to wire it all up yourself. And through Lovable's MCP server, you can create and deploy your project directly from the agents you already use. Turn ideas into software people love with Lovable.
Lovable.dev. Now, on to the episode. Wouldn't as a prerequisite for any of that to happen, like a model would have to have perfect knowledge of all of the different, like of its own weights being hosted within the company and then also of all of the teams like the pre-training team and the post-training team and like do models currently have that level of oversight or sort of like omniscience over their current state within the company even before we get to RSI? Is that something that they can do? Yeah, so this is actually like a really interesting question
“rsi is fundamentally a data problem it is not a model problem it is a data problem”
“the agents act accidentally leaked the code that was deleted in the commit”
any risks associated with rsi um because like a lot of what we've seen today on the timeline has just been like um we need to pause before but it seems like you're like thinking about it's been And specifically, like, a lot of freaking out, a lot of major updating based on this meter thing.
Yeah, yeah. But you seem to be approaching it from a more, like, I don't know, problem-solving way. I think my, like, impression of this is a little bit different than my friends at Enthropic, who are more, like, doomsdays here. Like, P-Doom is super high. I'm, you know, an engineer and an entrepreneur. And I really think about how can I be helpful in solving these problems? Like we obviously need like better AI policy and like better governance around this. But one way of being useful here is how can you engineer your way out of destruction? And I think this is a very solvable problem if you fix the data.
Hence why if the data is, let's say, if the data has word hacking behavior, which a lot of the vendors that we used to work with back when i was at xai we would have models that would do all kinds of catastrophic stuff um and that is really really bad because as a researcher you have so many priorities you're not going to spend a lot of time like looking at every single sample and you kind of just yolo things every now and then regardless of you know what kind of lab you are um but it's really important as a data vendor or a data lab to ensure that the quality if your samples are super high because you run the risk of models having all kinds of weird behaviors and those weird behaviors may be like in one percent of the samples and that one percent of sample like economically could be like worth hundreds of billions of dollars um which is why what we're doing is we're sort of taking guys who've worked at frontier labs who like understand the model recipes really well and making safe uh aligned uh but data sets focus around rsi capabilities to ensure models are safe and not destructive yeah and i guess before like we even
“almost every single time all the models gave the complete different answer and an incorrect answer”
a huge thanks to mts sponsors 11 labs ai that communicates at human level across every channel and modality 11 labs.io slash mts scale your startup on neon millions of developers and startups have already chosen neon for their back end start on the free plan or get up to $100,000 in credits for your startup at neon.com slash MTS. Special thanks to our sponsor Kong, the AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. KongHQ.com. Finally prove what your AI investment is worth with Lairdyn. Head to Lairdyn.com today to start measuring your AI's bottom line. MongoDB, trusted to run critical
applications by 75% of the Fortune 100, the intelligent data platform for the era. Apply today for access to up to $15,000 in Atlas credits, dedicated technical support, and exclusive events and go-to-market opportunities. MongoDB.com. Chargebee, billing and monetization infrastructure for the AI era. Chargebee.com. Blitzy, autonomous software development for enterprise code bases. Ship 5x faster. Blitzy.com. adquick make your brand a billboard out of home advertising as easy to scale as digital adquick.com arena measuring ai performance in the real world arena.ai support for the show comes from vcx the public ticker for private tech the u.s stock market started history's greatest wave of wealth creation from factory workers in detroit to farmers in omaha anyone could own a piece of the great american companies but today our most innovative companies are staying private longer, which means everyday Americans are missing out. Until now. Introducing VCX, a public ticker for private tech. Visit getvcx.com for more info. That's getvcx.com.
Carefully consider the investment material before investing, including objectives, risk, charges, and expenses. This and other information can be found in the innovations fund prospectus at getvcx.com. This is a paid sponsorship. You
