The alpha, twice a week
That was 2 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
MTS · 2 min · Nov 5, 2026
Test-time learning may be a bigger bottleneck than compute
Rohan Anil argues that models’ uneven performance comes from their inability to learn quickly from mistakes.
The episode’s best minutes
Full episode- 15:37Test-time learning may be a bigger bottleneck than compute2 min
- 1:47Compute islands could make scattered GPUs useful for frontier training2 min
- 9:20Butterfly mixing lets training tolerate asynchronous GPU updates2 min
- One more moment in the full episode
“the reason it's jagged is that we don't have mechanism in the architecture and optimization for models to be able to learn from its own mistakes much quicker.”
Takeaways
- Models can excel at hard tasks and still fail at simple ones.
- Labs currently rely on periodic pre-training and RL cycles to fix gaps.
- Anil wants models to learn at the fundamental level as companies evolve.
About this segment
Rohan Anil argues that models’ uneven performance comes from their inability to learn quickly from mistakes. He sees continual learning as a fundamental algorithmic bottleneck, beyond compute and data-center limits.
Rohan Anil is cofounder of Core Automation and former Anthropic pre-training researcher.
Theo Jaffee, Host · Co-founder & host, MTS
One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.
Sophia Dew, Host · Host, MTS
From “AI Models Will Learn at Test Time by Next Year | Rohan Anil”, Nov 5, 2026. Watch the full episode
Thanks. We'll look at and fix what's off.
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 15:42
Yeah, totally. So you also talk in this piece a lot about the bottlenecks to deployment of compute because we don't have like grid equipment. We don't have the power. We don't have the land. Maybe there will also be new political bottlenecks. um how much to these all in all affect your timelines to AGI, like real AGI? I mean, the reason I'm at CoraO and I'm working on the problems is that I think there is fundamental, like these models are special.
- 16:16
I love Astra, I love my opus, I love my fable. But I mean, I think everybody who's watching and both of you probably love them too in ways, but you probably hate them too in other ways. the number of times I've had incidents of rage when the models gaslight me, not able to do something simple, like when it can solve naivier stokes on one day and do something dumb on the other day. It's very jagged. I think it's something Karpathy coined. And the reason it's jagged is that we don't have mechanism in the architecture and optimization for models to be able to learn from its own mistakes much quicker. And today we have to wait for labs to do the pre-training an RL cycle every K months with the right environments and pre-training data to solve those tasks. So I feel like that is a limiting factor, at least personally. I feel like
- 17:08
beyond compute, there is an algorithmic limiting factor for us to make this useful in a way that I think could get us to like, we exist and we are able to use this technology to do something bigger. If companies could operate much more effectively, if there was one piece of model that can learn as a company evolves today the best we can do is compacted tokens in different forms of different types of memory and we would like to solve this at the fundamental layer and then build products on top so i do think i think i hedged the question to the answer by saying there is i think even without the data center power and um grid acumen there is some other algorithmic bottleneck that exist as well. Yeah. When we do get to that continual learning, or at least the first primitive type of continual learning, what do you think is the alignment risk