# Test-time learning may be a bigger bottleneck than compute

Markdown copy of https://realalpha.app/s/test-time-learning-may-be-a-bigger-bottleneck-than-compute/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Rohan Anil](https://realalpha.app/people/rohan-anil/) · Rohan Anil is cofounder of Core Automation and former Anthropic pre-training researcher.
- Host: [Theo Jaffee](https://realalpha.app/people/theo-jaffee/) · Co-founder & host, MTS · One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.
- Host: [Sophia Dew](https://realalpha.app/people/sophia-dew/) · Host, MTS
- Show: MTS
- Episode: [AI Models Will Learn at Test Time by Next Year | Rohan Anil](https://realalpha.app/v/rVMy8MSBKZo/), Nov 5, 2026
- Moment: 15:37 to 18:06 of the episode (2 min), playing on https://realalpha.app/s/test-time-learning-may-be-a-bigger-bottleneck-than-compute/; the episode from there: https://realalpha.app/v/rVMy8MSBKZo/?t=937

## Quote

> “the reason it's jagged is that we don't have mechanism in the architecture and optimization for models to be able to **learn from its own mistakes** much quicker.”
> — [Rohan Anil](https://realalpha.app/people/rohan-anil/) at [16:39](https://realalpha.app/v/rVMy8MSBKZo/?t=999) · MTS

## Takeaways

1. Models can excel at hard tasks and still fail at simple ones.
2. Labs currently rely on periodic pre-training and RL cycles to fix gaps.
3. Anil wants models to learn at the fundamental level as companies evolve.

## About this segment

Rohan Anil argues that models’ uneven performance comes from their inability to learn quickly from mistakes. He sees continual learning as a fundamental algorithmic bottleneck, beyond compute and data-center limits.

From “AI Models Will Learn at Test Time by Next Year | Rohan Anil”, Nov 5, 2026. [Watch the full episode](https://realalpha.app/v/rVMy8MSBKZo/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[15:42](https://realalpha.app/v/rVMy8MSBKZo/?t=942) Yeah, totally. So you also talk in this piece a lot about the bottlenecks to deployment of compute because we don't have like grid equipment. We don't have the power. We don't have the land. Maybe there will also be new political bottlenecks. um how much to these all in all affect your timelines to AGI, like real AGI? I mean, the reason I'm at CoraO and I'm working on the problems is that I think there is fundamental, like these models are special.

[16:16](https://realalpha.app/v/rVMy8MSBKZo/?t=976) I love Astra, I love my opus, I love my fable. But I mean, I think everybody who's watching and both of you probably love them too in ways, but you probably hate them too in other ways. the number of times I've had incidents of rage when the models gaslight me, not able to do something simple, like when it can solve naivier stokes on one day and do something dumb on the other day. It's very jagged. I think it's something Karpathy coined. And the reason it's jagged is that we don't have mechanism in the architecture and optimization for models to be able to learn from its own mistakes much quicker. And today we have to wait for labs to do the pre-training an RL cycle every K months with the right environments and pre-training data to solve those tasks. So I feel like that is a limiting factor, at least personally. I feel like

[17:08](https://realalpha.app/v/rVMy8MSBKZo/?t=1028) beyond compute, there is an algorithmic limiting factor for us to make this useful in a way that I think could get us to like, we exist and we are able to use this technology to do something bigger. If companies could operate much more effectively, if there was one piece of model that can learn as a company evolves today the best we can do is compacted tokens in different forms of different types of memory and we would like to solve this at the fundamental layer and then build products on top so i do think i think i hedged the question to the answer by saying there is i think even without the data center power and um grid acumen there is some other algorithmic bottleneck that exist as well. Yeah. When we do get to that continual learning, or at least the first primitive type of continual learning, what do you think is the alignment risk