Unsupervised Learning · 2 min · Mar 17, 2025

Use one model unless specialization earns its cost

Jack Rae · Research Director, DeepMind

Noam and Jack Rae argue that high-value uses like medicine can support a general model, while specialization makes sense when cost or negative transfer requires it.

In this episode

19:34 Inference is cheap enough to justify much more thinking6 min
25:29 AI math progress depends on asking better questions6 min
31:48 Research teams need both bottom-up and top-down bets8 min
9:39 Qualitative tasks need broad reward criteria2 min
15:57 Agent research depends on choosing the right environment4 min
42:03Use one model unless specialization earns its cost2 min
1:06:30 Inference-heavy AI can use more distributed compute3 min
58:15 Hallucinations made entertainment a natural early use2 min

“If there's positive transfer, put it in the same model, as long as it doesn't then become too expensive to serve.”

Jack Raeat 43:55 · Unsupervised Learning

Takeaways

High-value tasks may not justify a specialist model on price alone.
Specialize when negative transfer is measurable.
Keep positive-transfer capabilities together if serving cost allows.

About this segment

Noam and Jack Rae argue that high-value uses like medicine can support a general model, while specialization makes sense when cost or negative transfer requires it. The rule helps frame when a specialist is worth maintaining.

Research Director, DeepMind

Led Gemini Thinking, co-led Gemini pre-training, and contributed to Gopher and Chinchilla language-model scaling research.

Jacob Effron, Host · Partner, Redpoint
Growth investor in Abridge, Ramp and Physical Intelligence; ex-Flatiron Health.

From “Noam Shazeer and Jack Rae: Scaling Test-time Compute, Reactions to Ilya & AGI”, Mar 17, 2025. Watch the full episode

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

42:07
You guys are obviously both big believers in these models becoming more and more general purpose. I guess a question some folks are asking is, you think about domains. You talked about healthcare earlier, Nome. The model that ultimately is going to be our AI doctor, is that just a continuation of what we're doing in some giant model? Is there, like, a healthcare-specific version of the model that ends up being released that is, like, you know, only has some set of data, you know, inputted into it?
42:32
Or they're just a bunch of guardrails? Or, like, paint that picture for me of what you ultimately think, like, our, you know, AI doctor or AI, you know, biology researcher. Yeah, I don't really think you, you know, would need very, you know, specific, you know, test-specific models for something that high value. because, you know, you probably pay like $1 a token for talking to your doctor. So it's like the LLM is like way, way, way cheaper at this point.
43:06
So the only reason for task-specific models is price. And so if there's things that you wouldn't pay a dollar a token, yeah yeah yeah like you need targeted yeah something to analyze you know vast quantities of data for marginal value then maybe you want something yeah you can test specific there's always this notion of like there'll be tons of like negative transfer out there so like compartment things i don't really feel like that has ended up being the case if it could be measured then And that's a good reason to compartmentals, like models.
43:39
If there's no negative transfer, there's positive transfer, then just have one big model. That's like, I think, my personal philosophy. This is not like a thing which people have like uniform agreement over, though. It's like a continuous active area of research. Like how much do you want to like specialize and spin off these expert models? But yeah, from the way I see it, it's just very simple. If there's positive transfer, put it in the same model, as long as it doesn't then become too expensive to serve.
44:05
You know, obviously you guys have been at the cutting edge for a while.