# Distillation met a need for cheaper model serving

Markdown copy of https://realalpha.app/s/distillation-met-a-need-for-cheaper-model-serving/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Jeff Dean](https://realalpha.app/builders/jeff-dean/) · Jeff Dean is Chief Scientist at Google. · Co-founded AI startup Discovery Loop; previously Google chief scientist and co-creator of MapReduce, Bigtable.
- Host: [Diana Hu](https://realalpha.app/people/diana-hu/) · Managing partner, Y Combinator · CTO of Escher Reality; after Niantic bought it, ran AR for Pokémon GO.
- Show: Lightcone
- Episode: [Jeff Dean: The 1% Rule for Building in AI](https://realalpha.app/v/CxXgV54KzpQ/), Jul 30, 2026
- Moment: 47:44 to 50:02 of the episode (2 min), playing on https://realalpha.app/s/distillation-met-a-need-for-cheaper-model-serving/; the episode from there: https://realalpha.app/v/CxXgV54KzpQ/?t=2864

## Quote

> “we actually saw this was a super important problem because we knew **making cheaper, highly capable models** from larger scale models was something we desperately wanted to do”
> — [Jeff Dean](https://realalpha.app/builders/jeff-dean/) at [48:51](https://realalpha.app/v/CxXgV54KzpQ/?t=2931) · Lightcone

## Takeaways

1. The paper was rejected with a review predicting little impact.
2. Google needed cheaper capable models to serve more users.
3. Rejection did not stop the work from being shared and used.

## About this segment

Jeff Dean recounts a distillation paper being rejected despite a clear need for smaller, cheaper models to serve speech and vision. The work was shared on arXiv and became part of Google’s model development.

From “Jeff Dean: The 1% Rule for Building in AI”, Jul 30, 2026. [Watch the full episode](https://realalpha.app/v/CxXgV54KzpQ/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[47:50](https://realalpha.app/v/CxXgV54KzpQ/?t=2870) point founders or start your careers you will probably collect lots of rejections that will happen it has happened to you too Jeff I mean, there's a story that in 2014, you, with Jeff Hinton and Oriel Vinales, wrote a paper on distillation, which has to do with taking a big teacher model to train a much smaller and more efficient model that's a lot cheaper to compute, less model parameters. And it has become a trick that everyone is using right now in industry.

[48:29](https://realalpha.app/v/CxXgV54KzpQ/?t=2909) Yeah. And the thing is, this paper got rejected at UREPS. Yeah, I mean, yeah. I mean, I think I don't fault the program committee because, you know, a lot of times a paper gets three reviews and someone will look at it, one of the reviewers will look at it. And in this case, they said, oh, it's unlikely to have significant impact. Unlikely to have significant impact. But, you know, I think, you know, when we wrote the paper, we actually saw this was a super important problem because we knew making cheaper, highly capable models from larger scale models was something we desperately wanted to do because we wanted to serve models to more and more people in many different domains like speech or vision.

[49:13](https://realalpha.app/v/CxXgV54KzpQ/?t=2953) But sometimes the reviewer maybe didn't have that experience because maybe they're not thinking about large scale AI services and are thinking about is this a fundamental advance. um so so you know it gets rejected every so often that's fine we put it on archive people read it people use it it's all good uh and you know we do use it in making our flash models for example from our larger scale pro model that's partly why our flash models for example in gemini are so capable uh relative to their size and speed they're some of the best in the benchmark for their model size class just impressive and i think part of the lesson is that even if you get rejected keep going yeah that's that's the lesson i would distill from that