# Google built the TPU after estimating speech's server cost

Markdown copy of https://realalpha.app/s/google-built-the-tpu-after-estimating-speechs-server-cost/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Jeff Dean](https://realalpha.app/builders/jeff-dean/) · Jeff Dean is Chief Scientist at Google. · Co-founded AI startup Discovery Loop; previously Google chief scientist and co-creator of MapReduce, Bigtable.
- Host: [Diana Hu](https://realalpha.app/people/diana-hu/) · Managing partner, Y Combinator · CTO of Escher Reality; after Niantic bought it, ran AR for Pokémon GO.
- Show: Lightcone
- Episode: [Jeff Dean: The 1% Rule for Building in AI](https://realalpha.app/v/CxXgV54KzpQ/), Jul 30, 2026
- Moment: 6:01 to 9:23 of the episode (3 min), playing on https://realalpha.app/s/google-built-the-tpu-after-estimating-speechs-server-cost/; the episode from there: https://realalpha.app/v/CxXgV54KzpQ/?t=361

## Quote

> “that was like the equivalent of **20 years of advances** in speech recognition in just a few months of like fiddling with the model”
> — [Jeff Dean](https://realalpha.app/builders/jeff-dean/) at [7:01](https://realalpha.app/v/CxXgV54KzpQ/?t=421) · Lightcone

## Takeaways

1. Better speech models cut the error rate in half.
2. Three minutes of daily speech use could double Google's server fleet.
3. The TPU was specialized for low-precision, dense linear algebra.

## About this segment

Jeff Dean explains how better speech recognition threatened to double Google's server fleet. That cost pushed the team to build specialized hardware that was far more efficient and lower latency.

From “Jeff Dean: The 1% Rule for Building in AI”, Jul 30, 2026. [Watch the full episode](https://realalpha.app/v/CxXgV54KzpQ/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[6:07](https://realalpha.app/v/CxXgV54KzpQ/?t=367) Sounds funny. So one of the stories about you is that back in 2013, when speech recognition started to work at Google, you did the napkin math, where if every Google user used their phone and talked to it and used the speech recognition system for just three minutes a day, you found that the system requires a Google server, you would have to double the fleet. which would be really, really expensive just to do speech translation.

[6:39](https://realalpha.app/v/CxXgV54KzpQ/?t=399) Yeah. And instead, you basically built a custom ship and that was the origin story of the TPU. Yeah. Yeah. I mean, I sort of had done, you know, we were starting to see really good quality results on the sort of deep learning based speech systems, speech models we were training. but they were computationally expensive compared to the old speech system but they halved the error rate so that was like the equivalent of 20 years of advances in speech recognition in just a few months of like fiddling with the model and getting scaling it up a bit and getting better data and so we started to get worried that if speech worked a lot better people would use it more and so that that back of the envelope calculation was really about that like well what if people start to use speech recognition more to dictate emails or to talk to their phone

[7:33](https://realalpha.app/v/CxXgV54KzpQ/?t=453) or whatever? And yeah, it turned out that we realized that we needed some better solution than running on CPUs at the time. And so we came up with TPUs, which are sort of very specialized for essentially low precision, dense linear algebra, which is at the heart of nearly all of the modern machine learning algorithms we use today, and... If you build a specialized chip for low precision, dense linear algebra and can't do anything else, that turns out to be really useful for machine learning inference, even though it can't run Chrome or Word or whatever.

[8:11](https://realalpha.app/v/CxXgV54KzpQ/?t=491) And so that system produced a chip a couple years later that was 30 to 80 times more energy efficient than CPUs and GPUs of the day, and also much, much lower latency, like 20 to 30x lower latency. Which is incredible what the foundation that TPU has become today. No way you would have predicted that TPU would be so foundational now with transformer architecture, which was invented way later, before you actually invented the TPU.

[8:41](https://realalpha.app/v/CxXgV54KzpQ/?t=521) Yeah, I mean, that's sort of why we built a general purpose linear algebra system, which is what a TPU is, really. Because we knew ML algorithms were still evolving, and you didn't want to over-specialize, But you wanted to specialize enough that you got the dramatic performance benefits of, we could have very big multiplier units, we could have, you know, high speed memory, we can have high speed interconnect, or later TPUs that like brought many, many chips to bear on the same problem efficiently. And, you know, we've continued to scale those up and, and improve their performance over many, many generations now.