Waymo's early team favored software over custom-built hardware

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 2:49:17

   Well, one, there's a huge problem with noisy data that comes out of all of these sensors. You know, it's in a car in the desert getting rocked around, it's in the heat, it's in the sun. So common wisdom, and what Carnegie Mellon did, was to do as much as you possibly can on the hardware to mitigate that. So things like custom rigging and gimbals and giant springs to stabilize the sensors, Carnegie Mellon would essentially buy a Hummer and rip it apart and rebuild it from the wheels up.

2. 2:49:43

   We're talking like welding and real construction on a car. The Stanford team did the exact opposite. They viewed any new piece of hardware as something that could fail, and so in order to mitigate risks on race day, they used all commodity cameras and sensors that they just mounted on a nearly unmodified Volkswagen. So they only innovated in software, and they figured they would just kind of come up with clever algorithms to help them clean up the messy data later.

3. 2:50:09

   Very googly, right? Very googly. The second thing they did was an early use of machine learning to combine multiple sensors. They mounted laser hardware on the roof, just like what other teams were doing. And this is the way that you can measure texture and depth of what is right in front of you. And the data, it's super precise, but you can't drive very fast because you don't really know much about what's far away, since it's this fixed field of view, it's very narrow.

4. 2:50:36

   Essentially, you can't answer that question of, how fast can I drive, or is there a turn coming up? So on top of that, the way they solved it was they also mounted a regular video camera. That camera can see a pretty wide field of view, just like the human eye, and it can see all the way to the horizon, just like the human eye. And crucially, it could see color. So what it would do, this is like really clever, they would use a machine learning algorithm in real time in 2005.

5. 2:51:06

   This computer is like sitting in the middle of the car. They would overlay the data from the lasers on top onto the camera. feed. And from the lasers, you would know if the area right in front of the car was okay to drive or not. Then the algorithm would look up in the frames coming off the camera, overlaid, what color that safe area was, and then extrapolate by looking further ahead at other parts of the video frame to see where that safe area extended to. So you could figure out your safe path through the desert.

6. 2:51:35

   That's awesome. It's so awesome. I'm imagining like a Dell PC sitting in the middle of this car in 2005. It's not far off in the email that we send out. We'll share some photos of it. It could then drive faster with more confidence, and it knew when turns were coming up. Again, this is real-time, onboard the camera, 2005. It's wild on that tech. So ultimately, both of these bets worked, and the Stanford team won in super dramatic fashion.

7. 2:52:03

   They actually passed one of the Carnegie Mellon teams autonomously through the desert. It's like this big dramatic moment in the documentary. So you would kind of think, so then Sebastian goes to Google and builds Waymo. No. As we talked about earlier, he does join Google through that crazy, please don't raise money from Benchmark and Sequoia and we'll just hire you instead. But he goes and works on Street View and Project Ground Truth and co-found Google X.

8. 2:52:29

   David, as you were alluding to earlier, this Project Chauffeur that would become Waymo is the first project inside Google X. And I think the story, right, is that Larry came to Sebastian and was like, yo, that's self-driving car stuff. Like, do it. And Sebastian was like, no, come on, that was a DARPA challenge. And Larry was like, no, no, you should do it. He's like, no, no, that won't be safe. There's people running around cities. I'm not just going to put multi-ton killer robots on roads and go and potentially harm people. And Larry finally comes to him and says, why? What is the technical reason that this is impossible?

9. 2:53:07

   And Sebastian goes home, has a sleep on it, and he comes in the next morning and he goes, I realize what it was. I'm just afraid. Such a good moment. So they start. He's like, there's not a technical reason. As long as we can take all the right precautions and hold a very high bar on safety, let's get to work. So Larry then goes, great, I'll give you a benchmark. So that way you know if you're succeeding.

10. 2:53:32

    He comes up with these 10 stretches of road in California that he thinks will be very difficult to drive. It's about 1,000 miles, and the team starts calling it the Larry 1000. And it includes driving to Tahoe, Lombard Street in San Francisco, Highway 1 to Los Angeles, the Bay Bridge. This is the bogey. Yep. If you can autonomously drive these stretches of road, pretty good indication that you can probably do anything.

11. 2:53:58

    Yep. So they start the project in 2009. Within 18 months, this tiny team, I think they hired, I don't know, it's like a dozen people or something. They've driven thousands of miles autonomously, and they managed to succeed in the full Larry 1000 within 18 months. totally unreal how fast they did it and then also totally unreal how long it takes after that