Robots need useful real-world data, not just more data

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:02:43

   I mean, which of those two, which of these approaches is most expensive? How do you think about capital constraints here with training any of these? All of them are, yeah. So the biggest challenge in robotics, right, is data. Right. And the thing that most people miss is it's not just volume of data. There's now hundreds of companies out there trying to sell data for a physical AI. But a lot of the data is not that usable, right?

2. 1:03:08

   So you don't just need a lot of general. We're not just going to feed in. People are certainly trying to experiment with feeding in a whole lot of YouTube data, for example. But it's not just that data. We need high-quality data, demonstrations of robots doing things. you know the really important part for most companies is when you get beyond the first proof of concept how can you put robots out there in the real world and then learn from the data that you're getting on the particular tasks so you think about like a self-driving car example right like what separates uh what separates companies that are driving driverless around san francisco today from people that are still testing what was the the hardest thing that took the industry you know 10 to 15 years to get over this hump of like taking the driver out of the

3. 1:03:46

   car it was the ability to put those cars out in the real world and then learn from learn from that experience so look at all of this driving data discard the 99 of it that's just me driving along a straight road find that interesting like one percent or point one percent of like weird interesting things that happened turn that back into learnings turn that like turn that back into training for a fleet and then being able to know being able to like evaluate and validate a release to actually know that like that data had an impact yeah we've gotten better at driving in this particular and we know that across these like thousand other situations we haven't got any worse wasn't there i mean there was an approach using gaming to to simulate data is that always try this right but they tried it's not like simulation in general right like a lot of people um would

4. 1:04:29

   really like it to be true that we can go build some robots in simulation and then we can deploy in the real world and they will be successful and people are having limited success but you have of this big sim-to-real gap is what it's called, right? So you have like, hey, great, the robot completes the task 100% of times in simulation, but the real world is just different. It's more complicated. And some things, you know, in some ways, for example, self-driving cars are actually a little bit of an easier simulation problem than some of the things that we're trying to create in general purpose robots, right?

5. 1:04:59

   Because why? so you think about driving i mean it's a really hard thing to simulate it's not easy but at least there are rules of the road you've got vehicles you've got cyclists you've got pedestrians if you look at enough data you can kind of predict what what those different actors are going to do over time um but it's relatively structured but then if you try thinking how do i simulate everything that could possibly happen in a household for example right like i kind of i frame residential robots as like the final boss of physical ai because you just have so many different tasks that I can take everything from cracking eggs to folding laundry to, you know, all of these different things. And then you need to deal with model. If you try to simulate it again, which is talking about how do you model, model, you know, the properties of a sheet of a bed sheet or a

6. 1:05:41

   clothing or things like this? How do I model the squishiness of a strawberry? Or these are just like really, really hard things to, to model. And so you just kind of have this like combinatorial explosion of, of complexity. Right. Let me ask you one last question before we let you go.