Robotics is held back by a much larger data problem

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 35:23

   it feels like everything is moving so fast and like every month there's new model releases that are blowing everyone's mind and maybe the caveat is like robotics feels very behind um we do not have general purpose robots everywhere we have some room buzz and we have teslas and waymos um but like what's the current state of robotics like how do we get there is it a question of hardware software do we just not have enough data yeah it's uh of course i'm i'm biased i'm biased when i say this but i believe it's almost entirely a data problem And the reason I say that is because hardware, of course, there's still a lot of innovation to be done on, you know, robot arms, actuators, and just overall hardware capabilities.

2. 36:12

   But there are form factors that if they were to act intelligently would work, right? But the problem is the model itself, which comes down to the data. And the key thing is that for robotics, we don't have an internet-scale pre-training data set. And we have to build that first. And it's really, really hard to build that. Totally. And the main approach that a lot of our customers and just robotics labs in general are taking is egocentric data, which is you get lots of humans to do lots of diverse but basic actions, which is household tasks, some commercial tasks, et cetera.

3. 36:56

   Yeah. And then you sort of structure that, you annotate, you know, action labels and so forth. And then you take that as like the large pre-training internet equivalent data set. But the problem is, you know, that I think is like there's promising results and we can, you know, there's probably going to be some baseline capabilities that come as a result of that. But the problem is that the real world has a lot more, it has a much larger action space than the digital world.

4. 37:26

   Like you can just do way more. And there's, you know, it's obviously three-dimensional. And so it's a much harder data problem. And therefore, the number of environments that you can create is like orders of magnitude more. So the data investments required is like also orders of magnitude more. And we just have not gone to the stage of, like, having that type of funding, for a lot of examples, essentially.

5. 37:52

   How did, like, how did self-driving cars fix this? Have they just been collecting data for, like, you know, 10 years? So, yeah, it's a really good question. So, self-driving cars have, yes, they have been collecting data for a long time. Um, and if you think about what is, what, what, what is the difficulty of, uh, driving in terms of a kind of a general purpose capability, it's really impressive, but it's a very easy structure task.

6. 38:24

   You literally have like, you have lines and you just have to follow those lines. And, and like, you know, there's, there's sort of a red light, you stop at the red light. And of course, I don't want to undermine like how impressive of a technology this is, but, But compared to humanoid robots where they see a different cup each time on a different desk and the way you grab it is like fundamentally different because of its size and friction and so forth, it becomes a much harder problem than driving.

7. 38:53

   And also, if you think about like, you know, driving is one of those skill sets where it's like almost all of humanity can do it. It's like a very generalist type skill set. and and what what what is a self-driving robot it's it's a it's actually not a general purpose robot at all it is like the most specific purpose or whatever it's one task type which is driving so uh so that's why it works and it's very impressive um but but but what what we're you know what robotics companies are trying to solve today is like general purpose robotics which is a you know it's not even in the same order of magnitude of a problem set uh which points to the fact that like if you if you go the route of trying to solve these niche robotics problems yeah it is a lot more solvable and that that frankly might be the path to take totally like maybe we

8. 39:42

   see sort of like a lot of these specific um robot companies come out like this one's for laundry this one's for making coffee etc and then um at some point they can they can merge and have enough