Robotics companies still lack an internet-scale training corpus

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 18:00

   versus like other AI? So, I mean, there's a lot of similarities. And I think in a way like LLMs that started off as, you know, like chatbots that exist purely in the text world and robots, which are like physical machines, very multimodal in nature. You can see these things kind of converging because the latest models are multimodal. They can take in audio, images, other things. in the same way that your robot is expecting that.

2. 18:24

   And so over time, I think they're converging a little bit. And in fact, a lot of the training approaches, pre-training, post-training, those concepts exist in the robotics world. However, there's still a lot of things that are unique to robotics that you would never do if you're working purely on an LLM. And that's a lot, basically like mixing in real-world data, different ways of collecting it, different ways of using simulation and figuring out how to tie that to all the intelligence that's embedded in like a modern LLM.

3. 18:46

   And then the data is like super important here, obviously. Data is important today. I think this is like a now problem. You know, if you look at LLMs, I think the reason that you can see so many different companies, like 20 different companies all building foundational models and get, you know, within a stone's throw of each other in terms of performance, small teams, large teams, whatever it is, is because essentially they're all starting from the same data set, which is the internet and everything that can be downloaded from it.

4. 19:11

   And, you know, that data kind of determines the quality of the model that you can get. And there's certainly some alpha on top of that from individual teams. But in the robotics world, there's no corpus of data like the internet that exists. There isn't an entire internet of point clouds or camera images of robots manipulating objects. And so right now we're in this early days where you've got to either bootstrap that data yourself, you've got to pay people to collect it for you, or you've got to try to interpret or generate robot data from other things like watching YouTube videos and trying to infer from hand motions how a robot should do the same thing. And so we're just kind of in the early, early days of that for robotics.

5. 19:45

   Do you think there should be like a scale AI for robotics data or will it be that a company like yours just generates its own data and gets smarter as a result of that? I don't know. I think they'll probably be both. In fact, I've probably talked to at least a dozen companies who want to be. the scale AI for robotics. And I think that there's going to be plenty of customers for that in the near term, especially as, you know, as this data void exists. But when that starts to be filled and we start to see useful robots in the world, I do think the majority of data collection will come from robots and less from people getting data. I would also think that for like for your product, for example, any data set that is not your products in the wild is going to be approximating the data and the perfect data set I would imagine would be if you had armies of robots out in homes

6. 20:29

   giving you data. If your technology is sufficiently advanced that you can do transfer learning from other forms of data, other robots, YouTube videos, whatever it is, any source of data, and you can use that to train your robot, that's like an advantage because you don't, you know, then that total size of that data set may be much larger than just the data set that would be collected on your specific robots. However, where we are today, it's much easier to get robots to do amazing things if the data collected came from the exact robot that you're trying to deploy a model on totally yeah and we'll see if that that changes over time yeah it