Robot experience can help models learn from human video

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 26:03

   It sounds like on the way to generalizing, data is a very important part of that. And I was reading there's different types of data. There's simulated data. You could collect, like, physical interactive data. And I wanted to hear your take on what's the best data to get, what's the worst, and what are the pros and cons? This is, by the way, like a question that is, I guess, quite— there's a lot of discussion in the robotics community about this question, and some people have, like, very opposite opinions on it.

2. 26:35

   My own take on this is that a lot of different data sources are easier for the model to internalize if it can ground them in a thorough physical understanding of the world. So let me try to explain what I mean with a few examples. If you want to learn to fly an airplane, you will probably use a simulator, at least during part of your training. But the simulator makes a lot of sense to you because when you start using the simulator, you have a lot of world knowledge that you can use to ground what's going on.

3. 27:09

   Like, you know that when you are using the flight simulator to learn how to fly the airplane, you're not just, like, playing a video game. You're trying to acquire knowledge that you will then use with a real airplane. And you understand that there's sort of an abstraction there. Same thing if you're playing, like, a really cartoony, like, you know, Atari game or something, right? Like, you know that all the symbols on the screen, you can sort of connect them to things that you've experienced in your life and you can make an analogy there.

4. 27:35

   So a lot of that, like even though it kind of seems like these simulated environments reflect aspects of the real world, to us they make a lot of sense because we kind of bring to bear a lot of our own prior experience and we fill in the blanks that the simulation has. And also if you want to use data, yourself as a person, of somebody else doing something, If you watch something, On, let's say, cooking a meal, right, even though you don't experience every movement they're experiencing, you have a lot of that knowledge that you bring to bear, and you're like, okay, I see they're picking up the salt shaker, like, I've put salt on things before, so I kind of roughly know what's going on there, and I can file it away at this level of abstraction of, like, add salt without having to, like, figure out all their muscle

5. 28:18

   movements. So my point with this is that once you have that understanding of how you do things physically with your own body and how you experience the physical world, now all these other sources of knowledge can be connected up to it because that foundation you get from your experience helps you ground everything. So where I'm going with this is that if we have a robotic foundation model that is trained on lots of real embodied data that provides that grounding, it might actually be much better able to absorb other sources of knowledge.

6. 28:46

   And this is actually a little bit upside down relative to how some people think about it because it's very tempting looking at the success of internet data for LLMs to say, well maybe we should do the opposite and we should like start with like YouTube videos and then put robot data on top of that but I think it's actually the other way around and I actually I even have like a little bit of evidence for this so my colleague Suraj Nair together with Simar from Georgia Tech they had a project together a while back where they took our robot foundation model and they added human video data but they didn't start with human video data they actually started with a model train on robot data and then added video data on top of it. And what they did is they looked at the representations inside the model. Basically, how does the model represent

7. 29:30

   human experience versus robot experience? And they found that if you use a small model with a small amount of robot data, predictably the human experience and the robot experience are fully separated, meaning that the feature representations are different. But if you train on lots of robot data from lots of different robots, then the features are grouped much more by what task is being done rather than by whether it's a human or a robot.

8. 29:52

   And when we looked at the feature plots, it was just mind-boggling because literally when you crank up the amount of robot data to 100%, they just line up perfectly. You do this T-SNE embedding, you see the shapes of the embeddings, and it's just all task identity and minimal sensitivity to embodiment. And to me, that's kind of mind-blowing because the base model wasn't trained on any human data at all.

9. 30:15

   But once you start adding human data, it represents it exactly the same way and I think that's really exciting I think that that to me is like one of the strongest indicators that if you have that good foundation of robot experience you can put everything else on top of it it's actually better at absorbing that. Is it important that that base