Robotics may advance faster by solving the whole field

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 28:08

   We obviously have a number of companies that are really doing full stack kind of robotics development. Let's talk maybe as an example, Optimus, right, at Tesla. And then we also have folks who are working much more on like broad based horizontal capabilities with then kind of essentially integrators or kind of like folks who are deploying, figuring out how to combine them. how do you think this plays out over the next five years, right?

2. 28:35

   Like, obviously, pros and cons of each approach, and you have a very strong, I guess, you know, horse in the race slash point of view. But walk us through kind of like how you think about the vertical versus horizontal landscape for robotics today. Yeah, so I think that to me, the most important thing here is that the robotic problem is really difficult. And one of the really important lessons of, you know, all of the recent advances we've seen in AI is that when you're faced with a really difficult intelligence problem, somewhat counterintuitively, making the problem as big as possible actually makes it easier. So if your goal is to solve machine translation, naively we might think that the best way to do it is to build a machine translation model, like get lots of paired examples of like English

3. 29:20

   sentences and Chinese sentences, train a model that maps one to the other, that's what the early sequence-to-sequence models did, and that's your method. It turns out that's the wrong answer. The right answer is to solve all NLP problems all at once, and then you get a better machine translation system. And this is like, at this point, this is not a controversial statement. Like every single NLP researcher will tell you this is exactly what you do.

4. 29:40

   So what does that tell us about robotics? That if you want your humanoid to be really good, do you get lots of examples of a humanoid or do you solve all the robotics problems? I think the answer there is very clear. And I think that there's a good reason why AI systems have this blessing of generality, because when you have generality, you get more data. You see more of the unusual situations, more of the exceptions, more of the edge cases. And in robotics, that's even more important than it is in natural language.

5. 30:09

   So I think this lesson actually applies more to robotics than it does to NLP. It's just like the bitter lesson applied to like robotics. And I think that's actually an interesting answer, which I hadn't thought of, which is you are, I guess, you know, it's easy to kind of conflate humanoids with all robotics, but they're not actually the same thing. It's kind of like the point you're also making, right?

6. 30:28

   There's nothing wrong with humanoids. I think humanoids are amazing. And I would love to see humanoid robots all around us. But I also think that tech generally doesn't work that way. Take computers, for instance. It used to be that computers were like the humanoid companies. You would go to IBM in the 50s and 60s, and you would buy a computer, and it was a vertically integrated business. IBM comes in, they install the computer.

7. 30:52

   It's a room-sized thing. The IBM technicians come in. The IBM software engineers come in. Everything is IBM. And now, how does it work? Now we have operating systems. We have hardware. We have software. A software company usually doesn't worry about, like, building their own, you know, CPUs because everything is nicely factorized. And I think we'll see that in robotics. We'll see this kind of explosion of creativity with all sorts of different robots, all sorts of different robotic applications.

8. 31:19

   And the key to that is the right kind of physical intelligence layer that enables that factorization to happen. Yeah, it's interesting. I think one of the, you know, certainly, I don't know how many of you have done this or remember doing this, but certainly like when I was younger, I would actually kind of hand assemble all of my CPUs, right? Like you basically go figure out like what different components to buy, you assemble it.

9. 31:39

   It was kind of fun. But I think one of the things that has happened even to myself, I haven't hand built a computer, I guess, you know, for a long time. But I think our relationship to hardware over the last 15 years, ever since this magical device came out, is that our only mental model is that great hardware kind of like almost exists like in a vertically integrated way. And so it has made us, I think, almost, you know, almost forget that there are different ways of doing this.

10. 32:07

    Like, you know, 85% of the world still uses kind of like the Android operating system with its like insane variety of different kind of like components and parts and like configurations. And so I think your answer there is great because both can happen, right? Like you will see vertical, you will see horizontal, you'll see innovation at almost every different like layer in a bunch of different ways.

11. 32:27

    yeah um do you think that there is still time to create a new uh robotics foundation model company that's an interesting question um it's hard to answer that question because it really depends on kind of an answer to something that nobody really knows which is how will the ultimate robot foundation model work so i certainly have my own theory about that i have my roadmap and the kind of research that physical intelligence is pursuing. But if I'm being very upfront about this, like that is an open science problem. And when you have an open problem like that, it could be the answer is something that nobody expects. Like, you know, certainly if you talk to Jan LeCun, he'll have a very different perspective on this than I do. And maybe he's right. Like, you know, the great thing about science is that it's not a, you know, a purely

12. 33:16

    abstract philosophical debate where we just tell stories. Like there is an actual truth out there and somebody's right and somebody's wrong. Yeah, I'll see you out in the field, right?