How Close Are We to Robots That Can Do Anything? | Sergey Levine, Physical Intelligence

Summary · 3 min read

Levine says he moved from computer graphics and character animation into robotics after realizing that methods for simulated characters might help physical robots. He later worked on robotics at Google and co-founded Physical Intelligence with a team drawn from research settings.

Levine argues that robot learning needs both real-world experience and prior knowledge from foundation models, because robot data alone can produce narrow skills. He describes how Physical Intelligence’s models add reasoning and aim to generalize to tasks without task-specific action training. He says robotics can gain data at scale as robots become useful, and that broad capabilities can help solve difficult tasks by exposing models to more situations. He also emphasizes human interventions as a near-term learning loop and a research culture where people understand the full robotics stack.

Minute by minute

1. 0:28

   Prior knowledge complements robot data. Levine says he moved from computer graphics and character animation into robotics because techniques for virtual characters might help physical robots. At Google, robots gathering grasping data became good at grasping but not other tasks; he says robots also need prior knowledge, which foundation models can provide.

2. 4:19

   The founding team made the timing work. Levine credits Physical Intelligence CEO Carol Hausman with proposing that they start a robotics research lab, and says a network of like-minded researchers helped launch it. He says having the right people early made the difficult first days more manageable.

3. 6:36

   Pi Zero combines language and action. Levine says a robot foundation model takes images and language as input and produces continuous actions. Physical Intelligence started from an LLM because it provides knowledge that robot data lacks, while the robot data grounds that knowledge; he describes Pi Zero as an outline that later models could extend with reasoning and other data.

4. 9:32

   Reasoning helps robots generalize. Levine says Pi 0.5 added internal reasoning that breaks instructions such as cleaning a kitchen into steps before grounding them in actions. He says Pi 0.7 aims for zero-shot generalization: people can coach it in language without supplying action supervision for each new skill. It can also reason with images, such as generating a picture of a robot finger touching an appliance’s power button.

5. 16:39

   Robotics researchers need attention to detail. Levine says robotics work ranges from tightening gripper screws to training models on thousands of GPUs, so researchers need to understand many levels of the stack. He says credentials and publication records matter less than curiosity, willingness to learn, and attention to detail, though evaluating those qualities when hiring is difficult.

6. 21:08

   Useful robots can produce abundant data. Levine argues that the physical world contains far more information than the internet and that commercially useful robots could eventually generate enormous amounts of data. He says less-specialized robots can learn across many lower- and higher-stakes tasks, unlike cars, whose narrow role limits what they can learn from.

7. 26:01

   Interventions can lead to self-improvement. Levine says human interventions during supervised deployments can supply data that improves a robot’s performance. In his view, reinforcement learning could later help robots avoid mistakes rather than only correct them; he says this is working in laboratory experiments but is not yet widely used in deployments.

8. 28:02

   Broad capabilities support robotics progress. Levine argues that difficult intelligence problems can become easier when researchers solve a broader problem, as happened in his example of machine translation benefiting from progress across NLP. He expects robotics to develop a range of robots and applications, with a shared physical-intelligence layer enabling specialization rather than requiring every product to be vertically integrated.

9. 33:21

   Data strategy matters more than model labels. Levine says the key foundation-model question is where training data will come from and how its supply will grow, not whether the system is called a world model, video model, or LLM. He expects future models may combine these approaches, including world-model capabilities for visual and spatial reasoning, but says attention to model type can distract from the data problem.

10. 36:10

    Teams need expertise and shared context. Levine says Physical Intelligence benefits from respecting specialized skills while encouraging people to understand the broader stack. He points to ML infrastructure researchers attending a hardware talk as an example of people seeking context beyond their own areas.

11. 39:14

    Levine favors wheeled home robots. Levine says a robot using a plastic bag to pick up simulated dog poop surprised him, and he favors a wheeled base over a humanoid for a first home robot because it is less likely to fall and injure a pet or child. He recommends Abhishek Gupta’s work on universal world models and a paper called Neural Thickets for its account of why reinforcement learning may work.

Published Oct 22, 2026.