A 13-hour coffee run tested long-horizon robot behavior

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 22:41

   This kind of reminds me, I remember Figure had this demo where they live streamed the robot kind of sorting packages. When you watch that demo, does it demonstrate generalization, in your opinion? Yeah, yeah, I think it does. And by the way, this is something that I find very encouraging is that, like, I mentioned that it's hard to show generalization in a video. And it's clear that lots of people are thinking about that and that are thinking about how do you, like, how do you present something that somebody can watch and take in kind of at a glance what generalization is.

2. 23:13

   I think you can see a lot of creative steps towards that, live demos, these kind of like really long time lapses. I think it's a great idea, and I think that's like a really nice way to move towards elevating the importance of generalization in people's kind of consciousness. You know, when we were working on the PyStar06 project, the RL project that we did late last year, we wanted to do some longer horizon experiments.

3. 23:42

   In some cases, it's obvious like we had our robot assembling boxes at Dandelion chocolate factory So they're like, you know, it's an actual chocolate factory. So they need the boxes. So we run it for several days but we had this coffee task which was The robot was using an espresso machine to make espresso. So what we did is we ran it for 13 hours making espresso drinks We are, you know, we try to be like very, I guess, environmentally conscious about this.

4. 24:04

   So we didn't want to like throw out the coffee. So after 13 hours, everyone in the office was like a little wiry because someone had to like drink the coffee. But it like it ran for 13 hours and it was pretty cool. It screwed up a few times. Like it'll spill all the coffee grounds and then it needs to go and get like a cloth and wipe it down. But like it does it and it's, you know, nothing exploded.

5. 24:23

   13 hours went by. Probably the most negative consequence was like loss of sleep from too much caffeination. But when it spills the coffee grounds and cleans it up, it did that by itself. Well, so the way that that experiment was done is that there is a high-level prompting. So roughly the prompt is updated maybe every five minutes or so in between semantically coherent tasks. So you tell it, like, make espresso, clean up the machine, et cetera.

6. 24:49

   So those steps, the actual clean up the machine was commanded by a person. In principle, we could automate that. In fact, one of the things we're spending a lot of effort now is improving our high-level policy that does those commands. But for that experiment, like every five minutes, somebody basically updates what it's being asked to do. Like the way we intended it was like the commands would be like if you go to an actual coffee shop, you say like, oh, I want a latte.

7. 25:09

   I want an espresso. Like that was supposed to be the prompt, except then you also have to tell it, I want you to clean it up before you do the next one. That makes sense.