AI still lacks continual learning despite strong scaling returns

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 2:55

   And I think we're pretty much on track. What are the biggest bottlenecks when you look today in the documentary that you just never have enough compute? What are the biggest bottlenecks when you look at where we are today? I think compute is the big one, not just for the obvious reason of scaling up your ideas and your systems as, you know, the scaling laws, as they're called, you know, keeping on building bigger and bigger architectures with more and more parameters.

2. 3:20

   And as you do that, you get more intelligent systems. But the other thing you need a lot of compute for is for doing experiments. The computers, the cloud is our workbench, basically. So if you have a new idea, a new algorithmic idea, but you want to test it, you kind of got to test it at a reasonable scale. Otherwise, it won't hold when you actually put it into the main system. So you need quite a lot of compute if you have a lot of researchers with lots of new ideas.

3. 3:52

   You mentioned the word scaling laws. A lot of people suggest that we're hitting scaling laws and we're starting to see that plateauing effect. Yeah. Do you think that's true? No, I don't think so. I think it's a bit more nuanced than that. So, of course, when the leading companies all started building these large language models, you're getting enormous jumps with each generation of new system. You know, maybe they're almost like doubling in performance.

4. 4:14

   At some point, that had to slow down. So it's not kind of continuing to be exponential. But that doesn't mean there isn't great returns still for scaling the existing, you know, systems up further. So we and the other frontier labs are getting a lot of great returns on that kind of compute expansion. So I would say the returns are kind of still very substantial, although they're a bit less than they were, obviously, at the start of all of this scaling.

5. 4:48

   Where are we behind where you thought we'd be? I think actually in most areas we are ahead of where I thought we would be. If you think about things like the video models or even now with our newest systems like Genie, they're interactive world models, which I think is kind of incredible if you sort of step back and think about it. I think if you'd shown me that five, 10 years ago, I would have been pretty amazed.

6. 5:11

   So I think. In most domains, we are ahead of where the field thought. There's still some big things missing, though, like continual learning. These systems don't learn after you finish training them, after you put them out into the world. You know, they're not very good at learning further things. And I think some critical capabilities are missing. Why is that? I'm sorry to ask blunt and basic questions.

7. 5:25

   Why do we not have continuous learning? Well, people haven't quite figured out yet. And all the leading labs are working on this, like how to integrate new learning into the existing systems that you spent months training. Of course, the brain does this very elegantly, right? And probably through things like sleep reinforcement learning. So you just kind of get consolidation, it's called, in the brain, where your memories during the day are replayed.

8. 5:50

   And then some of that information is elegantly incorporated into your existing knowledge base. And perhaps we, I thought for a while, maybe we need something like that to incorporate new information along with the existing information base.