Reflection expects larger models as reinforcement learning scales

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 56:50

   coming. I imagine you're training a large model. Give me a sense of what you want to be pushing on as you continue to build these models and the cadence we can expect from you. You've been in stealth now. You've got this model. It's very exciting. But what can we expect in terms of cadence and sizing and all of that? Yeah, that's a good question. I think that both the mission and the vision of the company is to close the gap between the open and the closed frontiers and ensure there is one frontier and reflection is at the frontier of intelligence.

2. 57:23

   In order to do that, you need to both be scaling in terms of compute, in terms of the model size. So we'll have like, you know, significantly bigger models coming out like in 2027. And also in terms of like your scale of reinforcement learning. And then the thing about reinforcement learning is that it can happen in like waves. That's why you see that many of the other models like GLM 5.1, 5.2, 5.3, or like, you know, Fable 5.1, 5.2, 5.5 and so on and so forth.

3. 57:54

   So there's going to be a similar structure where we'll just have a bigger base model, and then we'll have multiple releases following after that. So there's a lot of more work that's happening already. We've already started building our next generation of models, and we'll be able to release and share more with the world in 2007. There's something about Yanis's work actually in AlphaGo and AlphaZero.

4. 58:23

   One of the things that I thought was so inspiring for me and why I switched from physics to AI is that an algorithm was developed that never stopped learning. And so by the time that AlphaGo beat Lee Sedol, it became superhuman or superintelligent. And at that point, it's just, well, you can put more compute in it and it can become even more superintelligent. It's just that it didn't make necessarily economic sense to maybe make it, you know, why, if you're already that good, What's the economic value of being even better at Go?

5. 58:54

   Now, when we actually started the company, we thought that this moment would come for general models, general coding agentic models. And one of the things the team really cracked is reinforcement learning just keeps scaling. And we've... This is, we believe, the largest reinforcement learning done... reinforcement learning run done in open source. It's been openly documented. It's around 10,000 GB300s that have been running for weeks, and it's never stopped learning.

6. 59:26

   It's a completely compute bottleneck that you just put more compute, and the thing keeps on learning. And so what we feel has happened, and this is across the field, and I think the closed labs have kind of figured this out a bit earlier, but it's extremely important that this kind of technology and science is also written about in the open, is the field is figured out how to scale models, not just scale pre-training, but also scale reinforcement learning.

7. 59:51

   And so now we're in this moment where you can really make the model super intelligent at anything. And it's just an economic task, like how much money do I want to put in and how do I get the data and so forth. But I think some of the fundamental scientific breakthroughs have been made and now it's about making them a lot more efficient. And that's really exciting because it's kind of a full circle moment, I think, for both of us for why we got into this field.

8. 1:00:17

   Giannis, I think AGI believer and I think you joined DeepMind in 2012. Yeah.