# AI has returned to an age of research

Markdown copy of https://realalpha.app/s/ai-has-returned-to-an-age-of-research/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Ilya Sutskever](https://realalpha.app/builders/ilya-sutskever/) · Ilya Sutskever is cofounder of SSI. · Co-founded Safe Superintelligence in 2024 and leads it as CEO, following earlier breakthroughs beginning with AlexNet and work at OpenAI.
- Host: [Dwarkesh Patel](https://realalpha.app/people/dwarkesh-patel/) · Host, Dwarkesh Podcast · Co-wrote The Scaling Era, an oral history of AI; named to TIME100 AI.
- Show: Dwarkesh Podcast
- Episode: [Ilya Sutskever – We're moving from the age of scaling to the age of research](https://realalpha.app/v/aR20FWCCjAs/), Nov 25, 2025
- Moment: 18:51 to 25:25 of the episode (7 min), playing on https://realalpha.app/s/ai-has-returned-to-an-age-of-research/; the episode from there: https://realalpha.app/v/aR20FWCCjAs/?t=1131

## Quote

> “So it's **back to the age of research** again, just with big computers.”
> — [Ilya Sutskever](https://realalpha.app/builders/ilya-sutskever/) at [21:53](https://realalpha.app/v/aR20FWCCjAs/?t=1313) · Dwarkesh Podcast

## Takeaways

1. Pre-training scaling offered a comparatively low-risk investment.
2. Finite data limits how far the original recipe can go.
3. Research can matter more than simply adding compute.

## About this segment

Ilya Sutskever argues that scaling pre-training was a useful recipe, but more compute alone is unlikely to transform AI. The next gains depend on finding better ways to use compute.

From “Ilya Sutskever – We're moving from the age of scaling to the age of research”, Nov 25, 2025. [Watch the full episode](https://realalpha.app/v/aR20FWCCjAs/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[19:04](https://realalpha.app/v/aR20FWCCjAs/?t=1144) the thing so so here is a perspective here's a perspective i think might be might be true so the way ml used to work is that people would just think with stuff and try to and try to get interesting results that's what's been going on in the past then the scaling insight arrived, right? Scaling laws, GPT-3. And suddenly everyone realized we should scale. And this is an example of how language affects thought.

[19:48](https://realalpha.app/v/aR20FWCCjAs/?t=1188) Scaling is just one word, but it's such a powerful word because it informs people what to do. They say, okay, let's try to scale things. And so you say, okay, so what are we scaling? And pre-training was a thing to scale. It was a particular scaling recipe. The big breakthrough of pre-training is the realization that this recipe is good. So you say, hey, if you mix some compute with some data into a neural net of a certain size, you will get results.

[20:20](https://realalpha.app/v/aR20FWCCjAs/?t=1220) And you will know that it will be better if you just scale the recipe up. and this is also great companies love this because it gives you a very uh low risk way of investing your resources yeah right it's much harder to invest your resources in research compare that you know if you research you need to have like go forth researchers and research and come up with something versus get more data get more compute you know you'll get something from pre-training and indeed you know it looks like i based on various um things people say on some people say on twitter maybe it appears that gemini have found a way to get more out of pre-training at some point though pre-training will run out of data the data is very clearly finite and so then okay what do you do next either you do some kind of a souped up

[21:12](https://realalpha.app/v/aR20FWCCjAs/?t=1272) pre-training different recipe from the one you've done before or you're doing rl or maybe something else. But now that compute is big, computer is now very big, in some sense, we are back to the age of research. So maybe here's another way to put it. Up until 2020, from 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling, or maybe plus minus. Let's add error bars to those years. Because people say, this is amazing. You got to Scale more, keep scaling, the one word, scaling.

[21:46](https://realalpha.app/v/aR20FWCCjAs/?t=1306) But now the scale is so big, like is the belief really that, oh, it's so big, but if you had a Honeydex more, everything would be so different. Like it would be different for sure. But like, is the belief that if you just Honeydex the scale, everything would be transformed? I don't think that's true. So it's back to the age of research again, just with big computers. That's a very interesting way to put it.

[22:12](https://realalpha.app/v/aR20FWCCjAs/?t=1332) But let me ask you the question you just posed then. What are we scaling and what would it mean to have a recipe? Because I guess I'm not aware of a very clean relationship that almost looks like a law of physics, which existed in pre-training. There's a power law between data or computer parameters and loss. What is the kind of relationship we should be seeking and how should we think about what this new recipe might look like?

[22:40](https://realalpha.app/v/aR20FWCCjAs/?t=1360) so we've already witnessed a transition from one type of scaling to a different type of scaling from pre-training to rl now people are scaling rl now based on what people say on twitter they spend more compute on rl than on pre-training at this point because rl can actually consume quite a bit of compute you know you do very very long rollouts yes so it takes a lot of compute to produce those rollouts and then you get relatively small amount of learning power rollout so you really can spend you really can spend a lot of compute and i could imagine like i wouldn't at this at this it's more like i wouldn't even call it a scale and scaling i would say hey like what are you doing and is the thing you are doing the the the the most productive thing you could be doing yeah can you find a most more productive way of using your compute we've discussed the

[23:39](https://realalpha.app/v/aR20FWCCjAs/?t=1419) value function business earlier and maybe once people get good at value functions they will be using their their resources more productively and if you find a whole other way of training models you could say is this scaling or is it just using your resources i think it becomes a little bit ambiguous in a sense that when people were in the age of research back then it was like people say hey let's try this and this and this let's try that and that and that oh look something interesting is happening and i think there will be a return to that so if we're back in the era of research stepping back what is the part of the recipe that we need to think most about when you say value function people are already trying the current recipe but then having llm as a judge and so forth you could say that's a value function but it sounds like you have something much more

[24:29](https://realalpha.app/v/aR20FWCCjAs/?t=1469) fundamental in mind do we need do we need to go back to should we even rethink pre-training at all and not just add more steps to the end of that process yeah so the the the discussion about value function i think it was interesting i want to like emphasize that i think the value function is something like it's going to make our realm more efficient and i think that makes a difference but I think that anything you can do with a value function you can do without just more slowly the thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people and it's super obvious that seems like a very fundamental thing okay so this is the crux generalization