# Frontier compute budgets could reach $100 billion by 2027

Markdown copy of https://realalpha.app/s/frontier-compute-budgets-could-reach-100-billion-by-2027/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Dario Amodei](https://realalpha.app/builders/dario-amodei/) · CEO, Anthropic · Co-founded Anthropic, the AI safety company behind Claude; earlier led OpenAI's GPT-2 and GPT-3 research.
- Host: [Lex Fridman](https://realalpha.app/people/lex-fridman/) · Host, Lex Fridman Podcast · Robotics and AI researcher at MIT; started the show in 2018.
- Show: Lex Fridman Podcast
- Episode: [Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity | Lex Fridman Podcast #452](https://realalpha.app/v/ugvHCXCOmm4/), Nov 11, 2024
- Moment: 15:32 to 20:44 of the episode (5 min), playing on https://realalpha.app/s/frontier-compute-budgets-could-reach-100-billion-by-2027/; the episode from there: https://realalpha.app/v/ugvHCXCOmm4/?t=932

## Quote

> “2027, there are ambitions to build $**100 billion clusters**. And I think all of that actually will happen.”
> — [Dario Amodei](https://realalpha.app/builders/dario-amodei/) at [18:41](https://realalpha.app/v/ugvHCXCOmm4/?t=1121) · Lex Fridman Podcast

## Takeaways

1. He expects $100 billion training clusters by 2027.
2. SWE-bench rose from 3–4% to about 50% in ten months.
3. He expects coding performance to approach 90% within another year.

## About this segment

Dario Amodei expects frontier training clusters to reach $100 billion by 2027 and cites SWE-bench rising from 3–4% to 50% in ten months. Those gains underpin his forecast of rapid capability growth.

From “Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity | Lex Fridman Podcast #452”, Nov 11, 2024. [Watch the full episode](https://realalpha.app/v/ugvHCXCOmm4/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[15:38](https://realalpha.app/v/ugvHCXCOmm4/?t=938) If we do hit a limit, if we do hit a slowdown in the scaling laws, what do you think would be the reason? Is it compute limited, data limited? Is it something else, idea limited? So a few things. Now we're talking about hitting the limit before we get to the level of humans and the skill of humans. So I think one that's popular today and I think could be a limit that we run into, like most of the limits, I would bet against it, but it's definitely possible, is we simply run out of data.

[16:09](https://realalpha.app/v/ugvHCXCOmm4/?t=969) There's only so much data on the internet and there's issues with the quality of the data, right? You can get hundreds of trillions of words on the internet, but a lot of it is repetitive or it's search engine optimization drivel, or maybe in the future it'll even be text generated by AIs itself. And so I think there are limits to what can be produced in this way. That said, we, and I would guess other companies, are working on ways to make data synthetic where you can use the model to generate more data of the type that you have already or even generate data from scratch.

[16:49](https://realalpha.app/v/ugvHCXCOmm4/?t=1009) If you think about what was done with DeepMind's AlphaGo Zero, managed to get a bot all the way from, you know, no ability to play Go whatsoever to above human level just by playing against itself. There was no example data from humans required in the Alpha Go Zero version of it. The other direction, of course, is these reasoning models that do chain of thought and stop to think and reflect on their own thinking. In a way, that's another kind of synthetic data coupled with reinforcement learning. So my guess is with one of those methods will get around the data limitation, or there may be other sources of data that are available. We could just observe that even if there's no problem with data, as we start to scale models up, they just stop getting better. It seemed to be a reliable observation that

[17:37](https://realalpha.app/v/ugvHCXCOmm4/?t=1057) they've gotten better. That could just stop at some point for a reason we don't understand. The answer could be that we need to invent some new architecture. There have been problems in the past with, say, numerical stability of models where it looked like things were leveling off, but actually, you know, when we found the right unblocker, they didn't end up doing so. So perhaps there's some new optimization method or some new technique we need to unblock things.

[18:09](https://realalpha.app/v/ugvHCXCOmm4/?t=1089) I've seen no evidence of that so far, but if things were to slow down, that perhaps could be one reason. what about the limits of compute meaning uh the expensive uh nature of building bigger and bigger data centers so right now i think uh you know most of the frontier model companies i would guess are operating you know roughly you know one billion dollar scale plus or minus a factor of three right those are the models that exist now or are being trained now uh i think next year we're going to go to a few billion. And then 2026, we may go to above 10 billion. And probably by 2027, there are ambitions to build $100 billion clusters. And I think all of that actually will happen. There's a lot of determination to build the compute to do it within this country. And I would guess that it actually does happen. Now, if we get to 100 billion, that's still not enough

[19:05](https://realalpha.app/v/ugvHCXCOmm4/?t=1145) compute, that's still not enough scale, then either we need even more scale or we need to develop some way of doing it more efficiently, of shifting the curve. I think between all of these, one of the reasons I'm bullish about powerful AI happening so fast is just that if you extrapolate the next few points on the curve, we're very quickly getting towards human level ability, right? Some of the new models that we developed, some reasoning models that have come from other companies, they're starting to get to what I would call the PhD or professional level, right?

[19:37](https://realalpha.app/v/ugvHCXCOmm4/?t=1177) If you look at their coding ability, the latest model we released, Sonnet 3.5, the new or updated version, it gets something like 50% on Sweebench. And Sweebench is an example of a bunch of professional, real-world software engineering tasks. At the beginning of the year, I think the state of the art was 3% or 4%. So in 10 months, we've gone from 3% to 50% on this task. And I think in another year, we'll probably be at 90%.

[20:06](https://realalpha.app/v/ugvHCXCOmm4/?t=1206) I mean, I don't know, but might even be less than that. We've seen similar things in graduate level math, physics, and biology from models like OpenAI's 01. So if we just continue to extrapolate this, right, in terms of skill that we have, I think if we extrapolate the straight curve, within a few years, we will get to these models being above the highest professional level in terms of humans. Now, will that curve continue?

[20:36](https://realalpha.app/v/ugvHCXCOmm4/?t=1236) You've pointed to, and I've pointed to a lot of reasons why, you know, possible reasons why that