# Physical and institutional limits slow AI-driven change

Markdown copy of https://realalpha.app/s/physical-and-institutional-limits-slow-ai-driven-change/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Dario Amodei](https://realalpha.app/builders/dario-amodei/) · CEO, Anthropic · Co-founded Anthropic, the AI safety company behind Claude; earlier led OpenAI's GPT-2 and GPT-3 research.
- Host: [Lex Fridman](https://realalpha.app/people/lex-fridman/) · Host, Lex Fridman Podcast · Robotics and AI researcher at MIT; started the show in 2018.
- Show: Lex Fridman Podcast
- Episode: [Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity | Lex Fridman Podcast #452](https://realalpha.app/v/ugvHCXCOmm4/), Nov 11, 2024
- Moment: 2:05:49 to 2:11:57 of the episode (6 min), playing on https://realalpha.app/s/physical-and-institutional-limits-slow-ai-driven-change/; the episode from there: https://realalpha.app/v/ugvHCXCOmm4/?t=7549

## Quote

> “And so if we bring AI systems into this, you know, into these human systems, often the level of intelligence may just **not be the limiting factor**, right?”
> — [Dario Amodei](https://realalpha.app/builders/dario-amodei/) at [2:10:03](https://realalpha.app/v/ugvHCXCOmm4/?t=7803) · Lex Fridman Podcast

## Takeaways

1. Physical experiments and hardware production take time.
2. Greater intelligence does not remove complexity or uncertainty.
3. Human institutions and democratic legitimacy constrain deployment.

## About this segment

Dario Amodei pushes back on instant-takeoff predictions, citing physical constraints, complex systems and human institutions. He argues that even powerful AI must work within social and democratic processes.

From “Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity | Lex Fridman Podcast #452”, Nov 11, 2024. [Watch the full episode](https://realalpha.app/v/ugvHCXCOmm4/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[2:05:54](https://realalpha.app/v/ugvHCXCOmm4/?t=7554) to figure out how fast two extreme positions both seem false to me so the singularity is on the one extreme and the opposite on the other extreme can you describe each of the extremes yeah so why so yeah let's let's describe the extreme so like one one extreme would be well look um you know uh if we look at kind of evolutionary history like there was this big acceleration where you know for hundreds of thousands of years, we just had like, you know, single celled organisms. And then we had mammals and then we had apes. And then that quickly turned to humans, humans quickly built industrial civilization. And so this is going to keep speeding up and there's no ceiling at the human level. Once models get much, much smarter than humans, they'll get really good at building the next models. And, you know, if you write down like a simple differential equation, like this is

[2:06:42](https://realalpha.app/v/ugvHCXCOmm4/?t=7602) an exponential. And so what's, what's going to happen is that, uh, models will build faster models, models will build faster models, and those models will build nanobots that can take over the world and produce much more energy than you could produce otherwise. And so if you just solve this abstract differential equation, then five days after we build the first AI that's more powerful than humans, then the world will be filled with these AIs and every possible technology that could be invented, will be invented. I'm caricaturing this a little bit, but I think that's one extreme. And the reason that I think that's not the case is that one, I think they just neglect like the laws of physics. Like it's only possible to do things so fast in the physical world. Like some of those loops go through, you know, producing faster hardware. It takes a long

[2:07:34](https://realalpha.app/v/ugvHCXCOmm4/?t=7654) time to produce faster hardware. Things take a long time. There's this issue of complexity. Like I think no matter how smart you are, like, you know, people talk about, oh, we can make models the biological systems that'll do everything the biological systems look i think computational modeling can do a lot i did a lot of computational modeling when i worked in biology but like just there are a lot of things that you can't predict how they're you know they're they're complex enough that like just iterating just running the experiment is going to beat any modeling no matter how smart the system doing the modeling is or even if it's not interacting with the physical world just the modeling is going to be hard yeah i think well the modeling is going to to be hard and getting the model to to to match the physical world is going to be all right so

[2:08:18](https://realalpha.app/v/ugvHCXCOmm4/?t=7698) he does have to interrupt yeah yeah yeah verify but but it's just you know you just look at even the simplest problems like i you know i think i talk about like you know the three-body problem or simple chaotic prediction like you know or or like predicting the economy it's really hard to predict the economy two years out like maybe the case is like you know normal you know humans can predict what's going to happen in the economy next quarter or they can't really do that maybe maybe a AI system that's, you know, a zillion times smarter, it can only predict it out a year or something instead of, instead of, you know, you have these kinds of exponential increase in computer and.

[2:08:54](https://realalpha.app/v/ugvHCXCOmm4/?t=7734) intelligence for linear increase in inability to predict. Same with, again, like, you know, biological molecules interacting. You don't know what's going to happen when you perturb a complex system. You can find simple parts in it. If you're smarter, you're better at finding these simple parts. And then I think human institutions. Human institutions are just, are really difficult. Like, Like it's, you know, it's been hard to get people, I won't give specific examples, but it's been hard to get people to adopt even the technologies that we've developed, even ones where the case for their efficacy is very, very strong.

[2:09:34](https://realalpha.app/v/ugvHCXCOmm4/?t=7774) You know, people have concerns. They think things are conspiracy theories. Like it's just been very difficult. It's also been very difficult to get, you know, very simple things through the regulatory system. Right. I think, you know, and, you know, I don't want to disparage anyone who, you know, you know, works in regulatory regulatory systems of any technology. There are hard tradeoffs they have to deal with. They have to save lives. But the system as a whole, I think, makes some obvious tradeoffs that are very far from maximizing human welfare.

[2:10:09](https://realalpha.app/v/ugvHCXCOmm4/?t=7809) And so if we bring AI systems into this, you know, into these human systems, often the level of intelligence may just not be the limiting factor, right? It just may be that it takes a long time to do something. Now, if the AI system circumvented all governments, if it just said, I'm dictator of the world and I'm going to do whatever, some of these things it could do. Again, the things having to do with complexity, I still think a lot of things would take a while.

[2:10:37](https://realalpha.app/v/ugvHCXCOmm4/?t=7837) I don't think it helps that the AI systems can produce a lot of energy or go to the moon. Some people in comments responded to the essay saying the AI system can produce a lot of energy in smarter AI systems. That's missing the point. That kind of cycle doesn't solve the key problems that I'm talking about here. So I think a bunch of people missed the point there. But even if it were completely unaligned and could get around all these human obstacles, it would have trouble.

[2:11:03](https://realalpha.app/v/ugvHCXCOmm4/?t=7863) But again, if you want this to be an AI system that doesn't take over the world, that doesn't destroy humanity, then basically, you know, it's going to need to follow basic human laws, right? You know, if we want to have an actually good world, like we're going to have to have an AI system that interacts with humans, not one that kind of creates its own legal system or disregards all the laws or all of that.

[2:11:26](https://realalpha.app/v/ugvHCXCOmm4/?t=7886) As inefficient as these processes are, we're going to have to deal with them because there needs to be some popular and democratic legitimacy in how these systems are rolled out. We can't have a small group of people who are developing these systems say this is what's best for everyone. I think it's wrong, and I think in practice it's not going to work anyway. So you put all those things together and we're not going to change the world and upload everyone