AI scaling runs into chip and power constraints

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:48:25

   So, there's this thing called, like, AI scaling laws. And these are kind of at the core of where we're at right now, geostrategically around this stuff. So what AI scaling laws say roughly is that bigger is better when it comes to intelligence. So if you make a bigger sort of AI model, a bigger artificial brain, and you train it with more computing power or more computational resources and with more data, the thing is going to get smarter and smarter and smarter as you scale those things together, right, roughly speaking.

2. 1:48:51

   Now, if you want to keep scaling, it's not like it keeps going up if you double the amount of computing power that the thing gets twice as smart. Instead, what happens is if you want – it goes in like orders of magnitude. So if you want to make it another kind of increment smarter, you've got a 10x. You've got to increase by a factor of 10 the amount of compute. And then a factor of 10 again. So now you're a factor of 100.

3. 1:49:11

   And then 10 again. So if you look at the amount of compute that's been used to train these systems over time, it's this like exponential, explosive exponential that just keeps going like higher and higher and higher and steepens and steepens like 10x every – I think it's about every two years now. You 10x the amount of compute. Now, you can only do that so many times until your data center is like a $100 billion, a trillion dollar, $10 trillion.

4. 1:49:38

   Like every year, you're kind of doing that. So right now, if you look at the clusters like the ones that Elon is building, the ones that Sam is building, Memphis and Texas, these facilities are hitting the $100 billion scale. Like we're kind of in that there are tens of billions of dollars actually. Looking at 2027, you're kind of more in that space, right? So you can only do 10x so many more times until you run out of money, but more importantly, you run out of chips.

5. 1:50:09

   Like literally TSMC cannot pump out those chips fast enough to keep up with this insane growth. And one consequence of that is that you essentially have this gridlock, like new supply chain choke points show up, and you're like, suddenly I don't have enough chips or I run out of power. That's the thing that's happening on the U.S. energy grid right now. We're literally like we're running out of one, two gigawatt places where we can plant a data center.

6. 1:50:37

   That's the thing people are fighting over. It's one of the reasons why energy deregulation is a really important pillar of U.S. competitiveness. So this is actually this is actually something we we found when we were when we're working on this investigation. One of the things that adversaries do is they actually will fund protest groups against energy infrastructure projects just to slow down, just to like just to tie them up in litigation, just to tie them up in litigation.

7. 1:51:06

   Exactly. And like it was actually remarkable. We talked to some state cabinet officials, so in various U.S. states, and they're basically saying, like, yep, we're actually tracking the fact that, as far as we can tell, every single environmental or whatever protest group against an energy project has funding that can be traced back to nation-state adversaries who are- They don't know, right? They don't know about it.

8. 1:51:32

   So they're not doing it intentionally. Right. They're not like, oh, we're trying to- No. You just imagine, like, oh, we've got, like, there's a millionaire backer who cares about the environment. He's giving us a lot of money. Great. Fantastic. But sitting behind that dude in the shadows is, like, the usual suspects. Wow. And it's what you would do, right? I mean, if you're trying to tie up the U.S.

9. 1:51:50

   If you're just trying to fuck with us. Yeah. Like, just go for it. You were just advocating fucking with them. So, of course, they're going to fuck with us. That's right. That's it. What a weird world we're living in. Yeah. But you can also see. See how a lot of this is still us like getting in our own way, right? We could, if we had the will, we could go like, okay, so for certain types of energy projects, for data center projects and some carve out categories, we're actually going to put bounds around how much delay you can create by lawfare and by other stuff.

10. 1:52:20

    And that allows things to move forward while still allowing the legitimate concerns of the population for projects like this in the backyard to have their say. But there's a national security element that needs to be injected into this somewhere, and it's all part of the rule set that we have and are, like, tying an arm behind our back, basically. So what would deregulation look like? How would that be mapped out?

11. 1:52:46

    There's a lot of low-hanging fruit for that. What are the big ones? Yeah. So right now, I mean, there are all kinds of things around, it gets in the weeds pretty quickly, but like there are all kinds of things around if you're going to, so carbon emissions is a big thing, right? So, yes, data centers, no question, put out, like, have massive carbon footprints. That's definitely a thing. The question is, like, are you really going to bottleneck builds because of that?

12. 1:53:14

    And, like, are you going to come out with exemptions for, you know, like, NEPA exemptions for all these kinds of things? Do you think a lot of this green energy shit is being funded by other countries to try to slow down our energy? Yeah. It's a dimension that was flagged, actually, in the context of what Ed was talking about. That's that's one of the arguments that's being made. And to be clear, though, like the this is also how like adversaries operate is is like not necessarily in like creating something out of nothing because that's hard to do.

13. 1:53:44

    And it's got it's like fake. Right. Instead, it's like there's a legitimate concern. So a lot of the stuff around the environment and around like like totally legitimate concerns, like I don't want my backyard waters to be polluted. I don't want my kids to get cancer from whatever, like totally legitimate concerns. So what they do, it's like we talked about, like you're like waving that rowboat back and forth.

14. 1:54:04

    They identify the nascent concerns that are genuine and grassroots, and they just go like this, this, and this, amplify. Well, that would make sense why they amplify carbon above all these other things. You think about the amount of particulates in the atmosphere, pollution, polluting the rivers, polluting the ocean. That doesn't seem to get a lot of traction. Carbon does. And when you go carbon zero, you put a giant monkey wrench into the gears of society.

15. 1:54:31

    And one of the tells is also like – so nuclear would be kind of the ideal energy source, especially modern power plants like the Gen 3 or Gen 4 stuff, which have very low meltdown risk, safe by default, all that stuff. And yet these groups are like coming out against this. It's like perfect, clean, green power. What's going on, guys? And it's because, again, not 100% of the time. You can't really say that because it's so fuzzy and around the edges.

16. 1:55:00

    A lot of his idealistic people looking for a utopia and they get co-opted by nation states. And not even co-opted. They're fully sincere. Yeah, just amplified. Just fund it. Because it's convenient. Amplified in a preposterous way. That's it. And then Al Gore gets at the helm of it. And then that little girl, that how dare you girl. What? Oh. How dare you take my child away from you? How dare you?

17. 1:55:20

    Yeah, it's wonderful. It's a wonderful thing to watch play out because it just capitalizes on all these human vulnerabilities. Yeah, and one of the big things that you can do too as a quick win is just impose limits on how much time these things can be allowed to be tied up in litigation. So impose time limits on that process just to say, look, I get it. We're going to have this conversation, but this conversation has a clock on it.

18. 1:55:46

    Because, you know, we're talking to this one data center company, and what they were saying, we were asking, like, look, what are the timelines when you think about bringing new power, like new natural gas plants online? And they're like, well, those are like five to seven years. And then you go, okay, well, like, how long, and that's, by the way, that's probably way too long to be relevant in the superintelligence context.

19. 1:56:07

    And so you're like, okay, well, how long if all the regulations were waived? If this was, like, a national security imperative and whatever authorities, you know, Defense Production Act, whatever, like, was in your favor? And they're like, oh, I mean, it's actually just, like, a two-year build. Like, that's what it is. So you're tripling the build time. We're getting in our own way, like, every which way, every which way.