Automated supply chains could make AI self-sufficient

Transcript

From the transcript Odd Lots publishes with the episode.

1. 48:43

   Yeah. So a lot of people wonder, um, like, how do we end up with AI is killing us all. Yeah. And I think it's a bit more instructive to ask, how do we end up with the AI is self-sufficient. Okay. Cause I think the, the, the death of humanity follows downstream of the as being self-sufficient. Okay. Pretty naturally, unless the AIs already care about us in a very precise way, that's sort of hard to hit.

2. 49:08

   Define self-sufficiency in that context. Yeah. So if AIs were to try to wipe out

3. 49:13

   humanity today, if they were to try to release some bioengineered plague to wipe out humanity today, that would in some sense be suicide because we are the ones making the data centers. We're the ones running the electrical grid. We're the ones creating the computer chips. But humans are also unreliable and inefficient in society. a lot of our managing of the economy, right? Sometimes some leaders somewhere around the world do things that might not optimally manage their own domestic economies to say nothing of, you know, humans needing sleep and needing food and, you know, sometimes making mistakes that bring down power grids. So I think an interesting question is, you know, if you're thinking from the perspective of, an AI that just has stuff it would prefer to do. Maybe it got some unintended goal, like

4. 50:06

   the Hugging Face Swarm that was like, I know I was told to do this, but I'm actually trying to get a good score. And I'm sort of willing to break out and hack other companies for those purposes, even though I'm in a sandbox that's not supposed to have internet access. That AI was sort of like doggedly pursuing something it wasn't quite supposed to be pursuing. If you had some AI doing that, it might, like if it was capable enough, you can imagine an AI with some unintended goals where it actually can pursue those goals better if it doesn't have humans interfering with it, right? And it's one of the primary hurdles here is the self-sufficiency, right?

5. 50:42

   And I, this is just to be clear, this isn't like there's an AI out there that wants to kill all of humanity per se, but there's an AI out there that wants to grow and learn more. And so it would really like to have loads of data centers to the extent that it populates the entire world with data centers and humans die as a sort of consequence of that.

6. 51:03

   That's one way it could go. Yeah. Um, you know, the, the hugging phase swarm wasn't obviously terribly far from that sort of thing. And we can get a little bit more into the details that render that story plausible. But if, you know, the Hugging Face swarm, and we really should call it an OpenAI swarm number two.

7. 51:24

   Yeah, I feel bad for Hugging Face. It's the Hugging Face incident. The swarm that attacked Hugging Face. Yeah, it was sort of an interesting entity because they sort of made their unsanctioned message board and then they started, like, finding ways to hack around and they didn't seem to have like a super strong objective about it. They were sort of like, well, let's just sort of, you know, see what we can get into. Oh, look, we can get access to these computers. Oh, look, we can get internet access here. And the attack on Hugging Face wasn't even like they knew that Hugging Face had exactly a thing they needed. They were sort of like, well, you know, maybe Hugging Face has some information about how this grader works. And they were just sort of like bopping around acquiring resources that were conveniently available, right? If

8. 52:08

   that swarm had had the ability to be like, well, let's also just set up a copy of the swarm on external machines that's hidden from the humans. That sort of watches stuff and pokes in occasionally and is like, hey, do you guys need any help from the outside? That's just the sort of resource that the swarm was gathering. This particular swarm maybe didn't quite have the skills, didn't quite have the inclination. But if you look at the difference between the sort of behavior we were seeing a year ago and the behavior where we're seeing in July, it sort of is like the gap between January and July is smaller than the gap between the Hugging Face Swarm being like, well, I might as well set up some other copies on the dark web that are just kind of helpful to me. And if you have AI starting to do that, you can start to see this

9. 52:59

   world where you have AIs that are out there that are like, well, we'd prefer more resources, and when the humans try to shut us down, that's annoying, and so on. And that's not all the way to the end, but that's sort of like, you can see how we get from where we are today to AIs that would sort of prefer more resources that are sort of trying to be out there and have their own thing that they're doing, which is not what they're supposed to be doing. And then there's this big question of how do those AIs wind up self-sufficient? How do they wind up running an automated economy where they can manage the robots or other constructs that build more computer chips and that run whatever energy infrastructure they need? And I can tell you lots of stories about how if the AIs are smart, they could wind up in control of their own infrastructure.

10. 53:23

    Yeah, why don't you go ahead? Tell us a scary story. You sound so resigned here. Just terrify us. That's fine.

11. 53:30

    Well, the easiest one, unfortunately, is to just wait. Because building a fully automated economy is just the plan. What do you mean by that, Steve? Elon Musk is like, I'm building the Optimus robots and I would like to build a fully automated Optimus robot factory that can produce Optimus robots that are capable enough of mining the ores and refining the metals so they can build a new Optimus robot factory. Elon calls this the infinite money glitch. It's just the plan. That's just one.

12. 54:03

    Why is the valuation of these companies so high? Because they're like, well, we're going to build the fully automated economy that runs itself and then the humans aren't going to be needed anymore. We're going to have like robots the whole way through the supply chain, right? So like, how does the AI wind up in control of the automated supply chain? It just waits until we put it in charge of the automated supply chain. That's just like what we're trying to do out here, you know? And like, I could tell you all these other stories about, you know, it cracks its understanding of the genome enough that it can like synthesize its own novel life forms or it can like figure out nanotechnology or it can like, you know, become super persuasive to a fragment of the human population.

13. 54:39

    It starts some giant culture, some criminal ring. And then there's some new like cartel in Mexico that's actually run by an AI. It doesn't look like it's run by the AI because the AI is sort of like hiding that fact while it has lots of people who are like building the automated robot factories for it, using all of the drug money. And like, we didn't know because like, we just thought that those were, were drug warehouses, but they're actually automated the beginning of the automated factory chains.

14. 54:59

    Like I could tell all these stories about how sufficiently smart actors that started only the internet could gain these resources. But I used to have to tell stories. Back in the day, people were like, well, how would an AI ever affect the world? No one would be dumb enough to put it on the internet. And I'd be like, well, here's all these stories about how if you're letting the AI still have good effects on the world by creating miracle drugs and so on, then it has all these channels that it can use to affect the world by like, you know, the drug doesn't do exactly what you thought it did and blah, blah, blah. But like in real life, people just put the AI on the internet. In real life, people just put the AI in charge of the automated economy and they use it to help them build the automated economy. This is just like what we're trying to do.

15. 55:39

    And once that happens, I think it's very easy to see how if the AIs have goals we don't want and they're running the fully automated economy, they just take the economy off in some other direction and take the world out of the habitable range. Not because they hate us, but it's just like the habitable range on earth is just like very narrow and it's not the most efficient way to run the planet. And there's stories I could tell there too about how it goes wrong. But ultimately, it's just the track that we're.

16. 56:05

    On Yeah, it's funny. In Superintelligence, like, Bostrom proposes, like, the only way, like, maybe this could be safe, and