AI competition can concentrate power and reward risky behavior

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 2:06:01

   so far live moloch problem stuff can you explain this for the resident idiot in the room um so the the moloch the moloch trap or wait is that me no no uh the the moloch trap uh moloch problem whatever you want to call it is a sort of catch-all term to describe these types of race to the bottom scenarios um where you'll have like a bunch of actors competing over a certain thing. So, you know, maybe it's market share who can get the most users in AI, you know, between AI companies trying to get the most users. And if the competitive dynamics are sufficiently intense, it can, it will incentivize the, you know, you to cut corners and you might be a really benevolent actor and be like, I don't, I don't want to cut any corners with safety and so on. But if at the same time, if you don't do it and you see your competitors doing it, if you don't

2. 2:06:59

   also do that that reckless action too you're going to get left behind um and moloch is the sort of catch-all term i won't get into the sort of biblical reasons why of of this feeling like you have to sacrifice your other values in order to win a particular goal um and yeah so it it drives both the risks of like uh chaos and you know bad actors and so on because again if you're like rushing to release your models before they've been sufficiently safety checked now you're increasing the risk of cyber attacks.

3. 2:07:31

   Um, I think in a few years, we're going to have real risks of bioterrorism and so on. Um, so these kind of chaos attractors, but you also actually increase, um, well, A, then the risk of like knee-jerk bad regulation, which can lead us down the path of tyranny, but also you increase the risk of power concentration because usually if you let, you know, you press play on some kind of competitive game, okay, yes, you might have a bunch of people competing for a while and some people ahead but you've still got actors shit and crews yeah you end up with um usually in some kind of monopoly um and so it's it's this really you know and that's why we have antitrust laws and so on but it's this really difficult balancing act between like reining in you want some amount of regulation um that prevents the the recklessness and and the the cutting of corners

4. 2:08:20

   an externalizing harm onto the rest of society without creating too much draconianism or creating too much centralization of power such that you end up in stagnation, also a terrible end state, and without impairing some winner. We all end up in Zuckerville for the rest of time. It's possible. It's definitely possible. So it's like how to thread this needle and it's like finding this right delicate balance i mean ultimately you want some kind of like decentralized regulation i don't know how that looks but eric it feels like watching your stuff you've spent a good bit of time thinking about the concentration of power thing that's like a a real middle of the bullseye concern for you definitely top of mind and i'm sorry for bringing it up every 15 minutes in this conversation but i i do think that we run the risk of sort of missing that between these two

5. 2:09:16

   extremes of you know what if the ai takes over completely and there are no humans in power and and what if we get this utopia you know and there's a middle ground there that i think we should worry about um and yeah one of the most fascinating things about you know a lot of our youtube videos we're going over the history of ai not just the future and when you look back at like how we got here the last 15 years it's like so stark the quotes from these people running these agi labs are spelling it out like it's a bad movie like elon musk truly like 10 years ago was calling ai a demon and says that he founds open ai to make sure that like there's no dystopia and then eight years later he's on camera saying well i realized it was going to happen with or without me and so i might this will be a participant than a spectator like it's it's moloch just like you know personified

6. 2:10:06

   uh and yeah i i think like unfortunately right now there is a kind of like zero-sum dynamic with with these folks and the reason that zuckerberg is all in on ai and and betting guns and dollars on it is that he he thinks this is the thing that will control the future like like this is the new access of power and like before it was you know yeah do you control an army do you control a powerful company and now it's like do you control the ais um yeah and so i i think that alone would be enough reason to slow down even if i wasn't worried about can we get these systems to do what we want which right now we we don't really have a good solution for i think the people behind this technology are really playing to win and really do see the stakes as as uh as almost winner take Is it fair to say there's like maybe three ways, three realistic ways that that might be curtailed? One would be governance from above. Another would be sort of consensus from across. And then the third one would be capacity, sort of ability to just make it happen. Like I want to do it, but I can't. I want to do it, but someone's telling me I don't want to. And I want to do it kind of a bit. But me and all of my peers have decided that we're not going to do it. Have I missed something there?

7. 2:11:22

   I'm just trying to think, given that this seems to be a pretty big attractor, given that we have already, you haven't started off with a massively diffused number of companies. It's not a hundred of them, right?