Two or three AI projects could balance safety and power

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:04:43

   Yeah, maybe we should talk then about some kind of contentious, politically salient questions that your disagreements might bear a bit more strongly on. sure um one idea we could talk about here is you know there are some people who think it would be a good idea to um have one big agi or super intelligence project um and you know that way it's kind of maybe easier to impose some safety infrastructure on just one project uh maybe it buys the ability to pause late like as needed um without like losing out to a competitor or something like that i would guess that this sort of thing sounds somewhat worrying to tom given that having one big project um probably does increase the risks of concentrating power um in the hands of a few people i'm curious firstly tom if that's correct and secondly Katya, if you also feel similarly worried about having one big project?

2. 1:05:53

   Yeah, I think, as you say, there's a single point of failure if there's a single project. My understanding is that it might be quite hard to set up a single project within the United States that couldn't be co-opted by the president. I really haven't looked into this, but in general, it's surprisingly hard to create government institutions that are not within the executive branch and it's at the moment quite hard to stop the president from gaining control of institutions that lie within the executive branch so i think there is a real challenge a direct challenge to creating a single project and then you know even if it's not the president there would be some you know some small group that's running that project that would pose a distinct risk and would have no other rivals um i also just disagree that it's going to be that

3. 1:06:38

   hard to coordinate multiple projects to slow down like it's not that hard to tell if projects are training powerful ai it's not that hard to just tell them that's illegal um i think it's you know totally totally feasible yeah i think i agree with that that it you know the difficulty of coordinating multiple projects seems pretty overblown um and i guess yeah i'd say overall i'm uh also decently worried about one big project like is it better or worse than the current state of affairs not sure i think it seems worse in that i i kind of expect this one project to then have quite a lot of power itself and to be like entirely full of people who really really want to build ai soon um and so it's not to actually be a very conducive um environment for a pause coming about uh unless it's like you're imposed from outside and the outside imposing seems like

4. 1:07:39

   it might be easier with numerous less powerful, less coordinated projects. Whereas it seems like the way in which it might be easier is if they're trying to coordinate it themselves. And I guess in terms of how humans operate, I less buy that they would coordinate it themselves once you have That entity, my preferred number of projects would be zero. Yeah, I previously said my preferred number of projects, two or three, because I think you get a lot of benefit of power distribution from just a kind of a fairly small number of projects.

5. 1:08:16

   I think once you're introducing 10 projects, it might get a bit harder to coordinate and you might have some more difficulties with kind of races to the bottom. Although I still think you could do it for sure. um but yeah i think i think you get a lot of benefits with just a second project because you can have the ais from each project check on it put the ais from the other project which decreases misalignment risk and also decreases the risk of secret loyalties um where human purposely makes ai secretly loyal to them and yeah you could just argue that um if there's a single project then um ai doesn't even have to bother colluding it's now all the super intelligences are like naturally, you know, of one ilk. And so maybe the AI takeover challenge