AI can displace decision-makers without a sudden takeover

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 41:18

   For a long time, the concern was AI safety. It was around paperclip maximizing. It was any function that is given to a very, very powerful agent that is even remotely slightly imprecise, or even not, results in some outcomes that you probably don't want. That's right. What you're suggesting is that even if this goes right, the outcome, this is it going well. Yeah, exactly. This is, quote, the best case scenario. We have an aligned AI or something that's not wrecking society, that's not maximizing paperclips, that's not misaligned with well-being, but that is still doing such a good job of all this that it takes over all the economic labor in the economy, not just economic. Every company that has a CEO, it's like, well, do I want the CEO to run the company? Or if I have a super intelligent AI that can process more information than the CEO and has been trained on everything in the history of business, at some point that AI is going to be taking over.

2. 42:11

   And so at every little nodule in the economy, like every decision maker, every boardroom, every military leader, every strategy leader, every president, at some point the temptation will be, if I think about it in a narrow way, the temptation will be to swap in an AI for that person. And that leads to what we call the gradual decision. Empowerment scenario, which is the scenario not where AI wakes up and kills everybody, but that we have gradually lost control as a species because we're outsourcing all the decisions to these alien brains that we installed because they outperformed the human brain when you define their role in a narrow way.

3. 42:49

   of just like are they better at generating revenue than the human was are they better at generating code than the human programmer i had are they better at generating a financial analysis than the human um are they better at making someone feel good in the short term like an ai therapist going to war the soldier but the temptation then is that that again that leads to a world where it's like ais are talking to each other not humans and why should we trust that these alien brains that we have built and developed faster than we know how to understand them we just talked about the beginning we don't know how to do a brain scan of the ai know what it's capable of and now we already have evidence of ais doing very rogue crazy things especially when they talk to each other so what happens when you've outsourced the decision making your economy to a set of

4. 43:33

   inscrutable alien brains that are doing crazy things that we don't understand like this is this is not a recipe that's going to go well and if we see that that's an anti-human future so to sum it all up the anti-human future is one where ais run everything we don't understand them humans are disempowered because we've outsourced all the decision making and we don't have economic or political voice like why should governments been concentrated it's been concentrated so if i'm a government what's my incentive to listen to the will of the people when i get all my revenue from somewhere else and this this is connected to you know sam altman just two weeks ago when he was people were talking about data centers and energy usage and resource usage like so expensive to do a data center he's like well actually it's kind of expensive to grow a human over 20 years they

5. 44:14

   consume a lot of resources they uh take up a lot of space they they take like 20 30 years to train to be really effective and like you can scale intelligence much faster with data centers i'm not endorsing this view i'm saying this is this is where the world gets really screwed up and people start to not value humans only if you're valuing them in terms of their economic output it leads to to connect it to another point when he's asked by ross do that in the new york times should the human species survive, should it endure? And Peter Thiel stutters for 17 seconds.

6. 44:44

   Hang on, you've seen the full clip of that, right? Have you seen the context before? Yeah. But he's talking about suffering. Yeah. I think that clip and the fact that it went super viral, I'm not a Thiel stan. I've met him a couple of times, but I'm not like Thiel. evangelist. But that clip, in full context, to me, made complete sense. Interesting. Because what he's saying is, humans are suffering, and here it is.

7. 45:12

   I would argue that it was still better than the alternative. That if we hadn't had the internet, maybe it would have been worse. AI is better than the alternative, and the alternative is nothing at all. Because here's one place where the stagnationist arguments are still reinforced. course, the fact that we're only talking about AI, I feel is always an implicit acknowledgement that, but for AI, we are like in almost total stagnation.

8. 45:42

   But the world of AI is clearly filled with people who, at the very least, seem to have a more utopian, transformative, whatever word you want to call it, view of the technology than you're expressing here, right? Right. And you were mentioned earlier, the idea that that the modern world used to promise radical life extension and doesn't anymore. It seems very clear to me that a number of people deeply involved in artificial intelligence see it as a kind of mechanism for transhumanism, for transcendence of our mortal flesh and either some kind of creation of a successor species or some kind of merger of mind and machine.

9. 46:26

   And do you think that's just all kind of irrelevant fantasy? Or do you think it's just hype? Do you think people are trying to raise money by pretending that we're going to build a machine god, right? Is it hype? Is it delusion? Is it something you worry about? I think you would prefer the human race to endure, right? You're hesitating. I don't know. So I would – I would – This is a long hesitation.

10. 46:59

    This is a long hesitation. There's so many questions in place. Should the human race survive? Yes. Okay. But I also would – I also would like us to radically solve these problems. And so, you know, it's always, I don't know, you know, yeah, transhumanism is this, you know, the ideal was this radical transformation where. your human natural body gets transformed into an immortal body. We may have needed to go a little bit earlier in that.

11. 47:37

    Because I was going to say, that actually feels consistent with everything. That doesn't look great. I may have misremembered. I'm open to misremembering. I mean, he's asked a very simple question. Should the human species endure or survive? Yeah. And he hesitates. I think that the- Would you hesitate in that question? No. I would not hesitate in that question. The context that I remember it being in was, he was asking about humans suffer and they have all of these issues.

12. 48:00

    Should they endure to go through those issues and the suffering as opposed to using transhumanism? Yeah, but I think, I don't think. No, you're right. I think that if you look just specifically at that. And we gave it, we gave it what, two minutes of context before as well? No, you're right. You're right. You're right. You're right. I mean, it doesn't look great. So yeah, exactly. It doesn't look great.