Fear of losing leverage is drawing talent into AI labs

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 7:01

   But you've also described coming across, and this is probably from a smaller subset of people, but some kind of nihilism about AI. So this kind of idea that it actually doesn't matter if technology serves humans at all. And I guess like a related idea that actually maybe we should be handing over the reins of the future to AI and away from humans. So in that sense, it's not just, you know, that it doesn't matter if technology doesn't serve humans. It might actually be a good thing if it's sort of not serving humanity, but serving AI as our successor. Can you tell us a bit more about those views and sort of how you see them influencing the AI scene these days?

2. 7:51

   for sure um yeah i'll share your caveat that um i would say that these what i call like successionist light views are more niche and definitely less common than the sort of previous set that I described. But I have been worried in hearing in just casual conversation, fairly prominent and influential people within the AI community express sort of an indifference or even a preference to a world where AIs are making most of the decisions in society rather than humans. I mean, one of them, these are, I won't disclose people's identities because they were Chatham House or sort of off the record contacts. But, you know, like I'd be having a conversation with somebody at a conference workshop and they would say, you know, like everyone at this conference is so worried about what's going to happen to humans in the post-AGI world. But frankly, I'm more worried, what if the

3. 8:42

   monkeys are still running the show? And in this case, the monkeys refers to the humans. He was more concerned that humans would still be in charge of governments, of corporations, of society, because he sort of had this view that AIs by in their super intelligent existence were more deserving and more important to preserve a future world for. Or I was, you know, talking to another very influential person in the AI scene and we were just on a walk and musing about, you know, like if all labor gets automated, what are people going to do? And he was like, maybe it'll be like Nozick's experience machine and we're all going to be, you know, know, wearing these VR headsets and watching perfectly personalized AI slop. And so we don't have any leverage in society, but in a way it's very pleasurable and people are very happy.

4. 9:28

   And I was like, I don't know, like, do you think that's actually the world we want? Like, we're sort of going back and forth on it a bit. And he's like, honestly, Jasmine, like, I'm a lot more interested in what the AIs are going to do with each other a thousand years from now than I am in what's going to happen to humans next year. And so in a way, I think that there are people, uh, influential and prominent people in the AI community who, whether for intellectual reasons or for like intellectual reasons, as in like a belief that like the more intelligent species is more deserving, or simply because it's a more interesting thought experiment have sort of arrived at an indifference towards whether humans stay in control.

5. 10:07

   Um, and I find that quite worrying. I don't think it's, you know, the majority belief or common belief within the AI labs or something like that. But I do think that it makes it very hard to have to move forward policy action, to move forward the safety conversation if we're not able to agree on even very foundational concepts such as humans should stay in control of our governments. And it is possible to decide what technologies we build and don't build. And our conversations should center what happens to people. And so I find that a little bit troubling. I think maybe I'm just like a jaded person. But when I hear you talking about people expressing these nihilistic beliefs or these deterministic beliefs, in my head, I'm just like, but surely these people like they don't genuinely believe what they're saying. You know, it's just like a bunch of tech bros who are doing

6. 11:05

   some kind of mixture of like posturing, like edgy humor, irony, that kind of thing. And I wonder if through this conversation we are paying too much attention to what's essentially just like an aesthetic choice that they're making rather than like a real set of beliefs. What do you make of that? I think it's fair. And I think that, you know, when I try to report, I try to have an eye for some of the hyperbole, like in the permanent underclass piece, like I try to make pretty clear that in a lot of ways it is an edgy joke. And most people do not literally believe in a permanent underclass, even if they believe in mass job displacement and increasing inequality, which I also think is somewhat concerning and worth talking about, even if it's expressed in more extreme ways. But in general, like I like paying attention to these cultural quirks and these memes

7. 11:57

   because I think they are a release. for things that you can't say or things that feel almost sometimes taboo or inappropriate. Like I would not have written that permanent underclass piece if I have not had, you know, lots and lots of conversations with young workers in Silicon Valley who have decided to try to get a job at an AI lab rather than doing a variety of other things specifically because they were specifically worried about being trapped in a permanent underclass. And maybe you could say, oh, they were actually just worried about having, being poor or being, having less money when AGI comes and returns to labor, stop working. But it's true that these memes, silly as they may sound, are shaping people's individual behavior. I mean, I've been thinking a lot specifically about this question of where people choose to work and brain drain in particular, because for example,

8. 12:50

   one of my close friends is a PhD student who studied NCS in AI at Berkeley. And she was telling me how like half of her cohort has decided to like graduate early within just a few years of beginning their PhDs in order to get jobs at AI labs. And they've actually curated their PhD dissertations for research that would be most likely to get them a job at an AI lab. I've seen in my own community, tons of people who were independent writers or independent policy thinkers and policy advocates decide to leave those roles behind in order to get jobs at AI labs.

9. 13:25

   And these views, sure, you could call it like a normal amount of people like money, but I think the scale of how much talent is flowing into a few specific labs really has to do with the fact that the AI community itself is promoting these widespread beliefs that if you do not have a stake, if you do not have literal equity in frontier AI, you are not going to have economic leverage in the future and you and your kids are going to be screwed.

10. 13:49

    And I've just heard in enough private conversations, these are motivating people to make such decisions. And it's clearly draining talent from the non-corporate independent ecosystem that I do start to take those memes somewhat seriously. So if you're trying to understand what the future of AI is going to look like and how