Safety work has different status inside and outside AI labs

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 22:13

   like how does that actually affect what's going on in the labs? Yeah, I would say again, to me, a huge part of this is the talent flows question. Just because talent is so scarce, as we see from the salaries that people are willing to pay, talent is incredibly scarce. And so when you have these cultural beliefs, the main thing that they influence is actually really what young people decide to do with their lives and how many people decide to say work on safety versus working on capabilities. One thing that I've noticed rise over the past few years, even outside of AI, but in Silicon Valley broadly is the rise of vice signaling. This idea that actually the edgy and cool thing to do is to build a startup like Mechanize where the tagline is that we're going to automate and replace all human labor or to build something like Cluely where the tagline

2. 23:02

   is cheat on everything. I think that the idea that trying to be careful and trying to be careful, trying to be constrained about the way that you build technology by labeling that as Doomer-y or low status or like as not being sort of a pioneer and a serious builder, you're basically preventing a bunch of very impressionable young people from going into a field that they otherwise would. And that's to say nothing of obviously the policy conversation in DC where Doomer has been employed as an extremely effective insult. I think one important thing actually also about the Doomer insult is that it's coming mostly from the tech right that has pretty effectively tied Doomerism and AI safety to, quote unquote, like woke identity politics and previous attempts to regulate technology by alleging discrimination, civil rights concerns, labor concerns, et cetera.

3. 23:53

   And so the folks like Marc Andreessen and David Sacks were using the Doomer label most loosely. building this perception on the American right in particular, that to be interested in AI safety is the same as being a woke regulator who wants Facebook to like censor all non-liberal content. I think what I'm interested in here is that, I mean, at 80,000 hours, right, we see a lot of people who are very attracted to careers in like the Bay Area, at AI companies and the general kind kind of like AI ecosystem, specifically because they like really genuinely want to positively influence the trajectory of AI development. I wonder in this environment, like how much you think, like how optimistic you are that safety focused people can actually succeed at pushing forward what they want to push forward? I think that folks who are working on technical safety

4. 24:57

   work, there are still a lot of opportunities and a lot of support, both like socially, financially, et cetera, professionally, professional ecosystems towards doing that kind of work. I think that there is a well-developed enough ecosystem of AI safety organizations. There's the fact that actually most people who work on capabilities at the labs really care that their models are aligned. It's also very hard to sell unaligned models. It sort of coheres with the business incentives of these companies that like you can get made fun of a little bit by like Mark Andreessen in order to do work that you think is really important, really well respected by a lot of other technical people in your community. Like one thing that I sometimes tell folks who are not an AI is like, you know, what's really notable is actually

5. 25:38

   the, a lot of the, a lot of the researchers who have made the most, the biggest breakthroughs in AI, people like, you know, Jeff Hinton are some of the ones who are raising the alarm the most on AI safety being an important thing to work on. And I think that affords working on technical safety quite a lot of status, at least within those technical research communities. I think where I start to worry a little bit more is on types of work outside of that core safety work, which is the technical safety work, which is more well-funded. Again, maybe it's just because I'm a journalist, but I think a lot about how much incentive there is for people to go into these fields that are where, one, I don't think that you can do them inside a lab. So I don't think that you can do especially credible either journalistic work or policy advocacy work if you

6. 26:30

   are employed by a lab. You are going to be seen as a shill. You are sort of correctly seen as a shill. You are basically doing marketing and lobbying for that company. And so if you completely 100% align with that company's overall position, then like, okay, great, go do that work because you'll help your firm win. But for the most part, I really wonder and worry about how many positions there are for very AGI-pilled and very safety-concerned people to work in these sort of independent roles. Even the journalism community has sort of appropriated these words to make fun of people who believe in things like existential risk. In the journalism and media world, people will make fun of you if you are seen as an effective altruist, if you are seen as someone who believes in existential risk, if you were seen as, quote unquote, buying into the Doomer hype,

7. 27:17

   like you mentioned, of these AI companies. And so it's actually like kind of low status in journalism to believe in these things. It's low status in the policy community. I mean, Politico, I think last year or the year before, sort of was running these stories, attempting to like weed out and witch hunt anyone who has any connection to effective altruist fellowships within the AI policy community so that people would then go and discredit them. I've had, again, senior media people tell me very similar things where they're going after all the Tarbell fellows and saying these people are just anthropic shills or these people are shills for like some crazy Silicon Valley billionaires. And so especially because these are the roles that I already think are less supported, like they are not as well remunerated as roles at the lab.

8. 28:00

   you don't have this like social and professional ecosystem. I tend to worry more so about people who are in academia and research and policy and in journalism who take AI really seriously and