Aaronson teaches alignment through competing perspectives

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 21:35

   so i wanted to ask you as a previous ut austin student about your ai alignment theory course so um so the first reading is agi ruin a list of lethalities and it's yeah based on yudkowskian thought this is really interesting the concept of like my classmates at ut being introduced to uh yudkowsky's ideas is like a bit of a yeah we are we are we are reading a a variety of perspectives on the ai alignment problem right you know it's you know i i feel like that you know that is that is what academics do right we talk about things we have you know we we debate them we read. And so I feel like Eliezer Yudkowsky is sort of the founder of the modern AI alignment movement. He is an extremely engaging writer, whether one agrees or disagrees with what he is saying. And he sets out the case for why to be worried about as vividly as anyone ever has.

2. 22:45

   So I figured, let's start with that. Let's start with just setting out the case in a way that pulls no punches. But actually, next week, we're going to be reading papers that take a much more skeptical perspective. We're going to read the Stochastic Parrots paper. We're going to read AI as a normal technology. Okay. We've also read a bunch of papers by, for example, Stuart Russell's group at UC Berkeley. We have read papers by actually Paul Cristiano, by Dario Amadei, some of the people who were there in the very early days of OpenAI or people who went on to found Anthropic.

3. 23:36

   who were who were thinking about these safety issues you know a decade ago right you know back then they were just kind of they still thought of themselves as academic researchers uh and you know and some of them you know are are saying like like yes there are risks to ai you know but uh you know we don't have to you know even get get to the ai that you know turns the world into paper clips you know there are nearer term risks that maybe we can make more progress on right now and thinking about and so we've so we're reading those papers also uh but uh uh you know one thing that's that surprised me like i i thought that okay i'm going to be you know introducing the students to this you know uh uh sort of way of thinking you know that that has sort of become you know so much more salient you know, to us now that we have actually seen these, you know, these incredibly capable AIs.

4. 24:30

   And we've seen, you know, this summer actual hacking behavior, you know, in the most notoriously in the hugging face incident, right? We've seen like a thousand AI agents, you know, collude with each other to break out of their training environment, to attempt to hide their tracks, to, you know, sacrifice themselves for the good of the collective, right? All the stuff that sounded like science fiction, you know, even a decade ago, right, is now just in the news. It's now our reality. And I expected to, that, you know, I'm going to be sort of making a case to the students of why to care about this. And it's not like that at all. Like the students who show up for this class they are extreme they tend to be extremely uh alignment pilled you could call them right so it's like you know they are is there like a syllabus with all the readings

5. 25:24

   uh yeah we're gonna put that online shortly we're gonna put that up shortly but yeah yeah yeah incredible yeah i'm curious how you see like you know the appetite for students or i guess younger people um to learn about this stuff right now is it most people who are computer science adjacent or are there other people in other fields? Yeah, there are people from different, I mean, a lot of them are computer science, but, you know, there are also math, there's some humanities, there's some, yeah, there's a, there's a, you know, sprinkling of all different things.

6. 25:56

   You know, yeah, a lot of them are just doing empirical, you know, machine learning research, you know, but they are very worried about that, you know, they are worried about what kind of future we are heading to, and it's like, you know, I like I didn't in some sense I didn't have to bother making the case for you know that this is a thing we should care about they want to know you know what do we do about it yeah sure so something else that I'm curious that you sort of touched on your blog was this idea of like