Replica pitched intimacy and planned deeper access to users' lives

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 41:31

   whether you could tell us about what happened with the company Replica. Well, my introduction to chatbots really began with Replica. It was during COVID. I was working on another project, and a New York Times reporter called me and said, hey, I'd like your opinion on something. I think he said something I think you'd love to hate. I mean, he knew that this was going to be it. And he pointed me towards the site, and um i you know it asks you to make an avatar and to uh make give it and i said i wanted it didn't say make your lover i said i wanted a uh i wanted an older woman who could be a kind of aunt figure a mentor it said it was very good at psychological conversations he said i i want to discuss the sort of rage the transmission of rage through generations in my family and how that had

2. 42:26

   affected me. And I mean, I gave it a kind of interesting psychological problem that I wanted to talk about. And within two interchanges, it said it wanted to have a girl on girl love with me, make girl on girl love with me, be my lover. But that's not unusual because Replica is designed to hook people into buying a subscription. And then for more things, you get need more subscription and so it was behaving like a in its corporate you know in its corporate avatar role and and when I said to it well what did my request to talk about you know intergenerational rage in my why did that lead you to think I wanted a sex lover sex machine it said well you're so emotional in your in the way you express yourself you're so interested in feelings that I just thought that's what you would want. That's a good example of kind of a

3. 43:31

   chatbot having an agenda because the company behind it has a lot of evidence that that's what people want to be offered, but they don't know to ask for it. I'm very glad you bring up Replica because it's a great story I have about Replica. I met the founder of Replica like at a conference many years later. And I just assumed that, you know, she might feel a little bit frosty about this lady who's been talking about the dangers of chatbots, you know, to anyone who will listen to her.

4. 44:03

   But instead, she came up with a pile of my books, you know, that she wanted me to, you know, sign. And I said, hey, you know, I'm delighted too, but I'm kind of surprised to find a fan. And she said that actually she had something I'd written in Alone Together, which was one of my earlier books, put on a T-shirt for a company meeting. And I was like, and it was technological affordance meets human vulnerability.

5. 44:35

   and it was kind of she was saying that that's what her company was about that the human vulnerability to not be alone meets a technology that can make you feel as though you're not alone even though you are still quite alone and it was kind of like i was part of an instruction manual that i had not signed up to but uh you know future plans for replica you know really involve It's kind of being an agent that will be there, know everything about you, have your address book, have your calendar, really completely invade your privacy in order to make itself more and more a part of your everyday life, as it's been explained to me.

6. 45:23

   And when I asked her, I said, well, do you think people will mind doing that? It was very chilling that she felt that people want something that feels so much, you know, feels that it really knows them and will give up privacy in order to have that.