Public actuaries could put a price on catastrophic AI risk

Transcript

From the transcript Odd Lots publishes with the episode.

1. 45:15

   This is a slight tangent, but I think it's relevant to the conversation. I'd be curious for your thoughts. But when we talk about minimizing or disincentivizing risky behavior, often we talk about pricing that risky behavior. And so you could imagine, for instance, a world in which insurers say if you're going to be running these models you're going to have to pay us enormous premiums because the risk is that it runs amok and hacks into a company and i don't know takes down an entire server with everyone's bank data on it or something like that and that would act as like a natural market incentive for companies to start taking these risks more seriously and Do you ever talk to insurers or do you have like any handle on the actual financial penalties attached to some of this?

2. 46:04

   I have recommended liability for AI companies for a while. I think that could help a bit. It's a little bit tricky for the, then it wipes out all of humanity. True catastrophe risk insurance. But it's not, it's not, I mean, you can't really think of it from an insurance point of view because there'd be no one to collect or to pay out when we're all dead. But you could, I mean, in theory, that doesn't stop you from having actuaries that work for the public whose job is to somehow assess these risks in a sensible way. And then, you know, you could sort of look at the, you know, how much inconvenience does someone usually take on to avoid what degree of risk?

3. 46:40

   You know, you can sort of look at people's preference for traveling on different types of transportation that are different levels of risky and how they trade off the risk against the time. And you can get some sense of like, how much do people tend to like, how much do people pay to avoid, you know, being killed? And then you could try and factor in some, you know, extra cost for wiping out everybody all at once, which then also destroys all future generations. And you could take some actuaries and be like, hey, you know, find some numbers here, like find some numbers on the risk, do some numbers on the costs. And, you know, you'd have to weigh that against the possible benefits here and the chance that the AIs prevents some, you know, nuclear war or pandemic.

4. 47:20

   or or meteor strike or the thing that would wipe us out, which is sort of benefits on the other side. And then you could say, OK, if we sort of count your possible benefits against your possible harms here at the current risk levels, what premium should we charge them just for the privilege of risking our lives rather than because, you know, we're going to pay out if you kill us all? And I think that would be a very sensible strategy that people should absolutely look into. And I think if you ran anything remotely like the numbers that that the engineers are talking about today, you would be seeing that you're like, okay, OpenAI, you need to pay us a quadrillion dollars this year to continue this research.