The alpha, twice a week
That was 6 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Moonshots · 6 min · Nov 13, 2026
AI safety plans should weigh the cost of inaction
The panel challenges takeover-focused warnings and asks labs to account for the harms of inaction.
The episode’s best minutes
Full analysis- 28:18NVIDIA’s next CEO needs CUDA and TSMC credibilityPro pick 8 min
- 46:48AI safety plans should weigh the cost of inaction6 min
- 57:44Closed-loop AI self-improvement raises oversight questionsPro pick 7 min
- 4 more moments in the full episode
“where do you calculate the downside risk of not doing anything of inaction”
Salim Ismailat 46:56 · MoonshotsTakeaways
- Risk assessments should include the downside of doing nothing.
- Labs should explain what their alignment work actually does.
- The panel argues AI research should pursue concrete public benefits.
About this segment
The panel challenges takeover-focused warnings and asks labs to account for the harms of inaction. They call for clearer alignment work and directing compute toward disease research.
Salim Ismail, Host
Founder, Open ExO
Co-wrote Exponential Organizations; was Singularity University's founding executive director; sold Ångströ to Google.

Peter Diamandis, Host · Founder, XPRIZE
Co-founded Singularity University and founded Zero-G; co-wrote the books Abundance and Bold.

Alexander Wissner-Gross, Host · Founder, Reified
Has backed and advised 40+ tech companies, directly and through 021T Capital and Reified; writes The Innermost Loop.
From “Jensen’s Departure, NVIDIA’s succession plan, AI doomer response, and Demis talks RSI | #311 (Live)”, Nov 13, 2026. Watch the full episode
Thanks. We'll look at and fix what's off.
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 46:57
I call, I get very angry when I see stuff like this, right? If you're doing a serious risk assessment and you're supposedly a serious guy well where do you calculate the downside risk of not doing anything of inaction and i think that's that's not a serious conversation if you're not going to make a serious conversation where the hell is pfab for example we spend an unbelievable amount of time tracking and intellectual energy calculating p doom right ai is going to destroy humanity well what's the possibility and probability that it goes uh incredibly well um how what where is that uh in the equation where's the probability that we cure cancer or solve alzheimer's or hunger or extreme you know give every kid a rich rich rich education the people that don't Look at that side at all and only go, oh, lots and lots of people are going to die without saying what the mechanisms are or how that may happen.
- 47:56
So what I'd like to see is, okay, if your PDoom is over whatever it is, and his seems to be, then what's the recommended policy? Is it just stop everything? You're on the board of the thing accelerating the whole damn thing. Yeah. What pisses me off is why is an OpenAI, why is it anthropic, if they really believe this, just going out and saying we are putting all our capacity towards alignment. We are we have 10 different alignment programs underway to make sure any future model has full alignment because they control where their flops flow.
- 48:29
I want to get in my car, drive up to Toronto and yell at these people. Go ahead. Sorry, if I could thanks if I can recall opening I did pledge 10% of their compute towards it for porkers 20% 20% for Ilias super alignment effort 20% and then they didn't deliver Where is Ilya? Where is safe super intelligence? Hashtag? What did Ilias see and super? Shut down, but I think there's an elephant in the room, which is alignment equals capabilities We've known this since the instruction tuning days.
- 49:04
All alignment is at the end, including, by the way, Paul Cristiano's RLHF technique, it's simply reshaping the flops toward a distribution that is more in line with human instructions or human goals. It doesn't, there is this misperception, again, that somehow alignment is orthogonal to capabilities. But all of these alignment efforts, including RLHF, actually amplify capabilities. RLHF and instruction tuning in particular, I think if I remember the original instruction tuning paper, increased for constant parameter count for the model, increased the capability level by the equivalent of something like 10,000 times the number of parameters at constant parameter count simply through instruction tuning.
- 49:49
So that's like exhibit A, that alignment actually amplifies capabilities. And that's why I think this is the elephant in the room. you want more alignment you're simply accelerating capabilities what i don't hear i hear the fear mongering what i don't hear all of these ceos and board members talking about is what they're doing about it thank you yeah what should they be doing iman what would if you were running open air anthropic and alignment was your principal objective what would you be doing um i define it properly for a start you know get together as a group just to find the damn thing get your ai models to do it and then you would put additional resources into sponsoring everything rather than coefficient giving and others have to do that like again this fits with the internal red teams and
- 50:43
other things that they've got around that um and they should be very clear as well maybe even again with ai oversight behaving objectively about things like advertising and how they can cause misalignment right about things like what goes wrong when you delete parts of the latent space to sell Budweiser lights etc and that goes again against corporate side but again I think this is part of the interesting thing of just like the board of directors like you can now have an AI that literally objectively checks things right that checks board minutes that checks board decisions that checks alignment decisions and so I'd like to see that as an independent lot of compute. And I'd like to see their strategies public on these as well.
- 51:26
And where's the government in this? Right? I mean, if everybody's so fearful, where's the government saying, you know, we need you to report on your alignment efforts publicly every week. You know, Alex, when we wrote solve everything we talked about making sure you got the proper benchmarks, the proper harnesses, and you can accelerate a field I don't see that here. Government should be shaping incentives.
- 51:50
And I would answer the question, what should OpenAI be doing? Solve all disease. Do it really quickly. You want to get the public on your side? You want to get the public on the side of all of the data centers you're going to need? Solve all disease right now. Right now, instead of OpenAI intimidating all the mathematicians out there with slop drops, instead of that, just solve all the diseases. I think Anthropic is following a better approach very publicly, going after rare diseases, orphan diseases that their own pharma customers aren't solving, pursuing the long tail of disease.
- 52:22
If I were, Sam, just to do something actually pro-social and good for humanity that's obvious on the face of it versus implicitly good, I would announce it's not just going to be the OpenAI Foundation pursuing just Alzheimer's. it's going to be OpenAI solving the top 5,000 diseases much faster than the rest of humanity can. Do a slop drop on curing all disease. And I think they're heading that direction.