# True AI agency may require two major breakthroughs

Markdown copy of https://realalpha.app/s/true-ai-agency-may-require-two-major-breakthroughs/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Diogo Almeida](https://realalpha.app/builders/diogo-almeida/) · Diogo Almeida is creator of Jev and cofounder of TypeSafe AI. · Built Jev, a decision model returning typed probabilities; co-invented RLHF and InstructGPT at OpenAI.
- Host: [Jacob Effron](https://realalpha.app/people/jacob-effron/) · AI Investor, Redpoint Ventures · Growth investor in Abridge, Ramp and Physical Intelligence; ex-Flatiron Health.
- Show: Unsupervised Learning
- Episode: [Jev Creator on Decision Models, Benchmark Fatigue & Why the Labs Missed It](https://realalpha.app/v/ham8dROHCrc/), Nov 5, 2026
- Moment: 55:49 to 59:20 of the episode (4 min), playing on https://realalpha.app/s/true-ai-agency-may-require-two-major-breakthroughs/; the episode from there: https://realalpha.app/v/ham8dROHCrc/?t=3349

## Quote

> “i would guess that we are something like **two breakthroughs** of like gigantic magnitude away from this”
> — [Diogo Almeida](https://realalpha.app/builders/diogo-almeida/) at [57:21](https://realalpha.app/v/ham8dROHCrc/?t=3441) · Unsupervised Learning

## Takeaways

1. He distinguishes autonomous agency from humans misusing powerful tools.
2. His estimate is two breakthroughs of gigantic magnitude from true agency.
3. He says evidence could change his view.

## About this segment

Diogo Almeida separates human misuse of powerful tools from autonomous agency, arguing they have different threat models. He estimates true agency is at least two major breakthroughs away, while acknowledging uncertainty.

From “Jev Creator on Decision Models, Benchmark Fatigue & Why the Labs Missed It”, Nov 5, 2026. [Watch the full episode](https://realalpha.app/v/ham8dROHCrc/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[55:56](https://realalpha.app/v/ham8dROHCrc/?t=3356) answer like um how good you know there's obviously so much discourse around it AI safety these days, like how concerned are you about a lot of these, you know, this AI safety conversation? I think AI safety is not one monolith and there's like a ton of different slices of it. I think that the ASI discussion is actually extremely different from the bio risk discussion, which is very different from like the cyber risk discussion, which is different than like trust and safety. Maybe like ASI versus like bad actors, you know, using bio and cyber and I get the sense that most people concerned with ASI are a little bit divorced from the reality of the technology.

[56:36](https://realalpha.app/v/ham8dROHCrc/?t=3396) And this has been the case since day one. You know, like, I feel like you need to, like, actually build a reliable system with AI to have a voice in the room that something is going to be, like, so reliably, maybe not evil, but misaligned that it can destroy the world. But I think we're pretty far from stuff like that. additionally there's actually a bunch of nuance towards like you know asi rsi and like whether or not the models have a sense of agency on their own like paperclip maximizer is actually very different than bad actor programs super employee to destroy the world type things and like actually i think that those end up becoming quite different in threat model i think that we're breakthroughs away from true agency as much as people like to like read these chains of thought and like try to

[57:27](https://realalpha.app/v/ham8dROHCrc/?t=3447) ascribe real agency to it um i would guess that we are something like two breakthroughs of like gigantic magnitude away from this i think even one breakthrough would not be sufficient is my guess but maybe one gigantic one could be um and i think the threat model of like humans with super tools who want to do damage is very different than the ASI-RSI thing. I think a lot of people who also believe in RSI both have not studied the history of AI very much and also have no appreciation of Amdahl's law, where if you assume that intelligence is the spectrum and it's the only one that matters and you have to map things into that spectrum and you can also map humans and chimpanzees, it's a horrible argument in my opinion.

[58:16](https://realalpha.app/v/ham8dROHCrc/?t=3496) uh then it does look bad but the reality is that it doesn't work that way and like the nuance of like the many different types of intelligence and how to do them reliably really matters a lot in this whole analogy could the field of ai development become a little bit faster uh yes but people have been trying this for a long time and it usually doesn't work out and it i don't think that people would count like a 10 speed up on ai research as rsi if it doesn't actually increase over time so i i i'm very open to being convinced by evidence but i i think that it's more of like a philosophical problem rather than a technological one um the kinds of people who like read uh like a bostrom super intelligence and treat it like reality over like a very interesting sci-fi book makes makes me kind of worried yeah um well this has been a fascinating

[59:11](https://realalpha.app/v/ham8dROHCrc/?t=3551) conversation. I thought we covered a ton of things. I want to make sure to leave the last