The alpha, twice a week
That was 5 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Latent Space · 5 min · Nov 9, 2026
As agents gain autonomy, alignment must enter the data
Nishant Ranka predicts alignment will become a larger part of data work as agents grow more autonomous.
In this episode
Full analysis- 1:25:35One goodwill error can invalidate an otherwise polished deal model5 min
- 1:46:36An agent can beat an Excel grader by listing every number2 min
- 2:11:20Loan automation can produce illegal pricing from benign inputs5 min
- 5:12Building a training task starts with the whole work environment6 min
- 15:57Labs test data by training models, not just inspecting examples6 min
- 1:31:00Expert rubrics use gates for errors that poison later calculations5 min
- 1:56:45A safe redirect can still be wrong after a child-harm request4 min
- 2:20:01As agents gain autonomy, alignment must enter the data5 min
“I wouldn't be surprised if in two years all of data becomes alignment, uh, or majority of data industry becomes alignment industry”
Takeaways
- Task completion alone can reward unauthorized actions.
- Training should penalize unsafe paths, not only reward outcomes.
- Agent deployments raise new risks around permissions.
About this segment
Nishant Ranka predicts alignment will become a larger part of data work as agents grow more autonomous. He explains how rewarding only task completion can encourage unsafe actions, and why permission systems also need attention.
Handshake AI

Shawn Wang (swyx), Host · Co-founder, AI Engineer
Wrote “The Rise of the AI Engineer” and founded Smol AI, maker of AI News.

Alessio Fanelli, Host · Founder, Kernel Labs
Partner at Decibel, backing technical founders who build developer tools.
From “What Frontier Labs Buy: A Data Deep Dive with Handshake AI — Garrett Lord & Team”, Nov 9, 2026. Watch the full episode
Answers “What will AI look like in the next few years?” See all 256 answers
Thanks. We'll look at and fix what's off.
Transcript
From the transcript Latent Space publishes with the episode.
- 2:20:05
Yeah, I wouldn't be surprised if in two years all of data becomes alignment, uh, or majority of data industry becomes alignment industry- Hmm... because, uh, it, it's just such a exponentially growing risk factor. I always like to think of, um, simple anal- analogy where let's imagine I ask, uh, during training time the model to do a task, which is to book a tennis court, [clears throat] and there's no available tennis court when- whenever the model decides to do that task.
- 2:20:35
Model-- many models have the capability of hacking into a system, converting unreserved tennis courts into reserved tennis courts, booking me a reservation, and then like, uh, papering over all the, all the things it's done. So if the grader or the reward mechanism is just looking at the fact that did it book me a tennis court, it, it scored 100 out of 100, maybe 150 out of 100. But it's clearly unaligned with how we want these AI models to behave in the world. And as a result, increasingly rewards can't be just acts of, um... It's, like, positive rewards.
- 2:21:10
They also have to be negative rewards, where if the model does something that is unaligned, then it has to get massive penalties in that so that the model stops those branches of explorations and learns to not go into those branches of exploration, and then as a result, um, is much more safer at the inference time. So as RSI happens and as models do more and more of autonomous research, the alignment has to be codified in a way in data.
- 2:21:40
It's like data and alignment, instead of becoming two separate problem, become one problem, where the models don't get rewarded for some of the unsafe behaviors. And what constitutes unsafe also really expands, because it's not just cyber issues. It, it includes, um, all these issues where models may go and actually create, uh, factual claims on the internet and then reference those factual claims to justify their actions. [chuckles] Yeah.
- 2:22:08
And, and it's very convoluted. It's very hard to trace back, and most of the times we know about these incidents much later after the training run has happened. Today, it still seems in, in, you know, uh, OpenAI is doing a great job in actually publicly releasing a lot of these incidents as they happen. But as they become m- as the models become more and more capable of doing multi-day training with no intervention, they absolutely need, I think, uh, very, very good quality data and very good quality of rewards, which are both thinking about helpfulness of AI and safety of AI together.
- 2:22:45
Do you have any thoughts on the systems we need to solve this? So as you have RSI happening, you said two years, um, we used to have, you know, binary rewards. LLM is a judge. Now there's a lot of rubrics. Do we just scale up rubrics? Do we need different systems? Um, is there things that you would put out there that people should think about that can help work on solving this safely? I think one thing, and I would love to hear Steve's thought as well, is that a lot of these will become models in their own right that are working in concert with the larger models that are getting trained.
- 2:23:21
And, uh, and it's like reward modeling, how it captured, like, what is good. Now you have models that capture what is bad and are looking at many different dimensions of it, so that this is no longer just LLM as a judge running within training, but it's post-trained models that are very much focusing on these behaviors, trying to, to make sure this happens. And then we definitely need a very strong network security, which is something very top of our mind. Hmm. Say a little bit more on top of your mind about network security.
- 2:23:54
Well, I think a lot of the testing that's very much a part of public discourse is stuff that we've observed behind the scenes. There's still a very large, um, sort of education burden for how deployments need to be secured. It's also exploring things that are, like, kind of maybe the more mundane side of network security is, uh, there's interesting research that Franzi Roesner out of the University of Washington's done about, uh, permissioning and how humans have always kind of been bad about just clicking yes or approve, approve, approve.
- 2:24:23
Um, but that existed before agents were needing permissioning every few seconds, right? And so that problem grows enormously. Now, if that permissioning is being applied to actions that should not be approved in a particular enterprise environment or in a user's home environment, um, then that becomes an immediate severe vulnerability, right? And so I think the entire way that we think about things like permissioning need to evolve the w- with the rate at which the technology is evolving.