# Continual learning keeps humans in the loop

Markdown copy of https://realalpha.app/s/continual-learning-keeps-humans-in-the-loop/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Ronak Malde](https://realalpha.app/builders/ronak-malde/) · Ronak Malde is cofounder of Trajectory. · Built SWE-1 at Windsurf and now leads Trajectory's continual-learning platform for improving agents from production use.
- Host: [Pat Grady](https://realalpha.app/people/pat-grady/) · Co-steward, Sequoia Capital · Joined Sequoia in 2007; backed Snowflake, Zoom, ServiceNow and OpenAI.
- Host: Lauren Reeder · Sequoia Capital
- Show: Training Data
- Episode: [Bringing Continual Learning to Every Company: Trajectory’s Ronak Malde](https://realalpha.app/v/32vxW6GMeLY/), Nov 5, 2026
- Moment: 20:11 to 24:19 of the episode (4 min), playing on https://realalpha.app/s/continual-learning-keeps-humans-in-the-loop/; the episode from there: https://realalpha.app/v/32vxW6GMeLY/?t=1211

## Quote

> “you need to allow these models and build the algorithm so that it **keeps humans in the loop**.”
> — [Ronak Malde](https://realalpha.app/builders/ronak-malde/) at [24:05](https://realalpha.app/v/32vxW6GMeLY/?t=1445) · Training Data

## Takeaways

1. Agents need to learn the transitions between actions and outcomes.
2. Curiosity means exploring what an agent does not yet understand.
3. Continual learning uses human interactions to guide improvement.

## About this segment

Ronak Malde connects curiosity to learning how the world changes when an agent acts, then contrasts continual learning with recursive self-improvement. The distinction is that humans guide what the agent learns to do.

From “Bringing Continual Learning to Every Company: Trajectory’s Ronak Malde”, Nov 5, 2026. [Watch the full episode](https://realalpha.app/v/32vxW6GMeLY/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[20:14](https://realalpha.app/v/32vxW6GMeLY/?t=1214) Very cool. Okay, so we talked a little bit about group size one with GRPO. What's next? Where is continual learning headed? Yeah, so a lot of what we've been talking about right now is mainly just helping these companies translate over what good looks like for them into improvements for the model. I think to get to this true world where humans, we are extremely efficient about learning from the real world. And we're able to have, I think, inbuilt curiosity as well about the world.

[20:46](https://realalpha.app/v/32vxW6GMeLY/?t=1246) And I would say that's kind of one of the missing pieces to get us to agents actually learning autonomously on their own via continual learning. The models right now, they do a task, but they don't actually accumulate any experience or learn anything about the real world. Whereas if you think about, let's say I'm throwing a football, I feel like as a human, maybe if I've never seen a football before, I actually have no idea what's going to happen.

[21:13](https://realalpha.app/v/32vxW6GMeLY/?t=1273) But I do that a hundred times over and now I suddenly build this heuristic. And the important part is I think with humans, like as kids, we are constantly exploring the world. We see a soccer ball, we see a football, and we kind of just try out things to build this mental model. And that makes us incredibly intelligent. Whereas with agents, that doesn't exist right now. And so we've done a lot of thinking and how do you actually quantify curiosity? And I think one interesting way to think about this is, so there's this idea in RL called a Markov decision process.

[21:48](https://realalpha.app/v/32vxW6GMeLY/?t=1308) and it's basically you can you can characterize the world as a bunch of state action pairs so a state is like I'm holding the ball in my hand and then an action is I throw it and then the following state is where does the ball end up what LLMs right now are really good at is basically going straight from state 0 to state 1 their next token predictors right but what's missing is that intermediate part of what what is that transition function when you actually like go from s1 or s0 to s1 what is that transition function that happens and llms don't really have a way of encoding that right now so i think the interesting thing here is with humans we are very aware of when we don't know what a transition function is the world. And we say, I'm basically going to try to figure out what this transition function is. And I think this is where curiosity comes in.

[22:41](https://realalpha.app/v/32vxW6GMeLY/?t=1361) And so if there's a way to basically quantify, here's all of the world that I don't know about right now. And my goal as an agent is to figure out if I do this task, what happens? I build this mental heuristic. Then I think we are starting to solve this curiosity problem and get to a world where you can autonomously kind of learn and continually learn from the world. How do you make sure that an agent's curiosity is aligned with a user's objectives?

[23:07](https://realalpha.app/v/32vxW6GMeLY/?t=1387) So I think that's actually one of the reasons why I'm very passionate about working on continual learning. So there's two kind of directions that it seems like research is going right now. There's RSI, recursive self-improvement, and there's continual learning. And both of these are very, very tightly tied together, right, are very similar concepts. They're essentially just how does an agent actually learn from experience. The thing is, RSI is all about taking a model, putting it in a black box, and basically making it the best AI researcher itself so that it can improve itself, right.

[23:41](https://realalpha.app/v/32vxW6GMeLY/?t=1421) But this is kind of a scary world, right? Because it exactly begs the question of are these models and agents actually going to be aligned and have the best interests of humans in mind? Where I think continual learning takes a very different approach is it says, I want to learn from experience, but learn from the experience of actually interacting with humans and then getting better at the tasks that they set out and that they want to do.

[24:05](https://realalpha.app/v/32vxW6GMeLY/?t=1445) And I think that's where the key difference is, is you need to allow these models and build the algorithm so that it keeps humans in the loop. I don't want to end up in a world where these agents are going rogue. And I think the way to do that is to keep on this alignment path.