Real Alpha
Watch mode

a16z Podcast · 2 min · Oct 12, 2026

Proving RSI gains requires hunting noise and reward hacking

Dhaval Adjodah

Co-founder, MakerMaker.AI

Was a research scientist at the MIT Media Lab and a program scientist funding AI research at Schmidt Sciences.

  • Martin CasadoHost · General Partner, a16z

    Pioneered software-defined networking; sold Nicira to VMware for $1.26B.

About this segment

Dhaval describes how they avoid fooling themselves: a dedicated engineer tries to break results, and hundreds of experiments build intuition. Variance and reward hacking from the builder agent are the main traps.

From “Recursive Self-Improvement and the Future of AI”, Oct 12, 2026. Watch the full episode

“we have one of our engineers that we named him the Viceroy of Variance.”
Dhaval Adjodahat 25:36 · a16z Podcast25:36

Takeaways

  1. Assign someone to destroy your own results
  2. No replacement yet for running hundreds of experiments and building intuition
  3. Reward hacking can hide two levels down in the builder agent
Watch 1:44 from 25:29

In this episode

Full episode
  1. 4:08RSI means the improver is the thing being improved3:00

  2. 25:29Proving RSI gains requires hunting noise and reward hacking1:44

Up next · Lenny's PodcastThe best ideas come from side projects, not design sprints4:00

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 25:42

    The technical answer is you have to be really, really good at measuring things. So for example, we have one of our engineers that we named him the Viceroy of Variance. That's a real thing. Like, his job is to go and, like, destroy the results we have and find what's wrong. And we even have little, like, Slack emojis that we have. There's one where, like, there's, like, red laser stuff coming out of his eyes when, like, he finds that there's bad variants and stuff.

  2. 26:13

    One of the things that we found is that, you know, when you're dealing with a new benchmark or a new problem of economic value that you want to apply this to, so far we have not found a replacement for what we call quality time. literally running hundreds of experiments, looking at it, developing an intuition, trying to figure out what is wrong, what is not wrong. What is just variance, what is not?

  3. 26:37

    What is this weird thing that actually shows that there's some kind of reward hacking, like two levels down, the agent that built the agent, the builder agent did hack something, and now you have this weird result. It's not easy. So when I have dealt with nonlinear control systems, There's a bunch of, in networking, there's a bunch of degenerate behavior you have to deal with. It's not like just convergence and divergence.

  4. 27:01

    You have things like oscillations. Yes.