Try Pro free

The Peterman Pod · 4 min · Nov 2, 2026

Strict one-to-one ports limited agent scope

Stephen Toub used agents for line-level work but intervened on architecture and scope.

In this episode

Full analysis
  1. 11:27Incremental ports keep fast-moving products shippingPro pick 7 min
  2. 22:11End-to-end tests matter more than unit tests for a port6 min
  3. 49:55A Rust port cost $120,000 in model tokensPro pick 6 min
  4. 5:36Rust won after performance and embedding tests tied4 min
  5. 32:02Strict one-to-one ports limited agent scope4 min
  6. 42:15Most regressions were caught before pre-release3 min
  7. 1:00:01The port grew to three times its estimated size3 min
  8. 1:03:24A rewrite needs a measurable business return3 min
“One of the rules that I put in place with instructions that wasn't always abided by was I wanted these ports to be as strict ports as possible”
STStephen Toubat 35:05 · The Peterman Pod
35:05

Takeaways

  1. Keep ports to existing behavior, not unrelated bug fixes.
  2. Agents sometimes declared a slice finished too early or too late.
  3. Unrequested changes were a source of regressions.

About this segment

Stephen Toub used agents for line-level work but intervened on architecture and scope. He wanted ports to preserve existing behavior; unrequested changes, including bug fixes, contributed to regressions.

ST

Stephen Toub

Stephen Toub is a Distinguished Engineer at Microsoft.

  • Ryan Peterman, Host · Host, The Peterman Pod

    Went from new grad to staff engineer at Instagram in three years.

From “Microsoft Distinguished Eng: Why And How To Port Your Entire Codebase to Rust | Stephen Toub”, Nov 2, 2026. Watch the full episode

Answers “Are AI agents ready for real work?” See all 157 answers

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 32:08

    I saw in the blog post that you have this cool bar chart where you basically categorized every single message from you to the agents and what your intent was. Looking over that and reflecting on how you guided the agents, where was most of your time spent and what were the most impactful times where you would step in and change something there were two or three even higher level categories of things that i was doing um there was one or two categories where i think this just speaks to the the state of sort of long horizon tasks with models and harnesses today where it was me being frustrated with their progress in one way, shape, or form, and either telling them to keep going in some fashion when they had stopped or telling them to stop when they were going off down bad paths. But the majority were more, were in a category that was more of, there were trade-offs to be made.

  2. 33:18

    And I wanted to make sure, and reasonable people or reasonable agents could choose multiple paths. and I wanted to make sure that we were going down the right one and a lot of that would be times where I would just have a conversation with the the main agent to basically evaluate where we were the decisions that were needed to be made and look and kind of talking through rubber ducking through sort of the pros and cons of those choices given the current situation we were in And some of that, to your earlier questions about sort of the incremental versus sort of pause the world or Big Bang rewrite approaches, some of those were less about the end-to-end and more about what I wanted to do for this one slice that we were putting up.

  3. 34:16

    um so you know the the agents would pause at a particular place saying you know i'm done with with this component and i'd look at it and say this is not a good place to leave things in terms of the the the tendrils that this has into other systems you know temporarily or either maybe we need to do just a little bit more to kind of get beyond this architectural boundary and then that I'll clean up a bunch of stuff so that when we merge this, the code base is in a reasonable place.

  4. 34:48

    A lot of times it would sort of declare success for the slice. And if you squint, yeah, OK, but if it were do just a little bit more or in some case revert and do a little bit less, it would have left the code base in a better place or situations where it actually tried to do more than I wanted it to do. One of the rules that I put in place with instructions that wasn't always abided by was I wanted these ports to be as strict ports as possible, not removing functionality, not adding functionality, not making any huge changes in algorithmic approaches, not trying to fix arbitrary bugs that it comes across along the way.

  5. 35:35

    I would have it note those for me so I could revisit those separately, but I wanted these to be very strictly as one-to-one functionality ports as possible. And there were times when I would see it violating those rules, and I would basically have it back off from that and go back to what it previously had, or at least talk through what it did to make sure that I was comfortable with that. particular change being included.

  6. 36:04

    A non-trivial percentage of the regressions that we have had were from places where it violated those rules and I didn't notice and it included fixes for things that shouldn't have actually been done.