Try Pro free

The Pragmatic Engineer · 3 min · Nov 18, 2026

CodeRabbit built its own CI as agent workloads grew

A CI vendor quoted about $13 million a year as agent-written code drove compute demand.

In this episode

Full analysis
  1. 20:18Software factories turn each release into a learning loop4 min
  2. 25:29Code review shifts from reading diffs to managing release risk6 min
  3. 55:17Startups have less time to win and need more AI capitalPro pick 8 min
  4. 15:37A six-week agent build shipped without Gill reading the code3 min
  5. 40:22CodeRabbit built its own CI as agent workloads grew3 min
  6. 46:02CodeRabbit routes models by task instead of letting users choose5 min
  7. 1:08:12CodeRabbit hires for agency and runs cross-functional war rooms5 min
  8. 11:50Copilot exposed a code-review bottleneck before ChatGPT3 min
“agents which are de-slopping the tests, removing unit tests, doing more integration end-to-end tests”
Harjot Gillat 41:14 · The Pragmatic Engineer
41:14

Takeaways

  1. Agents prune redundant tests and add more integration coverage.
  2. CodeRabbit aimed for three-minute CI runs; its runs were about four.
  3. More validation brings more compute demand and cost.

About this segment

A CI vendor quoted about $13 million a year as agent-written code drove compute demand. CodeRabbit built CI in-house and runs agents to remove redundant tests.

Harjot Gill

Harjot Gill is founder and CEO of CodeRabbit.

Co-founded CodeRabbit, the AI code review company that became the most installed AI app on GitHub and GitLab.

  • Gergely Orosz, Host · Host, The Pragmatic Engineer

    Spent four years at Uber heading mobile, web and backend teams; shipped part of its payments system.

From “Software factories and code reviews with Harjot Gill”, Nov 18, 2026. Watch the full episode

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 40:30

    Yeah, and I know you previously talked about things like de-slopping tests. Yep. Can you tell me what is that? Is that a skill? is that a oh it's massive like i'll tell you our ci has gone through the roof like i think we're probably spending as much on on ci as tokens and you're spending a lot on tokens yes and also on ci like ci i heard last time like vendor quoted us like 13 million or something a year we built our and by the way now.

  2. 40:55

    Yeah, but that would have been your run rate if you would have stayed on the vendor. That's right. So the thing is, like, these agents are now, because they're, like, writing code and CLI commands, they're driving a lot of compute. And all these code changes, you're still going to make them go through CI gates, your merge queues and everything. And you're, in fact, writing more linters, more simulations, because you want to be sure.

  3. 41:16

    There are two problems there. One is, like, a lot of these tests are actually useless. So we already have agents which are de-slopping the tests, removing unit tests, doing more integration end-to-end tests, redundant tests. Like we have those running all the time. Now, at the same time, it's a constant battle. Like your CI time goes up, but you want to keep them within three minutes, ideally. Like that's how much, I think we are like roughly four minutes right now.

  4. 41:38

    And at the same time, then you're sharding and all like the costs go up. And this is the reason, like one of the things is like at our scale, like it makes sense to just build our own. So we built our own CI. right running right now what we are hitting is like compute capacity are we talking about cpu capacity yes even gcp unfortunately has very little compute available anthropics took it all so we were in the central region and we couldn't even upgrade our database instance we run out of and we had to like and you're like just to be clear you're not the smallest of companies i mean you will not be a google but but you have plenty of budget uh you have good funding like you're you know you're ready to give the money to well yeah we have signed good contracts with them but unfortunately they're all locked in by the large because they're all reading sandboxes rl environments

  5. 42:27

    i don't know what they're scaling yeah but they're also not just scaling gpus anymore but they're you know the ram so because i i wrote about the the cpu crunch and and the primatic engineer a few months ago but you're you're a good example where it's it's happening right it is happening in a big way right now we have a big cpu crunch as well and it's just like a few companies shipping at that scale like i mean for the rest of the world to get there i think the biggest bottleneck is gonna be the data centers the energy and those kind of things the models are commoditized like the models are there but they drive a lot of that compute as they work but now a little bit talking