# CodeRabbit routes models by task instead of letting users choose

Markdown copy of https://realalpha.app/s/coderabbit-routes-models-by-task-instead-of-letting-users/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Harjot Gill](https://realalpha.app/builders/harjot-gill/) · Harjot Gill is founder and CEO of CodeRabbit. · Co-founded CodeRabbit, the AI code review company that became the most installed AI app on GitHub and GitLab.
- Host: [Gergely Orosz](https://realalpha.app/people/gergely-orosz/) · Host, The Pragmatic Engineer · Spent four years at Uber heading mobile, web and backend teams; shipped part of its payments system.
- Show: The Pragmatic Engineer
- Episode: [Software factories and code reviews with Harjot Gill](https://realalpha.app/v/4Cjb7691GJ8/), Nov 18, 2026
- Moment: 46:02 to 50:33 of the episode (5 min), playing on https://realalpha.app/s/coderabbit-routes-models-by-task-instead-of-letting-users/; the episode from there: https://realalpha.app/v/4Cjb7691GJ8/?t=2762

## Quote

> “we don't let the user drop down and choose the model like **we do it for them**”
> — [Harjot Gill](https://realalpha.app/builders/harjot-gill/) at [46:56](https://realalpha.app/v/4Cjb7691GJ8/?t=2816) · The Pragmatic Engineer

## Takeaways

1. Model releases arrive faster than the evaluation team can fully assess.
2. CodeRabbit selects models for specific sub-agents and workloads.
3. Cheaper models prepare context; expensive models handle deep inference.

## About this segment

CodeRabbit tests new models against price and workload as releases arrive every few weeks. It routes cheaper models to context preparation and reserves expensive ones for deeper reasoning.

From “Software factories and code reviews with Harjot Gill”, Nov 18, 2026. [Watch the full episode](https://realalpha.app/v/4Cjb7691GJ8/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[46:09](https://realalpha.app/v/4Cjb7691GJ8/?t=2769) Because of your business of offering a tool that gives you AI code review, model quality will be really important and costs and all of those things. How do you evaluate new model releases? And I'm obviously talking about the big ones, the OpenAI, the end traffic, but also open models and so on. Because I assume it's kind of a big deal for you to know how good of this model is in terms of the quality, the cost, latency, and so on.

[46:35](https://realalpha.app/v/4Cjb7691GJ8/?t=2795) That's right. I mean, right now we are also bottlenecked on our evals capacity. Oh, how so? I mean, the models, the cadence, they are coming out every few weeks now because the RL thing has taken off, right? So people are coming up with back-to-back releases, the training runs are shorter and shorter, right? which means that I mean every day we are like evaluating something and they're like of course three or four classes of open air models and tropic models and we have like an ensemble of models in our product so it's like unlike other products we don't let the user drop down and choose the model like we do it for them we know which sub-agents which workloads the model is going to be a good fit so a lot of the time that our applied AI team spends is figuring out the price to performance of each model and the workload to the point that we have zero bandwidth

[47:21](https://realalpha.app/v/4Cjb7691GJ8/?t=2841) to even onboard another vendor like we are a big token spender we have been working with like even you can see the jaminis and all but we don't have bandwidth that's been a big constraint even with the open models i know they're getting good enough each time we think that okay we could go use these open models open air brings down their models in price like luna is a great model like given the price point that sounds like we're using it a lot yeah we use a lot of luna like in our product we use other models we use expensive ones for deep inference and reasoning but a lot of code exploration a lot of the context preparation is done by cheaper models these days right but the the trick is like the open source is lucrative the only thing it's helping is like putting downward pressure overall in the industry we ourselves didn't get like much time to go and

[48:04](https://realalpha.app/v/4Cjb7691GJ8/?t=2884) evaluate those do you build your own models do you either fine-tune or or do some like post-tuning or have you experimented with this? Early days, we did. And that was an interesting realization that the frontier was moving faster than we could maintain those fine-tuned functions. Like, I mean, the shelf life of those investments were very short. So it was a little bit better for a short time and then a new model came and boom, your advantage was gone.

[48:31](https://realalpha.app/v/4Cjb7691GJ8/?t=2911) Then you're in a different zone altogether, right? So, and the other thing is like, we are in the coding space where I think other people have plenty of data. That's what they're RRing on. I mean, if we, let's say, had really very different data stream, then maybe it makes sense because the models are not good at training on that. But if you look in coding, code review, they're trained to some extent.

[48:48](https://realalpha.app/v/4Cjb7691GJ8/?t=2928) Where the real trick comes in is in fine-tuning the hardness. There are two parts of fine-tuning. One is a model. I think the bigger problem is like... What's the right context? Because these reasoning models can go off track if they overthink. I mean, you might have noticed extra is actually sometimes worse. It overthinks, right? Same thing. Like, I mean, depending on the context, we use a lot of like semantic analysis of the code, code graphs to make sure the token costs are like lower, while we also get very high quality leads to go and track.

[49:15](https://realalpha.app/v/4Cjb7691GJ8/?t=2955) So that's where like the effort goes in, is in fine tuning the hardness. And we have very large open source user base. So we are like very popular in public repositories. Yeah, yeah. Yeah, you have this thing where you make it really easy for open source projects to use CodeRabbit, right? Yeah, and big reason for that was the feedback loop. Because with open source, each review that happens every few seconds goes back into our pipeline to fine tune the hardness.

[49:37](https://realalpha.app/v/4Cjb7691GJ8/?t=2977) So basically, just from a business perspective, I mean, you're losing money on every one of these because you're paying for it. But you're getting it because now you're able to make your product better, which hopefully it will be more valuable on the long term, right? That's right. And this could be a win-win. Yeah. So the reason we have the price to, and this is like one of the very few use cases by the industry where a specialized harness is still applicable. Like everything else is getting eaten by general purpose cloud code. Even legal, finance, like it's doing general purpose work. But code review is one area where it's a breadth and a depth problem. You need a good recall precision. And over there, like a custom harness is very effective. And this is where like all this open source data is helping us tune that.

[50:19](https://realalpha.app/v/4Cjb7691GJ8/?t=3019) I mean, I have a couple of agents just running, looking at open source all the time. One more thing about quality is you mentioned briefly formal methods.
