# Software factories and code reviews with Harjot Gill

Markdown copy of https://realalpha.app/v/4Cjb7691GJ8/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

Harjot Gill argues AI software factories can help small teams ship faster, if they build agentic systems for testing, review and operations.

- Show: The Pragmatic Engineer
- Hosts: [Gergely Orosz](https://realalpha.app/people/gergely-orosz/)
- Guest: [Harjot Gill](https://realalpha.app/builders/harjot-gill/) is founder and CEO of CodeRabbit.
- Published: Nov 18, 2026
- Length: 1 h 28 m

## The best minutes

- [20:18](https://realalpha.app/v/4Cjb7691GJ8/?t=1218) [Software factories turn each release into a learning loop](https://realalpha.app/s/software-factories-turn-each-release-into-a-learning-loop/) (4 min)
- [25:29](https://realalpha.app/v/4Cjb7691GJ8/?t=1529) [Code review shifts from reading diffs to managing release risk](https://realalpha.app/s/code-review-shifts-from-reading-diffs-to-managing-release/) (6 min)
- [55:17](https://realalpha.app/v/4Cjb7691GJ8/?t=3317) [Startups have less time to win and need more AI capital](https://realalpha.app/s/startups-have-less-time-to-win-and-need-more-ai-capital/) (8 min)

## Also worth watching

- [15:37](https://realalpha.app/v/4Cjb7691GJ8/?t=937) [A six-week agent build shipped without Gill reading the code](https://realalpha.app/s/a-six-week-agent-build-shipped-without-gill-reading-the-code/) (3 min)
- [40:22](https://realalpha.app/v/4Cjb7691GJ8/?t=2422) [CodeRabbit built its own CI as agent workloads grew](https://realalpha.app/s/coderabbit-built-its-own-ci-as-agent-workloads-grew/) (3 min)
- [46:02](https://realalpha.app/v/4Cjb7691GJ8/?t=2762) [CodeRabbit routes models by task instead of letting users choose](https://realalpha.app/s/coderabbit-routes-models-by-task-instead-of-letting-users/) (5 min)
- [1:08:12](https://realalpha.app/v/4Cjb7691GJ8/?t=4092) [CodeRabbit hires for agency and runs cross-functional war rooms](https://realalpha.app/s/coderabbit-hires-for-agency-and-runs-cross-functional-war/) (5 min)
- [11:50](https://realalpha.app/v/4Cjb7691GJ8/?t=710) [Copilot exposed a code-review bottleneck before ChatGPT](https://realalpha.app/s/copilot-exposed-a-code-review-bottleneck-before-chatgpt/) (3 min)

Skip the sponsor reads, the discussion of work pace, sleep and burnout, and the closing recap.

## Full analysis: Harjot Gill says he merged more than 5,000 PRs in a month

- Gill personally merged more than 5,000 pull requests in one month.
- That pace depends on validation, release controls and production monitoring, not coding agents alone.
- The spending case fits fast-moving venture-backed software better than every company.

Its 6 sections, what it means for you and what to watch are on [the page](https://realalpha.app/v/4Cjb7691GJ8/#analysis), free with an account:

- [3:30](https://realalpha.app/v/4Cjb7691GJ8/?t=210) Five thousand pull requests expose the real constraint
- [18:27](https://realalpha.app/v/4Cjb7691GJ8/?t=1107) Factories make every release a learning loop
- [26:10](https://realalpha.app/v/4Cjb7691GJ8/?t=1570) Code review shifts from diffs to release risk
- [31:35](https://realalpha.app/v/4Cjb7691GJ8/?t=1895) Shipping privileges make quality a human responsibility
- [55:40](https://realalpha.app/v/4Cjb7691GJ8/?t=3340) AI spending buys speed only in a narrow race
- [1:11:03](https://realalpha.app/v/4Cjb7691GJ8/?t=4263) People still choose problems and staff the factory

## Summary

Harjot Gill is the founder and CEO of CodeRabbit, an AI code review business. He says he has spent 30 years building software and previously founded Netsil, an observability startup, and Flux Ninja, which worked on load shedding and graceful degradation.

Gill says AI agents have raised his and CodeRabbit’s output, but that the gains depend on building a software factory around validation, CI, releases and production monitoring. He argues that human code review has given way to engineer-led risk decisions at release time, with shipping privileges tied to regressions. Gill says competitive, venture-backed software startups may need larger budgets for models and compute as competitors copy products faster; Orosz challenges whether that spending makes sense outside AI devtools. Gill favors hiring people with high agency and broad systems skills, while Orosz questions the sleep tradeoff and supports the driving-license approach to accountability.

### Minute by minute

- [3:13](https://realalpha.app/v/4Cjb7691GJ8/?t=193) **Agents drive a surge in shipping.** Gill says CodeRabbit shipped more than 5,000 pull requests in a month as agents took on work. He says many of his own pull requests clear bottlenecks such as flaky tests and CI. The company also puts marketing assets and ad changes in code, with pull requests deploying changes to Google Ads.
- [8:34](https://realalpha.app/v/4Cjb7691GJ8/?t=514) **Early startups shaped Gill’s AI bet.** Gill traces his career from a telecom startup to Netsil, an observability company, and Flux Ninja, which worked on load shedding and graceful degradation. At Flux Ninja, using GitHub Copilot convinced him that AI would write most code. He says CodeRabbit began with the idea that code review would become a bottleneck, though the opportunity proved larger than he expected.
- [14:57](https://realalpha.app/v/4Cjb7691GJ8/?t=897) **Agents turn ideas into implementation.** Gill says coding by hand was a means to an end, and he now focuses on moving ideas to implementation and validation. He describes an agent that ran for six weeks to build a Rust-based write-ahead log for object stores; he says he has not read its code. He uses simulations and other checks to let agents verify their work.
- [20:27](https://realalpha.app/v/4Cjb7691GJ8/?t=1227) **Software factories run feedback loops.** Gill distinguishes hands-on chats for shaping a feature from factory-style agent loops for ongoing work. His agents tackle flaky tests, CI, and production health, learning from releases and expanding rollouts as they observe performance. He says local machines limit how many sessions developers can run, so CodeRabbit is working on cloud-based environments.
- [25:17](https://realalpha.app/v/4Cjb7691GJ8/?t=1517) **Human judgment moves to release decisions.** Gill says human code review is dead, but human review is not. With more code arriving from agents and non-engineers, he sees pull requests as a gate for deciding what is worth shipping and assessing risk. At CodeRabbit, engineers review releases and their risk indicators rather than reading every pull request.
- [30:34](https://realalpha.app/v/4Cjb7691GJ8/?t=1834) **Shipping privileges depend on reliability.** Gill compares directing agents to earning a driving license. CodeRabbit tracks production regressions back to the pull requests that introduced them; after enough strikes, an engineer loses the ability to ship without human review and can earn it back after a proving period. Gill says teams also need agents watching production so they can limit the exposure time of defects.
- [38:33](https://realalpha.app/v/4Cjb7691GJ8/?t=2313) **Agents clean up codebase entropy.** Gill says teams can let agents ship quickly and then use other agents to maintain the code, refactor abstractions and remove flaky or redundant tests. CodeRabbit built its own CI after a vendor quoted about $13 million a year, and Gill says compute capacity has become a constraint. He says better validation has helped lower CodeRabbit’s defect rate.
- [46:02](https://realalpha.app/v/4Cjb7691GJ8/?t=2762) **Models need workload-specific evaluation.** Gill says evaluating the steady flow of new models has become a bottleneck. CodeRabbit routes different tasks to models based on price and performance, and Gill says open-source reviews feed useful data into its review system. He also describes using TLA+ models and simulations to find race conditions in a distributed Slack bot.
- [52:01](https://realalpha.app/v/4Cjb7691GJ8/?t=3121) **AI and infrastructure costs are substantial.** Gill estimates his own monthly AI bill at $250,000 to $275,000 and says the team spends several million dollars a month on tokens. He says AI-generated work also drives CI and sandbox costs, and CodeRabbit has moved CI in-house. He sees the cost as acceptable when he can connect it to value.
- [55:28](https://realalpha.app/v/4Cjb7691GJ8/?t=3328) **Gill argues startups need more capital.** Gill argues that venture-backed startups pursuing fast growth need more capital because spending has shifted from hiring large engineering teams to tokens, CI and compute. He says small teams can direct millions of dollars of model usage, and that paying more for stronger models can get a product to market faster. Orosz asks whether this logic applies beyond AI devtools.
- [1:00:47](https://realalpha.app/v/4Cjb7691GJ8/?t=3647) **Gill says product windows are shrinking.** Gill says startups may have only weeks to establish mindshare before competitors copy a product, compared with a longer window in the past. He expects more companies to use software factories to build a range of products, while arguing that mass-produced software could pressure traditional SaaS businesses. Orosz points to competition from large tech companies as well as AI labs.
- [1:08:12](https://realalpha.app/v/4Cjb7691GJ8/?t=4092) **Hiring favors agency and range.** Gill says CodeRabbit is hiring quickly in sales, while rethinking engineering roles. He cites a hiring post seeking people who had shipped 1,000 pull requests in 30 days or played 500 hours of simulation games. He says high-agency candidates, including junior hires who can learn quickly, may adapt well; the team also uses in-person “war rooms” to shape products together.
- [1:12:57](https://realalpha.app/v/4Cjb7691GJ8/?t=4377) **Agent work strains sleep and schedules.** Gill says agents on his local machine sometimes stall and that he wakes around 3 a.m. to guide them, a pattern he says is not sustainable. He hopes cloud-based agents and a follow-the-sun team can make that less necessary. Reflecting on burnout, he says hard work was not the cause in a previous startup; the lack of progress was.
- [1:21:50](https://realalpha.app/v/4Cjb7691GJ8/?t=4910) **Agency and breadth matter.** Gill says he looks for people with agency, side projects, systems thinking and experience across domains, because agents can extend what they already know. He recommends The Black Swan as a book about the scale of change. In his closing remarks, Orosz says he remains skeptical of Gill’s overnight agent tending and broad funding claim, but likes tying shipping privileges to regressions.

## Transcript

The Pragmatic Engineer, hosted by Gergely Orosz, with Harjot Gill.

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[0:55](https://realalpha.app/v/4Cjb7691GJ8/?t=55) What does it look like when a startup CEO merges 5,000 pull requests in a single month thanks to going all-in on AI agents and software factories? This CEO is Harjit Gil, heading up CodeRabbit, the AI code review startup. Harjit had been writing code for 30 years, is on his third startup, and never shipped as much as he does now. Today we cover Why Harjit says human code review is dead, but AI code reviews are more important than ever. The concept of driving license for engineers, how devs at his company lose their right to ship to prod if they ship too many bugs. His spicy take that $8 million seed rounds are no longer enough to build a software startup and me pushing back on it, and many more.

[1:31](https://realalpha.app/v/4Cjb7691GJ8/?t=91) If you want an unfiltered look on how a truly AI-pilled founder slash software engineer operates who is extremely hands-on with AI, this episode is for you. And while I am nowhere near this AI-pilled as hard yet, I found it educational to understand his thinking and his views on the dev tools industry, where competition feels more intense than ever before. This episode is presented by TurboBuffer, and I want to show you the new and improved the Pragmatic Engineer search at pragmaticengineer.com slash search that I built on top of it. I was frustrated with Substack's search, how their search was keyword only, and I couldn't do a search like the article where me and Ken Beck made a fun of a big consultancy. I'm obviously looking for the piece about McKinsey, but Substack cannot find it for the life of it. So I decided to build a search that I needed. I did the whole thing from idea to deployment in three hours with Codex and TurboBuffer with TPUF doing the heavy lifting. I dumped all my articles into TurboBuffer, it handles creating embeddings and getting full-text search working.

[2:26](https://realalpha.app/v/4Cjb7691GJ8/?t=146) The result is a much better search than what Substack has. Check it out for yourself at pragmaticengineer.com slash search. But let's think bigger. What would it take for Substack to give every single publication this kind of advanced search? With most other search systems, you would need to create a bunch of shards, and it would be expensive to operate the search indexes. But with TurboBuffer, each publication is simply a namespace. Idle namespaces live in object storage, which has no compute cost. Active ones are pulled into NVMe and memory caching tiers, so they are served very fast. And this is how TurboBuffer can scale to hundreds of millions of namespaces and why multi-tenant AI products at companies like Ramp, Harvey, and Notion are built on top of it.

[3:04](https://realalpha.app/v/4Cjb7691GJ8/?t=184) With TurboBuffer, you really do get the best of all worlds. Ridiculous scale, low latency, and cheap operating costs. Whether you're building a side project like I did or a platform with millions of accounts, check it out at turbopuffer.com slash pragmatic. Harjit, welcome to the podcast. Hey, thanks for having me here. It's great to have you here. This is your pull requests per month. Like the last month, it was more than 5,000 pull requests. That's right. How? How? At this point, like, it's like really machines are building machines. Like, I mean, just good at keeping the agents busy at this time, I would say.

[3:43](https://realalpha.app/v/4Cjb7691GJ8/?t=223) What are these pull requests doing? When are you building it? Because, you know, like, you will have a lot of responsibility and time spent just running the company, I guess, coordinating people, pulled into fires, the usual stuff, right? Yeah, I mean, it's all started, like, I think, when the models, I would say, 5.3 Codex was kind of the first inflection when we started seeing these agents getting good enough to do, like, real-world work, like, we couldn't even ship to production. And around that time, like, it was very clear to me that, okay, we have been around for, like, two years. Like, I mean, so generative AI has been going on for a while, but last three or four months were like a big inflection point.

[4:19](https://realalpha.app/v/4Cjb7691GJ8/?t=259) I also see it here. It was up to July, you had like, I don't know, maybe 100 or 200 max, and then it went to more than a thousand in July. It was just something else. And you wouldn't believe like everything just breaks when you start shipping at that scale. So as a leader, like one of the key things that I keep doing is like, how do I make my team productive? And a lot of my pull requests go in just fixing some bottleneck in our SDLC at this time. I mean, of course, I'm doing like maybe one or two tactical features where I'm driving the agents with each message to craft the UI or something. But most of my volume of the code will be like, believe it or not, fixing things like.

[4:54](https://realalpha.app/v/4Cjb7691GJ8/?t=294) janitorial work like fixing ci or flaky tests or building some like distributed system like fixing something in caching or something right so all of that hill climbing is what i'm doing with these pull requests right now it's very not very optimal i would say that number should probably like be 10x of where it is we have we're not even hitting the peak here with what these agents can do and what agents or what tool really kicked this off around i see i was july august was Was this Opus? Was this Codex? Was it something else? Yeah, I think both the model providers have been in lockstep in terms of capabilities.

[5:32](https://realalpha.app/v/4Cjb7691GJ8/?t=332) A lot of our team members use Opus 5.5 with Cloud Code, but I personally use Codex. I mean, I've been a big fan of the Codex desktop app. I've been coding like 30 years. I've done three startups. And most of the time, I've also been kind of the CTO hat in every company I've been. And I have been a NeoVim person. I've been a code editor person all my life. even like had put out my dot files which a lot of people use but now we are in a different zone altogether i have not been in a code editor since i shifted to codex desktop app it's pretty awesome and ever since then like it's been always about like how many agents can i keep busy at a time so i'm actually not sleeping much that's the other side of like you get all this anxiety that you have to really um because the kind of value you are creating i mean if you know really what to build And the kind of value you're creating is phenomenal.

[6:24](https://realalpha.app/v/4Cjb7691GJ8/?t=384) Like, I mean, I've never seen things move in such a compressed time frame quickly within the companies. So it's not just the volume. Like, these are not like some silly PRs. They're like actually moving the needle forward for our company. Okay, you're not sleeping much, so it's eating into your night, clearly. And you're running ED agents there mostly with Codex and some other tools. But when you're commuting, you know, a lot of people listen to podcasts or music or just drive. And you have a cheat code there, right? Yeah, so I have, so I kind of live in the East Bay and I take the Bay Bridge on my way to work here. And I have a Tesla FSD and I love FSD. I'm like roughly 97% self-driving on FSD, like pretty much it's like driving on its own.

[7:05](https://realalpha.app/v/4Cjb7691GJ8/?t=425) And what I do is like, and FSD has gotten really good. Like earlier, it used to like nudge you that you have to hold the steering wheel or something. But now you can like, as long as you're like looking at the road every few minutes, it's okay. So as a cheat code, what I'm doing on this side is like I have got my laptop with the codex session. And I'm just like sending the instructions while I'm driving. So it's kind of funny, like the AI is driving the car while like the other AI is driving the company. Wow. Which I'm stating. As well, that word there, right? Right. And also, I assume, you know, you can now talk as well or like, you know, there's Simon Wilson was telling me how I think a year or two ago he was already talking.

[7:40](https://realalpha.app/v/4Cjb7691GJ8/?t=460) It's just crazy how different hands off or in different areas you can you can just use AI to like build stuff. Yeah, that's right. Like one of the things with companies like CodeRabbit is like as you start shipping faster, every part of your business also have to move at agentic speed. like what we are finding is we're shipping so much that the attention the marketing has become a bottleneck so I had to go back and also like refactor the marketing foundation to be very agentic which means that we built like a separate repository we call it the content studio all our slide decks are now in code html css we built a whole portal to like share those download pdfs and so on we actually are running even ads through code like to like we have millions of dollars of Google Ads spend. And the way we are going after all these keywords and the different copies on that is we have a loop where we look at the ads performance and then some agent comes up with the code changes and we actually create pull requests to deploy ads. So when you merge that pull request it deploys it on Google Ads.

[8:42](https://realalpha.app/v/4Cjb7691GJ8/?t=522) So it's like everything is inside CodeRabbit at this point. Okay, so you've been in the industry for closer to 30 years. I just want to go back before we get more into how we're building now, but how did you get started? how did you get into industry? Where did you first work? Yeah. I mean, I started my career in very early stage in a telecom company, which was actually a startup acquired by Alcatel-Lucent back in the day. That's where I first got away from, okay, people could start companies and build deep tech. I mean, that was like carrier grade systems, high availability systems. So I learned a lot about distributed systems in my first job. At that point, I kind of like felt that, okay, I want to do my own startup. And that was the reason I came to the US. And the route I took was, I mean, I always wanted to do a startup and that that's very clear it's not like very clear and now you're on your third one anyway so that out in turkeys i came into my phd at university of pennsylvania in the east coast at philadelphia and the goal over there was very clear that i mean i'll incubate some good idea that i'll take out and then commercialize and that's exactly what i did so we did a technology

[9:43](https://realalpha.app/v/4Cjb7691GJ8/?t=583) transfer from the university my first startup was uh kind of also like an inflection point right when microservices and Kubernetes were taking off, we saw that the observability is going to become a bottleneck. So that was my first start. I was called Netsil, listen spelled backwards. Oh, yeah. And this was observability for Kubernetes, right? That's right. So the idea there was like these prior to microservices, you had monoliths and you had like a Java monolith where people will install an APM agent like New Relic App Dynamics to instrument that. So your function calls were in the code itself.

[10:18](https://realalpha.app/v/4Cjb7691GJ8/?t=618) and when function calls became API calls when you were in microservices so you started decoupling then you had to like really observe the network your network became the app in a way so that was the idea behind NetSail was can we listen in to these network events to understand application performance and that was kind of precursor to eBPF and some of those technologies that came out later after the startup you you moved to a larger startup right or did you move or was it acquisition? we got acquired Yeah, it was a pretty... But noontics, right? Yeah, that was a public company at that time. And we got a pretty decent offer.

[10:55](https://realalpha.app/v/4Cjb7691GJ8/?t=655) Like we just did one round of funding, but then the offer was pretty good that we took an off-ramp. Then I stayed there for like a few years, trying to spend time integrating our technology into their stack and the product offerings. Then I quit to start another company called Flux Ninja, which was in like web scale. Like how do you do load shedding, graceful degradation, and controllability, right? So we built like a whole PID controllers to make sure that the applications don't crash with cascading failures during like Black Friday spikes and so on. Now, very interestingly, like at that time, the market had appetite for all these Kubernetes service meshes, but then generative AI happened all of a sudden.

[11:32](https://realalpha.app/v/4Cjb7691GJ8/?t=692) We saw that first, yeah, when it touched GitHub Copilot. I mean, we had Tab9. This was 2021, right? So at FlexNinja, we were using some AI, like Tab9 is one of the products we were using. Well, Tab9 was the OG of AI Autocomplete. Yeah, and it would like create its own local model or something. I don't know what it used to do, but it used to work like, but it wasn't like as great as like when the GitHub Copilot, that was like the aha moment, right? Yeah. And this was, I believe, even before ChatGPT had come out.

[12:00](https://realalpha.app/v/4Cjb7691GJ8/?t=720) **Harjot Gill:** At that point, it was very clear. Like we were a fast-moving team, and at that time, the code review was already a bottleneck.

[12:07](https://realalpha.app/v/4Cjb7691GJ8/?t=727) So people were doing stack pull requests. We built our own tool called SPR. We open-sourced it. But the thing is that, like, it was still not ideal, like, given the volume of the code just started growing after GitHub Copilot. It was very clear that AI is going to write most of the code. And so just to pause there, when you saw GitHub Copilot in 2021, this was before ChatGPT, you already predicted or kind of like fast forwarded saying, oh, AI will write most of the code. That's right. This was not a commonly held view, like a select view might have held it, but what made you assume this? Can you talk us through your thinking process at that time?

[12:48](https://realalpha.app/v/4Cjb7691GJ8/?t=768) I was in the arena, like as a leader, like I was actually coding on the ground. Even at that company, a lot of the work I was doing janitorial, like just. making sure the developers were productive, adding all the linters, the tooling, everything to go faster. As a leader, my goal has always been how quickly we can get to the next milestone. And when I touched that system, I don't know, it just came to me that, okay, this is going to change everything. The tool was really good in sensing what I needed to do next. And I was accepting the suggestions. It was like writing really complex algorithms. You just provide a few comments on what you want to do, and the code came out. Tab complete, right?

[13:25](https://realalpha.app/v/4Cjb7691GJ8/?t=805) it was pretty magical. And at that time, it was just clear to me, I don't know how, but I mean, once you are in the arena, I think you see these patterns long enough, the second order effects. And the same thing was clear to me three months back on how software is going to be built. Like I would say I'm still learning. It's not like we're not on a stable ground yet. Things are moving too fast. And some of the things I'm seeing now, we could not have predicted even six months back on how the software development is going to evolve from here. You found a code rabbit on this kind of notion or prediction?

[13:55](https://realalpha.app/v/4Cjb7691GJ8/?t=835) Was that how it happened? Actually, we didn't realize this would be such a big opportunity. One thing was clear, it would become a bottleneck, but we thought, I mean, the market cap, we were looking at Snyk and a few other companies in the GitHub ecosystems, right? Because we have an installation which is like inside the version control systems like GitHub. Like there was some precedence around previous generation companies like Sonar Source, based out of Europe, which are like code quality agents. So we were kind of like thinking, okay, the market cap would be like in that territory. what we didn't realize that this is like much much bigger market cap opportunity because people who were not even traditionally buying those tools would line up to buy AI code reviews even though we don't sell to individual users a large number of our users and our individuals paying for the product. So you've been a NeoVim user you you wrote a lot of code for like decades pretty much right you're handwritten how easy did you find it to just move over to AI because a lot of people especially some folks who use NeoVim, you know, like they're kind of into the craft as well.

[14:55](https://realalpha.app/v/4Cjb7691GJ8/?t=895) They might have been a bit more hesitant to just go there because you kind of have to, you know, get rid of that really familiar UI. You no longer even have a UI when you're using agents. So for me, like writing the code is just means to an end. As an entrepreneur, like you want to get to the final value and validation quickly. I mean, I did enjoy writing code, but I wouldn't say that was the most enjoyable thing in building a company. It's about like how quickly you can get an idea out, get validation, grow that business and so on. Coding was kind of a necessary thing you had to do. And a lot of the work around maintaining those code bases, DevOps, all comes alongside that, right?

[15:31](https://realalpha.app/v/4Cjb7691GJ8/?t=931) So for going from, I don't miss it. If you ask me, like, do I miss writing code by hand? I actually don't. What I love is now I'm able to go from ideas to very quick implementation and just validate. So I'm doing actually the fun part of software development. I'm still doing software development. It's not like I'm not doing software engineering, but now what I'm focusing on is a lot more fun than figuring out the syntax and autocomplete. I mean, my agents are writing Rust code. I have zero idea how Rust works. We recently had an agent goal running for six weeks to... Six weeks. Yeah, so for running, so building like Git write-head log on object stores.

[16:08](https://realalpha.app/v/4Cjb7691GJ8/?t=968) Yeah, write-head log. When Cursor's origin blog post came out, I appointed the agent, it ran for six weeks, and now we have that in production, powering a lot of the knowledge bases at CodeRabbit.

[16:19](https://realalpha.app/v/4Cjb7691GJ8/?t=979) **Harjot Gill:** So I didn't even have to have the real implementation, it just hill-climbed over time on its own, all written in Rust, and I have not read a single line of it.

[16:27](https://realalpha.app/v/4Cjb7691GJ8/?t=987) And for those of us who are not as familiar with this specific term, what does hill-climbing mean for agents? Yeah, so it's some sort of a loop. Like, it all starts at a very basic that agent can verify its work against something. Yep. And these loops are like inner loops, outer loops, multiple types. So when you're looking at software development, you want to provide agents means to verify its work close to where it's doing the software development. So we do a lot of... or not, a lot of formal methods. So when we first did the Slack bot product like eight months back, we invested a lot in like first modeling what this distributed system is going to do in DLA plus, then implementing it.

[17:07](https://realalpha.app/v/4Cjb7691GJ8/?t=1027) And also investing a lot in simulations. So the way we are scaling code development, code review is just one piece, which we of course market, but there were like a hundred other things you have to do, right? So we invested a lot in local deterministic simulations, right? very recently we started investing in lean theorems as well so a lot of that is lean is another form of formal methods yeah like you have theorems and it even like used for mathematical proofs where you can also try to build probably correct software things like billing service and all we have got like a bunch of lean theorems i can actually show you after the podcast it's pretty cool what we're doing there so what we have been doing is like these are all like hill climbing like you're first making them hill climb locally then you're going to the next environment where, let's say, emulators is another thing we have invested a lot in our own 12.

[17:53](https://realalpha.app/v/4Cjb7691GJ8/?t=1073) So you can now very quickly mock a real emulator, like the real APIs, like Google Cloud APIs or whatever, right? Then you are actually testing against a real Google Cloud or Vercel environment. Those are the cloud providers we have. In dev, you can have the agent actually spin up services, actually test them, right? I mean, this is all hill climbing. And then in production as well, like the thing is that I kind of see each release. So we release multiple times a day, right? every 30 to 45 minutes, we ship code. And each release is almost like a turn in a strategy game. If you have played Civilization, you end your turn, you learn.

[18:27](https://realalpha.app/v/4Cjb7691GJ8/?t=1107) That's how we see releases. Like each release is an opportunity for an agent to learn something from that environment. So a lot of the times we will release features under a feature flag, do like progressive rollouts, like maybe 10% for our open source users and so on. And the agent will observe the performance, the logs until it's stable, then do wider and wider rollout. And each release is the opportunity to learn for the agent. Interesting, because what you're describing, this kind of stage and wider and wider rollout, this is something that Facebook had been doing for a long time. And they just have a massive user base.

[19:01](https://realalpha.app/v/4Cjb7691GJ8/?t=1141) We're talking billions of people, right? And they built this automated system. And outside of Facebook, I don't really know. I didn't know of many other companies that had a kind of automated rollout for a few reasons. One is this only works when you literally have hundreds of millions of users. It's really hard to do it otherwise. It's a large investment to build. Automation is hard to build. You need, even pre-AI, you needed some sort of ML, which again, just goes up and up. And now you're describing that you, at a pretty small company in terms of headcount, compared to Facebook, 75,000 people, CodeRabbit is like...

[19:34](https://realalpha.app/v/4Cjb7691GJ8/?t=1174) 40 engineers, maybe? Yeah, and you've been able to build this for yourself. That's right. So the way we are seeing it is like the way Netflix, Facebook used to ship with thousands of engineers and the kind of tooling they needed like all the way from the process on merge queues and everything like they needed to ship at scale is something that small teams are going to need now with AI now teams of 30 40 people if they use agents in the right way have the right abstraction and this whole concept of software factory they're going to ship as much as Facebook means that all the sophistication they had in terms of testing in production managing the risk managing your CICD everything, caching. There's a lot of stuff around it. I mean, they will need it in a small team setting as well. So it's a golden opportunity if you were, if someone was at Facebook at that time, like when they were shipping and figuring out all these concepts, to go out and now actually build products, the market's grown. There's a lot more people who need those products than ever before. And you mentioned software factory. It's a term that is going around a lot.

[20:35](https://realalpha.app/v/4Cjb7691GJ8/?t=1235) What does it mean for you? Yeah, it's many things. I think, I mean, we have a slightly, maybe it's the same take as others but I mean I'll give you my take on it even when you're building with AI there are I think two ways of using AI one is like more like on a potter's wheel you are like shaping the software with each interactive chat with cloud code or codex like like for example like with ui development i have the browser open i'm doing annotation and each time i send a message i'm crafting the software but with ai and i can like manage as a human like my decision capacity is to do only like one or two of those tactical chats in a day i mean i usually have like 20 chats in codex right now two would be of that manner around 18 are more like the factory concept like where i know that i can have the agent hill climb on something whether it's like building a distributed system i can be hands-off with a goal just like the example i gave you with the git on object stores and also sqlite ltx that's the other thing we did with knowledge base is we removed our vector database provider to build our own in-house version on sqlite ltx right on an object store and those kind of things you can hill climb the other stuff is like each time you have a deploy ci cd is

[21:51](https://realalpha.app/v/4Cjb7691GJ8/?t=1311) happening it's an opportunity for the ai to learn and continuously improve it so i have agents running continuously to remove flaky tests right each flaky test resets the merge queue it's really bad so you have to have that continuously happen each time you have a deploy or a ci each run is an opportunity for the agent to optimize something do the next thing right and then you're running the agents against production as well so you have agents which own health of some components i care about for example i care about billing i care about the knowledge base performance like i care about the code graphs we have.

[22:22](https://realalpha.app/v/4Cjb7691GJ8/?t=1342) So each deploy, I have the agent running in a loop to observe the next experiment now and do either two things, like add more telemetry if it's like flying blind or fix something. So those are the kind of loops I'm running. And those are factories. So the thing is that right now, the chat is not kind of the right interface to run those agent fleets at scale.

[22:42](https://realalpha.app/v/4Cjb7691GJ8/?t=1362) **Harjot Gill:** The chat is great when you're doing, crafting the software one message at a time.

[22:47](https://realalpha.app/v/4Cjb7691GJ8/?t=1367) So some of these things we are still figuring out. First of all, that whole maintenance, the janitorial work around shipping software and maintaining the factory should probably be in the cloud on sandboxes and all. And I shouldn't be the one waking up at like 3 a.m. I mean, it's more like morning shift, night shift now in the factory. We luckily have a team in Bangalore. So we have a follow the sun model. So I want like if I'm babysitting the agent, so I'm not owning the cool daddy mode. We are owning agents. Let's say I have an agent looking at the health of a billing service and it stalls on something.

[23:15](https://realalpha.app/v/4Cjb7691GJ8/?t=1395) it as a question, then if I'm around during the US time zone, I'm able to answer that question. But I want someone to be able to unblock the agent while I'm sleeping. So I would say we haven't figured it out yet. So we are going on the upward trajectory, but most of our development right now is constrained by how many sessions people can run locally, how many releases they can make their PRs land in, like look at the release coverage and the PRs they landed. And we're also constrained on local resources. One of the things we found, and that's where I have a cheat code as a CEO, I have a very powerful machine. The developers don't have those. Like they have like 48 GB RAM, 24 GB RAMs, and they max out at like three, four sessions. Beyond that, if the machine just dies with the unit test, it has to run locally, right? So we are feeling like the real compute constraints right now in terms of the local development. And we are investing very heavily to go and make some of our cloud-based environments work. But I guess it's pretty obvious at this point that a lot of development will move to the cloud because of a lot of the constraints that you mentioned. And I guess the only downside I can think of moving to the cloud of running

[24:18](https://realalpha.app/v/4Cjb7691GJ8/?t=1458) coding agents in the cloud is, A, there's a bunch of work that needs to be done that you have an environment that's similar to your local one, security, all that stuff. But you need to build that. And there's potentially a cost question as well, because now, you know, like you have your hardware and when you're utilizing it pretty well, that's pretty good, right? You buy it once and you do it and there will be an additional cost part but the the cost i i wonder if that's a smaller question because you know like we already know ai has a very real cost and i know you're paying a lot for ai already it's actually better because in the way like if it's in the cloud and more like a collaborative setting you clearly see the value created because right now we have some people But.

[24:55](https://realalpha.app/v/4Cjb7691GJ8/?t=1495) who are spending tens of thousands of dollars on local tokens. We don't know if they're doing like side projects, like everyone's building their like home automation systems. See what you mean, yeah. Right, so there's no auditability as well on local development. Right, so sometimes we look at the price per PR, like mine's, I don't know, like 150 bucks a PR, like for some people might be thousands. Then you have to question like, are they really doing your work or someone else's work? So the thing is that like the cost perspective, I think I will be also as a leader like comfortable spending the money if I can see where it's adding the value.

[25:25](https://realalpha.app/v/4Cjb7691GJ8/?t=1525) like if it's in the Slack, so we have a Slack bot over there when CodeRabbit responds to all the datadog alerts. I clearly see it's adding value. CodeRabbit, one of the main things that you do is AI code review. Now, I have to ask you this question, is code review dead? Because specifically human code review, there's so many signs that it is on its last legs. The teams who are still doing it, who I talk to, it's kind of theater. It's like, you know, LGTM looks good to me, But people no longer have comments. They no longer read it. And companies like Uber are now making changes to just make it so much easier for people to, you know, like see what a code review does.

[26:05](https://realalpha.app/v/4Cjb7691GJ8/?t=1565) Because they also know people no longer, they spend less and less time reading it because there's just more of them, right? You're right. I mean, the human code review is dead, but the human review is not dead. Some review still has to happen. If you look at the software development space, there used to be like a upstream kind of a guard, like when you were like planning the work, because your engineering was an expensive resource. So when you were using the tools like linear JIRA, one of the reasons to do this Agile and Sprints was to make sure the developers are working on most important tasks.

[26:33](https://realalpha.app/v/4Cjb7691GJ8/?t=1593) We don't like spend a bunch of time and effort building something that was thrown away. Exactly. So there was a lot of regulation upstream earlier. Now that floodgate has suddenly opened with AI. Now you can go from idea to code without going through planning. Like you have a support ticket coming in. People will just ask Devin to look at it or code have it slackboard or something, right? At the same time, your designers and people who were not software engineers are now doing coding as they should be because they know what to build. A lot of designers are shipping code. You have background agents like Datadog is now going to be opening PRs.

[27:04](https://realalpha.app/v/4Cjb7691GJ8/?t=1624) Each time it sees an alert, it's going to say, I'm going to do it for you, right? So what that means is that your planning layer and the review gate still has to exist somewhere and that becomes a pull request because everything upstream, there's an influx now happening, right? And when you have a pull request open, it doesn't mean that you have to ship it. At that point, you decide whether this is worth spending my time, whether I take it to production or not, because the human taste is the bottleneck now, human judgment. So when a human is coming in, they're first making a decision. Is this something I need to ship?

[27:33](https://realalpha.app/v/4Cjb7691GJ8/?t=1653) Is it slop I need to close? I mean, and you make a decision after the code is there, right? Second thing you're doing is like, what's the risk? Can it just go through production without me looking at the code? Or do I need to understand the broader implication? Is it like a database migration it is doing? Why is it doing the migration? Because you want to sometimes look at the schemas because that's more structural change, right? So we do all that like risk management within CodeRabbit. So some PRs are on fast track to production. We are even letting our interns ship directly to production. So human code review is dead, but I would say humans still review some of the higher order abstraction and how we are like doing this with SOC 2 compliance is also interesting so we don't really so the definition of change at CodeRabbit is now a release so we have a Slack message we kind of get all the indicators around the risk we are taking each PR you have a minimal risk high risk and over there you kind of make a decision on whether this can go or some things have to be rolled back so each release is what we are reviewing now not PRs as a human And then when you say human review, just to be clear, human means software engineer, or can it also mean non-software engineer?

[28:39](https://realalpha.app/v/4Cjb7691GJ8/?t=1719) I'm just curious if you still think the accountability needs to sit with people who kind of know the internals, or can it be someone else? Can it be product manager or designer? I think it's still software engineers that code rapid. Like, they're not necessarily looking at the code anymore, but they are actually understanding the risk that we are taking with each. So it's about who can understand the risk. Yeah, because if something goes bad, then they are going to be the one dealing with it, like rolling back. Like it takes like 10, 15 minutes to roll back. I mean, that's the worst that can happen if something goes bad.

[29:05](https://realalpha.app/v/4Cjb7691GJ8/?t=1745) But they are still taking the responsibility. So we ideally want like developers to be around when something goes out so they can babysit what they've just shipped. That's why we try to release so frequently. But at the same time, like with the person who's approving the release, we have SREs and all. So they also like are taking some responsibility at that point. I think it makes perfect sense, especially for a business, because we're talking about a business where you actually have revenue, you users, you know, like if slop goes out and it takes down production, you're going to just lose customers or goodwill or cost frustration and so on. So I think it's great that you're saying, okay, let's have someone accountable. But the problem that us devs have is there's just a lot more changes. And I'm not going to have time to read through all of them. What ways are you thinking either now or in the future? What do you think needs to change so that as engineers, as the person who's making the decision, I can just have better information to, you know, like focus my attention where I need to focus.

[29:58](https://realalpha.app/v/4Cjb7691GJ8/?t=1798) Like, for example, I don't really want to review a manual refactoring that just move things around and it compiles. I mean, I could, but, you know, there's good use of my time and there's poor use of my time. And I feel, as you will know from yourself as well, I mean, you've shipped like so many poor requests, you will probably focus your time on the critical stuff. That's right. I think the human judgment is scarce now, right? And if you want to, like, first of all, not look at the code, there's higher order abstraction. You want to understand the risk, understand the blast area, especially if you are familiar with some area of the code.

[30:31](https://realalpha.app/v/4Cjb7691GJ8/?t=1831) You might also want to have inputs on the architectural change. Like, for example, like, if you're writing a migration on a large table, like, it's going to crash in production. You have some of these, like, tribal knowledge you want to bring in, right? Other thing is, like, also, like, I kind of see, like, now driving these agents almost like getting a driver's license, right? So we basically, even on the developers, put some responsibility on them who are shipping. So we let people ship if they are in the review exempt list. So we also have different lists and groups and teams where based on their experience and their past performance, they can either go to production without a human review or we put them in a group where they need a human review.

[31:07](https://realalpha.app/v/4Cjb7691GJ8/?t=1867) That sounds like based on a trust that you built up with them. The way we have done that is we do look at how long there was a regression in production. So because we look at the code, when you fix something, we are able to tie that back into the PR that introduced it. And now if someone has been irresponsible, like they did hard prompting there, or they did not follow through their change into production, you will have sometimes a hundred hours of exposure. I mean, someone had to point out, a customer had to point out that something regressed, you didn't QA it or something.

[31:35](https://realalpha.app/v/4Cjb7691GJ8/?t=1895) **Harjot Gill:** And once you have like enough strikes, we take your driving license away. We make you go through the code review.

[31:41](https://realalpha.app/v/4Cjb7691GJ8/?t=1901) Then you go through like a proving period with some human reviews, and then we let you ship on your own. So we're trying to set up a culture where I think responsibility is also with the developers to not drive these agents off the cliff. I like this driving license analogy, honestly, because we talk a lot about, well, I talk a lot about or worry about quality because I can see that quality seems to have dipped across the industry. And clearly it's because there's a lot more things going on and a lot less review. And it's nice to tie back, you know, the feedback loop that look like I think it's human, right?

[32:11](https://realalpha.app/v/4Cjb7691GJ8/?t=1931) Like the more you ship, the more likely you'll mess up. And as long as you own it or there's consequences or you say like, okay, from now on for this time period, either, you know, come up with better practices or have a second person, have a second pair of eyes, which is expensive. But this is what we used to do even before AI, right? The intern for you, you give the trustee intern to ship, but we didn't have it. The intern needed to have like two sign-offs, for example. Right. I mean, it's almost like that. I think people have to have responsibility. Like we have seen some developers are really good in asking the AI the right questions around the implementation they just generated.

[32:42](https://realalpha.app/v/4Cjb7691GJ8/?t=1962) they're more diligent and some developers will come and give like an ambiguous high level goal they will say just solve this problem for me and ai will like completely re-architect it sometimes we have to pull the plug when we start shipping seeing PRCOA, we're saying, okay, we need to stop that right away. We have seen those kind of things. So some people are like good, they've been prompting and steering these agents and others are just reckless. And that's a behavior we then try to penalize. Yeah, I do like the idea of responsibility as well. But there's a question of like, why if the AI will get even better, right?

[33:15](https://realalpha.app/v/4Cjb7691GJ8/?t=1995) Like we're just speculating here. But at that point, why would you pay software engineers? And I kind of keep coming back to the idea that, well, you know, why do I pay a plumber to come when I could just watch a YouTube video and I could fix my toilet. The problem is that, you know, like nine out of 10 times, it probably works. But the 10 times I made it, it might either clog it or break it or something. And I'm paying the professional because if they come in and they mess it up, they're responsible and they will fix it. And I'm kind of thinking that as software professionals, you know, like we will be responsible.

[33:48](https://realalpha.app/v/4Cjb7691GJ8/?t=2028) And, you know, one of the reasons we will hopefully be paid the big bucks is we can take responsibility for a complex system where an outage is just very costly, right? No, I think the responsibility to some extent matters. Like the things you can't foresee, we all have to be reasonable with the responsibility. Software are still like complex systems. They break in very complex ways, right? Fair. And you saw that today, the tweet around like railway employee hacking and then they're blaming agents going off track. But in the end, like it's really human who have to be responsible with how they're driving these agents, right?

[34:19](https://realalpha.app/v/4Cjb7691GJ8/?t=2059) And they can do your bidding, but they can also go in crazy directions right i think i think that's important and the other thing is like with this kind of when you're shipping at agentic speed your operations also have to be agentic so one of the things we are like agreeing is that okay we our goal is not to prevent every outage it's like to make sure that the exposure time of something bad happening in production is as little as possible which also means that agents need to be looking at production Yes. If that's out of lockstep, you can't ship fast. You're going to be more like AWS. You burn your hands with AI, and you're going to say, ban AI from my code base.

[34:54](https://realalpha.app/v/4Cjb7691GJ8/?t=2094) What you're saying, as I understand, is, okay, we'll have agentic feedback to verify the code or give AI code review, whatever that is, or run maybe fast tests, et cetera. But the observability layer should also likely be tied. It's all part of the QA. It has to be because your defect rates maybe are not going down, but now you're shipping more defects. because the volume has gone up, right? Which means that you have to stay on top of them. Like you have to have agents always looking at not just alerts. In fact, alerts are limited by your imagination on the failure scenarios you imagine. Like actually these agents are really good at looking at logs.

[35:31](https://realalpha.app/v/4Cjb7691GJ8/?t=2131) They're great looking at traces, right? So the way, in fact, those are the areas we are still not like fully scaled yet. Like there's so many areas of our code where we have had longer exposure in production with bad code because no agent was looking at it. Wherever we have agents looking, we are able to quickly resolve stuff at agentic speeds as well. So entire software development from left to right is now going to be driven by agents, including production and including marketing. Even stuff going out, you have to now be agentic. Harjit just said how software development is going to be fully driven by agents.

[36:05](https://realalpha.app/v/4Cjb7691GJ8/?t=2165) And when agents write, ship, and fix most of the code, you want to know why each change happened, which brings us to our seasoned sponsor, Entire. A few weeks ago, I told you that Entire was building a GitForge. Well, it's finally here. You can now host private repos directly in Entire in the US, Europe, UK, India, Australia, Brazil, and Singapore. And Entire has just reinvented a pull request for the agentic era. The problem with PRs is how agents write faster than how any of us can review it. So Entire replaced it with something called trails. A trail connects everything about the work in one place, the intent, the code changes, the agent sessions. A trail can be spread across multiple repos so the change to your backend and your web app gets reviewed once. With the trail you can configure customized evals. They look like a heads-up display in an airplane and score your work on every push. And you can see how each agent run passes or fails.

[36:55](https://realalpha.app/v/4Cjb7691GJ8/?t=2215) your user-defined gates and decide when you merge based on these. The point is that you control when you allow agents to be allowed to merge the code, like only merging after a human review or after all gates pass. Oh, and one more thing. Entire shipped continuous memory. Entire turns your repo into a brain so your agents can search past coding sessions to find the context behind earlier changes. Basically answer the why behind code changes. Memories get shared across the team and across the repo. If you want to see what an AI native Git Forge looks like, head to entire.io slash pragmatic and check out trails. And once agents are observing production and pushing fixes, someone has to decide what they're allowed to touch, which is why I'd also like to mention our season sponsor, WorkOS. WorkOS provides the auth infrastructure B2B and AI native companies need to sell to enterprise customers with SSO skim RBAC and now authorization for AI agents. WorkOS airlock is intent-based access control. Most access control only asks who you are. Airlock also looks at what the agent is trying to do at the moment it tries to do it.

[37:57](https://realalpha.app/v/4Cjb7691GJ8/?t=2277) Clear-cut rules like no agent deletes cloud resources are run as plain code. For the fuzzier calls, an AI judge evaluates the policy at runtime. You write these policies in plain English and Cloud can pull the ones you already had out of Notion or Jira and import them for you. Your agents, whether it's Cloud or Devon, connect to a single access point and your MCB servers are behind this access point. Airlock is a new layer on top of your existing permissions, which means that an agent can never get more access than the person who invoked the agent has. When something risky comes up, Airlock can escalate to human for approval, say over Slack. The idea is to write the rules once and spend human attention only on the calls that need it. You can request early access at workcalls.com slash airlock. And with this, let's get back to Harjit and what happens to code quality when agents write most of the code. Let's talk a little bit about quality because in your previous startups, you know, pre-AI, you've built some infrastructure tooling, which, you know, reliability is very important at those places. Accuracy, when you have the monitoring, I assume these are all non-negotiable. But what's happening now with AI, what we're seeing is when you're

[39:02](https://realalpha.app/v/4Cjb7691GJ8/?t=2342) using AI to generate code, it's great. It does it faster than I could do, usually similar or better quality than what I could have done. But it does get verbose. It can just repeat itself. It can just add a lot of tech depth you know like once you start generating enough you see that it's just gone a bit more bloated or a lot more bloated what is your observations here on how did you see this used to be for earlier models have newer models changed it and and what is it still a problem what are you doing about it yeah i mean once you unleash the agents on your code base there is some chaos i mean there's a saying like let the chaos reign and then you reign in the chaos that's a philosophy so rather than prevent the entropy the entropy is going to increase of the code base maybe maybe your security posture also weakens let me because you're not reusing the same auth layers or something like reusing stuff then you need to have agents which are self-maintaining that code the janitorial like the Roombas right so that's the approach we are taking like let people ship quickly let some of the chaos rain maybe things are not scalable right now but then you have the agents also cleaning up the mess like all the way from flaky tests

[40:09](https://realalpha.app/v/4Cjb7691GJ8/?t=2409) to bad abstractions to refactor the code. For example, recently we ran agents to introduce effect and X state into our code base. So refactors can constantly keep happening, right? So you only make sense of that mess once you actually have the mess there. Like then you figure out what's the best way. So I think both have to be like, it's like raining in the chaos as they say and then rain in the chaos, right? Yeah, and I know you previously talked about things like de-slopping tests. Yep. Can you tell me what is that? Is that a skill? is that a oh it's massive like i'll tell you our ci has gone through the roof like i think we're probably spending as much on on ci as tokens and you're spending a lot on tokens yes and also on ci like ci i heard last time like vendor quoted us like 13 million or something a year we built our and by the way now.

[40:55](https://realalpha.app/v/4Cjb7691GJ8/?t=2455) Yeah, but that would have been your run rate if you would have stayed on the vendor. That's right. So the thing is, like, these agents are now, because they're, like, writing code and CLI commands, they're driving a lot of compute. And all these code changes, you're still going to make them go through CI gates, your merge queues and everything. And you're, in fact, writing more linters, more simulations, because you want to be sure. There are two problems there. One is, like, a lot of these tests are actually useless.

[41:19](https://realalpha.app/v/4Cjb7691GJ8/?t=2479) **Harjot Gill:** So we already have agents which are de-slopping the tests, removing unit tests, doing more integration end-to-end tests, redundant tests.

[41:27](https://realalpha.app/v/4Cjb7691GJ8/?t=2487) Like we have those running all the time. Now, at the same time, it's a constant battle. Like your CI time goes up, but you want to keep them within three minutes, ideally. Like that's how much, I think we are like roughly four minutes right now. And at the same time, then you're sharding and all like the costs go up. And this is the reason, like one of the things is like at our scale, like it makes sense to just build our own. So we built our own CI. right running right now what we are hitting is like compute capacity are we talking about cpu capacity yes even gcp unfortunately has very little compute available anthropics took it all so we were in the central region and we couldn't even upgrade our database instance we run out of and we had to like and you're like just to be clear you're not the smallest of companies i mean you will not be a google but but you have plenty of budget uh you have good funding like you're you know you're ready to give the money to well yeah we have signed good contracts with them but unfortunately they're all locked in by the large because they're all reading sandboxes rl environments i don't know what they're scaling yeah but they're also not just scaling gpus anymore but they're

[42:31](https://realalpha.app/v/4Cjb7691GJ8/?t=2551) you know the ram so because i i wrote about the the cpu crunch and and the primatic engineer a few months ago but you're you're a good example where it's it's happening right it is happening in a big way right now we have a big cpu crunch as well and it's just like a few companies shipping at that scale like i mean for the rest of the world to get there i think the biggest bottleneck is gonna be the data centers the energy and those kind of things the models are commoditized like the models are there but they drive a lot of that compute as they work but now a little bit talking about this the cpu crunch so because i think this is i'm super curious to learn more on both what you can do to get more cpus and what you can do to maybe use your cpus more efficiently right you mentioned rust and i wonder if this is about it because rust is as efficient as you can get your code down to right that's what we did like i mean we were on typescript earlier like we just moved to bun i mean there was just another i was in tahoe on a beach i ran a goal and over the weekend all the services were migrated to bun dude your your idea of holiday yeah i mean i have the pictures of the team was so surprised i just ran a goal command and all the services moved to bun

[43:32](https://realalpha.app/v/4Cjb7691GJ8/?t=2612) over the weekend so so so just so i understand so you have this thing where you set up a goal and then you do a long-running session in the cloud or on your local computer unfortunately Interestingly, it's local. I mean, we do have a cloud agent, but I should be the one using it, but I have a powerful machine, so I always go. And then it goes off and it runs on your machine and it runs long-running tasks and it usually completes it. It does. I mean, it will sometimes stall to ask a few questions, but some of it is like, interestingly, the distributed systems are now far easier to build. They don't need much hand-holding because there are only a few ways to get something right when it comes to the architecture.

[44:09](https://realalpha.app/v/4Cjb7691GJ8/?t=2649) The product is like open-ended, like the UI you can build in 10 different ways. And that's where the taste, the judgment comes in a lot more. But we used to pay a lot for people doing complex scale distributed systems, things like Kubernetes and advanced caching observability scale, right? To be honest, those things are super cheap now. So you're saying that that part, that kind of this backend, even though they're complex, has been commoditized thanks to the models? Yeah, because they're like, as the saying goes, like there's only maybe one way of building Unix. If you were to build it today, you're going to come up with the same abstractions.

[44:38](https://realalpha.app/v/4Cjb7691GJ8/?t=2678) Yeah, and I guess going back to, for example, Kubernetes, I had Kat Cosgrove on from the Kubernetes team, but she was saying how Kubernetes is, on one end, it's complex to onboard to, because it's complicated, but they also have really good documentation. And I guess when you have something like this, and there's only, I guess, I assume there's only one. or maybe two ways to do things. But when you have this, agents are pretty good. When there's good documentation, no matter how complex it is, they can navigate the complexity. Those kind of things they're like great at. Like things like, okay, I'll optimize my Docker files.

[45:06](https://realalpha.app/v/4Cjb7691GJ8/?t=2706) They can just keep running it against Orbstack locally and optimize it. Optimize my CI cache or do this bundt migration. Those kind of things are super easy right now. I mean, if you have good unit tests and testing, like you can validate that, hey, things are coming up correctly and the smoke tests are passing, you can even try it on production. And usually things are working It's not like we are shipping a lot more defects. Our defect rate, I would say, is going down. I mean, this might be a hot take, but like humans were writing slop, not AI. I mean, that's what I'm realizing. Like AI is actually writing less slop at this point if it can validate against something.

[45:37](https://realalpha.app/v/4Cjb7691GJ8/?t=2737) So you're saying that in the past we shipped, like most teams would ship kind of slop or low quality stuff when we didn't validate it. And to be honest, if we're just being honest, we didn't always validate it, right? We had more outages in the past. I mean, that's my observation. I still have to measure it. But I would say when we were shipping with hand. And you're talking about just CodeRabbit, right? Now I'm seeing actually fewer outages. The more our code base becomes agent ready, the better validation layers become. We are seeing less defects, not more actually, as the velocity is going up.

[46:06](https://realalpha.app/v/4Cjb7691GJ8/?t=2766) So one more thing about quality is evals. Because of your business of offering a tool that gives you AI code review, model quality will be really important and costs and all of those things. How do you evaluate new model releases? And I'm obviously talking about the big ones, the OpenAI, the end traffic, but also open models and so on. Because I assume it's kind of a big deal for you to know how good of this model is in terms of the quality, the cost, latency, and so on. That's right. I mean, right now we are also bottlenecked on our evals capacity. Oh, how so? I mean, the models, the cadence, they are coming out every few weeks now because the RL thing has taken off, right?

[46:46](https://realalpha.app/v/4Cjb7691GJ8/?t=2806) So people are coming up with back-to-back releases, the training runs are shorter and shorter, right? which means that I mean every day we are like evaluating something and they're like of course three or four classes of open air models and tropic models and we have like an ensemble of

[47:01](https://realalpha.app/v/4Cjb7691GJ8/?t=2821) **Harjot Gill:** models in our product so it's like unlike other products we don't let the user drop down and choose the model like we do it for them we know which sub-agents which workloads the model is

[47:11](https://realalpha.app/v/4Cjb7691GJ8/?t=2831) going to be a good fit so a lot of the time that our applied AI team spends is figuring out the price to performance of each model and the workload to the point that we have zero bandwidth to even onboard another vendor like we are a big token spender we have been working with like even you can see the jaminis and all but we don't have bandwidth that's been a big constraint even with the open models i know they're getting good enough each time we think that okay we could go use these open models open air brings down their models in price like luna is a great model like given the price point that sounds like we're using it a lot yeah we use a lot of luna like in our product we use other models we use expensive ones for deep inference and reasoning but a lot of code exploration a lot of the context preparation is done by cheaper models these days right but the the trick is like the open source is lucrative the only thing it's helping is like putting downward pressure overall in the industry we ourselves didn't get like much time to go and evaluate those do you build your own models do you either fine-tune or or do some like post-tuning or have you experimented with this?

[48:12](https://realalpha.app/v/4Cjb7691GJ8/?t=2892) Early days, we did. And that was an interesting realization that the frontier was moving faster than we could maintain those fine-tuned functions. Like, I mean, the shelf life of those investments were very short. So it was a little bit better for a short time and then a new model came and boom, your advantage was gone. Then you're in a different zone altogether, right? So, and the other thing is like, we are in the coding space where I think other people have plenty of data. That's what they're RRing on. I mean, if we, let's say, had really very different data stream, then maybe it makes sense because the models are not good at training on that.

[48:46](https://realalpha.app/v/4Cjb7691GJ8/?t=2926) But if you look in coding, code review, they're trained to some extent. Where the real trick comes in is in fine-tuning the hardness. There are two parts of fine-tuning. One is a model. I think the bigger problem is like... What's the right context? Because these reasoning models can go off track if they overthink. I mean, you might have noticed extra is actually sometimes worse. It overthinks, right? Same thing. Like, I mean, depending on the context, we use a lot of like semantic analysis of the code, code graphs to make sure the token costs are like lower, while we also get very high quality leads to go and track.

[49:15](https://realalpha.app/v/4Cjb7691GJ8/?t=2955) So that's where like the effort goes in, is in fine tuning the hardness. And we have very large open source user base. So we are like very popular in public repositories. Yeah, yeah. Yeah, you have this thing where you make it really easy for open source projects to use CodeRabbit, right? Yeah, and big reason for that was the feedback loop. Because with open source, each review that happens every few seconds goes back into our pipeline to fine tune the hardness. So basically, just from a business perspective, I mean, you're losing money on every one of these because you're paying for it. But you're getting it because now you're able to make your product better, which hopefully it will be more valuable on the long term, right?

[49:51](https://realalpha.app/v/4Cjb7691GJ8/?t=2991) That's right. And this could be a win-win. Yeah. So the reason we have the price to, and this is like one of the very few use cases by the industry where a specialized harness is still applicable. Like everything else is getting eaten by general purpose cloud code. Even legal, finance, like it's doing general purpose work. But code review is one area where it's a breadth and a depth problem. You need a good recall precision. And over there, like a custom harness is very effective. And this is where like all this open source data is helping us tune that. I mean, I have a couple of agents just running, looking at open source all the time.

[50:24](https://realalpha.app/v/4Cjb7691GJ8/?t=3024) One more thing about quality is you mentioned briefly formal methods. I'm interested, when did you start to experiment with it? You mentioned TLA Plus and Lean. How is it going? Where are you using it? Because when I talk with Hillel Wayne, who was on the podcast, a TLA expert, you all know him, he said that he thinks formal methods, especially TLA Plus, is probably an overkill for almost all software, even with AI, although he did say this about six months ago, so the progress was not there. I'm just really curious on where you're seeing success. So we started using it back in February or March when we built the Slack bot product.

[51:03](https://realalpha.app/v/4Cjb7691GJ8/?t=3063) So that product had like some distributed states, like it had a queue, like queue up the messages, you could steer the messages. It had like complex states around, like shutting down the sandboxes. Like there was a lot of state machine there on the warm idle state. and so on. So that's where that TLA plus models, they're not great for like things which are like algorithms. They're great when it comes to like messages and actors and interactions. And that's what like we used because at 5.3 codex wasn't as great as now Astra. Now I think it might make pure mistakes, but the way we made 5.3 codex successful was first model it, make sure the state machines were correctly done before writing the code.

[51:39](https://realalpha.app/v/4Cjb7691GJ8/?t=3099) And we did find issues in the initial implementation after we deployed TLA plus. It could find all these like race conditions which we had overlooked. Yeah, simulations also help by the way. So there were two techniques we invested back in February or March. We built an entire simulation harness where we could fuzz and try different interleavings. At the same time, we used TLA Plus to model different states and explore those states and then find bugs. Nice. And I wanted to go back to the, not your commits, but the CodeRabbit commits. And this is the weekly commits, which are kind of similar to yours, except it's more granular, you can see but there's a huge inflection point in august very similar to how you went it went from about less than a thousand commits per maybe like 500 or 300 in july to more than 5 000 we're talking weekly commits at the last data point i wanted to ask what about the cost of of ai tools you obviously have ai code review tools you have the ai generation tools where is it at right now roughly so our ai cost i mean i mean mine alone would be 250k a month 275k something like that wow you're you're really spending it yes i mean that's just me alone and some of the people

[52:55](https://realalpha.app/v/4Cjb7691GJ8/?t=3175) the team are shipping close to me right now, right? And a lot of the also like the number two code committer is actually the CodeRabbit's background agent. So we also have like the agents looking at health and just creating PRs and we just take them to completion. So we are spending like several million dollars a month on tokens. And then we are also spending a lot on the CICD, right? And the sandbox environments. And it sounds like that was the CICD was on track to be a million a month, but you now brought it in-house. So you probably decreased it up a little bit. 50% roughly is what I'm hearing.

[53:27](https://realalpha.app/v/4Cjb7691GJ8/?t=3207) I have to see like, but we just shifted like over the weekend, this weekend, all to the local CI now. Okay. So for almost everyone else, this was absolutely freaked them out. Like having this thing and they were thinking, how can I rein it in? I also heard a lot of, I did research on startups and how their costs went. And for most of them, the costs spiked in August with Opus 5.5 and then it went down. Most companies are kind of just using it for coding. They're not running as many loops as you're doing. how are you thinking about the the cost of ai and why are you not nervous as a ceo and a founder so i think you're going to use both kind of models like this so if you have a good model routing layer then of course for some of the most of the tasks you can even use cheaper models but some of the things you still have to bring in the expensive model the other aspect is the fast mode like you pay double with ultra fast those are really great right i mean as a startup you want to get there faster than the next competitor right then the thing is like even though ai is getting commoditized but it's now driving the second-order effect of the CI cost and the CPU crunch we are seeing, right?

[54:27](https://realalpha.app/v/4Cjb7691GJ8/?t=3267) I think that's real. That's probably not going to go away unless, like, you are getting rid of unit tests or finding some other way of validation. Like, some people are doing local validation and using that kind of attestation to, like, pass the central CI. There could be some tricks, but the fact is, like, the software factories are going to be expensive, right? Earlier, when I wrote that tweet, I know that I got a pushback on that tweet. We can talk about it. Let's maybe talk about this tweet. So this tweet, I'll also put it on the screen, but I'll read it to you. You quote tweeted this founder engineer who shared how they're spending 32 million per year in clotspan building this massive multiplayer game. And you said that what we're seeing at CodeRabbit and elsewhere is this parable trajectory that even large seed rounds like 8 million plus are no longer enough to build the software business. Software factories need quality tokens at large scale compute to run CI sandboxes and other infrastructure to ship software at scale.

[55:17](https://realalpha.app/v/4Cjb7691GJ8/?t=3317) At this point, some smaller startups exist only to funnel ideas to larger startups with more tokens. There's a lot of spicy stuff in here. But let's talk about, I know X is not necessarily great for capturing the nuance, but you had an observation there. You said that you're seeing this trend. Is it startups spending a lot on software factories? I think that's going to be inevitable. There are multiple software businesses you can run. But if you're looking at hyper growth, VC funded startup in these competitive spaces, you would probably need more capital right now because now your bottlenecks have shifted.

[55:50](https://realalpha.app/v/4Cjb7691GJ8/?t=3350) Earlier, you needed to like pay mainly for the talent. So you'll get like ex-Netflix, ex-Google, ex-Facebook engineers. They themselves will be founders. They will hire from the same pool. Give them nice talk, some better, really good laptops. And that's it. Like it was all like the brain game. Like the IQ was the people. But you needed to pay for them, right? in Silicon Valley like the base salary would have been 300k plus and stock and all that we're talking a million plus in terms of running a business like in early stages I'm talking about Seed, Series A even before you hire sales like a lot of it is like the team like the dev team right now what we're seeing is like the IQs on the tap like the tokens like in fact even the smartest people need to be driving tokens

[56:31](https://realalpha.app/v/4Cjb7691GJ8/?t=3391) **Harjot Gill:** and a single person like me I can drive 2 or 3 million dollars of tokens a year

[56:36](https://realalpha.app/v/4Cjb7691GJ8/?t=3396) like blended models not just always using the best model. And perhaps it should even be more once we have the factory concept taking off. So my point here was like the bottlenecks are now shifting in how much you can like drive these tokens, number of agents, the CI, and the other infrastructure. Humans can make a lot more decisions in a day to drive. millions of dollars spent now you need like it's almost like marketing team if you remember like i mean 10 people can drive 100 million dollars of spend same thing we're going to see with engineers now a small team can drive 150 million dollars of spend of 30 million in that company right and that's how you stay competitive because now you're going to compete with everyone who's writing software in an inefficient way not in an economically viable way if you're going to mass produce you're going to see like companies composite startups now pinup, which will be a brand, but with like many products, they're going to mass produce, just like how you do in a factory. I feel you're still a huge exception in everyone I talk to, including inside of NVIDIA, there's a monthly budget of $10,000, and you can go above that, but most devs I talk to don't go about that. So I'll just play a little

[57:42](https://realalpha.app/v/4Cjb7691GJ8/?t=3462) bit of devil's advocate with you. So you're spending a lot on AI, like way more than most. You're also a founder, So you have a really good idea of where the business is and what value is adding. What value do you think you and people spending this much is adding in terms of like, does this mean that you're clear you're moving faster? But what is this resulting in? Are you getting more customers? Are customers spending more? Are you just, I don't know, getting more free user? You see what I mean? Because you're in a great position. Usually for an engineer, it will be maybe harder to connect. But you can clearly connect like what either you're seeing right now or what you're expecting to see as a result of this investment and spend.

[58:19](https://realalpha.app/v/4Cjb7691GJ8/?t=3499) Yeah, all of the above. Like one of the things you're going to see is like this point SaaS solutions, the days are over. Like now you're going to see really like companies doing a lot more software, right? And this is adding value in all parts of our stack. First of all, like, I mean, as I mentioned, like earlier, I would like rely on vendors for some of the things. Now you can just build in-house much cheaper. There are many tools like CI, we moved in-house, we moved vector databases in-house, a bunch of this stuff. Those are very clear areas. You can like go and build those differentiators quickly, right? And the other thing is even for the market, Like the people have infinite cap appetite for software and these solutions, right?

[58:52](https://realalpha.app/v/4Cjb7691GJ8/?t=3532) So there are companies which are going to like, for example, we are in the SDLC space. So all of our customers want to go and ship faster. Code review is not the only bottleneck they're running. Now they're on the market looking for the next thing, which is how do I solve my CI bottleneck? How do I do this? Like, so we're in that business. So we have to, first of all, go and learn quickly where the bottlenecks are going to be. I mean, we're getting arrows in the back, front, whatever, right? But what that also puts us uniquely in a position is that we can offer these products faster to the customers.

[59:17](https://realalpha.app/v/4Cjb7691GJ8/?t=3557) but even everyone wants to shift faster like that's the thing with with the with the vc funded companies right so that the the kind of like the range of tech you want to go quickly build that's your usually or more like i mean you have like a few months now the problem with the with the software right now is like even like so a lot of these startups they used to have a moat of a few months before they got copied but now they're getting copied within days and weeks so you can't just go to the market in this competitive market with a idea which is shallow like you you you get like even um and tropics gonna copy you in like two weeks right now oh so i see where you're coming from but i still want to push back on the fact that my sense is okay you're spending this crazy amount of money you know you personally are spending or you're on track to spend like two million per year or let's just talk about per month like you're you're you and the team maybe are spending 500k in this month maybe next month will be 600 maybe then 700 but in return what you're getting is you know you you are in the dev tool space with ai code review you're now realizing oh dang we also have this bottleneck with ci maybe we'll do something with ci or help

[1:00:21](https://realalpha.app/v/4Cjb7691GJ8/?t=3621) our customers do something with ci and we do this because we're now so fast we're seeing where they're going to be in like six months or 12 months so now we're now going to start this product we'll spend more then we realize maybe it'll be a feature flag and we'll be able to roll out and so on and by doing this you're almost like doing r&d for your business and you're also building your products and the bet is that you're you'll be expanding your market you'll sell more to customers but what i'm wondering is are you not a little bit unique that this might be true for anyone who's in a dev tools market but for some some company in any other market as well like so basically now you're moving in a compressed time frame Let's say it took you like a while to go to market and do a PMF, six, nine months if you're a YC company, right? Now you can like go there and try that idea quickly. And if it's a shallow idea, now the risk is you can get copied. So basically, we'll probably have to like do more, but you are now moving at a compressed timeframe. The earlier the bottlenecks were in scaling the team, right? So I needed to have the talent. You will like get all this VC money, scale the team. I don't need to scale the team at the same level, right?

[1:01:21](https://realalpha.app/v/4Cjb7691GJ8/?t=3681) I mean, maybe the sales still need to scale because they have to do the meetings, but product, marketing, engineering, you can now do few people burning a lot more tokens, right? And this token spend right now, if you, and it's quality tokens, like you can get cheap tokens, it will take you longer to get there. If you have the quality tokens, your price per task will be high, but you get there faster at a high quality, right? Okay, so what I'm hearing is what you're actually saying with this tweet, which wasn't that as nuances. If you are in the VC game, meaning you're raising venture capital because you want to build a billion-dollar company, which means you actually want to win your market.

[1:01:57](https://realalpha.app/v/4Cjb7691GJ8/?t=3717) Like inside of DevTools, you might have a niche. May that be like AI code review or AISRE or CICD, something like that. If you're starting today, you will probably need to raise a larger amount because you need to win this quickly. The game will be over pretty quickly and you want to get meaningful traction before the big guys and Tropic OpenAI can maybe copy you. Oh, and as an added benefit, you can have a small team to start with and you don't need to worry too much about hiring. But by the time you would hire anyway, you will want to build some solution and get customers and so on. So like everything is just compressed and you can pay more.

[1:02:33](https://realalpha.app/v/4Cjb7691GJ8/?t=3753) Right. I mean, the attention is compressed. Like if you don't get the mind share after building your MVP in like maybe one or two months, that's the window you have now. before someone just gobbles up that idea and runs with it. Okay, so that's pretty brutal. Yeah, I mean, that's the situation right now we're dealing with. Like the window of opportunity to get the mindshed has become shorter. It used to be that you had smart people, they had built something, right? And maybe it was a lot of product iterations. It was a SaaS product with a lot of like graft and care, or it was a complex backend system which just had a moat.

[1:03:06](https://realalpha.app/v/4Cjb7691GJ8/?t=3786) Like people couldn't replicate easily. So they would have like a shelf life of, I don't know, six to nine months before something. And they had that breathing room to get the mindshare. But now it's like a few weeks, unfortunately. And the company that's like spending a lot on tokens and all can get there faster. So you're going to see more companies like Rippling, which was a composite startup. You're going to see a lot more composite companies with these very efficient factories at the processes, gobbling up a lot of that software market. You will still have like people tailoring at the last mile.

[1:03:35](https://realalpha.app/v/4Cjb7691GJ8/?t=3815) But if you're buying these SaaS solutions and all, that's where we're going to see a big disruption. Okay. No, I think this is fascinating insight. And I mean, just a caveat, you're seeing this in Silicon Valley where everything is accelerated, always has been compared to the rest of the world. Let's say, you know, Germany will be always a lot slower, right? We've seen that with the Industrial Revolution, same thing happened. Like, I mean, one example is like, for example, toys, like an example, like earlier people used to make metal toys and very good craft. Even I grew up getting those Matchbox cars, like all metal, pretty good.

[1:04:02](https://realalpha.app/v/4Cjb7691GJ8/?t=3842) Oh, yeah, the old Matchbox, they were a lot heavier, right? When I was a kid, they were really heavy, really good quality, but now you're seeing it's all plastic. But mass-produced, the market has an appetite for that. So yes, the quality is going to go down, right? But the companies that are going to invest in these factories, which need now actually as much investment as running a factory, it's almost like building a steel plant now, 50 million out of the gate, are going to mass-produce software. But just to push you a little bit more on that, because I really want to push you on this year these software factories probably cost so much because there's not a turnkey solution and let's say given it's everything is so fast in a year there might already be software factories you can buy but you're saying that by that time i mean first of all everyone can have access to it and your window of opportunity might have closed for a lot of businesses software factories are expensive right now tokens but then cpu all the um like we talked about the ci like you can drive a a lot.

[1:04:55](https://realalpha.app/v/4Cjb7691GJ8/?t=3895) spend just with the velocity alone a single person cannot drive it right so so the economics are going to force you into that factory model it's almost like how like we've seen in the industry even like like let's take another example like watches like when the quartz watches came out you could mass produce and then you pay for rolex which is crafted now the question is like would artisanal software which is very well crafted people took a lot of time will command a premium or is the market just going to be okay with generic looking maybe slightly lower quality software which i think is what the case is and that software is going to be designed in a factory yeah no i i see where you're coming from and i really want to disagree with you in a lot of ways especially but i i see your point and i think i have a hard time disagreeing with the fact that look if you do want to win a market right now you want to give yourself the best time the best opportunity and and if if the if the theory is that to win you you want to be either either first or fast and iterate and get it right, you have less time because an interesting thing that I'm also seeing, which is one more thing we could reflect on is,

[1:05:57](https://realalpha.app/v/4Cjb7691GJ8/?t=3957) so far we've only talked about, we assume that your composition is either the big labs who see what everyone else is doing and they launch something, or the other startups who some of them raise more money and might be faster than you in their software factory. But the reality is, everyone else is a competitor, like Ramp launched their coding agent. Spotify is launching a platform for feature flags. So you have the existing tech companies who are also realizing, hang on, we built this tool inside because we're now big and we see and we think there might be an appetite. And what you said is every small company, like 50-person company like yourself, will need the tools that Netflix has.

[1:06:32](https://realalpha.app/v/4Cjb7691GJ8/?t=3992) Netflix at some point might say, like, hang on, let's open source. So now I feel there's just a lot more competition for the same thing, just coding harness. Again, what have you thought that a credit card company would have a coding agent and people take it seriously? Right. And the thing is that whoever is not on this parabolic trajectory you just saw, they're going to get disrupted. A lot of people are not even ready for what we're seeing already. Like their culture is not going to let them. So the way they're doing software development, like they have to rethink everything. Like first of all, how many people they need. Like people who are just good at writing code, they should probably look for other jobs if they don't know what to build.

[1:07:08](https://realalpha.app/v/4Cjb7691GJ8/?t=4028) Like people who are not software engineers, like artists, people who have taste, maybe should start coding. And then how do you allocate budgets? Like you have to now really think about if you're hiring someone at like, I don't know, half a million a year, expect they're going to spend more than a million on tokens now. So earlier, like when you do this plan to show the VCs, you will have the salary as the main component. But now we're starting to show tokens, right? Yeah, well, again, I just want to be clear, like this is for like a new startup VC building from zero because for existing companies that have a business, often like physical or in the world, okay, they could be disrupted, but, you know, they have something stable to be there. And they're the companies who are spending the $200 max plan for everyone, right?

[1:07:44](https://realalpha.app/v/4Cjb7691GJ8/?t=4064) I mean, if you have a hardware business and all, you'll probably have a better moat at this time. But if you're like just software with very high margins, those margins are going to compress. And you probably also have to like rethink. I mean, if you're not on the parabolic, then companies coming from behind. Now, this exponential is scary. I mean, these kinds of things start world wars, right? It's that scary. So that kind of companies are going to start coming from behind and start decimating some of these lower dinosaurs, unless they go and transform themselves on this metabolic. And those companies, they're actually investing in factories more than talent.

[1:08:14](https://realalpha.app/v/4Cjb7691GJ8/?t=4094) They're investing in just, I mean, paying for the best model possible. They're not cutting corners. They're just going fast. Let's talk a bit about hiring because this is also a very, very interesting topic. First, can you tell me how did CodeRabbit grow over the years and how has your hiring changed? Because you did mention that you're hiring a lot slower, right? In some areas, yes. In many areas, we are hiring rapidly. Like sales is one area where we have really doubled down. on the field sales because one of the things we found with um generative ai and is that the last mile is still a problem like people still need a lot of hand holding in how to set up the systems in the right way yes so those kind of things you can accelerate with ai you have to have those meetings you have to have on So those areas we are hiring rapidly, but other areas now we are like rethinking what we need to hire for.

[1:09:01](https://realalpha.app/v/4Cjb7691GJ8/?t=4141) Let's talk about software engineering specifically. Software engineering is the most trickiest area right now. Every month we are learning something. Like one of the things we have found is like the problem solving skills, system skills, like people like who came from like even I wrote a tweet about like I'm looking for people who have played strategy games. Let me read this tweet because that was a really interesting one. So you are hiring and you said, I mean, there's a KB. we're hiring the person who applies must meet one of the following two criterias to join either has tastefully shipped at least 1 000 prs in the last 30 days or has played at least 500 hours of simulation games such as civilization age of empires transport tycoon factory etc almost like now if you ask me about the software building looks feels like it's like playing one of those games right now i mean i talk about civilization like each like you basically make the moves and each turn is like a release for me right and you learn something in the next turn and while you are steering the civilization you still have like i don't know workers improving your roads and building fields and so on that's how the agents are and each turn you're like just making these

[1:10:04](https://realalpha.app/v/4Cjb7691GJ8/?t=4204) agents work on something meaningful right which we think is is a great ROI for your civilization right almost like now running these agents feels to me feels like buildings if people have grown playing strategy games i think they will be very successful in with these agents right now They can multitask. They have like a range of things. They can, I mean, at least they have a good problem that these agents can then work off. And how is hiring going for you so far? How easy or more like hard are you finding to hire these folks who have this kind of experience? So the way we are looking at it is like either someone who's already a tenured senior engineer at a good place, they have to come and first unlearn, right?

[1:10:43](https://realalpha.app/v/4Cjb7691GJ8/?t=4243) Or you hire someone even who's like fresh out of college, but they have high agency, but can learn faster. what we found that the people who are like coming with like are not set in the ways and have come especially from the background of like like playing this like strategy games and all have been very successful that's just my take on it and that's what we're looking for i think

[1:11:03](https://realalpha.app/v/4Cjb7691GJ8/?t=4263) **Harjot Gill:** the human agency now matters a lot more than it ever did person who was 10x knew what to build is

[1:11:08](https://realalpha.app/v/4Cjb7691GJ8/?t=4268) now suddenly becoming 100x with these agents right because now you at least have a problem list or the the most impactful areas these agents can work on and you could just unleash these agents as much as possible in parallel. So we talk about individuals, but I know you have a concept called pods that you also call it as war rooms. That's right. What are our war rooms? So what we found is like, that's another area, like I think we have to, like the collaboration has become even more important. So I've like in previous startups, on remote teams, we are like building distributed systems. So it doesn't matter like teams like in India or Eastern Europe, I've done all.

[1:11:40](https://realalpha.app/v/4Cjb7691GJ8/?t=4300) But now like what we're seeing is that now we are moving so fast, like somewhere one person is on the steering wheel, but other people are shaping the software as it's getting built. And everyone is able to touch the software, by the way. Even product managers get to touch it. And the kind of PRs they're making is relevant to change a copy on something or some information architecture flows, right? But everyone's touching the software at the same time. And it's great if they can be in a wallroom setting. So we kind of create these wallrooms where we share plans on a Slack canvas. We all agree on it.

[1:12:09](https://realalpha.app/v/4Cjb7691GJ8/?t=4329) And we all collaborate in a room in a more face-to-face setting. So you get everyone in a room, into a table. It's all real time. You kind of close the door and like, all right, let's solve the problem. And you're crafting this out in real time. Sometimes you will lay the foundation by running a goal, but when you're at the stage where you want to really craft the UI and experience, you want to get people in the room and really craft it live. Yeah, it's funny. At Uber, we used to have up to 2017 something called the War Room, which is like this. There's an important project, get everyone in, sometimes fly people in because they're distributed, and then spend maybe a week there, build up.

[1:12:39](https://realalpha.app/v/4Cjb7691GJ8/?t=4359) This was pre-AI, so with AI, it's a lot faster, but just build a proof of concept, whiteboard, code, and then have a prototype and then everyone goes back to their teams and just kind of builds it from there. But it's now working, the core thing. Yeah, now we're doing that more often. Like that's exactly the idea. And you need like people with different... It's like you have product managers in the same room, you have designers, you have people who understand back end, the front end, and they are like crafting the software together. It is hard work and sometimes it is late work. Because one of our first interactions on X was, I think I was writing something about how teams are working really late up to the evening or into the night.

[1:13:14](https://realalpha.app/v/4Cjb7691GJ8/?t=4394) And you replied, it's almost 2.30 a.m. and we're still shipping. And you're like, yeah, the kind of crying smiley face. I'm not sure this is good or bad. but can you tell me a bit on on the pace you talked about you're not sleeping as much before the recording we talked about how like okay at some point you need to get that under control because clearly there's a time for excitement but also when you're thinking of building something longer term like you'll you'll want to find the balance right how are you thinking about it it's crazy right now like i mean it does like impact your health to some extent because ever since these agents gotten better as i said like there's like a huge opportunity cost if they are not working all the time and they do stall they have questions you have to like steer them i read the reasoning traces i guide them whenever i have to and for that like i mean i've almost like changed my sleep cycle i mean i take a nap i wake up at like 3 a.m to send like a few instructions because every hour like one or two agents are going to get stuck and you have to go and like it's almost like Roomba is getting stuck you have to like unstuck them that's what it feels like but then then that's kind of become a habit but it's not sustainable and we haven't like

[1:14:15](https://realalpha.app/v/4Cjb7691GJ8/?t=4455) figured out the right interface to work with these agent fleet of agents at scale maybe it's really like a factory like companies have to have a morning shift night shift because instead of owning the code maybe you're now owning agents during the day i make the decisions but i can be rest assured we are seeing the same session the same agent that other people can make decisions while i'm sleeping right so we are we are right now because we're developing on our own machines we are like bottlenecked on on that collaboration right now yeah the the one thing i'll add is i I wrote this book called Why We Sleep.

[1:14:45](https://realalpha.app/v/4Cjb7691GJ8/?t=4485) I read it a while back. And it actually forces me to prioritize sleep because there's this thing with sleep, which there's some scientific evidence. I'm not an expert in this, but there's a suggestion that not getting enough sleep now, it's fine for tomorrow and it's fine for next week and it's fine for next month. But decades later, it can come back and bite you. And ever since, I don't know how scientifically sound it is, But just as when I was like, oh, it could happen, I'm like, OK, so like me getting a good night's sleep is not only important for me for tomorrow, because I'll be, you know, like you'll know, like you won't do as well.

[1:15:22](https://realalpha.app/v/4Cjb7691GJ8/?t=4522) But for my long term health, like I, you know, I'm starting to hopefully try to do stuff that will make me more healthy when I'm 60 or 70, et cetera. So that's my only kind of like addition to like not sleeping. And I would push a little bit like, is it as important to wake up and steer the agent? Can I not wait until the morning? So I think it's more like a polyphasic sleep. Maybe I'm like exaggerating the health. But I think even if you look at how people used to work pre-industrial AI, this 9 to 5 thing is a recent thing. And I mean, I come from like agriculture community. So people would wake up, tend to livestock because you are like working based on other aspects and you have to wake up and do that thing.

[1:15:59](https://realalpha.app/v/4Cjb7691GJ8/?t=4559) Now it's the agents. You have to wake up to tend to that livestock, which is now agents, almost like farmville. I don't know whether that's the right analogy. So people used to have polyphasic. Or like tamagotchi for a little bit. Yes. So people used to have polyphasic sleep and I'm doing that right now. Oh, you're right. Like this was a thing in the Middle Ages. People would wake up in the middle of the night and they would often like dine and stuff like that and then go back to sleep. That's what I'm doing right now. But I will say the only reason I'm doing that right now is because I have to tend to the agents which are on my local machine.

[1:16:28](https://realalpha.app/v/4Cjb7691GJ8/?t=4588) Now, once you've really figured out the right abstraction of the factory, once you move to the cloud, then maybe it's going to be like a factory, like nine to five, morning shift, evening shift, or like round the clock. Because these are not physical factories. You can have all of the sun model. Like we're still figuring out. But right now, it looks more like polyphysic with this generation of agents, at least for me. Yeah. Let's talk a little bit about burnout. You wrote about burnout, and I'll quote you here. CodeRabbit is my third startup now, and I have worked just as hard for my previous.

[1:16:55](https://realalpha.app/v/4Cjb7691GJ8/?t=4615) To NetSail and Flux Ninja, burnout never happens due to hard work. It kicks in when your work does not lead to desired outcome for a long time. On the family front, my two daughters were born during startups. They actually help prevent burnouts. In just your personal experience, like how have you thought about burnout and also just working so damn hard? Because it sounds like, you know, you're on third startup. Like you don't seem to be the, you're the intense type. Yeah, I mean, I really, when the second startup wasn't doing well, I mean, that's where I did feel dragged down, burnt out. because there was no outcome.

[1:17:26](https://realalpha.app/v/4Cjb7691GJ8/?t=4646) That's the first time I felt the signs of a burnout. Because yes, you were like working very hard, but without a very clear result or an outcome, just waiting, the anxiety kills you. Nothing's happening. You're doing all this work, but nothing's happening, nothing's moving. So that's a bad scenario to be. And I think that's a status quo, which I would never want to go back to. But for CodeRabbit, like I'm working hard, but things are on the upside. Like we are like seeing the revenue grow like crazy. Our team is growing, the funding's coming in. It's always something happening. We are actually moving the needle as well.

[1:17:56](https://realalpha.app/v/4Cjb7691GJ8/?t=4676) Customers are loving the product, the new capabilities. So even though I'm working hard, but I don't feel that I'm burning out or I have to stop at this point. And I think I can keep going on. And I wonder, like, I think in your case, it makes a lot of sense because you were founder in all the companies. So you immediately, you were connected to the outcome or, you know, for example, if the revenue goes up, you know, this is just good because it will help your company grow or raise the next round or get a sale or whatever, you know, like, you know what can come. I'm wondering just being empathetic for employees who are not founders themselves they might have equity but it will not be as big of a stake could it be that having autonomy having visibility for the business just knowing how their work is connected that it might actually help because inside of people typically work hard and long especially early stage but I wonder if the autonomy here and and seeing how your work is connected to the things that actually just helps you know get this kind of kick that you're you're getting basically yeah it motivates people Like, first of all, we don't force people to work like I do.

[1:18:55](https://realalpha.app/v/4Cjb7691GJ8/?t=4735) I mean, I lead by example. That's what I believe. And even if people can follow to some extent, it's great. But I do what I do. Like, I mean, rather than managing the people, I'm on the ground in the trenches with the developers. If there's incidents, I'm always like first there to help them out, no matter what the time. And I think that does motivate people. They're all like happy that, okay, the leader's right. They feel like much more empathy to their life, right? Secondly, like I found like the people who join startups are looking for this adventure. like it's a non-linear career trajectory so they are living leaving like very predictable salary rsus in some big company right to get like cheaper stock at startups but then they're looking at the upside and when that happens it's like even more inspiring right when the companies are on the upward trajectory so i would say like when the people who go and join the startups are looking for breaking that status quo which things can be very mundane and boring in a larger setting yeah and you're right like on a fast growth startup there are a lot of advantages like Like either, you know, if the fast-food startup goes great and you have equity and there's an exit, you could have life-changing amount of money, meaning that you can buy a house, you could have education set up for your kids.

[1:20:02](https://realalpha.app/v/4Cjb7691GJ8/?t=4802) And you now have the freedom to like either take a risky startup or do a boring job, whatever you want. And if it goes like this and it drops, for example, fast was a typical example in the, I think in 2022, which went bankrupt. You often, if the startup attracted really good people, you know, those people all go to different companies because now their company shut down. Yeah. But there's also often this network, this camaraderie that was there and they often call each other years later. They will some of them will end up at the next, you know, like really successful company. So I've actually changed my mind. And OK, the only downside, I guess, joining a startup is often you will need to work hard.

[1:20:38](https://realalpha.app/v/4Cjb7691GJ8/?t=4838) It will be often be in person. But typically when you're young and when this is not you don't have other huge priorities in your life, it can be a huge career boost. it is like even financially i wouldn't say it's crazy like that that's how the silicon valley is like i think it does reward people who took that kind of a risk i mean there's no fatal failure Somehow. Yeah. Like, and I've seen like people like who just been through that experience have gained a lot in like, even like if it's like a bad lesson, at least they know what is not going to work. Yeah. And that's valued in whatever the next job they're going to go look out for.

[1:21:11](https://realalpha.app/v/4Cjb7691GJ8/?t=4871) You're kind of saying it might be more valuable than sitting in one place, not doing much, not learning much, right? that's where you don't grow but if you go out and do this thing risky i mean there's no fatal failure there like the people love even in the valley at least people have taken risk whether your founders or early employees are foolish enough to join but but at the same time people reward that experience right and if it's successful then it's like you you're on an accelerated trajectory like and i think that's what i would encourage like even though yes i mean it's becoming a little bit cost prohibitive on the factory front but still it's still the best time for people to go out and try this out.

[1:21:44](https://realalpha.app/v/4Cjb7691GJ8/?t=4904) Like before all these like factories become more established and start blocking some of these individuals. But right now, it's a very magical time right now. Like in a single person, just driving these agents in the right way can create immense value. I wanted to ask that looking back pre-AI to like your 25-ish years as a software engineer, as a founder, what are the things that you learned there or skills that you built that you think are really useful and helpful right now working with AI and in this like, I guess, AI era? I mean, in terms of like entrepreneurship, it's always been like, I mean, again, it's almost like a video game.

[1:22:17](https://realalpha.app/v/4Cjb7691GJ8/?t=4937) Like, I mean, you're playing all these like games where, yes, you have to think about like the product market fit. I mean, all that is exciting. I think what I love is like creating value, like that's the end of the day. Like software is just means to an end. Yes, you have to write code. And I enjoyed it, but it was like just means to an end for me. Like the real thing was like our people, like it's really like making someone happy with the value you provide is always like the biggest reward, right? Everything is like then follows. Like you will get the VC money, you will get the team and people will join the mission.

[1:22:45](https://realalpha.app/v/4Cjb7691GJ8/?t=4965) So it's almost like being a missionary in this kind of a world, like you're spreading a religion. And that's the fun part of doing a startup. Like, I mean, I mean, I've always been on the frontier. Like I've seen like early telecom, I've seen SaaS, cloud takeoff. And each time it's always what excites me is like, OK, there's a lot of nonlinear value going to be created at the frontier then at other places. And I always try to be at the frontier. What would you advise a software engineer who one day would like to work at CodeRabbit or at a place like CodeRabbit, just as a fast-moving place? What kind of skills should they invest in?

[1:23:19](https://realalpha.app/v/4Cjb7691GJ8/?t=4999) What kind of practices? What kind of activities should they do outside of playing 500 pounds of factorial? Yeah, I think the agency is the number one thing I'm looking for right now when we interview someone. Because right now, even if they're in a bigger company, because a lot is happening on the side, if they're not doing side projects today, it's a red flag. because right now it's so easy to build that people who have agency are actually building something right now. They can hold themselves back. And that's all always like a great sign. If you have side projects, I love it, right? And the second thing we are looking for is like people who have a range of experience now.

[1:23:53](https://realalpha.app/v/4Cjb7691GJ8/?t=5033) For the depth, you have AI. It can go deep, drill, infinite to performance optimize something else. But if you have like a range of experience, you can like build, let's say distributed systems. Let's say you came in from one space doing, let's say observability databases. Now, if you were to give agents, now you can do 10x of that. You're not just building that, you're building 10 other types of distributed systems. So the range also matters now. So those kinds of things I'm looking for, right? I mean, it's like systems thinking, whether they have range in skills and a high agency to keep these agents busy.

[1:24:26](https://realalpha.app/v/4Cjb7691GJ8/?t=5066) As closing, what's a book or a paper you would recommend that you read and you found it interesting or fascinating? I think we're living in a black swan moment. I read this book. I mean, I'm not like a, I spend a lot of time reading books these days. i've just been like building um but i think we are in that like the black swan zone like what we are seeing right now and what was the book i mean this was like the black swan book i think i forgot who the author was but it was really yeah we'll put it in the show notes below yeah yeah yeah but when there was like pretty insightful about what we are right now going through is a kind of Like if you look at five years back, I would not have predicted that AI would get so good that it's first of all like driving my cars, it's driving the software and it's like even some of the capabilities we're still like discovering with what these models are capable of doing and we have not even hit the peak yet.

[1:25:11](https://realalpha.app/v/4Cjb7691GJ8/?t=5111) So yeah, so everything is going to change and some of the things we can't even predict today the way we're going to see the after effects. Well, Harjit, thank you. Yeah, thanks. Well, I cannot recall talking with anyone nearly as AI-pill-less Harjit is. To be clear, I was skeptical to start with when hearing about his $200,000 AI bill for a single month for a single person. And I remain pretty skeptical about several things. But Harjit explaining his thinking put a few things into perspective. Let's start with the part that I cannot agree with or recommend. Harjit's approach to waking up in the middle of the night to prompt his agents.

[1:25:45](https://realalpha.app/v/4Cjb7691GJ8/?t=5145) He calls this polymorphic sleep. But to me, it sounds like deprioritizing sleep all to unblock agents. This feels very inefficient to me. And I would rather look at wrong or learning tasks in the cloud if you really want to be there to have your agents running it all the time, even when you sleep. But really, I don't see the obsession with agents always running. This is not a competition about utilizing your local laptop to always run at 100%. The part on how Harchet doesn't think a startup with $8 million of funding has enough money to build a software startup, I'm also skeptical. I get it that when Harjit assumes that all startups will send millions of AI like his startup does with 40 people, you burn through funding fast. I also get it that in DevTools, it can make business sense to build new products like a custom CI really fast like CodeRabbit did. But outside of startups that sell AI DevTools, I just don't see it making sense to spend massively on AI just to be a little bit faster to market than your competition. The part I do get though is how much competition across AI DevTools startup is there today. It's not just CodeRabbit launching new products with their 40% engineering team. It's also larger companies like

[1:26:49](https://realalpha.app/v/4Cjb7691GJ8/?t=5209) Ramp or Shopify launching DevTools products, AI Labs and other large companies all trying to get the piece of the same market. So with this context I do get why speed matters especially for small startups and why spending on AI to build so faster can make some sense. Finally one practice I really liked is the concept of desk losing their driving license, if they cause too many regressions. AI pilled or not, I like this idea to force engineers to care more about reliability and shipping quality code. For more deep dives on software factories and engineering practices with AI, check out the link to Primatic Engineer Deep Dives below. If you've enjoyed this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating on the show. Thanks, and see you in the next one.
