Claude Code coincided with a 200% rise in PRs per engineer

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 21:14

   Yeah, and the gains have just been insane. Like, Adanthropic, over the last year, like since we introduced QuadCode, we probably, I don't know the exact number, we probably like 4X the engineering team or something like this. but productivity per engineer has increased 200% in terms of like pull requests. And like this number is just crazy for anyone that actually works in the space and works on dev productivity.

2. 21:35

   Because back in a previous life, I was at Meta and one of my responsibilities was code quality for the company. So this is like all of our code bases, that was my responsibility, like Facebook, Instagram, WhatsApp, all this stuff. And a lot of that was about productivity because if you make the code higher quality, then engineers are more productive. and things that we saw is you know in a year with hundreds of engineers working on it you would see a gain of like a few percentage points of productivity something like this and so nowadays seeing these gains of just hundreds of percentage points it's just absolutely insane what's also insane is just how normalized this has all been like we hear these numbers like of course AI is doing this to us it's just it's so unprecedented the amount of change that is happening to software

3. 22:16

   development to building products to just this the world of tech it's just like so easy to get used to it, but it's important to recognize this is crazy. This is something like I have to remind myself once in a while, there's sort of like a downside of this because the model changes. So there's actually like, there's many kind of downsides that we could talk about. But I think one of them on a personal level is the model changes so often that I sometimes get stuck in this like old way of thinking about it. And I even find that like new people on the team or even new grads that join do stuff in a more kind of like AGI forward way than I do. So like sometimes, for example, I had this case like a couple months ago where there was a memory leak. And so like what this is, is, you know, like quad code, the memory usage

4. 23:00

   is going up and at some point it crashes. This is like a very common kind of engineering problem that, you know, every engineer has debugged a thousand times. And traditionally the way that you do it is you take a heap snapshot, you put it into a special debugger, you kind of figure out what's going on, you know, use these special tools to see what's happening. And I was doing this and I was kind of like looking through these traces and trying to figure out what was going on.

5. 23:22

   And the engineer that was newer on the team just had QuadCode do it. And it was like, hey, Quad, it seems like there's a leak and you figure it out. And so like QuadCode did exactly the same thing that I was doing. It took the heap snapshot. It wrote a little tool for itself so it can kind of like analyze it itself. It was sort of like a just in time program. And it found the issue and put up a pull request faster than I could. So it's something where like, for those of us that have been using the model for a long time, you still have to kind of transport yourself to the current moment and not get stuck back in an old model because it's not Sonnet 3.5 anymore. The new models are just completely, completely different. And just this mindset shift is very