Code quality contributed double-digit productivity gains at Meta

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 17:05

   And the idea was, I think it's sort of like 2016 or 2018 or something. But Zuck mandated that every engineer at the company, 20% of their time has to be spent fixing tech debt. Oh, interesting. And we called this Better Engineering. And some of this is kind of bottom up where, you know, a team knows best the tech debt that they have to fix. And then some of it is dropped down where you need to do, you know, very big migrations.

2. 17:30

   You need to migrate to new language features, new frameworks, things like this. And at Facebook scale, you know, there was tens of thousands of these migrations every year. And so I just started leading all this. And I realized very quick that you just need a little bit more order to it. There was no goals. No one knew kind of like what the outcomes were. There wasn't any tracking. And so we developed a bunch of stuff.

3. 17:52

   One of the ideas was a centralized way to prioritize the different kind of code quality efforts. The second thing was figuring out the impact of code quality on engineering productivity, which turned out to be significant. How did you measure? What did you find there? There was a bunch of stuff. I think some of this has been published. I don't know if all of it has, but essentially you try to do like causal analysis and causal inference.

4. 18:13

   This is the methodology. You try to figure out like what are the factors that make it so engineers are more productive. Some of it is code quality. Some of it is outside of code quality. So, for example, meta went back to, you know, return to office instead of work from home. That was partially driven by this because we just found some, you know, fairly strong relations that we thought were causal about this.

5. 18:32

   But code quality actually contributes like, you know, double digit percent to productivity, it turns out, even at the biggest scale. It's kind of comforting to hear because I think it's rare to have a place where you actually measure this, but I think we feel it. Like when you have a clean code base, modular, or it can get easier to work with. And I think, you know, reasoning, could it also be easier for LLMs to work with it?

6. 18:56

   And my hint would be, yes, it should be, right? But I think there's just very little data. But that's a feeling that I would have. Yeah, I think a lot of the big companies have published about this. Like, I think Facebook published something. Microsoft publishes a bunch about this. Google does. But yeah, totally. If every time that you build a feature, you have to think about, do I use framework X or Y or Z?

7. 19:17

   These are all options that you can consider because the code base is in a partially migrated state where all of these are around the code somewhere. As an engineer, you're going to have a bad time. As a new hire, you're going to have a bad time. As a model, you might just pick the wrong thing. And then, you know, like the user has to course correct you. So actually, you know, the better thing to do is just always have, you know, a clean code base.

8. 19:38

   Always make sure that when you start a migration, you finish the migration. And this is great for engineers. And nowadays it's great for models too.