Experienced engineers get more leverage from coding agents

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 11:01

   So maybe I can give you like a historical view of like what's been happening. Yeah, yeah. In terms of like running a team and so on. And so, you know, when we first started AR, we're still kind of autocomplete. I remember like at the time we were using Copilot and IntelliJ and we're like, oh my God, look, it autocompleted these tests. And we're like, yeah, but it's just autocomplete. And very rapidly, things started evolving.

2. 11:26

   And what I've noticed is that really good engineers are able to somehow put it into their workflows in a way where the AI starts training them. Oh, interesting. In terms of like how to talk to it, how to get the best out. By the way, as an aside, do you think that's model-specific? Do you learn how to use AI writ large, or do you get efficient at a given model? I think there's some nuances that different models bring. I'm sure, looking at your tweets, you've...

3. 12:07

   There's, like, nuances, but there is a way to work with AI. And, like, I think that there's also a way to work with different harnesses. And I think, like, you know, we tried, for example, using open code or, like, different harnesses or, like, using codecs versus cloud, like, cloud code. different ones have different ways of managing context or thinking through problems or when to spin up the right sub-agents and what to tell it, like the kind of prompts.

4. 12:35

   For example, with Cloud Code, I find that if I don't tell it, don't spin up 20 Fable agents to do basic research, it'll go and spin up 20 Fable agents to do research and burn all your tokens. So a lot of these things, you start learning about the harness and the models and like how they interact with each other and then how they can interact with the rest of the world. But I digress, like I'm just gonna, in my mind, like this is kind of what I started noticing is that like some people were adapting and learning and kind of doing it that way and others weren't as adaptive.

5. 13:12

   And then as we started hiring more people into the team, that disparity between, you know, okay engineers and really good engineers just multiplied 10x. Can you predict who will, could you predict who would be good? Was it like good engineers got better and bad engineers didn't or was it really okay? Yeah. So like if you're a good engineer, you're just a better good engineer. It's not like there's some like bad engineers all of a sudden became good engineers.

6. 13:40

   No. Okay. No, no. It always ends up being like you have to really understand ultimately the setup because the more complexity that your brain can handle, the more you can break down the tasks and kind of assign them i think this might actually be uh like non-consensus opinion that like uh i mean i i do hear it sometimes but the idea that like the better engineers like are much better those are less and you found this to be pretty universally true i mean look my team is still small yeah we haven't done it across a hundred people yeah in my experience for the things that we do that is the case because we build distributed systems and distributed algorithms and like your brain has to handle the complexity because like AI doesn't do it.

7. 14:27

   Right. Yeah. Yeah. And so for the problems that we solve none of the models can actually think through the things that we do. Like there's a lot of, for example, we have a query translation layer. We were just talking about query optimization. Yeah. And so you have to really understand the semantics of the query and how to get to the semantics of the query and then explain it to the model and then have the model go and build the translation layer, let's say.

8. 14:53

   Yeah. So for the people that are listening that may not be so deep in computer science, what do you mean by semantics?