Humans own system design for Claude-written code

Transcript

From the transcript Latent Space publishes with the episode.

1. 1:21:02

   which I, which I thought would be fun to sort of bounce back and forth between the two of you. Um, how do you fight slop on your teams? Uh- Yeah... so we actually, this has been an interesting debate within our team. The, the-- we're no different than anybody else, right? I think plenty of teams are talking about this. Um, and what's interesting is what we're trying to figure out is if you're producing a document, you're producing something that you want other people to consume, code, documents, whatever, right?

2. 1:21:28

   If, like, you should use Claude to help you, of course, right? But if you use Claude to, like, totally generate a thing, you didn't spend any time on, like, actually reading it yourself and making sure that it was something of high quality, then you're putting a whole extra burden on the consumer [laughs] of that document or of that code or whatever to spend a whole bunch of time trying to parse it, and that's just, like, insane and totally unfair.

3. 1:21:51

   But what's funny is we, you know, we push back, we push on our team like, "You can't do that," right? Like, if you're producing something, use Claude, sure, whatever, but, like, you have to make sure that it's a high-quality thing. But then we get into these, like- Yeah, you as the human are still responsible for the quality. Right. You're still responsible for the quality, but then you get into this really funny thing where because of the Claude-isms, even just like silly, like, I don't know, there's like an em dash in the middle of the thing, right?

4. 1:22:13

   Like, people's brains trigger on this like, "Oh, no, I'm reading something that was written by Claude and not a human, and I'm, like, annoyed because I wanna be r- like reading a thing that was, like, fully produced by..." Not necessarily fully produced by a human, but, like, that has the quality bar that the human would hold, right? And so then we get into funny situations where maybe actually, like, the person did spend a lot of time making sure the thing was high quality, but they used Claude to help them, but tho- there's those little hints of Claude, and then the reader is, like, turned off, you know?

5. 1:22:43

   Mm. And, like, that's, like, unproductive as well. And so what we have kind of tried to do is, like, set a bar within our team that's very much like use Claude to help you because it would be really silly if you didn't use this hilariously powerful agent to speed you up in your job and in your life or to just, like, help you produce better things. But it has to, like, have a certain bar, and then for the consumer and the reader, if, like, it really actually is just slop, like, send it back. [laughs] Right? Like, just say no, just send it back.

6. 1:23:10

   Like, you're not gonna waste your time on this. But also don't be, like, too nitpicky about the little Claude-isms, right? Because, like, we have to find the right balance here. And so it's just really funny because we're, you know, we're going back and forth on this stuff within our teams, and just, like, so is everybody else in the world- Yeah... right now. Um, we're just not any different. But, but you have free tokens. [laughs] Uh, well, sure. But, um, I mean, it-- that doesn't tend to matter quite as much when it comes to like- As in, as in so you have more than anyone else in the world.

7. 1:23:38

   You probably see a lot- Yeah... you guys use a lot, you see a lot more people doing a lot more. Yeah. [laughs] Mm, yeah. Yeah. It's totally fair, but it's definitely, it's definitely a problem. Heavily discouraged. Yeah. [laughs] Yes. Uh, engineering slop? Yeah. I mean, it's the same thing, right? Same thing. Okay. Like, but, but I do think that where, where we've gotten to is a pretty good spot where, like, you know, humans play a large role in making sure, like, designs of systems are good, right?

8. 1:24:01

   And, like, implementation tends to be Claude, right? And, like, we're in a pretty good spot where for, like, a lot of the code that gets written, like, all you really need is just, like, a quick stamp, and you can keep going, and, like, you don't necessarily have someone, like, deeply reviewing every single line of the code 'cause you care more about, like, the overall plan, the overall, like, design is in good shape, less so than, like, the exact perfect implementation, and obviously rely on a whole bunch of verification to make sure the thing that you're gonna ship is high quality.

9. 1:24:30

   And so it's actually been, like, less of a conversation these days on, like, code quality being slop, in part because Claude's just, like, gotten better, but in part because we have a lot of the right verification in place to make sure that, like, you know- Okay... we're holding a high bar. I see. I see. Yeah. Trust your tests. Yeah. Um- Any product PM-ing slop, you can ship features a lot. You can- [laughs]... put stuff out in a day. Uh, so he said this to me recently. Mm.