Engineers still need to own judgment and system design

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 19:26

   is maybe what I would look at for the most part. And as agents do more, what human skill do you think becomes more valuable, not less? So, yeah, it's a good question. I think, well, right now the answer is that the agents are kind of like these intern entities, right? So it's remarkable. You basically still have to be in charge of the aesthetics, the judgment, the taste, and a little bit of oversight.

2. 19:51

   Maybe one of my favorite examples of, like, the weirdness of agents is for Menugen. you sign up with a Google account, but you... purchase credits using a Stripe account, and both of them have email addresses. And my agent actually tries to basically, like when you purchase credits, it assigned it using the email address from Stripe to the Google email address. Like there wasn't a persistent user ID that for people. It was trying to match up the email addresses, but you could use different email address for your Stripe and your Google, and basically would not associate the funds.

3. 20:26

   And so this is the kind of thing that these agents still will make mistakes about. It's like, why would you use email addresses to try to cross-correlate the funds? They can be arbitrary. You can use different emails, et cetera. Like, this is such a weird thing to do. So I think people have to be in charge of this spec, this plan, and I actually don't even like the plan mode. I would, I mean, obviously it's very useful, but I think there's something more general here where you have to work with your agent to design a spec that is very detailed, and maybe it's basically the docs, and then get the agents to write them.

4. 20:56

   and you're in charge of the oversight and the top level categories, but the agents are doing a lot of that under the hood. And so I think you're not caring about some of the details. So as an example also with arrays or tensors in neural networks, there's a ton of details between PyTorch and NumPy and all the different, like Pandas and so on, for all the different little API details. And I already forgot about the keepdims versus keepdim, or whether it's dim or axis or reshape or permute or transpose, I don't remember this stuff anymore, right?

5. 21:24

   Because you don't have to. This is the kind of details that are handled by the intern because they have very good recall. But you still have to know, for example, that there's underlying tensor, there's an underlying view, and then you can manipulate view of the same storage, or you can have different storage which would be less efficient. And so you still have to have an understanding of what this stuff is doing and some of the fundamentals so that you're not copying memory around unnecessarily and so on.

6. 21:48

   But the details of the APIs are not handed off. So you're in charge of the taste, the engineering, the design, and that it makes sense, and that you're asking for the right things, and that you're saying that, okay, these have to be unique user IDs that we're going to tie everything to. And so you're doing some of the design and development, and the engineers are doing the fill-in-the-blanks. And that's currently kind of like where we are, and I think that's what everyone, of course, is seeing, I think, right now.

7. 22:13

   Do you think there's a chance that this taste and judgment matters less over time, or will the ceiling just keep rising? Yeah, it's a good question. I would say, I mean, I'm hoping that it improves. I think probably the reason it does doesn't improve right now is again, it's not part of the RL. There's probably no aesthetics cost or reward, or it's not good enough or something like that. I do think that when you actually look at the code, sometimes I get a little bit of a heart attack because it's not like super amazing code necessarily all the time and it's very bloaty and there's a lot of copy paste and there's awkward abstractions that are brittle and like it works, but it's just really gross.

8. 22:54

   And I do hope that this can improve in future models. A good example also is this micro-GPT project where I was trying to simplify LLM training to be as simple as possible. The models hate this. They can't do it. I kept trying to prompt an LLM to simplify more, simplify more, and it just can't... You feel like you're outside of the RL circuits. It feels like you're obviously pulling teeth. It's not like light speed.

9. 23:21

   So I do think that people still remain in charge of this, but I do think that there's There's nothing fundamental, again, that's preventing it. It's just the labs haven't done it yet, almost. Yeah. So I'd love to come back to this idea of jagged forms of intelligence.