Human expertise must keep updating AI models

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 22:41

   And when you ask, you know, great lawyers to spend time creating data for you guys, I imagine an initial reaction is often, are we just training you to replace us? And what do you respond to them? Like, what do you genuinely think about that? yeah it's a really good question and it's definitely the most common misconception there's a few things here the first is the the premise of that question assumes that knowledge is static because if you if you are distilling all of your knowledge that is just one static set into a model at some point you will distill all of it and it will know it will know everything and then you can just use it and you don't have to do anything and you just replace yourself in but but the key thing is that knowledge is not static it is and the example that I that I used

2. 23:38

   and you could take this and apply to any domain but the one that makes sense is a medical uh there's some sort of stat around medical school which is um uh when students graduate medical school it's something along the lines of 30 to 40 percent of the knowledge is like no longer relevant or you have to like relearn and update and so forth and and that's literally like a few years and and like almost half of the knowledge you just learn that you, like, worked so hard to learn is not relevant anymore or you have to update it because there's a lot of, like, innovation that has happened, right?

3. 24:06

   Yeah, that's actually a very interesting idea. I haven't heard that. Yeah, you can apply that to any domain. I mean, law is, like, you can apply it to law even more so because law, I mean, the definition of law is, like, we're introducing new laws, right? We're updating and iterating, and so it's very, very dynamic in nature. And pick any other domain you can make the same argument for. so what this means is that you you are distilling the current state of knowledge and the current state of know-how which only humans know obviously to the models but as that changes you have to keep aligning the models and so so that's one part of it but but but one part that is actually more important is as you as you distill let's assume the rate of change is very slow because the argument against this is like the rate of change is very slow then like you can just get to close

4. 24:58

   to 100 percent of knowledge distilled and then and then you sort of stay close to 100 because the rate of change is slow but but what we argue is that the rate of change is actually not slow and it's increasingly fast and the reason is AI because if you think about like a let's just take a quick example of um of uh let's say um a lawyer right a lawyer might train models to be very good at writing legal memos for m&a let's say they're m&a lawyer and they may trust the model to to ultimately write the memo based on all the information that they've accumulated in the deal right and maybe the model gets so good at that where like it actually does it better than that what the lawyer has done at that point is they've they've automated that part of their job there's like a you got to think of their job as like a this this action space right it's like it's like

5. 25:49

   the vector span or action space, whatever you want to call it. And, and, and, and at, you know, at some point you, you automate a material percentage of that action space. And, and as you do, you free up your time as a lawyer and part of free up your time doesn't mean you actually do nothing in that free time. It means you're going to figure out other stuff to do. And part of, part of you figuring out other stuff that, that will be within the realm of your, or the action space of your job is you will invent new things to do. And that's the whole idea of like the economy expands, humans invent new things to do. And so the action space is this like ever expanding set. And you have to keep automating the set that is expanding that you've just invented as a result of getting free time. And what we argue is that with AI,

6. 26:36

   the inventions of humanity will accelerate and that action set will grow faster than it is today. In many ways, I totally agree with what you just said. To double click, do you feel like there's any examples like you're already starting to see? Like in this space, the action set actually expanded and I'm seeing people that did this do more of that. Yeah, it's a good question. There's two really good examples I'll give you. One is coding is the obvious one. I mean, coding is one that you didn't just change the action, you literally just removed, you changed it completely, right? Like most domains, you're sort of, you change a few things, you know, you sort of have like maybe 10% of the action changes and it's very iterative. And that's sort of what's happening in medical, legal, and all other domains.

7. 27:22

   And I think AI is going to make that happen pretty much in every domain. But coding is an example where this happened very quickly. And it wasn't, if you think about what a coder did a few years ago, it was like code, right? Like they would spend their whole day coding. It was like 90% of their actions plays. It was like literally just typing and coding. Now they're a conductor. Yeah, now it's literally zero.

8. 27:41

   Now it's like close to 0% of a coder's time is spent time actually programming. however programmers are uh i would say there's two things that are true the first is programmers are having a blast like yeah i think it's like the most fun time to be a programmer They're building more software than ever as part of that. And there is more programmers than ever. And we can discuss how the action space has changed, but it certainly has fundamentally changed. But it's now a lot more of a powerful role, which a lot more people can also participate in.