A firm's specialized model could become its core IP

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 16:04

   Yeah, I think this is going to be a, like, blueprint for the future, right? So right now, I mean, like, the total number of large-scale parameter fine-tuned or reinforcement fine-tuned models is, like, still pretty small. But if you kind of think about it, like, one version of the future is that every firm's core IP is actually their specialized model or their own fine-tuned model. And just in the same way that like, you know, today, you would generally think that the IP of most tech companies is their code base. In the future, you would generally think that their specialized IP might be the model that powers all of their internal workflows.

2. 16:47

   and what are the special things they can add on top? Well, they can add on data and environments that are somehow specific, very, very specific to the day-to-day problems or information or challenges or business problems that they see on a day-to-day level. And that's the kind of like really gritty real world information that nobody else will have because nobody else is like doing the same, the exact same business motion as them.

3. 17:13

   There's a lot of weird tension in that though. So I remember friends of ours from one of the top model companies came by and they were like, hey, do you think YC and YC companies would give us their evals so we could train against it? And we were like, no, dude, what are you talking about? Why would they do that? Because that's like their moat. And then I guess now that based on this conversation, it's actually, I mean, evals are pretty important as a part of RL cycles.

4. 17:39

   And then even the evals are not really the valuable part. But the valuable part is actually the, like, properly fine-tuned model for your data set and your set of, you know, sort of problems. Yeah, it's like these Lego blocks, right? If you have the data and you have the environments and then you have the, you have, you know, a base model, you, like, you know, can stack those on top of each other, get a fine-tuned model.

5. 17:59

   And obviously the evals are important. This is some of the tension. And this is basically, you know, in a nutshell, the sort of, like, does AGI become a Borg that just sort of, like, swallows the whole economy in, like, you know, as one firm? or do you still have a specialized economy? My belief, generally speaking, is that you still do have a specialized economy. Like these models are platforms, but the like alpha in the modern world will be determined by, you know, to what degree you're able to sort of like encapsulate your business problems into data sets or environments that are then conducive towards building like differentiated models or differentiated AI capabilities.

6. 18:38

   Yeah, that's why asking for evals was so crazy to me because it's like, okay, you get the evals, the base model is way better. And then not, you know, now all your competitors have exactly the same thing that used to be your advantage. I think we will undergo a process in AI where we learn what the bright lines are, right? I mean, I think that like, it's like very obvious and intuitive to tech companies that they should not give away their code base and they should not give away their database. Like they should not give away their data, they should not give away their code base. The analogs of that in a highly AI-fueled economy, I think, will identify over time, but are the evals, your data,