AI companies need different moats at each layer

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 37:01

   AI app layer companies are going to die. Not exactly. Well, what I said is that there's sort of three main categories of companies that have meaningful revenue scale outside of data companies. The first one is Frontier Labs. The second one is the inference providers. So first one being, you know, Anthropic OpenAI, second one being Fireworks Space 10, and the third one is the app layer companies like Harvey. And my thesis is that there are large portions of each of those businesses that get commoditized, right? Like if everyone's buying GPUs in the inference layer, then the GPUs got commoditized. If the app layer companies are building software layers, those will get commoditized as it becomes cheaper and cheaper to build software. But the two things where companies are able to build moats that I've seen that have been durable is first of all their brand

2. 37:58

   of which Harvey has a world-class brand certainly top in the law category and maybe more generally across application layer startups and then second of all their data moats which we can help with meaningfully of how do they understand this flywheel from the not training on customer data but learning the distribution of what do their customers care about and using that to inspire what are the benchmarks that they want to build, what are the training data sets that they need to curate to be best in the world at those.

3. 38:32

   And I think those are really going to be the two primary modes to look for in the application layer companies that are able to build durable and potentially legendary companies. What are the modes for the top two layers? Well, with the Frontier Labs, there's certainly a lot of economy of scale associated with just spending more on compute and more on data, where if you have the most distribution and revenue that you can generate from your models, you can invest more in having more compute and more RL environments, which the labs are scaling up their investments in exponentially.

4. 39:12

   I think there's also, so I'd say it's probably mainly economies of scale. And then there's some flywheels associated with a given model helping them to improve their subsequent model more, i.e. recursive self-improvement, which won't be a vertical takeoff, but it will contribute and make things slightly faster over time. And then brand modes, I would say, are the main ones for foundation models. For the inference companies, some of them are, I would say, the primary one that I think the inference companies need to get right and Fireworks is executing really well on is building custom models, right?

5. 39:51

   It's just the other side of the coin of what Harvey's thinking about because the switching costs of reselling GPUs are zero. As much as people try to tell this nice. story of how there's great margins, like of course, when you buy a bunch of GPUs and those appreciate like 50%, you'll have like good margins for a period of time. The real switching costs and durable margins come from how do you help a customer customize a model that has high switching costs and can't be ported over to other infrastructure. And so that's how I think about sort of the modes that are developing at the frontier model labs, and then also the modes that are developing in the inference layer and how those companies will build durable businesses.

6. 40:34

   Which is the most vulnerable?