Hard technical infrastructure can create an AI moat

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 30:32

   able to do things that the big labs are not going to do or eat your lunch and like actually be able build a good business. And then the other thing, what can LLMs do right now? And what scaffolding do they need around them? The scaffolding part is really important because this is where you can build a value on top of the LLMs. So the way I think about our business right now, we were in the business of training models now where we use all the different commercial models. And I see our businesses to build the milieu, the environment around LLMs. If you think about LLMs as like as a person, you want to create the habitat for them, right? So we built the best habitat, I think, for programming, right? So when we first did this pivot from like the human being of the programmer to the AI being of the programmer, I took the same teams at the company and I told

2. 31:23

   them now you're no longer built for humans, you're building for the AI. So the editor team built editing tools for the AI. The cloud infrastructure teams that have built deployment tools for humans, they now build it for the AI. So I would look at any different vertical, and I would think this way. If you want to build a financial agent, just think about what kind of environment you can build for that agent. Now, I see a lot of people, for example, trying to put LMs into Excel sheets, whatever, which I think works okay. But can you build an Excel interface for the AI? In a way, it's textual. It can use it based on that you want to hit the right...

3. 32:00

   distribution of the of the lms you can pick any vertical and if you want to build like a really great business just think about building your user your first user is the lm think about building that habitat for lm to live in to create like amazing experiences amazing things with the product the unlock of creativity the unlocks of what does that mean for more personalization things that work for individuals businesses you know creative thinking is great so let's talk a little bit about also kind of some of the business attributes and like one of them is you know you just mentioned that the that you know part of what you've done as opposed to home growing in your own models you're now using a fleet of models but that obviously makes you somewhat dependent on them makes you more copyable etc you know business attributes so one of the things that the people

4. 32:46

   who listen to our podcast a lot are entrepreneurs you're navigating a business challenge that they need to learn from so how do you think about that what are the things that you're doing to build unique and enduring business. What are the kinds of considerations that go into that? First thing I would say is user obsession. There's a massive platform called LLMs, right? That platform is useful on its own.

5. 33:07

   People can use it directly. But for you to build a very valuable business on top of what kind of value you're creating, one is if you're really obsessive over certain types of users, OpenAI can't build for every type of user in the world. So pick a user that you know very well. Hopefully it's you. You've had that experience. You're a salesperson. You were going to build the best LM for salespeople, right?

6. 33:28

   That goes into UI, UX, all of that. Now, second, and this might comment, I'll go a little deeper on the comment about the environment around it. So, you know, I said that people using Replit should feel like they're in a safe environment. It should feel like anything is reversible. Now, that turns out to be a huge technical challenge. Now, you could easily kind of hack around it It'll say, you know, we'll just restart the app.

7. 33:55

   We'll do whatever. But what we did is we built a transactional file system that allows undo. Like we have a proprietary file system that we spent two years building, that every action is like an immutable part of a ledger, that anything can be rolled back and rolled forward. That ends up being like a time travel system. And now we're finding more use cases for it. For example, one way to make models better and more performant is by doing sampling.

8. 34:24

   Because models are stochastic, every time you run the model, you might get better results, you might get worse results. And if there's a way to verify whether the result is good or bad, you can sample from the model. So what we do is, because the system is immutable, has this characteristics of being very cheap to fork. So you can take a file system, you can fork it 100 times, run the same prompt with different parameters on these different forks, and then pick the best one.

9. 34:52

   Do it repeatedly. Obviously, that costs quite a bit of money, but for the advanced users, they're willing to pay it, they will get the extra money. So do hard technical things on top of the models in the environment where the models are sitting, right? You want to build technical advantage. It doesn't have to be in the model itself because that costs a billion of dollars. There are huge industries around it.

10. 35:15

    You don't want to be training models, at least not yet when you're initially building a business. But you can do a lot of other hard technical challenges on top of it to make the models work a lot better. That's the second thing. Third is reaching scale.