Models make product layers obsolete, so build around durable needs

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 3:17

   How does your business change in a way that people don't see as the models get better? The main way our business changes as the models get better is we basically have to erase big parts of our product. So what I mean by that, and Cursor said this really well as well, I think this was one of their actual operating principles as a company is like erase the product or delete the product, I think is what they called it.

2. 3:37

   Where basically like these models get better and things that you had to do six months ago, you no longer have to do. And so this is very relevant in our open source. So our open source is all about basically the harness and the framework for taking these models and helping them do things. And the right way to take these models and help them do things changes every month, three months, six months, choose your time period, but it changes pretty rapidly.

3. 3:59

   How do you build for such an uncertain future? Do you just have to get in bed with the labs to know the roadmap so you don't waste six months building something that then narrates? There's no easy answer. On one hand, what are you going to do? Not build. That's what I like to say to founders who ask this, what do I do? Stuff is changing. The models are changing. If I build something now, it might not be relevant in six months.

4. 4:21

   What's the alternative? You just don't build anything? I think you have to make educated guesses as to what parts are going to be around and what parts aren't going to be around in the future. For example, early on, the way that we would do tool calling in LinkedIn was by having the model generate JSON, DSL, and then parsing it. And that was something that I think we all thought or knew was like a little bit of a hack at the time.

5. 4:44

   But we had to do it because that was the only way to get it to actually steer things, because you have to get it to do this type of tool calling. And then a few months later, OpenAI came out with tool calling. And I think we recognized... And this is a more native version of tool calling that's directly in the model and in the APIs. And I think we recognized pretty early on that that was the future, and so we invested a lot of efforts and just completely switched from JSON parsing to this tool calling.

6. 5:06

   And so I think you have to basically recognize what things are short-term hacks and what things are long-term will always be there. The models will never know your proprietary data. They will never be able to actually get the output of APIs. You have to take their API call, pass it to the API, get the output, and then continue in the loop. And so I think the system around the model, connecting it to other places, will always be there.

7. 5:29

   But a lot of the hacks we do around chain of thought or planning or tool calling, things that can just be much more closely integrated into the model or into the model API, will go away. And I think you just have to be quick to recognize that and go on. Why do we need neutral harnesses?