Open source pays off once AI workflows are stable

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 25:02

   So you've been doing a lot of interesting writing on X, a lot of great posts. And one of your most recent posts really intrigued me. it was about open source models. And you wrote that everyone is wrong about open source AI in the enterprise. What are they wrong about? Yeah, I would say to put it simply, it's, you know, there's a really strong narrative now that open source has taken over. It's getting a lot more adoption.

2. 25:26

   You see this in a ton of metrics. You know, the usage of the models on Hugging Face, the spend on the inference providers. I mean, they're all ripping. So that is true. Open source is happening more, but then actually if you look at the research, a lot of the stats are saying that the share of open source compared to closed source is actually shrinking. And so we were just thinking why that is because we are a big proponent of open source.

3. 25:52

   So we are pretty much 90 plus percent open source at this point. Wow. Have you always been 90%? When we started, it was all closed source because you're just trying to figure out the use case, right? Yeah. But my thesis was, hey, you know, why is it the case that, you know, open source is growing, but then the share is shrinking? Is that, you know, simply put, open source is great for scale production use cases where you know the structure of the use case and you know the structure of your agent or the model calls.

4. 26:22

   Everything else, closed source makes more sense. And so if you look at the enterprise, most of the use cases are still in that experimentation phase or still in that sort of not fully productionized scaled out phase. And that's why closed source is still so popular. So the point that I sort of read from your post was this idea that when you come up with a task for AI to do, you want the cutting edge model.

5. 26:50

   You want the frontier model because you don't know how much power, essentially, you're going to need to complete the task. The way I read what you were saying is that, you know, we had a lot of these use cases. People sort of thought about, hey, I think I could do this. Why not use a frontier model? Now we're at the point, though, where they're saying, okay, well, maybe I don't need that. I can use the open source models.

6. 27:13

   The question I have for you, though, is that, I mean, won't the applications of the technology continue to keep going? Like, we're going to keep finding new use cases, so frontier model use, conceivably, will still continue, right? Yeah, that's my main point. So it's not that, hey, open source is going to take over, and because they're smarter now, you don't need the frontier models. I would say there's always going to be new use cases.

7. 27:38

   And the other thing I wrote about is that the effort to get open source to perform at the same level as Frontier, actually, it does take effort. And that's not to say the effort's not worth it. I mean, again, we put a lot of effort in, and it's super worth it. Because once, you know, we do AI customer service, and so once that use case is solidified and you're already scaled in production, then open source makes a lot of sense, because it makes sense to put in that effort to train those models.

8. 28:00

   Because those open source models, especially the small ones, you get a lot of benefit out of them, right? You get better latency, which was the main reason for us. You get better cost as well. But out of the box, those models are just not good enough. So you have to put in a lot of work to make them good enough and make them be able to sub in for these frontier models in certain spots. And once you put in that work, now the results have to justify the work.

9. 28:24

   And they only justify the work if you're deployed in production at scale. Most of the use cases, to your point, are not like that. So people are just going to continue using frontier models. Which models specifically are you guys using in the open source category? It's a mix. So we have Gemma 4 is the main American one. A lot of the Chinese ones are quite good. Quinn, GLM. and we have our own evals to kind of decide which ones perform better in certain scenarios.

10. 28:51

    Right. How do you, I mean, you know, the race here, the open source race here, I guess, between open source models coming out of China and those developed here in North America, I mean, you know, at one point this was a race that people were paying a lot of attention to. Now it's sort of accepted, I think, that, you know, those models are great. People are willing to use them. I mean, if you were to sort of make a prediction as to how you see that race playing out, Do you think that the issue of it is kind of behind us now and now we can just sort of use whatever we want?

11. 29:22

    I think it's really up to people's choices, right? Like we've talked to enterprises where, you know, super fairly their leadership has made a decision, hey, we only want to use models that are trained in the U.S. Because even the Chinese open source models, they're all hosted in the U.S. Your data's not going anywhere. So that's not a concern. But it's, hey, you know, maybe there's some bias in those models.

12. 29:40

    Like we don't want those models in our system. And so that's kind of up to each enterprise. Other ones are very comfortable using open source models. I think China has taken a very open source forward approach because I think that's how they compete. In the US, a lot of the leading labs, I think just if you're in their shoes, it makes sense to put your best models not open source because you make more money that way.

13. 30:08

    So there is kind of that balance. And you also hear from China now that, okay maybe they want to kind of now that they've reached you know frontier level and in certain cases maybe they want to stop doing open source who knows we'll see but i think having a lot of choices is just better for everyone what do you think of meta's decision i mean meta initially the llama family of models and that was open source um muse unless i'm missing something that that is a closed source model and so this is sort of a a transition away from open source for the moment which is a choice why you know what do you think make of that decision um i mean again i think it makes sense if you're in their shoes in in the u.s that's the norm right all the all the leading models are closed source and if they're closed source kind of you you maintain more of your alpha

14. 30:58

    and you get to make more money because people are paying for the right model it's like that's like a self-reinforcing loop you make you take that revenue you can that helps cover the cost of trading the model and so that makes sense right and again in china like that those like if they did that no one in the u.s would use those models because people are afraid of their data going to china um so having open source having open source strategy there makes sense because it gets people to use your models give us an update on your company so we had you on the show almost a year