Enterprises may reserve frontier models for their hardest tasks

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 22:47

   what do you think is the state of the security world? Like, what do you think has shifted from this point forward? Certainly within two years, you're going to have local models that are all running on people's computers. You know, you may have a mixture of local models plus sort of last mile routing into frontiers in terms of what they look like. You're going to have a lot more open weight models. You know, you can just see the shift in the tokens, you know, over the last six months. What is a shift that you're seeing? And what's the order of magnitude there? How long do you think that continues for? I talk to CIOs, you know, almost every day. We had dinner with one tonight from Big Bank in New York. And basically the comment is, you know, we got into AI, we went all in great. The costs were out of control. You know, token maxing, you know, stopped when people

2. 23:33

   got big bills. And, you know, I said, well, how do you think about this? Like, what are you going to do? And they said, the models are so, the open weight models right now are so good. Even if they're three months behind, it's plenty for what we need. And one of the customers' comments, and they know I'm into Formula One, they said, any Formula One activity that we need for the bank will go to the frontier.

3. 23:57

   Like if it's alpha and we're making money and it's like, you know, super important, we'll pay the money, no big deal. But for all the other things, like it's really expensive and we're going to create all of our own open weight models, you know, and do more post-training on them, those sort of things, that that's the way they're going to operate. And I firmly believe that. I'm seeing that in customer after customer, that that's the way they're going to go. And I think part of some of the challenges in the current environment is I think people are all seeing that and there is some concern at the frontier level of what's going to take place.

4. 24:34

   And so I guess on the open-weight model, one of the statements that tends to be made is that they are at least in part distilled from the labs. In other words, you have Chinese open-source players who are effectively training their models against that anthropic or open AI model or, you know, other sort of frontier models. And many of those labs are now trying to roll out anti-distillation efforts.

5. 24:59

   So they're trying to build into the model the capability to not get distilled. Do you think it's just sort of a back and forth where the capabilities keep going almost like offense and defense and cyber? Or what's your prediction in terms of the state of distillation in two years or, you know, within some reasonable timeframe? I mean, I guess my first comment is what were the frontier models distilling when they got all that information? I mean, we know all of the, you know, sort of lawsuits around where the data came from and how that unfolded, right? So we kind of start from that. So it's like a fine liquor. It's triple distilled, you know?

6. 25:29

   It's triple distilled. Exactly. Like, so where, where, where does the distillation start from? You know? Um, so I think it, it, it is what it is, but the, the, the labs certainly are focused on, on trying to prevent distillation. And, and obviously, you know, when you look at the Chinese models that are out there, there's a lot of distillation takes, it's factories of systems that are, you know, bouncing all over the internet, distilling this. I think the challenge is that the genie's out of the bottle. I mean, there, there's a lot of distillations already taking place. The models are very good, but we also have to give credit where it's due. If you don't have all the chips and you have to be resourceful and you still want to have top models, you know, I think there's some innovative things that are taking place outside of the U.S. and we've got

7. 26:13

   to be mindful of that, right? So yes, it's hard. I think it's going to be. hard to put the genie back in the bottle on distillation. They're going to do the best they can. But, you know, a lot of the models are, you know, I think now, you know, what does it look like in two years? I don't know. You've got great models today. They can last for a while, and you have a lot of data that's already packed into them.

8. 26:36

   So we'll see how it unfolds. I wish I had the crystal ball on that one. Oh, no, I was just curious. And then I guess another, it's always hard to predict these things, but I'm a little bit curious. Do you think there's structural advantages for Chinese companies to stay at the frontier of at least open source and open weights? Or do you think U.S. open-weight models are likely to catch up in the near term?

9. 26:56

   Well, there are some open-weight models coming out. I'm sure you may be invested in them as well. I think having great U.S. open-weight models is going to be very important. So one of the reasons why we worked with NVIDIA was, of course, Pneumotron is open-weight and open-source, not just open-weight. But having other alternatives is going to be very important. And, and I think there's, you know, again, just being in technology long enough, you kind of look at how things unfold, like the, the operating system of the internet is Linux like period.

10. 27:29

    Okay. And there's a reason for that. And there's going to be an operating system of AI. And it's probably in my mind, it's going to be, you know, an open source, open weight when that happens and how it all unfolds. I mean, we'll see. And it doesn't mean there isn't room for any frontier model. It doesn't mean that people are not going to make a bunch of money off it. you know, the application stack sort of around it, you know, the harnesses and how things are connected and how you think about enterprise, all those sort of things are going to be really important. But ultimately, like, there's just too much cost involved right now. And when you look at a lot of these frontier models, I mean, they're being commoditized. If the Chinese models are three months behind, you know, a lot of companies are going like, three months is not,

11. 28:14

    What am I going to get out of it? It does everything I want. And that's the challenge. A top five financial said to me recently, like a large technology buyer there, that there was no chance that their organization had the capability to absorb a new model release every 10 days anyway. And so three months behind was faster than the acceptable regulated pace of their organization. So it made no difference to them to some degree.

12. 28:46

    I think open is here to stay as a big part of the puzzle. You know, one one question. And I know we're going to keep asking you to look into the future and it's kind of impossible.