Frontier labs are holding back models before public release

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 54:58

   we go if that's okay um you know the i think dylan you've said in the past that um the the people who are getting most of the value out of the ai models that are created are the end users right the people that are using the clods for instance at the you know at the base um and of Of course, Anthropic makes that model available via API and to go into all these various uses, you know, it's going to have to do more licensing and who knows if they'll be able to do products for all this.

2. 55:23

   I'm curious if you think if you see a scenario where given the big numbers that we talked about in the past, whether they keep their frontier models at home for longer, much longer and use them to build their own products. Like we know they have the wet lab, we know they have cloud code and co-work, and then they don't make them available because they want to get more margin out of the infrastructure that they've bought and built.

3. 55:48

   Yeah. So, I mean, I think we've already seen this, right? Anthropic had Mythos ready in February. Their model safety cards noted that they have two other models since Mythos that have released. And publicly, we still don't have full Mythos. We have a neutered version of it called Fable, and it took many, many months before Fable even came out. They had the Glasswing, which was like mythos, but for only some subsectors, and it was neutered in some areas, right?

4. 56:17

   Not all areas. And so it's clear that Anthropic has not released new models. OpenAI likewise, Astra was actually done a couple months ago, and it only came out like, you know, it took a couple months before they released Astra. And their new model, Bell, they even showed a blog where they showed the performance of it in math and how it, you know, solved all this stuff. And they have not released it.

5. 56:41

   So clearly they are just using it internally, not externally. And so, you know, the labs have already moved along this line, right, where it's like, oh, yeah, the gap between them and Chinese open source is still six months. They're kind of keeping that. But in reality, they kept, you know, months of progress internal only, because the moment you make it external, people can distill it, you know, and all these other things.

6. 57:05

   So I think, you know, Anthropic will likely release a new model in the coming weeks, you know, for their IPO. I think OpenAI will likely have to release a new model in the coming weeks as well, just because it's such a competitive environment. But at the same time, they're like keeping a lot of their stuff internal only. yeah we were discussing internally some of the uh capitalistic incentives if if the incentives are actually going to motivate people to release more models and get them in the hands of consumers because of the incredible gross margins on the per token API pricing, or if the incentive is to keep it internally so that you can pursue the end user applications, which are even more high value than selling the access of the model per token. It's going to be interesting to see how it plays out. But in one of those scenarios,

7. 57:52

   the capitalist approach will be to pace the frontier for yourself, right? And what's your gut on where it's going to go? which one of those scenarios you think plays out i think anthropic has publicly stated that they are pacing the frontier by keeping themselves uh restricted and that's the only way that their blog posts have manifest in reality nobody else is slowing down they're only slowing themselves down i think that's true i think that's what's happening and uh i think the incentives to both improve the status of ai by solving things that are you know like cancer that are just unequivocally good that everybody can agree is a good thing would be you know a positive incentive and then i think there's also plenty of money to be made if you do something like that so i my my bet is that we continue to see the minimum viable models released by both open

8. 58:47

   AI and Anthropic to keep a lead over Chinese and the rest of the open source ecosystem.