Mollick sees no clear technical wall to steady AI gains

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 31:12

   Let's talk about AI's progress. AI's progress has, in many ways, surprised the leaders of these AI labs with how good it's gotten in the period of time that we've dealt with. Why? Has it? Yeah. I mean, all right, let's take an example. Here's a good example for you with Anthropic. So Dario was at the Dealable Conference a couple of years ago, and he was talking about like if he buys you know if he overextends on his compute purchases then um his company could go bankrupt if the demand is not there uh it turns out that anthropic is compute constrained now he should have bought more so that's what i'm talking about like the capabilities there were even um so another here's another example ilia sitzkevath former chief scientist of open ai said very clearly a few years ago that the technology was about to hit a wall data wall and what happened like the the capabilities have just

2. 32:10

   exploded ever since so there has been a degree of surprise there so so not to be argumentative i think that the ai companies have their market has exploded best than they thought right which is different that like demand has been higher than i think they anticipated right and they're all compute constrained at this point um i think ilia's point was not that there was no forward progress He argued that the era of pre-training had generally ended.

3. 32:34

   That's correct. And for a while, he was right. But what's happened is just like in every other kind of case, there have been technological innovations that have allowed LLMs or LLM hybrid systems to continue to develop as things change. I think the more fundamental question is, were the AI companies, are the leaders actually surprised by the pace of actual development of the technology? I haven't actually seen a lot of evidence of that.

4. 32:57

   Like they seem to have been pretty spot on, actually shockingly so, about the continued development and doubling of the technological ability. So I think you could say the market's grown faster than they expected, right? But I don't think that they've, I think the technology is surprisingly to me, right? I thought we would have hit some sort of slow, I mean, I thought slowdowns were a real possibility a few years ago.

5. 33:17

   I no longer think they are. They've overcome a lot of technological barriers, but I don't know if they've, they don't seem surprised by that. They seem on target for where they think they're going. i think reasoning is something that they weren't expecting to work as well as it has i mean you take a look at and reasoning has been core to these uh math breakthroughs right so you take a look at um the the predictions of when people expected uh ai to be as good good at math as it is five, 10 years from now, but it's already solving millennium problems.

6. 33:49

   Yeah. That's something I talk about in the book, but those are predictors. So again, we have a little bit of a confusion because you were sort of saying the heads of the AI labs. I don't think they're surprised. I think super forecasters keep getting this wrong over and over again. Yeah. So why do they get it wrong? Because I think it's really hard to predict an exponential. And I think that it's almost the opposite of what you said before. The labs have been right. We have been on an exponential curve that has, with a fast doubling time. And by the way, the reasoner breakthrough is one nice example of this, right? The reasoner breakthrough was AI was starting to hit a wall of development, right? It couldn't do math. That was the big, it couldn't get the R's on word strawberry. And then what happens? People, the OpenAI develops reasoners

7. 34:27

   and a one preview comes out and suddenly the pace is even faster than before. So I think the issue is, is that, and this kind of goes back to your big question from the very beginning, which is, I think people keep not being able to forecast that we are still on an exponential curve. So they keep making plans for what's on today. Today's about vibe coding. Today's about token maxing. Today's about lowering token costs. And those are very temporary situations as long as the exponential continues. As soon as it eases off, it's easier to predict the future. I can then tell you what the world might look like. I can't tell you as long as we're on an exponential curve. Where could the next wall be? So when I talk to people at the labs, they don't think there is a wall i mean that's what they think i i all technology all exponentials

8. 35:10

   become s curves eventually um you know and there's all we hit some sort of limit at some point i don't have a lot of future insight you know like most of the things we thought would happen model collapse because of lack of training data like that turned out to be technically solvable right reasoners turned out to be a really good solution to problems i thought agentics um one of my biggest mistakes, I think, was I thought agents would be harder to implement than they were because I thought you'd have to build complex organizational systems for agents to operate in.

9. 35:40

   It turns out self-organizing swarms work really well. So like the barriers I've thought I've seen would see, I haven't seen yet. So I've gotten, you know, I've run out of being able to predict good barriers. I am sure that this S-curve has a limit, but like I don't have a technical point anymore where I can point to this and say, this is the big problem that's going to hold back AI development what about power so i mean there are limits to how much compute we can deploy right and limits the number of chips out there so you have to make choices about whether or not you using the remaining the power like the physical power and amount of inference we have available is limited, right? And that will be a trade-off between R&D efforts that the labs spend on training new models and how much inference they want to sell. But yes, there's obviously going

10. 36:23

    to be a bottleneck in the physical world in inference and compute. It doesn't mean that, like, I'm not predicting we go to superintelligence overnight, but I'm saying I don't see any stoppage to steady gains in AI ability. It does not mean the same thing as AI becomes super intelligent and wakes up and murders us all, right? There's enough people worried about that that it's worth a little of our time to worry about, but it's important to realize like we can get continual gains.

11. 36:44

    It will be constrained by the amount of compute and power we have. So we talked about reasoning, but I think it's worth stopping a minute and talking about just how impactful it's been, because I think you're looking at a completely different AI industry