Enterprises favor open models but still need hyperscalers

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 8:54

   the end of the day, I do want to say something to what you guys were talking about earlier, about the open source versus the sort of these hyperscalers providing inference. So we had this peculiar problem. We'd like to embrace AI as quickly as many of the startups. And we've gone through all the understanding, all the analysis, open source versus cloud models. And we preference open source, open weight, because it allows us to modify the weights and train them. And something which most people don't talk about is it allows us to interrogate the chain of thought.

2. 9:27

   Because over time, what's going to happen is all of those enterprises are going to need explainability, auditability, and verifiable. A hundred percent. If I can't validate, it's not going to be okay to go to your boss and say, I don't know what happened, but the IT guys put an AI model in there. It did it. That's not explainable to the regulator in financial services and healthcare. So they want explainability, verifiability, and auditability.

3. 9:50

   And we can only get that with open source, open weight models. We can interrogate chain of thoughts. That's kind of one reason we're gravitating towards them as opposed to getting caught into the closed race. The challenge, though, I think you guys talked about and I saw Chama's chart. The challenge we have is it's very hard to find a global, resilient, highly available inference provider outside of the hyperscaler community right now.

4. 10:14

   So if I'm a startup, great. I can go rent 10 megawatts in one data center in one part of the US and if it goes down for a few hours I'm out of luck but I'm a startup so I'm okay but I'm providing resilient services at 99.9% reliability it's very hard for me to find a global inference provider who does not want to sell me their model at this point in time at the price it just makes it economically useful to use open source let me ask you a question though do you think that there's a world where Palo Alto Networks says one of two things one is you go to the hyperscalers and say, okay, listen, guys, you run it for me, but I want the bare metal. I want it totally air gapped. And it's an AWS facility, but I'll just lease it from you. Otherwise, just get out of the way so that I can have everything inside of it. Or do you do what this one big G-SID bank

5. 11:04

   does and say, I'm going to build my own inside the United States? Is there a world where do you think other companies go to that limit? Or do you think that's an extreme case? That's probably not representative of the norm? So this is the interesting debate, Shumat. Eight years ago when I came to Palo Alto, I shut down all data centers. I said, we cannot run a global data center sort of footprint at the scale, speed, and reliability as I could probably rent it from hyperscalers.

6. 11:29

   And life was great, and suddenly what's happened is in the current sort of incarnation of where we are, compute is highly dear. It's very hard to find compute in the world, period. So I don't, I'm a buyer, I'm a taker of terms, not a dictator of terms. So it's very hard to go and dictate terms anywhere because, yeah, sure, you can make any plans you want, but those plans are, you know, you talk to Jensen, he can't give you chips until 2028.

7. 11:53

   He talked to, you know, where do you get land power and shell? So it's very hard to go say, yes, I can go build more data centers. It's kind of an empty threat. It's very hard to go build it, have it be resilient. And by the way, I don't run 50 data centers around the world where I'm running 10 to 20 megawatts of inference. So it's a good conceptual sort of idea that I should go become independent, build my own inference.

8. 12:14

   But most likely, the answer is somewhere in the middle. I'm going to have to get bare metal, yes. Most likely, I'm going to get from either a hyperscaler or a cousin of a hyperscaler, or somebody's going to be stitching sleeves of compute from around the world to meet my needs in the hope that I'm going to migrate to something more resilient and more robust over time. All right. So you have a hybrid solution down there.

9. 12:36

   One second. I just want to complete the thought. You have a hybrid solution down there. I just want to get to cyber stuff before we run out of time. Just, you know. Then you go and you basically get a bunch of open source models so that you can interrogate chain of thought tokens. Is that a representative solution for most of the Fortune 500 or Global 2000? I think we get there. I expect at some point in time we will start seeing hyperskinners giving bare metal and letting us run open source models, which they already do to some extent.

10. 12:58

    So that's probably what's going to happen because it's a matter of the profit pool, right? It all depends on where you shift the profit pool. Today, they all live in NVIDIA and the memory industry, and they're being shifted towards the hyperscalers, people who have large compute available because at a point, it'll become less about the chips. It'll become more about the availability of compute because I think demand is insatiable.

11. 13:19

    There's enough demand. There's more demand than you and I can imagine. So yes, I understand the enterprise diffusion comment, but it's going to get diffused. The question is, even if every company in the world today said, I'm in, I'm in, I'm ready, I'm embracing AI, let's go as fast as we can. There's not enough computer in the world to satisfy us.