Data center owners need open models and a software partner

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 49:13

   One thing is ecosystem building, and the other is these sort of commercial constructs. On getting the models out into the world, it's really important that the ecosystem doesn't build itself. You need to be an active participant in building that ecosystem and working with partners to to ensure the models are natively integrated, natively supported across the whole stack from some of the NeoClouds that exist today to other inference providers, to hyperscalers, to the Dells of the world.

2. 49:45

   And we have a whole open source team whose job is, one, build open source software around the model. So it's not just the model. The ecosystem is the software that you build around it and will have beyond the model releases of open source tools that are meant to be coupled with the model that help the community better understand well, better understand and build safety guardrails, agents and so forth.

3. 50:11

   And then obviously there's an arm of the open source team around working closely with the ecosystem to integrate things. So while ecosystems don't get magically built, you have to put effort into it. I think that on the kind of, on the commercial side, the way we think about these partnerships is this, This, again, going back to the notion that there are certain things that enterprises, sovereigns, public sector entities are good at, and there are certain things where they need partnership.

4. 50:38

   And typically, a place where there's some expertise among these is, well, finding land, power, shell. These are real bottlenecks. These are limiting factors to deployment of compute. If you have a rack sitting somewhere that is not powered, you can't monetize it. It's not useful. And so partnerships like this allow you to basically get power, land, shell, and even stand up the infrastructure to bare metal.

5. 51:08

   So Dell is one of the best infrastructure providers in the world. Then there's a question of, well, what happens from there? So let's say you've figured out, and this is in Korea, it's this 250-megawatt data center, so you figured out how to set up the bare metal. Well, what happens from there? Today, the market in compute is largely in offtake. So I take that bare metal and I sell it to hyperscaler.

6. 51:35

   That market is moving very quickly into inference. Like, I'm no longer selling offtake. I'm selling inference into other enterprises, into my own enterprise. And at that point, well, I need an open model because I can't get my hands on a closed one to run around my own stuff, right? Maybe a hyperscaler can through some partnership or a closed model itself can. But by nature of you owning your own asset, you need an open model to serve on it or an ecosystem of open models.

7. 52:03

   And you need a partner that will set up the stack for you because you don't have that expertise that will enable you to do that. And so that's where we come in, at the software layer above the bare metal with the model and the whole stack around it. And I actually think Yanis had a great analogy to it, so I'll let you speak to it. But you can kind of think about frontier model builders as kind of, you have to build all this software at scale for serving models in order to train models in the first place. So there's a really nice analogy that Diannis has to how AWS came from dogfooding web services at scale and how that translates to AI. Maybe you want to speak to that?

8. 52:47

   Yeah, it's the idea that the clouds, you have three main hyperscalers. You have GCP, you have AWS, you have Azure. And the way they actually started is because they want to solve their internal problems. So AWS had to support the Amazon business, so they required the servers. GCP was built because Google had this massive search engine and had just really served billions of customers. they had just like solve an internal problem and that helped them just develop the really dog food.

9. 53:18

   You know, the dog food thing is the idea of like using your eating your own kind of like products or like using your own products. So they had like this internal demand that kind of like led to the development of this technology and then realized that like this technology can be like sold more widely. And it actually gave rise to the era of internet and like all the SaaS companies that were actually built on the cloud.

10. 53:38

    So I felt like we would be entering like a similar era where you need to have the insight of building a frontier model and just really be at the frontier of intelligence in order to ensure that you design the system in a way that it's scalable and it's the right solution for the right problem. Yeah, so to kind of Yannis' point, what that means is that by training these models, we serve trillions of tokens into our own runs.

11. 54:06

    We serve billions of agents into our own runs, and we actually have built all the technology necessary for managing very large clusters across many different GPU fleets, serving the inference on top of them, sandboxing into agents at scale. And in a closed lab, that's, again, abstracted, and so you kind of just get the experience out of it. But if you're an enterprise or company that wants to own your stuff, we externalize all this software that we built and already battle-tested at scale for them and effectively make them have the same capabilities as a frontier closed model, right, through a partnership.

12. 54:43

    As we're describing all this,