New Nvidia chips require constant iteration before launch

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 10:16

   I guess I'd love to take our listeners inside Nebius, and I feel like there's so many interesting parts of the business, but maybe one place to start would just be, I imagine as you're getting new NVIDIA chips that come in, there must be a bunch of stuff you have to do within the business to get this all up and running. So what does that look like from, I don't know, you're talking to the NVIDIA folks and they're giving you early insight into the next generation of chips to actually getting these live for a set of customers?

2. 10:43

   that is a mess no i mean like in a good way you know like this whole this whole market this whole field is running at such a pace that like everyone is like uh trying to catch up right and NVIDIA despite being like in the lead of this like essentially leading and showing the direction for this market is not the exemption so first NVIDIA folks like way before show you the prototypes of what they want to build in the next like year right they kind of show you the schematics like how it will be like optimized internally where is your like PCI connections how would that look like what is the like future platform would look like then some time later they come to you and said oh like we've optimized things it's now gonna be different you're like all right then they come to you and like oh we have like the first lab samples do we want to have a look yes

3. 11:43

   of course we do want to have a look oh by the way it doesn't look like the stuff you've showed us two months ago oh we've changed stuff because like we found like right because they are so compressed in time same as they everyone else they are constantly iterating while these chips are being produced. Right? If it's not your, like, you know, there is like an ocean in the consumer market, like Apple iPhone that you see today was actually planned three years ago, and final design more or less was settled like a year ago, and they take the final year to polish it to build this product.

4. 12:17

   Like, AI market is running at such a pace, like, that whatever, like, the announce is, it's indicative. And then teams are constantly iterating. Well, because the architectures and the workflows are changing, right? Right, so I'm sure they're iterating as that. Yes, like they're iterating, like going further and further and further. And NVIDIA is a behemoth now, right? Whatever they do influences the whole market, supply chains, there's everything.

5. 12:41

   So they have to like do that. So at some point you get a sample and it's more or less like what you'll get in production. Then you start building, like you are testing everything. you like incorporate support for the new hardware for the new restrictions or capabilities into the software platform if you figure out a way to isolate workloads from from the chat we are we are cloud we are not providing bare metal hosts right like so we need to split we make sure that our virtualization works that our storage works that our networking works sometimes we are on the bleeding edge of this so we like for some of the generations we were probably the first team like like maybe one of the first teams that actually built like a cluster of a certain size.

6. 13:32

   So we've hit some issues with like with the software that no one has seen before. So we had to report that back to NVIDIA, like propose a patch, get there, like fix from it, make sure that it runs, et cetera, et cetera, et cetera. So like we kind of feel like we are at the forefront of it. But all in all, it's a pretty normalized process. it may sound messy right there are like multiple iterations you go back and forth you're in constant uh contact with like NVIDIA hardware folks with NVIDIA software folks uh you are trying to solve the issues on your end at the same time incorporating into a more complex platform thinking how can you expose that to customers because for example working with uh like your usual black hole chips like you know the like b300s and then transferring that experience to gb300s which i call different architecture in terms of isolation you are

7. 14:30

   talking racks rather than servers you need to find a way how to do maintenance on the rack instead of the server which you just switch off and you move the workloads to another one despite all of that this kind of works all the time and every day you like push it a little bit