Compute buyers can choose broad or exact GPU SKUs

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 20:23

   So we see the other side as well. Kind of depends on who you are. Cool. So, is it, okay, and I just really want to make super clear on this. Of course. Do you care about the different generations of chips, or do you just care about overall flops and you spread it among whatever chips you want? We care about different generations of chips because... Yeah, you care. I guess, do your customers care? Yes.

2. 20:46

   Right? Oh, wait, when you say different generation chips, do you mean like... So, like B300s or B300s? Yeah, exactly. Yeah, or like even AMD or whatever, right? Yes, I think people totally care about the generation of chips. I think part of this is because of things like VRAM, and part of this is because of larger NVLink domains and things like that. So there's just no way to abstract all this to some generalized unit of compute that you can sort of mush around, whatever you like.

3. 21:11

   I think when you were saying even oil isn't oil, oil is actually just a bunch of different types of oil. Yeah, like jet fuel or car fuel, whatever. I think it's the same thing. You're going to get a bunch of different types. It is, but you have oil refineries. Effectively, a black box is you can put oil in then get like whatever out um if compute it is so uh heterogeneous i guess is the word is that you just have to expose the underlying complexity to your end customer which is bad for you as a market maker maybe like you want to identify the one like choke point or like i don't know what the better word for choke point is like a shelling point where everyone agrees like okay like this is kind of the standard and we'll we'll all agree to this standard and uh you know it'll be somewhat inefficient yes uh so we've been able to reduce the um the the problems to a certain level but

4. 22:02

   yeah like your job is like dimensionality reduction yes we reduce a lot of the dimensions but nonetheless there are still dimensions yeah it's still going to exist do you know how many skews you have so um like five times a hundred times like five thousand equals i don't know like what's we actually made skews like a concept um that and the goal is to make it so that somebody else can make their own skew so if i want to buy blackwells i can say i'm willing to buy blackwells at six dollars an hour or less and you can match me against gb300s or you can match me against b300s and i don't care which blackwell as long as it's less than six dollars an hour um but you just said you care about memory and other stuff totally but not every customer does so some customers like um just can work on black holes whatever it is some customers are

5. 22:50

   super hyper specific and they need this exact thing and they're going to pay more for that because the market's going to charge them more okay um and that's what we do so and so people use your apis to programmatically define what they want and then you match that with whatever composition of sellers uh they might they might they might be able to fulfill yes um with the caveat that having a named query is a new thing we're doing um but the query system as a concept is how the order book works right now.

6. 23:17

   Okay, interesting.