Compute contracts must price interruptions and hardware differences

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 9:41

   okay what dimensions matter in this grid of things um so a bunch of different things um one is dimensions categories that i will say is like is there a workload running on it right now or is it capacity in the future so is it capacity right now um capacity right now in order for you to sell that um you have to interrupt your current workload so you have to think do i care if my current workload is interrupted or not capacity that's sold that's starting a month from now um you can buy and sell that all the time and so you get um liquidity on that really easily um there is some concern like oh no well i have capacity next month but it's very different than if you're interrupting a workload right now the way that the interruption happens matters so an example is if you're on a cluster and the cluster doesn't have as amazing storage as you want it to have so an example is

2. 10:29

   like do you have a weko or vast system under the hood and do you offer that and like what's the size of that this changes how you respond to being preempted or removed off the cluster meanwhile the location of that storage matters so if you're able to move that data from one cluster to another um quickly then maybe you might be able to spread across multiple different clusters um and you can be interrupted over here and then pop back up over there um to make that work you also need um like a fast transit path between one cluster another typically this is implemented with a wan but most of these clusters don't necessarily have um like a wide area network yeah a wide area network so um basically cables that connect the clusters together a private internet is another way to think about it um a lot of clusters that are being spun up today

3. 11:16

   They're not attached to traditional CPU clusters were next to the customer for latency reasons. In GPUs, they're next to the power, wherever you put the data center next to the power rather than next to the customer. This means that a lot of stuff is getting put in places that previously they weren't expecting data centers there. And so you may not have as excellent of a transit path from one to the other so this this starts to matter um other examples are um weird shit where we theoretically support but don't actually um uh like this is a thing that we're thinking about haven't done basically um the early gb200s broke the early h100s broke um and so if you were buying h100s or you're buying gb200s you might really want the ones that were deployed after a certain time because you're not getting the first shitty ones off the rack and so if you're a super

4. 12:17

   sophisticated customer you may be willing to pay more um because for that oh so it's it's about like what cohort they were they were produced yeah and they get better over time so like uh the first ones off the rack maybe had some faults or some problem. Right. And so if you were the ones who got the first ones, or, you know, first ones from the first shipment of them, then yeah, they would be broken or in some way. I assume the sort of process is, like, look, there's defects everywhere. Like, it just happens, right? Totally.

5. 12:46

   And so, like, you get a new shipment from TSMC or whatever, and you just kind of run your tests and, like, you figure out which ones are bad and you throw them out. And isn't that how it works? I didn't know about this element of cohorting where like let's say Q4 2026 B300s are way worse than 2027 B300s. Yeah, so I'll give an example. You're describing burn-in. Yes. So when you get a cluster, a good cloud, what it does is it tries to kill all the GPUs before they ever get in the hands of the customer. And then the GPUs that are broken or the nodes that are broken, you get RMA'd.

6. 13:25

   You basically go and request it. you return it and you get a new one and there's a lead time to that but whatever you'll get yeah you'll get some back um and then the customer never has to experience this um the um thing i'm referring to is that um like in early h100s there were like firmware bugs that may not even be related to the specific like they may not be NVIDIA they might be like super micro has a firmware bug in the um in the box that you bought um and it applies to all of them and it doesn't matter like um the solution to that was you need to get them to update their firmware um or they need to ship a new thing um in every new box yeah well commission had this for like the the b300s that that we got um yeah and the iq had to use devin to go fix it yes so like different types of problems like this can happen interesting okay so i'm like okay that seems small compared to the