Modal qualifies GPU vendors with specs, tests, and support history

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 21:55

   And I think a little bit of what we see in the market is also, this is more speculation, but I think we see, you know, smaller clouds that are a little bit more sort of risky to work with being more of an opportunity for us. Like where, you know, maybe NVIDIA, sorry, Anthropic and OpenAI, my feeling is like they prefer to buy from like the tried and true. Part of it is also because they do a lot of training runs, so they need to have like very large blocks.

2. 22:24

   but there's plenty of capacity out there you know you can get 32 nodes in armenia you can get 128 nodes in indonesia like you know like we love that kind of shit like you know so we can go out there and like get a lot of you know smaller capacity all over the world pretty good prices is it pretty much just like if someone offers you like a certain number of gpus like that's all you care about is there nothing like oh you care about like how reliable it is you know is it just some like janky warehouse that someone has these in i mean sometimes it's kind of a janky warehouse it feels like you know but but like we do have like a spec sheet basically like when we talk to vendors the first thing we do is we send them a spec sheet we say we need this number of iops we need you know like the cpu needs to be at least at least you know have have this many you know whatever the

3. 23:10

   pass mark scores or whatever it is we need to see like in a minimum you know north north south bandwidth, egress and ingress. So we do have a list of requirements. Often we try to do an acceptance testing period where we get one node and run some tests. Over time, to some extent, it also involves a little bit of a leap of faith. Sometimes we sign very big commitments, tens or hundreds of millions of dollars, and we start working with the vendor that we still don't fully know. And over time, we'll see. And then I think in a little bit of like there's a little bit of sort of just, you know, our own sort of like, you know how like semi-analysis has like a cluster max. I kind of feel like we almost like have our own like modal max score internally. Like I know which one there's I like more than others, you know, like and

4. 24:00

   And we sort of prefer to do additional deals with the vendors that we've seen, you know, very good reliability. Not just that also, like it's not just reliability. It's also, are they very responsive when we have, you know, support issues? Do they fix the issues we ask for, the turnaround time? If things go down, you know, how quickly do they fix it?