GPU sourcing depends on location, timing and fit

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 14:47

   that's the business i'm in so are you aggregating gpus you've raised 230 million reported and announced i don't know but is are you aggregating gpus yeah we are not i won't use the word aggregating we are buying we're renting gpus uh to build the scale of search traffic that we are seeing and expecting uh and as i said we grow alongside inference so inference is growing so we have to do a proportionate amount i don't know if it's like already a i don't think it's a core competency of ours like we have been in multiple mad gpu scrambles yeah in the last three months to be like oh shit we need gpus we don't have gpus can someone figure it out and like you're making a bunch of calls and like who are you calling sorry who are you calling everyone everyone like every hyperscaler every neocloud what's like the typical response back because

2. 15:52

   this comes up in pretty much every conversation now it's like how do you get I guess the allocation of gpus no I think it's like I think At least for us, I think it's true for many people, it's like, okay, which GPU is it? Physically, where is it? Sometimes we need GPUs within five milliseconds of where a large web index is sitting. Other GPUs are sometimes more flexible with multiple GPU workloads.

3. 16:21

   It's not just like, okay, do you have GPUs for me? It's like, do you have the right GPU in the right scale, in the right physical location? for me available at the right time and that is what makes the problem much harder i think so far we have managed to solve every problem we have encountered we've not had to turn away a customer because i didn't have a gpu right but we've had a bunch of gpu acquisition fire tilts i mean that's a really interesting line to go down it's like what is the sourcing process like for gpus like what are the main questions that you're asking what do you need yeah so i think you mostly so i think different workloads are different but for bulk of our workload it's like how many gpus which types are you open to when do you need them and where must maybe like there's too much more to it.

4. 17:23

   I think for certain large training clusters, there's a little bit more to it. And then for us, like, are you willing to commit enough duration to that physical thing? Yeah, but that's, it's not complicated. The reality is that I wouldn't say the market is entirely efficient right now. Like a lot of it is on iMessage. Really? Yeah. So, so yeah, but it works. the market works well that's crazy so you're doing deals on iMessage for GPUs not exactly figuring out availability on iMessage and then like the finance person then calls the finance