Containerized AI compute could support $5–35 million deployments

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:33:40

   size of these. So the DSX pod is the size of a standard intermodal shipping container. This is a shipping container computer. And that means it's portable. That means that almost like an internet packet, you want to compute on demand, you can just ship it via a shipping container. The other thing I would note, if you look at the history of compute and what the most popular form factor for compute was, it used to be monotonic. We had mainframes in the 60s, and then mini computers in the 70s, and then microcomputers, remember those, in the 80s, and then 90s personal computers. And then we got eventually to smartphones in the late 2000s and 20-teens.

2. 1:34:21

   And then the trend inverted. And now the main form factor where virtually all of the new frontier node transistors are going, it's going back in the opposite direction to larger and larger footprints. And I think what we're seeing is the dawn of the shipping container computer. That's the new form factor of the moment. It's not like transistors in your pocket anymore. It's getting larger and larger and larger again.

3. 1:34:48

   Well, really, really specific case study in that. Our very good friend Rob Fisher started a data center called Provocative on the Somerville-Cambridge line where there used to be an old carpet factory. And a huge amount of power coming into the old carpet factory, but it already has loading docks. So this product wasn't available, so it took him probably a year, year and a half to get up and running because he didn't have that.

4. 1:35:10

   With this product, he could literally have parked the shipping container right on the loading dock, run a big cable into it, and be up and running and making money in apparently two weeks. We've also talked about distributed compute inside your Tesla, inside your power wall. basically, you know, wherever the power is, being able to connect it to GPUs and connect it to bandwidth and provide, you know, distributed inference. Maybe. And I'm sure Elon will try to monetize that as part of the SpaceXAI or SpaceX SI Tesla merger. But if this theory of history is correct, that we've seen this grand reversal of the size of the form factor for most transistors, it's a falsifiable theory that would suggest actually, nope, sorry, cars as edge inference compute, not that promising. Things are trending more in the size of shipping

5. 1:36:00

   containers or larger. Two points. I think Dave's point about standardization is really, really important because you can just kind of drop these wherever. But I think the more interesting parallel here is to AWS, where they took a lot of complexity around cloud computing and made it just easily manageable. And you can see something happening here. The modularity and the standardization lead to just a plug and play and make it so easy to get up and running.

6. 1:36:27

   I could imagine electrical utilities doing this, just buying boxes of these and just plugging them in. And all of a sudden they're getting decent margin for marginal energy. Or drop them on the moon. Well, if anyone wants to do this, the minimal unit of Endeavor is about a $5 million NBL 72. You need to get up to that level of scale, 72 GPUs or above. Otherwise you can't run frontier models efficiently. But if you get at that level or above, you get about a 2x performance boost. And so if you have, and the shipping container would hold a couple of those, but the right unit of investment. It'll host seven of them.

7. 1:37:03

   Okay. So your range of investment is $5 to $35 million for the hardware. So it's bigger than building a new Airbnb, but smaller than building a hotel is your unit of effort, which is well within all local banks kind of range of endeavor. Really think about that if you're an entrepreneur in the construction world or just a business person. Permitting, local permitting will be the tricky part. It's the packet ties.

8. 1:37:27

   Yeah, which is great if you know the town, you know, if you're already entrenched with Tell them, you know, permitting at a hotel is really hard. Tell them, put it in your backyard. Don't tell them until you pay them STEMI checks for the privilege. I do think, I think we're seeing the dawn of the self-contained industrial economy. If you can fit a self-contained data center with all of its own mechanical cooling, maybe certain energy requirements inside a standardized shipping container, you can drop one of these on the moon. You could drop it into a war zone, drop it into a war zone. Yeah, this is what tiling the earth and the solar system with compute could look like. Something, some evolution of this becomes the standardized unit of not just compute, but compute plus energy, plus a bunch of other utilities. It becomes the packet of

9. 1:38:10

   civilization. Okay. So if you know the local government on the moon, that's enough. That's right. You can come to my town. I'll give everybody permission. So the full play would be you have