Snapshotting can make massive test parallelism practical

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 43:32

   use case for snapshotting? Like, what would that give us that we don't get right now? I think the big thing for me is like, today, a lot of sandbox workloads are kind of like lift and shift like people think a lot about sandboxes almost like you know i have my dev box now i'm going to shove it in the cloud i'm going to have an agent control it maybe i'll run the harness inside the sandbox i don't know if like long term that's like the right way to think about you know on like compute for I think a much better model would be you're going to have this like very long running agents, which almost like to me are like kind of durable execution workflows, like they're going to run, you know, 24 seven, you know, every once in a while, they're going to run commands and, you know, having the ability to like spin up a sandbox in, you know, 100 millisecond, run a few commands, and then like shut it down and like snapshot it.

2. 44:24

   I think the faster you can get that like you know wake up you know go to sleep cycle the more you can sort of create sandboxes that only run for like a very brief period and that will make compute much more efficient and it will also enable you to do some really cool things you know like forking and moving things around and you know having like imagine you know like clawed code instead of you know having sort of a very linear flow like sequential flow like kind of branching and running hundreds of agents at the same time.

3. 44:54

   I think a lot about CI is another illustrative example. For most of the history of CI, talk about GitHub actions and things like that, you've had a very sequential flow. You slowly install a bunch of stuff, then you run the test suite, and then generate some reports or whatever. And it's like you can break it up into shards, but it's kind of annoying. What I would love to do is take your entire test suite and just run every single test in its own sandbox, right?

4. 45:19

   like and just paralyze the shit out of that around like a thousand sandboxes why can't that be done i mean the problem is like there's too much there's too much overhead around like you know starting and stopping sandboxes so i think the more like you you create like you know frictionless like starting and stopping the more you can enter a world where like you can do this like extremely bursty like large scale like run as much as you want in parallel so you know i'm an infra geek Like I said, like I get very excited about these things.

5. 45:49

   You know, once we figure out all these primitives and the right building blocks, I think it's going to unlock a lot of really cool stuff that, you know, we previously could not do with software. Why hasn't that already been built? I would assume even previously people would want to parallelize as much of their CI as possible. You can't, but the overhead, it's like Amdahl's law or whatever, right? Like the overhead of parallelization right now is very high because like right now, like in GitHub Actions, for instance, like we parallelize in the modal test suite, But what that basically means is now if you take your test suite and you split it up in two, now each of those shards both need to do all the installs and the test setup.

6. 46:27

   And that takes a long time. It can take minutes. But let's say you can do the test setup once, and then you snapshot the container, and now you clone it, and now you can run the test suites independently. I think a lot of the benefits cannot be, you can't materialize those benefits until you get the friction and the overhead down, the transaction cost.