Real Alpha
Watch modeSubscribed

Latent Space · 3 min · Oct 22, 2026

Small training is underserved, while fine-tuning is optional

Erik Bernhardsson

CEO, Modal

  • Shawn Wang (swyx)Host · Co-founder, AI Engineer

    Wrote “The Rise of the AI Engineer” and founded Smol AI, maker of AI News.

  • Alessio FanelliHost · Founder, Kernel Labs

    Partner at Decibel, backing technical founders who build developer tools.

About this segment

Modal sees demand for modest training runs from researchers, biotech and traditional ML, not just giant clusters. Erik says open models have made fine-tuning optional for many users.

From “Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal”, Oct 22, 2026. Watch the full episode

“Today, I would say, like, the open source models are so good that, like, you can basically just, like, swap them out”
Erik Bernhardssonat 40:41 · Latent Space40:41

Takeaways

  1. Smaller workloads can train on dozens of GPUs, not 100,000.
  2. Modal sees demand in biotech, speech and recommendation systems.
  3. Strong open models weaken fine-tuning's old quality advantage.
Watch 2:36 from 39:14

In this episode

Full episode
  1. 18:04GPU price drops can be riskier than continued scarcity3:39
  2. 43:35Fast sandbox snapshots could make parallel work practical3:39
  3. 52:58Good developer experience also helps coding agents4:46
  4. 13:37Modal built a procurement team as GPU supply tightened3:41
  5. 28:28Neoclouds scale by pre-selling capacity to finance GPUs3:57
  6. 58:05Modal uses forward-deployed engineers to turn customer work into product4:17

  7. 39:14Small training is underserved, while fine-tuning is optional2:36

  8. 1:21:24Inference providers can shift capacity as model demand moves3:29Up next

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 39:26

    the bigger enterprises doing huge training runs? No. Okay. No, they're going to go and do these big hyperscale or rather NeoCloud mega deals and get 100,000 GPUs or whatever. But there's a little bit of an underserved market of, in my opinion, researchers iterating or people training smaller models. There's text-to-speech and speech-to-text models. Competition and biotech is actually a market that we see a lot of demand from.

  2. 39:56

    We have a lot of customers in that space. Training relatively modest models. You can train on 64 GPUs or whatever. Recommendation system. A lot of sort of traditional machine learning. There's a lot of things where you don't need 100,000 GPUs. That market, I think, we're not going to focus on that for a while. But I think there's a lot of stuff in the slightly smaller markets that are relatively underserved.

  3. 40:18

    It's very hard to set up training clusters. Yeah. Have you noticed since, I don't know, GLM 5.2 or maybe like Kami 2.5 that a lot of people are using those and fine-tuning them? If anything, I've actually seen the other direction. I feel like fine-tuning the value prop a couple of years ago was like, yeah, open source models, they're not quite as good. But if you take a small model and you fine-tune it on your specific domain, you can actually reach the frontier in terms of, you know, quality and cost.

  4. 40:55

    Today, I would say, like, the open source models are so good that, like, you can basically just, like, swap them out and just run, you know, instead of, you know, the frontier ones, you can run, you know, Kimi K3 or whatever, DeepSeek 4.1 or GLM 5.3. And they're, like, basically almost as good as, like, the frontier model at a much lower cost. So we still see a lot of demand for post-training. And I do think it's a really interesting space.

  5. 41:19

    and we're working on some products in that space, both sort of SFT but also reinforcement learning and things like that. I would also say fine-tuning is no longer the thing that you kind of have to do. It's very optional. Open source models work really well just off the shelf too. Back to sandboxes, actually, I forgot to ask you.