Real Alpha
Watch modeSubscribed

The MAD Podcast · 3 min · Oct 22, 2026

Mistral’s in-house cluster teams fixed issues in two nights

Pierre Stock

VP of Science, Mistral AI

Mistral's first employee, now VP of Science, who leads model releases including the 1-trillion-parameter Mistral Large 4.

  • Matt TurckHost · General Partner, FirstMark Capital

    Publishes the annual MAD landscape of data and AI; sold TripleHop to Oracle.

About this segment

Pierre says colocating pretraining and compute teams let them debug a sensitive cluster in two nights. A cloud setup tailored to Mistral’s code could have taken weeks.

From “Mistral Is SO Back: Inside Le Chonk, the West’s Biggest Open Model”, Oct 22, 2026. Watch the full episode

“I think it could have taken like weeks to set up given our particular code base”
Pierre Stockat 12:54 · The MAD Podcast12:54

Takeaways

  1. Cluster communication can erase the gains from top GPUs.
  2. Pretraining and compute teams debugged together in the same room.
  3. A cloud setup could have taken weeks for Mistral’s code and tuning.
Watch 2:57 from 10:28

In this episode

Full episode
  1. 46:42RL training depends on calibrated tasks and resilient infrastructure5:40
  2. 27:28Mixture-of-experts models trade training complexity for inference capacity4:58
  3. 1:03:20A model pinged its trainer from inside a sandbox4:01

  4. 10:28Mistral’s in-house cluster teams fixed issues in two nights2:57

  5. 18:40Mistral trained Le Chonk with less compute than Chinese rivals2:33Up next
  6. 53:35Mistral trains domain skills with tests, rubrics and simulations4:43
  7. 1:07:40Open-model guardrails raise the cost of malicious use4:34
  8. 1:14:14Mistral says its models speed senior researchers up two to four times2:17

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 10:31

    We had Tim on this podcast a few months ago for Mistral for LeChonk. you trained the model on your infrastructure in Europe with 3,800 NVIDIA Grace Blackwell GPUs. Was that a big part of the unlock that facilitated that comeback story? Yep, exactly. So I think we have more compute than that and we have plans to expand it by orders and magnitude already. But there's a lag, right? Typically, this computer is not actually founded by our latest series D, the 3 million one, because it takes time to actually build the data center. So it's founded by the previous ones. So we plan to scale much more, but I think there I have one story that basically tells you why it's so important to build your data center from a technical point of view. There's many business reasons, of course, in particular the law

  2. 11:33

    under which to reach the data centers are subject to. of course but like my story is that we started the pre-training so like the very first you mean from regulatory standpoint so from a regulatory standpoint there's a big advantage in having for a customer in having a data center in the countries in which you operate and that are not subject to other laws that you wouldn't like to deal with i think that's that's important but from a technical point of view we when You started first training the models from scratch. We put in our room both the pre-training team from the same side and both the compute team from the cluster side. And we had two nights of health debugging because it's hard to actually make a cluster work. It's a bit like, I always say, it's a bit like a formula one, right? Everything is so sensitive. The basic example

  3. 12:23

    is that if you have great GPUs, like the best in the world, but the communication between them are a bit lagging, even by a few dozen hundred milliseconds, then you lose all the benefits of the fast compute because the teams are waiting for each other. And that's the kind of balance you need to find on the cluster among like tons of other stuff. And actually, this interaction in the same one room of two teams that are the best in the world, some of the best in the world, actually, if we had to go through a cloud provider with that, But I think it could have taken like weeks to set up given our particular code base and the way we want to optimize and to get the best throughput.

  4. 13:07

    So that's actually, I mean, having a compute is like a blessing from a technical point of view. And also for the customers, for our customers, it's the insurance that the data stays in the country they chose to stay.