# Mistral trained its trillion-parameter model on its own compute

Markdown copy of https://realalpha.app/s/mistral-trained-its-trillion-parameter-model-on-its-own/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Sagar Vaze](https://realalpha.app/people/sagar-vaze/) · Sagar Vaze is a research scientist at Mistral, leading the multimodal team.
- Host: [Theo Jaffee](https://realalpha.app/people/theo-jaffee/) · Co-founder & host, MTS · One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.
- Host: [Sophia Dew](https://realalpha.app/people/sophia-dew/) · Host, MTS
- Show: MTS
- Episode: [Why Closed AI Providers Block Cyber Defense | Sagar Vaze](https://realalpha.app/v/gJZLJSIEbtQ/), Oct 29, 2026
- Moment: 9:52 to 13:02 of the episode (3 min), playing on https://realalpha.app/s/mistral-trained-its-trillion-parameter-model-on-its-own/; the episode from there: https://realalpha.app/v/gJZLJSIEbtQ/?t=592

## Quote

> “It's a **trillion parameter model** trained on 1E25 flops, which is a measure of the amount of compute that went into it.”
> — [Sagar Vaze](https://realalpha.app/people/sagar-vaze/) at [10:09](https://realalpha.app/v/gJZLJSIEbtQ/?t=609) · MTS

## Takeaways

1. The model has one trillion parameters and was trained on 1E25 FLOPS.
2. Mistral trained it on compute at its own data centers.
3. Open-weight releases let researchers build on each other’s work.

## About this segment

Sagar Vaze describes the model’s scale, training infrastructure and coding results. He also explains how open releases let labs learn from one another.

From “Why Closed AI Providers Block Cyber Defense | Sagar Vaze”, Oct 29, 2026. [Watch the full episode](https://realalpha.app/v/gJZLJSIEbtQ/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[9:56](https://realalpha.app/v/gJZLJSIEbtQ/?t=596) Lovable.dev. Now, on to the episode. So tell us about the model. What's new? What's different? What's exciting about Mr. All-Large 4? Yeah, so ML4, LeChonk, it's our first flagship OSS release in a while. It's a trillion parameter model trained on 1E25 flops, which is a measure of the amount of compute that went into it. This sort of flop budget and parameter scale is sort of a key milestone in any AI lab's development.

[10:28](https://realalpha.app/v/gJZLJSIEbtQ/?t=628) There was a bunch of infrastructure work that went into this, a bunch of things in the making. So first of all, actual real infrastructure. So we had to build out our own data centers. This model was trained from scratch on our own compute, which we built just outside of Paris. Also in pre-trained science, in order to be able to scale to the trillion parameter scale, being able to build predictors to understand the behavior of the models as you scale to this level and then also in post-training so to turn this strong pre-trained base into agentic capabilities requires a bunch of iteration on post-training we also did a bunch of research on that for the last 6 to 12 months and the result of it is this it's a ML4 LeChonk it's by far the best agentic coding model we've released both in our internal testing

[11:15](https://realalpha.app/v/gJZLJSIEbtQ/?t=675) and with third party evals it's the best OSS coding model. I think on the AA coding index, it's exactly tied with GLM 5.3, but needs 50% less output tokens. And then on these critical domains for us, which, as I said, are very important for our customers, things like cyber defense, finance, law, etc., where not only best OSS, but in many cases, the best model, period. Yeah, it's very nice to talk to open weights people because if I asked the same question to someone from OpenAI or Anthropic after they released their model, they would be like, oh, well, it's our best model yet.

[11:53](https://realalpha.app/v/gJZLJSIEbtQ/?t=713) We're very excited. We put a lot of hard work into it, and that's all they would say. So we are very excited. We did put a lot of hard work into it. But yeah, I think the magic of OSS is that you can share this with the community. We also benefit the other way around, right? So like the OSS landscape is in itself a community. In the past, for example, we would release, we released, I mean, people forget, we released the first open-weight mixture of experts model.

[12:19](https://realalpha.app/v/gJZLJSIEbtQ/?t=739) So it was called Mixtral. I remember Mixtral back in the day. Okay, so not completely forgotten. But 8X7B was the first mixture of experts open-weight. and now a mixture of experts are sort of the underlying technology to every really large model today. Then some of our friends in China, DeepSeek, they took this and they sort of turned the crank and they made this awesome V3 model, DeepSeek V3, which they made the model much bigger and they made it much sparser and then we in turn learned from that and if you look at our architecture that we released about a year ago, ML3, it bears a lot of similarity with the DeepSeek model.

[12:53](https://realalpha.app/v/gJZLJSIEbtQ/?t=773) So that's sort of the magic of, one of the magics of open source or also the stuff that I discussed earlier, and we can discuss again.