The alpha, twice a week
That was 2 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
MTS · 2 min · Oct 29, 2026
Mistral trained multimodal capabilities from the start
Sagar Vaze says the model was trained multimodally from scratch, with images included from the start. He describes the infrastructure and research needed to add modalities without regressing on text capabilities.
Sagar Vaze is a research scientist at Mistral, leading the multimodal team.
Theo Jaffee, Host · Co-founder & host, MTS
One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.
Sophia Dew, Host · Host, MTS
About this segment
Sagar Vaze says the model was trained multimodally from scratch, with images included from the start. He describes the infrastructure and research needed to add modalities without regressing on text capabilities.
From “Why Closed AI Providers Block Cyber Defense | Sagar Vaze”, Oct 29, 2026. Watch the full episode
Thanks. We'll look at and fix what's off.
“rather than training a model on text and then fine-tuning it on images and text or videos and text from the very beginning it's got images baked into it”
Takeaways
- Images were included from the beginning of pretraining.
- Images create infrastructure challenges because they are heavier than text.
- The team worked to add multimodality without losing text capabilities.
In this episode
Full episode- 14:05Closed-model refusals can leave cyber defenders without recourse2:38
- 9:52Mistral trained its trillion-parameter model on its own computePro pick 3:10
- 3:26Fine-tuning can beat prompting on narrow enterprise tasks2:13
- 5:45Compute is Mistral's main gap to the leading AI labs2:10
- 1:01Open weights let firms turn proprietary expertise into a moat2:23
16:44Mistral trained multimodal capabilities from the start2:04
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 16:48
What is that, where is it on the most important model capability now, which is being able to make really cool, like, 3D visualizations and models that you can post on Twitter? Yeah, so when we said that we wanted some early feedback, one of the things that indeed we found was that it wasn't as good at 3D generation as it should be. So I think you should cry out the new checkpoint, and I think it's pretty good at 3D asset generation.
- 17:11
So I'm biased, but multimodality means all the modalities, and one of those is text. So it's a strict superset of all the capabilities. It's a strict superset of the capabilities, and in practice for us this means understanding images videos etc and on these domains we're finding that uh this model is really outstanding is it's the best uh it's by far the best multimodal model we've trained um it's also i think uh on all of internal evals the best oss model out there at the moment and on some capabilities uh for example visual grounding we're finding it to be the best uh uh you know competitive with all the best closed source also um for us this was a real technical journey so ml3 so our model from a year ago wasn't quite at the bar that we wanted and we made some some pretty big changes in
- 18:00
pre-training uh pre-training and post-training to get to get the model up to scratch on on multimodal um particularly something i think we'll talk about in our tech report is that this model is what we call a natively multimodal train from scratch so rather than training a model on text and then fine-tuning it on images and text or videos and text from the very beginning it's got images baked into it and so uh this has a bunch of interest infrastructure challenges because, you know, images are a heavier modality than text and also a bunch of scientific things to make sure that you don't regress on the critical capabilities like coding, et cetera.
- 18:33
So a bunch of research to make sure that we could take multimodal to the heights that we think are possible and also to make sure that we didn't regress on text and that's sort of been the work for the last year on our team. Right, well, it's very exciting.