# Multimodal models are less reliable than text models

Markdown copy of https://realalpha.app/s/multimodal-models-are-less-reliable-than-text-models/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Diogo Almeida](https://realalpha.app/builders/diogo-almeida/) · Diogo Almeida is creator of Jev and cofounder of TypeSafe AI. · Built Jev, a decision model returning typed probabilities; co-invented RLHF and InstructGPT at OpenAI.
- Host: [Jacob Effron](https://realalpha.app/people/jacob-effron/) · AI Investor, Redpoint Ventures · Growth investor in Abridge, Ramp and Physical Intelligence; ex-Flatiron Health.
- Show: Unsupervised Learning
- Episode: [Jev Creator on Decision Models, Benchmark Fatigue & Why the Labs Missed It](https://realalpha.app/v/ham8dROHCrc/), Nov 5, 2026
- Moment: 34:39 to 37:53 of the episode (3 min), playing on https://realalpha.app/s/multimodal-models-are-less-reliable-than-text-models/; the episode from there: https://realalpha.app/v/ham8dROHCrc/?t=2079

## Quote

> “It's very measurable that multimodal models will be **less reliable than** the text models.”
> — [Diogo Almeida](https://realalpha.app/builders/diogo-almeida/) at [35:40](https://realalpha.app/v/ham8dROHCrc/?t=2140) · Unsupervised Learning

## Takeaways

1. Each added modality can lower reliability, at different rates.
2. He puts image-model progress at roughly 10% of text-model progress.
3. He wants to study easy failures before harder automation.

## About this segment

Diogo Almeida says longer context and added modalities degrade reliability, and puts image capability at about 10% of its text-model progress. He says easy text tasks should be automated first.

From “Jev Creator on Decision Models, Benchmark Fatigue & Why the Labs Missed It”, Nov 5, 2026. [Watch the full episode](https://realalpha.app/v/ham8dROHCrc/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[34:47](https://realalpha.app/v/ham8dROHCrc/?t=2087) I feel like you all obviously have released some, you know, stuff around images, right? And, you know, obviously multimodal, I imagine, is a focus. I think most of our listeners probably think of Jeb in the context of, like, you know, these decision models or type 1 things and kind of in classic, like, LLM replacement. Maybe talk a little bit about, like, the multimodal world, like, how you see the shape of problems there, how, like, Jeb models fit into those.

[35:11](https://realalpha.app/v/ham8dROHCrc/?t=2111) So there's a couple of, like, branches to go off of here. I will say branch one is that being aware of the reality of AI is so unbelievably important. And by its nature, just like long context tends to degrade intelligence no matter the model, multi-modality tends to degrade intelligence too. And this is like a well-known fact by like anyone in ML circles. And actually every modality has different rates at which it is degraded.

[35:37](https://realalpha.app/v/ham8dROHCrc/?t=2137) And this is very measurable stuff, except when you put them into chatbots, because chatbots are optimized for human preference. It becomes a very, very fuzzy thing on whether or not it's better or not. It's very measurable that multimodal models will be less reliable than the text models. And my philosophy has always been automate the easy work before the hard work. Like, will JEV models be used to automate software engineering?

[35:58](https://realalpha.app/v/ham8dROHCrc/?t=2158) Maybe one day. But software engineering, maybe I'm arrogant, seems a lot harder than like really easy tasks that LMs fail at. And in order to like be calibrated to the world, we should study why they fail at those easy tasks and like really honestly, like go in depth into studying them, not to cheat them, which I think is like quite common, but like actually like dig in and figure out what is missing and why.

[36:20](https://realalpha.app/v/ham8dROHCrc/?t=2180) And sometimes it's not the model. Sometimes it's the prompt. Sometimes it's every other stuff, you know, like it's a collaboration between software and model. But it's through that that I think we like learn the truth. I have, like, stories about how we got to even be anti-benchmaxing more. But this is a thing that LLMs try to avoid because they will look bad on benchmarks, as far as I can tell. And so that is one aspect of multimodality.

[36:45](https://realalpha.app/v/ham8dROHCrc/?t=2205) We have to take into account text tasks will be automated before the image tasks. Thing, too, is we invested really heavily on text tasks. And very much, like, we've done, like, a tiny amount of work in comparison to the image tasks. Like I am like kind of jazzed that our model does as well as it does. I think that that's great. Like as far as I can tell, our lead in images is not as large as our lead in text, but we do have an overall lead in images is my vibe.

[37:14](https://realalpha.app/v/ham8dROHCrc/?t=2234) Could be totally wrong, but that's kind of what I've sensed. And I would actually describe the image model as completely insufficient. Like I would be truly embarrassed to say that this is prod ready because there's so much jaggedness left in it that I'm jazzed if people find stuff to actually deploy. But I will say that this does not have like my seal of approval yet. We need to do so many rounds because like it's not quite as bad as like other people just shipping a shape of something.

[37:45](https://realalpha.app/v/ham8dROHCrc/?t=2265) But it's like in the grand scheme of it,