# Self-hosting open models can cost more than it appears

Markdown copy of https://realalpha.app/s/self-hosting-open-models-can-cost-more-than-it-appears/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Aidan Gomez](https://realalpha.app/builders/aidan-gomez/) · Aidan Gomez is a Cohere co-founder and Transformer co-author. · Co-founded Cohere, the sovereign AI company, and co-authored the Transformer paper, Attention Is All You Need.
- Host: Eric Newcomer
- Show: Newcomer Pod
- Episode: [Aidan Gomez Unfiltered on AGI, China & AI's Biggest Risk](https://realalpha.app/v/GfBEBfbW6YE/), Jun 26, 2026
- Moment: 6:08 to 9:43 of the episode (4 min), playing on https://realalpha.app/s/self-hosting-open-models-can-cost-more-than-it-appears/; the episode from there: https://realalpha.app/v/GfBEBfbW6YE/?t=368

## Quote

> “In reality, I think it's actually **quite expensive to adopt** open source models, and it looks cheap.”
> — [Aidan Gomez](https://realalpha.app/builders/aidan-gomez/) at [6:50](https://realalpha.app/v/GfBEBfbW6YE/?t=410) · Newcomer Pod

## Takeaways

1. Token prices don't include infrastructure and maintenance.
2. Self-hosting means keeping model-serving software current.
3. Enterprise customers often need customized models.

## About this segment

Aidan Gomez says open models look cheap until teams account for infrastructure and maintenance. He explains why Cohere builds customizable models rather than competing at the frontier.

From “Aidan Gomez Unfiltered on AGI, China & AI's Biggest Risk”, Jun 26, 2026. [Watch the full episode](https://realalpha.app/v/GfBEBfbW6YE/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[6:11](https://realalpha.app/v/GfBEBfbW6YE/?t=371) I want to talk about the response or one answer to token maxing, which is open source or the idea that people move away from state of the art models to find, you know, I think there's a lot of conversation about routing to various models in a cost effective way. What is your view, I guess, putting aside the geopolitical part of it, which we'll get into, what's your view on sort of the appeal of open source models from a sort of cost structure point of view?

[6:40](https://realalpha.app/v/GfBEBfbW6YE/?t=400) I mean, in some ways, they're cheaper if you're willing to build up all of the infrastructure around them, like all of the software stack, the harness, maintain this thing, believe that you have the best serving efficiency. see. In reality, I think it's actually quite expensive to adopt open source models, and it looks cheap. If you're willing to use, like, DeepSeek's API or Alibaba's API, you see the token prices, and you're like, oh, look, open source is cheap. These companies release open source models, and their prices are way lower than the American equivalent.

[7:17](https://realalpha.app/v/GfBEBfbW6YE/?t=437) What's wrong with that thinking? What's wrong with that thinking? Well, one, you have to be comfortable sending your data to those parties. If you try to self-host, I think it's a very different equation. And then there's also the maintenance burden of that, right? Like these harnesses, these pieces of software that the models power, you're going to have to keep that updated over time as capabilities evolve, new models come out.

[7:41](https://realalpha.app/v/GfBEBfbW6YE/?t=461) And so that's another huge expense. I think similar to databases, in the same way, yes, you could take an open source database, self-host that thing, try to maintain and manage it. Nobody does that. Like very, very few people do that. Instead, it's all managed services. There are enormous databases. companies because it's a hard, painful, and expensive thing to operate. So part of the pitch of coming to Cohere is that you don't want to manage your own service, models are going to keep changing, we'll be here with you, make sure it's safe.

[8:14](https://realalpha.app/v/GfBEBfbW6YE/?t=494) How do you think about why are your models shifting to being more open source in the first place? And then why Cohere build models versus sort of being a facilitator of your own models and other models? So on the first question, like the, yeah, I think the piece around which model you use matters a lot. And for Cohere, it's very strategic for us to build our own models because we offer more leverage to the customer. Increasingly, what we're seeing is folks care a lot about cost. They care a lot about sovereignty and off the shelf models aren't good enough or they're too expensive for what they need to do. So the ability to customize becomes an essential part of the product and offering to our customers. The other thing is that we're not trying to compete at the very frontier. We're not trying to get into a like who can spend the most

[9:13](https://realalpha.app/v/GfBEBfbW6YE/?t=553) money competition. It's just a losing race. And increasingly what we see, the model is not the bottleneck. It's actually the market that's the bottleneck. So frontier models are so far out in front of what the market is doing that even if you have a model from a year ago, 18 months ago, two years ago, the companies are still trying to catch up to that. So the model itself is largely not the bottleneck. And when it is, they can't afford these massive models. And so they're