AI subscriptions can hide the true cost of usage

Transcript

From the transcript The Diary Of A CEO publishes with the episode.

1. 12:00

   We've just found a slop machine. But then also there's the problem of cost. So when you use AI services, you burn tokens. And it's per million tokens. What's a token? So it's around three quarters of a word. So it's characters. So the AI companies have a currency in which they charge you, like a taxi in New York has a meter. Yeah. And they call it tokens. Yeah. And every word, let's just say for ease it's a word.

2. 12:23

   Yeah, about a word. Yeah. And it's per million tokens. So you'll be charged per million input tokens, the stuff you feed into it. like a document or a code base. And the output tokens are both the stuff it spits out at the end, but also when it thinks. So, okay, you've asked me to give you the best restaurants in this area of New York. I should find the best restaurants in New York. All of that's output tokens as well.

3. 12:45

   However, when you're paying for a monthly service, you don't see any of that. Put all that crap to the side. They just have rate limits. So you can use them a certain amount, and then when you run out, but they kind of obfuscate what that was. Now... Someone recently found, Semi Analysis actually found this, a big analyst group. They found that on a $200 a month chat GPT subscription, you can burn $14,000 worth of tokens.

4. 13:08

   And on Anthropix, you can burn $8,000 for... 200 bucks that is how most and even on the 20 buck a month service you can burn 400 dollars now most people don't realize that most people have no idea what ai costs most people just think oh it's 20 bucks a month no all of these companies run a horrifying loss open ai lost 20.9 billion dollars last year because people can burn as many tokens as they want And when they tried to move everybody on the enterprise side, so companies bigger than 150, onto actually paying the cost of AI in around March of 2026, to quote Sam Altman, they said, people have a big problem with it, I think.

5. 13:47

   It's a huge issue, which is not really what the... heir apparent to tech's history is meant to be saying but the point is enterprises immediately started freaking out uber burned through their entire annual token budget in three months so suddenly after everyone's saying ai is the most productive thing ever it's amazing it's changing everything the moment people actually had to pay for it they go oh i don't know actually um maybe it's it's obviously we all love it it's all great right But it's costing too much, so we now need to reduce the cost.

6. 14:18

   Because people are just dumping stuff into it, being like, what do I do here? And getting whatever the median is out. Because that's what these things do. They provide the median answer. So essentially, someone like me who's a power user of these tools... I could be costing Anthropic or OpenAI $1,000, but they're only charging me $100, let's say. So they are having to subsidize $900 of my usage because of the electricity costs and the costs at their data centers.

7. 14:45

   And so your assertion here is that that is unsustainable. Yes. And just to be clear, they're probably not one for one dollar. It might be 34. We don't we don't know. I think it's unprofitable. These companies don't disclose them. Even in their auditive financials, they play funny games with how they categorize things. But nevertheless, yes. And on top of that, the way that you stand up inference, which is the thing that creates the output within these data centers, is...

8. 15:08

   You're not just saying, okay, turn the inference machine on, let's go. You are standing up the GPUs necessary to take in the demand, and if you buy too much, you've wasted the money. You have to pay for the hourly GPU use regardless. If you buy too few, your customers can't use it. They get pissed off at you, they can, so they go with someone else. But nevertheless, yeah, they would get demand selling $20 or $40 for a dollar.

9. 15:32

   And that's what these services do. And really, the simplest way to explain it is, If they were actually profitable, if they believed that these services were worthwhile and that they were worthy of the cost, they'd charge it. Regular people wouldn't be able to get a monthly subscription. They'd just be paying what it's worth. Unless, of course, there was an economic problem. And it's very simple. You pay when you use an LLM, regardless of whether you get what you want.

10. 15:57

    When these things hallucinate, say you're doing something, you're coding something, and they go through a code base and they fuck up a bunch of stuff, they break a bunch of stuff, you're paying for that. You're paying for it whether it works or not. Unless, of course, you're using one of these subscriptions.