AI subscriptions hide the real cost of a unit of work

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 12:06

   So with Uber, for example, a lot of their money was marketing and R&D. I like to equate Uber to Groupon more than anything. Heavy, heavy marketing, and that was where they did the subsidies for the rights and all that. Also, R&D efforts within autonomous driving that I think went nowhere. Forgive me if you're not having looked recently. Also, the burn rate on Uber was just a different planet. I think they burned.

2. 12:28

   Way lower. It was scary then, but now it seems like a joke. It's about $32 billion in history. And Anthropic has raised over $75 billion in the last six months. So I think that that's altogether different. And also the level of burn, the scale of burn, and the linear reality of it. It's a word, I think. Basically, as Anthropic's revenue increases, so do its costs. Except the costs are always up here.

3. 12:52

   Same with OpenAI. Uber had a messy cost structure, but they didn't change the unit of work. their unit of um revenue so they didn't have a thing where it wasn't they tried the ride pass where you could just pay one fee and drive as much as you want but you still paid for a ride a customer was still paying to go from point a to point b with the subsidized ai labs subscriptions they are training people and they've trained millions of people to use these things in a way that does not make sense and they pay the costs the reason that you're seeing companies burn so much money is imagine if you were driving your car and you just never really had to think about fuel fuel was just included with it perfectly and then one day and so you actually no let's reframe this use a cab service and it will drive you as much as you want you get 300 rides a month

4. 13:37

   you could drive from la to san francisco you can drive from just the upper west side to the upper east side no matter what it's always the same costs then suddenly one day they say actually gonna have to pay on a mile per mile basis you would have no idea how to plan your life you planned your life around this subsidized unlimited thing and perhaps you could say the the oh the ride point to point from the original uber was like that no this is like if every uber ride used to cost three dollars and the new ones cost three hundred five hundred a thousand dollars people do not know how much their ai costs they have no idea how to measure a unit of work because they've never had to think about it like if you were used to just loading up claude going fix these bugs and it kind of does it and then it messes some stuff up and you poke it bonk it on the head a few times

5. 14:24

   maybe it gets it right maybe you mess around with it but you don't know what it costs and you weren't paying the cost suddenly doing that would give you cte look at the people so github copilot which is microsoft's coding tool they just moved from a premium request model to a token-based billing go on the subreddit people are screaming like they're being stung to death by bees because they are finding they're burning through their entire monthly allotment which is just the price of tokens in like a day there's someone who did it in three prompts there's so much i want to respond to this so i'd like to take a move sorry but i do the specific one major contradiction that seems to appear to me in your argument is that people see the cost being high and they freak out because they want the product desperately like how do you reconcile with the idea that

6. 15:14

   But it's like people who are bullish on AI see the fact that people want to spend money on it, want access to it as evidence that it is doing well, that they see some value in it. Like, do you not see that? I do see value in it. I'm not disputing that. I'm saying that the value they get out of it is based on the idea of subsidies because they have evaluated it based on it not costing what it really does.

7. 15:35

   So if you use the product and you could do all this stuff with it, however accurate or inaccurate it is, you can say, well, that's a great deal at $200 a month. is it a good deal at 300 bucks a week probably not you wouldn't be able to do as much and i also think psychologically you look at it differently if something costs 200 bucks for a month and you run up against rate limits fine you can accept that i want to talk through the uber analogy as i see it i mean I don't know.

8. 16:00

   Yes, I think Uber was a similar case where people felt like, oh, it might work at one price, but it won't work at another. And I think what happened was people got really upset in this amazing period where Uber was overpaying drivers relative to what they needed to pay them to get them to do the work, discounting rides to keep the riders from going to Lyft. And basically we had this like free giveaway and there was like a crowd of people online who were like very angry about this even though they were getting stuff at a big discount.

9. 16:35

   My treats. Right. They were getting it. And so then that period ends. Basically Uber and Lyft sort of come to a sort of equilibrium where it's, I don't know, it's probably 85, 15 or even more aggressive in Uber's side. And yeah, I mean basically they have to squeeze drivers more especially on airport trips. but there is a price where people take Uber. It works. It's not like the insane Google level business that people thought. And things like Uber pool, which we're supposed to allow it to sort of like get to scale don't work. So there are pieces. I just don't think there's this like something's valuable at one, you know, it's sort of a sliding scale. And I feel like we didn't learn the lesson from Uber that, you know, that this sort of like big hustle was had on us right but here's the thing when you take from brooklyn to grand central 50 60 bucks it's

10. 17:27

    still within the realm of what you would pay for a cap i am saying that what's happening with ai now is people went from just for the github copilot example this is not hyperbole there were people who were paying 39 bucks a month and being able to burn 1500 to 3000 dollars worth of tokens that is not the same that's not the same at all that would be if like uber rights were five dollars and then became fifteen hundred dollars or three hundred dollars it's just a different economic scale and also you can plan around that still even though it's more expensive you can say okay this ride was 10 bucks damn i wish it was still 10 bucks but it's 50 bucks now with ai it's you don't know how to measure a unit of work that's the biggest thing like it's tough across models across harnesses use open code cloud code using a deep seat model using jpt 5.5x high

11. 18:15

    plus alpha you're using whatever you're using each prompt oh what are you having it do it might act differently because hallucinations are a mathematical guarantee per open ai so you don't know how to measure the unit of work you also pay regardless of whether there is a mistake or not if it gets it right if it doesn't get it right you are still paying on a per million token basis so it's not like you can say well okay was spending 200 bucks a month now i'm spending 500 bucks a month you're unaware you have no idea the more complex the work you do and because ai code is very verbose the more ai you use the more convoluted things are going to become and the more reliant you are on the machine to kind of work out what it's doing and those costs are dramatic they are unrealistic for most people they're unrealistic for most organizations the models that we have