Large companies report AI returns before full organizational change

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 7:28

   this ROI, they're also telling their employees to pull back on the spend. And this era of, you know, we're going to spend tokens freely, you know, that's ending. And so, yes, there's some ROI, but what their actions suggest is there's limited ROI right now. I will disagree with you again on this. So first of all, we keep whipsawing between different views, right? Like token maxing was big this spring, and then it was like, oh, no, everyone's spending too much on tokens. It is becoming a fast-growing expense. Companies are worried about this. When I talk to a lot of Fortune 50 companies, though, they are viewing this as something that is an R&D expense to try and get under control to figure out what's going on. A lot of them don't know in the same organizational change ever, like I was talking to a very large company the other

2. 8:14

   day, and they're saying some people are 100 times more productive, and some people aren't really getting any benefit out of AI. We don't know yet how to spread the 100 times more productivity to more people. So I think that there's news articles about people pulling out. Now, most of the people who are talking very publicly about how they're cutting AI spend tend to be people with competing AI products. So like the fact that Microsoft, Meta, and Palantir are telling you we're spending too much money on Claude, I don't know if that tells you that much about the world, right?

3. 8:40

   Our revenues are still growing for these companies. Absolutely nobody has cost under control, use under control, but I don't see any sign of people saying they're pulling back because AI turned out to be useless. Not useless, it's just the question is how much return are you getting on the investment? I mean, companies are getting large returns that we're talking to, but they're also in experimentation and evolution phase, right?

4. 8:59

   You can't get returns without making changes. Most organizations I know know that. Taking changes takes time, right? And they need to figure out how to swallow this new technology. So I think that there's too much of a sort of back and forth, like AI is changing everything, AI is changing nothing. When I look inside organizations, right, C-level people are deeply spending a lot of money and a lot of time trying to figure out how to make things operate well.

5. 9:23

   I'm seeing transformation in lots of organizations, including really old school biopharma companies and financial service companies, very large ones, who don't want to talk about exactly what they're doing yet because it's a source of competitive advantage. And I think people are worried about token costs, but that's often a management problem. Like if you don't make any decisions about how tokens should be used, right?

6. 9:42

   Who should be using them? What process should be using them? Then, and you incentivize people to use as many tokens as possible, they'll use as many tokens as possible. It feels like, I feel like we're kind of in this chicken and egg where we're like, you know, where is the return? I keep telling you it's there because companies keep telling me it's there. And the surveys show 85% of companies have positive ROI from AI as of last year.

7. 10:03

   So I think though the major transformation hasn't happened yet, except it's starting to in some organizations. But again, you have 100,000 employees. How are you supposed to even think about mapping out all the processes in your company? How do you start deciding about who gets to use tokens under what circumstances? How do we start to adopt the genetic models when they're basically less than a year old?

8. 10:21

   There's a lot of stuff happening all at once. Yeah, okay, so let's go bit by bit here. So first of all, you've said that the companies are already starting to see ROI. Where are they seeing that ROI? Where are the areas that they're seeing it? So in terms of, so productivity and cost savings are already there. I've actually been worried that companies are pushing too much and productivity is the main goal.

9. 10:42

   So my colleagues at Wharton have been surveying a thousand large companies. And like I said, last November, a year ago, 85% of large companies with over a billion dollars in revenue reported positive ROI from their AI use, right? We don't have detailed breakdowns on every piece of whether it's cost savings or spend, right? but you must have some anecdotes of like a company put uh you know ai into action and yeah i mean sure but but so tell us tell us the stories i mean a lot of them i can't even talk about necessarily i mean so i talked to uh you know there's a another A Fortune 100 company that gave its senior leadership team access to basically open clause with access to the full company data and completely changed how management works.

10. 11:23

    Like C-level teams are using this as a super analyst to find problems inside the company they weren't able to identify. I have spoken to a large financial services company that has launched a whole bunch of new product lines that they never had before as a result of AI. Talked to a couple of very large hedge funds, the PE funds, that are seeing huge returns to management technology right now. And all the companies are growing faster inside their portfolio.

11. 11:48

    Like lots and lots of cases like this, right? Oftentimes, they don't want to talk about it in huge detail. And they're all still in the experimental phase. I mean, a lot of these are outside of pilots now. They are rolling out at the organizational level. But it's tough because it's a complicated kind of set of choices at the company level rather than a product you adopt and get an instant return from.