Enterprise AI is spreading across models as token use surges

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 8:17

   Now, on to the episode. You're starting to see this new paradigm. You're having, I mean, traditionally, it's humans who are calling the different APIs, and now you have agents who are calling the different APIs, which opens up a whole new slew of potential problems or architectures that we haven't really thought about. I want to get your thoughts because I know what recently just wrapped was the annual API AI Summit.

2. 8:41

   This wrapped a few weeks ago that Kong does. This is your guys' biggest in-person event ever. I want to talk to you specifically about your keynote, what you shared on the stage, because I think it talks a little bit about where we are right now and where the world is heading to. So can you just summarize some of the high-level learnings there? Yeah, we did our annual user conference. 1,200 people was a mix of infrastructure leader, AI leader, and platform leaders.

3. 9:08

   We have this concept of be AI movers. It's like Salesforce are trailblazers, and you create a movement, right? And we have this concept of AI movers. You're not really moving tokens or moving API requests. You're moving AI. But obviously, everybody can move AI, But then you have token maxing, security risk. You have to move AI safely, secure, and economically efficient. That's the only way to move AI, to make it long-term positive outcomes.

4. 9:37

   So that's how we help our customer. That's how we thought there was a general thinking where platform engineering is going back in charge into the global 2000s to govern AI traffic. Because when the traffic went kind of out of control, then governance comes back in. And so these teams, this type of budget gets back in charge. So that's another change we've seen in the last year. Another one is models.

5. 10:10

   So there's all open-weight model. Probably you've seen a lot talking about it. There was no open-weight model a year ago at our last summit. Like just talking or what, today enterprises are using like one third are now open-weight model usage. But the crazy stuff is, on average, like a sophisticated, advanced organization has at least dozens of models inside. And some of them, we govern hundreds of models, all type of it.

6. 10:40

   It's not like, oh, one big one used OpenAI and Entropic. No, they have dozens and they use also different inference providers. So it's going to this like what happened in the cloud was one model late and then it gets microservices distributed and then it got multi-cloud. In the AI era, it's the same. We started with one model to consume more model to get a bit of closer than open wake model and even going to tune and train our own models with our own proprietary data.

7. 11:04

   And so it's a perforation, a large, like an organism of this nervous system of dozens and a hundred of models. That's another change that happened, that it wasn't even discussed or a problem to govern about a year ago. Wow. Yeah, I never even thought about that. Just the fact that all the things that you have to think about and connect and govern today in 2026 just wasn't totally true back in 2025.

8. 11:28

   So just the speed that things are changing, especially in enterprises, is something we haven't totally seen before. I do want to ask specifically about your keynote. You had the title, Moving AI. I want to understand, where did that title come from? It was like, you know, we're on a mission to build the connectivity layer of AI. And it's very, obviously, it's engineering, it's a product. But to connect humanity around it, like what actually you do, you're actually helping the world move AI.

9. 11:56

   And so it comes actually from our customers to come together. to the idea, like, you know, we are moving AI. And I was like, wait a sec, what did you just say? I said, that's amazing, like, that's it. Like, if you go in a bar and meet somebody and say, oh, I'm, you know, platform engineering, you know, I'm really limiting APIs and token control cost. But if you say I move AI, like, it's very, it's like, okay, tell me more about it.

10. 12:23

    It's much higher, I think, higher humanitarian. so that that resonated very well from with the folks in the audience and so we decided to kind of make the theme of this year i like that this is the main theme of this year how did this differ from last year's theme what was last year's theme what was this year's theme what specific trends do you notice that inform what the theme of the conference will be so last year we all the bet we made actually happened a year later but it happened faster than what we thought last year thing was agentic enterprise agentic enterprise nobody people weren't quite talking yet but like now it's all over now enterprise are like one percent to agentic enterprise what they really use is the chatbot and coding harness that's it a little bit of agentification in support business

11. 13:08

    unit but 95 95 of the real enterprise has not been agentified which is the whole opportunity but we thought we say like we were going to the age of ai connectivity that was the big thing Because AI had to, in the digital world, you know, was networking. In the AI world is connectivity. And we're going through different models, different protocols. And we needed help to connect models, agents, and APIs together.

12. 13:34

    So that was the big bet. Enterprises were starting to become agentic in some work streams. And companies were going multimodals. We did not envision open-way, but we envisioned a multimodal world. So that's kind of the bet we made a year ago. And then, like, here we are. Wow, that's interesting. So you guys had the certain bets a year ago that you were too early on a little bit or you just were able to predict before.

13. 13:55

    What's that right now? So if you were to have to predict what maybe next year, 2027's big API AI summit would be about, what trend do you see happening that isn't totally true right now? Yeah, it's the same like direction of a genetic enterprise, but more pieces of organization will be actually identified. So that's one, meaning not just support chats, but it will be other work stream in our workflow.

14. 14:23

    The numbers, I think, of models will keep growing exponentially. The numbers of tokens, by the way, is growing exponentially. Like we, for what we can see, because some enterprise are here gapped, we don't see everything. But for a subset that we see, we are running about 500 trillion tokens, going to quadrillion soon, because it's growing 140x year over year. And that is like 100 times bigger than a year ago.

15. 14:49

    So the amount of tokens enterprise are using, it's insane that the cost per token is going down. And I think next year is going to go even more pragmatic that the cost per millions of tokens going down. Inference will still be exploding. So we're going to live in this enterprise war that is going to be more multimodal, cheaper tokens, cheaper level of intelligence, and more about how do we secure it?

16. 15:14

    How do we govern it? How do we cost control it? Those are going to be, I think, the next thesis to making a genetic enterprise an economically efficient outcome. Yeah, how do you cost control all of these tokens? I think that's one of the big areas of concern