Only 5% of large enterprises run non-engineering agents

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 19:28

   Yeah. So one way I think about this is I think about enterprises adopting AI as almost like these different milestones in a race. And I think, especially at AIE, when I tell people about what percentage of companies have reached these milestones, they're kind of like mind blown by it. But it's like, OK, first milestone in the enterprise AI race is just rolling out a general purpose LLM to your people.

2. 19:53

   And I'd say of the, you know, let's call it 100 enterprises we're talking to today, 90% have rolled out either Claude or Codex to their employees. Then the next milestone is like actually teaching people how to use it, like training and enablement, which like never gets the love that it should get, but is arguably more important or as important as the technology itself. I would say that's at like 60%.

3. 20:21

   Oh, that's pretty good. Yeah, yeah. Like there's some group there of companies that are like, our people will just figure it out. But then there's like a good chunk that is like, okay, we need to teach our people what skills are. We need to teach our people the features and functionality of these surfaces. Then the next after training and enablement within a general purpose LLM is coding agents. And the crazy thing is like, I was asking a few people here, like what percentage of enterprises do you think have rolled out coding agents to the engine product org in their company?

4. 20:52

   And most people are like, it has to be 70 or 80%. The number is closer to 50%. Like every other enterprise that we speak to either hasn't rolled out a coding agent or what they are using is closer to not having a coding agent than having a coding agent. Then the next, and this is where like the biggest, I would say, drop off is, where like the biggest diffusion gap is from like raw intelligence to enterprise usage, is agents in production for non-engineering work drops to like 5%.

5. 21:22

   Wow. It's like, I mean, you're in these conversations all day long, but like most enterprises that we talk to that have, let's just say more than a thousand employees do not have an agent in production that is doing work that is not building software. Sure. And so I think there's a few things. One is there is because like there's just this whole wave is driven by a combination of some small group of people understanding what is possible from this technology in a way larger group of people driven by FOMO.

6. 21:53

   And by the way, FOMO isn't bad. It creates some forward momentum. But it also means that people will latch onto the things that are most in the zeitgeist. I guess now for the last year and a half, people have been saying it's the year of the agent or however long it is. Every year is the year of the agent. Like enterprises are like, can you help build us an agent for some back office task within the organization?

7. 22:14

   Like that is one of the most common things. And it's why now a lot of the work we're doing at 10X is deploying agents into production outside of the entor for companies for repeatable work that they're doing. The second is permissioning and like RBAC. Basically, as every employee in a company has Claude or they have Codex and they need access to certain data, it is turned into an absolute nightmare for CIOs and CISOs.

8. 22:41

   and they don't have basically a good gateway for easily doing almost like a control panel for data permissioning. Literally just what can you read, what can you not read. Exactly. And that is a huge issue. So that's like something we spend a lot of time on. So you make them for people or do you like buy something off the shelf? So basically we started by building bespoke for companies and now we actually productize that where we have our own gateway solution.

9. 23:07

   Exactly. Exactly. Is there anything else that comes to mind that's like we're hearing a lot from customers? I think that covered what we're hearing when it comes to like particularly the agent deployments. I think one of the pains that just always gets discovered, I phrase it to the team, is like you can't automate a process that does not exist. So when you have 40 people that are all roughly doing the same function, but they're all doing it a little bit of a different way, there's actually it's it's way harder to build an agent that's kind of proactively doing that in the background you can totally sit down with them give them the tools walk them through how they could augment their own workflow but when you really start to get the leverage is when you have these agents that are surfacing work to you you're reviewing it you're throwing it back

10. 23:53

    and you become more of this like you know everyone's been talking about becoming an agent manager and i think we're on the cusp of that but when you zoom out oftentimes our recommendation to companies is like, hey, first, we need to build a process around this organization within your company. And then here are the steps for us to actually hand off that work to an agent in the background. But if we don't have some form of standardization, it's going to be impossible for the agent to know whether it's doing it right for us to build evals around this system and actually deliver the value that an agent that is doing this function would have. And even one thing I'd be curious your thoughts on, we were literally talking about this right before, is