Model approval can stall enterprise AI agents

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 13:08

   There's a lot here. And it's not simple. The reality is every enterprise is a little different. And what they need is different. And what they need for different workloads is not always the same. So, sure. In a lot of regulated enterprises, people have AI helpers. doing work, there's not a lot of regulated enterprises today who have unleashed autonomous agents to run mission-critical work. And that's a maturity thing for the industry part of it.

2. 13:40

   Yeah, stuff like SRAS, you know, the reliability of the system and is the model always there and able to respond uptime, et cetera. But it's way more than that in most of these organizations. And I've got customers who are well down, and I think this is true, by the way, of a lot of technology providers, customers who are well down the path of an agentic system they have built and designed and tested and they're super excited about.

3. 14:07

   And it's sitting in a compliance backlog today somewhere in their organization where it's pending the approval of the use of a specific AI model under the covers for that work. And the obstacles that get in the way are things like, well, we need to understand whether all the data used to train that model was legally acquired by the person or the entity that trained that model. There is a set of requirements that exist in many of these organizations that make it extraordinarily challenging to trust a workload to an autonomous agent.

4. 14:54

   And this is some of these are just like, all it's doing is reading data out of PDF documents and turning it into structured data. And yet, there is this type of compliance concern that slows progress. So I think enterprise ready means a lot of things, right? Yes, of course, it means the systems have to work when you need them. But it also means that they need to meet a rather extensive set of requirements that are non-functional in, you know, not just like does it do the thing when we press the button or say the words, but does it help us meet all of our regulatory requirements?

5. 15:37

   contractual requirements, et cetera. So there's a lot when it comes down to what enterprise-ready means in the market today. And a lot of it hasn't changed with AI. Some of it has. That whole model provenance question, that's a relatively new one. But a lot of it isn't. How do I know exactly how every bit of PII is being managed in this complex workflow across six systems to get this thing into production. And that's a lot of where the governance comes in, where a lot in the form of the security model comes in. So, how do I ensure that the model is executing with access to only the data that's appropriate for that? It's not consuming stuff that it shouldn't. And those are all the kinds of core primitive capabilities you need in a platform to be ready for enterprise AI.

6. 16:36

   When it comes to these apps that are built and in this sort of compliance backlog that you describe, and feel free to pick it apart and subdivide your answer, but does that strike you as just prudent risk management? Or is that ultimately a mistake by leadership to not expedite some of these things in the spirit of getting the transformation that I think everybody kind of agrees is inevitable underway now?

7. 17:06

   It's certainly some of both. The calcification processes inside organizations is real in large enterprises and is something that organizations need to challenge and break in order to advance. But a lot of it is also very real. In a lot of places, the regulatory constraints are significant, the liability risks are big, and reputation risk is also high. And so a lot of this is prudent decision-making. And the reality is organizations are taking different risks in different parts of their business for different kind of workflows, right? There's one level of risk I'll take for managing, for example, the allocation of IT resources to people who work in the company.

8. 17:58

   banks need that insurance companies need that right regulated enterprises of all kinds need that kind of stuff take more risk there than perhaps the loan origination process where they are making decisions contestable by regulators and they need to be able to prove right what and how the decisions were made at every step of the process because it was all auditable. And so they need to make sure that they have what they need to be able to defend themselves in the case that they ever need to for any decision they're making. So I think it's a mix in terms of appropriate prudence and also just legacy conservatism. What are you seeing in terms of indemnification from the big model companies? Does that move the needle for people? And what about the role of insurance on some of these things? I've previously done an episode with

9. 18:55

   a company called AIUC, the AI underwriting company. They're developing standards and trying to seed the insurance market to hopefully grease the wheels on this. Are you seeing? I've talked to a couple of entrepreneurs doing work in this domain, you know, basically AI agent insurance type offerings. We haven't actually seen that uptake of that with our customers, with the workloads they're managing on our platform. So I think that is a very interesting idea. I do see that probably could be an unlock, but I think for a lot of these organizations, it's also another unproven thing and also needs some maturation for the kind of regulated industries that often operate on our platform to like jump in on that too.