Background agents need gradual, permissioned autonomy

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 29:33

   And it can actually walk you through the process and help you explore new opportunities that you wouldn't even have even thought of. So one thing that seems a little bit under discussed right now is background agents. Could you tell us more how you think about that and what you think is coming in terms of this next wave? Yeah, I always describe background agents to the company as like, this is the ultimate moonshot in enterprise software, just these agents that go off and do work for you and magically discover all your problems and solve all your problems. And it would be amazing if we could get that deployed and it could work.

2. 30:00

   could follow our permissions. And I think no one's really figured out how to do this well, because on the one hand, you want these to be very autonomous, to go off and figure stuff out and then solve problems without somebody babysitting them. On the other hand, autonomy is very scary in the enterprise. And so you do want to see what exactly can they read. This is kind of the OpenAI hugging face incidents or other things where agents are given a task and they do something that's unexpected in order to accomplish that task.

3. 30:25

   And so what are the guardrails you can put in place? Exactly, because my best employees, what do you do with them? You say, here's the general high-level mission. Go off, don't talk to me, and go do it. And you'll come back and you'll have done amazing things. I want to be able to trust agents to do the same thing for me, where I can give them this high-level task and they can come back and solve this problem and come back to me. I don't think we've figured out how to do that in a way that feels safe in the enterprise.

4. 30:46

   What do you think is a way to make it safe? I mean, people have experimented with different things over time, everything from constitutional AI to certain forms of RL. I'm just sort of curious if you have a solution in mind or is it just... We think of it in the large enterprise, it's not perfect, but it's kind of an incremental escalation of capabilities. So you go off and you do these things and it comes back and says, hey, I would like to be able to do all this stuff.

5. 31:08

   This would help me do my job. And you approve them. And all this stuff are deterministic workflows. I want to call this tool. I want to run this script that does X, Y, and Z. can I do that? And you just kind of approve these with certain permissions and guardrails around them. And you say, OK, yeah, that's OK. That's OK. That's OK. And over time, the agent becomes pretty competent because it has a big enough toolkit that it can solve most problems. And it's not perfect because it's annoying to have to babysit that process. But I think it's what you have to do because otherwise it could theoretically just start writing code that could take action all your systems. And that is an unbounded security risk that I don't think anyone's going to be able to take anytime soon. Do you think to your point of being excited or you and Alex being

6. 31:49

   excited about future capability, like we're still missing some model capability here? The reason I ask is, you know, the ability to be proactive and then predict what an individual, Jake as CEO, me as an investor, or me as a human being, you know, and get it right is still very hard, right? You need a lot of preference data. Think about an employee working for you like, okay, What are Jake's actual priorities when he gives me this high-level mission?

7. 32:16

   What does he mean? Do you think that's a model problem or it's entirely like a guardrails and harness problem today? I think it's both, honestly. I think memory obviously has to be better and better because you want to understand the preferences. You want the context. to be insanely large because i don't want to have a bunch of different chats with you i want to have kind of one single chat that goes back all the way to our very first conversation and you remember the key points of that that are relevant um and then i think the permissions problem will always remain the biggest thing because even if i feel like you are a very very valuable smart agent and i you can do all these things for me how much do i trust you to just run wild and not have to babysit your actions you have to do that with people too right i mean you also have anybody

8. 32:59

   It works for you. It has to come back and check in, and this is what you wanted, and here's three options.