Reliable workflows are a feature when automation must scale

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:22:19

   wide loop very early on i mentioned i wanted to come back to a particular point and it's fundamentally around like you know when you think about architecting around decision models or whatever we want to call the the broader category uh that jeff sits in um that would require like more traditional kind of workflow orientation versus like where we've kind of arrived at with agents and agentic software is like, okay, well, let's generate some context and give it to an LLM and let it spit back some answers and call some tools, which are essentially the next steps.

2. 1:23:02

   And like, I guess I want to poke at like, you know, in some ways it feels like, you know, if you set aside the fact that this world that we move to doesn't really work all that well reliably, right? Which is kind of important. It's kind of, that's a kind of... It's kind of, yeah, it's kind of important, yes. Yeah. But like, let's set that aside. Like, there's a certain agility to being able to hand off all your decisions to this one thing and let it run in a loop and let it call, you know, next actions and like return to this loop.

3. 1:23:35

   Whereas like decomposing everything into these much more like static or rigid workflows like creates a lot of debt over time creates a lot of complexity and like you know does Jeff require us to move back there or how do you think about that like I've kind of set this up okay yeah you set it up very unfairly because you put aside the I recognize I recognize but I'm open to that and actually I think that um this is a fun hypothetical question because i never not think about reliability but i actually think that there's it's actually a feature to many actual uses like obviously in there's a long tail and there's going to be a lot of uses that is not worth automating this is the tragic the tragic problem of like just doing work as a software engineer do i write a bash script for this that will take me four hours or do i just do it in five minutes and you know i'm the kind of guy who would

4. 1:24:35

   probably write the bash script more often than not. But there's a dilemma here, right? So I would guess for a lot of work, it's probably worth it to just give it to the unreliable LLM and hope that it works right. And then you can just check its work if it's not. But if you're actually trying to scale something, like if it's a workflow that is so valuable that you'd want tons of people to do this, or you'd want to run it a lot in the background, or, and this is the part that I think no one really gets in LLM land. If you want it to be like closed away in the box and run in the background and have it be a dependency that you just call forever that's the kind of thing where it makes sense to really engineer that system up front and you it actually makes sense to maintain it over time because that provides value forever versus like having to rediscover the logic every

5. 1:25:19

   single time so um i think that like people just don't think of llms as composable and layerable and something that can be the foundation of greater and greater cognitive abstractions because LMs are so unbelievably bad at this. And it is a feature to be on the rails. You know, like you don't want to go an amusement park that doesn't have its rails. Or at least you can't do the most fun stuff without like the rails and the seatbelts and all of that, right?

6. 1:25:47

   So I think it's going to be a very similar thing for the most valuable parts of AI software. You're saying that the automating with this, you know, unreliable-ish thing is kind of a it's an oasis and you're not going to fully get there and the fact that we force you to that jev like would force you to like more traditional code and like workflows and all that isn't necessarily it's a feature i think is the positioning that you're it's a feature for automation it is not a feature for just-in-time stuff um so like if you're doing stuff on the fly that's a one-off like of course it's not worth it to write a big old automation for that and there's lots of in between like actually if you've seen cloud code's dynamic workflows dynamic workflows are super cool and nothing stops you from actually having an llm right

7. 1:26:41

   you know understand your task write a workflow for map reducing across your documents in your company brain or whatever is hot these days but actually it could call jev models in it because there's a lot of documents and you don't actually want opus 5.5 to reason over the whole thing so there's a lot of middle grounds over there um and the world is going to be our oyster for