Real-world AI takes longer when the environment is unconstrained

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 53:33

   You said, oh, I don't know if it's going to be solved or not. I'm like, yeah, Martina, it's going to be solved. And in my mind, like there's so many other things like for example self-driving cars like for a long time we thought that's that's not going to happen but it's clear that it's actually going to be done um and it's going to happen it's just taking time because of a lot of different things and so what are like what are the other next stages the next areas that you think are kind of like lined up right after code.

2. 54:06

   I mean, my sense is AI is going to continue to take off in a wildfire for a constrained domain, where the solution space is in a constrained domain, right? Where the distribution of the solution space is the same as the distribution of the training set. The distribution of the syntax for code is the same as the distribution of the training set. The distribution of the generated images is the same as the distribution of the training set.

3. 54:28

   In those areas, I think it's going to continue to take off as a wildfire. The domains where, like, the distribution of the solution is the natural world, I think it's going to, you know, have, you know, it's going to be much harder to take longer. And I think the economic equivalency is going to be much more difficult. Self-driving is a great example. I remember when Sebastian Thrun won the DARPA Grand Challenge when I was at Stanford.

4. 54:50

   Remember that? And we were like, hooray! You know, self-driving cars is around the corner. Seriously. And it took 20 years and $170 billion of investment to get to worse unit economics than Uber, right? So that's, you know, so I think that, like, solving, like, solving real problems where you have to deal with the distribution of the real universe is just much, much harder. And so I view AI as this general tool.

5. 55:18

   It's a very effective tool. It's one where like you have, you know, a distribution of data and then hopefully you can apply that to like, you know, some problems and we're going to apply it to a bunch of stuff. And I think economic equilibrium is going to determine what works and what doesn't work because I think it's a very, very general tool. It's like a universal function approximator. And wherever that nets out, I mean, again.

6. 55:38

   You know, one of the things that I said earlier was like how AI was training humans to like use it. And I very often think about, like, you know, some of the problems that seem intractable is because the world is kind of that way. And then we're trying to fit AI or the tools that we're building to manage the world as it is. But in some situations, we actually tend to change the world to fit to what we can actually do.

7. 56:03

   Oh, 100%, of course. I think this is the story of robotics. Yes, exactly. It's literally the story of robotics, yeah. The automated data warehouse. And so the question like that always comes to my mind, like when we were looking at these data warehouses and whatever, like the only one that worked was the one where it was like, you know what? just automate the whole fucking thing. Yeah, yeah, yeah, yeah.

8. 56:22

   And I wonder to like what extent as we kind of improve or change how we, or kind of expand the domain of what we want to solve with AI tools and so on, that we actually start modeling the world or changing the world and the things that can actually happen to fit what AI needs it to look like. Yeah, I mean, listen, I think that right now a lot of AI just operates in the digital world. And for that, like literally the solution set is the same distribution as a training set, in a sense.

9. 56:54

   And so that's fine. And I think that, again, like in the case of equilibrium, like I think like when you're dealing with the real world, like you can solve any problem you want. It's just a matter of like compute and money. And that the equilibrium point is going to be an economic one. And I think one way you can impact that function that you're doing equilibrium over is to constrain the environment, it's to constrain the system.

10. 57:20

    And so I think that we're going to, like we're now in this meta-economic equilibrium flywheel where we're kind of optimizing this super general but very inefficient process with a solution set that people value. And I think, listen, we've got 30 years of this and a lot of value is going to be created. You know, you want to hear a funny story? Yeah. So, you know, we built this AI thing and we're like talking to people, We're, like, reaching out to customers.

11. 57:47

    We're like, hey, we built this AI thing. You know, we're helping customers become more reliable. Here's, like, what we're doing. They're like, great, we want to talk to you. And we go on the call and we're like, hey, here's what we do. You know, we built this system, helps you optimize your data, helps you reduce your costs, and then it helps you become more reliable because now we can find these problems and solve them before they hit your incident.

12. 58:07

    They're like, wait, wait, wait, wait, wait. Roll back. You said you can save 90% of our Datadog bills? Yeah. That's what I want to buy. But the AI thing, they're like, no, no, no, no. That's what we want to buy right now. Yeah, yeah. And it's interesting, like a lot of these, like for code, you know, I always think about it, it's like, oh, for code, people just get it. They're like, oh, shit, it just works.

13. 58:30

    Yeah, yeah, yeah. And on the ops side or like anything that's beyond that. Yeah, it's very tough to understand the actual value and put a dollar value to it and be able to pay out the door, no, 100%. And I also think from a coding standpoint, literally it's engineering productivity, which is one of the highest line items on these things, so it's just very straightforward. Exactly, but you can also think about it, although it's SRE productivity, or it's like your business going down, maybe you can save an incident and what the cost is.

14. 58:55

    I don't know. It's weird. I think like how people think about it outside of the...