Agrawal would rather overuse tokens than be too cautious

Transcript

From the transcript Grit publishes with the episode.

1. 13:16

   portfolio now, like, obviously token spend is going up. Like, we're spending a lot of money on cloud code, basically everybody. Are you starting to look at that like, uh, as you dig right now, do you actually care or are you more curious, like, is this do you think this is a line item that's going to show up at the board level where you're like, this is getting insane. Yeah, I ask most of my CEOs at this point and we talk about it because I'm trying to understand the curve of token use, like use and cost coming down. And at what point does it sort of stabilize or does it keep going through the roof where if it's right now, maybe like one tenth of a person's salary, does it go to 50% of someone's salary to one times a person's salary? And I don't know where it sort of lands, but I'm trying to make a sense. And that's part of our job is to understand, like,

2. 14:05

   where do these, uh, macro trends land for, you know, for knowledge work for other types of work in the future? I will say my I have two takes on this. Number one, I think most token maxing a lot of token maxing that is happening isn't that valuable, including a lot that I have done over time in the last few months. So at the same time, if you have to. So clearly there is a failure mode to just token max. Not because it's expensive in terms of tokens, but it's expensive in terms of like AI, psychosis or time you spend token maxing, right? It takes some effort and energy to token max. And sometimes it's actually not super productive. So I think it's there are some people who are really, really good at it and actually drive themselves and accelerate themselves forward. And not everyone is. So I think it's really important to not measure by token

3. 15:11

   maxing. The second framework that I have is like, listen, none of us know what the perfect line is at any time on this. Which way would you rather be wrong on? Right? Would you rather be wrong because you're too conservative and just not spending tokens? Because you're too afraid of either spending tokens the wrong way, or spending your time on tokens the wrong way? Or would you rather go on the other side and spend a little bit too much time with the models? Aren't good enough. Spend a little bit too much tokens, and at least my personal decision is I'd rather be wrong on the token maxing side than the non right, and you're never going to be right. But if you know which way you'd rather be wrong, that tells you what you should do. You track it. And Roadrunner. Uh, my co-founder Eugene tracks it. We first it was like,

4. 16:06

   let it rip. That was like, that's been the last six months. Was let it rip and everyone let it rip. And then you're like, okay, this is getting expensive. And so then we started to make sure that it's all in one credit card. So you can track like, all right, where is it actually being spent. And then you started to ask yourself, okay, well what model is it being spent on. Okay. So then we started figuring out like, all right, should we just let everybody use whatever model. And so we've kind of like kept pushing in a little bit. And the short answer is we track it, but we don't enforce anything today. Um, we're also watching to make sure that at some point, if an engineer is spending their salary on tokens, they better be pretty good. You know, they better be pretty good. And so we I would say we,

5. 17:00

   um, we watch it, but our best engineers are doing such unique techniques and how these agents are working that I think limiting the ways that they're using these tokens feels like a mistake to me, because they're on the bleeding edge of pushing the boundaries of like, sequencing five agents together to do all these interesting things. And I would rather spend more now to let them be on the bleeding edge. And look, they happen to also be the best engineers that we have. Do you agree with that? Yeah, I think there are. We definitely see, I think one way people perhaps undersell the use of tokens is like, oh, you're doing the same work that you would do faster. I actually think the what excites me is people like broadening what they could have done, or taking more end to end ownership of things by

6. 17:54

   using agents and models. So the example is like a a back end engineer will now do more front end work, or a front end engineer will change more of your API ship or. Someone will use agents to do a bunch of security reviews. Someone will use agents to push a bunch of optimizations which they would otherwise not have been doing, and design something and do it. So it's actually people expanding their span of influence and ownership on the product. That's where I think there is true business value being generated in my mind. And so people who end up sort of feeling comfortable about that and like really pushing their own sort of learning curves really fast. That's where I think the real value is, at least for me, that I see. Mhm. Yeah. The, the, the real life use case that came up this morning for me was every time