How AI Breakout Harvey is Transforming Legal Services, with CEO Winston Weinberg

Right. And so I think that the largest misconception with kind of the GPT. wrapper companies was a lot of what you were starting to do didn't have like a massive delta between what the foundation models provided and what your small company could do, right? So one of the largest advantages that we had in the beginning was how much emphasis our product had on citations. Like literally just, it was very important in the legal industry to be able to cite like line by line and the accuracy of those citations, right? There were a bunch of other things as well but like from day one that was something we put resources into if that's all our company was if we just said okay we're going to be a citation company right then yeah i think you're just a gpt rapper and it's going to go away but if the ambition of the company is to actually partner with an industry to completely transform it this is a trillion dollar industry it's incredibly messy there are tons of different data sets there's tons of ways that people work There's tons of different specialized workflows, et cetera.

And so I think over time, it became clearer and clearer that these models weren't going to just automate entire industries away, right? They were going to change and, like, fundamentally serve as kind of like the groundwork for changing an entire industry. But, I mean, one way to even look at this is when a new feature comes out, we usually think of that feature as this is a great piece that we can put into part of a workflow. So an example of this is like OpenAI will release deep research, right? And my first reaction to that is, oh, my God, there's all of these capital markets use cases, right, for like studying markets, looking at all of this data that's on the Internet, et cetera.

It's not everything a capital markets attorney does, but you could put that into a piece of a 100-step process in your product, and it makes that process better. And so the best way to think of this is literally whenever I see either the foundation models can do something better than they could before or a completely new feature, it just unlocks part of the TAM or part of the actual process that you can build on top. Let's pull this thread a little bit. So at time zero, you've got the model down here. You've got the customers up here. And you've got Harvey doing citations in the middle. For a month or so, yeah.

What came next? Kind of walk us through some of the major leaps in the product or some of the major value creation opportunities you've seized upon along the way.

building these specific solutions and then you collapse them back in. How much of the magic do you think is in the models themselves versus what I think you just described as an agentic system or cognitive architecture yeah um and you know how much is it is it the workflow that's hard is it training things there that's hard is it the combination of all that's hard so i think like

So what is your view of the ideal end state for how a lawyer should be interacting with Harvey?

Is it kind of that email chat interface and it's just merge company A and B and we're good? I don't think that we will get there anytime soon. And that is not for me to say that I'm not bullish on the foundation models getting better. I mean, you know this. I am incredibly bullish on the foundation models getting better. And we have designed the company kind of without a constant push as a driving force. I think the biggest problem with that is that doesn't allow the user to have enough kind of exercise enough judgment on the workflow itself. Right. So I think one thing that when people are talking about agents, they're talking about these tasks that are like quite simple and don't have massively high economic value.

When we're talking about building workflows and putting kind of agents in those, we're talking tasks that cost hundreds of thousands of dollars, right? I mean, one reason the legal industry is so good for LLMs is if you kind of think of this as is the industry text-based and then how valuable is a token, right? And a token is incredibly valuable in legal and professional services. If you look at like a merger agreement, like a 50-page merger agreement, the token in there, like each piece of a word, is worth so much money if you think about how much it costs to produce. So this is all to say I think the end state is you keep building these agents and workflows, and then you chain them together as much as you can, right?

And that makes it so that your UI actually looks the same, but your suggestion and your routing model gets better and better. Your orchestration level model gets better. So you can kind of think of this maybe like, let's go to the law firm. I think that we're building out the specialized associates that can do different tasks. But it's also incredibly important that you have the partner or the managing partner operating model as well. And I think that as the models allow us to build more and more of these specialized kind of specific associates, you also have to put all of this effort into actually having the orchestration layer that pulls all that together.

Right. And so I think our UI. actually looks similar in the sense that it does look like a kind of like a text window, right? But the ability for a user to upload a bunch of documents and effort just to suggest what you do or the ability to say, this is what you did last time. Would you like to do it again? Things like that improve. Kind of like a colleague that's just like really good at knowing what you want to do. On that note, are you selling software or are you selling work?

is the tech ready like how much i guess legal work can already be automated with today's models and if you were to freeze mobile development yeah you know how much can be automated yeah i mean i think that's a good question in terms of and especially the second one of i guess like how there's the reasoning of the models and then the capability of the models, right? And if you froze the model's reasoning ability, I think we'd be fine, actually. So if we somehow kind of like froze the ability for the models to process data, that doesn't mean that they know which process to actually analyze that data, but just their ability to ration over it, to make rational decisions over it, we'd be in a really good spot. So in other words, I think we're at like small percentage points right now of like legal and professional services. But if you paused, we would increase pretty high, even if the models didn't get better, right? Because I think the reasoning ability of these models is there, actually. I think the bigger problem is like evaluation, improving process,

and collecting more data, too. The data isn't there. is model development going to freeze here um i i don't think so no um i mean i i think we're we're seeing a pretty large evidence of if you throw more compute at the model and you let it make more and more reasoning steps it just gets better and better yeah exactly right um and it's interesting if we go all the way back to 2022 in the early 2022 the thing that Gabe and I had done was we were doing a bunch we got access to GP3 this is public by the time this is like early 2022 and we the thing that we found was we were just doing chain of thought prompts before anyone was thinking about chain of thought prompts and the way that we actually started was we were doing that over a bunch of kind of like legal questions and we cold emailed the general counsel of open ai and sent him that and he basically his name is jason kwan um and he basically responded oh my god i had no idea these models were this good at legal and i think the main reason people weren't looking at it is because they were just doing one model call over a set with like one static prompt and just calling it a day and they weren't doing what are the actual process steps

Like they weren't telling the model, these are the steps that you need to take in order to answer this question. Right. And so my point is, that's what we saw in like early 2022. And that's like the direction that these models are going. Right. And I think that it's really good for legal and for kind of these these industries where you have to have these complex decision making processes for two reasons. One, the model get better at that. That just increases the ability of them. But two is that process data doesn't exist, like I said before. It's not online. How to book a flight is online, right?

There are easy ways to train that. It's harder to do. How do you do an LBL? And you guys, you now, deservedly so, have an unfair advantage around the process data because you have some of the very best law firms in the world as customers of Harvey, which I think was a contrarian strategic decision that you and Gabe made. couple years ago, and you made it with high conviction. And I remember a time when there were lots of law firms who wanted to come work with Harvey, and you basically said no. So you could focus on some of these big, prestigious firms. Can you say a couple words on what gave you the conviction that that was the right strategy? And then maybe more importantly, once you determined that that was the strategy, how in the world did you get them to trust you?

Yeah.

well, not just exactly what, you know, is special about your product too. How have you elicited the behavior change out of your customers and kind of taught them to use AI? Yeah, this has been really hard. I think that it starts with product by far. So when I was talking earlier about the expand and collapse, the most important piece of the collapse is to make it so you don't have the blank page problem. That is by far the most important thing. So that when you go onto the landing page of Harvey, there are all of these buttons that you can click that will help you get started. And that is super important. And now we actually have it to a point where you put in how many years you've been practicing, what kind of lawyer you are, things like that. And it will change what that screen looks like when you start.

So that's been super helpful because if you make it so that those very specific systems are exactly what they do day to day, it is really easy to get them to start. And if they start using it, they'll be creative. They won't be creative right off the start. It's hard to get people to do that, right? So on the product side, I think that's the most important piece is just making it very personalized and making it so the time to value is really short, like as fast as possible. And then they'll go out and explore. On the CS side, we've hired a lot of lawyers. And I think that's been super helpful, is you need to hire domain experts who can say, and we're doing the same too in tax and these other areas, you need domain experts who can come in and say, this is how I would use it. And this is, you know, I've been doing this for six years. I was in the same shoes you were in. I know exactly what work you do. Talk about the trust thing, hallucinations.

And what do you think are the developments that are most interesting for you?

Yeah. I mean, costs going down are always good. I think like one thing to think about here is there are so many use cases that we have gotten to work 70% of the time, right? And the O-series models have been incredibly helpful for us because not even just the O-series models themselves, but it unlocked kind of like our product strategy for the next six months to a year, right? Where there were so many systems that I don't think we thought that we would be able to build in the next year if it was just the GPT-series models, right? And I think that the O-series has massively kind of changed that for us.

And so our product roadmap has drastically changed because of that. What's an example of something that you couldn't do before that you can do? So examples of this are like things that you need to do multi-step reasoning for and pulling from many sources at once, right? So the thing that the O-series models is really good at is orchestrating a plan and also executing on that plan. So if you build a bunch of systems that are really good at basically extracting information from Edgar, then extracting information from case law, then extracting information from all your internal documents, the missing piece for us was what do you do once you extract all that information?

Once you have all of those different pieces, how do you combine them into the correct work product? That's what the O-series models allow us to do. If Sonia had a magic wand and she were to wave her magic wand and you and Sam Altman all of a sudden switch places and you're now CEO of OpenAI and he's now CEO of Harvey, what would you do different at OpenAI and what might he do different at Harvey? Yeah. Well, at Harvey, he'd raise more money. He is pretty good at that. He's really good at that. Maybe I'll start with what I think OpenAI has done an incredible job of. and I would maybe even double down on it more.

I think that they have done an incredible job of capturing the consumer zeitgeist. So I am not from Silicon Valley and I don't have, I mean, now I have tons of colleagues here and tons of friends, but I didn't beforehand, right? And almost none of my friends know what any AI tool is other than ChatGPT, that's it, right? And I- The thing that I think I would do differently, and to be honest, I think they are going this route anyway, is just put way more effort into productionizing that for consumers as much as possible, right? And that's not just model performance at all, right? So I think that's still one of the biggest problems with ChatGPT and just kind of all of these tools in general is they're looking so much on the performance from the model side and not at how do you make the experience easier for the user, right? How do you make it so that it extracts more information from the user? How do you help the user figure out what it can do and what it can't do? How do you help the user figure out how it combines with different pieces of information, et cetera?

So I think on this side, it would just be putting more and more effort into understanding the consumer behaviors and how they use AI right now and making that easier for them. What you guys have been partnering with OpenAI from the very beginning,

What are the biggest threats to Harvey?

I think not moving fast enough. I mean, I tell this to my team a lot, and I think it's becoming very obvious in the past couple of months, too, that we're really living in a time when all of your timelines are compressed. I would argue this is, in all of human history, the most compressed timeline. in terms of what you can change in the world right um and i think that you have to move so incredibly fast in order to keep up with that and the speed is compounding in a couple different ways if you are constantly moving fast you are required to constantly test everything around you pay attention to every single every single part of your industry how your every single customers using your product how your sub processors are changing their models everything right and if you do not do that fast enough you'll make a bunch of mistakes i think and you constantly will make mistakes right by moving very quickly but the scariest mistake is you move too slowly and you miss a massive thing right so there is part of a feature that someone releases something and you have steps 12 through 13 complete on that feature but that 13th step you just cannot get the models to do it and somebody releases something that unlocks that you need to put that

in the product immediately right because you need to start testing it you need to see actually did it solve it etc and if you aren't moving quickly and you just say ah that's something new we'll try it out eventually i think it's a huge problem speaking of moving fast good segue into our lightning round since starting this company two and a half maybe three years ago how many days have you taken off it's a loaded question uh i have not taken a day off i probably should though to be to be in in all honesty i think like i've definitely a little bit worn the you know in like full transparency i've worn the badge of honor of like you know don't take any time off obsession etc and i think on on one side i actually very strongly believe in that so going back to my point about the timelines being incredibly compressed you need to be obsessed right like you massively need to be obsessed um but i do also think the other side of that is like you need to transition how you are a leader like that needs to change um and you know our company last year we started the year with around 40 people we have 260 right now and you just Thank you.

need to change how you are spending your time. Right. And I think that I've definitely learned how to spend my time differently. And there's also stuff I've held on to and I actually like deeply believe in. So one of the things I deeply believe in is I actually think you should do part for a little bit of every single job at your startup for a little bit. Like I actually I did I do it too much and I did it for too long. But it is all most of my hiring mistakes have been I didn't understand what the role did, like at all. And a bunch of people told me what it was, but it doesn't help you hire if you don't understand what the actual role is. And so that one I feel really strongly about, but at the same time, I also took way too long to hire. And I was probably doing too many low-level things for too long, right? And so it's a combination of both of those.

For those 260 people who work at Harvey today, what is the best thing about working at Harvey and what is the worst thing about working at Harvey? Yeah. I think the best thing is every day something new is happening, right? From the product side, from the GTM side, you'll see, you know, a bunch of changes with the model providers that you want to integrate very quickly. It is stimulating above all else, like for sure. um and i think we're seeing like a lot of impact too and it changes too and i think that's if you ask folks that have been here for a year or a year and a half and there aren't tons that have been for a year and a half um the thing that i think is most fun for them is how much our market has changed right where i mean we used to it was brutal in the beginning for kind of how you do your sales process the requirements people had and things like that and now most of our customers like they're partnering with us right and they're letting us do their onboarding they're doing all of these things where it really has seemed like we're kind of this wave of all of us together doing it together yeah doing it together and that that changed a lot there was a lot of pushback in the beginning like a massive amount of push and there's still some but i think

that has changed pretty drastically um the worst thing is the expectations are really high um and And, you know, we had a really good year last year. And I think we had our offsite recently. And I went up to the offsite and I basically said, hey, we did a really, you know, we had a great year last year. This year needs to be like significantly better. We need to raise the bar. And, you know, I think that's not always what people want to hear. Sometimes people are like, wow, we did such a great job last year. Let's take it easy. Yeah, let's take it easy. Let's just do the or do the same as we did last year and it'll be fine, right?

And the reality is, again, going back to those timelines being so compressed, you can't do that. And I think that the main thing I ask people is basically, look, like, I don't know what your goal is. I don't know if your goal is to make a large industry change, to learn as much as you can. make money, whatever it is, your options to do that in the next decade is the best it will ever be. Like, it just is, right? You will make more impact than you ever will have the chance to in your life. How have you changed as a CEO? And what prior have you updated the most? yeah um uh hopefully i've changed somewhat um i i think the so that the prior that i've updated the most is teach not do like i'm i'm bad at that um because i i kind of want things done so quickly i have a massive problem of whenever i start seeing friction i just go okay I'm going to go do it. Right. And that's actually really, really bad. I think it it makes it so other folks can't learn. It makes it so that it is kind of like too top down. Right. And it's something I'm working on a lot. And I think I've gotten better at it. Hopefully you can ask my direct reports. I'm not sure if I have, but I think I have. And I think that that's the area

that I want to keep getting much better at is taking kind of, you know, slowing down in some instances and actually like setting ourself up to scale um instead of everything being like go fix that fix that fix that fix that um yeah who's a better athlete you or your co-founder gabe oh gabe is a much better athlete than i am i mean it's it's really unfair so he he played professional soccer um and he is just a much much better athlete but he had a knee injury recently and I will say I do a little bit like we live together and every morning like I'll get up to go to go to the gym and I definitely slam the door like a little bit loudly just so he knows that I'm going to the gym and he can't quite yet I got it like we we met each other uh before this and we and we were we like became best friends and had no plans of doing a startup um and he was just like always a better athlete than me and so this is my like revenge a little bit it's not gonna last long he's healing right now and so he's probably right now working out and getting better all right last question i'm gonna steal the last question from guy ross i don't know if you ever listened to his uh show how i built this but he has the same last question every time which is how much of your

success has been luck and how much of your success has been skill um it depends i need to find luck we have been in a place where we have the options to apply skill and we have the leverage to apply skill and if you apply it correctly enough times to actually have a large impact from that and so the luck is the timing the luck is do you actually have the option to make a difference to make an impact and actually i think you were kind of asking earlier about what have i learned as a as a CEO, one of the things that I think I've actually tripled down on, quadrupled down on is young talent, like by far. And that goes back to giving them the luck.

or the opportunity to actually try something they've never done before, it works out really well. And they don't get it right every time, but I don't either, right? But you adjust really fast and their ability to adjust, I think is better than a lot of folks that have been doing this for a long time. And so maybe the best way to phrase that is their skill is their ability to adjust to luck and seize luck opportunities more than anything else. I like it. Winston, Thank you so much. Yeah, thank you. You