Recursive updates compound ranking gains without rebuilding models

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 39:33

   You mentioned or you referred to, is the harness and recursive like the same concept or is the harness a subconcept or? Harness is one component of the entirety of our recursive stack. So you can think of the recursive stack as like being this engine with many, many different components. We have a proprietary knowledge drawer. We have a harness library. We have what we call frontier ranking models, all of which work together to power the recursive stack.

2. 40:08

   And so our core IP is the recursive stack itself that produces these insane ranking models that are able to, you know, 4, 5, 10x Fortune 500 companies revenue in just a matter of weeks. and that's why we are able to allow them to then own the ranking models right because the real meat of the matter is the ability to continuously build ranking models better and better and better and better ranking models that continue to drive revenue and so when you talk about this harness what makes it a harness is it just a library and like harness is the cool word or is there something harness-y about it in the same way that, you know, Codex or Quadcode is a harness?

3. 41:00

   Yeah, so it's actually a collection of harnesses that we have. Definitely not, certainly not a cool word at the time that we started building it in. We just needed a way to specifically have what we call eval harnesses. to find what we call, uh, the commercial ground truth of interactions. And so that's why we started to build these harnesses and we quickly realized in short order that, uh, even, you know, in the, in the, in the space of evaluating these ranking models, uh, how we were able to do so offline in, let's say, the travel space, which is, you know, considered obviously higher ticket items, has a ton of overlap with the furniture space, for example, which also has higher ticket items. But guess what? It doesn't have a lot of overlap with the ad space because the user is just clicking on things and that translates directly to revenue, but it's not coming at a

4. 42:12

   cost to a user so how you evaluate these different kinds of ranking models um you know requires to you to have a very very rigorous approach and so we came up with a collection of um these evaluation harnesses and that's that was the start of our of our harness library you call the the tool or the platform recursive like what's recursive about it recursive ranking intelligence is kind of the first name.

5. 42:41

   Is that, is recursive just a name or is recursion a part of what you're doing? Actually, if I were to be very, very specific about it, it's most similar to like TL recursion, meaning that you're able to, instead of having to rebuild the entire ranking, iterate on the entire ranking algorithm and rebuild it from scratch, which I would consider iterative and not recursive, you don't have to rebuild the entire ranking stack every time um you're learning so you're able to learn continuously and make uh minor modifications here and there and in doing so you're able to compound um gains at a much faster rate at a significantly less at a fraction of the compute so that is if i were to get very technical that is why we call it recursion is that we don't have to rebuild um ranking models um previously when we started the business um we actually did so we would train the ranking models

6. 43:45

   a daily sometimes hourly but it would be like from scratch and it took us quite a bit of time actually to figure out a way how can i have how can i um quote unquote iterate on this without having to rebuild it from scratch every single either day or hour or whatever cadence and and insert that recursive element in there.