Memory retrieval can improve from logged questions

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 40:51

   What do you think I might be leaving on the table, especially as you think about multiplayer that demands an even more sophisticated approach? yeah so i believe the approach you're talking about and we we still have some rag in our database and we use that for our rag as well it's called like hypothesis driven retrieval i don't know if you've if you've heard that it's a sort of reverse hypothesis driven retrieval so there's both there's both so what hypothesis driven retrieval does is it generates like suppose it's asking like where was justin bbill born right and like and then you you basically instead of searching for that question using rag you search for your hypothesis like justin bbill was born in Paris, Justin Bieber was born in New York, in Berlin, and you search for this bucket of hypotheses in your RAG database, because obviously, semantically, each of these

2. 41:35

   hypotheses is a lot closer to the answer you might be looking for than to the question where was Justin Bieber born. So that's the first thing. Then what you do is you also do the opposite, which is when you find an answer in your RAG base, you generate a bunch of questions that map to that answer and you attach them to the answer. So now the retriever can both look for hypotheses and can still look for its answer.

3. 41:56

   for the question. And that's going to tend to increase retrieval quality. That's totally fine. I think one thing you may be leaving on the table is it is really healthy for all of the memory to be managed by one agent. Even though that agent, and both retrieval and updating of the memory. And so even though the memory is in a file system and so your agent could actually just go mess with the file system, what we have found actually, and can still do that, But we tell the agent, we prompt the agent to be like, if you're looking for a memory that you don't have, please ask the memory agent.

4. 42:31

   And the reason why is because when we ask the memory agents, then what we do is that we log the query, we log the answer, and we log how many hops it took to retrieve that. And then when the memory agent is napping and dreaming, it looks at this log of like, hey, this is the kind of stuff people have been, it's almost like a librarian. You know, it's like, ah, like I've been hit up left and right.

5. 42:56

   This is a question that I get quite often. So right here, by the way, it acts as a sort of cache, you know, because automatically, so, you know, you always keep, like, the memory of the memory agent, like, the last, like, thousand, like, question and answer pairs, you know? So it's like a rolling window, and so it is a sort of cache. But also the memory agent is not going to just rely on that cache because every cache at some point has got a TTL, it grows a stale, and all of that stuff.

6. 43:19

   So what the memory agent will also do is, like, it will restructure its own memory to reduce the number of hops that is needed to retrieve the most frequently asked questions. And that tends to work quite well to improve the memory retrieval and the memory speed. Cool. Interesting. Another system we experimented