Agent memory turns into a retrieval problem

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 32:22

   like for you to be a user yeah i guess i mean gbrain was like a distillation of a bunch of stuff i wrote like i first worked on gary's list which was like agentic it was like an agentic newsroom and i had to build up like basically a research system for rag and for like policy and papers and it would like read every newspaper article about a given topic and then try to synthesize um and then i took that and then distilled it into uh open source retrieval for open claw because the idea was like okay i had it in rails and that was kind of like a foxconn factory for agents it's like i treated every llm call is very expensive and incredibly dangerous and then when i saw open claw it was this you know it's kind of like esalen it was just like a very different way for uh you know you to work with ai because then it's like oh the ai smart

2. 33:19

   it's not trying to hurt you and then you should just let it like do stuff and you should just give it data and capabilities and so I recognized a million tokens is actually an incredible working memory like most human beings only have like seven plus or minus two things that they can keep in their head whereas like a million tokens is like basically like three Harry Potter books but then even that is like a little bit limiting for the agents like keeping the correct three Harry Potter books in your context in a given time is a real challenge. So it felt like there was this thing in between for a company brain or like my personal stuff. It's like I have half a million markdown files on like every communication I've ever had. And so this thing really does have context on me and everything that I care about. But then at any given moment,

3. 34:11

   like the agent, they might need like two pages or something. It's a retrieval problem. And so So at that point, you know, at many, I think it's like 30, 40 gigabytes, you need PG vector, you need Postgres, you know, you can do graph databases very easily in SQL. And so that's what Gbrain is. It's like, you know, rag retrieval plus like a graph database that creates like structure out of unstructured data.

4. 34:38

   And so it's like dynamic schemas, like a little bit like what I remember Palantir was back in the day. Yeah. And then the entire idea is that agents are much more powerful when there's sort of this intermediate thing where the agent can just say, like, this is what I'm looking for, and it'll traverse the graph. It'll do all of the, you know, vector search, and it'll come back and say, like, here's a report and, like, load this into context.

5. 35:05

   And so that's what Gbrain is. And then Superbase was just like very straightforward. It's like, I need hosted Postgres and I need it to be able to support like a 50 or 100 gig or 300 gigabyte, you know, database corpus really fast. And it's been great. Like, you know, the auto scaling works really, really well. And now the funniest thing is like with browser use, like I use a side browser now for browser use.

6. 35:30

   And it's like, oh, yeah, like OpenClock can go into Superbase and, you know, use the database or the new APIs you made. and it just like managed it itself it can like self-manage were you having to prompt it to like figure out the data structures like graphs or like rag or anything like that or uh i mean what's funny is like i i remember going into cloud code and just saying like use exa and download every download and diarize uh every paper you can find in ai research which...

7. 36:00

   Today, I could use like alpha signal or something, but it's like, go and find every piece of research about RAG and graph databases and like, you know, text retrieval, and then summarize it for like my use case, which is like, how do I help an agent get access to this stuff? And then after it did that, it's like, oh, okay, well, here's how we're going to do it. yeah i was at a dinner the other night on like the future of databases and one idea was like databases essentially are like a retrieval mechanism for data and you know you can put indexes so that they'll search across the data in certain formats so a graph or or rag and then one of them which was super interesting was just like putting an agent on a loop to create the perfect data structure for the search like if you're searching this data set this many times and

8. 36:47

   it's taking 10 seconds then why don't you just put an agent on a loop to create a new type of index or like find the best index for returning that in less than a second i mean dynamic retrieval is pretty clearly going to happen the second you have really really smart code gen yeah so i mean for now i mean i use it for open claw so then uh i use my production instance and then the traces from that, I extracted out like a eval data set. And then I have a separate repo called Gbrain evals. And it's got 40 different categories of like specific types of retrieval, retrieval efficiency, like cost efficiency, latency. And then the cool thing about evals is then you can just tell the agent like loop, you know, just auto research and loop and anything that improves your stats on this eval like ship it into master on by default and anything

9. 37:43

   that you tried that didn't work write a markdown file that you know explains why you think it didn't work and then file it away in G brain evals I use Cappy there's a you know Cappy has this concept of captains which is kind of like similar to grok bots bots and so you can just give it a context and so I have one that is always going that's like come up with new categories and I have another captain that's always going that's like go through all the categories in evals and auto research. And every single day, try to find a way to improve our scores on our own internal evals. And that's just going all the time. It's a very, very primitive version of RSI for a particular use case. But I guess if you go up one level of abstraction, you could do that not for my use case, which is for OpenClaw or something. It

10. 38:32

    could just be for generically like anyone's retrieval use case like the only thing that changes is like the use case and uh the traces and the data set you obviously know about evals