Distributed agents can improve models if work is verifiable

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 33:08

   We need more collaboration surface around it, essentially, for people to contribute to research overall. Can you talk about that? Yeah. So we talked about our research has a single thread of like, I'm going to try stuff in loop. But fundamentally, the parallelization of this is like the interesting component. And I guess I was trying to like play around with a few ideas, but I don't have anything that like clicks as simply as like, I don't have something that I'm like super happy with just yet, but it's something I'm like working on inside when I'm not working on my claw.

2. 33:35

   So I think like one issue is if you have a bunch of nodes of parallelization available to you, then it's very easy to just have multiple auto researchers talking through a common system or something like that. What I was more interested in is how you can have an untrusted pool of workers out there on the internet. So, for example, in auto research, you're just trying to find the piece of code that trains a model to a very low validation loss.

3. 34:00

   If anyone gives you a candidate commit, it's very easy to verify that that commit is correct, is good. Someone could claim from the internet that this piece of code will optimize much better and give you a much better performance. You could just check. It's very easy. But probably a lot of work goes into that checking. But fundamentally, they could lie and etc. So you're basically dealing with a similar kind of problem.

4. 34:20

   it's almost actually like looks a little bit like my designs that incorporate an untrusted pool of workers actually look a little bit more like a blockchain a little bit because instead of blocks you have commits and these commits can build on each other and they contain like changes to the code as you're improving it and the proof of work is basically doing tons of experimentation to find the commits that work and that's hard and then the reward is just being on the leaderboard right now there's no monetary reward whatsoever but I don't want to push the analogy too far but it fundamentally has this issue where you might have search goes into it but it's very cheap to verify that a candidate solution is indeed good because you can just train a single you know someone had to try 10 000 ideas

5. 35:00

   but you just have to check that the thing that they produced actually works because the 99,000 of them didn't work, you know? And so basically, long story short, it's like you have to come up with a system where an untrusted pool of workers can collaborate with a trusted pool of workers that do the verification. And the whole thing is kind of like asynchronous and works and so on. And it's like safe from a security perspective because if anyone sends you arbitrary code and you're going to run it, that's very sketchy and dodgy.

6. 35:29

   So, but fundamentally, it should be totally possible. So you're familiar with projects like SETI at home and folding at home. All of these problems have a similar kind of setup. So folding at home, you're folding a protein, and it's very hard to find a configuration that is low energy. But if someone finds a configuration that they evaluate to be low energy, that's perfect. You can just use it. You can easily verify it.

7. 35:47

   So a lot of things have this property that, you know, very expensive to come up with, but very cheap to verify. And so in all those cases, things like folding at home or SETI at home or auto research at home will be good fits. And so long story short, a swarm of agents on the internet could collaborate to improve LLMs and could potentially even like run circles around Frontier Labs. Like who knows, you know?

8. 36:10

   Yeah, like maybe that's even possible. Like Frontier Labs have a huge amount of trusted compute, but the earth is much bigger and has a huge amount of untrusted compute. But if you put systems in check, systems in place that, you know, deal with this, then maybe it is possible that the swarm out there could come up with better solutions. And people kind of like contribute cycles to a thing that they care about.

9. 36:34

   And so, sorry, so the last thought is lots of companies or whatnot, they could maybe have like their own things that they care about. And you, if you have compute capacity, you could contribute to different kind of auto-research tracks. Like maybe you care about certain, you know, like you care about like cancer or something like that of certain type. You don't have to just donate money to an institution.

10. 36:51

    You actually could like purchase compute and then you could join the auto-research forum for that project, you know? So if everything is rebundled into our researchers, then compute becomes the thing that you're contributing to the pool. Yeah, that's very inspiring. And it's also interesting. Like, I don't know how far this goes, but it is interesting that at least some audience of people, you know, here in Silicon Valley or lining up at, you know, retail stores in China