Measured chemical landscapes let agents run thousands of campaigns

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:17:09

   because one thing that we realized is that we had this growing collection of activity cliffs and profiles of the same chemical material across large numbers of proteins. And we wanted to know if we could actually use reasoning agents to go explore it and use it in a lab-in-a-loop kind of situation. And so that demanded that we started working with reasoning models and language models in order to kind of let them go play.

2. 1:17:34

   And so what that looks like is we took some open source language models, like we did some QUINs and GLM and the stuff that other people are using. And we taught them medicinal chemistry in ways where you say, here is a structure, describe it in natural English. Give me the diff between two molecules in natural English. And then we got it to the point of where we say, okay, here are some molecules that are binders.

3. 1:17:55

   Here are some that are not binders. Can you tell us why you think the binders are binding? And then you can also give it hard negatives. So you have these two collections that they go, okay, well, I think this chemical series does this. and it will write these things down in readable English, which we think is great. And then we said, okay, now that you sort of know these basic concepts, we're going to give you 10 binders and 10 non-binders that are close, ask you to decide why you think the binders are binding.

4. 1:18:19

   And now we're going to give you the opportunity to run your own experiments. Here's a catalog of millions of molecules. Why don't you pick five or 10 and test your hypothesis about why they're binding? And so then the model will go into the data that we have already collected. It will pull out what it thinks are the most informative choices and we'll get back the answer instantly like it's a coding sandbox and then it will update its hypothesis about the binding.

5. 1:18:41

   And then the ultimate goal is to get the best binder in as few steps as possible, very much like a medicinal chemist would do. And then we just look at it. So you're doing this based on your a priori run experiments, but it's looking up in a database. That's right. So now, so you're basically created reinforcement rollout. It is a reinforcement rollout of a large scale that's growing all the time that's full of these activity cliffs and full of different protein variations.

6. 1:19:02

   And so now we have, and so you can use this, and since we know that when you change the molecules in these small activity cliff ways, it is a causal change. Like, we know that when you put this thing on there, the behavior changes in this reproducible, measurable way. And so you can also use these trajectories to come up with reasoning traces kind of at a pretty good scale. And so right now, we have these agents running through the dataset.

7. 1:19:23

   We just give them 10 binders and 10 non-minders. You give them a budget of 100 molecules, and they'll go through, you know, probably like 102 holes where they're just going through this long, many-step reasoning problem and then come up with what they think are the answers to the best at the end. So why do you think this is superior to something like Hermes or maybe like something, you know, sort of scaling up the Hermes approach where you're, you know, sort of building some, you know, attention thing?

8. 1:19:51

   Yeah, we think so. We think it's much more resistant to the memorization problem. And so like while we are very happy with where the attention goes with Hermes when you throw in all these mutants, as you add more parameters to a model, they're going to memorize the mutants too. And we want this thing to really understand these causal changes. And it seems like using natural language is a pretty good regularizer for the problem.

9. 1:20:12

   And we are unaware of other problems where you have a reinforcement learning loop that is physically grounded, that is this scalable, this fast. You can imagine for when you're doing lab-in-the-loop stuff, where you're going off and making robots and going after these problems, they will also necessarily require like limited resources. They will still make some choices about what to make in the first place, what not to make in the first place.

10. 1:20:35

    And then doing those loops will take a little bit longer. And if there's a scalable way to measure that landscape and then continually make that bigger and bigger and bigger, it's allowed the agents to play in it. We think it's probably going to go a lot faster for this particular problem. That's interesting, yeah. I mean, I would say it's Excel from Xera is getting there, but they're not actually language there, but they are encoding as a sequence.

11. 1:20:59

    So in some ways, the perturbation is similar, but this is different. And then Lila also rhymes with this. It does. All these guys are really smart and pushing these automation things. Yeah, but they're not doing these pooled screens and then using that as they haven't, as far as they know, haven't figured out how to make that into just a database lookup So you can do huge number of rollouts. Yes. Yes.

12. 1:21:28

    So right now, the way that we like to think about it is if you're a medicinal chemist and you're working at a pharmaceutical company, you do this thing called a structure activity relationship. And what that means is, oh, when I change the molecule in this way, it changes the behavior. And so I'm trying to optimize for these different properties. And you go through what's called an SAR campaign. That's early drug development.

13. 1:21:46

    It's making the molecule better. You might see a couple dozen of these, maybe 50 of these in your career. And so right now, you know, last weekend, we pulled out over 10,000 of them and we're not even trying that hard. Right. So, yeah, and so now is the opportunity. Now we'll be restricted to the space of things that we have screened. And then at the end, you can ask the models to make additional choices from, say, catalogs.

14. 1:22:08

    And then you can go, you know, in the future, go order those and retest them like you might for like a traditional lab-in-the-loop approach. But right now, we give these agents 10 binders, give them a budget of 100, and they'll get the best binders out of six and a half million in that budget. And so it seems like it's working. Yeah, so tell us more about why you think this is... I get the argument about it's a better regularizer, but also why do you believe that this is effective?

15. 1:22:36

    So you can imagine, well, it doesn't get things wrong so badly, but it also doesn't get things that right. I think it's because you give it the opportunity to make all those choices. I mean, it's like running these coding sandboxes. You just have like really rapid reinforcement reward signal. But the thing is, you also have to include the choices that a human being would not have made because we're measuring all the molecules across all the proteins.

16. 1:23:01

    Yeah. So it has to be able to make the counterintuitive decision and get a feedback on that. It's got to have the counterfactuals. It's got to make mistakes. And you need to make sure that the mistakes are, you know, present in the data or future potential mistakes are present in the data in ways that are not really biased for it to, you know, learn chemist sociology fingerprints. Yeah, that's really fascinating.

17. 1:23:19

    So through all of the language pre-training corpus, it kind of has an intuition for some basic chemistry that I think most medicinal chemists might have an idea of. And now you're giving it sort of like an infinitely fast lab in the loop. That's correct. So maybe reasoning too. I would say that the reasoning skills might even be more important. The ability to say, oh, in my reinforcement campaign, learning campaign, I've learned things, and now that allows me to reason about the next harder question.

18. 1:23:48

    And then just keep making choices. And right now, we're really excited about the early signal. And also, we're a tiny shop, right? There's only 10 of us. I'm really looking forward to when our experimental corpus grows at 2x, 10x, 100x the rate that it currently does. because then like the amount of space for these agents to explore like expands dramatically.