A science AI needs data, simulations and robotic experiments

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 10:16

   When you think about kind of where we are with regards to AI for science, like clearly it's been something that we've been talking about for a long time and the promises is there. Are we further ahead than where you might have expected us to be? you know if you you know five years ago looking forward or are we not quite as far ahead as you would think um you know that one it comes to my what one uh example that uh is drawn to mind is like ai is going to replace all of the radiologists right like we haven't necessarily seen that but you know we certainly see the you know the promise is still there the you know their acceleration is still there of that you know some of the aspects of that job part of the question is like is science harder than you know is it easier to say science than it is to do science and these things sound

2. 11:15

   good but they take much longer like yeah lots to unpack there so I think like I think CEOs get away with making incorrect predictions as long as they actually make a lot of progress and their companies make more revenue on the way to, you know, maybe different timelines than they predicted. And so I think, you know, like people said, GPT-2, one CEO said, GPT-2 is too dangerous to release to the world, right?

3. 11:47

   And now you can train a GPT-2 model within five minutes on your laptop. And so the world has not gone down. And I think there are lots of like predictions by now. Around the full extinction of all of humanity that are completely overblown and unrealistic that people are coming up with. And a few people are held accountable. There's no really big downsides to making wild predictions in one's careers often.

4. 12:15

   And so the sort of more subtle takes and nuanced takes get less press than the sort of crazy, crazy wild ones. But I think indeed, of course, whenever you stay in software or math or coding, iteration speed is faster. Once you go into biology, chemistry, physics, and so on, and just hardware, that is just by definition slower molecules. Sorry, molecules and atoms are harder to move around and iterate on than software is.

5. 12:49

   And so at the same time, we are seeing real, like, like really large data sets now that you need to get to that kind of progress. Like another one is called Tahoe Therapeutics, like open source, like massive data sets for these perturbation studies where you look at like, how does a molecule actually change and interact with a cell? And those are all data sets that eventually we can try to put together to create a real virtual cell that is like, you know, an approximation, but like useful enough.

6. 13:19

   And so I'm not, I'm rarely ever that surprised on things. I think a lot about the future. So a lot of different scenarios that are in my head. And like, it's also very clear that AI can code, AI is code. So recursive self-improvement is almost inevitable from that perspective. There's a lot of things to figure out to do it well, especially on the ideation. Um, but, uh, but yeah, I'm, I'm very optimistic that we're going to see just this new era of scientific discovery.

7. 13:47

   When you think of the Eureka machine, is it like an extension of the stuff that we're doing now or are there innovations that you can see maybe post transformer architectures or some new way of thinking about data or quantum computing? I don't know what these things are. Uh, do you know what these things are or do you think that like we are, we have the right tools and we just need to continue in the same direction it's a mix i think we can get much further still if we just collect more data on biology then we can uncover more useful things in biology in fact i think neural nets and the current language of ai is kind of what calculus was to physics it's like a new language that can allow us to make sense of make predictions off and work with biology because biology has all these like very complex systems that

8. 14:40

   don't have a very simple beautiful linear relationship or one small equation that summarizes the whole microbiome or the whole brain the way like physics had and so I think it's often now science is about weaving these these smaller pieces we understand well like we understand the synapse understand the neuron we understand like this particular bacteria But when they all put together and someone gets hangry because they had too much sugar and then the blood sugar dropped, now why does that change the brain?

9. 15:09

   Like, no one really knows how to, like, thoughts really form and why do people act in certain ways and so on. Like, there's, like, this complexity and we need this new language. So, and then concretely, the Eureka machine has these four pillars. World knowledge, which we kind of can approximate to a good degree with large language models the way they are now. And we can certainly make them better, faster, cheaper, easier to train with less data and more energy efficient and so on.

10. 15:37

    But, you know, they do a decent job already to capture a lot of world knowledge that's on the Internet. Then we need to have more scientific measurement data as a second pillar. We need to have simulations on things. And then we need to have real robotic experiments to way kind of periodic labs, for instance, does really cool in chemistry. and then on top of those four pillars we need to have like a community of AI scientists, agents that basically try to automate the scientific method of ideating, implementing, and validating ideas. And so in many ways, the pieces are there. They can all be much more scaled and they can all be done much better and more efficient, sample efficient, energy efficient, and so on. And then, you know, you have to put them into the right environments. You have to give them the right rewards. And then, yeah, I think that you can

11. 16:31

    kind of see that future in the next five to 10 years to really change all of the different