AI can solve problems before it can choose the right questions

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 37:07

   just broadly. So there's a essay from Daniel Cocotaglio, Scott Alexander, and others that outlines steps along the way to get to ASI and has a lot of interesting ideas in it. One of which is including a superhuman. encoder and a superhuman AI researcher. And in that, there's a term of research taste that's really interesting. So in everything you've seen, do you think it's possible for AI systems to have research taste, to help you in the way that AI co-scientists does, to help steer human brilliant scientists, and then potentially by itself to figure out What are the directions where you want to generate truly novel ideas?

2. 37:56

   Because that seems to be like a really important component of how to do great science. Yeah, I think that's going to be one of the hardest things to mimic or model is this idea of taste or judgment. I think that's what separates the great scientists from the good scientists. Like all professional scientists are good technically, right? Otherwise, they wouldn't have made it that far in academia and things like that.

3. 38:19

   But then do you have the taste to sort of sniff out what the right direction is, what the right experiment is, what the right question is? So picking the right question is the hardest part of science and making the right hypothesis. And that's what, you know, today's systems definitely they can't do. So, you know, I often say it's harder to come up with a conjecture, a really good conjecture than it is to solve it.

4. 38:44

   So we may have systems soon that can solve pretty hard conjectures. um you know i i am in maths olympiad problems where we you know alpha proof last year our system got you know silver medal in that really hard problems maybe eventually we'll be able to solve a millennium prize kind of problem but could a system come up with a conjecture worthy of study that someone like terence tau would have gone you know what that's a really deep question about the nature of maths or the nature of numbers or the nature of physics and that is far harder type of creativity and we don't really know today systems clearly can't do that and we're not quite sure what that mechanism would be this kind of leap of imagination like like einstein had when he came up with you know special relativity and then general relativity with the knowledge he had at the time

5. 39:29

   and for conjecture the you want to come up with a thing that's interesting it's amenable to proof yes so like it's easy to come up with a thing that's extremely difficult yeah it's easy to come up with a thing that's extremely easy but that at that very edge that sweet spot right of of basically advancing the science and splitting the hypothesis space into two ideally right whether if it's true or not true you you've learned something really useful and um and and that's hard and and and and making something that's also uh you know falsifiable and within sort of the technologies that you have you currently have available so it's a very creative process actually highly creative process that um i think just a kind of naive search on top of a model won't be enough for that okay the idea of splitting the hypothesis space into is super interesting so

6. 40:22

   i've heard you say that there's basically no failure in or failure is extremely valuable if it's done if you construct the questions right if you construct the experiments right if you design them right that failure success are both useful so yeah perhaps because it splits the hypothesis basis too it's like a binary search that's right so when you do like you know real blue sky research there's no such thing as failure really as long as you're picking experiments and hypotheses that that that meaningfully spit the hypothesis space so you know and you learn something you can learn something kind of equally valuable from an experiment that doesn't work that should tell you if you've designed the experiment well and your hypotheses are interesting it should tell you a lot about where to go next and um and then it's you're effectively doing a

7. 41:08

   search process um and using that information in in you know very helpful ways so to go to your dream