Nature's structure may make hard problems learnable

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 2:02

   in your nobel prize lecture you propose what i think is a super interesting conjecture that quote any pattern that can be generated or found in nature can be efficiently discovered and modeled by a classical learning algorithm what kind of patterns of systems might be included in that biology, chemistry, physics, maybe cosmology, neuroscience. What are we talking about? Sure. Well, look, I felt that it's sort of a tradition, I think, of Nobel Prize lectures that you're supposed to be a little bit provocative. And I wanted to follow that tradition.

2. 2:37

   What I was talking about there is if you take a step back and you look at all the work that we've done, especially with the Alpha X projects. So I'm thinking AlphaGo, of course, AlphaFold. What they really are is we're building models of very combinatorially high dimensional spaces that if you try to brute force a solution, find the best move and go, or find the exact shape of a protein, and if you enumerated all the possibilities, there wouldn't be enough time in the time of the universe. So you have to do something much smarter. And what we did in both cases was build models of those environments. And that guided the search in a smart way.

3. 3:15

   And that makes it tractable. So if you think about protein folding, which is obviously a natural system, why should that be possible? How does physics do that? Proteins fold in milliseconds in our bodies. So somehow physics solves this problem that we've now also solved computationally. And I think the reason that's possible is that in nature, natural systems have structure because they were subject to evolutionary processes that shape them. And if that's true, then you can maybe learn what that structure is. This perspective, I think, is really interesting one you've hinted it at it which is almost like uh crudely stated anything that can be evolved can be efficiently modeled think there's some truth to that yeah i sometimes call it survival of the stabilist or something like that because you know it's it's of course

4. 4:05

   there's evolution for life uh living things but there's also you know if you think about geological time so the shape of mountains that's been shaped by weathering processes right over thousands of years but then you can even take it cosmological the orbits of planets the shapes of asteroids these have all been survived kind of processes that have acted on them many many times so if that's true then there should be some sort of pattern that you can kind of reverse learn and a kind of manifold really that helps you search to the right solution to the right shape and actually allow you to predict things about it in an efficient way because it's not a random pattern. So it may not be possible for man-made things or abstract things like factorizing large numbers, because unless there's patterns in the number space, which there might be, but if there's

5. 4:55

   not and it's uniform, then there's no pattern to learn. There's no model to learn that will help you search. You have to do brute force. So in that case, you maybe need a quantum computer, something like this. But in most things in nature that we're interested in are not like that. They have structure that evolved for a reason and survived over time. And if that's true, I think that's potentially learnable by a neural network. It's like nature is doing a search process and it's so fascinating.

6. 5:23

   that is in that search process is creating systems that can be efficiently modeled. That's right. Yeah. So interesting. So they can be efficiently rediscovered or recovered because nature is not random, right? These, everything that we see around us, including like the elements that are more stable, all of those things, they're subject to some kind of selection process, pressure. Do you think, because you're also a fan of theoretical computer science and complexity, Do you think we can come up with a kind of complexity class, like a complexity zoo type of class where maybe it's the set of learnable systems, the set of learnable natural systems, LNS?

7. 6:02

   Yeah. This is a new class of systems that could be actually learnable by classical systems in this kind of way, natural systems that can be modeled efficiently. Yeah. I mean, I've always been fascinated by the P equals MP question and what is modelable by classical systems, i.e. non-quantum systems, you know, Turing machines in effect. And that's exactly what I'm working on, actually, in kind of my few moments of spare time with a few colleagues about is should there be, you know, maybe a new class of problem that is solvable by this type of neural network process and kind of mapped onto these natural systems.

8. 6:40

   so you know the things that exist in physics and have structure so i think that could be a very interesting uh new way of thinking about it and it sort of fits with the way i think about physics in general which is that you know i think information is primary information is the most sort of fundamental unit of the universe more fundamental than energy and matter i think they can all be converted into each other but i think of the universe as a kind of informational system so when you think of the universe as an informational system then the p equals mp question is a is a physics question that's right it is a question that can help us actually solve the entirety of this whole thing going on yeah i think it's one of the most uh fundamental questions actually if you think of physics as informational uh and and the answer to that i

9. 7:24

   think it's going to be you know very enlightening more specific to the pnp question this again some of the stuff we're saying is kind of crazy right now just like the christian enfance and the book speech controversial thing that he said sounded crazy. And then you went and got a Nobel Prize for this with John Jumper, solved the problem. So let me just stick to the P equals NP. Do you think there's something in this thing we're talking about that could be shown if you can do something like polynomial time or constant time compute ahead of time and construct this gigantic model, then you can solve some of these extremely difficult problems in a theoretical computer science kind of way.

10. 8:09

    Yeah, I think that there are actually a huge class of problems that could be couched in this way, the way we did AlphaGo and the way we did AlphaFold, where you model what the dynamics of the system is, the properties of that system, the environment that you're trying to understand. And then that makes the search for the solution or the prediction of the next step efficient, basically polynomial time. So tractable by a classical system, which a neural network is. It runs on normal computers, right? Classical computers, Turing machines in effect.

11. 8:44

    And I think it's one of the most interesting questions there is, is how far can that paradigm go?