Cell models need information-rich data and useful inductive bias

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 14:42

   start by saying that, you know, I really think that kind of in the limit of, you know, data and the limit of infinite data and compute, if you're thinking in limits, kind of all model architectures probably kind of end up being the same, right? They're just kind of function approximators, Right. And so if you have the right inputs and the outputs and you, you know, you learn a very, very good general function approximator, then then then in that sense, in those limits, kind of the model architecture itself falls away.

2. 15:11

   But I think I think in biology, you know, we're very far from that point. We're very far with some exceptions, right? Like with protein biology, you know, we have gene sequencing at scale. You know, we really can kind of, you know, get closer and closer to kind of that limit of of of infinite data. But for for cellular biology, you know, we're just so far from that right now. And I think historically, when you're far from that limit.

3. 15:37

   You know, inductive bias becomes really important. When I think about, you know, a model, I mean, I think where do we ultimately want to get? You know, we want to get to a model of the cell. It's kind of what I was saying before. You know, it has that complexity at the molecular level. It has a representation of that. You know, it could, you know, imagine kind of being able to project out the state of the cell down to the level of, you know, atomically resolved structures of its components, you know, would have that.

4. 16:12

   It would also really be able to model the cell at the level of its programs, and it would be able to, you know, predict. It would be able to allow development to unfold, to allow the interactions within tissue to unfold and tell you what would happen. So in that sense, you could think of it as a world model, right? Kind of a world model of life, a world model of biology. I think we're very far from that. So what I think is possible now is an information theoretic model of the cell. And I think that there are, you know, really these programs that run life. They're encoded in the genome.

5. 17:06

   and i think it's it's historically you know there's there's been these kind of two dual ways to think about biology one is this kind of bottoms up first principles approach kind of trying to build it from the basic chemical and physical principles but i think the other view that's been really productive is to kind of really think about the the emergent level of the informational model of this of the cell you know the cells and information processing machine which has a genome, which encodes the programs that run life. And I think that's the level of understanding of the cell that could be attainable via AI in kind of the next five years, let's say.

6. 17:52

   I think that we really have, you know, we're in the age of information theory at scale. And in a sense, you know, we have these incredibly powerful methods which can perform, you know, they do pre-training, they use self-supervised learning on these, you know, vast data sets. And from statistical patterns that link inputs to outputs, they're able to capture really high fidelity representations of the underlying latent variables that are driving the relationship between input and output. And so you can imagine generating data on the cell that could really start to reveal how information at the genetic level, at the transcriptomic level, at the molecular protein level, would link to the programs and responses.

7. 18:45

   of the cell that can be observed through different molecular readouts, such as transcriptomics and gene sequencing and epigenetic measurements, but also through imaging and the power of just being able to see how the cell responds. So I think you were exactly right to kind of think about it as one of the first things that we can really think about doing is really learning this transition function that takes us from a state and a set of inputs to the next state.

8. 19:15

   And it's really a probabilistic function. It's not going to be deterministic. The cell is going to transition, I think, to a distribution of possible downstream states. So it's a very complex problem to learn. But I think today, we kind of have the technologies now that are going to make it possible to create that kind of model. Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is sponsored by Parallel,