Reinforcement learning scales when experiments give clear feedback

Transcript

From the transcript TBPN publishes with the episode.

1. 48:53

   Okay. Let's say you're talking you do have a panel in a second. I wish we could You're go talking to somebody who is now willing to admit that this AI thing is real. They see the revenue ramps. Let's say they're using various models. They're like, this is very capable, but they're getting into it. And they're saying, like, okay, a lot of a lot of the revenue is tied to coding right now. These tokens are deflationary over time. Consumption? Where is it gonna come from?

2. 49:26

   Right? Because, like, finance is an interesting category, but it's I I don't see, you know, 200,000,000,000 of, you know, financial modeling spend on tokens. Right? So where where are you predicting or thinking about these sort of, like, next legs next legs up after coding?

3. 49:50

   So the the gift that keeps giving still is reinforcement learning. Like, RL is just working. Yeah. Like, as a technology, it's so simple. Sure. It's like training training a dog. Right? Like, hey, fighter. Go fetch. You did it right. You know Here's a treat. Here's a treat. And reward modeling is is used to be like this bespoke craft thing like two years ago. Now, we're we're getting to the stage of industrial grade, like repeatable reinforcement learning as a service.

4. 50:17

   Mhmm. And so I think taking models and doing post training where you have the weights and then post training on data that you care about on some small set of samples that are high quality where the reward is very clear and you're doing RL gets you your own jagged frontier very fast. Yeah. You because know, you guys know this. Like, capabilities, if there's a jagged frontier, you know, coding is clearly one area because of formal verification Yep.

5. 50:40

   With unit tests Yep. The progress has been super fast. Mhmm. I think verification is people just still haven't truly internalized, like, how generalizable this concept of, like, the context feedback loop with verification built in Mhmm. Is. And the algorithm is so simple. Like, RL just works Yeah. That wherever you can do, like, formal verification and RL, you're gonna see crazy capabilities. So one example is Periodic Labs.

6. 51:06

   It's one of our portfolio companies. Yep. They're trying to discover a room temperature superconductor. The pace of progress there has been extraordinary in the last, like, six months. They've got a 40,000 square foot facility in Menlo Park That's awesome. Where you got AI models that are predicting new materials Mhmm. New candidates. Mhmm. And then you have robots synthesize the material and then test with an x-ray diffraction machine. Does the material have the superconducting properties that the the model said it would?

7. 51:33

   And because that's verification from reality, like from nature, like physics. So you can do x-ray diffraction to test Yeah. Does it have it or not? You the feedback is the signal is very clear, and then you take that signal and pipe it back into RL, like, as many times as you need. And It's exciting. The the the capabilities ladder is is extraordinary.