AI researchers may already be driving AI research

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 39:57

   Alex, over to you. Yeah, if you look at... The charts as Meter released several really excellent to some perhaps stunning charts on this exponential or hyper exponential that we're on. There was some interesting commentary. If you remember the AI 2027 scenario that we're within 10 to 15 percent of where AI 2027 predicted that we'd be by November 2026. Look at model four on that chart. So there's this codename model four that appears on meters charts, it is above the exponential trend line. We are so deep in recursive self-improvement at this point. I'm not even sure if meters fitted trend line adequately captures how deep into recursive self-improvement we are. And this is why I say you could stop all data center construction tomorrow. And yes, it would be painful. And yes, this is not investment advice,

2. 40:52

   but presumably public equities would collapse if all new construction on data centers were halted. But at this point, so much of the AI R&D is itself driven by AISI. I'm not sure it matters at this point. It would be a blip. Maybe we'd lose a few months of headway over China, probably fall behind or further behind. But basically, this is all being driven. All the research is increasingly being driven by superintelligence.

3. 41:20

   And so we're past the point of, I think, no return. So the metaphor here is that the car is moving pretty quickly. And even though it's running out of gas, it's going to have enough momentum to get us there. I think so. I think we're at a, I mean, you pick your metaphor, escape velocity. We're at superintelligence escape velocity at this point and stop all data center construction. Yes, it'll be painful.

4. 41:44

   But yes, I think we have enough algorithmic insight combined with the compute that's already available if we had to in a crisis to, maybe some wouldn't view it as a crisis, but call it a counter-singularity crisis to just get to where we're going, which is, I think, the perfect model. Yeah, this is so understated. It's almost, we are so deep into RSI. Because on that last podcast, remember we said that the use of tokens for internal development in Anthropic is up 235x in the last six months?

5. 42:18

   You don't do that unless, yeah, of course it's self-improving. Why would that happen? Also, in the meter report, they showed the fraction of all compute that's going into internal work as opposed to external product. development. And if they showed it a year, I think it was a year or two ago versus today, I mean, this entire thing has been hugely redirected toward itself, inside of itself. You only do that when you're in recursive self-improvement. Not to mention, we have entire companies being founded called Recursive, you know, it's by top AI gods. So yeah, we're so, so deep into the self-improvement cycle at this point. But then you have to ask yourself, like, well, who is going to tell you that and why would they? And it used to be Dario because he was just being the honest guy in the industry. Now he's in a quiet period going into an IPO. So why would

6. 43:12

   anyone ever say that? And the only motivation that I know of is ad views. You're trying to put a negative spin on it and create panic and whatever to try and get ad view revenue. That's the only reason you're going to, you're going to hear the story. So I think as a listener, you have to be really, really conscious of the fact that like, okay, there aren't a lot of people out there that have a lot of upside in telling you what's going on. It used to be the labs, but you saw what happened to the labs, you know, how lambasted they've been by, by mostly Washington. Uh, so they've kind of quieted down. They don't have a lot, you know, roiling the, uh, the voter up doesn't really help you too much. So they've, they've gotten quiet. Who else is going to tell you what's going on. And so it's something to watch out for. If we hit RSI, how would we know?

7. 43:57

   Like, what's the benchmark that would tell you that? I mean, I would argue we have, and we have so many benchmarks. So I think we're really deep into it already. You can look, the original benchmark as floated by, I think, execs from both OpenAI and Thropic was something like, whether for unmargin every new flop of compute, whether it's more likely to get assigned to an AI researcher versus a human researcher. And Anthropic has been pretty good, I would say, about putting out charts regarding how much supervisory compute versus, call it grad student level compute versus more menial engineering compute is being assigned to humans versus being assigned to pure AI or SI researchers. And every chart, every one of those sigmoidal curves points to us being either at or past the point where substantially all research at this point is

8. 44:54

   actually being conducted by computer, by machine researchers rather than by human researchers. I think that's as decent. a definition of RSI as any. Yeah, absolutely. Absolutely. And, you know, and a little subtlety to add in there. If you go to Noam Brown over at OpenAI and you say, hey, Noam, you're a brilliant AI researcher. Give me your 10 best ideas on what to work on next week that might make the algorithms faster, smarter, bigger, better. Okay, write those 10 ideas down. Now let's go to Astra and, you know, you just solved Navier Stokes. Okay, give me your 10 best ideas. If Noam has six out of 10 good ideas and the AI has one out of 10, you're way past RSI because the AI is effectively a multi-billion person research force and Noam is still one guy. And so, and they're all going to get tested. It's not like there's like a, you know, it's not like being a major

9. 45:43

   league batter where it matters how many times you strike out. As long as you're hitting balls over the wall, you have an infinite number of pitches. You just test them all. And so that's why all the compute is getting sucked internally. So you're way past RSI if it has any good ideas at all. Does anyone believe that it doesn't have good ideas after Navier-Stokes? I mean, of course it does. So you ask the AI to generate a bunch of options, then it tests the options and ask it to run a bunch more. You just keep pushing that paradigm. Yeah. And within that area of endeavor, there's no physical experiment required. It's all just run the code. Yeah, I completely agree, Dave.

10. 46:21

    And I would say Navier Stokes, what we saw was purportedly 10,000 agents being used to solve it. But then fast forward to the slop drop, and it was three agent hours per solved problem. So it may not even be the case. It may be the case that needing a fleet of brute force agents to solve a hard problem is itself in the rearview mirror at this point. And the agents have sufficiently good taste. I agree with Dave's point that it's sufficient to have agents with just a little bit of good taste, and then you can brute force, you can amplify that up with lots of horizontal scaling.

11. 46:58

    But there is every indication that we're past what is sufficient. And at this point, we're at or nearing the point where models like Bell, which is, as we're recording this, expected to drop.