GPT-6 handled research-level physics problems for about $3 each

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 29:24

   is that right and it's still not enough crazy all right i'm gonna go to our third breaking story this week, again, from Uemod. So last week, we discussed how OpenAI dropped 722 math proofs in GitHub and sort of shocked, obliterated, cooked, made glorious in the mathematical world, whatever you want to call it. And, you know, Alex has been saying forever, and as have Uemod, we've talked about this, that physics is next. And so this week, researchers built a test of 50 unpublished research-level theoretical physics problems with verifiable answers.

2. 30:00

   Because the problems had never been published before, there was no lookup on the internet that it could make. So they estimated that a world-class physicist would take about a day to answer each of these 50. And GPT-6 Astra answered it with about a cost of, what was it, $3? Yes, $3 per problem. $3 per problem. So employing a world-class physicist for a day or answering it for $3. Interesting detail.

3. 30:28

   Well, originally, it was reported that Astra got 49 of the 50 physics problems correct. But when it looked into it, it found out that the published answer for the 50th was wrong, and Astra got it right. So what are we seeing here, Imad? I mean, again, like, for these verifiable domains now, the AI is just simply better than us. Like, for the math problems, it only cost a few hundred bucks to solve these long-standing math problems.

4. 30:58

   Again, physics is verifiable. It does this as well. Even without internet lookup, so it has to rely on its own knowledge, it still scored 42 out of 50. And as you said, it corrected that final thing. So this creates an interesting thing whereby anything verifiable, we suddenly have a tool to verify. And that's fantastic because it means we can build much more complex systems and processes. I mean, like, Salim, this must be great for organizations, right?

5. 31:23

   Unbelievable. The implications for how you think about specific tasks, which are well understood, KYC in a bank, you know exactly the outcome you want to get to. It's a very bounded domain. You know where to get the data. In the pilot that we're running, people are taking 22-step workflows and dropping it down to four steps. That's just going to make organizations 10x more efficient and effective across the board in workflows.

6. 31:49

   You know, I am curious, and we'll maybe chat with Alex on the next episode as well about this. So these were 50, shall we say, you know, what level of difficulty would you say? But when it comes down to like the Millennium Physics Prizes, right, if there's that describing that category. Level, yeah. Yeah. How far are we from that? So these would be at like, again, professor level because the professor could check it within a day.

7. 32:14

   So it's a verifiable set of things in various advanced places of theoretical physics, which tends to be very mathematical. the nature of the way it is. Like, the big problems of physics like quantum gravity, you know, like where is time and quantum mechanics, and Unified field theory. Unified field theory. I don't think they'll be solved this way, actually. I think they'll be solved another way, but I don't think it'll be more than half a year.

8. 32:39

   Oh, my God. There's one observation I've been thinking about, which is, you know, a couple of years ago, Eric Schmidt, on stage of A360, said, you know, imagine a time when you have the world's best physicist or equivalent in every lab in the world. And we asked him when, he said five to six years, and now it's like two years, and here we are. So I think this is another example of stuff happening much faster than we ever predicted, because now every lab has access to an incredible competence at a few bucks an hour.

9. 33:11

   And that makes it unbelievable. But then you think about, you know, Anusha, what this means for the XPRIZE and things. You've had so much incredible science coming through. It can now be checked and verified and extended with augmented capabilities to improve that knowledge base that's been built, right? This definitely can help the validation process. What I'm curious about, and perhaps it will be part of the next topics, when you have AI solving, you know, bulk solving a lot of these problems, and you have young people entering universities or picking a field to study, they probably will not be very excited to go in a field where AI can do the job better and can solve all the problems.

10. 33:54

    At the same time with this article, it talked about how it's still not at the level, and I don't know if it will ever get there, that have the same curiosity to identify new problems to be solved. So if we don't have a new generation of mathematicians, physicists, astrophysicists identifying those, you know, new challenges and mysteries, and we have AI bogs solving all the, you know, problems that problem sets that we've defined, what's going to, how are we going to advance?

11. 34:25

    Well, I think, again, that's where the humans we can still direct because it's about what's good for humanity, right? so you've had a bunch of problems but like with fermat's last theorem being proved was fantastic mathematics what difference did it make to real people i think we have these frameworks and again physics is one that i believe will be solved because there's only one physics there isn't different physical laws so there must be something that connects them and these new generations of models will help us figure that out but then once we have that framework what matters is how we apply physics to make flying cars to explore the universe to explore biology you know and more things like that. So I think, again, What this means is that we can move away from theory, trying to find the contours of the

12. 35:05

    universe, to applying the frameworks to make a difference on the universe. I think that's the super exciting part.