OpenAI Math Breakthroughs Show We Are Near Full RSI | Prinz

critical systems to make sure that some of the worst effects won't happen um but we can't know

Prinzfor sure, right? And I think That right now, anything that is vulnerable is going to be attacked by private groups, by state hackers, what have you. So we'll find out. I highly recommend, as a non-technical person, I highly recommend that everyone listening to the show do as much as they can to secure their personal stuff. Two-factor authentication, like change your passwords, like do as much as you can.

Yeah, I think this is going to be a project for me over the next couple of days. Yeah, I mean, it, you know, people say this about, people use the COVID analogy all the time. And I think it's worth remembering for people that COVID was very far from the worst case scenario for what could have happened. Like, looking at the state of COVID-19 in, like, Wuhan, China in, like, November, December of 2019, seeing some of the headlines that were coming out of, like, China and then Italy where it was like, oh, my God, like, they literally are, like, filling up hospitals. They have to, like, build new hospitals because there's so many people getting sick.

Like, the morgues are filled up because so many people are dying. And now this is spreading to northern Italy. And, you know, my vacation to Italy got canceled because of that. Um, and like, holy shit, like, what is this going to look like? And it turns out COVID was like on, you know, on the grand historic scale of things that have happened to the world and to specifically the day-to-day lives of people in the developed world, COVID was moderately bad, but it wasn't anywhere near as bad as say, world war two. Like it could have been, like it could have killed like 10% of the population of the country or something like that would have been really, really bad.

Yeah, it could have been like the Black Plague in the Middle Ages, right? Like, there was no way. Yeah, that's exactly right. And it is, I think that there are lots of analogies to COVID because that was a big expect the unexpected moment for a lot of people, right? Prior to COVID, I think a lot of, at least people in my circle, had this kind of nothing ever happens type of mindset for everything, for everything. Like, whatever happens in the world, whatever, the stock market is going to crash by 2%, 3%. You know, a couple of years later, we're going to be out of it. You know, for me personally, it was the second time something crazy like that happened because my childhood was spent in Eastern Europe under a communist regime.

And so we had our own nothing ever happens, right? Right. And, you know, the USSR and all its satellite states were supposed to, you know, last for decades, centuries, like the American intelligence predicted this. And then in the span of a couple of years, it all completely collapsed. The economy collapsed. The regime collapsed. It was chaos everywhere. um and um you know a lot of depths and a lot of civil wars so i am totally prepared for number three uh for me and number two for many of you listeners um you know hard to know where it's gonna go but the best you can do to prepare is is the right move right now i think right i want to shout out our sponsor lovable you know the app you've been meaning to build the internal tool, side project, or product you'd otherwise lose a weekend to, with Lovable, you get software you can ship today.

That includes an editable code base you can inspect and change, plus two-way GitHub sync. Lovable also handles the backend and infrastructure, including managed Postgres, auth, storage, hosting, payments, and more, so you don't have to wire it all up yourself. And through Lovable's MCP server, you can create and deploy your project directly from the agents you already use. Turn ideas into software people love with Lovable. Lovable.dev. On to the episode. So we're also going to talk about RSI and Foom. What exactly do you picture when you think of RSI? I think, you know, sort of like how cyber apocalypse means different things to different people.

RSI means different things to different people. Where when some people conceive of RSI, they think of like, oh, we're already experiencing RSI because OpenAI and anthropic researchers are like heavily using models to like write experiment code for them or whatever. and it's accelerating the returns to AI R&D to some extent, maybe 2x. Whereas when other people think of RSI, they think of it as like AI fully automates the AI R&D process and this leads to some incredibly discontinuous, like fast takeoff, hyperbolic growth in AI capability that takes us to like for real, for real superintelligence in a very short period of time.

Yeah, look, so I think that both of these things are true, and the only question is of definition. So it's a good thing that you're talking to a lawyer because definition is what we do. um i to a certain extent definitions are not particularly important right because the process as you outlined has already started and eventually i think it will be um this automated growth and capabilities that will be very fast um but for me i think i focus on the word self and self improvement right so until the models can improve themselves and in fact significantly improve themselves not just you know like train and a non-model like under supervision of a bigger

Prinzmodel you need to be able to train a model that is your successor and to make it more capable than you are until we have that i don't think it's rsi so i think that right now we don't have rsi

at least it's not been publicly disclosed and i don't think that the labs have it internally either um i think we will have it when um an ai model at open ai and or anthropic is going to be able capable of fully autonomously not just um building the next model but also coming up with really creative ways to make the new model more capable right like big um uh kind of like ilia suits cover type like incredible brilliant moments and in a r&d that lead to a gigantic jump in capabilities and i think that's coming pretty soon how soon i mean it's hard to say i've always been um i've always thought that the most important thing for rsi is the ability for models to come up with really novel solutions to you know like random like various problems right and so what we saw up until august of this year primarily was models coming up with like brute force approaches to solve various problems like in math um and indeed like when for example i use gpt6 astra i've had a couple of uh i've kind of messed around with a couple of ciphers but were previously unresolved like like uh from like historical like from world war one and um i kind of successfully had astra decrypt a couple of them which was really cool but every one of the results that

i achieved with astra was um the model having the key but um it was it was in in In each case, it was a scenario where humans were not able to use the key for whatever reason, like the cipher was missing some characters or some characters were hard to read or or whatever, like those kinds of things. And then Astra could brute force the solution by, like, searching for the entire possible space of possibilities and going, oh, actually, like this approach works. And here's how the cipher should read. um i have personally not been able to get astra to like come up with the key on its own based on some novel cool idea like oh you know like this person lived in modena italy and so this there's a clue here that says tower and it must be the tower that's 55 miles from his house so like let's look at that and oh that's the key you know i mean like it's like stuff like that like really big creative leaps um so all of which to say that the math results recently achieved by open ai's internal model have really updated me because um it does seem like some of these results were like involved really creative thinking um i think terence now said about the quasi reeman solution that it was a combination of three or four existing ideas but arranged in a very unusual

way it's right right so it's when you can arrange ideas in a very unusual novel way um you kind of start thinking about creativity but human genius humans exhibit um and i think that generally speaking these kinds of abilities generalize and eventually they get you creativity in AI, R&D, and then that enables the models to unlock significant new advances. I think there's a delta between creativity and math and creativity and other things. I say this just because I think that Rune has said publicly that there's a delta. I think I've seen a tweet or two from him on that, and I believe him at his word.

um so it doesn't necessarily mean that like an internal model that the open that open ai currently has is able to already achieve rsi i'm guessing probably still not but it feels like we're really not very far away um because once the models are able to do something you know how it is in 12 months or maybe sooner they'll be really really really really good at it and we'll be able to do it faster and cheaper. So I think we're on the brink. Right. Do you think that RSI will just, you know, many people think RSI will just solve all of LLM's current weaknesses, solve jaggedness. Do you think that this will happen?

I mean, I have to think yes. If I were one of the Frontier Labs, one of the first directions at which I would point RSI would be some kind of novel model architecture that is better than the transformer maybe more efficient maybe more human-like maybe more energy efficient but it feels like we kind of stumbled on the transformer architecture right and we've been using it ever since and it's not clear but it's the final thing um and it seems that to the extent there are limitations with it it's going to be the automated ai researcher that'll be able to discover it um and so you know it's it's kind of it's hard to say um whether that's low-hanging fruit or not it's possible that it might take years for even a couple a couple of years of models to become creative enough to discover or something really But I think it's a possibility. I would not be surprised if in five years' time, the AI architecture that everyone was used were completely different and alien from what we're used to right now.

Yeah, but I don't understand how this would magically solve jaggedness. It seems to me that a lot of the reasons that LMs are jagged are because of data bottlenecks. So like something like bio, for example, it seems like, you know, maybe not completely impossible. I mean, in AI, you can't rule things out, but pretty unlikely that an AI would just be able to like solve its constraints in biology just by doing RSI on a computer, whatever that means. no i think that's exactly right um look like if you took albert einstein and you put him in a lab and said like you need to cure cancer like tomorrow um he would not be able to do that even if he had the knowledge right like he wouldn't be able to do that without having the right data the right experiments right um but i don't i don't know if that's um evidence of jaggedness i think it's evidence but sometimes a brilliant intellect whether human or artificial needs to be given um the you know the right environment and the right data um but on the other hand for example um i'm a lawyer right so if i think a lot about llms and their ability to like write a good legal research memo right for which you don't need albert einstein but llms right now we're not particularly good at it they're becoming better

but they're not incredible and i don't really think it's about the data i think it's just because llms are not as efficient um at learning some of this this non-verifiable stuff as humans are um and uh one answer may be that in a year they will be really really good at it regardless but another answer may be that a better architecture will just make them more human-like and learning and then an llm will be able to learn to write a legal memo just as well as a law firm associate can do that we'll continue monitoring right after this message from our sponsors 11 labs ai that communicates at human level across every channel and modality 11 labs.io slash mts scale your startup on neon millions of developers and startups have already chosen neon for their back end start on the free plan or get up to a hundred thousand dollars in credits for your startup at neon.com slash mts special thanks to our sponsor kong the ai connectivity platform Connect APIs, LLMs, agents, and systems with serious security and governance.

KongHQ.com Finally prove what your AI investment is worth with Lairdyn. Head to Lairdyn.com today to start measuring your AI's bottom line. MongoDB. Trusted to run critical applications by 75% of the Fortune 100, the intelligent data platform for the era. Apply today for access to up to $15,000 in Atlas credits, dedicated technical support, and exclusive events and go-to-market opportunities. MongoDB.com. Chargebee, billing and monetization infrastructure for the AI era. Chargebee.com. What are the current issues with LLMs writing legal memos right now? I think it's, you know, LLMs produce a lot of slops still.

like right now i feel like if i feed even the best tell alarms um like a legal question right i am going to get output that will be a haystack with a needle in it. The needle will be there.

PrinzSo if I'm a specialist lawyer, if I know what I'm doing, I will immediately extract the needle and it'll be incredibly helpful.

So instead of me coming up with that needle for 10 hours, the LLM will give it to me in an hour and it's perfect. But the rest of the haystack, like for a non-lawyer, right? For a first-year associate or something, is um like it's it's it's not very useful and so what happens is that in the context of a legal memo it's the same point right so if i ask an llm a complicated legal research question it'll give me a 35 page memo for something that can be said in two pages and much sharper and much quicker right a lot of the time its answers are going to be correct if you use the right model It's a right reasoning effort, as opposed to, you know, like Sonnet 5.5 instant or whatever.

But it takes a lot of processing to get past that result. Right. So do you think that the economy will really grow 17% per year, which is the, I believe, the median expectation of anthropic employees by 2030? um well i hope so you know uh it's it's really funny right so um we in in the in the western world we're very used to our economy growing at only a few percent per year right that's the way it's been for many many many decades and if you like zoom in a lot of it is because we our economy is extremely developed right we don't really have like an incredible scientific advance or a million scientific advances every year to like really jump us like get us jumped forward um a lot of the time our economic growth really comes from like a private equity shop like buying a supermarket and figuring out how figuring out how to make its logistics two percent more efficient um and that's all great like that's in in the long run that's a really big and winning combination but you're not going to get 17 economic growth out of that sure um i do think though that if you are a gi pill then if you expect lms to start discovering seriously like novel things in science you know drug discovery material science you know uh discover how to

operate robots and like go to space and colonize the moon what have you um if you think of big scientific discoveries being made at scale and what you can do with that you know like for all kinds of aspects of the economy right like like imagine just people like living longer by 30 years on average and having more productive careers at vertel and it could unlock huge huge economic growth um so i am not really sure that expectations based on our priors from the past couple of decades are really the right way to look at it um i will say that um for example like china after Mao Zedong's death in the 1980s and the 1990s, it was pretty common for China to be growing at 10% or more per year. It was a smaller economy back then, but a lot of it was because under real, like under Maoist communism, there wasn't much the economy was not very advanced, right? And so you could land in the middle of China.

PrinzBuilt a fast food restaurant or something. And it just created so much net new value, but the economy grew rapidly. And I think that if you substitute new scientific discovery, like a new drug for fast food restaurant, that you could get that kind of analogy. I don't know if it's 17%. I don't know if it's by 2030. But I would not be surprised if we did see some major economic growth.

right well we're at time but it's been really interesting having you on prince thanks so much for coming on mts thank you so much thanks for having me absolutely a huge thanks to mts sponsors blitzy autonomous software development for enterprise code bases ship 5x faster blitzy.com adquick make your brand a billboard out of home advertising as easy to scale as digital adquick.com arena measuring ai performance in the real world arena.ai support for the show comes from vcx the public ticker for private tech the u.s stock market started history's greatest wave of wealth creation from factory workers in detroit to farmers in omaha anyone could own a piece of the great american companies but today our most innovative companies are staying private longer which means everyday americans are missing out until now introducing Investing VCX, a public ticker for private tech. Visit getvcx.com for more info. That's getvcx.com.

Carefully consider the investment material before investing, including objectives, risks, charges, and expenses. This and other information can be found in the Innovations Fund prospectus at getvcx.com. This is a paid sponsorship.