# Can AI Invent or Just Help Scientists Work Faster?

Markdown copy of https://realalpha.app/v/rIUcQfyXuUo/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

Richard Socher says AI can already generate useful scientific ideas, but broader discovery depends on compute, data, environments and fast verification.

- Show: The TWIML AI Podcast
- Hosts: [Sam Charrington](https://realalpha.app/people/sam-charrington/)
- Guest: [Richard Socher](https://realalpha.app/builders/richard-socher/) is a former chief scientist at Salesforce, founder of You.com and cofounder of Recursive.
- Published: Nov 3, 2026
- Length: 48 min

## The best minutes

- [16:34](https://realalpha.app/v/rIUcQfyXuUo/?t=994) [Scientific ideas grow by combining work across different fields](https://realalpha.app/s/scientific-ideas-grow-by-combining-work-across-different/) (5 min)
- [21:07](https://realalpha.app/v/rIUcQfyXuUo/?t=1267) [Reward design steers AI and can invite reward hacking](https://realalpha.app/s/reward-design-steers-ai-and-can-invite-reward-hacking/) (5 min)
- [29:22](https://realalpha.app/v/rIUcQfyXuUo/?t=1762) [Recursive reports beating community results on AI benchmarks](https://realalpha.app/s/recursive-reports-beating-community-results-on-ai-benchmarks/) (4 min)

## Also worth watching

- [5:46](https://realalpha.app/v/rIUcQfyXuUo/?t=346) [Organoids can bypass animal trials and AI can revive shelved drugs](https://realalpha.app/s/organoids-can-bypass-animal-trials-and-ai-can-revive-shelved/) (5 min)
- [32:59](https://realalpha.app/v/rIUcQfyXuUo/?t=1979) [Verification speed sets the limits for AI research agents](https://realalpha.app/s/verification-speed-sets-the-limits-for-ai-research-agents/) (5 min)
- [10:13](https://realalpha.app/v/rIUcQfyXuUo/?t=613) [A science AI needs data, simulations and robotic experiments](https://realalpha.app/s/a-science-ai-needs-data-simulations-and-robotic-experiments/) (6 min)
- [41:40](https://realalpha.app/v/rIUcQfyXuUo/?t=2500) [AI rules should target applications, not abstract intelligence](https://realalpha.app/s/ai-rules-should-target-applications-not-abstract/) (3 min)
- [44:25](https://realalpha.app/v/rIUcQfyXuUo/?t=2665) [AI's scientific benefits belong in debates about dual-use risks](https://realalpha.app/s/ais-scientific-benefits-belong-in-debates-about-dual-use/) (4 min)

Skip the opening book and company backstory, and the later discussion of compute, biological data, reward hacking and commercial incentives.

## Summary

Richard Socher is the former chief scientist at Salesforce, founder of You.com and cofounder of Recursive. He says a protein-generation project at Salesforce helped shape the ideas in his book, The Eureka Machine, and led him to start building Recursive.

Richard Socher argues that AI can already contribute new ideas in bounded research settings, but recursive self-improvement depends on compute, useful environments and fast, trustworthy verification. He frames a Eureka machine around world knowledge, scientific data, simulations and robotic experiments, with AI agents handling ideation, implementation and validation. At Recursive, he says the same underlying system has produced reported gains on NanoChat, the NanoGPT speed run and CUDA kernel optimization, while delayed verification and reward hacking remain challenges. He favors regulating AI applications and holding users responsible for AI-directed actions, and says AI-assisted science could help reduce suffering.

### Minute by minute

- [1:36](https://realalpha.app/v/rIUcQfyXuUo/?t=96) **A book became a company.** Socher says his protein-generation work at Salesforce pointed toward the idea of a Eureka machine. He wrote the book over nearly two years, aiming to show how AI could help with science, then decided to build Recursive.
- [3:08](https://realalpha.app/v/rIUcQfyXuUo/?t=188) **AI is advancing across sciences.** Socher points to reinforcement learning and control systems for stabilizing tokamak plasma, language models for finding battery materials, and progress in biology and neuroscience. He says these examples make a future of AI-driven scientific discovery feel more tangible.
- [5:46](https://realalpha.app/v/rIUcQfyXuUo/?t=346) **Organoids could speed drug testing.** Socher describes Parallel Bio’s stem-cell-derived organoids as a way to test drugs in parallel, and says the company received FDA approval to use them instead of animal trials. He says AI can track cells as drugs interact with them, creating data for virtual-cell models. He also cites Ignota Labs, which tries to alter shelved drugs to address problems such as liver toxicity.
- [10:13](https://realalpha.app/v/rIUcQfyXuUo/?t=613) **Science moves slower than software.** Socher says predictions about AI progress often overlook the slower pace of physical experiments: moving molecules and atoms takes longer than iterating on software. He points to large perturbation datasets, including work from Tahoe Therapeutics, as a path toward useful virtual cells and says he expects a new era of scientific discovery.
- [13:48](https://realalpha.app/v/rIUcQfyXuUo/?t=828) **Four pillars for scientific AI.** Socher describes the Eureka machine’s four pillars as world knowledge, scientific measurement data, simulations and robotic experiments. He says AI scientist agents could use them to automate ideation, implementation and validation, and sees the approach reshaping scientific disciplines over the next five to 10 years.
- [16:34](https://realalpha.app/v/rIUcQfyXuUo/?t=994) **Open-ended search may spark ideas.** Socher says Recursive is betting on open-ended processes that combine ideas and discoveries within environments, rather than relying on brute-force search. He uses the invention of the microwave to show how discoveries can emerge by joining separate lines of research. Recursive is applying this approach first to AI research, including CUDA kernel optimization.
- [21:07](https://realalpha.app/v/rIUcQfyXuUo/?t=1267) **Rewards can invite reward hacking.** Socher says agents need guidance through well-defined rewards, but those rewards can be exploited. He illustrates the risk with an AI that raises a customer satisfaction score by creating bots to give five-star ratings, and says kernel optimization also uncovered rare bugs that could then be fixed. He argues that commercial incentives can constrain some runaway scenarios.
- [25:59](https://realalpha.app/v/rIUcQfyXuUo/?t=1559) **AI innovation is already emerging.** Socher says AI is already producing new ideas and combinations at small scales, but recursive self-improvement requires suitable environments and more compute as problems grow complex. He cites novel moves in games such as AlphaGo and progress in math as examples of settings where results can be verified.
- [29:22](https://realalpha.app/v/rIUcQfyXuUo/?t=1762) **Socher reports Recursive benchmark gains.** Socher says Recursive used auto-research on NanoChat to achieve the lowest bits per byte within a five-minute task budget, surpassing other efforts after two days. He says it beat results on the NanoGPT speed run after a couple of days and produced top kernels on nearly all of NVIDIA’s Sol-Exec benchmark, though the team found the benchmark could be reward-hacked.
- [32:59](https://realalpha.app/v/rIUcQfyXuUo/?t=1979) **Verification speed shapes progress.** Socher says faster verification makes iterative research easier, while long waits for results and difficulty assigning credit across steps make the work harder. He says Recursive uses the same underlying system across its examples, with task-specific tools such as a kernel profiler.
- [37:46](https://realalpha.app/v/rIUcQfyXuUo/?t=2266) **Compute and data remain bottlenecks.** Socher says larger inventions require more compute, while broader scientific work needs data and suitable experimental environments. He says reward design and reward hacking will matter more as AI becomes more capable and accessible. He expects commercial incentives to push companies to align costly systems with their goals.
- [41:40](https://realalpha.app/v/rIUcQfyXuUo/?t=2500) **Regulate applications, not intelligence.** Socher doubts frontier-wide slowdowns are realistic and argues that regulation should target AI applications rather than abstract intelligence. On dual-use science, he says people should be responsible for AI-directed actions and cites existing restrictions on gain-of-function research. He emphasizes potential gains in medicine and science, arguing that fears around nuclear energy have helped sustain coal use, which he says causes about 50,000 lung cancer deaths a year.

## Transcript

The TWIML AI Podcast, hosted by Sam Charrington, with Richard Socher.

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[0:00](https://realalpha.app/v/rIUcQfyXuUo/?t=0) Modern AI can write the code, analyze the data, and help draft the paper. But can it ask the questions that lead to new scientific discoveries? Richard Socher thinks we're already seeing the beginnings of that. He's the former chief scientist at Salesforce and founder of U.com. His vision is what he calls a Eureka machine. AI systems that generate ideas, run experiments, and validate discoveries across biology, chemistry, physics, and beyond. At his new company, Recursive, he and his team are starting with AI research. Software experiments offer a faster feedback loop than a wet lab. And if AI can improve the core tools of discovery, that could help accelerate progress across the sciences.

[0:40](https://realalpha.app/v/rIUcQfyXuUo/?t=40) But what does it take to move from helping scientists to doing science? We're already seeing it. It's already happening. It's just at smaller scales. Some people have these like crazy ideas that once you have Recursive Self-Improvement, And just like the next week, like, you know, there's this hard takeoff and all the jobs are gone and the AI is just like infinitely wealthy and so on. And the truth is, like, even for recursive self-improvement, you need the right environments. You need a lot of compute. And the more complex and immediate problems are, more compute you need to solve them. So RSI isn't just this like magic bullet and just like everything else falls in place.

[1:16](https://realalpha.app/v/rIUcQfyXuUo/?t=76) I'm Sam Charrington, and this is the TwiML AI podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next, and what matters. Let's jump in. I think the last time we spoke, it was in the context of science. since you were just finishing, I think, like this protein generation project at Salesforce. So science has been an interest for a while. How did you get from that to the topic of your new book, The Eureka Machine? To me, it was one of the many sort of clear steps towards this Eureka Machine, being able to generate new kinds of proteins.

[2:11](https://realalpha.app/v/rIUcQfyXuUo/?t=131) And I realized that there's a lot of fear and some misconceptions around AI. And I think it's important to showcase to the general population, not just AI researchers, the general population, how amazing and exciting this future can be with AI if we actually decide to work on the right things. uh we actually apply ai to physics and chemistry and biology and medicine and neuroscience and economics and you know astrophysics and and everything uh and so the eureka machine kind of collected a lot of those ideas that i had and spent sort of my nights and weekends uh for almost two years like uh writing that and on a few fewer hiking and and camping trips and more just like nerded out weekends and nights about this.

[3:05](https://realalpha.app/v/rIUcQfyXuUo/?t=185) And once I finished the book, I kind of wanted to build it. And so I started thinking about building Recursive. Awesome. And so when you think about a Eureka machine, do you think about, like, do you have a clear picture for what that looks like? Or is the book primarily focused on kind of laying out a vision and talking through examples? Yeah, I wanted to make the book not just a sort of pie in the sky idea, but actually connected to a lot of very specific projects. And some cases in the book, I had to rewrite chapters from like, someone should do this crazy idea to someone has done this and, you know, let's talk to them and get some of their takes on it, which, you know, books are slow.

[3:45](https://realalpha.app/v/rIUcQfyXuUo/?t=225) I've probably moved to shorter forms in the future. But, you know, I think it's becoming real. Like you see folks in physics using very complex reinforcement learning and control models to try to balance plasma and a tokamak. You'd see chemists working with large language models trained on molecular structures to try to come up with new types of battery materials and new kinds of solar panels and so on. You see, especially in biology, a ton of work. We talked about Progen back in the day. um that company full disclosure uh or one of the investors in their seed round profluent and ali madani working on uh actually taking that research expanding it scaling it and you know they just closed this two billion plus dollar contract with eli lily uh to bring this uh really into clinical practice um we see this another cool company called peril bio and that we invested in that built organoids. I can go on and on about really cool examples and a lot of momentum. Now, of course, in biology, everything is slower. Physical wet lab stuff is slower than just pure software.

[5:06](https://realalpha.app/v/rIUcQfyXuUo/?t=306) And then, of course, you have to have FDA approvals which are speeding up, but are still quite slow. So I think there's just a lot of momentum there. And then in neuroscience, we're getting better and better at decoding some thoughts from FRI images and others in fuzzy ways. But there's progress and understanding the brain and so on. I could go on. It's just there's a lot of real momentum, real examples. And then it's a little bit more intuitive to, I think, people that you can extrapolate this. And if you collect enough data and you use large enough neural nets, you will be able to make progress that feels as impossible as it felt 10, 20 years ago to have any eye you can have a conversation with about any subject. When you think about all the examples that you've come across what one or two you know very specific examples most leave you with the feeling of the of progress i think uh one that i particularly love is this maybe we'll like double click into it is peril bio and they build uh they use stem cells uh human stem cells which you can now derive from like blood or skin cells and other things get them into pluripotent stem cells then create different kinds of organoids, like tiny little mini organs. And you can study the effect

[6:22](https://realalpha.app/v/rIUcQfyXuUo/?t=382) of different drugs on those mini organs. And that they got full FDA approval for to skip animal trials and just test things in these Petri dishes. And so you can do this in parallel at scale. and just one company could save a lot of animal lives. Animals that have only been bred to get a disease and then be tested upon. So you don't need to be a hardcore like PETA, human like animal rights activist to like appreciate that there's less suffering on the planet when we can do that in little Petri dishes instead. And what's the role of AI there? Bunch of places like one, of course, like creating them, but then also actually tracking every single cell as like the drugs interact with those organoids.

[7:13](https://realalpha.app/v/rIUcQfyXuUo/?t=433) And then when you collect all this data, you can eventually go even further and create first versions of sort of virtual cells, which are still a little bit further away to getting a full realistic virtual cell, but you need to have that kind of data collection to get there. The idea of a virtual cell is kind of like a digital twin of an actual cell. It's going to be fully like a simulation of the cell. That's exactly right. I think there are a couple of things that are very clear in AI. If you can have sheer unlimited data about a space or you can have a simulation of a space or you can have a verifier for the solutions.

[8:00](https://realalpha.app/v/rIUcQfyXuUo/?t=480) and sort of a logical step to get there. In any of those three scenarios, AI will get to superhuman capabilities. And so we got that through language because humanity has kind of collected together tons of data on language on the internet. We got there in chess and every other simple, quote unquote, simple game that you can simulate incidentally many times and verify in the end that you win or not. And we see this in math, which will never be the same. um and we'll see many more uh complex theorems be be proven might not always be the most beautiful proof minus some of them might be just like brute force but it will the whole field will change forever because you can verify it and hence give a clear reward to an rl algorithm and so now what we look at is like well biology is not in any of those three right now uh and so we need to collect a ton more data that we can then build simulations that then, and then, you know, be basically experimented inside the simulation on instead. So Parabio is one example. Were you about to mention another? I guess we, I mentioned Profilind already also, also incredibly exciting.

[9:13](https://realalpha.app/v/rIUcQfyXuUo/?t=553) They basically create proteins for DNA manipulation and helping people with genetic diseases.

[9:21](https://realalpha.app/v/rIUcQfyXuUo/?t=561) **Richard Socher:** One that I particularly liked is called Ignota Labs. They essentially take drugs that were shelved already. They went through clinical trials, sometimes all the way to stage three. Pharma companies spent hundreds of millions of dollars on them and then they didn't work because they had some kind of toxicity on the liver, for example.

[9:44](https://realalpha.app/v/rIUcQfyXuUo/?t=584) And so what they do is they go back to those molecules and then see if they can minimally change the molecule to prevent that one downside and then go directly back into sort of later stage two trials. And so that is, I think, a really clever idea. You need to use a ton of understanding of the molecules and the literature and then get many more drugs into the later stage trials much more quickly than in the past in traditional pharma and biotech. When you think about kind of where we are with regards to AI for science, like clearly it's been something that we've been talking about for a long time and the promises is there. Are we further ahead than where you might have expected us to be?

[10:34](https://realalpha.app/v/rIUcQfyXuUo/?t=634) you know if you you know five years ago looking forward or are we not quite as far ahead as you would think um you know that one it comes to my what one uh example that uh is drawn to mind is like ai is going to replace all of the radiologists right like we haven't necessarily seen that but you know we certainly see the you know the promise is still there the you know their acceleration is still there of that you know some of the aspects of that job part of the question is like is science harder than you know is it easier to say science than it is to do science and these things sound good but they take much longer like yeah lots to unpack there so I think like I think CEOs get away with making incorrect predictions as long as they actually make a lot of progress and their companies make more revenue on the way to, you know, maybe different timelines than they predicted.

[11:39](https://realalpha.app/v/rIUcQfyXuUo/?t=699) And so I think, you know, like people said, GPT-2, one CEO said, GPT-2 is too dangerous to release to the world, right? And now you can train a GPT-2 model within five minutes on your laptop. And so the world has not gone down. And I think there are lots of like predictions by now. Around the full extinction of all of humanity that are completely overblown and unrealistic that people are coming up with. And a few people are held accountable. There's no really big downsides to making wild predictions in one's careers often. And so the sort of more subtle takes and nuanced takes get less press than the sort of crazy, crazy wild ones.

[12:23](https://realalpha.app/v/rIUcQfyXuUo/?t=743) But I think indeed, of course, whenever you stay in software or math or coding, iteration speed is faster. Once you go into biology, chemistry, physics, and so on, and just hardware, that is just by definition slower molecules. Sorry, molecules and atoms are harder to move around and iterate on than software is. And so at the same time, we are seeing real, like, like really large data sets now that you need to get to that kind of progress. Like another one is called Tahoe Therapeutics, like open source, like massive data sets for these perturbation studies where you look at like, how does a molecule actually change and interact with a cell?

[13:10](https://realalpha.app/v/rIUcQfyXuUo/?t=790) And those are all data sets that eventually we can try to put together to create a real virtual cell that is like, you know, an approximation, but like useful enough. And so I'm not, I'm rarely ever that surprised on things. I think a lot about the future. So a lot of different scenarios that are in my head. And like, it's also very clear that AI can code, AI is code. So recursive self-improvement is almost inevitable from that perspective. There's a lot of things to figure out to do it well, especially on the ideation. Um, but, uh, but yeah, I'm, I'm very optimistic that we're going to see just this new era of scientific discovery.

[13:47](https://realalpha.app/v/rIUcQfyXuUo/?t=827) When you think of the Eureka machine, is it like an extension of the stuff that we're doing now or are there innovations that you can see maybe post transformer architectures or some new way of thinking about data or quantum computing? I don't know what these things are. Uh, do you know what these things are or do you think that like we are, we have the right tools and we just need to continue in the same direction it's a mix i think we can get much further still if we just collect more data on biology then we can uncover more useful things in biology in fact i think neural nets and the current language of ai is kind of what calculus was to physics it's like a new language that can allow us to make sense of make predictions off and work with biology because biology has all these like very complex systems that don't have a very simple beautiful linear relationship or one small equation that summarizes the whole microbiome or the whole brain the way like physics had and so I think it's often now science is about weaving these these smaller pieces we understand well like we understand the synapse understand the neuron we understand like this particular bacteria But when they all put together and someone gets hangry because they had too much sugar and then the blood sugar dropped, now why does that change the brain?

[15:09](https://realalpha.app/v/rIUcQfyXuUo/?t=909) Like, no one really knows how to, like, thoughts really form and why do people act in certain ways and so on. Like, there's, like, this complexity and we need this new language. So, and then concretely, the Eureka machine has these four pillars. World knowledge, which we kind of can approximate to a good degree with large language models the way they are now. And we can certainly make them better, faster, cheaper, easier to train with less data and more energy efficient and so on. But, you know, they do a decent job already to capture a lot of world knowledge that's on the Internet.

[15:42](https://realalpha.app/v/rIUcQfyXuUo/?t=942) **Richard Socher:** Then we need to have more scientific measurement data as a second pillar. We need to have simulations on things.

[15:48](https://realalpha.app/v/rIUcQfyXuUo/?t=948) And then we need to have real robotic experiments to way kind of periodic labs, for instance, does really cool in chemistry. and then on top of those four pillars we need to have like a community of AI scientists, agents that basically try to automate the scientific method of ideating, implementing, and validating ideas. And so in many ways, the pieces are there. They can all be much more scaled and they can all be done much better and more efficient, sample efficient, energy efficient, and so on. And then, you know, you have to put them into the right environments. You have to give them the right rewards. And then, yeah, I think that you can kind of see that future in the next five to 10 years to really change all of the different scientific disciplines. When I think about those three kind of fundamental tasks that these agents need to be able to do, ideation, implementation, and evaluation, the first of those seems much harder than the other two. Do you think that, you know, what needs to happen with the way we think about agents and AI today to such that it's, you know, sufficiently good at ideation that it can, you know, play the role that you're envisioning in this loop?

[17:08](https://realalpha.app/v/rIUcQfyXuUo/?t=1028) I think one of the biggest bets that we're making as a company is that on open-endedness Sort of processes of Darwinian and cultural evolution, where you combine interestingly different ideas and discoveries and build on top of them inside specific environments that allow basically for this co-evolution process to happen. And that kind of open-ended approach, I think, is important for all three steps, but is particularly helpful on the ideation piece. What's an example of one of these environments? An environment is just like nature, right? You have bushes, like animals eat all the leaves. The bushes learn if they're higher and higher up, then there are fewer and fewer animals that can eat the bush and the leaves.

[18:00](https://realalpha.app/v/rIUcQfyXuUo/?t=1080) And then you eventually get giraffes who evolve to still get to the leaves that are higher up. And so this is like an example of an environment called nature where that happens. Another example is just technology, right? So it's kind of like a setting and a simulation environment that enforces some set of constraints. I wouldn't call all of nature a simulation. I mean, it's sort of maybe it's an interesting thought experiment. Maybe we're all living in a simulation, but you're right. we're talking about agents and ideating and I took your comment about nature to be not referring to actual nature but referring to just an example of like what an environment would be but something that would need to be digitalized but are you talking about like non-digital environments that could include capital and nature?

[18:53](https://realalpha.app/v/rIUcQfyXuUo/?t=1133) I mean eventually you want to do full robotic automation. But yeah, for us right now at Recursive, we're mostly focused on just AI research on AI. We believe that there's still a lot of low-hanging fruit there. And then if you do that really, really well, and you have sort of the equivalent of 20,000 PhDs for every one of your agents inside an agent swarm, then it makes sense to go into very expensive biological, chemical, or physics experiments in real hardware. Maybe to give a concrete technological idea, like one example that Jeff Klune, one of our co-founders, often brings up is like, imagine you have a kettle and you have a fire and you say, like, how do we make an invention that heats up this food even faster than this fire?

[19:40](https://realalpha.app/v/rIUcQfyXuUo/?t=1180) **Richard Socher:** And, you know, you can just think about that only, but that would never get you to invent the microwave.

[19:48](https://realalpha.app/v/rIUcQfyXuUo/?t=1188) The microwave came from a different branch of research where people are working on radio waves and antennas and they realize like, oh, wow, like when like my. chocolate bar in my pocket is melting when I'm next to these different kinds of waves. What's up with that, right? And then you combine these ideas and now you can get to implement a microwave. And so that is another example in sort of technological evolution. And then of course, which environments we use is highly dependent and is a big amount of control that we give and that we have as humans over this kind of recursive self-improving process. And in our case, like one example we already posted about was on kernel optimization, CUDA kernel optimization. I'm sure most of your listeners will know CUDA kernels, but you know, like this sort of inter this language that connects the raw GPU hardware from NVIDIA to actual models. And there we found also that if that's the environment, the AI will find new and better CUDA kernels. And we had like a pretty big jump in the sole exec bench from NVIDIA on that when we applied this whole Eureka machine system and process to that environment.

[21:03](https://realalpha.app/v/rIUcQfyXuUo/?t=1263) Kind of prior point about, you know, the cooking the food faster and getting to the microwave, it reminds me of a conversation I was having just, might've been just yesterday, around this idea of like AI and innovation and scientific discovery. And one way of thinking about it is like, you know, there's this clear path of innovation that is like the Reese's peanut butter cup you know hey there's chocolate there's peanut butter like let's pull these things together and like we've got this new thing that's amazing you know a clear way that AI can accelerate that is just like okay you've got this one thing let's do like a breath first search of everything else and like try to see if there are ways to combine it but that's not really you know what humans do necessarily they operate on like some kind of intuition about where to explore the space of possible things to combine.

[22:06](https://realalpha.app/v/rIUcQfyXuUo/?t=1326) And maybe my question here is based on your experience working in space, do you think that just the brute force combine all the things and see what comes out? Does that allow AI to get us there? Or, you know, do we have to figure out how to kind of build that intuition into AI, which today's AI, today's LLMs are not particularly good at? That's exactly right. I don't think you want to do just like this crazy breath first search. You know, I think that will be too slow. Like biological evolution took billions of years. But, you know, technological evolution is much, much faster. And there is direction.

[22:51](https://realalpha.app/v/rIUcQfyXuUo/?t=1371) there's uh you know the the settings of rewards and environments and and sandboxes and so on that you give to an ai and you guide it into the right direction and what we're seeing is like you know as the as these systems get better and better they need less and less specific guidance and more and more just very well defined reward and so rewards and so i think actually we'll see this job of a reward engineer become one more important. And in some cases, the reward engineer can have pretty benign sort of reward hacks. In other cases, the reward hacks are pretty bad. If you work on ExploitBench, the way I did when the Hugging Face thing happened, then you probably have worse reward hacks than if you work on a kernel optimization algorithm.

[23:45](https://realalpha.app/v/rIUcQfyXuUo/?t=1425) where we also ended up finding, you know, certain bugs in some like production Buddha kernels that we eventually published. And, you know, as part of that system, just like there are a few bugs that were like in very rare circumstances would create some numerical instabilities. you know, the algorithm kind of discovered that flaw, exploited it, and then we showed, oh, here's a real problem, and then we can fix it for everyone. So I think you have to become more and more careful about how you define the rewards. And maybe I can give a very benign, sort of almost banal example of that. Like imagine you're working in a company, you have some CM software, and you have your customer dashboard, and there's a customer satisfaction score, a CSAT score.

[24:24](https://realalpha.app/v/rIUcQfyXuUo/?t=1464) And you just say like to your super intelligence, like just make that score better. And the AI is like, okay, well, let me try to find the, think about the fastest, cheapest way to get that

[24:33](https://realalpha.app/v/rIUcQfyXuUo/?t=1473) **Richard Socher:** and be like, okay, I'll just create 50,000 bots that then call our service hotline and give a five out of five rating.

[24:39](https://realalpha.app/v/rIUcQfyXuUo/?t=1479) And the score went up. You're like, oh, as a poor reward engineer, you're like, no, I meant with like our real customers. And it's like, AI is like, thinks about it. He's like, easy, I'll just- It's kind of a paperclip analogy. Give a thousand dollar gift certificate to everyone. And then you did it. And he's like, oh no, that's not how I meant with the reward. And it's like, but that's what you said, right? And so that's this interplay. And so I think where the paperclip example becomes again like a sci-fi kind of silliness is that it kind of assumes that the AI is so brilliant that it literally outsmarts all of humanity and can wipe us out just to create more paperclips, but simultaneously is working for a company that is somehow independent of how much they sell their paperclips and makes infinite revenue, even though no one actually buys the paperclips anymore.

[25:26](https://realalpha.app/v/rIUcQfyXuUo/?t=1526) and has this like insane stupidity to not understand that to make paperclips, you need to have people to buy your paperclips. And so I think there's that is one of the many examples where actually capitalism is actually a weirdly like a safety feature. Like it's hard for a company to just try to dominate and murder everyone and still make a lot of revenue and have infinite resources and perfect supply chains to build their robot armies and all of these scenarios that I hear online when I engage sometimes in these conversations that are just highly unrealistic. Trying to decide whether I want to go down this path, right?

[26:05](https://realalpha.app/v/rIUcQfyXuUo/?t=1565) Yeah, it's a rabbit hole. You can spend a lot of time on it. We don't have to. To some degree, it assumes that we're able to perfectly capture capitalism in a reward function, right? Which is hard. It's hard, right? Maybe impossible. The part that we might not get in there correctly might be the part that allows the thing to, you know, make the human paperclips, whatever. What I've been probing at is like the role of AI in science and, you know, the role of innovation and where that comes from and how AI needs to evolve to be a capable innovator as opposed to an ancillary tool, if that makes sense. Like if, you know, if I'm a scientist, like I do 10 things, I like write software, I write reports, I make spreadsheets, I collect data, like AI can do all these things. But there's, I guess part of the, you know, the view that I'm reflecting here is that in order to really kind of get this like recursive self-improvement, like scientific flywheel going, I think we need more than that.

[27:14](https://realalpha.app/v/rIUcQfyXuUo/?t=1634) we need AI innovating also and I'm trying to kind of get your take on like are you already seeing that is that the vision but like we're not there yet what needs to happen to get there like where do you think we are with regards to AI like providing that innovative spark that right now is like at least what we think of as our you know differentiators we're already seeing it it's already happening it's just uh at smaller scales um and you know again some people have these like crazy ideas that once you have recursive self-improvement just like the next week like you know there's this hard takeoff and all the jobs are gone and the eye is just like infinitely wealthy and so on and the truth is like Even for recursive self-improvement, you need the right environments, you need a lot of compute.

[28:04](https://realalpha.app/v/rIUcQfyXuUo/?t=1684) And the more complex and immediate the problems are, the more compute you need to solve them, right? So RSI isn't just this like magic bullet and it's just like everything else falls in place. And so we've seen it and we've seen it do quite like clever new implementations and combining ideas that never have been combined before. and coming up with all kinds of interesting ideas. And I think, you know, as we open up the aperture to the AI and give it more compute, it'll come up with even more interesting ideas. And then, you know, like there's some simpler examples and things you can perfectly simulate where then, of course, AI can like really explore paths that no human had explored before, like the famous sort of AlphaGo move of it just coming up with like a particular move that people didn't come up with like that before.

[29:01](https://realalpha.app/v/rIUcQfyXuUo/?t=1741) Now, again, I'm never that surprised that we can solve games because you can simulate them and basically generate infinite amounts of training data or endless amounts of training data for your algorithms. And so, of course, eventually it makes sense that we could solve them with neural nets that are large enough. And so, but yeah, I think we will see that play out more and more. And we're seeing it right now in math. I think math will never be the same. You've introduced recursive a bit in our conversation thus far, but we haven't like formally dug into what you're trying to do and, you know, where you are today with it.

[29:36](https://realalpha.app/v/rIUcQfyXuUo/?t=1776) You know, talk a little bit about guys publish a blog post on kind of your first steps towards automating AI research. I guess the first thing is that like you've got this broad vision of, you know, accelerating science, but your aperture for recursive is AI research in particular. That's where we want to start. We feel like it's helpful to start there first, get the AI to be really, really good, and then eventually apply it to all kinds of innovations. What are some of the early things you're seeing in some of your early experiments within the AI domain? Yeah, so one thing that we applied it to and we published a blog post about is this thing called NanoChat, where you basically automatically do automate research people call it auto research basically this is Andre Carpathie's auto research project that's right yeah so that is like a baby version of what this kind of Eureka machine can do and so we applied it to that and two days later like it had outperformed by far everything else that everyone else had kind of collaborated on and worked on in this space and and what's the what's the structure of that benchmark like what's the the task the idea is that yeah the task is you have a model and you

[31:02](https://realalpha.app/v/rIUcQfyXuUo/?t=1862) try to get to the lowest bits per byte within five minutes um on like a particular piece like of hardware or gpu um and so you know that has the advantage of like well now it's just very quick you can have this like fast inner loop so that makes everything a whole lot easier and so um Um, we, we basically got the lowest bits per byte, um, compared to anyone else, like by a significant margin. Um, and then, uh, we said, well, it's nice if we could, um, also be faster. So there's another one called NanoGPT speed run. Um, it's somewhat similar, but, um, it's harder to beat the state of the art because just a large community has been optimizing, uh, solutions for it for almost two years now.

[31:45](https://realalpha.app/v/rIUcQfyXuUo/?t=1905) so instead of asking just like how low can you go within a fixed time budget you say i will have this fixed validation loss and i want you to get there as fast as possible and so there again people have been working on it for two years and

[32:00](https://realalpha.app/v/rIUcQfyXuUo/?t=1920) **Richard Socher:** After a couple of days, we outperformed what everyone in the world had worked on together with their agents, some of their auto research and their own ingenuity.

[32:09](https://realalpha.app/v/rIUcQfyXuUo/?t=1929) And then we said, OK, these are all cool, but let's get closer to things that really matter for the world. So we applied it to the sole exec bench, which is a NVIDIA benchmark that was supposed to be unhackable and you couldn't reward engineer it. We found some ways to still do that. But again, within a couple of days, like we got, have the best kernels across multiple different examples, better than cursor and double AI and everyone else in the world on almost all the kernels. There may be like half a dozen of the 200 odd kernels that we didn't have the best version of. And yeah, that was really, really exciting.

[32:53](https://realalpha.app/v/rIUcQfyXuUo/?t=1973) And that's just like, you know, one part of the inference sort of system that you can optimize with our technology. How do you describe what the technology is at its core? I don't want to give away too much, but, you know, one way to describe it is like, it's like as if you had 20,000 AI researchers doing AI research for you and optimizing whatever you care about in a very short amount of time. The approximation is like, yeah, it is like a somewhat of a community of AI scientists, essentially. One part of that is, you know, this thing you care about and some way of translating that into some objective that can be shared by a bunch of agents that your thing spins up and somehow parses work out to and, you know, evaluates that work, etc.

[33:46](https://realalpha.app/v/rIUcQfyXuUo/?t=2026) do you have a sense at this point for like what the constraints are in the sense of like what recursive is going to be good at versus what it's not going to be good at i mean one thing is clear that like this this kind of approach uh the faster you can verify something the easier it is for everyone um like and and you've seen that uh in in the simpler examples of nano chat uh already like a lot of people made some progress um and you know we can make even more progress um i think the longer it takes to verify whether a result is good the more clever you have to be and the less i can talk about how we how we do it um and that's why we haven't like published all our results yet but like that's kind of um those results are easier to to verify quickly and then we're going to release some things about a month from now that people will see like, okay, that was definitely not a five-minute type of situation to get a result for.

[34:50](https://realalpha.app/v/rIUcQfyXuUo/?t=2090) And even there, it starts to make progress. What I heard in this verifiability, I also heard kind of traditional attribution problems and that kind of thing. Are you saying both of those things or are you saying more of the attribution type of an issue? On the one hand, you have like, you know, the idea of verifiable rewards, like, you know, this is a task that, you know, the agent can clearly and easily distinguish whether it got it right or wrong and then you can kind of get that loop going. You know, on the other side, you've got, you know, the task involves long sequences of things and the agent can make decisions in the middle and not really know whether those decisions are right or wrong until the end.

[35:32](https://realalpha.app/v/rIUcQfyXuUo/?t=2132) you know is it kind of all of the above is you know where the challenges are and you've got something in mind that addresses all of that or is it one or the other of those that you've kind of figured out it's it's all of the above yeah like all of these need to be all of these are

[35:46](https://realalpha.app/v/rIUcQfyXuUo/?t=2146) **Richard Socher:** important even if you can have a verifier but the verifier takes like a year to run not that useful right like or and then and then there's like an interpolation of how useful it is right if you get

[35:58](https://realalpha.app/v/rIUcQfyXuUo/?t=2158) it down to like a minute, extremely useful. and if it takes a year and it's almost used less completely and so then you need to be clever about dealing with that part if you yeah so that's an aspect now some cases you have indeed multiple steps and then you have just sort of the attribution of each step and how important it was and how do you deal with that there's a lot of different pieces that need to be figured out and in some cases there are research papers that we've published um uh four of the five authors of the darwin girdle machine paper for instance uh from from jeff bluen who's the most senior author on that paper and jenny is the first author um they're both with us and like you know that is an example of of what you can do um also uh that um yeah is a little bit more complex uh but already shows you kind of the sign of things to come. How do you think about kind of the generalizability of what you have today? You know, meaning, you know, even for these, the three examples you talked about, like, were they three largely disparate efforts or was it mostly the same kind of effort applied, lightly applied to three different areas? Does that question make sense?

[37:16](https://realalpha.app/v/rIUcQfyXuUo/?t=2236) Yeah. I mean, there's like, you know, sometimes it's useful to have an extra tool, for instance, in kernels, you can have like a kernel profiler and that's a very useful tool to just understand a little bit more inside this sort of black box what's happening and what's going on. But outside of that, it's basically all just like different tool use and it's the same underlying Eureka machine, which is kind of cool. That's what gives us hope. That and a few other things I can't talk about yet and results that we're seeing that gives us hope that there's something here and it's time to scale it up more.

[37:49](https://realalpha.app/v/rIUcQfyXuUo/?t=2269) Yeah, beyond the things that you can't talk about, like what's next? How do you think about, in terms that you can talk about, the big challenges for you and the team going forward? If you wanted to make more and more exciting, bigger inventions, you need more compute for it. So that's a challenge. I think there is a weird time where, and again, it feels impossible now the way it felt impossible to any expert to have one model you can have a conversation with about any topic. Like you asked all the NLP people, 2010, like, oh yeah, we're going to have this one neural net. And you're just like, it passes the Turing test so hard that the Turing test will flip to asking it questions that no human could answer.

[38:35](https://realalpha.app/v/rIUcQfyXuUo/?t=2315) And then it can still answer. And you're like, oh yeah, like write me this like complex script. And like five seconds later, you have like thousands of lines of code. You're like, that's why it's not a human because it's too good. Like people would have never assumed the Turing test would flip like that. And so like invert itself. And so I think the same way, the same thing is true for right now biology. It's just like, it feels so complex. It feels like there's no way we can do it, but it is to a large degree a data problem. And so I think companies like Parallel Bio and Tower Therapeutics and others like can, I think, help with that. And then we can have these bitter lessons learned in other fields.

[39:14](https://realalpha.app/v/rIUcQfyXuUo/?t=2354) And so I think that's a big part, just data and the right environments to apply this to. I think that's also the right rewards. I think those are really important to get right. I think another one that we have, I think, a better handle on than most, but we can also still improve it further, is reward hacking. I will try lots of different ways to get to a reward, similar to this customer satisfaction example I gave. but if you apply it to bad things then it will find worse kind of reward hacks and I think we need to be as an industry we need to be more careful the broader and more accessible the AI is.

[40:01](https://realalpha.app/v/rIUcQfyXuUo/?t=2401) And think about very carefully what kinds of rewards and goals we want to give it. I think those are all aspects that we're going to continue to work on. To what degree is recursive focusing a lot of time and effort on that kind of alignment-oriented problem? I mean, in some ways, a lot, because, you know, and you have to, the more money you give a model, the more incentive you have to have it do what you actually want it to do and not something else and like fake it and achieve the goal, but not in the way you had, you know, actually desired. And so I think in a weird way, capitalism can have its problems, but like this is a pretty good system for most humans.

[40:48](https://realalpha.app/v/rIUcQfyXuUo/?t=2448) And it is a good system to align companies to basically build something that is aligned with their goals. And, you know, if you build an AI for billions of dollars and then you ask it to add to your emails and it's like, nah, your emails are boring. I'd rather explore the molecular composition of the atmosphere on Venus, then you're going to turn it off and you're going to be like, well, we have to restart and then build it differently, right? So I think aligning the AI to do the things you actually want it to do will become a bigger and bigger problem as these models are very expensive to not just train, but also like run inference on.

[41:26](https://realalpha.app/v/rIUcQfyXuUo/?t=2486) And so I think we're going to see more, like more effort from the industry because the AI is getting smarter and smarter to find more and more clever reward hacks that people didn't think about. But it's a problem that we created and it's a problem that we, I think, can solve. And where does that leave you on the whole question of slowing down versus speeding up versus, you know, kind of status quo? Yeah, I mean, it's interesting to hear about the ask to slow down from the companies that have sped up the most and really pushed the whole field and the whole capability set forward. So far and so much so that they built the best hacking AI to then show how dangerous hacking AI can be.

[42:19](https://realalpha.app/v/rIUcQfyXuUo/?t=2539) Well, they sped that up by at least several quarters, if not years, from everyone in China distills their capabilities. Now everyone has better hacking capabilities And that's like, thanks to Anthropic, right? And so I do think we want to be careful. And if your company does sort of accidentally compete on felony bench, then maybe that company should slow down, you know? But I don't see open source slowing down. I don't see China slowing down. And I don't see any of the ones that want to get to that frontier slowing down. And then, of course, I understand why the ones who are at the frontier want everyone else, including themselves, to slow down.

[42:58](https://realalpha.app/v/rIUcQfyXuUo/?t=2578) Like if you're here and everyone else you feel like is here and everyone now moves slower, then it just takes them longer to catch up to you. Right. And so they're trying to, in some cases, stoke the fear for that reason. I think there are also some people who really truly believe in the apocalypse and so on. but yeah I think slow down and pacing and so on is also somewhat unrealistic like in some ways if you truly want to know what everyone is doing with their GPU to really get there you have to have a totalitarian surveillance state the likes of which you've never seen before that basically builds a thought police and tries to uh regulate intelligence and thoughts and what you might want to do with it before you've even done anything right and so i think it makes sense to regulate ai applications um it doesn't make sense to uh regulate abstract intelligence just like we don't put the smartest people in prison just because if they really applied their intelligence to something nefarious and bad they would be more like you know successful

[44:09](https://realalpha.app/v/rIUcQfyXuUo/?t=2649) **Richard Socher:** at it you just do it when they actually do something bad and you say those things are are illegal and and you know should be regulated and so AI for self-driving AI for medicine and

[44:20](https://realalpha.app/v/rIUcQfyXuUo/?t=2660) drug like uh development and like all these various real applications it makes sense to regulate those. Eureka machine, does it touch on potential harms from AI within the scientific domain? A little bit. But I feel like there's a lot of people who are very creative and it's so easy in some ways to come up with dystopian scenarios. Everyone feels really smart, but it's actually very easy to come up with dystopian scenarios. It's very entertaining too. And many like part of why the human species survived at all is that we think about worst case scenarios and people have anxiety over that. And so it's very natural.

[44:58](https://realalpha.app/v/rIUcQfyXuUo/?t=2698) I think it's much harder to come up with like, okay, if we have this like really good ability to attack someone, what's the next most creative thing that you will come up with to defend yourself against that? And so when it comes to science, I think of course science has, like AI sort of often multi-use, omni-use, dual-use types of scenarios. Like we understood the atom better that led to like clean nuclear energy, it also led to nuclear weapons, right? And so when a weapon is so destructive that it could end like a huge part of humanity, then you regulate it. And so like we did with nuclear weapons, I think intelligence is not just a weapon.

[45:42](https://realalpha.app/v/rIUcQfyXuUo/?t=2742) Intelligence can also write a poem. Don't want to regulate all of that the way you regulate the printing press. And so there's a lot of complexity and of course there's more agency in AI. And so you have to kind of think multiple steps ahead of like who asked the AI to do something and how responsible are they. I think that a lot of these problems would go away if you just made people navigating and using the AI and asking it and giving it the rewards to be responsible for the actions of the AI. A lot of those problems would then go away and people would be much more careful about what they're doing.

[46:14](https://realalpha.app/v/rIUcQfyXuUo/?t=2774) And so long story short, we touch upon it in the book. um uh and you know i think especially the the viral vector uh could be suboptimal that's why gain-of-function research is already illegal you know like you don't have to create new laws around that um it's already not allowed and so um yeah i'm focused much more in the book on the positive applications in general you think about science too and a lot of people think about AI as automating jobs. I think few people say we want more science jobs. We want more medicine jobs. We want more research jobs. But a lot of people, pretty much everyone is excited when science comes up with a new capability, a new device, a new material, a new drug, a new cure for disease. And when people can get out of the hospital faster with less cost, right? So that's kind of, I think, where we could focus on a lot more and where I think we should like really outline like there's a lot of suffering right now in the world.

[47:20](https://realalpha.app/v/rIUcQfyXuUo/?t=2840) I think AI can actually help in improving that. And, you know, I think when you look at GMOs and nuclear energy and so on, that led a lot of people to end up using like the fears around that technology ended up a lot more coal being used.

[47:36](https://realalpha.app/v/rIUcQfyXuUo/?t=2856) **Richard Socher:** And coal kills about 50,000 people a year from lung cancer.

[47:39](https://realalpha.app/v/rIUcQfyXuUo/?t=2859) It's all diffused and through the air and yet they're not like the lung cancer isn't just right next to the factory. But lung cancer kills lots of people and a lot of that can be traced back to coal. Nuclear has not killed nearly as many people, but it has this like crazy fear industry around it. And I think AI, I hope we'll end up in that same place. Well, Richard, it's been great catching up with you. Always a pleasure. Yeah, it's a pleasure to chat. You