AI researchers helped build Recursive's first coding model

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 0:32

   Thank you, and thanks for having me. Absolutely. So tell us about Flint. What is it, and who is it for, and how does it differ from other models? Flint is the best and fastest local coding model that you can actually download. It's fully open source. It's better than any of the other models in the category, pretty much all of which are Chinese. So it's also one of the first American-made open source models that's sort of on the Pareto frontier and beyond.

2. 1:03

   We think for us, it's, you know, a great first step, get into the habit of shipping. I know these things usually take companies a lot longer as they get started. And, you know, we used AI and a very small but very strong team of AI researchers to actually develop this model. So the AI did a lot of the work for us. Interesting. Can you go into a little more detail about that? How is the AI helping improve the AI?

3. 1:31

   Yeah. So a big part of recursive is building this Eureka machine. We call it the Eureka machine, somewhat aligned with my book, but focused first exclusively on AI research. And the idea is that inside this Eureka machine, you have a community of AI scientists that essentially ideate, implement and validate ideas in an open ended fashion. Open-endedness is sort of this idea that's inspired by biological, cultural, and technological evolution to combine interestingly different ideas, allow some branches of evolution to go on for a while, and then recombine when they come up with interestingly different ideas.

4. 2:16

   And that is what we've done here. So pretty much every aspect of this model, AI helped us come up with new ideas, implement them, and then validate them. And that's something that we think no other company has been able to automate to that degree. So you said coming up with ideas, implementing them, and then validating them. Which of these steps right now is easiest for AIs to do and which is hardest?

5. 2:44

   I think almost every company, when they talk about recursive self-improvement, say, oh, we're doing a lot of it because our developers are using AI to implement ideas. So that one, I think, is most consensus. I think how good you can make AI at coming up with really novel ideas is probably the least consensus. But then also truly validating things is non-trivial. So there's like it's sort of people don't think about it as much to make it either consensus or non-consensus.

6. 3:13

   But there's also a lot of complexity there. You know, people know that as these models get more intelligent, they often try to solve problems based on the letter of your prompt, not the meaning and the underlying intent of it. And so we think we have a better handle on some of those problems as well. hmm what will it take for ai to actually become as good at ideation as like the best human ai researchers so yeah there's there's a lot to unpack there i think at a high level people underestimate uh the crazy complex combinatorial space um of already existing ideas If you just combine all the concepts and all the ideas of physics, chemistry, biology, like computer science and, you know, theoretical computer science and, you know, like complexity theory and systems and operating systems, all of these different things, like there's actually a lot optimization.

7. 4:19

   and so on. There's a lot there that you can already kind of play around with. And similar to chess, you have to not just like sort of just try them all out because they're combinatorially too many, but you have to build an intuition of which of these idea combinations is the most viable. And so, yeah, I think this combinatorial space is really underexplored. I think the next level uh is to have like even completely new concepts and to be honest like even there not many human researchers are uh able and capable of doing that and in many cases uh the next invention is kind of an incremental step across um uh you know existing ideas uh even like let's look at the transformer right the transformer the title of the paper is attention is all you need Why? It's not because attention was invented in the transformer paper.

8. 5:15

   In my papers, lots of other people had used various levels of attention before. They learned, and the big insight was you don't need any recurrent neural network layers. So sometimes even deleting an idea is novelty in and of itself, right? To simplify things, it can be very beautiful and then create more generality. So there's a lot to unpack there, but I have no doubt and we're seeing already that AI is getting better and better and better at ideating.