# Karpathy uses autocomplete for novel code and agents for boilerplate

Markdown copy of https://realalpha.app/s/karpathy-uses-autocomplete-for-novel-code-and-agents-for/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Andrej Karpathy](https://realalpha.app/builders/andrej-karpathy/) · Andrej Karpathy is an AI researcher and educator building Eureka. · Joined Anthropic's pretraining team; founded Eureka Labs, an AI-native school, after leading Tesla's Autopilot AI.
- Host: [Dwarkesh Patel](https://realalpha.app/people/dwarkesh-patel/) · Host, Dwarkesh Podcast · Co-wrote The Scaling Era, an oral history of AI; named to TIME100 AI.
- Show: Dwarkesh Podcast
- Episode: [Andrej Karpathy — “We’re summoning ghosts, not building animals”](https://realalpha.app/v/lXUZvyajciY/), Oct 17, 2025
- Moment: 30:27 to 35:23 of the episode (5 min), playing on https://realalpha.app/s/karpathy-uses-autocomplete-for-novel-code-and-agents-for/; the episode from there: https://realalpha.app/v/lXUZvyajciY/?t=1827

## Quote

> “they keep misunderstanding the code because they have **too much memory** from all the typical ways of doing things”
> — [Andrej Karpathy](https://realalpha.app/builders/andrej-karpathy/) at [32:14](https://realalpha.app/v/lXUZvyajciY/?t=1934) · Dwarkesh Podcast

## Takeaways

1. Autocomplete lets Karpathy stay in control of novel code.
2. Agents suit boilerplate that has many examples in training data.
3. He uses agents for unfamiliar Rust code when tests can check it.

## About this segment

He explains why coding agents struggled with NanoChat's custom training code, while autocomplete fit his workflow. He uses agents for boilerplate and for Rust tasks he can verify.

From “Andrej Karpathy — “We’re summoning ghosts, not building animals””, Oct 17, 2025. [Watch the full episode](https://realalpha.app/v/lXUZvyajciY/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[30:32](https://realalpha.app/v/lXUZvyajciY/?t=1832) You tweeted out that coding models were actually of very little help to you in assembling this repository. And I'm curious why that was. Yeah. So the repository, I guess I built it over a period of a bit more than a month. And I would say there's like three major classes of how people interact with code right now. Some people completely reject all of LLMs and they are just writing by scratch. I think this is probably not the right thing to do anymore.

[30:57](https://realalpha.app/v/lXUZvyajciY/?t=1857) The intermediate part, which is where I am, is you still write a lot of things from scratch, but you use the autocomplete that's basically available now from these models. So when you start writing out a little piece of it, it will autocomplete for you and you can just tap through and most of the time it's correct. Sometimes it's not and you edit it. But you're still very much the architect of what you're writing.

[31:17](https://realalpha.app/v/lXUZvyajciY/?t=1877) And then there's the, you know, Vibe coding. You know, hi, please implement this or that, you know, enter, and then let the model do it. And that's the agents. I do feel like the agents work in very specific settings, and I would use them in specific settings. But again, these are all tools available to you, and you have to, like, learn what they're good at and what they're not good at and when to use them.

[31:38](https://realalpha.app/v/lXUZvyajciY/?t=1898) So the agents are actually pretty good, for example, if you're doing boilerplate stuff. Boilerplate code, that's like just copy-paste stuff. They're very good at that. They're very good at stuff that occurs very often in the internet because there's lots of examples of it in the training sets of these models. So there's like features of things where the models will do very well. I would say NanoChat is not an example of this because it's a fairly unique repository.

[32:02](https://realalpha.app/v/lXUZvyajciY/?t=1922) There's not that much code, I think, in the way that I've structured it. And it's not boilerplate code. It's like actually like intellectually intense code almost. and everything has to be very precisely arranged. And the models were always trying to, they kept trying to, I mean, they have so many cognitive deficits, right? So one example, they keep trying to, they keep misunderstanding the code because they have too much memory from all the typical ways of doing things on the internet that I just wasn't adopting.

[32:27](https://realalpha.app/v/lXUZvyajciY/?t=1947) So the models, for example, I mean, I don't know if I want to get into the full details, but they keep thinking I'm writing normal code and I'm not. Maybe one example. Maybe one example is, so the way to synchronize, So we have eight GPUs that are all doing forward backwards. The way to synchronize gradients between them is to use a distributed data parallel container of PyTorch, which automatically...

[32:48](https://realalpha.app/v/lXUZvyajciY/?t=1968) It does all the, as you're doing the backward, it will start communicating and synchronizing gradients. I didn't use DDP because I didn't want to use it because it's not necessary. So I threw it out and I basically wrote my own synchronization routine that's inside the step of the optimizer. And so the models were trying to get me to use the DDP container. And they were very concerned about, okay, this gets way too technical, but I wasn't using that container because I don't need it.

[33:11](https://realalpha.app/v/lXUZvyajciY/?t=1991) And I have a custom implementation of something like it. And they just couldn't internalize that you had your own. Yeah, they couldn't get past that. And then they kept trying to like mess up the style. Like they're way too over-defensive. They make all these try-catch statements. They keep trying to make a production code base. And I have a bunch of assumptions in my code and it's okay. And it's just like, I don't need all this extra stuff in there.

[33:33](https://realalpha.app/v/lXUZvyajciY/?t=2013) And so I just kind of feel like they're bloating the code base. They're bloating the complexity. They keep misunderstanding. They're using deprecated APIs a bunch of times. So it's total mess. And it's just not net useful. I can go in, I can clean it up, but it's not that useful. I also feel like it's kind of annoying to have to like type out what I want in English because it's just too much typing.

[33:53](https://realalpha.app/v/lXUZvyajciY/?t=2033) Like if I just navigate to the part of the code that I want and I go where I know the code has to appear and I start typing out the first three letters, Autocomplete gets it and just gives you the code. And so I think this is a very high information bandwidth to specify what you want. If you point to the code where you want it and you type out the first few pieces and the model will complete it. So I guess what I mean is I think these models are good in certain parts of the stack, I actually use the models a little bit in, there are two examples where I actually use the models that I think are illustrative.

[34:23](https://realalpha.app/v/lXUZvyajciY/?t=2063) One was when I generated the report, that's actually more boilerplate-y. So I actually vibe-coded partially some of that stuff. That was fine. Because it's not like mission-critical stuff and it works fine. And then the other part is when I was rewriting the tokenizer in Rust, I'm actually not as good at Rust because I'm fairly new to Rust. So I was doing, there's a bit of vibe-coding going on when I was writing some of the Rust code.

[34:45](https://realalpha.app/v/lXUZvyajciY/?t=2085) But I had Python implementation that I fully understand and I'm just making sure I'm making a more efficient version of it and I have tests. So I feel safer doing that stuff. And so basically they lower or like they increase accessibility to languages or paradigms that you might not be as familiar with. So I think they're very helpful there as well. Because there's a ton of Rust code out there. The models are actually pretty good at it.

[35:07](https://realalpha.app/v/lXUZvyajciY/?t=2107) I happen to not know that much about it. So the models are very useful there. The reason I think this question is so interesting is because the main story people have about AI exploding and getting to superintelligence pretty rapidly