# Neural nets gradually replaced Autopilot's hand-written code

Markdown copy of https://realalpha.app/s/neural-nets-gradually-replaced-autopilots-hand-written-code/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Andrej Karpathy](https://realalpha.app/builders/andrej-karpathy/) · Andrej Karpathy is a former director of AI at Tesla; worked at Stanford and OpenAI. · Joined Anthropic's pretraining team; founded Eureka Labs, an AI-native school, after leading Tesla's Autopilot AI.
- Host: [Garry Tan](https://realalpha.app/people/garry-tan/) · President & CEO, Y Combinator · Co-founded Posterous and Initialized Capital, an early Coinbase backer.
- Host: [Harj Taggar](https://realalpha.app/people/harj-taggar/) · Managing partner, Y Combinator · Co-founded Auctomatic with the Collison brothers and later Triplebyte.
- Host: [Diana Hu](https://realalpha.app/people/diana-hu/) · Managing partner, Y Combinator · CTO of Escher Reality; after Niantic bought it, ran AR for Pokémon GO.
- Host: [Jared Friedman](https://realalpha.app/people/jared-friedman/) · Managing partner, Y Combinator · Co-founded Scribd, which YC funded in 2006; has advised 20+ YC unicorns.
- Show: Lightcone
- Episode: [Andrej Karpathy: Software Is Changing (Again)](https://realalpha.app/v/LCEmiRjPEtQ/), Jun 19, 2025
- Moment: 3:51 to 6:24 of the episode (3 min), playing on https://realalpha.app/s/neural-nets-gradually-replaced-autopilots-hand-written-code/; the episode from there: https://realalpha.app/v/LCEmiRjPEtQ/?t=231

## Quote

> “so the software 2.0 stack quite literally **ate through the software stack** of the autopilot.”
> — [Andrej Karpathy](https://realalpha.app/builders/andrej-karpathy/) at [5:18](https://realalpha.app/v/LCEmiRjPEtQ/?t=318) · Lightcone

## Takeaways

1. Autopilot shifted camera and timing tasks into neural nets.
2. As the networks improved, some C++ code was removed.
3. Choose between explicit code, trained models and prompts.

## About this segment

At Tesla, neural nets took on cross-camera and temporal work that had been hand-coded in C++. Andrej Karpathy says developers now need to choose among code, trained weights and prompts.

From “Andrej Karpathy: Software Is Changing (Again)”, Jun 19, 2025. [Watch the full episode](https://realalpha.app/v/LCEmiRjPEtQ/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[3:59](https://realalpha.app/v/LCEmiRjPEtQ/?t=239) GitHub code is not just like code anymore there's a bunch of like English interspersed with code and so I think kind of there's a growing category of new kind of code so not only is it a new programming paradigm it's also remarkable to me that it's in our native language of English and so when this blew my mind a few I guess years ago now I tweeted this and I think it captured the attention of a lot of people and this is my currently pinned tweet is that remarkably we're now programming computers in English now when I was at Tesla we were working on the autopilot and we were trying to get the car to drive.

[4:36](https://realalpha.app/v/LCEmiRjPEtQ/?t=276) And I sort of showed the slide at the time where you can imagine that the inputs to the car are on the bottom and they're going through a software stack to produce the steering and acceleration. And I made the observation at the time that there was a ton of C++ code around in the autopilot, which was the software 1.0 code, and then there was some neural nets in there doing image recognition. And I kind of observed that over time as we made the autopilot better, basically the neural network grew in capability and size, and in addition to that, all the C++ code was being deleted and kind of like was, and a lot of the kind of capabilities and functionality that was originally written in 1.0 was migrated to 2.0.

[5:17](https://realalpha.app/v/LCEmiRjPEtQ/?t=317) So as an example, a lot of the stitching up of information across images from the different cameras and across time was done by a neural network, and we were able to delete a lot of code. And so the software 2.0 stack quite literally ate through the software stack of the autopilot. So I thought this was really remarkable at the time. And I think we're seeing the same thing again, where basically we have a new kind of software, and it's eating through the stack.

[5:41](https://realalpha.app/v/LCEmiRjPEtQ/?t=341) We have three completely different programming paradigms, and I think if you're entering the industry, It's a very good idea to be fluent in all of them because they all have slight pros and cons, and you may want to program some functionality in 1.0 or 2.0 or 3.0. Are you going to train a neural net? Are you going to just prompt an LLM? Should this be a piece of code that's explicit, etc.? So we all have to make these decisions and actually potentially fluidly transition between these paradigms.

[6:06](https://realalpha.app/v/LCEmiRjPEtQ/?t=366) So what I want to get into now is first I want to, in the first part, talk about LLMs and how to kind of like think of this new paradigm and the ecosystem and what that looks like. What is this new computer?