# Human-generated data is running short, so AI needs synthetic data

Markdown copy of https://realalpha.app/s/human-generated-data-is-running-short-so-ai-needs-synthetic/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Elon Musk](https://realalpha.app/builders/elon-musk/) · Elon Musk is a technology entrepreneur. · Co-founded PayPal, founded SpaceX and xAI, and has been Tesla's CEO since 2008.
- Host: [Garry Tan](https://realalpha.app/people/garry-tan/) · President & CEO, Y Combinator · Co-founded Posterous and Initialized Capital, an early Coinbase backer.
- Host: [Harj Taggar](https://realalpha.app/people/harj-taggar/) · Managing partner, Y Combinator · Co-founded Auctomatic with the Collison brothers and later Triplebyte.
- Host: [Diana Hu](https://realalpha.app/people/diana-hu/) · Managing partner, Y Combinator · CTO of Escher Reality; after Niantic bought it, ran AR for Pokémon GO.
- Host: [Jared Friedman](https://realalpha.app/people/jared-friedman/) · Managing partner, Y Combinator · Co-founded Scribd, which YC funded in 2006; has advised 20+ YC unicorns.
- Show: Lightcone
- Episode: [Elon Musk: Digital Superintelligence, Multiplanetary Life, How to Be Useful](https://realalpha.app/v/cFIlta1GkiE/), Jun 19, 2025
- Moment: 31:56 to 34:30 of the episode (3 min), playing on https://realalpha.app/s/human-generated-data-is-running-short-so-ai-needs-synthetic/; the episode from there: https://realalpha.app/v/cFIlta1GkiE/?t=1916

## Quote

> “we're at the stage where there's **more effort put into synthetic data**”
> — [Elon Musk](https://realalpha.app/builders/elon-musk/) at [33:28](https://realalpha.app/v/cFIlta1GkiE/?t=2008) · Lightcone

## Takeaways

1. High-quality human-generated tokens run out quickly.
2. Synthetic data needs accurate checks against reality.
3. Musk says hard-science material can help train reasoning.

## About this segment

Elon Musk says high-quality human-generated training data is scarce, making synthetic data increasingly important. The challenge is checking that it is grounded in reality rather than hallucinated.

From “Elon Musk: Digital Superintelligence, Multiplanetary Life, How to Be Useful”, Jun 19, 2025. [Watch the full episode](https://realalpha.app/v/cFIlta1GkiE/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[32:09](https://realalpha.app/v/cFIlta1GkiE/?t=1929) Well, there's other various elements that decide competitiveness for large AI. There's for sure the talent of the people matter. The scale of the hardware matters and how well you're able to bring that hardware to bear. so you can't just order a whole bunch of gpus and if they don't then you can't just plug them in so you've got to you've got to get a lot of gpus and have them um trained trained coherently and stably um then it's like what unique access to data do you have i guess distribution matters to some degree as well like how do people get exposed to your ai those are those are critical factors for if it's going to be like a large foundation model that's competitive um You know, as, as many have said, I, my friend Elias has said, you know, we've kind of run out of pre training data for human generated pre, like human generated data, you run out

[33:09](https://realalpha.app/v/cFIlta1GkiE/?t=1989) of tokens pretty fast, certainly of high quality tokens. And, and then you and you have to do a lot of you need to essentially create synthetic data um and and be able to accurately judge the synthetic data that you're creating to verify like is this real synthetic data or is it an hallucination that doesn't actually match reality um so achieving grounding in reality is is is tricky but but we're at the stage where there's more effort put into synthetic data um and like right now we're training grok 3.5 which is a heavy focus on reasoning going back to your physics point uh what i heard for reasoning is that uh hard science particularly physics textbooks are very useful for reasoning whereas um i think researchers have told me that uh social science is totally useless for reasoning uh yes that's probably true um so yeah um there's something that's going to be very