Free with a sign-in.
The best 18 min of 59
The Startup Building World Models From a Billion Video Game Clips
Pim de Witte argues that diverse gameplay data can pretrain agents for games and robotics, with targeted real-world fine-tuning.
NPNewcomer Pod · Eric Newcomer
- Pim de Witte is CEO of General Intuition.
The best minutes
- 17:04Game data can pretrain robots, but real-world tuning still matters9 min
- 32:42Specialized robots may beat humanoids in homes4 min
- 42:26Build scientific skills alongside AI literacy5 min
Also worth watching
- 36:10Medal matches game skills to paid AI-training work6 min
- 12:28A mobile game can have sound mechanics and still stall3 min
- 55:34Claude found an S3 key during reverse engineering3 min
- 27:28A top game using General Intuition models could arrive next year3 min
- 5:55Factorio and RuneScape are strong hiring signals3 min
Skip the opening RuneScape history and the long debate on AI demand, Chinese models, and distillation before the coding-agent security anecdote.
Full analysis · 6 min read
Real Alpha · Nov 8
Pim de Witte says roughly a billion uploads could aid robot training
- Medal gets roughly a billion video uploads yearly, and some recordings carry captured controller actions.
- De Witte says game pretraining can cut robot tuning to minutes, but real-world data remains necessary.
- The test is whether action data transfers safely across tasks, not whether the upload count sounds large.
Free with an account
Read the rest of the analysis
6 sections, the 3 other voices it tests Witte against, and what it means for you. Every episode we cut gets one.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Signed up before? The same buttons sign you in. The episode, its moments and the transcript stay free.
A billion uploads are not a billion action examples
Medal receives roughly a billion video uploads a year, De Witte said. He said some recordings capture controller movements and button presses through features such as controller overlays.
That distinction matters because an uploaded video is not automatically an action-labeled example. General Intuition uses captured relationships between frames and inputs, according to De Witte.
His training task resembles language-model next-token prediction, except the output is an action after observed frames. That setup could suit agents that must act inside changing visual environments.
But he answered, “Yeah, partially,” when asked whether General Intuition is built on Medal recordings. The episode gives no share of uploads with usable actions or breakdown by game and recording quality.
De Witte leads General Intuition and built Medal, giving him a direct stake in this data thesis. That stake does not disprove the approach, but makes independent transfer results important.
“People upload roughly a billion videos a year.”
Our read
The distinctive asset is video linked to action, not a billion uploads as an abstract number. Buyers need evidence that the pairing improves transfer on tasks they care about.
Game pretraining may shorten, not erase, robot tuning
De Witte said models pre-trained mostly on game data transfer to novel robotics tasks using minutes worth of fine-tuning data. He said physical robots currently run on the foundation models.
That is a consequential claim, but the interview gives no task list, sample counts, baseline or success measure. Those details separate repeatable transfer from a selected demonstration.
The company’s framing is pretraining, not a replacement for real-world data. De Witte said both game and physical data remain necessary.
That qualification matters because a model’s starting knowledge and its final task behavior are different deliverables. Robotics teams need to budget for both, not count game clips as field validation.
In October 2026, Sergey Levine said on Minus One that robots could eventually generate physical-world data. He described human interventions correcting errors before reinforcement learning improves later behavior.
That is another data loop, not evidence that game pretraining alone transfers. Also in October 2026 on Minus One, Levine emphasized annotations, task planning and visual reasoning for generalization.
His point makes the training recipe part of the test, beyond corpus size. In November 2026 on MTS, Josh Gruenstein said simulation can scale quickly, but real robots validate it.
That offers a useful comparison, though Gruenstein’s point concerns simulation rather than Medal recordings.
“the end of the day you just need both”
Sergey Levine · Minus One · Oct 22General-purpose robots can bootstrap their own data- Sergey Levine · Minus One · Oct 22Reasoning and better annotations help robots generalize
- JGJosh Gruenstein · MTS · Nov 3New models make robotics simulation more scalable
Our read
The case is strongest as a pretraining argument, not a claim that games substitute for robotics data. The unresolved question is how consistently those minutes of tuning work across tasks.
Clip diversity does not prove real-world coverage
De Witte said clips uploaded with car-crash labels surpass daily physical car crashes in the United States. That is his reported label count, not a claim that the clips represent unique crashes or matched conditions.
The interview provides no labeling method, duplicate rate or definition of those labels. Without those, the comparison signals volume but cannot establish coverage of a target robot’s operating environment.
For example, De Witte said varied game bridges may teach shared structure better than bridges in one city. But that is a hypothesis about representation, not a measured result in this interview.
He also said different games teach different strengths, from spatial reasoning to interface use. Yet physics, pixels and rules can diverge from the deployment environment.
De Witte named UK and American driving behavior as one reason fine-tuning must reflect the destination. Game data can widen pretraining, while task-specific tuning must close those gaps.
Our read
The crash comparison supports a case for varied training material, not a safety or coverage guarantee. Buyers should ask what the data represents and where it differs from their target environment.
Safety still depends on controls outside the model
De Witte called safety a zero-tolerance problem after discussing Cruise’s accident and response. He said partners pair model-predicted actions with deterministic or sensor-based systems, including LiDAR.
Those rule-based systems may invalidate actions if they come too close or might put something at harm. The model therefore sits inside a larger control stack, rather than carrying the safety case alone.
Training on negative events can help a model predict consequences, De Witte said. But recognizing a harmful outcome does not guarantee the model will prevent one during deployment.
He also said the company uses evaluations and benchmarks, but offered no results in the interview. That missing evidence matters more for physical systems, where failures can have consequences beyond a bad game.
Teams should ask who can stop an action, how often safeguards trigger, and what happens when sensors disagree. Testing the model alone would miss the system’s other safety dependencies.
“you have to be on top of this stuff”
Our read
Game data may help models anticipate consequences, but it does not provide a complete safety system. Buyers should evaluate the model, its guardrails and its failure behavior together.
De Witte expects practical robots to beat home humanoids
De Witte said humanoids are further out in homes than people think, but closer in factories than people think. This is his forecast about expectations, not a claim that factory humanoids are already broadly practical.
He said training humanoid policies is harder than training systems steered through game controllers, such as robotic arms. He also described humanoids as bulky and heavy, with leg stabilization adding difficulty.
Those are separate reasons for his caution, not proof that a specific alternative will win. De Witte favors wheeled robots with cameras and arms for some household uses.
He said wheels offer a stable platform, while simpler hardware can save energy and space. For instance, a robot built for one task may not need walking legs or a human form.
In October 2026 on Training Data, Josh Gao also said constrained jobs would come before household robots. He described laundry folding as difficult and emphasized safety diligence inside homes.
That aligns on deployment sequence, not on which design or timeline will prevail. Teams should compare the required motions, space, cost and safety checks before choosing a form factor.
“humanoids are further out in the home than people think but closer in the factory than people think”
Our read
De Witte is making a forecast, not reporting a settled market outcome. Teams can test it against real task requirements before committing to a general-purpose design.
Data permissions and tool access need explicit controls
De Witte said he wants users whose data enters models to have ways to participate in the resulting value. He gave no consent flow, revenue share, opt-out or user-control mechanism in the interview.
That leaves an operational diligence question separate from model accuracy. Ask what rights cover source recordings, how they were obtained and whether permitted uses can change.
An employee example shows another access boundary. An intern used Claude to inspect an app while waiting for an SDK, De Witte said.
The model found a secret S3 bucket access key and retrieved documentation. The intern did not realize how Claude had obtained it.
De Witte said he disclosed the vulnerability, and no harm was done. His larger warning was that workers can trigger security incidents without noticing.
That makes tool permissions, credential isolation and review of agent activity practical controls, not abstract policy.
Our read
Data access is both the product’s input and a source of governance risk. Founders should verify user permissions and set boundaries around employee use of agentic tools.
What it means for you
- Check tuning results. Ask for task-level results, tuning data volume, baselines and failure rates. Separate broad pretraining claims from performance on your own work.
- Map action coverage. Check which games, controls and environments contribute to the training set. Measure performance on conditions that differ from those examples.
- Test the safety layer. Define which sensor checks or deterministic rules can block model actions. Evaluate near misses and changing real-world conditions.
- Audit data and tool access. Confirm the permissions covering recordings used for training. Review employee policies for model access to code, credentials and external systems.
What to watch
- Whether General Intuition publishes task-level results explaining its claimed minutes of robot fine-tuning.
- Whether a top video game built on General Intuition models arrives next year, as De Witte expects.
- Whether user participation terms and employee tool-access controls become concrete practices.
Send this to the robotics lead choosing training data and safety controls.
Watch the minute where he gives Medal’s annual upload figure.
Written by Real Alpha from the episode's transcript and the other moments it links, then checked against the transcript. Quotes are the speaker's exact words.
Minute by minute
- 1:45RuneScape became his first business
De Witte says he still uses Rocket League to gauge his mental sharpness, though he plays far less than before starting General Intuition. His RuneScape private-server business generated about a million and a half a year in revenue by age 18 and held about 60 percent of that market. He says he built the server by modifying the Java-based game, then shut it down after the company behind RuneScape became stricter on private servers.
- 5:55Games reward different kinds of skill
De Witte says games with changing situations and self-directed goals can build long-term planning and discipline; RuneScape also taught him English. He names Factorio and RuneScape as strong hiring signals, with Factorio the stronger of the two, followed by StarCraft. He puts Rocket League lower because he sees it as relying more on motor skills than transferable strategy.
- 9:07Humanitarian work shaped his startup path
De Witte joined Doctors Without Borders during Ebola, building medical-record systems and hardware, then satellite-based heat maps for disaster response. He helped create MapSwipe, which he says helped millions of people get mapped, but says its $50,000 budget had to cover four people's salaries for six months. Frustrated by the limits of nonprofit funding, he started a mobile game studio and later pivoted to Medal after its brawler lacked session variation and player liquidity.
- 15:15Gameplay actions can pretrain agents
De Witte says Medal records gameplay clips alongside controller movements and button presses, allowing General Intuition to train models to predict actions from frames. He compares this to language models predicting the next text token, and says diverse game data supports spatial and temporal reasoning across games, computer use and robotics. He says models pre-trained mostly on game data have transferred to novel robot tasks with minutes of fine-tuning, while stressing that both game and real-world data are needed.
- 22:13Negative examples support safety checks
De Witte says gameplay that includes harmful outcomes can help models learn cause and effect and evaluate what not to do; he says AI labs wanted the data for alignment. He describes pairing action models with deterministic or sensor-based systems such as LiDAR to invalidate unsafe actions, alongside evaluations in harmful scenarios. He says General Intuition may open more negative-event data to other labs.
- 25:30AI must preserve game integrity
De Witte says General Intuition's agent plays many games at a high level and beats him at most of them, though not Rocket League. The company limits access to avoid bots undermining game mechanics, and he argues that AI companies should work with developers and disclose when players face bots. He expects AI-driven non-player characters to be an early showcase, with promptable behavior making games more interactive.
- 31:07Purpose-built robots may beat humanoids
De Witte says some robotics demos use sped-up footage or controlled stage settings, and points to Generalist as a company that shows its models' generality. He argues that humanoids are bulky and harder to train, while simpler robots built for specific tasks may be more practical. He favors wheeled robots with cameras and two arms, and expects humanoids sooner in factories than in homes.
- 36:10Gamers want AI to solve real problems
De Witte says gamers' skepticism is warranted because AI has brought botting and efforts to replace artists, while game studios face shutdowns. He says General Intuition wants to help with uses such as anti-cheat rather than replace art workflows. The company launched Nerve to match people with work based on action sequences they excel at; for example, it could use Factorio players to label factory-specific data.
- 42:37Science and coding prepare workers
For young people, De Witte recommends math and physics for rigorous problem-solving, plus coding and AI as basic literacy. He says biology has major potential, but scientific progress is constrained by missing measurements and data, making measurement tools an attractive career area. He cites Stanford's Eterna, where players used puzzles to help predict RNA structures, as an example of games contributing to science.
- 47:21AI demand hinges on infrastructure
De Witte argues that better models generate new questions and support more agents, so he sees no sign that demand for intelligence is capped. He says the economic outcome depends on infrastructure, energy and margins, and warns that lower-cost providers could erode Western model margins. He says open-source models can help General Intuition because it has a unique dataset, while reliance on Chinese models raises supply-chain concerns.
- 52:59Data access and security need safeguards
De Witte says he understands the emotional objection to using users' clips to train models and wants Medal users to share in the benefits through job or game-development opportunities. He says distillation is difficult to prevent, so labs need to be careful about what they train and how they evaluate models. He recounts an intern's legitimate reverse-engineering task in which Claude found an exposed S3 access key and retrieved SDK documentation, illustrating how AI can surface security risks users may not recognize.
Published Aug 3, 2026.
Transcript · 50 min read
Newcomer Pod, hosted by Eric Newcomer, with Pim de Witte.
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
On RuneScape, I built the largest private server in the world, so that's the only one where measurably, I think, did pretty well. Is there money in that? Yeah. The company generated about a million and a half a year in revenue by the time I was 18 years old. You're saying there are more car crashes in the video games than in the real world? Yeah, we've counted. The amount of clips uploaded with car crash labels surpasses the amount of physical car crashes that happen in the United States every day. Are there games you're prioritizing to show off your capabilities? I think we'll leave a bit of a surprise.
At 18, Pim DeWitt was running the biggest RuneScape private server in the world. Today, he runs Metal, the app gamers use to clip their best moments. They upload about a billion clips a year, and every clip records the exact buttons the player pressed. That's what his new company, General Intuition, trains AI on. The result is a model that reads a screen and works out what to do next. And that skill transfers straight to robots. Pim says his AI already beats him at almost every video game. almost, not Rocket League. This is my conversation with Pim DeWitt, CEO of General Intuition. Pim DeWitt, CEO of General Intuition. Thank you so much for joining the Newcomer Podcast.
Thank you for having me. We have so much to talk about. We'll get into some of the big questions of artificial intelligence progress. You're building in what might be, you know, one of the buzziest categories in Silicon Valley right now, world models. But I thought I'd start off in more fun, lighthearted territory. You were at the top of your field in RuneScape and Rocket League, I believe. And they sort of helped you get into entrepreneurship, right? So yeah, tell us about your RuneScape days and are they all behind you yeah um runescape uh no i still play both games occasionally actually so i got so good at rocket league that i can kind of tell how my mental state is by how i play and whether i should be making important decisions that day so if i if i'm on it and sharp i can see that i'm playing really well i can essentially measure my my level of decision making by uh like if i play like a few games in the morning i kind of know whether or not it's like a good day to make big decisions and do you do that you're like i have a board meeting today let's play some rocket league and see if i'm on on admittedly my best admittedly recently no um uh but uh you know i think uh you know in the past year uh you know i i probably get 10 percent of the games in
Read the whole transcript
before i started the company uh but i don't know runescape i built the largest private server so in rocket league i was um i got up to i got up to gc1 so definitely not the top of the field but it's it's a it's a good place to be you know what top percent what percentage i don't know if i guess so it's double so i'm guessing top five percent or something like that i don't know what the today's numbers are um on runescape i built the largest private server in the world so that's the only one where measurably i think did pretty well is there money in that yeah yeah company generated about a million a half a year in revenue by the time i was 18 years old um and we owned about 60 percent of the private server market on runescape yeah is that still running no uh shut it down when the company behind runescape got sold i believe in 2015 it got sold to a chinese mining company at some point and like physical mining or crypto yeah there was the no physical mining yeah there was some there was some program at the time where the government was uh encouraging non-technical companies to go buy overseas technical companies And I believe they somehow ended up buying the creators of RuneScape.
And then, yeah, as a result, we had to shut our servers down. Because they made everything private? Yeah, I think they were more strict on private servers than the previous ownership. Did you, you know, were you one of these people, like, harvesting in-game items and selling them to you? So you could, so in the private server, so I built a private server, which basically means... It means that you host and distribute your own version of the game, both the client and the server, which is fairly easy to do on RuneScape because it was a Java-based application, so you could essentially just decompile and de-obfuscate the code, and you could modify it, and then you could see exactly the interactions going to the server, and then you could essentially construct a server that acts in the way that the client would expect, and there was a whole community of people that was doing this.
it wasn't just me it was like probably a thousand people that were all excited about this stuff um i just happened to create one that became very large um and yeah item spawning was one of the things that people made money on um but are you breaking the game like are there people who object to this for sure yeah this is on the edge of allowable uh behavior the private server got extremely popular after runescape removed the wilderness which was um sort of their primary draw for a lot of the like player versus player oriented people that played the game um and then yeah i think you also asked a lot of people come with runescape origins and i think the reason for that is is because runescape uh was free and world of warcraft was paid so all the rich kids played world of warcraft and all the poor kids or people that didn't want to pay the money uh played runescape and they they had to in order to get to the top you had to get the hustle you couldn't pay for it and so I think it basically self-selected into uh the people that actually wanted to because like RuneScape notoriously did not have pay-to-win mechanics right so you couldn't buy your way into stuff there wasn't like you could pay for membership but everything else you had to grind
and so it was both free so it self-selected into the people that actually just wanted to work hard and then it actually was a very very great simulation of economies like I learned English there like I have my some of my colleagues that I played with still remember when I didn't speak can work at word of english so it teaches you in a very sort of general way um how systems work so your language started out in the most internet forum way possible yeah yeah i'm sure there are lots of parents you know they're yeah like oh my son plays too many video games oh there's a path
“i found factorio to be even higher signal than runescape and then probably starcraft uh is probably number three”
papers yeah i love it and so metal yeah explain your sort of first foray and is that your first company or obviously the servers is that your first startup are there other startups too um so i started uh so my private server was my first company right which did yeah did very well um i I joined Doctors Out Borders after that. What is it, sir? MSF, the medical charity. I joined during Ebola. The reason was I love doing hackathons.
So I would go compete in hackathons and build products. And I placed second in one for MSF. When Ebola happened, it just happens to be the case that private server backend orchestration with local networking on different servers all around the world was very, very similar to setting up Android and Java-based medical record infrastructure for Ebola treatment, which was a very strange overlap. And I had built an Android client for my private server. So it was like a pretty good fit for me to, and my private server was shutting down at this point. I kind of knew that. And so I joined Dr. Subborders, spent three years there, worked first on Ebola.
So building medical record systems and hardware for Ebola treatment, and then worked on generating satellite-based heat maps for disaster response. So like when houses are far apart, You can assume there's more income when there's playgrounds. You can assume that there's children nearby. And so you can sort of use these to optimize planning and disaster response for minimizing loss of life. And you can automate a lot of these things. So I built a lot of algorithms that help. This is the benevolent surveillance state. It's called MapSwipe. No, but I'm saying there's a broader sense that you're inhaling all this data about the world.
I think there are lots of people who have a negative intuition about that. And you're saying, well, if you suck up all this data, you can learn things to help with public health. It's critical. It's critical to have this data when you write Doctors Without Borders. So the motto is always the first on the chopper. Well, it's probably good if you know where to land. Right. So, you know, like where to actually start, start, start your response. And the way that you do that is using things like map data, really good planning. So. So three years of Doctors Without Borders. uh and then uh yeah so started a large project here called mapswipe which is it's um it's a non-profit but it's the thing i'm most proud of uh which has helped millions of people get on the map for the first time for these type of situations um what does it mean to get on the map i there's about in 2015 there was about a billion people that were not on maps uh which means that uh their areas it may have been in the satellite image but it's probably wasn't labeled um and so if like you would be out of the economic calculations you would be out of the disaster response is this like getting your town onto google maps kind of yeah so but google
and microsoft at the time were not mapping for instance myanmar or south sudan because it was either economically not viable are too dangerous and so that's also where a lot of these things sort of tend to happen and so there's this huge overlap where when you're not on the map it's kind of like partially you don't exist for for purposes of information systems um and so getting people on the map is really important uh and getting like good labeled data on on on towns and geometries of of uh of houses and all this went into open street maps so it's all open source um and uh we handed it over to heidelberg university in germany who now uh runs the project um so that that was the that was my second what i call sort of company that that's actually the thing i'm the most proud of
“we got the gameplay mechanics correct, but the gameplay loop and variation within the sessions, I think, was too low.”
I mean, we needed to know in particular about Metal because General Intuition is built on those recordings.
Is that right? Yeah, partially. So we use, similarly to how LLMs use sort of internet skill text data to bootstrap their tokens, we use the recorder actions. So LLMs are trained by, right, given a sequence of text, predict the next text token. And our models are trained by, given a sequence of frames, predict the next action token. It's very similar. and so uh what we do is when people are playing with With Metal, there's lots of features for gamers that enable this too, like for instance, controller overlays and things like that. So we capture, for instance, the controller movements, the DX, DY, STI, or the buttons that you press.
Like as they're received to the computer, basically. Or it's like you press this button, I got it this time, this action happened, and the relationship between those. Yeah, exactly. And so now you have a data set for understanding how you reason in space and time. And if you do that across enough environments, it turns out it transfers incredibly well to problems like robotics. But even think about how much interface use is present in things like Factorio. So it even transfers to things like computer use. And so we found that sort of pre-training on this incredibly diverse gaming data set, actually gaming in a way sort of perfectly marries the spatial temporal dynamics of the real world, but also the information density of the internet in a way that the internet text pre-training doesn't do. So it's this pre, like we look at it as just the next phase of pre-training where we can actually transfer out to both digital use cases and physical use cases using the same
“we have models that are pre-trained on mostly video game data that transfer over to novel robotics tasks in minutes worth of fine-tuning data.”
right thing to do um specifically around things like like negative events yeah there's so many things your technology can do and i want to get into the robotics of it yeah but are you are you going to create like i don't know the best halo player or whatever or like to to build like how much are you creating like ai is capable of just crushing every human being at uh runescape yeah They're actually the same problem.
It's both just a general agent in pixel space, right? It's both a general agent in pixel space that you can steer using game controllers and keyboard and mouse. It's the same. So you're building it? Yeah, we're building general agents that can reason over environments that require deep spatial and temporal reasoning, which is mostly pixel space. Are there games you're prioritizing to show off your capabilities? I think we'll leave a bit of a surprise. But I think for us... because obviously what dota was key to some of the labs early on yeah yeah we have we have currently a general agent that can play many games at the highest level uh using the same model that can also steer robots because again it's the same problem it's pixels you play against it a lot yeah personally yeah it's it's better than me at most games not yet at rocket league yeah really yeah the i mean input type games is it really interesting to compare give i would think it would dominate a rocket league because it can input instantaneously and perfectly like how are you better at it on rocket league that one surprises me i'm not sure yeah it's yeah unknown you're just so good you know no uh yeah it's really interesting similarly to how on early
llms you saw that was really good at some things and really bad at others because of how you've either modeled the architecture or the recipes it's the same thing you have a name for your system or what do you refer to it yeah we're not yet announcing it but uh the we did announce our world model class uh two weeks ago okay uh mirror but the agent is not the agent we're not announcing yet but you work in sort of private partnerships with people or that's not even customers yeah we already have customers that are using this in video games um we purposely don't want to enable the api for gaming because we don't want people to use it as a bot uh because it breaks
“i would be shocked if next year we don't have a top video game that was built on top of the general intuition models”
you think will be video games versus like real world uh it's hard to say uh we're very focused on the games industry and robotics at the moment um i like robotics because it's uh when you solve robotics robots can then assemble more robots and so it's it's a it's a it's a it's a it's a it's a constraint that you need to unblock and then you can unblock many other use cases once you unblock that constraint video games is really interesting because we have such a big mode there but the models work incredibly well out of the box with no fine-tuning uh and and so game developers love love them uh you're like robots very societally important going to change the world video games were awesome at it and it's fun yeah yeah which i understand the how silly that sounds uh but that that is the reasoning uh and and i think the um with with robotics like to take that head on i think you know it's hard to know how much the robots are secretly controlled with the joystick like what's your sense of smoke and mirrors versus reality on the public demos we're seeing with robotics right now there's a lot of speed ups that people don't realize especially when you have like vla or vlm based approaches where the models are completely incapable of
responding to real-time dynamics so they'll they'll they'll move the robot at like you know 100x slow down or like 50x slow down and then they'll stitch the frames together in a way to make it look real time interesting uh that one's interesting so you're saying just spell that out you're saying you think there are cases where sure the company could do that but they like pretend like they do it at a much faster speed than they do yeah pretend or i mean we're saying there's some but it's a lot of it looks cool yeah A lot of times, a lot of the demos are in stage environments, not in like when there's people walking around or different lighting conditions or inside versus outside versus one of the companies that I really like is called Generalist.
They do a really, really good job at just actually showing the generality of their models. And but a lot of other labs, they they just kind of show it in one. One thing that's interesting is as humans, right, we can play video games using the same strategy that we can control a robot. If you can actually control robots using keyboard and mouse, it actually feels like you're playing a video game. So, a sufficiently good model, in my opinion, should be able, if it's a good general agent, it should be able to generalize to video games and robotics at the same time, which is, I think, one of the reasons why people are going to choose our models.
Every robotics company starts in simulation anyways. And so, these models will be really, really good at simulation of the box, and then we'll need some work to transfer over into real robots.
“i think that supply chains actually converge on gaming inputs instead of humanoids this is my bet”
“So like a Factorio player, we might use them for labeling like very factory specific data, right?”
For a teenager, right, if I could, if I, let's say I'm an 18-year-old and I can make a few hundred bucks a month, that's actually a lot of extra life.
“mathematics, physics as sort of a foundation, because those are very, very good at abstract problem solving and validation”
I mean, I witnessed self-driving cars. Today, I mean, we watch Waymos. It feels like, okay, we're getting pretty close. If you've been to San Francisco and spent time in them, you feel like, all right, we're here. But back in 2016, there were a lot of smart people, a lot of AI researchers. day now we're almost there and then they just seem to misunderstand either they had the incentive to raise lots of capital and say like oh this is almost here when they it wasn't or more charitably it's just really easy to underestimate how much like you know the last five percent is hard to solve and matters a lot yeah to success like what do you think is the risk that it seems like we're close but then near enough just isn't enough what i bring this back to is when people get a really good model um the first thing that model does is uh clarify for instance what other questions you should be asking and then it spawns sub-agents right the better the model the more sub-agents it can support and so there's this i think there is this effect where intelligence just demands intelligence and it's this loop we're essentially consuming tokens if the tokens are high quality
actually make you then use more AI because you can solve more problems. And so I see this pattern just come back all the time. And then it's not just humans consuming the tokens, right? It's agents that you have prompted choosing to consume tokens. And even in that part of the curve, we're still super early, right? Think about how much more tokens you consume after Cloud Code than before Cloud Code where you had to manually prompt everything. So I just see no reason to think that the demand for intelligence is not uncapped at the moment. However, with that said, economically, I think it's a question of where is it infrastructure, right?
Is it in the US? Is it in China? Is it? And like, where are the margins, right? To translate what you're saying, you're saying every new model that comes out, businesses and people want to spend more and more money on it. It's like, oh, there's more to be had. So the demand goes up even though it's more expensive to deliver, but you're saying we're just going to hit the capacity of our ability to keep sort of improving these models. What is a company? A company is a function to convert, depending on, I'll just say the simplest example, right? It's capital into more capital, more efficiently given energy and time, right?
And so if you can do that more effectively by spending tokens, then yes, of course, you're going to do that. uh and um and so yeah i see no reason why um uh why any of that is is is looking to stop but i do see i am concerned for instance about um the margins on top of like infrastructure and model providers and i think it's it's you know there is a chance that um if the west doesn't win kind of the frontier of models and usage and we don't stay sort of really far ahead that our lunch gets eaten by companies that can provide the the intelligence at much lower margins because they have better energy grids or or can sustain more demand. Kimmy like this week Kimmy is like a wake-up call that we're not as far ahead as we thought we were. Does that affect you or?
Open source model development generally is really good for us because we can leverage a lot of the capabilities of open source frontier models to advance our own systems. And because nobody has our data set, it's a net win. However, I think that it's not great for, especially the distillation and all these things that happen, it's not great for the industry overall. because it does probably mean that a lot of enterprises start using Chinese models and, right, you know, the risks are currently unknown. I would say, like, there are some people that say there are risks, there are some people that say there aren't risks.
I've read papers in both directions, right? There's concepts of, like, specific tokens that are very hard to find that can trigger sequences. There's also I, but there's what you're saying that specifically, like you use some foreign countries model, and they hide some malicious, yeah, like, look at how good these things are at cyber attacks, right? Like the the supply chain risk, I think, is quite real, where you cannot possibly know, given every input, what output the model is going to give. And there's there's papers where very, very subtle kind of token sequences can trigger very aggressive responses in, for instance, domain of cyber. And then, you know, there's also the economic sides of it, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, if, If a model has economic biases towards recommending cars that are not made in the U.S. or not made in Europe, then and people are relying on these models sort of recommendations and economically, a lot of people are going to buy products from other countries that could be bought domestically.
And so I think that there's just it's really, really important that the frontier of AI stays in the countries where people live. And then secondly, I think it's really important to just deeply invest in the infrastructure build out and the energy build out because otherwise, right, the margins will get eaten eventually. This surfaces the area of like distillation, right? Like there's this debate of whether, you know, it's OK for models to sort of like extract all the, you know, learn basically how to build a great model by studying somebody else's. It seems right now like that's allowed. Like, you're basically allowed to use other people's products, see how they work, and then build your own products, right?
I mean, do you disagree with that? I mean, maybe we need to change the law. Yeah, I think there's two sides to this coin. One, LMs are trained on large-scale internet data, so it's – That's what I was going to get at. It's like an industry that took stuff from everybody else and then it's going to whine and say you're taking stuff from us. Yeah, I actually subscribe to this as well. From an intellectual property perspective, I think the law has been pretty clear that there are net new works. And therefore, legally, I think it doesn't transfer quite like that. But I think, you know, emotionally, it does.
What do you say to like your metal customers who say, oh, I had no idea that all this footage was going to be used for this, for you to get rich in this totally different domain? I think that's why it's so important that you get the economic benefits right. One of the things that I want to do is just find ways to also, if your data is in the models as a Metal user, I think you can construct really, really interesting loops, for example, where you actually make game development more about creating environments that extract more intelligence. So I think it's really important to think both from a jobs perspective, but also a user participation perspective, that there is actually just ways where you can make sure that people can participate versus be at odds economically.
So I think all these things matter. I think a lot of the reasons why people are so upset at the LM companies is because, yeah, they did just scrape all the books and all the works. And then you can ask the model to write a new book and compete with you. right we're in a little bit different regime because we predict actions and not pixels so we're a bit sort of not the same kind of problem space but I think broadly in terms of distillation the the answer is that models can be distilled it's almost impossible to prevent that from happening as as models get more efficient they need less data to to sort of be distilled right and so and it's also i think it's also crazy to assume that it was just distillation right like i they clearly knew what they were doing exactly it clearly this is a great team uh clearly um uh right like clearly there was some distillation going on but perhaps they just
“Like, the biggest risk to companies is probably your own employees bytecoding stuff, not realizing they're hacking things, right?”
It's a new world. It's a new world. Him do it. We could talk all day. This has been fascinating. Thank you so much. Excited for your agent. Someday maybe we'll get to play against it. Thanks so much. Great to see you. Likewise. Thank you, Eric. That's our show. I'm Eric Newcomer. Thanks so much for listening to the Newcomer podcast.
go check us out on Substack at newcomer.co. Please like, comment, subscribe here, and give us a review on your favorite podcast app. Thanks so much.
