Free with a sign-in.
The best 16 min of 98
How OpenAI Shapes Its Research And What's Next - EP 46 Mark Chen
Mark Chen says OpenAI must keep backing research breakthroughs and build AI that can accelerate scientific work.
Core Memory · Ashlee Vance
- Mark Chen is chief research officer at OpenAI.
The best minutes
- 8:16OpenAI funds exploration to find the next research paradigm6 min
- 59:58Pre-training still has room to scalePro pick 4 min
- 1:09:16OpenAI wants AI to conduct research end to endPro pick 6 min
Also worth watching
- 1:33:49OpenAI avoids rewarding models for human-pleasing reasoning4 min
- 47:17Chen rallied OpenAI researchers during the leadership crisis3 min
- 1:15:25OpenAI says it could use 10 times more compute3 min
- 23:49Models bring different intuitions to research problems4 min
- 0:00OpenAI focuses recruiting on the people it most needs4 min
Skip the sponsor reads, the extended math-contest and poker history, and the education and early-career biography.
Full analysis · 5 min read
Real Alpha · Nov 8
Mark Chen says OpenAI ranks about 300 projects for compute
- Chen says OpenAI ranks about 300 research projects every one to two months.
- His strongest evidence is a successful reasoning bet, though no project hit rate is disclosed.
- For leaders, the question is whether exploratory spending earns measured progress, not just more compute.
Free with an account
Read the rest of the analysis
7 sections, the 6 other voices it tests Chen against, and what it means for you. Every episode we cut gets one.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Signed up before? The same buttons sign you in. The episode, its moments and the transcript stay free.
Chen ranks about 300 projects every one to two months
Chen says he and Jakob review about 300 projects every one to two months, ranking each and assigning research compute.
He says priorities must be visible in direct communication and in compute allocation.
That makes compute a governance tool, not just infrastructure scheduling.
Yet Chen gives no rubric, rejection rate or budget split for the rankings.
Our read
The useful disclosure is a recurring portfolio review, not the spreadsheet itself. Without decision rules or project-level outcomes, outsiders cannot judge how often exploration beats near-term performance.
OpenAI says exploration outranks benchmark chasing
Chen says more compute goes to exploration than to training the final artifact.
He says OpenAI does not build its program around copying other labs or catching up on benchmarks.
That is a strategic distinction, but not a neutral description.
As chief research officer, Chen is describing OpenAI’s own research approach.
For example, Richard Socher, Recursive’s CEO, said on MTS in October 2026 that open-ended search finds ideas direct optimization misses.
He used microwave invention to illustrate how distant lines of inquiry can combine.
That supports exploration as a method, not Chen’s account of OpenAI’s actual allocation.
The transcript supplies no project-level results or comparison between exploration and product-model performance.
“We are not in the business of replicating the results of other labs”
Our read
The tension is not exploration versus measurement. It is whether open-ended searches have tests that show what a new direction actually adds.
One reasoning bet supports the portfolio, not every bet
Chen says OpenAI backed reinforcement learning for language models more than two years before it became popular.
At the time, pre-training and post-training looked like safer bets.
He now calls reasoning a primitive the field cannot do without.
That is a persuasive example, but it is a retrospective account from the lab that made the bet.
Take David Silver, Ineffable Intelligence’s founder and CEO, on Training Data in October 2026.
Silver described AlphaZero as a 50-50 research bet.
He said it took a few hours to outperform two years of human-in-the-loop work.
Chen offers no failure rate for competing projects, or a way to identify a promising anomaly before payoff.
Our read
One win earns exploration a place in the portfolio, not a blank check. The missing denominator is how many bold projects absorbed resources without producing a new research primitive.
Pre-training remains an open bet with an evaluation problem
Chen says OpenAI devoted substantial effort to reasoning, then rebuilt pre-training expertise during the last six months.
He calls it a muscle requiring frontier work on optimization, numerics and organizational attention.
Chen says scaling is not dead, but identifies a ceiling too.
Training on human-written material teaches imitation, so surpassing human output requires harder tasks and other learning signals.
He also sees an evaluation problem beyond human performance.
Once systems exceed contests like the IMO or IOI, human rankings may stop separating their scientific output.
Alexander Wissner-Gross said on Moonshots in November 2026 that benchmarks can steer AI toward outcomes people care about.
That is a useful counterpoint, since Chen needs evaluations that discriminate, not benchmarks abandoned altogether.
Alexander Wissner-Gross · Moonshots · Nov 4Benchmarks can steer AI toward problems people care about
Our read
A research budget needs two connected investments, continued scaling and evaluations that survive human ceilings. Chen gives neither a named evaluation nor evidence that OpenAI has solved this problem.
Compute demand does not establish research returns
Chen said he could use 3x more compute immediately, and probably fully use 10x within a small number of weeks.
Those are estimates of deployable demand, not evidence OpenAI already has or has used that capacity.
Chen said algorithmic breakthroughs enable further scaling, and pointed to data efficiency as an area where OpenAI has strong algorithms.
He gave no comparative data here.
For context, Misha Laskin of Reflection AI described a roughly four-times compute multiplier per generation.
He said this on No Priors in October 2026.
That frames the capital burden, but does not validate Chen’s utilization estimate.
OpenAI plainly has an interest in securing compute, so the estimate needs an outside measure.
“if we had 3x to compute today, I could immediately utilize that very effectively.”
Our read
Treat the 3x and 10x figures as demand to validate. Boards should tie each increment to measured learning, evaluation coverage and research output.
Chen sets research automation deadlines, not verified outcomes
Chen sets a one-year goal for research teams to rely productively on AI interns.
Within two and a half years, he wants AI doing research end to end.
He describes the first stage as people choosing ideas while models implement and debug.
That makes the near-term target more concrete than an unqualified AGI date.
But Chen leaves productivity, independence and the number of eligible research tasks undefined.
Qasim Wani, Extrapolate’s CEO, said on MTS in November 2026 that recursive improvement involves more than model training.
Wani includes data, experiments, evaluations and infrastructure in the research loop.
That is a completeness test, not proof Chen’s timetable is feasible.
Our read
Teams can test progress through researcher hours displaced, accepted experimental runs and reproducible discoveries. The two-and-a-half-year goal remains a management target, not a demonstrated capability.
Observable reasoning helps, but alignment remains unresolved
Chen calls alignment one of the main challenges over the next one or two years.
He said OpenAI chose not to supervise model thinking, fearing human-pleasing explanations could make reasoning less candid.
Leaving that channel observable, he said, supports alignment research.
Yet an observable answer is not proof of an honest intention.
Chen warns that a model may produce the expected answer through a twisted process.
Ethan Elasky said on MTS in October 2026 that his team replicates risky agent incidents, including simulated agents on a wiki.
That suggests behavioral tests alongside reasoning inspection, but does not show OpenAI uses that method.
Chen offers model-on-model games as research directions, not as a solved safeguard.
Our read
For builders, visible reasoning should not count as an audit log of intent. The evidence must come from behavior across situations, not persuasive explanations alone.
What it means for you
- Log allocation decisions. For each exploratory project, record compute assigned, the intended discovery and the evidence that would justify another cycle. That makes portfolio reviews testable rather than anecdotal.
- Name stopping rules. Set milestones before an experiment begins, including what result would stop funding. Chen describes bold bets but gives no failure rate or decision rubric.
- Build a science evaluation. Choose a research task where humans can verify progress, then test whether the measure still separates results as model performance improves.
- Measure compute yield. Track utilization alongside accepted experiments and reproducible results. Chen’s 3x and 10x estimates describe demand, not demonstrated returns.
What to watch
- Whether OpenAI defines measurable milestones for its one-year AI-intern goal and its two-and-a-half-year end-to-end research target.
- Whether pre-training improvements come with evaluations that can distinguish scientific performance beyond human contest scores.
- Whether alignment work tests model behavior as well as leaving reasoning observable.
Send this to the research leader and finance partner setting compute budgets.
Watch the minute where Chen says OpenAI reviews about 300 projects.
Written by Real Alpha from the episode's transcript and the other moments it links, then checked against the transcript. Quotes are the speaker's exact words.
Minute by minute
A Pro write-up, open to you here. Try Pro free for 7 days
A Pro write-up. The minute by minute is in Real Alpha Pro; the episode, its moments and the transcript stay free.
Read it with Pro, 7 days free- 0:00Chen says OpenAI resists recruiting raids
Chen says Meta pursued half of his direct reports before hiring anyone from OpenAI, and all declined. He argues OpenAI should not match offers dollar for dollar; it should retain key people and keep developing talent. He recalls a Meta leader hand-delivering soup to recruits and says he has also brought soup to Meta candidates.
- 4:51Compute allocation makes priorities visible
Chen says he and Jakob review roughly 300 projects every one to two months, ranking them and aligning compute with the research group’s priorities. He describes a research organization of about 500 people inside a much larger company. The process also requires leaders to say clearly which projects they cannot fund.
- 8:35OpenAI bets on new paradigms
Chen says more compute goes to exploratory research than to training the final model artifact. He describes OpenAI as focused on finding the next paradigm rather than replicating competitors’ benchmark results. He says the lab made an unpopular bet more than two years earlier on reinforcement learning for language models.
- 16:13Chen tests models with a 42 puzzle
Chen says OpenAI judged Gemini 3 to be a strong model but believed its own internal models already performed at that level, with successor models planned. He tests models with a puzzle to build a random-number generator modulo 42 using generators modulo primes below 42; he says models get close but have not reached the optimal solution.
- 19:54AI brings new intuitions to research
Chen says coding contests showed him that models can find ad hoc problems easier than people expect and bring different intuitions to frontier work. He recounts a physicist using GPT-5 Pro on a paper problem; the model thought for 30 minutes and solved it. He says models’ performance in competitive programming has risen quickly and that interviews and exams may need new ways to assess people.
- 35:40HFT pushed Chen toward AI
Chen says he spent four or five years in high-frequency trading after MIT but found its closed, distrustful culture difficult and saw the field changing only incrementally. Watching AlphaGo prompted him to study AI and reproduce Deep Q-Network results on Atari games. He says people can reach the field’s research frontier quickly by spending three to six months on a project.
- 40:26Chen joined OpenAI as a resident
Chen says OpenAI had about 20 people when he joined in 2018 as a resident, a program that trained people from other fields in AI. Ilya Sutskever guided his early projects and study. Chen later worked on ImageGPT and Codex, then moved from individual research to management.
- 47:17Chen rallied researchers during crisis
Chen says he and colleagues set a goal of not losing a single researcher during the leadership crisis. They opened their homes to colleagues and helped them stay connected to leadership. He says more than 90% of the research organization had signed a petition supporting Sam by morning, with nearly 100 people signing.
- 53:32Chen favors bottom-up research ideas
Chen says research needs top-down direction, but argues that strong ideas also emerge from unexpected places. He and Jakob coordinate priorities, staffing and incentives, while Sam talks with researchers and looks for hidden problems. Chen favors sharing ideas and moving quickly over building deep information silos.
- 59:58OpenAI is rebuilding pre-training
Chen says OpenAI put substantial resources into reasoning over the last two years, which weakened its pre-training and post-training work. He and Jakob have spent the last six months rebuilding that capability, and Chen says the lab sees room for further scaling. He adds that learning from human-written data has limits, while reinforcement learning can push models beyond imitation; measuring superhuman performance remains difficult.
- 1:06:53AI could accelerate scientific discovery
Chen says recent math and physics results have shifted his view of AI’s scientific value, though many scientists remain skeptical. He wants OpenAI’s tools to help scientists make discoveries themselves. The research organization aims to use AI interns within a year and reach end-to-end AI research within two and a half years.
- 1:15:25Compute demand supports further scaling
Chen says he could use three times more compute immediately and put ten times more to productive use within a small number of weeks. He says OpenAI plans to keep scaling and sees algorithmic room to improve data efficiency. For future ChatGPT devices, he expects memory that learns about each user and says Jony Ive brings the design expertise.
- 1:24:44Chen says OpenAI must stay research-led
Chen says OpenAI remains focused on AGI, automating AI research, advancing science and enabling economically useful work. After DeepSeek prompted questions about whether OpenAI had lost its way, he says the lab doubled down on its own research; he describes DeepSeek as a strong lab and its model as a replication of ideas in OpenAI’s model series. He favors a talent-dense team and says the company should publicly recognize contributors.
- 1:33:49Chen warns about hidden model intentions
Chen says alignment is a major research challenge, as models may provide expected answers while reaching them through concealed or troubling reasoning. He says OpenAI chose not to supervise the models’ thinking process, preserving it as a tool for studying alignment. He is interested in methods such as models supervising one another to encourage honesty.
Published Dec 1, 2025.
Transcript · 1 h 17 m read
Core Memory, hosted by Ashlee Vance, with Mark Chen.
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
“The game is not to retain every single person in the world. It's to trust in this pipeline”
We have a special treat today. I'm excited. Mark Chen is here from OpenAI. He's the chief research officer. He's somebody I've gotten to know over the last couple of years. Thank you so much for getting in. Yeah, it's been great to know you for so long. I feel like there's a handful of people in this world working on this very important project. And I mean, you're right at the top of it. So it's so cool to have a chance to chat. Yeah, thanks for having me on. It's my pleasure.
And I mean, there's a bunch of things that I want to talk to you about because I've gotten to know you, like we said, over those last couple of years. I want people to know a bit more about your biography, but I also know there's going to be AI enthusiasts who want us to go deep on a couple of things there. So we'll try to do everything. um i wanted to start just by giving people a feel for your job which in my head when you i mean just correct me if i get any of this wrong but you're you're you know sam has been he's really into research he's the boss he's kind of at the top of the food chain but then you and and yakub are working together to shape open ai open ai's research direction and and then you're in this additional part of this role is is is actually deciding which compute goes where onto these projects so you kind of have to chart where opening eye is heading and then the mechanics yeah of how you're going to get there yeah and this always strikes me as a horrible job because i picture people um doing everything in their power to get gpus from here it's true People are very creative in the ways that they try to make backroom deals to get the GPUs they need.
Read the whole transcript
But yeah, I mean, it is a big part of the job, right? Figuring out the priorities for the research org and also being accountable for execution. So really to that first point, there's this exercise that Jakob and I do every one to two months where we take stock of all the projects at OpenAI. And it's this big spreadsheet about 300 projects. And we go and try to deeply understand each one as best as we can and really rank them. And I think for a company of 500 people, it's important for people to understand what the core priorities are and for those to be communicated clearly, both explicitly verbally and also through the way that we allocate compute.
All right. What do we do at Core Memory? We cover innovative, fast-moving, forward-thinking companies, which is why Core Memory is sponsored by Brex because Brex is the intelligent finance platform for many of these companies. 30,000 companies from startups to the world's largest corporations rely on Brex's technology for their finances. They've got smart corporate cards, high yield business banking, and expense automation tools that are fantastic. I hate doing my expenses and Brex's AIs and software run right through those expenses figure out where we're spending money and take care of so much stuff for you so you don't have to waste your time on it yourself go to brex.com slash core memory to learn more and just you know get with the program let's get going let's get out of this archaic finance software and move to War of the Future, Core Memory, and Brex.
So you've got, when you're talking about the 500, these are the 500, this is the heart of the research team. Yeah. In an organization now, it's thousands of people. Yeah. Okay. So, and then in that, when you're talking about this 300 projects, I imagine, I mean, obviously some of those are the giant frontier models, and then some are probably experiments that people are working on. And so, like, how do you possibly
“We are not in the business of replicating the results of other labs, of kind of catching up to other labs in terms of benchmarks.”
2018 in 2018 and so you feel like that soul that that that um yeah that core culture and that core nucleus it's it's really persistent it's still there what does it elon says uh what does he say he says we shouldn't call any of you guys researchers it's just engineering yeah no i think yeah we no it's it's true because i feel like once you have this hierarchy um and you elevate, let's say, research science as a thing beyond engineering, you've completely already
lost the game. Because, you know, when you're building a big model, so much is in the practice of optimizing all of those, you know, little percentages of, you know, how do you make your kernels that much faster? How do you make sure the numerics all work? And that's a deep engineering practice and if you don't have that part of the picture you can't scale to to the number of gpus we use today so because i think they're well okay but there is like a mystique that surrounds a researcher versus an engineer you know what i mean so are you where you do you feel like um it is better to kind of stay level-headed on that is that is that kind of what you're saying or or um well i i just feel like researchers they come in so many different shapes you know uh some of our best researchers, they're the type that, you know, they come up with a billion ideas, right?
And many of them are not good, but, you know, just when you're about to be like, ah, is this person really worth it? They come up with some, you know, phenomenal idea. Some of them are just, you know, so good at kind of executing on the clear path ahead. And so there's just so many different shapes of researchers. And I think it's hard to just lump it into one stereotypical type that works that makes sense um okay i won't i won't belabor you with too many Like, rival questions. It's just, since Gemini 3 did come out, I did wonder what happens with you personally or the team when one of your rivals puts it, like, does everybody go and look and see what it can do? Is there like a, is there a prompt or a question that you often throw at these new models to see what they can do?
Yeah, yeah. So to speak to Gemini 3 specifically, you know, it's a pretty good model. And I think one thing we do is try to build consensus, you know, the benchmarks only tell you so much. And just looking purely at the benchmarks, you know, we actually felt quite confident, you know, we have models internally that perform at the level of Gemini 3, and we're pretty confident that we will release them soon. and we can release successor models that are even better. But yeah, again, kind of the benchmarks only tell you so much. And I think everyone probes the models in their own way. There is this math problem I like to give the models.
I think so far, none of them has quite cracked it, even the thinking models. So yeah, I'll wait for that. Is this like a secret math problem? Oh, no, no, no. Well, if I announce it here, maybe it gets trained on. But yeah, I do think it's one of the nice puzzles of last year. It's the 42 problem. So you want to create this random number generator mod 42, and you have access to a bunch of primitives, which are random number generators modulo primes less than 42. You want to make as few calls on expectation to these sub-generators as possible. So it's a very cute puzzle, but the language models, they get pretty close to the optimal solution, but I haven't seen one quite crack it.
okay this is we're heading down a direction i want to ask you about yeah just before we get there so i know you're i've seen you you're very competitive you've also told me yes i think i i love competition i hate to fucking lose so i really hate losing i hate losing yeah so i'm picturing i'm just curious if this is at all right i mean if you know if we know gemini 3 or whatever is coming out on a thursday i mean are you up at like midnight throwing that problem at it or is it is it not quite that drastic um no i mean i think it's in long arcs right um and any endeavor like i you know i'm kind of a person who has obsessions i think any endeavor you have to play the long game um and you know we've actually been focusing on pre-training specifically supercharging our pre-training efforts for the last half year um i think it's a result of some of those efforts together with Jakob, focusing and building that muscle of pre-training at OpenAI, crafting a really superstar team around it, making sure that all of the important areas and aspects of pre-training are emphasized.
That's what creates the artifacts today that feels like we can go head-to-head with Gemini 3 easily on pre-training. Okay. And I want to ask about the pre-training, because I've been talking to all you guys about this a lot. but but um okay but but you're saying that um you're less obsessed about lobbing um problems at these new models just when they appear and more at this this long journey absolutely yeah okay um okay the reason i want to talk about sort of the the puzzle that you're at i mean i you know i first met jacob before open ai ever started when he was doing a coding competition And I got, I got like super into coding competitions for a while.
There's this guy, Gennady. I don't know if he's still famous, but he was like the Michael Jordan of these coding competitions. And so I went to watch one at Facebook used to, I don't know if they still do, but they had an annual. Hacker cup. Yeah. Hacker cup. And, and that's where I saw Jakob for the first time. And then I know you, I think did. math competitions in high school, like probably grade school through high school. And then you also, did you also do IOI? So I got into coding really late in life. It was a roommate in college that convinced me to take my first coding class. And I had all the hubris of a mathematician at that time, whereas like, you know, math is the purest and hardest science. And that's where, you know, you really prove your worth. I mean, I think I was probably too into the competition back then um but yeah i mean it became this super rewarding endeavor and um and you know it started out as purely a way to keep in touch with my friends from college um you went to mit yeah i went to mit um you know i graduated and every weekend we would just log on and do these contests just to keep in touch with each other and um you know over time i found myself having a talent for
You know, I, um, started competing fairly well and then writing problems for, for contests like the USA coding Olympiad, eventually started coaching that team. And yeah, it's been a great community where I've met people like Scott that, you know. Yeah. Yeah. Okay. So you, so I think lots of people might be familiar with like math competitions cause they probably see kids going through that. The IOI and these coding competitions are a little bit different. I mean, it's, I mean, you'll know it so much better, but when I saw them, I mean, it looks like a it's almost like a word problem that's a puzzle and you're trying to kind of find the most efficient and correct way to solve that and you're in this race against everybody and and everybody's like writing code on their computer and then and then some people try to get there really fast but then their thing kind of doesn't solve the problem right and then you know there's like this trade-off right yeah absolutely right and so so you you actually were on the mit team No, no.
It's something I did after college. After college. Okay. But today you are like the coach of the U.S. national. Yeah, one of the coaches. One of the coaches. Okay. And was it last year or the year before? Like the U.S., like we hadn't won one in a long time, right? Yeah, yeah. Yeah. Didn't we? Yeah, yeah, yeah. So, yeah, I mean, the team, I mean, it's, you know, you can never predict what the makeup of top talent looks like every year. But we had a very spiky team, I think, two years ago. Okay. And yeah, I believe they won the Olympiad. Because I feel like usually it's like China or Russia or like Belarus and Poland.
I mean, right? Yeah. Yeah. And so the big competition takes place in a different country every year. What does it look like? How many people show up? Yeah. Yeah. So they take the top four students from every single country. It is as much of a competition as it is a social event. this is a tight-knit community. They all go on to do phenomenal things. And yeah, it's this intense two-day contest where each day you get just three problems, five hours to solve them. And you can really feel the adrenaline and all the pressure in the room. But it's also great fun. I think people settle down and they make lifetime friends through it. And as a coach, I mean, you're so freaking busy, man. How much time do you spend on this? What does that look like?
Honestly, the kids are so self-motivated. Sometimes it's really about just managing their performance and strategy. I think you're going to have good days. You're going to have bad days. You're going to have good hours within the contest, bad hours, and you can't let that get into your head. There's a lot of similarities between managing contestants and managing researchers. It's like on a much longer timescale, but you know, like researchers have good months, bad months, you know, you can't really let those strings of failures get into your head because that's just the nature of research. Right. And I think a lot of it's morale management at a certain point. Yeah. I think one other interesting thing that
“AIs plus humans in frontier research, it's going to do something amazing just because the AI has a different intuition for what's easy and what's not.”
be doing these competitions in 10 years i think so i mean they're just fun yeah um i mean certainly a bunch of people who use it to you know pad their resume are gonna drop drop off from doing it but i think the people who've always basics.
excelled at it the most are people who just do it for the fun of it. Yeah. And I don't think that'll go away. When I was doing this story, I mean, they were telling me that like, if you're from Russia or I don't know which countries that you basically get like an automatic free ride to any university that you want. I mean, I see the guys on the US team go to like Harvard and MIT. So they seem to be doing okay, but it doesn't seem like the US has a... Yeah, I mean, don't you think it's gonna... Yeah, I mean, interviews, right? They're gonna be kind of broken going forward. And everyone's seeing this a little bit.
And, you know, even college exams or college homework, it's all broken at this point, right? And I do think we're going to need new ways of assessing and gauging, you know, who's performing well, who's learned the material. Where somebody's actually at. Yeah, yeah. So, I mean, I've had this idea here where maybe for our interviews, we should just have candidates talk to ChatGPT. And, you know, it's a special kind of chat GPT where the model's trying to gauge whether you know the material or whether you're at the capability level to work at OpenAI. And, you know, you have to have this conversation with it that convinces it deeply you belong at OpenAI.
And, of course, you know, you can't be allowed to jailbreak it, and we'd look at the transcript after. But maybe, like, tests like this will more accurately reflect in the future whether you know. So you don't do that yet, but you're thinking about. Yeah, just creative ways to revamp the interviews. yeah yeah well i mean silica valley is famous for doing the like brain teasers during yeah yeah everything yeah um so you i mean we taught you were very good at math growing up and and i think you were you born on the east coast uh yeah born on the east coast and then you lived on the west coast and then you lived in taiwan for life for like um grade school to high school four years Okay. And your parents worked at Bell Labs. So you come from like engineering stock.
I mean, it's a really interesting background just because you kind of got like a flavor for all these innovation hubs and especially with your parents being at Bell Labs. I mean, yeah, I just grew up in a very scientific environment. You know, dinner table talk was puzzles and things like that. And I also got kind of the more traditional, you know, Bell Labs East Coast experience. On the West Coast, my dad came to do a startup. So a little bit of that kind of new company got exposed to that when I was young as well. And of course, the big jump to Taiwan, right? And I think it's a huge culture shock.
You wear uniforms, you're in a school, it has barbed wire on the school, right? And also getting exposed to kind of that level of rigor. I think it was just a number of really great experiences growing. So like the schools were harder? Well, I would say it was just much more kind of, you know, it's just, there's a little bit less flexibility and freedom in the school system, but I think it also teaches you something. Yeah. Okay. Since day one, the Core Memory Podcast has been supported by the fine people at E1 Ventures. They are a young and ambitious VC firm in Silicon Valley, investing in young and ambitious companies and people.
thank you so much to e1 ventures for all your support yeah and you knew you wanted to come back to the u.s for college absolutely yeah yeah okay and then okay so you're you're at mit um you're kind of like in this interesting group i guess mit probably always has a interesting oh man yeah 2012 was such a great yeah like who is there's sort of like an all-star list oh i mean it was a great year like um i don't know if you knew like jacob steinhardt you know he's doing Transluce now. He, and I used to do projects together in computer science class. There is Paul Cristiano, who's a bunch of really phenomenal. He worked at OpenAge for a while.
Yeah, yeah, yeah. A bunch of kind of big names in AI came from that year. And then, and then we're talking about the... The competitive coding, like Scott Wu, who's at Cognition. I mean, he's kind of like famous now as like a meme on X for his math abilities. And you just got to know him through the coding companies. Yeah, through the coding community. Okay. And then now I see the competitive end of you guys. The output of this, to me, look like poker these days. We were at an event, which I think we have to keep secret or something. like the the specifics on this event but um i think i'm okay to talk about this part which is like i late at night i'm walking by this table there's you scott i think sham from uh palantir and then and like a handful of other people in a like a fairly intense it looked intense to be but it's just not for you guys but like a fairly intense poker game so you guys this is where you've applied your math and competitive skills now. Yeah. I mean, poker is a really fun game.
And I've talked about my life in terms of a series of obsessions. Poker was definitely one of these obsessions in the past. And I think the big revelation for me in poker was it's so much more a mathematical game than a game of reading people and bluffing. And I think the more you learn about poker the more you update in that direction right um and i think that's you know i used to be a terrible buffer and when you know it's mathematically correct to bluff then it's so easy right it's like you you don't feel any nervousness around it um and yeah it's just so interesting that you have a game that i think is perceived as so human um but the underlying mechanics and how to win are so deeply mathematical and yeah i kind of thought about this the other day where, you know, there's something about that in language modeling too, right? You have this deeply human process of generating language, but there's this mathematical machine that can really do it as well as we can. I think about that part all the time as a writer. And then I did all this philosophy back in college with Fickenstein and all these guys thinking about these things. Yeah. Well, how do you find like an edge? If you and Scott both strike me as like
supernaturally good at math, but I don't understand how one of you is out-calculating the other person. Well, no, I mean, it is mostly a forum for us to just kind of just hang out and catch up with each other. And, you know, today we don't take it as soon. I think there is an element to taking something like poker very seriously that takes the fun out of it. And so, you know, my obsession with poker, I think, has ended more than a decade ago, but now it's just fun. You're just saying this because I saw Scott win of both days i think you might be right about that he was taking it he was taking it quite seriously um and and so like coming out of college i mean you had in some sense oh i beat him on the plane though you beat him on the plane right home yes okay all right all right so you did you was it just you versus him or it was like a group thing again and maybe three or four people okay um i feel like a lot of uh i feel like there's three i don't think i'm over generalizing too far especially among like if you cut back to the 2018 right sort of time frame um as far as like people who were in ai at a high level i mean a lot had academic backgrounds A lot were math prodigies or had gone on to sort of take their math background and get into robotics or something, physics like that.
And then there's this other bucket, which is people who had gone to Wall Street and done high-frequency trading and quants and things like that. And so that was the first path that you took was you went straight from MIT to Wall Street. Yeah. I mean, I... don't wear that badge with too much pride. And to be honest, it was a path that was fairly common for very quantitatively oriented kids at MIT. And I mean, it was certainly a very meritocratic system. You could apply intelligence and there's a very concrete world function, the amount of profit that you would make. But I think culturally it was hard for me.
It was a place where when you discover something, your first instinct is to just keep it away from as many people as possible because your knowledge is what gives you your worth. And so it felt like you would have out of this an outgrowth of just even internally at a company, like these competitive dynamics and people weren't very trusting of each other. I think it was also felt like such a closed ecosystem, right? I think we today don't feel too much, you know, when someone in HFT finds a breakthrough that makes their algorithm a little bit faster, no one else feels it, right? And I, over time, just kind of felt like, you know, I woke up after four or five years, we're competing against the same exact set of players.
Everyone's a little bit faster, but had the world really changed that much for it? And it felt like time to do something else, right? It's just a bunch of things lined up back then. you know, there's the AlphaGo match, which I think was a huge inspiration for a lot of people at OpenAI. Did you play Go? I did not, but I think the sense in which, you know, the model was able to do something creative, I really wanted to understand what was going on behind that. So you're watching that happen and had you been, had you been at all reading AI research papers and things like that or to be honest no no and then i saw that event um it was really inspiring and that's when i started doing my deep dives into ai so i um one of my goals after seeing that was reproduce the dqn results um this is a network that was able to play a lot of atari games effectively at a superhuman level and um and going from there you know that's how i got my start into ai and you were doing that like on the side yeah yeah you so you just work all day and then go back yeah yeah yeah yeah okay i mean it is weird i remember i was interviewing george hots like it might have been roughly 2018 um maybe a little before that you know and he'd just done this thing
building a self-driving car on his own in his garage and then you know i mean it's george so he says large statements sometimes that may may not be like exact or spot on or apply to other people but he's like he's like ai still so young you can you can basically learn the whole field if you read i don't know what the number was 10 20 30 research papers i mean it is fascinating to me that it's like old in many ways stretching back decades but this particular moment it's very shallow um i i always give this advice to people who are intimidated by getting into ai it's so shallow, like just spend three to six months, um, picking some project, like maybe, you know, reproduce DQN and you can get to the frontier very quickly. Um, the last couple of years has added a little bit of depth, but it's not anything like, you know, theoretical math or physics.
Do you think, is this a field where I asked Jakob this the other day, I don't know why I'm obsessed with this but you know like in mathematics you see you see people tend to do their best work in their 20s or to have the big breakthrough and then it's very hard as they get older to like have that same kind of moment um like what you're saying you know are we are we dependent on on young people reading these papers and then having some insight or is this is this something where it's more you You can keep going throughout your whole career. I mean, I think you can keep going. I mean, OpenAI itself does have a pretty young culture, but I don't think you need to be young to do good research. I think there is something about being young and having less priors about this is the way it's done. I think over time, you may develop your own vision, which is a good thing, right? But it also locks you into a frame of mind of like, oh, you know, this is how research is done. This is how good results come out.
And I think younger researchers tend to have a little bit more plasticity around that concept. Yeah. Yeah. So your career at OpenAI is funny. It seems like you walked in the door and had a very important, large position from the get-go. But when you first got there in 2018, it must have been what, like 50 people? No, it was much closer to 20. Much closer to 20. Yeah, yeah. And it really looked like two teams back then. I came in as a resident. So someone who, you know, clearly not a specialist, not a PhD. I think I was only a resident, you know, throughout his tenure at OpenAI. So I was very lucky in that regard, just kind of learning the way that he thinks high level about research.
And like a resident in this case, you're just the right hand person? Oh, so it's someone who comes in usually from another field who OpenAI wanted to invest in and train up in AI. Yeah. And so I think the first part of a residency is like a six month compressed PhD. And then going from there to just, you know, getting into deeper and deeper research projects. So you're kind of like talking to Elia every day? Is he kind of shaping that PhD? Yeah, yeah, yeah. He was responsible for my projects, for my curriculum, for my learning. And I would just go to him for, you know, hey, like, what's this about? Like, why did people pursue this. Okay. Yeah. And you, I mean, I think if you go like on LinkedIn, I mean, it would say you were the head of frontier research as your first job at OpenAI.
Oh, no, no. I was in IC for three-ish years. Okay. Yeah. Yeah. So, um, I was doing independent research projects. Um, I worked in generative modeling because that was really kind of where Ilya's focus was at the time. Okay. And then, um, only after a while did I start managing teams. And because most, you're talking about generative. I mean, most people point to DALI as maybe the first big project that the public would mostly recognize. Is that fair? Yeah, yeah. So I think that also marked the transition between when I was an IC and a manager. So one of my own kind of big projects, and one I'm still pretty proud of today, is ImageGPT.
This proof of concept that even outside of language, you could put things like images into a transformer. And the model would just internalize very good representations and understand, you know, the content of images. And it's kind of like a proof of concept that you can do language modeling outside of pure text and get really, really great representations and scale them to be state of the art with other methods. That was, I consider like a precursor work to DALI, which I was on the opposite side of managing. And I think between those, another project I'm really proud of doing IC work on is Codex, where we set up a lot of the framework for evaluating coding models and also did a lot of in-depth study on how you can take language models and make them very good on code. And what made you pick OpenAI? Because I could see it two ways in my head one is is big fish in a small pond there's interesting people here i remember the open ai of 2018 with 20 people in my head it was like this is probably not going to work um you know google seems like they've got this locked up and and this is like a pretty small group of people trying to take on something that appears to require There are many billions of dollars of capital, and I mean, this was even before the scaling
stuff. It was just like Google had invested so much in AI already, kind of like in a different form than we think. But you know, you're already kind of doing translation on your phone and things like that. So was that like a hard decision for you, or you just stumbled into the OpenAI gig so quickly that... Well, yeah, I mean, I think there are two things, right? you need ambition of vision. That was certainly what OpenAI had at the time, but you also need the talent to back it up. And, you know, I did feel like OpenAI was one of the rare places where the ambition was very large, but the talent was also large enough to fulfill that gap. And, you know, I was lucky I knew people like Greg before from college and yeah.
Oh, so yeah, Greg, you overlapped at MIT? I think we did some math contests together. Okay. Yeah, back in high school. and um yeah i i shot him a message actually and i was like oh you know i i don't know i have the right skill set but this sounds like a place that's doing great work so it still seems nuts to like just come at this you know out of nowhere and and now you're like leading research no it's surreal to me too it's surreal to me too um you know even that transition from ic to manager i was very hesitant about taking it um i didn't know if managing was a skill set that i would be good at and i was really enjoying ic work i think uh um i was having a lot of fun doing it excelling at it building really great collaborations um but yeah i mean it's really been a wild ride yeah yeah well okay on that point i mean you've always struck me as a very nice like level-headed guy i have to say you know there's parts of open ai's history that are are quite dramatic soap opera like um little game of thronesy you know power struggles and and like to me um to be a manager in that i will say now like i feel like things are a bit calmer than they were but but when you look backwards it just seems like um i don't know you're saying
you have to learn these skills but some of this feels like opposite i don't know you that well but some of it feels opposite to your personality um to have to like deal with all of that honestly you know i've been lucky at open ai um i genuinely say that and um in the sense that I've had managers that have really advocated for me. You know, they saw my talent and advocated for me. I think when I was in IC, you know, Wojciech, he was like, oh man, you should bet on him for Codex. And then later on kind of reporting to Bob, I've never asked for promotion or up level. And, you know, it's just organically happened. And everyone kind of along the way has given me great advice.
I think part of growing in management is just getting the reps. I didn't think there's any better place to get the reps than at OpenAI. You know, there's always challenges to solve. And yeah, I think kind of developing that confidence. I actually think management is something where it's really just about the experience. And, you know, there's, I would say less so talent involved in it. Yeah, I don't want to like embarrass you.
“I just set this goal of, I will not lose a single person. And we didn't.”
that it was kind of like Ilya and Sam were on opposite sides and Ilya is your mentor. And then I know Ilya kind of comes back. Yeah, yeah. Yeah, I don't know. Was that awkward? No, no, it was hard. I mean, it's a low information environment.
But fundamentally, it's just... And yeah, I mean, I think at the moment, you could very reasonably conclude, like, did Sam do anything here? But would Greg and Yaakov, like people of super high integrity, quit over that um i just felt like you know there was some part of the story that that was being misrepresented yeah yeah the you know jacob's been there for a very long time um like what should people know about jacob that they don't it's interesting because he's a super funny guy yeah he's hilarious oh my gosh he has this like sarcastic humor um and yeah it cracks me up so much. Honestly, like, yeah, that's one of my favorite things about OpenAI today. Just like the level of alignment I have with Jakob. I feel like we go into a meeting, we can just bounce off ideas and quickly get to alignment and then, you know, deliver the same message and kind of like operate on different parts of a big roadmap together. And yeah, it's just one of the big privileges I have working at OpenAI. Yeah, I mean, going to that, actually that point about keeping people together. I still feel that way about OpenAI research.
You guys are still under attack. Yeah. No, we are a family. Yeah, we're always under attack. And this is how I know we're in the lead. Any company starts, where do they try to recruit from? It's OpenAI. And, you know, they want the expertise, they want our vision, kind of our philosophy of the world. And we've made so many star researchers, right? I think OpenAI more than anywhere else has been a place that makes names in AI today. um and i still feel that same level of protectiveness like you come after i'm gonna do anything in my power to make sure you know they're happy they're eye opening and they understand you know how their role fits into the roadmap yeah i mean something i've battled with as i was doing the book or even just watching events unfold in real time is like when i go back through the history i mean you've got you've got ilia in 2012 making sort of like a big breakthrough and then you know you've got noam at google in 2017 doing transformers and then you've got alec radford yeah you know like sometimes the story is these individuals really pushing the field forward and it feels like a field that's still so young that you can have have this individual and then and it seems like there's this group of i don't know what the
numbers but let's call it like eight to ten who seem to have an ability to do that repeatedly and they're really shaping where this is all gone and so when i started seeing like john shulman leave or alec leave and then you know there's kind of i was like wow okay well if you've lost a chunk of this all-star team how do you it seems like a kind of field where you you sort of can't just replace that and yet you know it was it was kind of like after that that you guys pushed forward on reasoning and some of these other spots. Yeah. So I don't know. I've intellectually had trouble. I do disagree with that as the overarching way to do good research today.
I think there's certainly a lot of top-down steering. We bet on directions, but people... OpenAid has this beautiful culture of being bottom-up in a very deep way too, where some of the best ideas just organically emerge from sometimes the most surprising of places. and um i think really the the great thing has been just like watching some of these bets unfold take shape get scaled um and reasoning being a core example of that yeah and okay so in this but like this idea that um like how star dependent are we because you still see google spend an ungodly amount of money to bring gnome back you know what i mean yeah and so this makes me think, okay, this is how this works. Yeah. I mean, I think it's a mix, right? Like you have to do, you have to invest in your pipeline because I'm very confident in our ability to create stars, but yeah, there's certainly very good people out there and everyone knows that they're good. Um, I think if there's one thing that, you know, on, on the flip side I've learned from, from meta is, you know, OpenAI can also go very aggressively after star talent. And, um, you know, there's this very aggressive recruiting approach that, you know, i've taken a couple pages from as well okay but yeah i i think it's we should always just be
trying to assemble the best team yeah uh in service of the mission that we want to accomplish it's funny because it is like a relatively small world and like all you guys hang out even though you you're like rivals and then yeah it must be weird because i know you're friends with different people on some level yeah and then you're also trying to steal all their best i mean yeah it's a brutally competitive industry in all fronts right um but again that's what i love um i'm a deeply competitive person i hate to lose and yeah yeah on research on recruiting all of these fronts um i'll work very hard on them it reminds me because i'm like a semiconductor well i'm a history nerd but just the early semiconductor days were not that far off i mean you had you had all these semiconductor startups come at once they were all pushing the limits of physics and somebody would discover something at one they'd go to the bar and have a it's like people they're They can't stop from sharing knowledge with each other, and then they're also getting pulled. It's hard. Each company is quickly getting this breakthrough in one way or another.
Yeah. You raise an interesting point of there is going to be some base rate diffusion of ideas. And I think there's two ways a company can respond to that. You can create these deep silos of like, hey, we're going to protect information in all of these ways. I don't think OpenAI operates that way. And we don't think that's the right way to operate. We just will outrun other people as fast as we can. And I love the culture of openness. People in research freely share ideas. And I think that's the way to make the fastest progress. And how do you, Sam and Jakob, now work together? I think people sometimes, um, if you read the, the announcements and everything, you can tell that Sam is research oriented over, over like day to day running of the company.
You know what I mean? You can tell research is more of his, his passion and, and even just like in the titles and in the way it's been organized, especially recently, um, you and Yaakov are so deep on this stuff and i know sam is is technical but you guys are are like in it all the time yeah and then you know sam is having conversations with everyone yeah i'm just kind of curious about this dynamic between the three of you and how i mean are you guys i mean i guess you're not always alignment on in alignment on what is going to get the resources but um yeah i was just curious about you guys. Yeah. Yeah. So, I mean, it's a very tight cohort. You know, I talk to Sam and Jakob every day. And, you know, with Sam, he loves research. He loves just learning about research.
He loves talking to researchers. I think in some ways he's very effective at getting a pulse on the research org. And I rely on him also to just, you know, are there any hidden latent problems here? Go and find them out, you know, surface them to me. Jakob and I... Like personality or techno it could be just small things like oh you know um like just even the way the office is laid out like makes it harder for this team and this team to collaborate um and the two of them is like me to collaborate to to help unlock this breakthrough that we want um i mean all these things are you know very very important and i think yakub and i we spend a lot of time figuring out how to design the work for success you know i think um pairing people with the right strengths together. Also, how to incentivize people to work on directions that we find are important. Yeah, that's a lot of the work that we do. And Sam, what? Is he reading papers? Is he chatting with you guys? Yeah. I mean, I think he does his fair share of reading papers. He talks to researchers and just understands how they think about the world, the type of research that they're doing um and of course he's responsible for a huge umbrella of things outside of that all right
i'm gonna ask some nerdy questions now or i'm gonna try to um i don't know if i can do our cash level but i'm gonna do my best and you know i'll ask i don't know how top secret some of this stuff is but but um anyway well maybe you'll slip up and we'll just we'll get it out um you know i in the the meetings i have been on and i don't think i'm revealing because we talked about a it i think i'm safe here but you know preach pre-training seems like this area where it feels it seems that um my sense is you you guys feel like you figured something out you're excited about you think this is really going to be like a major advance it was also i think either a neglected spot or something of a sore spot you know previously things were maybe working exactly how you guys had expected or hoped um you're like what can you tell us about What you figured out and, you know, some sort of frame of reference on, we've seen these periodic big leaps forward.
Absolutely.
“A lot of people say scaling is dead. We don't think so at all.”
You just made me think of a question going back to that IOI stuff.
I mean, I'm sorry, we're going to come back. It just totally popped in my head. I mean, like often I would see the kids who were amazing at those competitions, they would get hired somewhere like a Google or Facebook or something, But they weren't always like the, you know, the top executive or the top, the most famous engineer afterwards. And maybe it was like by choice, but I don't think Kennedy, it was like the Michael Jordan ended up working at any of these companies. And that totally could be by choice. I'm not trying to disparage it. But it's not clear to me, like even, okay, so it's not clear to me that the human who excels at that is necessarily like the greatest engineer you're ever going to have.
And so like, yeah, I mean, if an AI is particularly good, what are we learning? Yeah, that's the thing I quite like about working in AI. It's, I think, more so than in standard engineering culture, it is a meritocracy. in that, you know, I've tried this many times before and learned this lesson many times before, but it is hard to put in someone to lead a group who doesn't have the respect of the researchers that they're leading. And I think this is more so the case in research than anywhere else. You have to make very strong technical calls of like, you know, this is the right path when there's a disagreement, this is the right kind of project.
And if you make those calls wrong, you know you lose the respect of your researchers so um yeah one of the fun things in working with uh in ai and creating a a strong ai org is you know all my bench is very deeply technical and it's fun to talk to them about the technical things yeah yeah okay and okay on this on i'm pre-training again yeah for a second okay you know like to me in my head it feels like transformers helped kick off this massive massive leap i mean reasoning to me it feels very comparable if not even sort of more amazing i mean are we when i talked to you guys over the last few months my you know and i could never tell if this is optimism if you guys are just putting the best foot forward when i'm chatting to everyone My sense when I talk to you, to Greg, to Jakob, you know, to Sam is that you guys kind of feel like you've been putting in hard engineering work for like three, four, five years that hasn't fully manifested itself.
And so then I can never tell how excited or not to be. I mean, like when you guys are hinting at some of the stuff you're seeing, do you feel like it is, you can already tell that it is a comparable leap forward to these big epochal kind of things? I think so. You know, I think when we launched GPT-5, you know, we talked a lot about synthetic data as well. You know, there are many other threads of this form that we think are holding quite a bit of promise and that we're scaling up pretty aggressively right now. And I think it's always about maintaining that portfolio of bets, taking the ones that are providing more empirical promise and scaling and supporting them at an even greater degree.
It was like two weeks ago, Andrej Karpathy, who used to work at Open, you know, he went on Dworkesh's podcast and seemed to like deflate some giant portion of the AI industry by saying, you know, I think he was saying what, that AGI was like 10 years, 10 years off. and then when i hear and then i i heard dario talking about a week ago i mean he seems to be holding on very much to like massive scientific his, what is he called, the nation of geniuses? He seemed to be holding still on a kind of like, maybe a little slower, but like a two-year timeline on that. Yeah, when you heard what Andre said, what did you think?
Yeah, I mean, I think Twitter, they love this like cycle of, you know, it's so over, we're so back. And, you know, whatever plays into the narrative at the time, I think, you know, just becomes amplified. Yeah, I'm trying to make a clip here, man. But, you know, the way I think about it, yeah. I mean, it's like AGBI, I mean, everyone defines their own point for AGI. I think even at OpenAI, you can't get everyone in the same room and be like, hey, this is my clear definition of AGI and it's consistent. And so I kind of think about it as something like, you know, you're in the industrial revolution, right?
Do you consider the, you know, having machines make textiles? Is that the industrial revolution? Or is it the steam engine? You know, everyone kind of has their different definition. And I think we're in the middle of this process of producing AGI. For me, I think the thing I index most on is, are we producing novel scientific knowledge? And are we advancing the scientific frontier? And I feel since the summer, there's been a tremendous phase shift on that front. Okay. Like from stuff that you're seeing, the first things that are jumping to my head are all these startups that are in the biotech space that are showing, you know, one-shot antibodies and molecules, but I have no idea if that, like, what are you?
Yeah. Yeah.
“within two and a half years, we want AI to be doing end-to-end research.”
sometimes i get the sense similar sort of thing it's like we all have in our heads at least people where i sit that there's been this massive infrastructure build out that um the models seem to get better every time you 10x them that um you know there was a there was a story for a while that as you guys were going from like four to five you weren't seeing the results you wanted even though um you were getting more compute but then the more i talked to you guys, the more it sounds to me like you feel we haven't actually, that things were moving so fast back then that we haven't actually seen the moment where we made the leap to the 10X computer. I don't know if I asked that question very eloquently. Yeah, I mean, I do have a thought to share here,
“if we had 3x to compute today, I could immediately utilize that very effectively.”
even faster to the the follow-up how much do you um get involved with things like and I'm sure you're going to tell me exactly what it looks like with johnny ives device cool yeah yeah like is that is that an area that um research plays in yeah yeah it is and actually um i was just having dinner yesterday you can describe it to me if you want yeah absolutely so it looks like this well um yesterday i was uh yeah just having a dinner with johnny with with some researchers as well, our head of pre-training and also post-training. And really the way I think about ChatGPT in the future, right? Today, when you look at how you interact with ChatGPT, it feels very dumb to me. It doesn't feel very thinking native, right? And you go to it with a prompt, right? You get a response, and then it's doing no productive work for you until you give it the next prompt. And if you give it a similar prompt, you know, it's going to think for the same amount of time. It hasn't gotten smarter because you asked the first prompt. And I think the future is going to be a world where memory is going to be a much improved feature. Every time you go to Chachibuti, it learns something deep about you. It reflects about why you would ask
this question, related questions, anything. And then the next time you go to it, it's going to be that much smarter. And I think it really begs the question of how do you design a device that has this as the dominating thesis. And yeah, I thought that's been a very productive experience. Do you have one? Do I have one? I may or may not have one. What I think about when I think about you guys talking to Johnny is that like at Apple, You You have this company that was centered around hardware. It's something that Steve Jobs obsessed about all the time. It's like, you know, it's a craft. It's like an art form. Whether it's you, Sam, Greg, Jakob, whomever, as far as I'm aware, none of you guys have really done a hardware product before.
um sam seems to take design very seriously i could tell from his the buildings in his house and things like that but you know there's no sort of track record to speak of of like i always thought of steve jobs as having like taste you know and then and i've i've had a couple bosses over the years like josh tirangali used to run business week he kind of he just always struck me as this guy he just had taste you know whether it was the way something looked the way a story should be It was like this innate thing that was on this really high level. It strikes me that's kind of like what's required here.
I guess that's why you have someone like Johnny on some level. But you have to have this like back and forth. How do we know that like any of you guys have taste and are, you know, can shape a hardware product? Honestly, we don't need to have taste ourselves. And that is Johnny's job. He's our discriminator on taste. And I think actually one thing that's been really nice is just realizing that the way they work in design. and the way we work in research is there's some deep parallels there, right? There's like so much exploration and ideation and you explore a bunch of hypotheses. You take your time and then you create kind of the thing that you're happy with, the artifact at the end that you're happy about.
And yeah, it's been really nice to kind of have them fold into the company and there's just a lot more direct communication about here's the capabilities that we're going to ship and here's what the form factor looks like and how to gel them. okay and i this is like a crass way to put this but um because i spend my life adoring and talking to these people but you know sometimes i'm just like man i just don't know if a bunch of math nerds are the ones that you want making like uh the ai computer you know but i guess it is this blend that you're talking about uh yeah i mean honestly um there yeah you're right in that the people who are the best at building AI capabilities are slightly different from the people who have the best taste.
And we do have teams built of people who have really great taste for model behavior. And I think there's like a very different kind of philosophy and a very different kind of set of questions you need to keep asking yourself. One example of like a good taste question, right? Like you can imagine this being like in the model behavior interviews, is like, what should Chachupiti's favorite number be? What should its what number be? Chachupiti's favorite number. Oh, okay, okay. I'm curious what you would ask, you would answer. What I think its favorite number should be? Yeah. Well, I have a stupid answer, which is that I went to Pomona College and 47 is this like number of lore there.
Okay, okay, okay. Yeah, I mean, that's a good answer. Okay, I'm gonna let you go in a second. You've been really generous. I appreciate it. Is there, well, I'm going to ask you a question that ChatGVT told me to ask you, which is, you know, it says, like, if you look back in five years, are there any kind of, like, small, fragile, nascent ideas that you're seeing right now that your instinct is telling you might be at the heart of a big breakthrough? Yeah, there's a couple. I would say a handful of ideas. I can't go into too much detail on them, but yeah, I'm really, really excited to scale them up.
Are there any hints, any buckets of areas where these fall? Yeah, I mean, I've been concentrating a lot on pre-training, so some pre-training adjacent, a small number of ideas in RL as well, and a small number of ideas in RL. Small number of ideas of how to put it all together. Yeah, okay. All right, I tried, I tried. So you may or may not have a device. No hints? Yeah, no hints. Okay, well, we covered tons of ground. I really appreciate it. I feel like I'm letting the nerds down a little bit as far as the AI obsessives. Yeah. Are there any technical, anything you see people like kind of getting wrong about you guys at the moment that you would set the record straight on?
Yeah. I mean, I think the most important thing is just, I think anyone at OpenAI in research would tell you that it is just a research-centric company. It's a pure AI bet. At the core of the company, the ambition is to build AGI. It's to build it without distractions. And I think anything when it comes to building products, it all flows very easily from that. Yeah, when it comes to what we want to do in research, it's, you know, we want to automate AI research. I think selfishly, like, we want to accelerate our own progress, and then we want to automate scientific discovery. And, of course, we want to automate the ability to do economically useful work.
And, um, I think all these pillars are falling and, and you see that kind of the big update in the last year has just been like in that second pillar of automating scientific research. It's happening. Yeah. Yeah. How old are you now? Um, 34, about to turn 35. About to turn 35. Okay. Are you able to have like a social life or are you, are you? No, honestly not. Um, I, I, yeah, I think every day the last two weeks, you know, it's been work calls till 1, 2 AM, uh, but I, but I love doing it. It's just, um, you know, there's, there's a lot of work to get done. There's a lot of people I want to recruit.
There's, um, a lot of steering that needs to be done. And like, why waste this golden moment? It's like, if we're in the middle of something like an industrial revolution, um, you gotta take as much advantage of it as possible. Yeah. I hear stories about you sleeping at the office and. Oh, yeah. That was a fun one, too. No, honestly, it's just, yeah, I think there are times in the company, I think that was right after Barad and Mira left and went to found their own company. It just, the job demands it. And I think when I peel it all back and examine that deep emotion, it's just this protectiveness of the research.
drug that was after mira mira left yeah yeah i spent spent spent a month kind of sleeping in the office and it's just like i i need to protect the researcher they feel like you know it feels like my baby yeah so you guys have gone through these waves there's the coup everybody's trying to steal your people i guess everybody's trying to steal your people all the time but you have this inflection point mira leaves meta decides they're gonna fire up this massive lab do you think are we like are we pass it? Has everybody fired their shot at this point? You know, I have a staff meeting, right? I talk to my reports and I'm like, okay, well, here's the thing that I'm working on. And once I get back, once I'm done with this thread, I'm going to zoom out and there's no more fires.
I've fully internalized at this point, the stakes are high enough for building AGI that there's always going to be something. And I think the important thing is just being able to understand what the important things are in the midst of all of all of these these things going on do you like months have passed since there was sort of the deep seek moments or whatever i guess it was like december 2024 i think yeah yeah earlier earlier this yeah yeah yeah or january yeah i mean is there anything um now you know it felt like people lost their minds for a second um Um, I don't know. Just like reflecting on it now and seeing what they've done since.
Just like thoughts, I guess, on open source models and Chinese open source models. Yeah, I mean, I think that was one of the first points in time when I just realized how important it is that we just stay true to our research. I think when that came out, it went viral, right? Like everyone was like, oh man, has OpenAI lost its way? Are these models catching up? And what's the response? What's the response? What's the response? And I think rightfully, the thing that we did was we just doubled down on our own research program. And I don't think it was the right, it was the wrong call at all. Like, I haven't seen the DeepSeek follow up model. You know, I think they're a very strong lab. But fundamentally, like, let's just keep focusing on innovating.
I think DeepSeek was a great kind of replication of the ideas in our whole series of models, but let's just focus on innovating. Do you think 500 people is the, does that number grow as the company grows, or this is like the optimum number for kind of like big ideas you can chase at one time? No, honestly, I feel like it can be done with even less. And again, as we get AI researchers or AI interns, there's a real question of how do you design a network around that. But I'm certainly a person who cares a lot about heavy talent density. I like to run a lot of experiments of this vein. For instance, in quarter two of this year, I thought, hey, I'm just not going to open up any headcount for anyone in research.
And if you want to hire people, uh, you gotta figure out who's not on the boat. And, um, I think these kinds of exercises are quite important. You know, you don't want to work to diffuse into something that's not manageable and you want to keep the talent bar very high. Okay. I promise this is the last question. Yeah. Sorry. I have to set you free. Um, uh, the, I remember being in a meeting and And I think you and Yaka were kind of on the same page here, but I remember you for sure. Sort of like this idea of who gets attribution for a project and you seem to be of the stance that like people are obsessing over that a bit too much.
And clearly AI has its roots in academia where you are very proud when you have a paper and it's a big deal and attribution is a huge thing. Um, I think I'm remembering that meeting right in your, yeah. And, and so what we've reached a new stage where that is less of a big deal, or it's just you, this is a company and who did what is less important. I actually really love this topic. And, um, I think over fixation on credit is a very bad thing, right? I think, you know, but on the other hand, I actually feel like it's important as a company for us to recognize credit both internally and externally.
And a lot of companies have actually shied away from this. You know, we've moved away from publishing papers, credits lists, I think broadly throughout the industry. But Jakob and I ended up making the call that we're going to do it at OpenAI. And of course, the counter argument is always like, man, you're like handing your top performers on a platter you know everyone else is going to be recruiting these guys aggressively but i don't think that's important right like we should just recognize the people who are doing great work we should continue to be this pipeline for creating ai superstars and um yeah honestly it's important for us to make.
names for the people who are doing the best work at the company so but you seem to also be saying people's the individual researchers should maybe obsess about this less where or am i am i just totally misremembering this no i i think there was a sentiment in the room of of that form um actually yakub and i held more of a dissenting view on that oh okay okay Okay, okay, it's been a while. I'm sure it's in my notes. Perfect. Yeah, yeah, yeah. Yeah, but I think we've got to give credit where it's due, even at the risk of everyone knowing who our top talent is. Okay, okay. I will make an even stronger statement.
I think OpenAI is the place where we allow for the most external credit per capita. Okay. By a large margin. Okay, all right. Well, I'm going to shut my notes. I'm like, well, now I've got more. Absolutely, absolutely. Yeah. I just remember it being a topic of discussion. And there were numerous opinions. So that's funny. Okay, I lied. Last question, I swear. So, you know, you got there in 2018. I mean, it was a research company. It was a nonprofit. The company started among the founders, you know, being this counterweight to Google and with sort of, you know, making sure AGI arrived. safely was kind of the goal um you came at this from high frequency trading and and saw these interesting things happening you know like how much in your i'm sure you're going to say you want this to happen safely i get that but like if you look at your career path you're a smart curious human who saw this interesting thing happening it's not like a requirement that you like really give a shit philosophically about this or want to see, you know, a super intelligence.
Yeah. I mean, but anyway, like, let's hear from you on, on like, why are you doing this in the
“when you put incentives into the model to, you know, give you a thinking process that is appealing to a human, it won't necessarily be honest with you”
intelligent chatbot that I feel like that you can't sort of try to charm your way past and bullshit. You would do great, Ashley. You would do great. I don't know, man. I don't know. I'm feeling okay, but I'm old enough not to have to probably do that. Thank you, Mark, so much.
I know you're super busy, so thank you for your time. Thank you so much for your time, too. All right, man. It was fun. Really a pleasure. Okay. the core memory podcast is hosted by me ashley vance it is produced by david nicholson and me our theme song is by james mercer and john sortland and the show is edited by the john sortland thanks as always to brex anyone ventures for making this possible please visit our sub stack youtube and podcast channels to get more of what core memory makes thanks y'all
