Free with a sign-in.
The best 13 min of 57
Jeff Dean: The 1% Rule for Building in AI
Jeff Dean says founders can win by solving focused problems that general AI models do not yet handle well.
Lightcone · Diana Hu
- Jeff Dean is Chief Scientist at Google.
The best minutes
- 6:01Google built the TPU after estimating speech's server costPro pick 3 min
- 25:30Small teams can win where general models stay weakPro pick 6 min
- 42:04Fast learned evaluators can change the pace of science4 min
Every moment plays free. PRO marks a write-up that's in Pro.
Also worth watching
- 19:43A benchmark skill lets agents iterate on performance code3 min
- 22:08Agent search can make long-running work more reliable3 min
- 31:18Clear specs matter more when agents write the code4 min
- 36:30Revisiting reliability assumptions can produce new system designs6 min
- 47:44Distillation met a need for cheaper model serving2 min
Skip the opening predictions and search-history discussion, the latency and energy explainers, and the closing career, hiring, and future-problem advice.
Full analysis · 6 min read
Real Alpha · Nov 8
Jeff Dean says use 0% or 1% model success as a screen
- Dean says founders should first choose useful work, then seek tasks models solve 0% or 1% of the time.
- That screens for capability gaps, not demand or durable advantage.
- Measure the gap, workflow value, and how quickly general models close it.
Free with an account
Read the rest of the analysis
6 sections, the 5 other voices it tests Dean against, and what it means for you. Every episode we cut gets one.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Signed up before? The same buttons sign you in. The episode, its moments and the transcript stay free.
The 0% or 1% screen does not prove demand
Dean's model-failure test is his second selection criterion, not his first. He says founders should first choose work they care about and believe useful. Then test whether general models succeed 0% or 1% of the time, not 20%.
Partial competence can be a warning, he says, because more training or scale may improve it. A zero or one percent result suggests room, but does not automatically describe a company.
That distinction matters because polished demos blur capability boundaries. A model that handles one in five attempts may already be on a path to improvement. A near-zero result on a defined task points somewhere else.
Dean gave no benchmark, test set, or sampling method for those rates. Founders must define the job and measure failures without selecting only flattering prompts.
In November 2026, Trajectory co-founder Ronak Malde said on Training Data that Harvey's legal benchmark rose from 4% to 18%. The moment also reports six times lower cost than frontier models. That shows how domain data and measured tasks can matter, not that every low-scoring task has buyers.
Our read
Treat 0% or 1% as a reason to investigate, not proof of a market. Next test whether the failure is valuable, repeatable, and addressable.
Private data and niche models can create openings
Dean offers two routes around general models. A product can bring relevant private data into reach, or a focused model can serve a domain where general systems struggle.
For example, a personal-information tool may access data the underlying model lacks. Dean points to AlphaFold, a specialized model for protein folding, and names materials science and chip design as possible domains.
Those are plausible openings, but Dean warns that frontier models keep improving. Founders should ask whether a gap may close in six or twelve months, or persist for a couple of years.
In November 2026 on Unsupervised Learning, Nebius CTO Danila Shtan said 95% of customers modify open models. That supports the relevance of specialization, but does not show which niche will pay.
Dean is Google's chief scientist, speaking from an organization building Gemini and hardware infrastructure. His map of startup openings reflects that position, not a neutral survey of customer demand.
Our read
A narrow model or private data can create a product edge only if the advantage survives model improvement. Validate customer access and durability separately from benchmark performance.
Inference economics can shape the product itself
The TPU story began with a cost bottleneck, not a chip roadmap. Hu described a calculation in which three minutes of daily speech use per Google user could double Google's server fleet. Dean said better speech models had halved error rates, making higher usage plausible. The team needed an alternative to CPUs, so it designed hardware for low precision, dense linear algebra.
Dean said it was 30 to 80 times more energy efficient than CPUs and GPUs of the day. It also had 20 to 30 times lower latency. Those figures describe that generation, not today's hardware.
Hu put data movement at about 1,000 times the energy cost of a calculation. Dean agreed that the gap affects system design, including batching, which saves energy but can increase latency.
For example, a product promising immediate, one-at-a-time responses faces different hardware economics from a workload that can batch requests. Inference cost and waiting time belong in product design, not only infrastructure budgets.
Our read
The historical lesson is to price the bottleneck before choosing the architecture. For AI products, latency, energy, and data movement can determine which user experience is viable.
Skills and evaluators make agents less brittle
Dean's performance skill encoded a sequence engineers already used. It taught an agent to run microbenchmarks, change code, measure results, and iterate, rather than merely suggest an optimization.
He said agents can degrade when tasks move beyond their experience. Skills keep work on familiar paths, while multiple candidates and evaluators let systems discard weak attempts. That is search over solutions, but only if evaluation can distinguish promising output.
In October 2026 on MTS, Google DeepMind's Nenad Tomašev described formal specifications and provers that give math models definitive feedback. That is a strong evaluator because correctness can be checked, unlike many subjective product judgments.
In November 2026 on Latent Space, Warp founder Zach Lloyd described declarative, versioned workflows. He paired them with shared agent data as foundations for a software factory. This complements Dean's skill by showing how teams can encode processes for replay and inspection.
On Sources in October 2026, Robinhood co-founder Vlad Tenev said agent-led product work needs context and tests. He said testing changes with live customers remains a bottleneck.
Nenad Tomašev · MTS · Oct 20Clear verification helps AI advance faster in math and coding
Zach Lloyd · Latent Space · Nov 3A factory stack starts with declarative code and shared traces
Vlad Tenev · Sources · Oct 13Agent-led product work depends on automated testing
Our read
Agents become more dependable when teams package both the procedure and the test. The hard part is not adding more agents, but making their work verifiable.
Fast learned evaluators can raise experiment throughput
Dean's clearest science example was a learned approximation of a costly simulator. His colleagues trained it on simulator inputs and outputs, making evaluation 300,000 times faster while remaining nearly as accurate. The speedup and accuracy refer to that specific example.
He said a calculation that might take a night could then screen many possibilities quickly. That makes a larger search practical, but the reported speedup belongs to this particular chemistry example.
The broader opportunity is to lower the cost of evaluation. Cheap evaluation can reject weak candidates before expensive tests, fitting more experiments into the same budget.
But approximation creates a new failure mode. Teams need to map where it is reliable and check uncertain answers against the simulator or a real experiment.
Dean expects automated experiment loops to help machine learning, science, and engineering. That forecast depends on measurable objectives and evaluators fast enough to keep the loop moving.
Our read
A fast evaluator can change the scale of exploration, not guarantee better discoveries. Budget for validation against the expensive source of truth, especially before acting on uncertain results.
Clear specs make problem choice more valuable
Dean says agents need explicit specifications because they cannot reliably infer an underspecified request. As execution gets cheaper, he expects choosing the right problem to matter more than writing code.
Code translation works well, he says, because existing software and tests define expected behavior. For example, a Python implementation can guide a Go version, with tests exposing behavioral differences.
That points to a practical hiring and product question. The scarce skill may be defining the outcome, its constraints, and its tests, rather than supervising each generated line.
Dean's MapReduce example shows the value of separating the task from reliability machinery. The team moved checkpointing and fault handling below a simpler abstraction, so users could focus on the computation.
Still, good specifications do not rescue a low-value problem. Dean advises choosing work that matters and suits the team, then asking whether success would make a positive difference.
Our read
Write down the user, the desired result, and a test before expanding agent capacity. Then ask whether success would be valuable enough to justify the work.
What it means for you
- Set a capability baseline. Choose one narrow task and test frontier models on representative cases. Track success rates over time, including whether the model closes the gap.
- Price the response. Measure latency, energy, and data movement for your actual workload. Compare one-at-a-time responses with batching before making a product promise.
- Write the acceptance test. Define the expected result and an automated evaluator for one agent workflow. Record failures and the human checks still required.
- Check evaluator error. If you use a fast learned evaluator, compare its decisions with the expensive source of truth. Track both speedup and accuracy on cases it may misjudge.
What to watch
- Whether frontier models close the measured gaps founders use to choose narrow domains.
- Whether learned evaluators remain accurate enough to support larger automated experiment loops.
- Whether inference latency and energy constraints change which AI product workflows are practical.
Send this to product and engineering leaders choosing an agent workflow.
Watch the minute where Dean gives the 0% or 1% screen
Written by Real Alpha from the episode's transcript and the other moments it links, then checked against the transcript. Quotes are the speaker's exact words.
Minute by minute
A Pro write-up, open to you here. Try Pro free for 7 days
A Pro write-up. The minute by minute is in Real Alpha Pro; the episode, its moments and the transcript stay free.
Read it with Pro, 7 days free- 0:50AI agents are advancing beyond coding
Dean says models are now close to junior-engineer capability, depending on the definition, and that he underestimated how quickly they would handle more complex tasks. He sees agent systems gaining ground beyond coding and predicts that ML systems will increasingly improve themselves through automated experiments on measurable objectives.
- 2:33Inference calls for specialized hardware
Asked what could be today's equivalent of fitting Google's search index in RAM, Dean points to high-performance, low-energy inference hardware. He says specialization could reduce latency and energy use compared with general-purpose devices, and imagines what applications might be possible with latency 50 times better.
- 4:34Agents can work for weeks
Dean says capable agents can already run for days or weeks on some problems, though not everyone has internalized that possibility. He gives the example of implementing a new version of software in another programming language to improve its safety or performance.
- 5:49Napkin math led to the TPU
Dean says deep-learning speech models halved the error rate, an advance he compared to 20 years of progress in a few months. A calculation showed that three minutes of daily speech recognition per Google user could require doubling the server fleet, prompting Google to build TPUs for low-precision, dense linear algebra. He says a later chip was 30 to 80 times more energy efficient and 20 to 30 times lower latency than the CPUs and GPUs of the day.
- 9:15Find bottlenecks from first principles
Dean advises founders to question how a problem is solved today and look for ways to improve performance or capability by an order of magnitude or two. For AI systems, he says engineers should learn memory and interconnect bandwidth, energy per operation, and how network bandwidth changes when connecting 10,000 chips instead of 500. He agrees that moving data can cost about 1,000 times as much energy as a calculation, which makes batching more efficient but hurts latency.
- 16:06Context shapes the whole AI system
Dean says useful AI systems combine a model with tools, retrieval, relevant context and ways to break problems into steps and evaluate results. He recommends improving model performance by adding guidelines and tool-use skills. As an example, he and Sanjay Ghemawat built an agent skill to benchmark code, make changes and measure performance; Dean says their 30-page Performance Hints document has also helped models reason about code optimization.
- 22:08Search helps agents stay on track
Dean says agents can falter when tasks move outside the work they have learned to handle. Skills and hints can keep them on more familiar paths, while multiple agents can try approaches and an evaluator can select promising ones. He describes this inference-time search as a way to improve reliability in long-running workflows.
- 25:20Focused products leave room for startups
Dean says small teams can build excellent products for domains that general models do not serve well, especially when they can use relevant data or a specialized model. He advises testing how well current models handle the task: a 0% or 1% success rate may signal an opening, while 20% may indicate a capability that will improve soon. He points to AlphaFold and mentions material science and chip design as possible areas for specialized models.
- 31:18Clear specs make agents more useful
Dean says agents do better when founders specify what they want, because vague instructions leave agents to infer the goal. Translating a tested Python program into Go works well, he says, because the existing code and tests provide a detailed specification. With coding increasingly automated, he argues that choosing worthwhile problems—and developing the taste to identify them—becomes more important.
- 36:30Question long-held assumptions
Dean describes considering chip designs that tolerate 20 transistor errors per day instead of one every million years, using redundancy to handle failures. He says such thought experiments can reveal different design options, even when they do not prove practical. He also traces MapReduce to asking whether a common abstraction could separate simple computations from parallelization and reliability code.
- 42:04Automate the experiment loop
Dean says AI can help automate cycles of proposing, running and evaluating experiments across machine learning, science and engineering. He gives an example from quantum chemistry: a learned approximation to a simulator was 300,000 times faster and nearly as accurate, making much broader screening practical. He sees scope for models to run more of their own improvement experiments.
- 47:54A rejected paper still found an audience
Dean says a 2014 paper on distillation was rejected after a reviewer judged it unlikely to have significant impact. He and his coauthors saw cheaper, capable models as important for serving more people, so they posted the work on arXiv and others used it. He says Google uses distillation in making Gemini Flash models from larger Pro models.
- 49:56Choose work with positive impact
Dean says the choice between a large organization and a startup depends on what work someone cares about and the impact they hope to have. He recommends finding skilled, low-ego teammates with complementary strengths, and says working together can add new tools to each person's capabilities. He suggests asking whether the best possible outcome of a project would make the world meaningfully better.
- 55:01Build more efficient ways to learn
Dean names more efficient inference hardware and more data-efficient machine-learning algorithms as areas he hopes builders pursue. He notes that large models may see 1,000 times as much data as a human does by age 18, and raises continual learning and learning from actions as open challenges. He also mentions multi-agent interaction and tools for better discourse and helping people meet others with shared interests.
Published Jul 30, 2026.
Transcript · 39 min read
Lightcone, hosted by Diana Hu, with Jeff Dean.
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
all right should we go should we get started jeff sure sounds great all right jeff welcome and again thank you so much for being here especially i just got a cold and thank you for yeah i'm afraid i've lost my voice i don't normally sound quite like this but we'll we'll do what we can So you built MapReduce, Bigtable, TensorFlow, the TPU, Gemini. We could spend a whole hour on all the things you've done. But what I love is that you're still making bold predictions in public. Last year, yes, last year in May 2025 at AI Ascent, you said that AI is at the level of a junior engineer. That was about a year ago. How close are we to that prediction? Yeah, I mean, I feel like the models have been getting a lot better at sort of agent-based, longer-running coding tasks, and it seems pretty clear that they are now actually pretty capable, and depending on exactly your definition of junior engineer, it seems pretty spot-on, I would say. What did you underestimate from that prediction? I mean I think the the ability to do more and more complex tasks has been growing faster than I thought and I also think outside of coding these these agent based systems are really starting to shine in other domains and I think you know that's that's going to be an
important trend in the future. So give us another bold prediction. What do you think is going to be the 2027 edition? I think you will see a lot more automation of ML systems themselves. Basically getting ML systems to improve their capabilities by running lots of experiments, breaking things down into sub-problems, you know, running those sub-problems in a tight automatic experimentation loop, putting the results together, and being able to then, you know, get some improved system out from that sort of fully automated problem decomposition and automated experimentation. I think that's going to be really exciting. I think that also applies not just to ML, but also to other fields of science and engineering. Basically, anything where you can have a measurable objective, I think you can actually make a lot of progress these days.
now let's go back to a little bit in history back in way back in 2001 google search used to run on hard drives yep and you and sanjay did the math and realized at some point the whole search index would finally fit in all of the ram of all the computers you had running and you made that radical realization and you basically in a few days with Sanjay shipped in production a whole new search version that worked in RAM rather than hard drive and that was the thing that got Google to be so fast Google searches so history tends to remix what is the it fits the memory moment right now in 20 2026 that everyone in this room is still should be thinking about and designing yeah i mean it's a little different but i think uh you're gonna see more and more uh uh high performance and um low energy uh inference hardware systems because i think everyone is now realizing that inference is the key to making you know these agent-based systems be available to more and more people and That latency is really important, and that specialization of the hardware is a really key way you can make things that are more energy efficient and lower latency than more general-purpose computational devices like, say, GPUs or TPUs.
Read the whole transcript
Because I think everyone here is used to waiting for responses on models. Yeah, waiting's no fun. Master of speed. So you're saying, what if we don't have to wait anymore? Yeah. I mean, I think, well, imagine what you could do with something where the latency is, you know, 50x better. Hmm. Interesting thought. Now, what's one assumption that perhaps 6,000 people in this room hold as already false about AI? Yeah, that's a good question. I mean, I think probably one thing is people don't quite realize how possible it is to have, you know, agent-based systems that can run not just for an hour or two hours on a problem you care about, but for some problem domains and with highly capable models underlying them, you can get them to run for days or weeks and do really, really complicated tasks.
And I think some people are starting to see inklings of this, but I don't think everyone has really internalized this, and that's going to be really a pretty big deal. What's a particular task that you have run that has run for weeks? What did you tell the agents to solve? Yeah, I mean, I think you can tell agents to go off and implement completely new versions of software in different programming languages that might have better safety properties or better performance properties, and then they can go off and actually do that in a pretty serious way. That's pretty cool. Now, one thing that you've been very well known for is you're really good at napkin math.
“that was like the equivalent of 20 years of advances in speech recognition in just a few months of like fiddling with the model”
Incredible napkin math. So what's a good napkin math that everyone here who wants to be a future founder should run tonight to potentially build something as consequential as the TPU? Yeah, I mean, it's always hard to say. I think, think about what problems you see and whatever it is you're thinking about, what bottlenecks you see, and are there very different ways of thinking of the solutions to some of those problems that would get you an order of magnitude or two orders of magnitude better performance or capability or whatever it is? Because sometimes if you just squint at a problem and you think about not necessarily being anchored on exactly how that problem is solved today, but how you would solve it from first principles, you can come up with really good ideas that are, you know, maybe not what other people are thinking about. That's a good tip. Now, for everyone here who doesn't know, years ago, Jeff wrote a very famous list called the latency numbers every engineer should know.
And these are numbers around, for example, how long a cache miss takes, uh disk seek a network package traveling let's say from california to netherlands um lots of numbers like this about distributed systems and systems engineering and it's been sort of taped and become the bible for a lot of distributed systems engineers okay yeah now fast forward that list is up for an update give us the ai edition for now 2026 yeah i mean i think if you looked at what is important in AI systems these days you would want to know things like the bandwidth between you know your main memory system on your accelerator to the on chip memory to the you know the multiplier unit or whatever you want to know how much energy does it take to do a single multiplier operation you know what is the interconnect bandwidth between chips and how much does that, how many chips can you connect with that bandwidth and then if you go beyond that domain, like what is the fall-off in network bandwidth when you need to talk to 10,000 chips instead of 500 or something. I think these are all really important numbers to learn.
and really affect how you think about solving particular kinds of problems. And one interesting thing that I've heard you talk about is that nowadays the unit that you measure everything is energy. You pointed out that doing a calculation or math costs about one picajoule, but moving the data and doing data I.O. costs a thousand times that. Yeah, just bringing it in from HBM on an accelerator into the processor so it can actually compute on it. Yep. That gap kind of quietly decides what products are possible and how these algorithms in AI are built. So what are the kinds of problems that founders keep calling model problems, but are in fact actually energy or data I.O. problems?
Yeah, I mean, I think the example you raised of a thousand X difference in bringing moving data versus actually computing on it in terms of energy is a pretty significant one. And it shapes a lot of aspects of what we do in machine learning. Because if you didn't have that thousand X difference, then, you know, you wouldn't have to do batching. But you have to do batching of many examples or maybe many tokens at once in order to amortize that data movement so that you can not pay 1,000x slowdown, but pay 1,000x divided by batch size energy cost. And for really low latency, batching is not really very good.
So I think these kinds of things and the energy behind various decisions in the computer hardware we use really affects a lot of decisions we make in building higher-level systems. A very concrete example is just how training models is done. There's this whole concept of batching the data sets and running epochs. That's basically, people perhaps may confuse that as a model problem, but it's really a systems data I.O. problem, right? Yeah. Yeah. I mean, you have to assemble batches to get better efficiency in your hardware. You know, ideally, you might do batch size one training, but, you know, it's not as good in terms of efficiency.
So people use pretty large batches these days. Do you think it's possible for, I know you're well known for taking off on a long week or weekend and coming up with this brilliant solution. Is there such things of Jeff Dean going and working on it for a couple of weeks and getting batch size equals one training done? Yeah, I've been thinking more about inference, actually. So I think inference is a pretty interesting problem because you do want very low latency. You know, training, you don't necessarily need incredibly low latency. And I think there's a lot of room for specializing hardware more for inference than we are today.
What are some of those interesting things that are on inference that you're really thinking a lot about? I mean, just trying to minimize data movement, trying to think about incredibly low precision operations, and maybe not supporting lots and lots of different kinds of precisions. if you feel like you have a good answer for what kinds of precision you need maybe just build that into the hardware and and not much else which i think it brings down to a core analogy i heard from famous computer scientists that really the whole process of ai is a big compression problem because in order to have the data to be fully lossy and compress it and then restore it, you basically need to understand it.
Yeah, I mean, if you truly understand the data, you should be able to compress it really well. And... Now, Transformer architecture is basically one of the ways that has turned out to work really well. Yeah, yeah, I would say. Working pretty well so far. Good work by my colleagues. Yes. Now, let's zoom out a bit. AI progress used to mean just better models. You had more data trained, bigger models with bigger parameters. But increasingly in the last years or so, it's everything around the model, not just the model size and number of parameters or more data. It's everything around things like retrieval, tools, memory, agent tools, and it might kind of get consolidated into what people call context engineering, right? Yeah, I mean I think the model is really only one piece of what you're trying to do, which is build an overall system that can solve really interesting problems.
And that involves a model that knows how to use various tools. It maybe knows how to retrieve relevant information, maybe has a history of other information that has retrieved for past problems. And it can put information into the context of the model. And the nice thing about that is that information is really clear to the model. unlike the training data the model was trained on where it's all kind of like trillions of tokens stirred together into a soup of hundreds of billions or trillions of parameters but it's all less clear than the actual context that the model sees directly for this particular problem or use case and then I think being able to understand what tools are available which ones are going to help the model solve this next phase of the problem, how to decompose the problem into a sequence of tool calls, maybe trying multiple approaches to solve the problem and seeing which ones work and be able to evaluate that.
This is the whole orchestration of complex agent and multi-agent systems that I think is going to be more and more important and super exciting times, I would say. And I think the fun thing about this particular problem domain set is actually something that everyone in this room can actually do because before to train a model you needed incredible amount of resources incredible amount of access of to gpus and data but for context engineering everyone here could do it you have you just need the api to something like gemini and then work on your own setup for your own retrieval your own tool calls and etc et cetera. So what are some tips for everyone here? How does everyone get better at and become exceptional at context engineering? Yeah, I mean, I think a really good way to do it is to use these models and sort of harnesses and tools and so on to try to solve problems.
And then sometimes you can actually see where the models are failing. And often you can actually make the model work better and succeed at that kind of problem by not just adjusting the model parameters which is hard to do from the outside but from you know creating better guidelines for the model you know writing skills for the model to know how to use different tools that would be incredibly useful for solving this particular class of problem and I think as you do that you end up on this kind of improving self-improving of the setup that you're trying to use to solve things and you know that that's a really good way to get better at understanding what what additional information the model would want in order to become more capable. Can you give an example of some context
“self-improving, benchmark measurement, benchmark code changes, measure the performance improvement, and then iterate on that.”
“stick to the ones that seem most promising and discard the ones that didn't seem to work”
This section is one that I personally care a lot about because also everyone here in this room needs to decide what to build in the future of your future founder.
“look for something where the model succeeds 0% or 1% of the time, not 20”
accurate, but niche model to do things that are hard today. That's a good example. So if some of you find a problem that's similar shape like AlphaFold
“it can sort of take all the tests that are in Python, make sure they pass in the Go version”
a lot about right now in this current era with agent encoding. How do you exactly build taste and do that? I mean, yeah, that sounds so esoteric. How do you make it concrete? Yeah, I mean, it is a difficult thing. It's not like there's a measurable objective of taste in a lot of cases. I think some of it is from experience. You know, working on a lot of different problems in the past kind of teaches you about what kinds of problems might be interesting in the future or what kinds of things might be just barely possible by cobbling together these previous approaches and then some open problems you might have to work on in order to get to something kind of magical or highly useful.
Another way you can get more experience for yourself is to just write down a bunch of things you think might be important in the next 12 months. And maybe you pick one of them to work on, but go back and evaluate in 12 months of these other things, which ones actually seemed important or which ones did other people in the world go out and create and which ones did not seem to do yet. But that can give you a lot more samples for your own sort of taste creation capability. And that's an important skill to have.
“what would happen if you tried to build a system out of transistors that might have 20 errors per day rather than one every million years”
“they made something that was 300,000 times faster and nearly as accurate as running the full-scale simulator.”
What are some of the spaces and problems that you're super excited that this super sped up scientific method is going to solve or achieve? What particular problems or spaces?
Yeah, I mean, I think, well, clearly machine learning itself is one. So can we have a model that is able to recursively self-improve itself by running lots of experiments? And, you know, if you think about how models are improved today in large research teams, you know, what usually happens is people think of some ideas, they run a bunch of small-scale experiments, they see if those small-scale experiments worked out well, if so, they take the most promising ones of those and they try them at larger scale, and that gets then evaluated, and then the results get integrated together into a new recipe for your model.
But I think there's no real impediment to making that be a much more automated loop where the model itself decides it's going to explore or maybe with a nudge from some people at the various highest level, like, oh, why don't you try some new ideas around model architectures that incorporate this? And then it will go run lots of experiments, see which ones work, and then those will get incorporated at a much more rapid rate. And effectively, you want to optimize your discoveries per unit of compute input. very cool yeah now going back to the room as all of you will become at some
“we actually saw this was a super important problem because we knew making cheaper, highly capable models from larger scale models was something we desperately wanted to do”
um now i think the fun thing is that you basically join when you when you join google's a 20-person startup back in 1999 now if you were to take the young jeff dean from way back then to teletransport him to now, today, in this era, with your skills.
I'm feeling so vigorous and young now. What would you do? Do you join a frontier lab, start a company? I don't know. What would you do, Jeff Dean, today, 25-year-old Jeff Dean? Yeah, I mean, it's always hard to say. It's a very personal choice of what it is you want to spend your time on. To me, some of the most important questions are, are you going to work on something you really care about? Will your working on that, and if you're able to make progress on it with a bunch of colleagues you like working with, if you're able to collectively solve it or make progress on it, will that make a difference in the world in some positive way?
Right. Like, will you suddenly be able to do something and offer that service to, you know, maybe it's a very niche thing, but it will tremendously help biochemists or something. Or maybe it's a broader thing. It'll help programmers or it will help all consumers on the Internet or other things. um what you're you know what you should strive to do is to have impact in the world that is positive and to work with people you enjoy working with and to you know uh work hard and and do your best um so in terms of say the particular trade-off you offered joining a frontier lab versus say starting a company with just one or two or three of you and your close friends um i think those are different experiences, right? In a, in a large established organization.
You have some structure, you have lots and lots of amazing colleagues who know lots of things you don't, you have lots of interesting problems that you can work on, and you already have a platform for impact by your work, you know, influencing lots and lots of people in the world already. And then as a very small startup, you know, you have to have something you're passionate about. And there's a lot of risk in working on that particular problem in a way that you're going to succeed and you're going to grow an endeavor in order to do that. But that can also be incredibly rewarding, I would imagine. So I think it's really up to personal taste. But at the very least, regardless of what path you take, ask yourself, if I work on this problem and the best possible outcome happens you know will the world be a lot better in some way or will the world go yeah that's kind of cool but whatever that's not the kind of thing you should spend your time on now let's talk a bit about more about that second path of working with people that you really like in a small team you've been able to be an incredible mentor and manager to many many engineers and you've been able to build huge systems what are some some of the lessons
for everyone here on how to get the most and how to work with smart people or find smart people yeah i mean you always want to find people who have really good skills in some some area that's needed in, you know, a team you're trying to form, whether that's inside a company or starting a company. But you also want to find people that are people you delight being around, right? Because you're going to spend a lot of time around people working on really hard problems. And you want people who are low ego, that are team players, that, you know, have complementary skills to your own perhaps. I always find working in a small team where people know things that I don't know and where maybe I have some skills that other people don't have as much of, you know, is super fun because you're collectively building something or working on something that none of you could maybe do individually. But in the process of working on that, you actually gain a lot of new knowledge and new skills for yourself and so do they and you kind of want to view your engineering or research career as you have an amazing tool belt of techniques and you always want to be adding new tools to that tool belt because you never know when you might come across a problem where you need
these four specialized tools rather than these three and adding more tools makes it more likely that the problems you you encounter in the future will be solvable by you. Hmm. Now, one last thing. I'm pretty sure someone in this room or multiple people will eventually build something as consequential as you've done with MapReduce, TPU, distillation, et cetera, et cetera. What problem do you hope they would be working on? Oh, yeah. I mean, I think there's a lot of interesting problems in the world and I'll just rattle off a few. this is not exhaustive because the world is a very big place and full of problems.
You know, I'm particularly excited about new approaches to hardware. You know, we've had thought experiment there was kind of, you know, a, you know, indication of that or much more efficient inference hardware. You know, I think there are radically different kinds of algorithms for machine learning that might be much more data efficient than the approaches. we're using today. If you think about our large-scale models today, they probably see 1,000 times as much data as a human does by the age of 18, yet the human by the age of 18 is better in a lot of things and on par with those frontier models that have seen way more data.
So could you come up with much more data-efficient systems that can learn continuously, learn from their own actions? Continual learning is a really interesting thing. I think multi-agent interactions is an interesting thing you know I think you know creating ways of having better discourse among people in the world could be interesting are there ways to have much more civil conversations and you know helping people meet other people are all over the world that they should know based on their interests you know these are kind of interesting things I think there's lots of cool things in the world and we should all go and strive to make even cooler things occur that sounds wonderful thank you so much jeff dean that's all we have today thank you appreciate it thank you all
