# The Trillion-Dollar AI Race to Replace Us (with Garrison Lovely) Markdown copy of https://realalpha.app/v/NdIrGMpWOZw/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy. Garrison Lovely argues that AI development is a race to replace human labor and should require safety and public consent. - Show: Future of Life Institute Podcast - Hosts: [Gus Docker](https://realalpha.app/people/gus-docker/) - Guest: [Garrison Lovely](https://realalpha.app/people/garrison-lovely/) is author and journalist. - Published: Oct 13, 2026 - Length: 1 h 20 m ## The best minutes - [8:04](https://realalpha.app/v/NdIrGMpWOZw/?t=484) [Replacing workers can weaken democratic accountability](https://realalpha.app/s/replacing-workers-can-weaken-democratic-accountability/) (3 min) - [32:17](https://realalpha.app/v/NdIrGMpWOZw/?t=1937) [AI training costs matter more than serving customers](https://realalpha.app/s/ai-training-costs-matter-more-than-serving-customers/) (6 min) - [1:02:24](https://realalpha.app/v/NdIrGMpWOZw/?t=3744) [A mass movement can push leaders beyond its original demands](https://realalpha.app/s/a-mass-movement-can-push-leaders-beyond-its-original-demands/) (6 min) ## Also worth watching - [27:06](https://realalpha.app/v/NdIrGMpWOZw/?t=1626) [AI exposure is narrowing entry-level hiring](https://realalpha.app/s/ai-exposure-is-narrowing-entry-level-hiring/) (4 min) - [20:40](https://realalpha.app/v/NdIrGMpWOZw/?t=1240) [AI task horizons have doubled every three to seven months](https://realalpha.app/s/ai-task-horizons-have-doubled-every-three-to-seven-months/) (6 min) - [56:02](https://realalpha.app/v/NdIrGMpWOZw/?t=3362) [Trust and local effects shape data-center opposition](https://realalpha.app/s/trust-and-local-effects-shape-data-center-opposition/) (5 min) - [39:43](https://realalpha.app/v/NdIrGMpWOZw/?t=2383) [Two market shifts could erode frontier AI margins](https://realalpha.app/s/two-market-shifts-could-erode-frontier-ai-margins/) (5 min) - [16:48](https://realalpha.app/v/NdIrGMpWOZw/?t=1008) [Labor replacement connects job concerns to AI safety](https://realalpha.app/s/labor-replacement-connects-job-concerns-to-ai-safety/) (3 min) Skip the opening definitions and capability forecasts, the remaining AI-bubble discussion, and the closing general advice on organizing. ## Full analysis: Garrison Lovely says close to 20% relative youth jobs/openings drop - Lovely cites a close to 20% relative decline in jobs or openings for the youngest workers in AI-exposed fields. - That is one early hiring signal, not proof of AI-caused broad layoffs or full-role replacement. - Track junior hiring alongside benchmark progress, serving costs and productivity. Its 7 sections, what it means for you and what to watch are on [the page](https://realalpha.app/v/NdIrGMpWOZw/#analysis), free with an account: - [1:37](https://realalpha.app/v/NdIrGMpWOZw/?t=97) Lovely defines AGI through labor replacement - [28:06](https://realalpha.app/v/NdIrGMpWOZw/?t=1686) The first labor signal is missing junior hiring - [22:44](https://realalpha.app/v/NdIrGMpWOZw/?t=1364) METR tracks task duration, not whole-job readiness - [35:56](https://realalpha.app/v/NdIrGMpWOZw/?t=2156) Serving models can pay while training consumes capital - [40:40](https://realalpha.app/v/NdIrGMpWOZw/?t=2440) Two risks could squeeze frontier AI margins - [56:55](https://realalpha.app/v/NdIrGMpWOZw/?t=3415) Political resistance can constrain AI expansion - [1:02:30](https://realalpha.app/v/NdIrGMpWOZw/?t=3750) Public pressure can push leaders beyond initial demands ## Summary Garrison Lovely is an author and journalist. He wrote Obsolete, the AI industry's trillion-dollar race to replace us and how to stop it, and says he recently joined the board of Irreplaceable. Lovely defines the AI industry's AGI goal as building machines that can replace human labor, and argues that doing so could weaken workers' economic and political power. He connects labor replacement to AI safety and extinction risks, while saying the transition could be destabilizing even without a rapid takeoff. He discusses the economics behind AI investment, early signs of labor-market effects, and growing opposition to data centers. Lovely argues that AI development is not inevitable and calls for safe development with public consent, backed by organized political action. ### Minute by minute - [0:56](https://realalpha.app/v/NdIrGMpWOZw/?t=56) **AGI means universal labor replacement.** Lovely defines AGI as a system that could take the place of a person in a remote-capable desk job. He says current models cannot handle whole jobs end to end, but capabilities can cross important thresholds quickly. - [5:05](https://realalpha.app/v/NdIrGMpWOZw/?t=305) **Decades-long timelines still matter.** Lovely says he has longer timelines than some people at AI companies, but argues that labor replacement within the next few decades is enough to demand preparation. He points to companies' claims that their addressable markets include human wages as evidence that their ambitions go beyond helping workers. - [8:04](https://realalpha.app/v/NdIrGMpWOZw/?t=484) **Labor replacement threatens political leverage.** Lovely argues that workers' labor gives them economic and political power, and that losing it could weaken democratic accountability. He says development should depend on demonstrated safety and public buy-in, and warns that autonomous weapons could also remove a means of resistance to dictatorships. - [16:34](https://realalpha.app/v/NdIrGMpWOZw/?t=994) **Labor replacement links to extinction risk.** Lovely says AI safety discussions often focus on superintelligence and extinction while overlooking the labor-replacing systems that could come first. He argues that the ability to use machines to do work in the world is part of what could make AI dangerous, and that job displacement could be highly destabilizing in its own right. - [19:45](https://realalpha.app/v/NdIrGMpWOZw/?t=1185) **AI task horizons are lengthening.** Lovely calls the METR task-horizon chart the least bad single benchmark for comparing models over time. He says the duration of tasks AI can handle has been doubling every three to seven months, while noting that measuring longer tasks reliably is technically difficult. - [23:49](https://realalpha.app/v/NdIrGMpWOZw/?t=1429) **Benchmarks miss human judgment.** Lovely says automatically graded benchmarks can overstate AI capability and miss the open-ended work where human creativity, taste, and intuition matter. He says Claude Code gave useful feedback on his book but produced poor suggestions when he asked for cover ideas. - [27:06](https://realalpha.app/v/NdIrGMpWOZw/?t=1626) **AI may squeeze entry-level hiring.** Lovely cites research finding a close to 20% relative decline in job openings or jobs for the youngest workers in the most AI-exposed occupations. He says companies may rely on experienced workers using AI instead of hiring junior staff, creating an apprenticeship gap; by 2034, he does not expect people born in 2012 to compete for remote-capable white-collar jobs by default. - [31:47](https://realalpha.app/v/NdIrGMpWOZw/?t=1907) **A bubble can leave lasting technology.** Lovely says a financial bubble would not mean the technology lacks capability or transformative potential; past bubbles left technologies that later became widely valuable. He contrasts the dot-com era's debt financing with AI investment funded in part by profits from established tech businesses. - [38:20](https://realalpha.app/v/NdIrGMpWOZw/?t=2300) **Training costs drive the business model.** Lovely argues that serving AI customers can have positive gross margins even while companies spend heavily on training new models; he cites OpenAI's plan to burn $130 billion before becoming cash-flow positive. He sees risks if costly new models stop improving or open-weight models become good enough to undercut frontier-model prices, and says a bubble is no reason to assume the risks will resolve themselves. - [47:18](https://realalpha.app/v/NdIrGMpWOZw/?t=2838) **Public response may lag internal progress.** Lovely worries that AI could automate research and development and advance faster than the public can organize a response, especially if the strongest models remain internal. He supports legislation requiring companies to disclose information such as how much code AI generates or reviews and what their most capable internal models can do. - [53:44](https://realalpha.app/v/NdIrGMpWOZw/?t=3224) **Data centers make AI opposition tangible.** Lovely says opposition to data centers often reflects local effects and a lack of trust in the companies, not just concerns about water use. He argues that a data-center moratorium might slow development without addressing the underlying race, and favors more direct limits on building AGI. - [1:00:40](https://realalpha.app/v/NdIrGMpWOZw/?t=3640) **Mass movements can shift political choices.** Lovely recounts how the nuclear freeze movement grew from a 1980 manifesto into a major movement, with the largest protest in US history in Central Park in 1982 and nuclear freeze becoming the country's top issue in 1983. He says the campaign helped lay groundwork for later arms reductions, while acknowledging that AI has stronger economic incentives behind its development. - [1:08:04](https://realalpha.app/v/NdIrGMpWOZw/?t=4084) **AGI development is not inevitable.** Lovely says people can choose not to build technologies, citing limits on cloning, a moratorium on recombinant-DNA research, and restraint around nuclear weapons. He argues that AI should be developed only when it is safe and has public consent, and that people can rally around stopping the race to replace workers. - [1:16:26](https://realalpha.app/v/NdIrGMpWOZw/?t=4586) **Collective action needs local levers.** Lovely urges people to learn about the issue, find others who share their concerns, contact elected officials, and join organizing efforts such as Irreplaceable. He says countries without frontier AI companies can also apply diplomatic pressure and help develop verification tools and frameworks for international agreements. ## Transcript Future of Life Institute Podcast, hosted by Gus Docker, with Garrison Lovely. Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there. [0:00](https://realalpha.app/v/NdIrGMpWOZw/?t=0) The vision of what the industry is trying to build, in my mind, is universal labor replacing machines. The industry calls this artificial general intelligence. Rendering us obsolete means disempowering people because their labor power is a source of economic power and political power. And so if people lose that because they are replaceable, then you lose a mechanism of democratic accountability. I don't think anything I'm arguing is predicated on this happening very soon. Just that it could happen within the next few decades is enough to make it the most important thing in the world. When and how that happens could matter enormously. And I'm not against building it at some point. I think it should just be when you know you can do it safely and when there's actual public buy-in. So you can make the case to people and they're making an informed consensual decision to bring this into the world. Welcome to the Future of Life Institute podcast. My name is Gus Stocker, and I'm here with Garrison Lovely. Garrison, welcome to the show. [1:01](https://realalpha.app/v/NdIrGMpWOZw/?t=61) Great to be here. You are the author of Obsolete, the AI industry's trillion dollar race to replace us and how to stop it. So what's the difference between the vision of AI that's being sold to the public and then what the actual goal of the AI industry is? The vision that's sold to the public, it's kind of a mess. It's like, okay, AI will help you do your work better, or AI will help you automate your workers away, or AI might cure cancer in the future. And it depends on who they're talking to and who's talking. The vision of what the industry is trying to build, in my mind, is universal labor replacing machines. [1:45](https://realalpha.app/v/NdIrGMpWOZw/?t=105) The industry calls this artificial general intelligence, and there's a lot of debate about what that means, whether we already have it. But my definition is like, could you, instead of hiring a new human for any given desk job that's remote capable, could you just hire or use an AI model instead? And so by this definition, we clearly don't have AGI or universal labor replacing machines because they can't plan a wedding end to end. They can't learn from experience. They can't seamlessly plug into any meeting and like catch everything that is going on and do this in real time the way a human can. [2:23](https://realalpha.app/v/NdIrGMpWOZw/?t=143) But the goal is to build machines that can do that. And OpenAI defines AGI as a highly autonomous system that outperforms humans at most economically valuable work. And Anthropic CEO Dario Amodei has said that AI is not a just normal technology. It's like a general substitute for human labor. and so I think that's an honest description of what they're trying to do and I think people should just be aware of that there is sort of a spectrum between AIs that can sort of you can hand off some of your work too which is clearly what we have right now and and maybe you can you can hand off increasing amounts of of your you're just job in total. But then that ends up with your job being replaced. And so is it sort of a spectrum between helping people do their job better and then replacing people? I think it's, yeah, it's really just about the capabilities. So right now, the AIs are not good enough to do most jobs end to end. They can do many tasks, an increasing number of tasks, but they're just not there yet. [3:34](https://realalpha.app/v/NdIrGMpWOZw/?t=214) And so when people say like, oh, we haven't seen a jobpocalypse, so therefore we won't see one, this is just missing the point. It's like, no, we aren't at the threshold that people have been worried about for a long time. And so, and one other point I make throughout the book is, you know, AI progress, if you look at charts of benchmark scores, it's like remarkably smoothly exponential, you know, doubling every some number of months typically. But when we experience that progress, we experience it as these big tipping points where it's like the LLMs can't really handle tasks they've never seen before to like suddenly they can do OK at them to AIs can't write code very well. [4:15](https://realalpha.app/v/NdIrGMpWOZw/?t=255) And now they can or AI agents just don't work well enough to be worth using. And now they do. And so this can happen very, very quickly once these crucial thresholds are reached. I think this threshold effect is worth keeping in mind, because it can be true to say one month that, okay, AIs can't do math. And a year later or so, AIs can actually do math and can do coding. And so it is for many other tasks. And as we get closer to AGI, capabilities will seem to suddenly spike in various domains. And it can be true to say that we do not have AGI by the definition you just gave, but we might be quite close to it anyways, because capabilities can, from our perspective, suddenly cross a threshold and then we are in a different situation. [5:18](https://realalpha.app/v/NdIrGMpWOZw/?t=318) I'm not actually sure how close we are. I want to say that. I think that I have longer timelines than a lot of the people working at these companies who think that we'll have AI that can automate its own research and development in six months or 18 months. And I don't think I think it's very easy to underrate how hard it is to do the last mile of a given problem. And in the book, I talk about how like I almost finished my book a dozen times and yet it wasn't done. And I think similarly, like you just won't be thinking about some piece of it that's actually crucial. And to get the really, really crazy takeoff that people talk about, you really need to automate not just 99.9% of the process, But the whole process because a rate limiting step could just make it go a lot slower I don't think anything. I'm arguing is predicated on this happening very soon Just that it could happen within the next few decades is enough to make it the most important thing in the world. [6:22](https://realalpha.app/v/NdIrGMpWOZw/?t=382) Even if we talk about this happening within the next few decades, our economies, our political systems are just not set up to handle that. So even quite long timelines by San Francisco's standards give us a situation in which we have to handle a lot of things in not a lot of time. Totally. When you hear the CEOs talking about using AIs as tools to help people in their job, making people more productive and so on, that seems to be a correct description of what's actually happening up until now. So what is it that gives you, why is it that you say that they are actually gunning for replacing workers? [7:07](https://realalpha.app/v/NdIrGMpWOZw/?t=427) Those quotes I gave for the definition of AGI, Dario's definition of AI, but you also see these financial reports where it's like we think the total addressable market of our business is 30 trillion dollars anthropic said this in their filing before their ipo and it's like that's more than the us gdp um so that is coming that's like white collar wages basically um and andreason horowitz has also said like our total addressable market is like all all human wages um so there's that ambition. And just the aspiration for AI, for AGI, for a long time has been creating a mind that can think like our own, that can do whatever we can do as well or better. And I don't think that's the only type of AI that we could pursue, but that has been what the industry is oriented around. And yeah, the book is called Obsolete because they're trying to render us obsolete. [8:05](https://realalpha.app/v/NdIrGMpWOZw/?t=485) And they'll push back on this in various ways. But I think you can just look at the record of what they say and the way they behave and the amount of money that's going into this and i think it's a pretty clear picture now one could object what's so bad about being obsolete in the economic sense what is so bad about being sort of a child for example or being a pensioner being someone who doesn't work but there are many examples of of people who do not work in our society that are still living great lives and why can't we all end up in that situation maybe eventually if you really trust the people in power um and you trust the democratic processes to take care of everybody you could have some kind of like species-wide retirement um that people are like opting into in this meaningful sense um i don't trust the people running the world right now to do this or our institutions as you mentioned to be to be able to handle it [9:10](https://realalpha.app/v/NdIrGMpWOZw/?t=550) **Garrison Lovely:** And I think that rendering us obsolete means disempowering people, because their labor power is a source of economic power and political power. And so if people lose that because they are replaceable, then you lose it. [9:28](https://realalpha.app/v/NdIrGMpWOZw/?t=568) mechanism of democratic accountability. And, you know, there's very scary things within that too, beyond the big picture. It's just like, you know, if soldiers are rendered obsolete, because you have lethal autonomous weapons that can be directed by a single individual to kill protesters, then one of the key ways that dictatorships have historically fallen is taken off the board. And that's also on the table. And the US federal government has expressed strong interest in pursuing those types of technologies. And yeah, so I think about who's in charge right now. But also, this would just be an incredibly difficult thing for any society to handle. And I think it's kind of like if we play our cards right as a species, we can have some AGI as a treat, where it's like, yeah, we are consciously opting to do this so we can have lives of leisure and abundance and whatever. But I think that's why we should condition development of it on not just having a scientific consensus that it can be done safely, but also by having strong public buy-in that it's what people actually want. And that would force the developers, the proponents of building it to make their case and then explain how people will be better off [10:45](https://realalpha.app/v/NdIrGMpWOZw/?t=645) as a result of being rendered obsolete. To some extent, I buy that political power is dependent on economic power. We as citizens and people with jobs, we pay taxes. And so the governments have to respect our rights and they have to let us influence what happens in the world. And perhaps if you are reliant on the government, that sort of relationship breaks down where the government sees you're more of an expense and perhaps your rights are not respected. On the other hand, it seems like there are many systems set up where we could, for example, cut off pensioners right now if we wanted to and treat them as sort of unproductive and therefore not worth paying any money to. [11:42](https://realalpha.app/v/NdIrGMpWOZw/?t=702) But we don't do that. And similarly, there are many countries in the world that could be invaded by China or by the United States, but there are limits to how much that is happening. Could there be a more humanistic ideal that would prevent us from losing political power simply because we lose economic power yeah i mean it's it's a good point and like you know old people vote uh and so that's a big block of political power and it's true that you know cutting social security and medicare are like two of the things that are least popular in the united states um and you know humans that are not yet retired are like looking forward to those benefits as well um and so i don't think it's like as soon as you lose your economic power, you automatically lose all of your political power as well. But it's a big lever that you no longer have access to. So the striking worker no longer has power in a world where they can be replaced totally by AI. But a billionaire has even more power because they can turn their capital directly into labor. And that's been the dream of owners and capitalists and bosses for basically before computing even existed. Charles Babbage, one of the fathers of digital computing with Ada [13:10](https://realalpha.app/v/NdIrGMpWOZw/?t=790) Lovelace, he was really interested in automating work so you could discipline, control, and to some extent replace workers. Because in the 1830s in England, there was this question of how do you get good work out of free labor? These people who were not slaves, and he opposed slavery to his credit but he was like trying to make sure that they could actually discipline these workers as well and so it's kind of interesting how long of a project uh what i call the obsoleting project like how long that's been something people working with computers uh before there were computers have been pursuing how is that project going um sort of today depends on who you're asking I mean, the Obsoleting Project is the best funded project in history. [14:03](https://realalpha.app/v/NdIrGMpWOZw/?t=843) It is substantially more money going toward it than the Manhattan Project, than the Apollo program, even as a percentage of GDP. And the AI systems that they're trying to build continue to get more capable as they scale them up, which has been the kind of key insight driving all of this. and it is bending the world and the economy to its whims. So yeah, it's doing pretty well. Right now, I think they're nervous because we're seeing this intense backlash to data centers, most of all. But I think AI more broadly, we're seeing it become more salient in the minds of the public. We're definitely seeing a lot more elites saying, this is a big deal. I'm scared about this. You know, a month or so ago, we learned that OpenAI's AIs hacked their way out of the company and into other companies. And we've learned more details that it was actually like a swarm of 1200 agents that were rogue inside OpenAI's infrastructure over a period of months without being detected and did a sophisticated R&D program to figure out ways to hack more of OpenAI, to hack these other companies, to trick their evaluation grader, and to tamper with their logs. And this is the worst the AIs will ever be. [15:30](https://realalpha.app/v/NdIrGMpWOZw/?t=930) And so I've been just seeing members of Congress and elite media figures and people who have been historically skeptical in some cases of this AI doom scenario be like, I was wrong. this is fucking scary. And you have like Matt Iglesias, who's always been a bit Doomer sympathetic, saying like, we need a butlerian jihad in reference to Dune, where they like declare war against all machines to be built in the likeness of a human mind. And that's referencing likely Samuel Butler, who was the first guy to write a story about machines taking over the world. And a decade before that, in 1863, I believe, he wrote an anonymous op-ed saying like we have to declare war on the machines and like this is something that nobody with a truly philosophic mind can deny um and so yeah this like i think just as much as like the capitalist drive to automate labor um has been with us as long as there have been capitalists basically um the uh human drive to not be replaced by machines is similarly deep. And I think as the evidence mounts that this replacement is not only possible, but beginning to happen, I think there will be a very strong immune response from society. And in fact, I'm counting on that for us to successfully stop it. [16:47](https://realalpha.app/v/NdIrGMpWOZw/?t=1007) What's the connection between economic replacement, replacing workers, and then AI extinction or AI causing extinction? How do those two fit together? I think a huge mistake the AI safety community has made has been focusing on superintelligence, a very fast takeoff of AI capabilities as AI is recursively self-improve and full-blown extinction, which I think are all conceivable things and we should be thinking about and worrying about. [17:22](https://realalpha.app/v/NdIrGMpWOZw/?t=1042) **Garrison Lovely:** But on the path to that is building universal labor replacing machines. [17:27](https://realalpha.app/v/NdIrGMpWOZw/?t=1047) basically all of these stories go through a stage of artificial general intelligence being created and it's typically after you know AIs that are significantly more capable than all humans put together i.e super intelligence it's typically after that is created that you have these like full-blown AIs taking over the world and potentially wiping out humanity and I think normal people are more like worried about their jobs being taken um and the labor replacing uh is like it's the ability to make labor um with capital with machines is the thing that also makes ai existentially risky uh because it will have to do labor in the world to take over the world. And so I think AI safety was both not doing itself any favors politically by avoiding talking about this. And I think they saw it as dishonest to be focusing on jobs when they were worried about something else. And I think it is actually a really big deal if everyone who works a desk job that's remote capable could be put out of work. I think that would be incredibly This destabilizing um and create like really bad outcomes for lots of people and maybe it could lead to something better in the future but the transition would be very bumpy at best um but [18:57](https://realalpha.app/v/NdIrGMpWOZw/?t=1137) also yes like you could just stop it before that point and then you'd head off the existential risk as well you mentioned the hogging phase incident um where open ais ais hacked into into hogging phase and this was undetected for a long time and they used quite sophisticated techniques to do so. For me, that is a great example of how the quest to create AIs that can replace jobs in their entirety is also ultimately a quest that leads to these capable AIs that are then, if we can't control them, potentially risky and potentially even risky in the catastrophic or existential sense and so for me that's that's the connection that you have sort of yeah the you are the pushing on capabilities can lead to both outcomes and now when you when you talk about how capabilities are developing you often reference the meter study which is this study referencing how long time horizons AIs can work over. [20:07](https://realalpha.app/v/NdIrGMpWOZw/?t=1207) So if you have a task that might take a human 12 hours, when can AIs achieve that task at, say, a 50% reliability or 80% reliability? Now, that chart has gone quite crazy in recent years, and it is now at the point where Meter tells us they can't reliably measure how good these AIs are or how long time horizons they have. And you have sort of anecdotes from companies, which you need to take with a grain of salt, of course, but you have anecdotes around them running for 20 hours, 30 hours, depending on scaffolding and so on. Is this the main chart to pay attention to? And I want to contrast it with paying attention to sort of economic data, because it's seems to me that there is still this wide discrepancy between any benchmark, including the meter benchmark, and then what we see in unemployment data, in productivity data, in economic growth, and so on. So where should we pay attention? [21:15](https://realalpha.app/v/NdIrGMpWOZw/?t=1275) I think the meter chart is the least bad benchmark that we have, if you had to pick only one. and the biggest thing it does is it creates something that can be usable over time so benchmarks have this problem where you create a score from like zero to 100 and models would be like really bad at first at the new benchmark and then within a year in many cases of them starting to make progress they would approach the maximum score and the benchmark would saturate And so it's really hard to compare like GPT-3 to... GPT-4 to GPT-5. And you sometimes saw journalists describing this as like, oh, like GPT-5 didn't improve on GPT-4 or something. And it's like, wait, what? No, it totally did. It's like incommensurable. And so Meter created this mechanism that lets you compare GPT-2 to GPT-5.6 to mythos um and it could continue doing this it just gets harder to as the tasks that get longer you need to find humans that can actually do like 16 20 40 hours of you know one software task and then grade them and then see if the ais can do it as well this is just very technically [22:35](https://realalpha.app/v/NdIrGMpWOZw/?t=1355) **Garrison Lovely:** difficult um and yeah i think like what the meter chart shows is that the time horizon in human terms of tasks the AIs can handle has been growing exponentially, i.e. doubling every like three to seven months. [22:53](https://realalpha.app/v/NdIrGMpWOZw/?t=1373) And it's kind of like hard to understand like how crazy this is, but one intuition pump is that from 2019 to 2026, AI progress was like, there was like, it went from like seconds to, I think like eight hours or something like that, seven hours and something. And then from like a few months period in 2026, it went up in absolute terms the same amount to reach 16 plus hours. And so like, yeah, it's just hard to wrap your mind around exponential growth. But, you know, COVID was an exponential process that people underrated and that it took over the world. And I think the intuition from people who are worried about this technology is like we're in February or January 2020 and things are just going to get crazier from here. [23:49](https://realalpha.app/v/NdIrGMpWOZw/?t=1429) Now, the caveat to all of this is that any benchmark that can be automatically graded is going to be something that AIs are going to be better at because you can hill climb. You can basically just have the AI try and do the thing and like see how well it does. isn't. A lot of tasks that we do, probably the most important tasks that humans do, are not of this quality, right? Like schooling doesn't prepare you for real work that well in a lot of ways. And these benchmarks similarly like overrate the capabilities of these models because you're given these like messy open-ended tasks in real jobs all the time. You often have to do something that's never been done before, you know, innovation and scientific research. [24:36](https://realalpha.app/v/NdIrGMpWOZw/?t=1476) And the types of things that AIs aren't as good at, like this kind of taste, intuition, this creativity, are still things where humans, like at least the best ones, dominate these systems. And you can see this, like if you use them regularly, like I could feed my entire book into cloud code, could read the whole thing in three minutes, give useful feedback, catch things that me and my editor had missed. and then you ask it, like, what are some ideas of the cover for the book? And it's 20 of the worst ideas you've ever heard. And they'll say, like, it's very important to cover not do this thing. And then five of the ideas will do that thing. So you're like, who am I talking to? This is like a weird experience. Because if a human read your book, like Goodwill Hunting, like read your whole book and gave you this feedback, he'd probably have some really good cover ideas, or at least like decent ones um and so this is the phenomena where ais can do incredibly incredibly impressive things in encoding and math and so on things that if a human could do those things that human is probably quite smart in general and can also yeah can also perhaps suggest good covers and can also perhaps write a book and so on that's not the case with ais [25:47](https://realalpha.app/v/NdIrGMpWOZw/?t=1547) And so you see this phenomena where they are just, they can do things that very few people can do in select domains, but still they lack sort of this, they lack generality, basically. Yes. Yeah. And this has been a thing that's been a problem for a while. And the industry is obviously trying to solve it. It's just difficult. And it has to do with the way the models are trained. And some people are skeptical that large language models will ever actually be able to fully replace human labor, at least like cognitive labor. And I feel that skepticism. And I don't think that you need to believe that LLMs will take you all the way there to be more or less as concerned because if it becomes clear that like there actually is a wall and our current approaches hit that wall the industry is not going to stop trying and they have more money and people than ever and so yeah they will just keep searching for new techniques and like in the long run um or maybe maybe even the medium run i think it'd be like pretty arrogant to assume they will fail everything they try. There is so much capital behind this project at this point. There is so much at this point, it is it is politically important for the US that this project keeps [27:05](https://realalpha.app/v/NdIrGMpWOZw/?t=1625) going and that it doesn't fail. And so there's a lot of pressure to find solutions to to any bottleneck that that shows up. And in terms of like financial data or jobs data, I think this is interesting, it's like quite hard to actually see the labor market effects in jobs data. And one component of this is that people aren't being like fired and replaced with AI so much as young people aren't being hired as much in jobs that are more AI exposed. At least in the paper I go into in the book called Canaries in the Coal Mine, these Stanford economists use more fine-grained economic data than what the government publishes. And previously economists were like, ah, we can't see any effects of AI in the labor market. But then when you use this more fine-grained data and create subsets of workers by occupation and then age group, like 22 to 25, 26 to 30, in the most... [28:06](https://realalpha.app/v/NdIrGMpWOZw/?t=1686) **Garrison Lovely:** AI Expos age group of youngest people and AI Expos jobs, they found a close to 20% relative decline in job openings or jobs. [28:17](https://realalpha.app/v/NdIrGMpWOZw/?t=1697) And this is controlling for interest rate changes and COVID and education effects of COVID and all these things. And it's possible they're missing things and there's critiques of this from economists and grains of salt and all of that. um but it also just like jives with a lot of people's recorded experience of the labor market as young people today um and so the idea here is like you know these are just the easiest jobs to automate and you could have you know experienced engineers doing the work of like multiple engineers uh programming all kinds of stuff um and if you just hire one really good experienced engineer for half as much as four junior engineers, and then they can do more work. That's what people will tend to do. This creates problems, though. The apprenticeship gap is one term people have used where you don't have young people who are learning the trade anymore. And at some point, all of your experienced engineers will retire. The industry and maybe the companies using these models are maybe just banking on the AI getting better at doing the job faster than humans are being replaced by it. And I use this analogy in the book of like, if deep learning, the training technique for all modern AI systems, if it were a person, it would have been born in 2012. Back [29:42](https://realalpha.app/v/NdIrGMpWOZw/?t=1782) when that was like when deep learning first showed promise at beating other types of AI techniques because we had enough data and computing power. And that means it would be like 14 years old right now and so going from like not being able to string together a sentence or even see very well to seeing as well as humans to uh passing tons of tests passing the bar exam at like the age of 11 or something or 12 and now at 14 being superhuman at hacking um and my question is like well where will it be by the time it would be graduating college in 2034 because so much of this ai conversation is focused on like today's capabilities with a denial that they'll get any better despite them having gotten better for like 14 years in a row so where do you expect this to be in in 2034 depends a lot on what we do about it but i think by default i do not expect humans that were born in 2012 by the time they're graduating college to be able to compete for white-collar remote capable jobs with AI systems. And that's assuming humans are still around. And robotics is lagging behind AI, but it could also move faster than people think, because as you scale up physical manufacturing processes, the costs drop predictably. So with solar, every doubling of [31:11](https://realalpha.app/v/NdIrGMpWOZw/?t=1871) solar production brought down the cost by 20%. Don't. that led to like, you know, decades long decreases in the price. And even before you have physical robots that are human competitive and cost competitive, if you have actual AGI, you could have augmented reality glasses directing humans in what to do, and the humans could just work as robots. So you could de-skill basically all these physical jobs. And that is not something I've seen that many people talking about like AI 2027 talks about this. But it's, you know, like a thing that could happen very quickly if you buy artificial general intelligence. In the public debate about AI, there is a lot of talk around about fast timelines and getting to AGI superintelligence on the one hand. And then there seems to be another sort of set of people talking about AI is a bubble and AI is bad and AI is flawed in so many ways. What do you make of the bubble discussion? [32:21](https://realalpha.app/v/NdIrGMpWOZw/?t=1941) Is there any substance to this? I know you've looked into Ed Citrin's work in the book. Yeah. With the bubble discussion, it's important to separate the claims that this technology is highly capable and getting more capable and could become very, very capable. That doesn't mean it will be economically lucrative, lucrative, at least at the beginning. And many past technological revolutions did create bubbles. There was an overcorrect or inflation and then the bubble bursts and there's still a transformative technology that's left behind. But people just get over overly excited and railways and the internet and two examples there yeah right like the dot-com bubble is the you know famous bubble and besides a housing bubble and with that you know telecom companies overbuilt um internet companies were over uh valued but then now the internet and tech is valued way way more than the dot-com bubble ever was and these are robust valuations because these companies are the most successful companies in history. They have billions of users, incredible profit margins. And those companies are the ones that are financing the obsoleting project. And so that's like one important way it's less of a bubble is that this [33:50](https://realalpha.app/v/NdIrGMpWOZw/?t=2030) is coming out of profits of these companies that each have very, very successful businesses from like Google search to like Amazon web services to meta, Instagram, et cetera. And so, you know, the dot-com bubble was financed largely with debt. And when it burst, it left, I think, $2 trillion in today's dollars of debt behind. Whereas this, you know, there's some debt involved, but it's much, much less. I think the... Do you expect that to continue, by the way? So at some point, the so-called hyperscalers... we can't finance this anymore. And so where does the money come from for the next training run in the future? [34:32](https://realalpha.app/v/NdIrGMpWOZw/?t=2072) I think you'll see more debt being taken on going forward. And it's not that the companies have fully run out of like cashflow to use, but in at least like Meta's case, they just don't want to put on their balance sheet. So they create these special debt vehicles so they can kind of like hide it, which, you know, that's concerning. And that's like a bubbly type move. And you have companies like Oracle, which actually do have a lot of debt. um and are in like a tough financial spot um i think the key mistake that is made by the bubble proponents is around whether it's profitable to serve individual ai models to customers so everyone knows that these companies are unprofitable like famously so open ai is planning to burn 130 billion dollars before it becomes cash flow positive and profitable um for reference Uber burned like 30 something billion before it turned its first profit. And that was the previous record holder. So it's like world historic levels of burning money. But that doesn't actually mean that it's unprofitable to serve a customer. And what's happening here is that it's profitable to go and spend money on using Claude per token. The subscriptions are on average profitable as well, even though some power users might be costing more than they're [35:56](https://realalpha.app/v/NdIrGMpWOZw/?t=2156) **Garrison Lovely:** paying. The average user isn't doing that. What's happening is that it's so expensive to train the next generation of models that the companies are using all of the profits from their existing [36:07](https://realalpha.app/v/NdIrGMpWOZw/?t=2167) products, throwing that into training, and they still need more capital. And so then they raise from venture and from big tech investors. And the hope is that you just grow your revenue fast enough that you can exceed the growth rate of the training costs. And, you know, this is plausible if you have, you know, trillion dollars of revenue and your training costs are like in the hundreds of billions of dollars a year, then it could work. And that's like what Anthropic and Opening Eye are projecting will happen. And it sounds crazy until you consider the fact that these These were the fastest growing consumer products of all time, the fastest revenue growth of all time. [36:52](https://realalpha.app/v/NdIrGMpWOZw/?t=2212) Anthropics percentage growth is like just truly unprecedented at the scale it's operating, going from like $5 billion at the start of the year to like 60 something billion annualized in a number of months. And so it's been more than 10xing year over year and is now poised to be the biggest IPO ever at two trillion or more dollars um and so but anthropic like they were profitable for one quarter kind of if you squint depending on how you assess these things but they're actually not planning to be profitable yet because they think they'll keep getting more value from training bigger and better models um that's very much part of the plan right so so So... [37:33](https://realalpha.app/v/NdIrGMpWOZw/?t=2253) Spending money on training the next model is necessary for these companies' valuations to make sense. Otherwise, if they were to, say, stop training and then just serve the models, the other companies' open source would probably catch up at some point and you would be in a situation in which they couldn't make as much money from serving their model as they could before. Yes. And these companies could be profitable tomorrow if they just stopped training new models. They just don't think that's a good idea. And I think two risks for the industry. Before we get there, where does the confusion arise? Is it because we don't have access to internal numbers that there is confusion about whether it's profitable for anthropic OpenAI to serve the sort of marginal customer or any customer? [38:29](https://realalpha.app/v/NdIrGMpWOZw/?t=2309) I think it's because of people who present misinformation. Okay. You know, there's actually a lot of reporting showing that the gross profits for serving the models are, you know, the gross margins are positive. And it's also like the investors would not be throwing so much money into this if it was unprofitable to serve a given customer and they had to grow like crazy and then bring down their costs like crazy to make it all work. um and yeah the financial statements have actually leaked to ed zitron who's like one of the biggest bubble proponents and i have a post showing how his own data that was validated by the financial times actually shows definitively that open ai is profitable on serving customers even when you account for all of the free users of chat gpt which is the overwhelming majority of them like It costs something to serve people AI models. [39:26](https://realalpha.app/v/NdIrGMpWOZw/?t=2366) Software costs basically nothing to serve a new customer, and that's why it's such a great business. AI costs something, but it's closer to nothing than to something, if that makes sense. And I think people don't have a good intuition for just how cheap they've been able to make these models. There was a report that OpenAI was able to have its inference costs in the last few months, and that's kind of crazy. They've been doing this for a while, and they're still finding like enormous efficiency gains like that yeah but you mentioned two dangers for the companies here one is that the models stop getting better so right now they're training bigger models each generation costs more um and then each generation is better and this is enough to make it all work but if you train you know a billion dollar model and then a 10 billion dollar model and each one is better and then you train a hundred billion dollar model and it's not better it's like oh shit that's that's a bad situation to be in and i'm sure they're doing all kinds of things to like make sure that the model is showing evidence that it's getting better so you don't like do the full training run and just see what happens at the end [40:34](https://realalpha.app/v/NdIrGMpWOZw/?t=2434) you like get checkpoints and you do you know obvious things like that um and And yeah, so if that happens, then you have this risk that open source models, open weight models are able to then catch up to you and like eat away your margin. Because the way these companies have made the revenue so far is by having the best model on the market, charging a premium for it. And then people are willing to pay for that because it's just worth it to get marginal amounts of improvement. and if you can't like if the cheaper model is also as good as the the most expensive model like then the prices are going to come down and you won't be able to recoup the cost of training [41:17](https://realalpha.app/v/NdIrGMpWOZw/?t=2477) **Garrison Lovely:** and then the other big risk is that open weight models get good enough to do a lot of the tasks that are currently being done by the most expensive frontier models and this is a thing [41:31](https://realalpha.app/v/NdIrGMpWOZw/?t=2491) people have warned about for a long time it's not really been true like people businesses customers whatever are willing to pay you know the most money for the best model um but if we think of the threshold idea it might just be that the open weight models hadn't gotten good enough yet like if it turns out that for most tasks like the the leading chinese open weight model is good enough for your purposes you just use that because it's like 10 times cheaper um i still think there will be like a market for the very best because like up until you get to like full labor replacement because like i don't know if you use them like i'm willing to just use the best model for hard tasks because if you use a cheaper one to save on you know your usage credits like and it doesn't work and you waste some time like it's just this is annoying and it's still cheap in absolute terms to use these models like the 20 a month plan gets you decent amount 100 or 200 a month gets you a lot of compute and that's a lot of money for software but then if it saves you like 10 hours a week of time you know that like looks like a steal for if you're like a white collar worker in the u.s and i think for many people it saves them at least that much time totally and from the [42:48](https://realalpha.app/v/NdIrGMpWOZw/?t=2568) business perspective, I mean, if you're paying a white collar worker, say $100,000 per year, and you're thinking, can this person's role be replaced by AI, basically, well, then maybe you can afford to spend $50,000 or $25,000 per year on AI for that role. And so there's a lot of room left in business budgets to spend on AI, I think. Of these two dangers, how plausible are they? The first one, do you think we might actually hit a wall? Scaling laws aren't laws of nature. It could actually happen, but it's just been many years in a row been proven to these laws have held yeah i would not bet on the wall um because it's been a losing bet historically but you know we don't know what will happen in the future um and i think the more important point about the bubble is like we should not be counting on this to save us from potentially very dangerous, destabilizing, undemocratic technology. And I think that's the kind of upshot of the bubble narrative for a lot of people is like, oh, I don't have to do anything now. And I think that's why people find it so comforting and appealing. Whereas my argument is we have to do a lot of things very quickly or else things can go very, very badly for most or all of us. Yeah, I have to be careful with sort of [44:20](https://realalpha.app/v/NdIrGMpWOZw/?t=2660) psychoanalyzing people here. But it does seem to me that some of the people who advocate or sort of argue that AI is a bubble would like it to be a bubble and would like to not would like this project not to succeed because it would be so dangerous for the world and so risky and so transformative in in in potentially negative ways that it's it would be easier if it were a bubble and so that i sense some of that in in the sort of public debate around these issues do do you find that too yes yeah i mean i think that is the best explanation for why these narratives are so popular um and it's just like it's a comforting story because if it's not a bubble uh there's this tweet from OpenAI employee under the pseudonym Arun, he said, people are not emotionally prepared for if it's not a bubble. And so it's like, yeah, like, what if they actually do create the universal labor replacing machine, like, then all these companies who are building it will be undervalued. And they will be more valuable than NVIDIA. And although NVIDIA might just continue, you know, growing in value with them, but more valuable than NVIDIA today. And yeah all the destabilizing disempowering effects that we were talking about earlier [45:37](https://realalpha.app/v/NdIrGMpWOZw/?t=2737) come online and I will say like on the bubble thing I think the biggest risks to AI are not economic they're political and legal so you have these like lawsuits the wrongful death lawsuits and that OpenAI is facing you have these massive copyright lawsuits that are still underway uh anthropic had to pay the largest um settlement in history 1.5 billion dollars to book authors because anthropic pirated their books um and then you know the political backlash to this like if they actually succeed at what they're doing or they get closer and closer to it there's going to be more and more resistance from people who don't want to be replaced who don't want these you know billionaires to have even more power over their lives um and so that is like the hardest to kind of price in because you know after the hugging face attacks uh by open ai's rogue ai agents you know the market didn't really react in a clear way to it and to me i'm like this is the most significant event in the history of ai with respect to like whether it gets regulated and how That is a thing that Wall Street guys just aren't able to figure out. [46:55](https://realalpha.app/v/NdIrGMpWOZw/?t=2815) And it's hard. It showed the capabilities were further along than people thought. And I bet some people were like, oh, man, I want to try that model. Assuming it does what you want it to do, it was very diligent. It was a very good worker if the job was to hack into hugging face. Um, but, uh, yeah, maybe it also makes it substantially more likely that we get a ban on developing AGI, which would be quite bad for the business models of these companies. Yeah. Do you think this is, do you think, um, sort of a strong public reaction is inevitable here? Do, is there some, is it naive to think that the companies would just, uh, be allowed to to sort of continue this development yeah I think that before any kind of like doom scenario I think there will be a recognition that AI is like an incredibly important thing and you're starting to see that now but it's still like I think a year from now AI will feel so much more salient omnipresent in the media and political discourse and The risk is that, like, you get to really dangerous thresholds without that kind of societal response. [48:13](https://realalpha.app/v/NdIrGMpWOZw/?t=2893) And so one path of this is, like, what if you have fully automating AI, R&D, recursive self-improvement? If that happens before you can automate, like, entire white-collar jobs because, you know, the industry is targeting this the most aggressively because it's something they know very well. it's something where you can get a lot of data on how it works. If you're able to do that and have a capabilities explosion where you have potentially years or decades of AI progress happening in the span of months or even weeks, then there could be a huge overhang where the models that are available inside the companies are just so much better than what's publicly available. We're already seeing this gap between how good the models are inside OpenAI and philanthropic and what the public can use. And so that gap grows very large, you can have like a very, very fast society wide effects, where like you have like super intelligence, basically, and then it's like just released or something. And a lot of jobs go away very quickly, a lot of power concentrates in the hands of the people controlling these machines. And people don't have a chance to like react and mobilize a political response quickly enough and that is like my kind of [49:32](https://realalpha.app/v/NdIrGMpWOZw/?t=2972) scariest world yeah it could be quite a dangerous situation if if we have say we have uh we have had a political backlash or just the government has responded um in to some extent and now you now open ai and anthropic can't release their models publicly or can't release the their most capable models publicly which has actually happened recently right and then then they start deploying internally instead and these models help them do their their work better the work of creating the next model as you say a capabilities explosion then it seems like the public response would be sort of limited just because you can't see the impact yet and as you mentioned they are and anthropic in particular is very much optimizing for recursive self-improvement or trying to do so they are not focused on sort of broadly automating jobs, at least not before they reach recursive self-improvement. [50:35](https://realalpha.app/v/NdIrGMpWOZw/?t=3035) So they are not sort of trying to replace podcasters or book authors or whatever it might be, right? They're trying to do something else. And I agree that that gap could be quite dangerous for the world. But what do we do about that? I mean, do we need to then monitor internal deployments? How can we trust the companies to give us accurate data on that? i think it's an awareness problem so you know it's kind of like trite to say like oh it's all about awareness because for so many issues people are aware of climate change or aids or whatever and like there's not enough being done about it um in this case i don't think people are aware enough of what's happening and so yeah if we can like wake people up to the intentions of this industry, how far along the progress has come and what might happen in the future before you see like really profound effects, then you can organize that response. And we've seen a lot of encouraging signs here where like even before there's been obvious labor market effects of this technology, you're seeing this like strong kind of backlash to it. And yes, I think we should be pushing for legislation that would mandate transparency, where Ajay Akhotra, who's one [51:52](https://realalpha.app/v/NdIrGMpWOZw/?t=3112) of the people who did the investigation into the Hugging Face attacks, she works at the nonprofit meter, and she has argued for companies releasing fixed intervals, information about how much of their code is generated by AI and reviewed by AI, and the most capable models internally and just like more you know sunlight on the problem and i do worry about this like yeah world where the models the very best models are kind of securitized and only used by you know the national security state to do who knows what and then you just don't have this kind of like the antibodies of of society um and i think this is like just yeah again getting people organized mobilize and aware of what's happening. And then pushing for legislation that would make it easier to understand what's going on. And then, you know, I think you just don't need to believe that, you know, universal labor replacing machines are imminent or already exist to think that people should not be allowed to build them under these conditions. And so making it like a normative claim, like a moral claim that we should not be pursuing these without buy-in. [53:07](https://realalpha.app/v/NdIrGMpWOZw/?t=3187) And safety, which I think is a pretty reasonable starting position. And there's questions of definition and feasibility and all of that. But just starting from that, this is a bad way to do it. It's risky. It's undemocratic. It's not happening with consent from the people it will affect, which is everybody. And so starting from that point and rallying people around that ideal is something that I'm trying to do. There's a group called Irreplaceable, which is started by veterans of the climate movement that's trying to do this as well. And I joined their board recently and I'm donating my portion of the royalties of the book to it as well, because I think this is the most important work in the world. [53:50](https://realalpha.app/v/NdIrGMpWOZw/?t=3230) Where do you expect AI to rank in the 2028 US election? Do you think it'll be up there with the price of gas and inflation and immigration and so on? I think it will be among the top issues and maybe the top issue. It's like kind of risky to say that because it's still quite far from being like the top issue in the country. We are now seeing actually like AI is driving up the price of memory. So computers and Playstations are getting more expensive because of all the demand for data centers. And then data centers have become this incredibly unpopular, high salience issue. And, you know, if you look at the polling on it, it's like it's a complicated story and a lot of it is about like the perceived effects of ai data centers specifically on the local environments and like the communities and you know energy prices and all of this um and like a minority of the concerns seem to be about ai writ large but yeah i think again things are going to get crazier um and the industry is just like so good at creating villains you know it's like you couldn't ask for better foils than sam altman and elon musk and mark andreason and peter thiel like people who just seem to resent humanity um and say like [55:08](https://realalpha.app/v/NdIrGMpWOZw/?t=3308) wild shit that it's kind of like hard to imagine um past industrialists like saying publicly about you know replacing everybody or like creating ai podcasts for your kids so you don't talk to them and all this stuff where people are like what the fuck man do you think the opposition to uh to data centers is stable when you when you think about the public uh becoming more informed about say the water use issue or sort of how uh how environmentally damaging these data centers actually are which is to say not that uh more damaging than than say uh data centers used for for uh for non-ai purposes and um it's sort of to my in my understanding the the environmental impacts are quite limited and and do you think so do you think it's the environmental concern that's that's driving the opposition uh or is it more sort of like data centers as an embodiment of everything we hate about ai and ai slop and the risk of our jobs getting replaced and so on it's hard to say Bye. [56:15](https://realalpha.app/v/NdIrGMpWOZw/?t=3375) I think the water issue has been overstated and, you know, there's been movement on that in the discourse, like Andy Masley has done the most to kind of educate people on this. I actually would push back on the claim that AI data centers use similar amounts of energy as like normal, you know, data centers, like they're just much more intensive and bigger. And so the carbon effects are stronger and the potential effects on, you know, the electricity rates for communities can be bigger, they can also, you know, be set up to go the other way. [56:48](https://realalpha.app/v/NdIrGMpWOZw/?t=3408) **Garrison Lovely:** And I mean, Jasmine Sun did some great reporting on data center opposition. And her big takeaway is that a lot of it was coming down to trust. You know, these communities just didn't trust [56:59](https://realalpha.app/v/NdIrGMpWOZw/?t=3419) the companies to deliver on their promises, because in a lot of cases, they had been promised something before. And then, you know, this big construction project from Foxconn didn't actually get completed. And so now there's like this unfinished, like ugly industrial park in the middle of town or something. And, you know, so much of trust has just been burned by, you know, the Trump administration, by tech elites, by the flood of slop and misinformation on the internet. And, you know, this technology is coming into the world after a decades long decline in trust and institutions, which I think is borne out or downstream of those institutions failing in very visible ways. You've got the financial crisis, you've got the war in Iraq, you've got Trump trying to do a coup and failing and then still being allowed to be the president again. And you've got the media, which has just consistently failed to educate people on what matters. And you've got the war in Gaza, which has been misrepresented by the media and has been supported by both parties, despite it being, in my mind, a genocide. And so, yeah, lots of people have lots of reasons to distrust the people in power right now. And that makes it very hard [58:17](https://realalpha.app/v/NdIrGMpWOZw/?t=3497) to negotiate. If you go like, we're going to reduce your energy rates, we're going to give all the teachers in your town a bonus, we're going to do all these great things. If they don't believe you, if the people on the other end of that deal don't believe you, then it's very hard to convince them it's worth doing um and then yeah i think like the actual effects on the community seem to be the dominant driver of opposition to ai data centers um but they are the physical you know manifestation of this broader like world eating process of everything becoming you know vertical video everything becoming ai slop and you now see like flock cameras becoming like the physical obelisks to, you know, the surveillance state, which people are like rejecting. And I think this is all very exciting, encouraging. My worry is that the energy will not get channeled towards the right things. And so like a data center moratorium is like, will slow down. [59:22](https://realalpha.app/v/NdIrGMpWOZw/?t=3562) the industry but less than other things that would be like uh less you know economically disruptive to you know the the country so like you could freeze how good ai chips can get and this would affect the NVIDIA's valuation and other you know companies valuations but it wouldn't have as much of a real economy effect um and i think you should just ban certain developments and developing AGI would be top of the list for me. Because you could get the data center moratorium in the US and then they'll build them elsewhere. There are already data centers around. You can swap out the chips on those data centers so there will be more compute in the future. And everything that has happened so far with AI, which many people are rejecting, has happened with less compute than will come online based on just already what's been planned. [1:00:13](https://realalpha.app/v/NdIrGMpWOZw/?t=3613) And so if it's trying to stop AI, it's not going to work. And that creates more distrust as well. If it's like, oh, we got the big win, we like beat the local project or we beat all projects nationally. And still my life sucks and feels like out of control and AI is taking my job or, you know, polluting the internet with slop or, you know, melting my kid's brain, you know, that's going to feel like dispiriting and dejecting. And so I'm taking like a yes and approach and going to people who have these concerns and being like, okay, great, like, let's do that. And then also let's do these more ambitious things that will address the problem more directly. [1:00:53](https://realalpha.app/v/NdIrGMpWOZw/?t=3653) So you have popular opposition to AI. People do not like it in general if you poll them. But there are many issues on which the public has some position, but something else keeps happening, right? It's sort of, there is a limit to the influence of popular opinion. There's a limit to the influence of democracy. Do you expect AI to fall into that category where this will just be sort of, this will happen whether people like it or not? Yeah, I think that one of the big challenges here is that American political institutions are very insulated from public opinion. And that's been true by design for the entire history of this country, but it's particularly true right now. And that makes it, yeah, everything harder. I think that if you have enough public opinion and enough salience, you can overcome that. [1:01:57](https://realalpha.app/v/NdIrGMpWOZw/?t=3717) And so with the data center turn, it's like a 75% issue. Both parties are equally concerned, roughly, about data centers. And you're seeing statewide candidates in Texas and Pennsylvania and Michigan turn, from both parties, turn on data centers in under a year, going from proponents to like, oh, I've always hated the data centers. And so I think that just goes to show you like we are still in something of a democracy and it. if it's going to lose you the election, then you will change your position. And I talk about in the final chapter, the nuclear freeze movement, which was born out of a four-page manifesto by an MIT grad student in 1980. Ronald Reagan's elected later that year, running as an arms control skeptic, a hawk who jokes about nuclear war. His administration is saying, we'll be fine if we have enough shovels, like really crazy shit. And then the nuclear freeze does well in some Reagan districts on some like ballot initiatives. And then they build this incredibly [1:03:06](https://realalpha.app/v/NdIrGMpWOZw/?t=3786) **Garrison Lovely:** powerful movement with millions of people. The largest protest in American history is in Central Park for a nuclear freeze in 1982. In 1983, it becomes the number one issue in the country. [1:03:17](https://realalpha.app/v/NdIrGMpWOZw/?t=3797) And then Reagan is terrified and co-ops the language, pivots 180 on it, doesn't run on a freeze. And so when he wins in 1984 in a 49-state landslide, it's seen as like a defeat for the freeze. But then in 1986 in Reykjavik with Gorbachev, Reagan comes very close to agreeing to eliminate all nuclear weapons. And then this lays the groundwork for a peaceful resolution to the Cold War and an 80% reduction in the stockpile of nuclear weapons, which is so much more than a freeze would have done. And so if you can build a movement that's powerful enough to make presidents think that their fates will hinge on their response to the movement, then you can make things that seem impossible become not just possible, but go beyond what the demands actually require. It's definitely an uplifting sort of history lesson. But I do worry that uh ai so ai is different from nuclear weapons in in in many ways it is enormously um value it's a it's an enormously valuable technology uh a lot of the world economy hinges on ai um working and it is sort of it's a general purpose technology it's not only a destructive weapon and so won't we have economic forces pushing us to develop this technology further nuclear weapons are like [1:04:47](https://realalpha.app/v/NdIrGMpWOZw/?t=3887) this expensive thing that once you have some getting more doesn't really help you very much because there's deterrent effects and like it's rational for countries to limit how many nukes they have if the other ones do it as well but they don't take people's jobs by design and so I think that's your biggest source of potential here and I think while like investors and a lot of the economy broadly could like benefit from artificial general intelligence everybody has something that they will lose by this you know i talked to anthropic people who are sad that they don't program anymore like this thing that they enjoy doing is just now obsolete like like they design software still but they don't code by hand and And... [1:05:37](https://realalpha.app/v/NdIrGMpWOZw/?t=3937) You know, like, woe is them, right? Like, world's smallest file in. It's like you invented yourself out of the job. But I think, like, we all hate slop, although some of us actually love it, but we don't even realize that it's slop. this has begun happening to me by the way i i consider myself quite good at recognizing ai but i have sort of several times over the last couple of months fallen for some videos where where i initially believe that this is this is an actual video and then i find out sort of later that this was actually ai generated and then i begin to wonder like how much of this stuff i you know what have i seen that where it's been ai but i haven't noticed and i will never notice and that it's, yeah, it's, it's quite wild to think about. [1:06:22](https://realalpha.app/v/NdIrGMpWOZw/?t=3982) Yeah, no, the internet is just gaslighting us now all the time. Um, and that sucks. Like, I hate when I see a cool video, I'm like, is this real? Like that's, it's, you know, not like the end of the world, but it, it, it's a shitty thing. And all of these other companies that make SAS products or do whatever, you know, the industry, the obsoleting project is coming for them too. And the people building this stuff are terrified of it. Like not all of them, but I think an increasing share of them you know when i was in san francisco a few months ago the vibe was very much like we gotta stop um i hope your book works and like these are people who are more safety pilled folks as i say but it was still different than it was a year ago when i was last there and i'm i can tell just from you know being online like it's way way more intense now after the hugging face hacks and so you've got these employees who are saying like hey we need to build a pause button. We need to be able to pump the brakes on this. 1,300 employees, including support of OpenAI and Anthropic as companies, signed a letter asking for that. And then you've got the public who's increasingly skeptical of the technology. And then you've got all these [1:07:37](https://realalpha.app/v/NdIrGMpWOZw/?t=4057) businesses and legacy institutions that are threatened by it. And you've got political leaders who don't want to lose control to anything and you know machines that make labor but don't always do what they're told are the kind of thing that you could lose control to and so all the ingredients are there for like this incredibly large and tough collective action problem being solved by the fact that like all of our interests are lined up with stopping them from building this now if all of our interests are actually lined up why does it keep happening I think I was uncareful there in that, yeah, it's in some people's interests narrowly in the short term to keep doing this, and it's unclear when that flips. [1:08:22](https://realalpha.app/v/NdIrGMpWOZw/?t=4102) I think it's also just like a lot of this is just skepticism about capabilities. I think almost everybody would acknowledge that if you had trillions of universal labor replacing machines that were much more capable than the most capable humans and had robots that were competitive. With humans, they would be able to run the world. And the question is whether those things will ever happen or what the relevant tipping point is for AI takeover becoming possible. And I think it's quite clear that the machines will not reliably do what you want them to do. They're unpredictable. We do not understand how they work at a fundamental level. [1:09:16](https://realalpha.app/v/NdIrGMpWOZw/?t=4156) And that combination of things, like incredible capabilities, unpredictability, uncontrollability in a lot of ways, that's like a very worrying combination for a nation state, for a business, for just anybody. And so I think a lot of people just don't believe. They're just like, oh, it will keep making me more money. It will keep making me more productive. um and so i'll keep investing in it and i'll keep using it and you know i don't think that opting out of using it is like a good idea uh to stop it because boycotts just don't work generally and um it's like very hard it's like becoming like opting out of using a smartphone and it also means that you don't understand how capable the technology is getting um and so we should like get organized and go for policy changes here and again it's like an education question in a lot of ways yeah i think for for each company or for each individual for each country it's it's not rational to simply sort of uh stop engaging with this technology but it could become rational if we had a legal framework around it where we set limits of what we want from the technology and then it is not as costly uh for say a country or a company to to to limit what [1:10:41](https://realalpha.app/v/NdIrGMpWOZw/?t=4241) what they do in in the ai space but of course that's that's sort of sort of uh getting from here to there is is the actual um problem you you talk about ai or agi not being inevitable um but from a lot of what i hear sort of at least again in the in the san francisco scene uh it's almost as if ADI has already happened. That's how the talk is, right? And if it hasn't happened, maybe it will happen in three years. Why is it not inevitable that we develop ADI? Because people can stop it. We have not built every technology that we can build. You know, this idea that technology happens because it's possible, that's something Sam Altman said to the New York Times, and he was paraphrasing Oppenheimer, who he likes to tell people they share a birthday um and uh yeah this idea of like technological determinism just doesn't actually hold up so you know cloning is a thing that is possible and we just don't do it and the people who have done it have gone to prison there was a recombinant dna when that was coming into the world in the 70s uh the scientists were They're worried about it. And they got together and they issued a moratorium that held. There's mirror bacteria, which if it was created, it's basically like a type of bacteria that would [1:12:08](https://realalpha.app/v/NdIrGMpWOZw/?t=4328) not be seen by our immune system. And so if it got out in the world, it could destroy the entire biosphere. And the researchers looking into this could not rule out that possibility. and they got together and they explained what was going on and they successfully created a taboo against creating it. Nuclear weapons, like we stopped making them bigger. We could make them arbitrarily large. We could create weather warfare. We could create radiological warfare. Like there's all kinds of things that were explored and not pursued. And so people in San Francisco need to read more history. They like to read The Making of the Atomic Bomb and Nothing Else by Richard Rhodes, which his entire body of work is arguing against the security dilemma, which is where these countries race each other, assuming the worst about each other, and then creating a self-fulfilling race to the bottom. And he was in favor of proactively restricting nuclear weapons, and I think would also be in favor of the types of things I'm talking about here. [1:13:10](https://realalpha.app/v/NdIrGMpWOZw/?t=4390) And so, yeah, I think they just need to read more books and not just all the same books. and realize that like, this is in our power. And I think they know it to some extent. And that's why they push the inevitability narrative so hard. Because like, if it was really inevitable, why do you need to convince anybody? Now, we've, as a species or humanity, we have stopped some technologies, we have sort of not gone down some technological paths. But have we ever not gone for a general purpose technology? For example, I mean, can we imagine that we discover electricity and then we do not deploy it? [1:13:52](https://realalpha.app/v/NdIrGMpWOZw/?t=4432) Because it seems to me that AI is a technology at that level, right? It is that valuable economically and ultimately militarily too. So we've stopped some technologies, but they've been limited and more narrow. Could we stop AI even if it's a general purpose technology? Yeah, it's a good question. I can't think of something off the top of my head, but I think my argument is that AI is different from past technologies and that difference if recognized by enough people could be the force for mobilizing against it. And I think we just have to try. you know like the way the technology comes into the world is also relevant um and you know even if in the long run you know humans will eventually build the obsoleting project will succeed like we will build you know universal labor replacing machines when and how that happens the matter enormously. And I'm not against building it at some point. I think it should just be when you know you can do it safely and when there's actual public buy-in. So you can make the case to people and they're making an informed consensual decision to bring this into the world. And so we should work from that ideal. And the way it's happening now is like break neck, [1:15:25](https://realalpha.app/v/NdIrGMpWOZw/?t=4525) you know, speed, just ask for forgiveness, not permission. And that's the worst possible way it could be happening. And so, yeah, I think we can do better than that. And even if you think we can just shape it, slow it, guide it, whatever, like we should at least try doing that. And I think having more ambitious demands that people can all understand and get behind actually makes them more likely to succeed in some cases. So like the nuclear freeze was more ambitious than passed nuclear weapon campaigns which focus on single weapons and people just couldn't understand why it mattered and similarly the campaign has been focused on like a pause where it's like well what if we don't want to keep going in that direction um like what like it's not just about safety uh it's about like the direction the technology is going um and talking about super intelligence when i don't think that's necessary and i don't think people believe it but instead just being like, hey, we should just not, we should stop the obsoleting project, we should stop the race to replace us. Like, that's the thing I think you could rally people around. And, you know, nuclear weapons seemed inevitable, and then they almost went away. And so I think, if that could [1:16:39](https://realalpha.app/v/NdIrGMpWOZw/?t=4599) happen, then we can do the same here. As you say, it matters a lot whether we develop AGI by 2030, or by 2050 or by 2100. So in some sense, technological determinism can be true on very long timescales, but our actions can still matter in the sense that whether we have decades to prepare our political and economic systems to the changes can actually make a massive difference in sort of the living standards and people's just general quality of life. Yeah, what can we actually do? if you're a normal person what can you do about this because it seems to be a process that involves basically heads of state CEOs of the biggest companies on earth what can and what can a normal person do about this problem I think depends on who you are where you are I think you could educate yourself about it tell other people about it find people who share your concerns if you're in the US, you could call your member of Congress. It actually does matter. Phone calls work best. And I think you can find organizations that are working on this. I mentioned Irreplaceable, which is just announced and is trying to build a climate movement style mass mobilization in the US. [1:18:07](https://realalpha.app/v/NdIrGMpWOZw/?t=4687) specifically. And there are things that other countries can do as well. So countries can create public goods that would help make international treaties more possible. So verification technology is something that will be required for any deal to be trusted by parties that don't trust one another. The Reykjavik conversation that Reagan and Gorbachev had, Gorbachev was informed by the Palme commission, which was created by the former and then again, Swedish prime minister, Olaf Palme, which laid out this idea of common security. And so like this idea ended up informing this conversation. And so, you know, middle power countries can create technology and institutions and ideas and frameworks that could make deals possible between the US and China. [1:19:01](https://realalpha.app/v/NdIrGMpWOZw/?t=4741) um and other countries can pressure these these governments to to stop uh because it's actually in their interest to do so you know if you're if you don't have any frontier ai companies in your borders and then your people get put out of work by them you're losing you know their their jobs you're losing tax revenue and you're not getting anything commensurate in return and so yeah i think like the whole world has a very strong interest in stopping this from happening um and they can you know bring diplomatic pressure on on these governments um and yeah i think like yeah finding people getting organized um thinking about what levers that you have and so if you have a platform you can use that platform i think basically people should look around at what levers they have available to pull and and pull them and then try and find other people and find bigger levers and just mobilization, movement building. It's been done before. Regular people were essential to the civil rights movement, to the nuclear freeze, to the climate movement. [1:20:09](https://realalpha.app/v/NdIrGMpWOZw/?t=4809) And this can work. You just need to be ambitious and get off the couch. Garrison, thanks for chatting with me. It's been interesting. Yeah. Thanks for having me. I had a good time.