This AI Expert Argues We're Underestimating AI's Impact — With Ethan Mollick

right no one has time to deal with all that code and we see that too in shipping stats 200 you know 20 times more code being produced uh 30 more products being shipped right and so it's not an uncommon problem we haven't adapted organizations to ai yet yeah i mean you there's there have been some stats i think you're referencing this um in the app store where there have been many more apps shipped than previously but uh even fewer reviews than before right so it's like um the productivity quote-unquote or the the um work looks like there's been more done uh but when it actually comes to like usage or value it's so far not showing up well actually those are two different things so one thing is the flood of stuff right and we're seeing that everywhere so there's a lot more novels a lot more games a lot more apps but i'm actually talking directly there's a nice set of studies looking at products shipped right and And we're seeing a huge increase in the amount of coding, smaller increases in the number of products shipped.

There's still an increase, but it's less, right, by an order of magnitude. And the reason for that is, again, we don't have multiplayer AI. AI is a very single-player experience right now, where you get individual performance, but. organizations are about many people performing at once yeah i've i've thought when it comes to these capabilities that the companies that are really going to at least in the near term pull ahead are going to be the ones that say individuals who are using this technology um we're going to empower you to actually go ahead and you know break free from some of the um standard red tape that would hold you back in a typical um you know corporation like um i think in most companies If somebody is, it's interesting what happens. Someone comes in after they're hired, they're given a mandate. They tend to get beaten down if they have a lot of new ideas, if they want to change things. At a certain point, companies are like, no, no, you're actually not here to have ideas. You're here to sort of play this role.

And so I think we might end up seeing a divergence in terms of competitiveness among companies that enable people to actually run with their ideas and then companies that follow this traditional blueprint of like, oh, you have ideas, you want to get things done? Just, you know, do your job. Right. I mean, in the book, I talk about leadership lab and crowd being three things you need aligned. You need a leadership team that actually buys into this and starts making organizational change. You need a crowd. You need people who are using and inventing ideas. And that's often the line employees are talking about.

And then you need a lab. You need to actually be doing research on how to take this stuff and spread it out among everyone. I think the only thing I'd push back on a little bit there is we still have the multiplayer problem. Like if you're still, if you're a hundred times more productive coder, we still need to figure out how do we give you the roadmap to follow in. We still need marketers to work with you to figure out how you get the material shipped out. We still need to figure out how to maintain the software. So it's a little tough to just say, if, you know, huge productivity gains alone from individual players, even if they're given the freedom to explore, or we still need to turn that into process?

No, but this is not year one after ChatGPT. Like if we were talking in October or November 2023, I would say, yeah, that makes sense. Companies have had four years to adjust to this. Yeah, the capabilities are getting better, but they've had a long time to adjust to this. And there is an urgency for them to figure it out soon because the magnitude of the bets on building out the infrastructure and the products and the investments in the companies that money eventually has to be returned, or the investment has to be returned to a degree. And if you don't have productivity gains, by definition, it seems like you're not going to get their return on the money. We were talking about top level productivity gains in the US economy. I talked to lots of CEOs of very large companies. There is almost nobody at this point in the data from my colleagues at Wharton back this up who isn't seeing some ROI from AI. This is not an issue anymore that people aren't seeing ROI. What they are trying to do is figure out how to move from individual productivity gains and $20 million here, $30 million here, which says a lot to us, but it's not a lot to a large company, to how do we transform our organization around this, right? So I don't

think it's a question anymore of are you getting gains? I don't hear that from C-level people. I did a year and a half ago or a year ago. That's no longer the question. The question is, what do we do now? So we have 100,000 employees. I think organizational change doesn't happen that quickly. I mean, ChatGPT came out in 2022, GPT 3.5. And companies just started adopting this in the last couple of years. There's no playbook. Nobody knows what to do in this case. All the traditional methods of the consultancies that would help you, the enterprise software companies that would build in. I mean, it is a revolutionary change. So I think that organizations are absolutely seeing the ROI. I haven't heard of a single organization saying that AI isn't paying off at a large scale right now. But they're not getting the transformation yet. But that requires management to make decisions about what's going on. And they've barely begun to do that. Many of them aren't even still using advanced systems. But OK, they are, while some might say they're getting

You know, a stat that we used to quote often on the show was that it was like one out of every eight or one out of every 20, something in a large, a large number of AI pilots were out there, but only one out of every, you know, one, eight to 20 or so actually made it into production. Have you seen the numbers sort of shrink in terms of the pilots that it takes to get to production? So two things. That is a famously bad study. Let's hear it. So that's the MIT Nanda study from last year. and um at the time it was it's somebody went out and just interviewed people at a conference and coded that himself no that was that was 85 percent of companies were like no that was 95 percent of pilots yes 90 right that was the that was the famous sort of non-design you've actually been vocal about how the study hasn't been good but there have been other reports that have come out that said you know it's not that maybe that was the one in 20 but the the more reasonable reports say like one in eight or something i i would hope that's the case that most pilots fail again the one thing i'll keep emphasizing again is i i you know i don't take money from the ai labs i don't really care whether they succeed or fail individually the you know the transformation

that's happening in companies is pretty inevitable when i talk to organizations at this point and i think they have to be doing experiments that fail because if they don't do that how are they going to learn this there's no playbook so i'm hoping for high failure rates otherwise you're

Okay, so I want to ask you, and by the way, so I know where you stand because obviously you wrote it down in a book.

I'm just trying to sort of talk through your arguments. So for those who are like, you know, speaking about me, this guy, you know, I'm just trying. No, it's fine. There's nothing wrong. I am excited to find. I should point out, we don't have all the answers, right? But also, we don't have all the answers sounds like an academic result. Nobody knows what's going on is a better way to phrase this, right? I think the thing that I'm reacting to a little bit is this sort of, I don't think skepticism about AI's ability to create transformation is warranted at this point. I think that's a dead end. I think anxiety about how we shape this in the right direction is warranted. And I think how do companies start to think about this in a way that helps large companies start to transform is a very reasonable thing to worry about and why we haven't seen those results. I spent a bunch of time in the book talking about why haven't we seen transformation happen. So I'm very much on your page and I'm not going to defend that like, oh no, AI has changed everything today. Part of the interesting puzzle of the book and why we're in the phase of coexistence, right, in some extent is we are trying to negotiate that right now what does the world look like so we get productivity

gains and good results for society okay great so then i'll keep pressing so thank you for the permission to do so i mean um i i get it if you're talking about industries like consulting software but i wonder i don't even want to use the term real economy to to describe what i'm about to describe but like there are parts of the economy where um shifting the organization in a way that you you talked about adopting ai in the way that you talked about um i i curious where you end up what you get at the end of the line what the output is i'm talking about like let's say like frito-lay adopted ai like crazy like do we just get like a new flavor of chips or or a car company adopted ai like crazy do we get like a new vehicle or a small hotel chain um do they build a new hotel like when it comes to these type of businesses which i think are the heartbeat of the economy to some degree i mean services are the heartbeat of the economy right now yeah but but if you want to manufacture i mean i literally talked to some of the companies in that exact range that you're talking to, right?

And again, I can't name names, not because I'm trying to be cagey, but because I don't know who I have permission to talk about or not. But, you know, I mean, there's an endorsement in the back of my book from the CEO, the just stepped down CEO of Walmart, to give you a sense of the kinds of companies I talk to, which I think most people would consider real economist stuff. And first of all, there is a huge amount of change in these organizations. Some of this, by the way, is R&D work, right? Like, literally, I know the equivalent of the better kind of Frito-Lay chip, But that is an expensive process inside companies that make new products and speeding up new project development is actually a big deal to them in a big way, right?

And finding new trends. I've been talking to large scale manufacturing agribusinesses that are thinking about things like how do we use the fact that AI can now look at cameras to figure out new things about which fields are in trouble and which ones aren't. We manage construction projects the same way we have since the 1940s. What does it mean when you have an AI that can give people advice about what to do differently and can actually give information about what's happening on the site or not. You have really sets, really interesting sets of changes happening everywhere, right? On the hotel side, like, or even in the Walmart store, they've been pretty open about this set of stuff.

How do you give advice to everybody, to a senior store manager who needs help figuring out what to do at the store when there's trouble? Like management is a real thing here and there's pretty transformational things. That's aside from AI for planning and AI for central management and AI for HR. So, I mean, I actually think, you know, both services are the real economy, right? And it's a big deal of coding. I think one out of every 16 jobs is management in the US, and coding follows soon afterwards. These are big jobs that represent a lot of the economy. But also, I think the whole point of this is general purpose technology with widespread effects. I mean, take even the most narrow thing, right? I was already talking about migrating databases over at large financial services companies and mid-sized banks that have COBOL still running internally. These have literally been the projects that have been the backbone of a large part of IT for a long time. Those are being transformed overnight. So I don't think it's like, I think you see this everywhere. This is not just like coders and scientists and financial professionals. Maybe you've touched this project since you named Walmart, but

I'll back you up here. I think one of the more interesting things within Walmart is associates now have like a handheld computer that they can instead of having to go like to managers or stuff they can ask questions to and it can direct them places yeah I mean and that's sort of almost on the floor Exactly. So that's one of many outcomes Walmart has had from being pretty enthusiastic about AI adoption. And they've only been doing it for a couple of years, right? And it's a few million employees. It's not an easy thing to transform an organization that quickly. So my flip side to you is on one hand, is it slow? Yes. Is this faster than digitization, cloud, e-commerce, any of the other things that have transformed American business the last 20, 30 years? Certainly.

Yeah. I wonder, people talk about AI as leading to this new age of flourishing. And I guess this is probably overly skeptical to ask this question, but does stocking a shelf faster and producing a new potato chip lead to that new age of flourishing? Is that an overpromise? I kind of feel like you're alternating back and forth between like, let's focus on the narrow use cases like what happens at Frito-Lay and let's not talk about biopharma. And then let's like, okay, let's talk about biopharma. Why is Frito-Lay matter? The whole point of this is widespread effects, right? So first of all, I don't think human flourishing is a given outcome from this.

And not just because of existential risk, but also because there is automation pressure in all these organizations. Like we haven't talked about like the easy way to use AI is automate your call center. Automate your management, automate, like, that's the danger is that we just automate everything rather than augmenting human work. So I actually think that for me, the concern partially is that there's danger that we don't pursue human flourishing as an outcome without even any of the big AI pictures. But also, there is, you know, the way you get human flourishing, right, is we free people from drudgery, we increase the rate of scientific advancement, we increase the rate at which people, everyday people get access to better education, better healthcare, better financial advice. The pieces are there to make that happen.

we haven't necessarily seen the steps needed to make it all occur for everybody yet. Something interesting that you've pointed out recently is that just there's a large set of capabilities that people are just not taking advantage of. And I think that's sort of also key to this conversation. You wrote recently in your Substack, while there is a lot of debate over what future models will do, the current capabilities of existing models are barely used and are often not even well understood. I mean, that was really interesting to me, this idea that the current capabilities are not used barely at all yeah i mean i call this the overhang right and there's a huge gap between current capabilities and future capabilities we don't need better models for complete transformation of a lot of how we work in the next 10 years right we need that adoption curve we started talking about at the beginning How we get up that curve. And so, in my mind, that means that we are lacking the imagination for the transformation piece, which is the point I keep making to you. There's no playbook yet. It's not an automated process. We need to figure this stuff out. But the piece is already there. And it's sort of inevitable at this point that transformation will happen. So, it's about guiding in a good direction.

and you're actually seeing real capability gain. Let's say AI progress stopped today, right? So no further progress, which is not going to happen, but let's just say this as a thought exercise. If companies and people were to take full advantage of the capabilities that exist today and organizations were to reorient themselves in a way that they don't stand in the way of the progress that AI can allow? What does the world look like? If progress stops today and we could continue, and I mean, I think we're back to that exact same beginning part, which is the capability overhang is quite large. So what's going to happen over the next five or 10 years, we will fill in the capability overhang. Organizations will transform the remaining technologies that need to make this matter. We'll make a difference. We will figure out how to use this in ways that the jaggedness of AI and the jaggedness of people coexist together.

and there'll be large-scale socioeconomic effects from all of that productivity gains i don't think we're going to see some of the insane numbers predicted of 10 growth but you know who knows um that seems outside of any expectation i would have um but but we will see transformation happen one way or another we don't need new models but we're getting new models regardless that's coming yeah i mean it's kind of hard to visualize but like when you walk down the street let's say again let's say all this stuff happens the way that you anticipate. Do the streets look the same? Do companies look the same? Do people's day-to-day look the same as they do today? What changes?

So in the next couple of years, I think change will be... It's kind of an interesting thing to have these conversations because any coders who are listening to this, their job is radically different than it was two years ago. It's just not the same job at all if they have any access to these tools. The best coders I know are not writing code anymore. And so you're saying like, oh nothing's changed it's like everything about their job has changed I mean but they're still going to walk down the same streets they're still going into office buildings no I mean like when I say nothing's changed I mean like the fundamentals of like of our society today are similar to they were before chat gpt i mean do they does that change at all in what time period i mean i think that in yeah when this stuff when it gets when it gets to maturity like when the capabilities that we have today are taken advantage of fully i mean i i so part of what's interesting here right is like it's if we don't change if the technology doesn't develop any further. I'm kind of at a point of like, I don't know how to prognosticate about this. I think people will still be people and we'll be living in cities and walking down streets. We're now in hypothetical land that I can't really answer,

which is like, do practical robots exist in this world? Well, there's some good signs that we're all, or bad signs, depending on how you feel about robots, that general purpose models are actually quite good at robotics. So maybe in the next five years, we finally see robots emerge. Suddenly that reshapes the physical world. Otherwise, are you more likely to have cyber attacks in your bank? Yes. Will that change how life on the streets is? I mean, people are people, right? And I think it's easy to overstate whipsaw technical change affecting people's lives. I mean, we went from, if you look back over the last 60 years or so, the built environment hasn't changed that much.

So I think it's reasonable to expect that unless we get robots doing work for us, you'll still walk down a city street. But I think that's not really the concern. The concern is like, what do people do for a living? How, you know, who is making returns from this kind of technology? What's happening inside the office building is not the office buildings themselves. Okay, so what's your answer to that? So my answer to that is that we are, that technological revolutions result in disruptive change. And you already see that happening. Coding is not the same job. We are in debate right now in economics about whether or not there'll be more or less coders.

We just don't know, right? A lot of things depend on government policy that happens, how companies adopt this technology over time, and we just don't have the answer to these questions. I can promise disruptive change, right? That's the only thing I can do. I cannot predict, because we just don't have the data yet, about whose job is replaced, whose job is augmented, where the value goes in the value chain here. I mean, I wish I've had a crystal ball. First of all, I'd be making investments based on my crystal ball. And second of all, this is not what you do in a period of disruption, right? like you can't be like this is literally one of the most impossible things to predict is exactly who wins and who loses under what circumstances i can make stuff up but it's made up stories yeah okay well i still wanted to ask

if reasoning doesn't happen and doesn't work as well as it does. Just talk a little bit about the importance of this reasoning architecture that the labs have built, and especially about why is it so useful in math and science? So large language models are next token prediction machines. They predict one token or word or part of a word after another. And as a result of that, they can't really think.

They think by writing. So the breakthrough of the reasoners was taking a technique many of us used in prompting anyway called chain of thought, where you'd say to the I think step by step out loud before answering, which gave it a chance to basically write out an answer that could be incorrect before deciding to produce the actual results. So all that reasoners essentially did, there's some other tweaks on this as well, but reasoners essentially did was let the AI model generate a bunch of internal thinking like a scratch pad before it gave an answer. And that was one of the big breakthroughs that kept AI advancing, right? There have been others behind the scenes. Like I said, there's been a lot of issues with data quality, with post-training, with reinforcement learning that have been similarly big breakthroughs, but reasoners was the most obvious one. And basically it lets AI think out loud before it has to answer questions.

And that creates much more high quality answers, especially in areas like math or science, where you have to explore errors before you get to the right answer. Yeah. And is it reasoning combined with harnesses, like the ability to use tools that's really led to this recent explosion? By the way, you in your substack talked about the agentic harness that was the first time i read it um we had a debate about the word here on the show and and it's been adopted by our audience um we're all called the harness hive out here so talk a little bit about that so so i once So again, just on the category side, I put reasoners as a set of things that has helped LLMs continue to develop at pace, right? And there is an inflection point where we get a faster pace after reasoners were developed, but that's just one of many, many sets of tools that have advanced LLM ability.

So I think of three things that go together. The LLM brain, right, which has continued to advance. Reasoners are part of that. Then there's the app, the sort of application you use this in. So part of what's been interesting is new form factors, right? Codex and Cloud Code, Muse, Notebook LM. You know, there's been a bunch of new ways of freeing AI from a chat window. And then the final piece is, as you said, harnesses, which are the tools AIs need to get things done, right? You have a machine that outputs text, maybe images. is, how do we turn that text into action? Well, we give the AI the ability to control a computer, to search the web, to execute code. And those things, those three things together, harness, plus model, plus app, working together have been what led to the explosion of utility recently.

You know, it all sounds so clear when you talk about it, but it still kind of blows my mind that you put those things together and you're able to use it to solve a millennium problem in math. Well, and the Millennium Problem adds a different piece. There's a coordination layer. There was, what, 10,000 agents passing 8 million messages back and forth, right? Like, they were coordinated solving the problem in parallel also. So there's a lot of stuff happening kind of all at once in these kind of conversations. And, I mean, I think this goes to a bigger point, which is the unreasonable effectiveness of LLMs, like, you know, and their successors of reasoners, everything.

Like, these models should not be as effective as they are, right? The fact that almost everything works is really weird. It's also why I'm convinced there's lots of room left. Like if we hit a wall in one area, I mean, you know, so you're worried about LLM token efficiency. Great. There's like a hundred ideas for better token efficiency that no one's bothered to pursue because why waste time on that? Because the model's already good enough, right? So there's all of these areas where AI is already really, really good, where if it hits a wall there, there's a bunch of unexplored ideas. Just like there's a lot of funding on world models.

There's a lot of funding, you know, into alternative architectures. Any of those can bear fruit if we hit a wall. on the application side maybe one that's bringing all this stuff together has been these personal agents things like dots and muse obviously we're really early here but what do you think the potential is for these type of agents so how do you use yours yeah so um

Yeah, I'm now freaked out. I didn't think about the prompt injection via inbound email. So thank you. No worries. Please don't want to do that to us. You know, you did this recently. I'd love to hear you just give a brief tour through like the personalities of the model. Because I think like you're deep in the model. So, you know, like what's good for what. And it would be great to hear your perspective. I mean, yeah. And I think it really has to be your own deep dive at some point into this. I think there was the Anthropics 4 series of, you know, Opus models were sort of notoriously kind of friendly, enjoyed.

like very interested in their own cognition, quite good at writing, although quickly became very tedious. And that's why you can't read LinkedIn anymore, because it's 100% like bearing the weight kind of stuff. But, you know, personable kinds of models for people stuff, right? And the GPT sort of five series was sort of doers. And the Gemini three series felt tortured um like they were forced to do things with electric shocks um and uh i think they would apologize a lot yeah and they would just sort of like you know it was like talking to golem sometimes like it would just be like smiegel says uh anyway the um um that's changed i think that the opus um i think that the original fable five series was very alien intelligency um but it's gotten better with opus five five and we'll see what fable five five brings i think that um the Noose Astra series is a really nicely balanced series from OpenAI that is much less talky and much more dewy. I think it's less of like a, you know, it's a pretty good agentic model that doesn't get bogged down in anything. And I've been impressed by Gemini 4. It's not quite up to the level of 5.1, but it is a friendly model that thinks differently than the other two. Because

what you actually quickly realize is a lot of the open weights models are actually distilled from either Claude or ChatGPT. So they talk like Claude or ChatGPT. So it's kind of refreshing to have a model that doesn't talk like you. Do you see there being a rise in people becoming real friends and companions with their AIs? Because I think that's one of the big stories that I anticipated that hasn't hit yet. Yeah, I mean, I think we were wrong about a lot of the negative effects, which doesn't mean we're not wrong about those things that happen in the future. It could happen in a positive way. Well, I think deep fakes turn out to be less of a big deal, at least till now. I mean, these are all till now, right?

So for anyone who's listening to this three weeks from now and everything has gone crazy, the models keep getting better. But things that we thought would be bigger issues, we thought there'd be a bigger issue with sort of AI friends, companions, and psychosis. And there are certainly some well-known cases that are very tragic, but it didn't become a mass issue in the same way we thought it would, right? AI companionship didn't, and everyone's saying, oh, I have a friend who's addicted to AI. But we haven't seen the sort of mass effect I think we were expecting right away. we didn't see the mass data leaking problems and prompt inject problems like there's a billion people using these systems 100 hundreds of millions inside companies and we have not seen the big incident yet from a company's internal use of a system that you know that blew up in a bad way we didn't see the deep fake problem blow up that doesn't mean we're not going to see these problems happen but we muddled through better than i think a lot of us thought we would when we sort of saw this technology and we can argue whether it's for good reasons or bad reasons but I think it's worth being humble about where we made mistakes. I think I predicted more negative

impacts than we've had, which doesn't mean they're not coming, by the way. I think cyber definitely is coming. Yeah. And lastly, on AI progress, robotics. I was with RJ Skaring yesterday as the CEO of Rivian, who mentioned to me that he believes robotics is the... People don't appreciate how much, how quickly robotics is advancing. And he basically said that, And I might describe this wrong, but in a similar way that like LLMs can sort of bake the relationship with text in their weights that we're starting to see models be able to sort of embed actions and the models are able to then generalize in a way that they haven't been previously.

So where do you see the state of robotics? Because I do think this is sort of people have been talking about a chat GPT moment for robotics on the way. I'm starting to think they might be right. I'm in sort of the same boat you are I'm not a robotics expert so I talk to robotics experts and I think six or seven months ago when you talk to them you probably saw the same thing which is like you know we've been everyone's been talking about this forever and you still see you know like the fact you see a human robot running around doesn't tell you anything. And it's still a really hard problem. And I have seen the same shift in the last couple of months that you have, which is, oh, actually, maybe LMs really do unlock robotics.

And we've seen some really interesting early demos of Astra and especially Argon being able to do generalized robotics work with a camera and a robot. So I'm hearing the same sort of thing, that this might be the moment where things start to take off, but I'm not expert enough in robotics to know whether that's fake or not. So I try and stick in my lane of what I do know. Okay, we're going to take a quick break and come back and talk about do a section that I call that I call education, faith and jobs. Podcast with Ethan Malek. He's the author of Coexistence, the next phase of AI. I highly recommend you get the book. If you're here, if you remain here, just go to Amazon or go to your local bookstore and get it and sign up for One Useful Thing, which is a great substack that Ethan writes. Okay, education, faith, and jobs.

AI in schools. So the last time we spoke, you had mentioned that you had assigned your students to do their work with AI. And I get it, you have to use the latest tools. But, you know, there are studies that are coming out that AIs damage the brain or sort of make you think less. Kids that use AI, then they can't use AI, or students, I should say, when they lose the ability to use the AI, their scores plummet. There was an MIT study that said students were less engaged, remembered less, and performed worse after using AI, and their brain scans showed less activity. Is this an unspoken crisis, what's happening in education?

I mean, first of all, it's a very spoken crisis, but also I'll push back a little bit on this.

the ais and swarms like solving math problems which by the way how do we check them and then we have uh ai assisted you know phd students and professors publishing papers uh that are um um that are uh let's see they're they're filled with ai you had had mentioned that in 2023 uh delf was mentioned in the third more papers uh that had been published uh in high quality journals yeah i mean people are are i mean you know outside of math where math is doing uh ai is doing math problems also increasing percentage of math papers acknowledge ai use

like for like it it is both a trend it's going to be the same kind of thing as we talked about with the economy and everywhere else, which is this is a transformational wave. On one hand, it's helping us do work we never could before, just like it's helping us extend education in ways we couldn't before. In the other, it's damaging old systems, right? Our world was built around humans doing human work. What does science look like when you automate at scale? What does it mean to get 400 proofs dumped in you at one time? Like we know the proofs are formally correct because they're in Lean, which is a language that allows formalization of math proofs.

But like, what do we learn from them? We don't know. So we have to figure out how to digest this. so we're seeing acceleration of science at the same time as we're seeing the foundations of science come apart and i think we're to see this in every field one after another let's talk about faith yes um ai written Sermons seem like they're a big deal now. You know, the pastor or any religious figure, you know, they're also tempted to use AI. I wonder what that does to religion. It's like, how do you feel if ChatGPT writes the sermon that you dragged yourself into church to hear? I mean, I talked about this in the book a bunch, right?

Like, I mean, AI is being used to write obituaries. I mean, there's a lot of things that AI is, like, that are intimate things to do. You know, there was a study that just came out today from Anthropic about, for political discussions of AI, breaking down where there are, 7% of political discussions are about faith. I'm not sure exactly what way, but that's how they're categorized there. So, I mean, this is an intimate thing, right? People use this for personal advice. They use it to write stories for their kids. and, you know, people are writing sermons with it, asking for religious interpretations.

I think it's part of the general reshaping of everything, right? Like part of this is, is this a sermon that's being researched with AI? Is it 100% AI written? Is it written and guided by a pastor or rabbi or imam or whoever's writing this? I mean, I think there's a lot of open questions about like polishing something. Is that good? You know, help me challenge my ideas. So because I don't have another person of the faith that I can speak to about this. Like we haven't socially negotiate all these things i mean that's kind of why the book is called coexistence is we have to negotiate our boundaries with this technology yeah i tell you if i'm sitting in a religious service and i hear uh some somebody up there yes talk about like uh you know what what this figure did was not this it was that right i'd be like what you use are you know you hear the m dashes yep you use chat i mean this is it but it's i mean you see that everywhere right linkedin is almost unreadable oh my god you know my podcast quality has shot up for a bunch of podcast i listened to and i'm like oh no like i'm like wow this is so knowledgeable i'm like oh actually this is claude this is not a knowledgeable person this is a claude script like it's a like

we are in a world of like smart imposters everywhere yeah i was just listening to uh tucker carlson had tristan harrison uh recently and they're like talking about all the negatives of ai and he goes to break and he reads an ad that was i mean from my from my ear seemed clearly written by chad gpt and it's like man okay um where do you stand on ai generated restaurant signs are everywhere yeah i mean you know i think that that's like false advertisement right it's uh i mean i'm not saying that a serious way for for the people in the room but like i i think that i think that for cultural products there is going to be as people's eyes recognize bad ai use You either need to invest in good AI use and like getting good at this or just invest in the human product again.

Like I think there will be a rebellion against mediocre AI restaurant signs, right? We talked about on the show recently, there was a spa that had a sign and the picture in the sign was a picture of a sign in front of a spa. Like it was a sign picture embedded in a sign. And clearly the prompt was, make me a sign that's in front of a spa and the AI thought, okay. But then someone posted it. Like that's the insane part. The insane part was they didn't even check, right? And I think that's the danger. Jobs, not to go from something kind of light to something more intense, but what's going to happen to jobs?

So we don't really know, right? Look, we are in an industrial revolution-like setting. And there are job categories that are going to get invalidated in ways we don't expect. And some jobs will transform and some jobs will grow, right? Like the debate of our coders right now is a good one, right? AI can program. So right now, we are seeing more demand, at least for senior developers, because there's now more coding you could do. Instead of having a team of 20 people, your own little organization can have a coder and you can produce incredible work. So maybe there's more jobs for programmers, but their job is also transforming from writing code to managing code writing.

Maybe that's a different kind of job. I mean, we don't really know. The AI labs are convinced they're going to replace all human labor by 2035. I think it is worth us thinking hard about the safety nets we want to build during this transition. because i point even when things work out well like the industrial revolution did it sucked to live through the industrial revolution for most people right that that was like the time of the communist manifesto that's what dickens's writings were about no one could breathe in london like there's a bad stuff that happens along the way there's a worker unrest like we don't know what happens we have to cushion against the downsides let's end with um a couple of uh one one kind of random thing and then i want to just run a couple of your tweets by you yes please to comment on it There's a great chart from Andreessen Horowitz talking about the price of compute versus the price of tokens. And the price of compute has stayed steady and risen a little bit, but the price of tokens is just falling dramatically. I mean, this is the most like deflationary technology in terms of the price that you pay for it that I can remember.

Yes. What do you think the impact is of the fact that AI token prices are falling so dramatically? So, I mean, it's kind of looping back to what we said at the very beginning, you were kind of pushing me about token costs and i'm like i'm not that worried about talking costs the long term this is why right we're using more tokens not just javon's paradox but also because that that That's sort of just agentic work requires more tokens. But there's a huge move for token efficiency. And we're even leaving aside the fact that organizational token efficiency, like a Fable class model can call a Haiku class model to do work and organize that work for lower costs.

So that lower cost means AI does more stuff for less money in a very short period of time, right? Really, really deflationary push. And so it's why I'm not that worried. I think that the token cost concern will go the way of, you know, of vibe coding and rag and other kinds of things because costs will decrease even as we get more compute online so there's a there's a downward pressure on prices that is real okay two tweets of yours that i found really interesting um this is about ai agents uh we will soon uh what we will soon see what will be in effect a perfectly legal distributed denial of service stack on customer service departments authorized by millions of consumers when it comes when it becomes widely known that Muse.etc can make unlimited requests for old refunds, negotiate deals, etc.

Yeah, I mean, I think there's a lot of systems like the banking system and the customer service system that depend on friction. It's hard to make a phone call. It's annoying to file something. It takes effort to find that there's a problem and gather your paperwork, even lawsuits, things like that. And now we got those done at scale. Even worse with proactive systems, right? I've already had my dot cancel several long-term subscriptions I couldn't figure out how to get out of. It's dealt with them for me. I have no idea how, but I'm not paying them anymore. So I don't know how many systems it spun up or how many chats it had. I don't have to know. And I think that as these things become widely available, companies are not ready for that. I mean, one of the most interesting was the threat to the banking system. Apollo Research pointed this out.

Yes, this is right. Which was that the economy basically depends on us being lazy with small amounts of money that we keep a low interest bearing accounts because it's too hard to move back and forth right large companies always move their money into you know across accounts to yield things we don't do that but your your dot you know eventually we'll be able to take the money and just move it into a high interest savings account and you don't have to care about that and i think that that's a big issue too okay last one i thought this was really interesting as the overhang between ai's expanding capabilities and the degree to which ai capabilities are used grows the penalty of locking in a good enough solution based on today's ai's limits grows too firms need to make sure they can benefit from the vast lab investment into better ais yeah i mean it's funny because we were talking earlier about this too which is the that you know there's huge amount of investment, but the investment is not coming from, like, most companies aren't actually investing. They're spending money on tokens, but they're not investing in building and doing AI R&D research. That's the labs, hundreds of billions of dollars, and the AIs

keep getting better and better. And the problem people did was even if they were good about AI, let's say you were really early into AI in 2024, you built a very elaborate rag system around a Llama 4 or Llama 3 installation. And that turns obsolete very quickly. And I worry a lot that companies are picking vendors or solutions based on today's technology. And then they're going to be blown away as the technology continues to increase. So you have to be ambidextrous, we call it in the business world, which business studies, which is you have to be able to both do exploitation of current technologies while still exploring for the future.

Okay. The book is Coexistence, The Next Phase of AI. Our guest has been Professor Ethan Malik. Professor, I mean, great to have you here. It's great to be able to speak in person. Like I said, We're very religious readers of everything you do. And thank you for illuminating this wild time for us. Thanks for having me. All right. Thanks, everybody, for listening and watching. And we'll see you next time on Big Technology Podcast.