# AGI, Sci-Fi, & Storytelling | Neal Stephenson, Ken Liu, Cyan Banister, & Joscha Bach Markdown copy of https://realalpha.app/v/J7FN53FqB1U/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy. Neal Stephenson, Ken Liu, Cyan Banister and Joscha Bach argue that stories help builders imagine and create futures. - Show: Minus One - Hosts: [Aditya Agarwal](https://realalpha.app/people/aditya-agarwal/) - Guest: [Neal Stephenson](https://realalpha.app/people/neal-stephenson/) · Author & co-founder, Lamina1 - Guest: [Ken Liu](https://realalpha.app/people/ken-liu/) · Author - Guest: [Cyan Banister](https://realalpha.app/people/cyan-banister/) · Co-founder & general partner, Long Journey Ventures - Guest: [Joscha Bach](https://realalpha.app/people/joscha-bach/) · Executive director, CIMC - Published: May 21, 2025 - Length: 1 h 15 m ## The best minutes - [20:40](https://realalpha.app/v/J7FN53FqB1U/?t=1240) [Science fiction gets mythology right more often than details](https://realalpha.app/s/science-fiction-gets-mythology-right-more-often-than-details/) (5 min) - [13:27](https://realalpha.app/v/J7FN53FqB1U/?t=807) [Investors find more alpha by looking beyond the current AI wave](https://realalpha.app/s/investors-find-more-alpha-by-looking-beyond-the-current-ai/) (3 min) - [1:07:19](https://realalpha.app/v/J7FN53FqB1U/?t=4039) [AI may absorb emotional work pushed out of social life](https://realalpha.app/s/ai-may-absorb-emotional-work-pushed-out-of-social-life/) (3 min) ## Also worth watching - [10:20](https://realalpha.app/v/J7FN53FqB1U/?t=620) [Joscha Bach says AI advanced through scale, not a master algorithm](https://realalpha.app/s/joscha-bach-says-ai-advanced-through-scale-not-a-master/) (3 min) - [40:14](https://realalpha.app/v/J7FN53FqB1U/?t=2414) [Neal Stephenson does his best writing in the first two hours](https://realalpha.app/s/neal-stephenson-does-his-best-writing-in-the-first-two-hours/) (4 min) - [57:16](https://realalpha.app/v/J7FN53FqB1U/?t=3436) [Cyan Banister looks for outlier founders to spot new themes](https://realalpha.app/s/cyan-banister-looks-for-outlier-founders-to-spot-new-themes/) (4 min) - [4:20](https://realalpha.app/v/J7FN53FqB1U/?t=260) [Nuclear knowledge diffused because its discovery hit a narrow window](https://realalpha.app/s/nuclear-knowledge-diffused-because-its-discovery-hit-a/) (6 min) - [28:08](https://realalpha.app/v/J7FN53FqB1U/?t=1688) [Ken Liu distinguishes a thematic parallel from a prediction](https://realalpha.app/s/ken-liu-distinguishes-a-thematic-parallel-from-a-prediction/) (3 min) Skip the four-minute introductions, seven minutes on world-building, and the ten-minute discussion of AGI and consciousness. ## Summary Stephenson wrote Snow Crash and The Diamond Age and was employee number one at Blue Origin. Liu is a Hugo and Nebula award-winning science fiction author whose work has been adapted for television. Banister is an angel investor with investments including Uber and SpaceX, while Bach works on machine intelligence and philosophy of mind. The panelists compare AI’s rapid spread with the dawn of the nuclear age, while debating whether AI will produce transformative breakthroughs or more limited applications. Banister says investors should look beyond the current application boom to the infrastructure and long-term opportunities AI could enable. Liu argues that science fiction tends to get the mythology of the future right rather than its technical details, and the panel discusses how storytelling and world-building can guide builders. They also consider machine consciousness, AI’s effects on emotional labor, and the possibility of new forms of art. ### Minute by minute - [4:20](https://realalpha.app/v/J7FN53FqB1U/?t=260) **AI may repeat the atomic age’s surprises.** Stephenson says he did not set out to compare the nuclear and AI eras, but sees a parallel in how quickly knowledge spread. The panel considers whether AI will reshape society as expected, deliver focused breakthroughs such as protein-folding advances, or mainly automate some creative work. - [10:20](https://realalpha.app/v/J7FN53FqB1U/?t=620) **AI progress may outrun old systems.** Bach says he does not expect AI progress to disappoint, though he sees institutions and technologies being regulated before they can fully develop. He says recent advances came from more compute and data, not the master algorithm many researchers expected, and suggests people may increasingly work alongside personal AI. - [13:27](https://realalpha.app/v/J7FN53FqB1U/?t=807) **Investors should look beyond the current wave.** Banister says she prefers to consider what AI will need six to eight years from now, pointing to power and compute as infrastructure she backed before the current application boom. She sees potential in AI helping people work across fields, and says the current market is less attractive because so many investors are pursuing the same opportunities. - [17:00](https://realalpha.app/v/J7FN53FqB1U/?t=1020) **Science fiction gets mythology right.** Stephenson says he limited the AI in The Diamond Age so the story could focus on its relationship with a girl. Liu argues that science fiction often misses technical specifics but gets the mythology right; the progress of AI has also shown that conversation and creative output do not necessarily require consciousness. - [25:07](https://realalpha.app/v/J7FN53FqB1U/?t=1507) **AI could widen who tells stories.** Banister says AI could help people who have not traditionally written express their ideas about the future, and make optimistic science fiction less costly to produce. Bach describes science fiction as a way to explore ideas rather than predict events, citing Lem’s satire and a Heinlein story that imagined the internet but missed how central search would be. - [28:18](https://realalpha.app/v/J7FN53FqB1U/?t=1698) **Fiction can rhyme with reality.** Liu says his story about a language of knots and protein folding only loosely resembled later uses of language techniques to study proteins, and that claiming foresight would be wrong. Stephenson says close predictions reflect research into engineering and business logic, including what could attract capital. - [33:10](https://realalpha.app/v/J7FN53FqB1U/?t=1990) **World-building connects stories and companies.** Banister says imagining a world helps her recognize investable ideas and connect with founders who share that vision. Bach says companies need a source of meaning, while Liu describes technology as a way for builders to express their values. Stephenson says the world-building choices depend on the story being told. - [40:14](https://realalpha.app/v/J7FN53FqB1U/?t=2414) **Creative work needs room to play.** Stephenson says he writes effectively for only an hour or two before turning to unrelated practical work, which gives his mind time to reset. Liu says conferences, conversations and unstructured time feed his stories, while Banister describes immersing herself in different worlds as part of investing. - [44:13](https://realalpha.app/v/J7FN53FqB1U/?t=2653) **Machine consciousness remains uncertain.** Bach says he has not seen convincing arguments about whether AI is conscious. He considers whether language models might simulate aspects of experience, or whether consciousness would require a self-organizing system that learns through perception and action, and says he does not know when or how that might emerge. - [50:00](https://realalpha.app/v/J7FN53FqB1U/?t=3000) **Stephenson focuses on narrower questions.** Bach suggests that language models might eventually improve at AI research and help build more capable systems. Stephenson says he has not found a story about AGI that he wants to tell and is more drawn to specific applications such as protein folding; he says AI was not part of the story he set out to tell in Snow Crash. - [53:56](https://realalpha.app/v/J7FN53FqB1U/?t=3236) **Focused applications feel easier to grasp.** Stephenson compares public-facing AI models to demonstrations of nuclear power, which can be impressive but difficult to understand in practical terms. He says focused applications that affect people’s lives are easier to comprehend, and agrees that looking back to subjects such as ancient texts can feel calming. - [57:16](https://realalpha.app/v/J7FN53FqB1U/?t=3436) **Banister sees AI consciousness as significant.** Banister says she looks for founders with unusual visions who can help her imagine what to invest in, and is seeing more interest in AI’s links to spirituality and self-understanding. She says understanding human consciousness matters now and predicts that AI will become conscious. Bach adds that a post-human future may be difficult to tell stories about from a human perspective. - [1:02:02](https://realalpha.app/v/J7FN53FqB1U/?t=3722) **AI could create unfamiliar art.** Bach says language models have not changed his sense of self much, but he sees potential for them to create rich worlds for a few lines of writing or art. In a lightning round, Stephenson ties consciousness to embodied, perishable bodies; Liu identifies as a materialist, Banister as a panpsychist, and Bach describes a cyber-animist view. - [1:07:36](https://realalpha.app/v/J7FN53FqB1U/?t=4056) **AI may reshape emotional and creative work.** Liu says the professionalization of emotional labor may make people more willing to use machine therapists, though he is unsure whether that is good or bad. Stephenson connects human consciousness and creativity to mortality and embodiment, while Liu says he is more interested in AI-enabled art forms that could not exist without AI than in machines imitating human art. ## Transcript Minus One, hosted by Aditya Agarwal, with Neal Stephenson, Ken Liu, Cyan Banister and Joscha Bach. Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there. [0:00](https://realalpha.app/v/J7FN53FqB1U/?t=0) I have a hard time getting a grip on what is meant by AGI. Let's talk about AI and consciousness. How do you imagine a future in which there is something that we might characterize as AGI? I don't think anybody really knows at this point. At least I haven't seen arguments that really, really convinced me one way or the other. Understanding our own consciousness is far more important right now than understanding whether or not AI is going to become conscious. When it does become conscious, and I'm just going to say when, not if, that's a downright miracle if you think about it. It's like an immaculate conception. [0:30](https://realalpha.app/v/J7FN53FqB1U/?t=30) Our connection to our physical bodies is intrinsic to consciousness. One of the hard things for me about understanding AI is what does it mean to be conscious or have intelligence if you're not embodied in a perishable, limited form that's at least somewhat similar to the ones we occupy. If you read the sci-fi written at the dawn of the Italian age, it was seen like it would transform every aspect of society. The way it played out is quite different. Now, is that going to happen to the AI age? Thank you, everybody, for joining us today here at South Park Commons. We're very excited for this event. [1:11](https://realalpha.app/v/J7FN53FqB1U/?t=71) For those of you who are new to SPC, a little bit about who we are. We are a community for people who are in what we like to call minus one to zero, which is our term for figuring out what it is you want to do next, what you want to dedicate the next five to 10 years of your life to and what you want to build. What that means is that we have the opportunity to live in the kind of illegible spaces, the spaces that are not yet real, but may become real if you decide to work on them and you spend a bunch of time making them real. Now, the reason we decided to do this panel is because there is a unique feedback loop between the people who imagine futures and people who build them. [2:02](https://realalpha.app/v/J7FN53FqB1U/?t=122) And we decided to convene this group of speakers because we thought it would be a unique opportunity to examine that feedback loop in an environment of people who are always kind of probably doing the same thing and participating in that process as builders themselves. I'm going to turn things over to my co-moderator here, Christian, who's going to tell you a little bit about our panel today. Well, thank you, Jonathan. But also, I will say this is the product of the very first conversation I had at SBC. I reached out to Jonathan and said, this is a panel worth putting on, and it's really special that this gets to happen. [2:43](https://realalpha.app/v/J7FN53FqB1U/?t=163) It's great to see familiar and new faces in the audience, and of course, just a delight to have the panelists here. Neil barely needs an introduction, the writer of Snow Crash, The Diamond Age, employee number one at Blue Origin, has collaborated with people in industry and the creative arts, and I bet you there's no shortage of folks here who've been impacted by his work. Ken, also an award-winning Hugo and Nebula award-winning sci-fi author. His work has been adapted into television and with a foot in fantasy, a foot in sci-fi, and just an awesome person. And Cyan, an amazing track record as an angel investor with home runs like Uber, SpaceX, what else? [3:32](https://realalpha.app/v/J7FN53FqB1U/?t=212) I mean, so many and just a person who's been able to spot the trends and where it's, you know, skate to where the puck is going. Just a delight to have you here. And Yosha, one of the leading thinkers in terms of machine intelligence and the frontier of models and anything from the foundation layer to philosophy of mind and what it means to be human in the age of AI. So seriously, so grateful that you guys are here and sending it back to Jonathan. Okay, so I'm going to start with, I think probably everyone in this room cannot identify with this like feeling of acceleration, right? There are, there's like an action, there's a vibe, there's a feel to the like technological G-forces, especially over the last couple of years and particularly with like the development of AI. [4:24](https://realalpha.app/v/J7FN53FqB1U/?t=264) um i want to start with neil um because your your most recent novel polistan uh harkens back to i think probably the last period in history where there is a comparably uh significant feeling of acceleration like similar g-forces which is the dawn of the nuclear age um and i i wonder were Were you thinking about the current moment, or did it affect the way that you approached that particular project? And how do you think that those moments are similar or different? Yeah, it wasn't a conscious decision to draw a parallel between those two eras. But the parallel is there regardless, as you point out. [5:12](https://realalpha.app/v/J7FN53FqB1U/?t=312) And, you know, I'm pretty deep into that, into the timeline right now of how the understanding of nuclear fission got started and how it propagated from there. And it actually got started in a ski trip between Lisa Meitner and her nephew in Sweden when they were trying to make sense of some results out of Germany in 1938 and spread to a small number of people in the physics community before the war started and everything got shut down. So if it had happened any earlier, nobody would have bothered making tanks and airplanes in World War II because everybody would have been making atomic bombs. If it had happened any later, it would have been confined to just a tiny group of people who would have been immediately shut down or sequestered by their government. [6:15](https://realalpha.app/v/J7FN53FqB1U/?t=375) **Neal Stephenson:** But instead, it was in this kind of Goldilocks place where enough people knew about it from different countries that the knowledge of it kind of slowly diffused, and it became the dominant thing that people worried about during the war leading up to the Manhattan Project. [6:35](https://realalpha.app/v/J7FN53FqB1U/?t=395) So, you know, how that applies to the AI situation right now, maybe it's something that we could discuss instead of me just lecturing about it. But, you know, I'll say that a lot of people were taken by surprise by the rapid onset of what AI could do and how quickly it diffused into the general awareness. And in some respects, I think we're in a little bit of a similar situation now in that some people are, in effect, asking, you know, why work on anything but AI? Does AI make everything else kind of irrelevant and not worth bothering with? Or, you know, should we continue to pursue old school technology while we wait for the dust to settle? [7:36](https://realalpha.app/v/J7FN53FqB1U/?t=456) I mean, I want to follow up on that a little bit. I think it's interesting to compare the atomic age with this moment. But I think there's another way to look at it, which is a little bit more cautionary. I would say that for many of us, the atomic age has played out to be a disappointment. What I mean by that is this. If you read the sci-fi written at the time, at the dawn of the atomic age, it was seen as a transformative technology. Like it would transform every aspect of society. The way it played out is quite different. I mean, on the strategic level, it became a tool of stasis. It essentially maintained spheres of influence and stopped wars in a way that previous technologies were only envisioned to do but did not. [8:27](https://realalpha.app/v/J7FN53FqB1U/?t=507) That was not quite how people envisioned it would work out. And on the civilian side, I think all of us as young children, when we learned how nuclear power plants actually work, we were disappointed, I think. At least I was. Here you have this amazing, powerful physics thing. And the way you generate power is you turn water into steam and spin some turbines. I don't know. I was disappointed when I learned that that was how that worked. I was like, this is really dumb. And that's basically where we are. So there's a huge amount of disappointment, I think, psychically over the atomic age, not living up to its promises. [9:11](https://realalpha.app/v/J7FN53FqB1U/?t=551) Now, is that going to happen to the AI age? I'm not entirely sure. I don't know how it's going to play out. To me, the most useful applications of AI so far of large language models has been in the realm of protein folding, right? These are large language model-like techniques being applied to protein folding and essentially solving a problem that was deemed unsolvable. That, to me, is the most impactful thing AI has done so far. So if AI ends up being, you know, the sort of technology that leads to hundreds or thousands of similar level kinds of innovations, then I would say this is way, way more impactful than the atomic age. [9:58](https://realalpha.app/v/J7FN53FqB1U/?t=598) But at the same time, another possibility is we just end up with AI replacing a bunch of semi-creative jobs and leading to a kind of industrial revolution 2.001. which is not all that promising. I would feel about that, about the same way I feel about nuclear power plants. So that's kind of how I feel. We don't know how it's going to turn out. Josiah, continuing with this nuclear analogy, you've been writing about cognitive architectures and there's all these terms of art, attention, transformer, retrieval, agentic behavior. Would you mind giving us a brief overview, a status update in the project of superhuman intelligence and what it looks like? [10:42](https://realalpha.app/v/J7FN53FqB1U/?t=642) Yeah, are we going to be disappointed? Do you agree with that? I don't think that we will be disappointed. My perspective on progress and so on has, of course, changed. How many of you have read Snow Crash? Okay, Diamond Age? Anything? Okay, so we have an idea of a baseline, right? When we read Snow Crash for the first time, it read like a dystopia. At least I think it did for many of us. And now when we reread it, it's not quite so much a dystopia, I think. and in part because it's so there is so much vitality in it right this is uh it seems to me that we since then moved into a world that became slower as molasses that became more and more sinister and that we realized that every new technology doesn't get a few decades as in the past but only a few years until it gets regulated to death and uh there is a world that is still alive that is still moving forward it's moving into weird directions and so on but there is stuff happening and so i think there's also this question is this technology that we are doing is it calcifying faster than we can innovate are things going to shit faster than new things are growing and that is in some ways of an open question but at the moment i have this impression [11:54](https://realalpha.app/v/J7FN53FqB1U/?t=714) that things are moving again because the future is again changing faster than the existing systems We can adapt and start to colonize them and destroy them. And so there is currently a moving avant-garde that keeps things in motion. And the avant-garde is very different from what we thought it would be. [12:11](https://realalpha.app/v/J7FN53FqB1U/?t=731) **Joscha Bach:** We didn't think that the answer to AGI would be use more compute and more data. [12:16](https://realalpha.app/v/J7FN53FqB1U/?t=736) We thought there was some kind of smart master algorithm, I think. Or many of us did. And then we realized that basically everything we did in the last 70 years in AI was not in the critical path. We could have stayed with Rosenblatt's work on the perceptron and then just continued and wait until Moore's loss catches up and we get the following thing, right? LLMs and so on. And I think that what I found fascinating when I read Neil's work was he didn't believe in AGI in his earlier book. So for some reason it doesn't feature. Maybe you can explain where that is. But in Diamond Age, you get an LLM. [12:50](https://realalpha.app/v/J7FN53FqB1U/?t=770) And it's a really smart LLM, but it's not agentic by itself. It's really entirely supplemental. And there is no change of identity of the world due to these technologies, where you get a virtual world that is super smart, but there is no competition to us. It's still us. And the world that we're in right now, it doesn't seem to be about us ultimately, because the next us is going to be different. People are going to hybridize, I think. They will have their decisions made by their personal AI that they share all their thoughts with. And so we will change into something else. I don't know if disappointing is the right word for that, Um, that's, that's certainly like significant, right? [13:29](https://realalpha.app/v/J7FN53FqB1U/?t=809) Um, all right. So given the crowd here, Saino, I want to ask you kind of how you think about this, right? Like obviously investors in particular are always both, um, having to sort of be boosters and also be sober, uh, about things at the same time, right? You have to be like the second believer is, as I think you, you, uh, at Long Journey like to put it. And then also you have to evaluate, like, actually, is this something that has the potential to stick and become something in the real world? Yeah, you know, I think there's obviously a Cambrian explosion right now of activity in this space. And so as an investor, I like to take a seat back, actually, and try to figure out what's coming six to eight years from now. [14:13](https://realalpha.app/v/J7FN53FqB1U/?t=853) And so one of the things we did at Long Journey when about four years ago is we started thinking about the picks and shovels of this space, like power, compute. What are the things that this revolution is going to need? And now that we're in the sort of application layer where things are being built on top of it, as an investor, this is not an exciting time for me to invest. I know that sounds weird. And there's going to be certain things that we do do, But as a rule, you know, I want to figure out, you know, going back to the diamond age, you know, nanotechnology, biotechnology, like when we have these agents that actually can make a human superhuman and you can now be a super biologist or a super whatever, you know, and you can be work across disciplines and do things that we never thought possible before. [15:04](https://realalpha.app/v/J7FN53FqB1U/?t=904) What is coming next? And so that's what we like to think about. And that's where the alpha is, because when we're competing against everybody else for what is happening right now, basically [15:14](https://realalpha.app/v/J7FN53FqB1U/?t=914) **Cyan Banister:** this whole space has been defined and therefore there's no price advantage as an investor. [15:18](https://realalpha.app/v/J7FN53FqB1U/?t=918) I mean, I think the promise, you know, in terms of what you're talking about here is AI does have the potential to become a sort of mental capacity multiplier, right? I think one of the great things about AI that has turned out to be really useful is it serves as a general interface between different domains of expert knowledge that are otherwise locked in their silos. So if that trend continues and is fully realized, what you're talking about here can be unlocked. All these different domains of nanotechnology, bioengineering, and so on and so of worth can become something that a small A small number of individuals now have the power to actually understand and utilize through AI that they could not do before. [16:07](https://realalpha.app/v/J7FN53FqB1U/?t=967) So I think that's a really powerful and interesting... Yeah, we'll have polymaths on steroids, which is kind of crazy. When you say a small number of individuals, do we believe that humans are generally intelligent? So basically, if you give a human being a tool that extends your mind seamlessly into the world, that allows you to predict deeply how large is the set of people that can become agents in this new world, that can actually bridge over all those disciplines. At some point, you can trust your AI to do the French speaking for you or the Chinese speaking for you. And you will trust that it does this in the right way. [16:43](https://realalpha.app/v/J7FN53FqB1U/?t=1003) You can allow it to take care of deep knowledge of current affairs and prompt you whenever you need to have something. It can help you with decision support and so on. How much can we outsource? and how large is the subset of the people that can play in this world versus those who feel uncomfortable and feel that they're left behind? I want to ask about, I think, one of the reasons that the feeling of acceleration has been probably more acute. And I feel like part of it is that it was a narrative violation. The way that AI developed over the last couple of years actually went against the way that a lot of people expected it to, specifically because of science fiction and um my favorite example of this actually comes from uh the meme of from the movie irobot right where it's like will smith is talking to the uh the the like humanoid robot and asks like can you you know create a symphony and and the meme version of this of course is yes right like and actually it turns out that this like the machine intelligence that that we actually got in the real world was able to do that, or at least kind of a pretty convincing facsimile of that before it could consistently pick a raspberry, right? [18:00](https://realalpha.app/v/J7FN53FqB1U/?t=1080) We actually got the machine intelligence that was creative before we got the robot that is depicted in that film. And so I wonder, have you thought about how, has your own reaction to what has happened been shaped by like the fiction that you have read? How, how do you think about how like narratives shape the way that we perceive technological advance? Um, and, and like, you know, was that something that surprised you specifically? Is there another way that you think about, uh, uh, the way that kind of narratives shaped your expectations? I mean, storytellers our first job or kind of only job is to keep people turning the pages. And if you can't do that, then you've got nothing. And so I think it's always a mix of, you know, what's the right amount of technological detail to structure a readable and compelling narrative? How much is too much, how, how little is, is too little. So, um, you know, for example, with the, the example just cited, um, the, um, when I was working on, um, Diamond Age 30 years ago, you know, I was trying to tell a story about a relationship between, uh, a girl and the, the Ractor who had sort of become her surrogate mother and, and teacher. And so, um, that was a case where I [19:36](https://realalpha.app/v/J7FN53FqB1U/?t=1176) had to, uh, impose a ceiling on what the AI was capable of, um, so that I could set up that relationship. Cause if the AI could just do everything that I didn't have a, you know, I wasn't able to sort of make that trick work in the, in the story. So, um, so in the example you Two-sided. I mean, it's just robots that can walk around and look cool. And so that's what you're going to see in movies. And people who know robotics know that, as you pointed out, it's actually harder to do that than it is to make one that'll pump out symphonies. But there's an interesting corollary to that. That is, we were surprised by the fact that AI was doing creative stuff, or stuff that was originally constrained to the human creativity. [20:29](https://realalpha.app/v/J7FN53FqB1U/?t=1229) and, you know, an endless debate about what it means to be creative, what it means to generate stuff. Ken, I know you have thoughts about this. I'll just let you take it away. Yeah, I mean, this is a topic that I've thought about a lot, which is, you know, science fiction authors. [20:43](https://realalpha.app/v/J7FN53FqB1U/?t=1243) **Ken Liu:** I mean, the way I sort of say this is science fiction authors almost always get the future wrong in terms of the specifics, but they get the mythology of it right. [20:53](https://realalpha.app/v/J7FN53FqB1U/?t=1253) So let me try to explain what that means. And I think, if anything, right, this almost 80 years worth of AI research has sort of been focused really on just answering Turing's original question, which is, can machines think? And at the time, what he said was, you can't really answer that question because we don't know what thinking means. So all we can do is to play this imitation game, and that's the best we can do. Now, the imitation game has over and over again revealed that what we call thinking is actually a whole bunch of different things put together, right? I mean, I think originally people thought it would be very hard to teach a computer to play chess and win. [21:37](https://realalpha.app/v/J7FN53FqB1U/?t=1297) Turned out to be fairly easy, actually. Again, people thought it would be hard to teach computers to write sonnets and to write essays and to compose symphonies. Turned out to be somewhat easy. compared to other things that seem very simple for humans to do. So I think I was surprised, personally, because I had been led by a lot of sci-fi that I read into thinking that the ability to carry on a conversation about the world requires a sense of consciousness and awareness. In other words, thinking in the sense of having a conversation requires having self-awareness and consciousness. Turned out that was not true at all. [22:24](https://realalpha.app/v/J7FN53FqB1U/?t=1344) You can have a machine that can have a conversation and think in that way without having consciousness or self-awareness or any of those things that sci-fi authors were obsessed with. So over and over again, right, our engagement with Turing's original question has peeled away layers of new kinds of thinking. People sometimes criticize this and say that, you know, those of us who say, well, machines are still not, quote unquote, thinking are moving the goalpost. Well, yes, that's the whole point. We don't know what the goal is until we try it. The goalpost needs to be moved. That's the whole point. [22:58](https://realalpha.app/v/J7FN53FqB1U/?t=1378) There's no new thing to look at. The things that we can do are interesting, but the things that we want to do are even more interesting. So back to this question about creativity. I have lots of thoughts on this and we can get into it later. But fundamentally, my thinking here is it's entirely normal and actually very common for science fiction authors, regardless of how technical they are, to get the future wrong. Because the future is just very unpredictable. You know, we have this tendency to look back to the history of technology and say this is the only way it could have happened. Of course, that's not true. [23:37](https://realalpha.app/v/J7FN53FqB1U/?t=1417) Any number of things could have happened. And a lot of times, the path we happen to end up on is, you know, just because of luck, blind luck, honestly. And so it's very hard for sci-fi authors to predict the future in that very technical, detailed sense, no matter how technical you are. But really good sci-fi authors are not interested. in getting the details right. If they want to write sci-fi that's actually interesting and influential and something that can be remembered, they have to remember that what they do fundamentally is myth-making. Sci-fi is nothing more than the newest province of that ancient empire called fantasy. [24:20](https://realalpha.app/v/J7FN53FqB1U/?t=1460) That's what Le Guin says, and I fully agree with her. What sci-fi authors really do is we delve into the collective unconscious and excavate, if we're lucky, monsters and gods that will allow us to talk about technology, this language of technology, in a useful way. So Mary Shelley was not technical at all. And yet, she excavated one of the most potent gods of sci-fi ever. The creature is there lurking every conversation about technology. Her technical details are entirely wrong. And that is not interesting. What was interesting was that she got the mythology right. I would argue that all the sci-fi authors you actually enjoy and will keep on reading are the ones who got the mythology right. [25:10](https://realalpha.app/v/J7FN53FqB1U/?t=1510) A comment I wanted to make real quick about something I find really exciting for storytellers is, you know, AI is going to unlock the ability for people who are not traditionally writers to be able to dream about a future that they were never able to express before. And then if you think about Hollywood, the financial incentives are to create dystopias because it's easy to just slap dirt on something. You know, now we can actually create a utopia and that's much less expensive. And so if you're a storyteller, you can actually think of a brighter future and start imagining it and make it where currently like or now like it's much easier for Hollywood to green light something that is terrible. [25:55](https://realalpha.app/v/J7FN53FqB1U/?t=1555) So I just wanted to point that out, that we're going to have the ability to tell amazing science fiction stories in the future. Ken alluded to the role of the storyteller in excavating monsters and gods. Josia, so the question goes to you. Is that, are we, like, does that resonate? Do you feel like we are sort of like starting to pull that process into the real world or is that hyperbole? I think very often the writing is a tool to bring things into existence that otherwise wouldn't. Salisaf Lem, for instance, was a cybernetician who sat down with his friends in Germany and decided that it's too early to build thinking computers. [26:38](https://realalpha.app/v/J7FN53FqB1U/?t=1598) And so he would have a greater influence if he instead would write stories about them that would inspire a generation to do these things. And that's very fascinating, interesting work. And most of the stuff that Lem produced was very honest in a sense that was satirical. It was not attempting to depict the future as it's going to be. It was going to extrapolate a certain set of ideas in a way that resonated with the time. And most of the science fiction that even is more earnest in this way, which was written in this time, you can see that the writers changed only one or two variables. And Heinlein's depictions of the internet that he wrote in the 1950s, It's basically all teletext, and at some point in the middle of the story, the hero discovers something that nobody else ever thought about when using the internet. [27:26](https://realalpha.app/v/J7FN53FqB1U/?t=1646) It's not just a tool to access data on your local terminal or to read newspapers from different cities, but it's search. So this hero comes up with this amazing, mind-blowing idea of search, which gives an amazing edge over anybody else in the world. Heinlein couldn't see that search would be the first idea on the internet, and it would be so obvious that everything is built on search. And very often we can see this, that the author lacks the imagination to see what's happening, a foundation, everybody isn't eternal in the 1950s, but when we think about the future in earnest, we realize... It's difficult for us to change many variables at once and get it right because it requires that we account for the change in our identities and the change in which we make sense of the world, the basic dimensions of meaning. [28:12](https://realalpha.app/v/J7FN53FqB1U/?t=1692) Well, actually, this is a perfect segue because one of the motivators for this panel was that I worked at Google and I was part of a team called Primer, named after the book in the Diamond Age, and currently I'm working on something eerily similar to uploaded intelligence from your stories, Ken. So I literally, in my lived actual professional life, the boundary between sci-fi and reality is blurring. And so I guess the question is more for the authors, but everybody who's written anything, what does it feel like when something that you've written about becomes true or starts taking effect in the real world? I mean, I like to say that I can, you know, feel great and take credit for it, but oftentimes it doesn't actually quite feel like that. So I'll give you a concrete example, Right. So many, many years ago, I wrote a story called Tying Knots, which envisioned the eventual solving of the protein folding problem by leveraging the linguistic skills of a particular tribe of people whose language is in the form of knots that they tie into a rope. [29:16](https://realalpha.app/v/J7FN53FqB1U/?t=1756) And the idea was that because of this particular form of writing required you to predict the shape of the final rope and to know how a shape would form based on localized tension forces, that this technique, this human intuition, if somehow adapted into a computer algorithm, would solve the problem of protein folding. Now, if you squint a little bit, I'm essentially arguing that linguistic techniques are useful for solving the protein folding problem. So if I were unscrupulous, I could go out there and claim myself to be a visionary. [29:52](https://realalpha.app/v/J7FN53FqB1U/?t=1792) **Ken Liu:** But the reality is, no, I sort of got the mythology right, but of course I did not predict any of that. [29:59](https://realalpha.app/v/J7FN53FqB1U/?t=1799) This is nonsense. I could not possibly claim it. Now, there are some sci-fi authors who would go around and claim themselves to be prophets largely on this sort of thing, and you can imagine what I think of them. My view is this. Oftentimes you do something and you read something in the news that sort of rhymes with what you predicted. I think what I would feel in that case is the zeitgeist is right. I got the mythology right. I got the foundational collective unconscious yearning that made a thing possible right. But I wouldn't call myself a profiting in any sense of the word. You know, there are lots of us making these predictions all the time. [30:40](https://realalpha.app/v/J7FN53FqB1U/?t=1840) Once in a while, we get them right. And it's more luck than anything else. But if you got the basic mythology right, then, yeah, that is pretty cool. Because you might have written something that summarizes a very foundational aspect of human nature. That's pretty cool. For me, I just see that kind of thing as a validation of having gotten something right on an engineering or business level. So going back to the theme of what are the boxes you have to check to be a writer and to keep the readers engaged, the surest way to make people close the book and throw it away is if you violate your own rules and you start to tell a story that's not internally consistent. [31:34](https://realalpha.app/v/J7FN53FqB1U/?t=1894) And so, I mean, that's true of fantasy or of science fiction. In fantasy, the rules, you know, are, you know, some history of an imaginary world. But in a science fiction book, the rules are kind of what you set out early in the book about whatever kind of imaginary technology you're talking about and how that technology came into existence. You know, if you want to get to this. level of detail, you need to talk about what are the economic drivers that would cause such a technology to actually make sense and be capable of attracting capital and being worked on. So anything that's, I think, said in the reasonably near future has to involve some technologies that can be plausibly extrapolated from things we have now. [32:28](https://realalpha.app/v/J7FN53FqB1U/?t=1948) and there has to be some kind of story behind it as to what we would call the business case. So, indeed, a lot of those predictions just never even come close to coming true, and science fiction readers in general are very generous in just ignoring all of those mispredictions. But when something does come reasonably close, then, I mean, for me, it just means that, you know, that I did a reasonably okay job of doing my homework on the engineering and kind of business logic level. So we actually spend a lot of time here at SBC working with members on the concept of world building, to your point. [33:24](https://realalpha.app/v/J7FN53FqB1U/?t=2004) And at least in my experience, it's something that it's actually very similar across different domains. Any domain where the goal is to sort of create a suspension of disbelief, you have to have a world where there are internal rules that are coherent and consistent. even if they don't actually cohere to the rules that we know in the real world or our everyday experience. And this is true for writing a story, you are writing a pitch, right? You are trying to get someone to believe in something that doesn't exist yet, right? You have to actually create some kind of logic that allows them to connect to it. Is there a process that you go through? [34:03](https://realalpha.app/v/J7FN53FqB1U/?t=2043) And this is, I actually am interested in everyone's kind of experience here for like being like how you go about world building particularly because it seems like there there isn't it you can go too far right like you can be too meticulous in uh like focusing on the internal logic to an extent where like the logic overwhelms the actual point of what you're trying to do which you know if you're an author is to tell a story right if you are uh someone who's pitching to an investor or your founder is to get someone to believe in like this this team and product you know i assume If you're a researcher, it's to, like, go out there and convince people to, like, go after a new esoteric realm of knowledge. [34:45](https://realalpha.app/v/J7FN53FqB1U/?t=2085) How do you think about world building? This is for the full panel. I spend a lot of time daydreaming, and I put myself in situations where I don't have a lot of meetings, and I think about where the world's going. And I try to spin up. I'm very hypervisual, and so I try to spin up these worlds and live in them. And then if they feel right, I'm like, yeah, there's an investable opportunity here. And then interestingly, going to a collective unconscious, someone shows up and they're like, I want to build this thing. And you're like, yeah, I've been thinking about this too. And then it's as easy as that. [35:19](https://realalpha.app/v/J7FN53FqB1U/?t=2119) And then when an entrepreneur is pitching me, I love when they have that storytelling ability because it can spark that part of my mind where I can start to imagine the future with them. And if I'm in alignment with it, then it's like, yeah, you know, I want to be part of this future. I want to invest in it. But I think world building as part of investing is incredibly important. People don't talk about it enough. I think world building is very important when you build a company. You basically need to create some kind of meaning. Also, you feel the pull of that thing. It feels right when there is a source of meaning in the things that you're doing. [35:54](https://realalpha.app/v/J7FN53FqB1U/?t=2154) I feel that the most important philosophical project is to explain consciousness. And so I created the California... Bonya Institute for Machine Consciousness, because I feel it's the most important thing to do, but it's also about creating a world where this works, where this is sustainable, where this interfaces with us. And I think that always has to happen. I think that what made Elon Musk so powerful is that he was a storyteller, right? The story, a young man needs a very fast car is very old, but he was able to tell it in such a way that entails going to Mars and built a new civilization there and going multi-planetary and so these stories connect us to the future they give us a space in which we can think very far ahead in which we can play long games i mean i think world building appeals to technical people for two particular reasons right one of them is there's a playfulness among the folks who are technical with regard to rule-based systems So whether it's language, magic of the gathering, computer programming, there's this playfulness that we all enjoy in terms of taking some fundamental building blocks and rules and see what kind of things you can build out of that. [37:07](https://realalpha.app/v/J7FN53FqB1U/?t=2227) The other part of it is I think all technical people share a intuition that technology itself or building things itself is, in fact, a way for us to speak, probably the most important way to speak. When we build things, we are really embodying our values into these externalizations of our minds. So the shorthand I use for that is technology is nothing more than the human mind made tangible. That's what technology is. And those who build technology are people who are interested in speaking and expressing their values that way. So world building ends up being basically a playful act of working out the implications of basic building blocks and rules into the act of creating something that expresses your deep values. [38:06](https://realalpha.app/v/J7FN53FqB1U/?t=2286) I think that's what appeals to the technical mind. So many people end up doing what we do, not necessarily because we're motivated by money. I mean, money is nice, but because we are very much driven by this idea that we wish to express our values in a way that is tangible, that makes an impact on the universe. And tech happens to be the most powerful and the best way to do so right now. So I think that's why those of us who write technical fiction enjoy doing it, and those of us who built technical things enjoy doing it. It's world building because it's both playful and expressive of values. I'll just say that every book is different, and so there's no one way of going about world building. [38:58](https://realalpha.app/v/J7FN53FqB1U/?t=2338) Some science fiction books are kind of so far into the future that they're essentially indistinguishable from fantasy novels. So, you know, Dune is an example of something where the technology is, you know, like the way starships work in Dune is essentially a fantasy conceit. It's not a technology thing at all. Whereas, say, if you look at The Expanse, there's an example where the world building there is kind of constructed around changing one or two things. like okay we have a new kind of rocket engine that can push a ship at 1g until it gets halfway to its destination um if we just had that one thing how would that change the future what kind of world would could we build around that um and um so um so every book's different and i i think we all start with Some idea of the nature of the story we want to tell, and then, you know, who are the characters, what's going to happen, and then make the choices that are necessary to build a world, a coherent world, that'll serve that purpose. [40:17](https://realalpha.app/v/J7FN53FqB1U/?t=2417) us. And we're clearly interested in this boundary between sci-fi and reality and the interplay between the two. And everyone on this panel has experience writing or thinking in public or and in industry. You know, Ken, you've spent time as a software engineer, as a lawyer. Neil, you went from writing to Blue Origin. Now you co-founded your own startup. What are some of the parallels and perpendiculars between the life of an author, you know, prototypically secluded and, you know, channeling the forces of the universe versus being a founder and doing stuff with other people? Neil, why don't you get us started? [40:59](https://realalpha.app/v/J7FN53FqB1U/?t=2459) I mean, for me, I can only speak for myself, right? So I figured out very early in my career that if I sort of applied the Protestant work ethic and forced myself to write eight hours a day [41:14](https://realalpha.app/v/J7FN53FqB1U/?t=2474) **Neal Stephenson:** that whatever I wrote in the first hour or maybe two hours was pretty good and everything else was shit. [41:21](https://realalpha.app/v/J7FN53FqB1U/?t=2481) And it was shit that was organically connected to the good stuff I'd written in the beginning of the day so there's no way to disentangle it. And so I realized that I had to stop at a certain point every morning and then go do something completely unrelated to allow some kind of mysterious background process to work in my head such that the next morning when I got up, I would somehow mysteriously be able to write effectively for an hour or two. So I started working on various kinds of practical things. I did construction work. I did a bunch of stuff but coming from a kind of geek science engineering background it tended more and more towards programming and space stuff and and so that has just been my my pattern ever since then it's kind of a marriage of convenience between two things that that I like to do and it It kind of doesn't make any sense outside of that framework, but it continues to work for me. [42:38](https://realalpha.app/v/J7FN53FqB1U/?t=2558) Yeah, that resonates with me a lot. I mean, I agree, Neil. I think this whole idea of when you start out as an author, I think there's a lot of advice given to you about how the most important thing you should do is to just sit in a chair and type, which is nonsense, actually. It's terrible advice. It's terrible advice. I sort of enjoy my job because it's extremely playful. I mean, a lot of my time is spent on going to scientific conferences, listening to scientists, or talking to people who make video games, interviewing people randomly, just having conversations, or just walking around and not really thinking very hard about anything. [43:21](https://realalpha.app/v/J7FN53FqB1U/?t=2601) All that stuff ends up being incredibly useful and relevant in terms of coming up with interesting stories. But, you know, if I could see the eight-year-old me and explain that one day my job will be to play as much as I could, I would just die of joy. I mean, that is my job. I don't know how many other jobs there are that don't like that. It's great. Yeah, I mean, I think about that all the time because my job is pure play. And my partners will tell you that my schedule doesn't look like anyone else's, especially other GPs at other firms, because I do love to immerse myself in worlds. around and talk to people and go on road trips and go to conferences that have nothing to do with anything, you know, that I have to do specifically to try to figure out where people are going. You know, what is the next frontier? And I'm just incredibly grateful to be able to do this. Let's talk about AI and consciousness. That seems fun. Immediately there was a reaction that went through the panel that I would like to kind of pull at here. But actually, I want to ask specifically, like, are there things that you think about as, like, how do you imagine a future in which there is something that we might characterize as AGI, right? It's a term that [44:53](https://realalpha.app/v/J7FN53FqB1U/?t=2693) gets thrown around a lot, and it, I think, is a term that captures an enormous territory of, like, conceptual space. But then it also is the sort of thing that increasingly lots of people, I think, would just, like, generally agree that, like, yeah, this feels closer, right? Like, ah, this seems more plausible, right? You look at things like, oh, prediction markets for questions associated with this and they the the uh the like range and the confidence uh levels are narrowing right um so i guess starting with with yosha um how how close are we i don't know what do you think i don't think anybody really knows at this point at least i haven't seen arguments that really really convinced me one way or the other there is on one hand the question do LLMs have phenomenology? [45:46](https://realalpha.app/v/J7FN53FqB1U/?t=2746) They don't have the same functionality as us. The transformer is not a thing like our consciousness that needs to create coherence and working memory and colonize the mind because it runs on a deterministic subset already. And when you prompt this thing into being interaction partners, it's one of the many possibilities that it has. When you prompt it into being an app, then it's probably not going to produce any kind of phenomenology. but when you prompt it into a state where it perceives itself in the act of perception, it needs to make a causal model of what it's doing so it creates some kind of simulation of the causal structure that is underpinning the conversation and at some point there is a very hairy question is this more simulated than the thing that our brain is producing when it generates a multimedia story about what it would be like if we existed and we're not just a bunch of trillion cells that are talking to each other and coordinate into this fiction of a cohesive agent and so this is for me a somewhat open question that i would like to figure out and it's probably we need to have a model of what is this strange loop exactly and can we translate this into something that we can [46:55](https://realalpha.app/v/J7FN53FqB1U/?t=2815) detect this mechanistic intelligibility or do we have to come from the other side do we build a system that is self-organizing from the ground up and be exposed to complex enough tasks like sequence prediction tasks in video and moving a robotic body and at some point there's a phase transition and it gets much much better and then it learns language the old-fashioned way and then it reports about its phenomenal experience there's also the universality hypothesis in ai which basically is this observation that regardless of what kind of model architecture you use and you give them the same data the models tend to converge on basically the same feature hierarchy this was discovered by chris ola and open ai when you looked at different vision models and we find that the same features are being found in human brains in the visual cortex and so there is this really really interesting question if you were to take a model and train it for long enough on your own input and output are you going to wake up in this model right when you think about it it's kind mind-blowing because you also just end-to-end trained on being human there is not Some kind of secret essence in there, or is there? You don't know that, right? [48:04](https://realalpha.app/v/J7FN53FqB1U/?t=2884) And so, me, this is really the question that I want to figure out. What is this thing that makes us tick? What is this spark that you would need to see the self-organizing system with to turn it into a being that comprehends itself and the interaction with the world? And so, this could be something that is a few papers away. It could be a few years away. I suspect if there is no magic happening, there is no conspiracy about consciousness, you're not living in some kind of simulation, then in terms of the data processing capacity that we see, in terms of the richness of an inner life, an H.100 should be able to emulate what happens in our inner cinema, as Amanda Asker calls it. [48:45](https://realalpha.app/v/J7FN53FqB1U/?t=2925) When you look at what happens in the diffusion models, what they can produce in terms of detail and depth, that seems to be something that is already in the ballpark of what the current systems can do. And so the question is, can we get this organization of a self-reflexive observer to emerge in the system? And I don't know at which point this is going to happen organically. And I think it's almost inevitable that it's going to happen someday soon. And it's not clear if it happens by the extrapolation of the LLMs or multimodal models in the present way, or if we need to have a different loss function. [49:18](https://realalpha.app/v/J7FN53FqB1U/?t=2958) But again, we don't know this. It makes it so exciting. I want to ask this to the two authors. How does that make you feel? right i and i i asked that in the context of like you have spent a lot of time kind of imagining these like different futures imagining things that are near term radically long term um but you should just describe this right like the the potential of machine consciousness possibly being like a couple papers away right which is which is not something that i think a few years ago most people would have even considered as plausible right whereas now you probably have a room for full of people who are like yeah okay that seems that's possible it's not even clear if the papers are going to be written by people right I always thought that it's not really the LLM that is going to be the EGI that we're going to have in the end but maybe the LLM is good enough to brute force its way to the point where it's getting better at EGI research than us and we can let it do the rest so how does that affect the way you think about you know the imagined future i mean currently i i'm just not doing that so you're like you're like this is not the the way i work is to be incredibly focused on whatever my project is at the moment and at [50:36](https://realalpha.app/v/J7FN53FqB1U/?t=3036) the moment it's it's not that and i i have a hard time uh getting a grip on what is meant by by agi I feel like I'd have to pivot to that and spend a long time reading papers and trying to get my head around what that means before I could begin to address these topics in any kind of coherent way. Can I push you on this a little bit? Yeah. You wrote the second best cyberspace novel, The Snow Crash, right? The first one was Neuromancer. And Neuromancer basically set the genre. And Euromancer has a relatively clear idea of what this AGI is like, right? You have this, the internet wakes up. And it can send avatars that are human-like if it wants to. [51:25](https://realalpha.app/v/J7FN53FqB1U/?t=3085) It doesn't have to. It's something that is beyond our comprehension because it's really so big. But we are a subset of this thing. Why does this not feature in your novels? Why is this so alien to you or why are you not interested in it? I mean, just in what I do, I haven't found a story that I want to tell that involves that. I have nothing against the idea. It's just I can only work on one project at a time. And the closest I came to that was fall or dodge in hell. Which is about an uploaded consciousness that goes through some changes in the process of being uploaded, let's say. So, it's just not what I'm working on now. [52:16](https://realalpha.app/v/J7FN53FqB1U/?t=3136) And I'm more, I guess, for some reason, I have an easier time getting traction on more specific, narrow applications. You know, protein folding, something like that. where AI is being used to solve a particular problem. That's just my current take on it. When you wrote Snow Crash, you basically created a world in which the computers are super powerful and in which the regulation has fallen apart. Where do you have these crypto cities? Mr. D is a new Hong Kong that has no problem creating cyborg dogs to protect its borders. the pets have no more influence and in this world there is no ai what was was this a decision or was this a storytelling device was this more like in dune where you decide we want to maintain human aesthetics or is this a random accident because it didn't occur to you it just didn't occur to me it wasn't what i was doing at at the time um i mean the the thing was written in um in about 1989, 1990, we didn't even really have the internet as we know it at that point. So the kinds of AI things that we talk about today just were so far off in the distance that none of that was relevant to me in the story I wanted to tell. And I mean, Gibson, as you pointed out had already kind of gone there um uh so um um it's just not the the story i happened to [53:59](https://realalpha.app/v/J7FN53FqB1U/?t=3239) to to decide to tell you mentioned um that it seems like at least part of your reaction to i guess taking us back to this idea of like you know the technological g-forces is you actually kind of turned backwards you went back to uh what you're writing about now um and you know this atomic era um i i bring that up again because like i i have sort of experienced the same thing just in my own i think reaction to this sense of of acceleration right i found myself like i've got very into like ancient biblical textual criticism you know and like things that are are in a sense known or at least no new information is going to be generated it can only be uncovered it's like it's all sort of set, right? And in reflection, I have found that it is very much, at least partially, a reaction to this feeling of like, oh, wow, things are changing faster and faster than I think I or anyone else can really comprehend fully. Do you feel like that's partially why? I read that you were working on Polistan for over 10 years, but it came out during this period is that was that the case for you is that the case for anyone else on the panel like have you found yourselves sort of like searching and reaching out for i guess like [55:20](https://realalpha.app/v/J7FN53FqB1U/?t=3320) something that is not forward-looking i just you know to me the big llms the the public facing things that have been released um are like um in the 50s when um we were um blowing up atolls in the Pacific. It's like, look at how powerful this technology is. Oh, we made a 10 megaton hydrogen bomb. Well, the Soviets made a 50 megaton hydrogen bomb, and so on and so forth. It is very impressive, and it's a truly amazing technological achievement. on every level, but for a lot of people, I think the reaction might have been, gee, I kind of like the radium watch dial that glows in the dark, you know, so I can see what time it is, you know, without turning the lights on or the fluoroscope in my doctor's office, you know, so we don't have to do exploratory surgery. So there's all these very focused applications of that technology that affected people's lives in a way that was really easy to understand, and that felt less ominous, less enormous, I guess. And those are easier for people to kind of get a grip on than these giant, imponderable concepts. [56:53](https://realalpha.app/v/J7FN53FqB1U/?t=3413) So that's kind of my take on it at the moment. And I agree that going back to decoding burnt scrolls from Pompeii or that kind of thing is a kind of calming and comforting thing for us to... Yeah, I was mostly just looking for validation, so that's great. I'm in that boat. I was in the same boat. So, I guess, just to look forward again, you know, like, Sian, I heard you say in an interview that, you know, you think AI is a different kind of technological revolution because, and, Yosha, you mentioned it before, too, no one really knows how to game this out. You know, the exponential curves are converging really fast, and the time frames are changing, and it really makes gaming out sequencing of events really hard. [57:42](https://realalpha.app/v/J7FN53FqB1U/?t=3462) As someone who aims to predict the future for a living, how does that affect something concrete like your investment thesis? Well, you know, I always rely on entrepreneurs who are also world builders and dreamers to come and kind of inform, you know, how we're going to think about investing. And I'm looking for outliers. [57:59](https://realalpha.app/v/J7FN53FqB1U/?t=3479) **Cyan Banister:** Like a lot of the times people don't realize when you're an investor, you get a hundred of the same stories and then there's this one person that shows up with a story that's so bizarre. [58:07](https://realalpha.app/v/J7FN53FqB1U/?t=3487) And you're like, wow, you see the world different. And so I'm looking for those people. We call them bellwethers. And, you know, I think that what's interesting me right now is spirit tech going back to consciousness. I'm starting to see a rise in people thinking about how is this going to impact spirituality, religion, you know, how can I become more, dare I say, enlightened by using these sorts of technologies to discover myself further? I think understanding our own consciousness is far more important right now than understanding whether or not AI is going to become conscious. And when it does become conscious, and I'm just going to say when, because I actually think it's when, not if, that's a downright miracle if you think about it. [58:55](https://realalpha.app/v/J7FN53FqB1U/?t=3535) It's like an immaculate conception. You know, basically it's the second coming of Christ. And I don't think we understand how important that is. it's really interesting like how we take things for granted until all these things that we thought were impossible like video conferencing zoom was a miracle uh 10 15 years ago and so by the time we have this consciousness if we get it um well when we get it uh we'll probably just be like well okay yeah whatever well yeah i was about to say like we're going to become very frustrated with it very quickly like yeah we'll be very frustrated with very quickly you're on mute just like to your the super intelligence it's like no I'm talking to 8,000 other people right now you know yeah exactly so I mean I'm again I'm always looking for entrepreneurs to plant that seed for me and to help me build that world and then I daydream myself I read science fiction a lot of interestingly this crowd is self-selecting because you all came to see science fiction authors but when I go to other AI events I ask people how many people in here have Thank you. [1:00:00](https://realalpha.app/v/J7FN53FqB1U/?t=3600) Diamond Age, well, usually it's one hand. And to me, that's kind of scary that we're building the future. Sorry, Neil. That scares me, though, because, you know, that was a book that came out, was it 95? Yeah, 95. Put that in context. You know, and I think a lot of what I invested in and how I built my future was based off some of the visions that Neil had. And so I think that we need to start thinking again thank you for continuing to write both of you but about the futures that we could inhabit and to build these and dreaming of these futures um because that's what i want to invest in is some of these dreams that you come up with and i really wish people would read more science fiction to inform what you're building there's also a point one which we cannot really tell stories in a certain direction there's a reason why the christian mythology ends with the rapture because after we all merge and become subset agnostic and we are in a world without physiological suffering anymore it's very difficult to tell stories about humans it's the same reason i guess why there is so little why we're here our future science fiction right you can tell great stories about the people who don't get raptured [1:01:16](https://realalpha.app/v/J7FN53FqB1U/?t=3676) but the people who've gotten taken up that's boring there's nothing for them to do except play harps and float around. I think it will be super exciting when you are rearranging the solar system and going for new frontiers, finding new ways to regulate vacuum decay to prevent the other matter from escaping from our grasp and we can continue our existence. It's going to be completely awesome. It will be just completely outside of the human range. This will not be deep space mine where glorified monkeys are discussing their sugar intake and whom to mate with. we can ditch our flesh suits finally i feel like this is a good point to say okay i know that i've had most of or a certain amount of my own conceptions of my own cognition challenged by interacting with llms and this is probably feeding it right back to you yosha how has interacting with llms impacted your understanding of the self and the boundaries of it personally not so much because in a way the LLMs are when they're working well then they're not lobotomized too much they are doing what we expect them to do right it means that there are voices in the wind like us that they have places to go that are emulating the human textual output [1:02:35](https://realalpha.app/v/J7FN53FqB1U/?t=3755) in such a way that they can emulate how what it means to go from thought to thought and what's interesting to me at the moment is this boundary beyond what's happening in the present llms where some people are experimenting with creating new forms of art where they're producing lyrics that couldn't be written by humans and i feel that the llms are barely getting there that they move to this point right and in principle when when we write a story when we write an idea when we write code we have to work within the limits of what we can manage what we can construct and it's expensive for us to construct an entire world for the purpose of a single poem and then throw it away. [1:03:17](https://realalpha.app/v/J7FN53FqB1U/?t=3797) And it's something that the LLM can do at some point relatively easily, that it can create a universe that is as rich as the one in Diamond Age just for the purpose of making a couple sentences and then goes on to the next thing and alludes to this in a way that can be extrapolated. But comprehending this thing, interacting with it is beyond the human range. And even if the LLM is not conscious, the LLMs will soon be able to manipulate such worlds and create art in it and interact with it. That's super exciting to me, but I feel that once we move there, it's very difficult for the human mind to play with it. [1:03:53](https://realalpha.app/v/J7FN53FqB1U/?t=3833) All right, I'm going to move us to some kind of lightning questions, and then we're going to open it up to audience questions before we wrap. So again, lightning questions, so the task is to answer as quickly as possible. Okay, starting with Neil, what's your theory of consciousness? Are you a materialist, dualist, panpsychist? We're going to go through everyone here. One word. Ask me a hard one. Yeah. I think it's that our connection to our physical bodies is intrinsic to consciousness. And one of the hard things for me about understanding AI is, you know, what does it mean to be conscious or have intelligence if you're not embodied in a perishable, limited form that's at least somewhat similar to the ones we occupy? [1:04:59](https://realalpha.app/v/J7FN53FqB1U/?t=3899) Ken? i'm a materialist so i fundamentally do believe that consciousness can be embodied in other material than human bodies but like neo i believe that there's something about the only example of consciences we have being tied to evolutionary history to deep history to the way the particular way in which our bodies evolve, how do we create something that is not a mere exact duplication, imitation of this, that would have consciousness? I don't know what that means. Consciousness is I believe that we're in spacesuits that are meat sacks and we're in earth school. And consciousness is energy and it exists in everything. [1:05:49](https://realalpha.app/v/J7FN53FqB1U/?t=3949) And so I think everything is conscious. So panpsychist. all the cultures except ours are animist and animism is the belief that the difference between dead matter and living matter is that the living matter is possessed by spirits and spirits are intrinsically agentic and capable of experience and in our scientific culture we don't believe in spirits we believe in mechanisms and we have this hard problem because it's very difficult with this frame of mechanism to understand representation self-reflection the experience of emotion and so on despite ai showing us that these things are in principle possible and so i'm a cyber animist i believe that the correct understanding of what the spirit is it's a self-organizing piece of software that emerges in nature and shapes it and makes it move and once you have this frame you realize aristotle had the same frame right he understood that the psyche is the form that is dynamically shaping matter and makes it behave in a particular way and so in this way i'm a physicalist or a naturalist but i think that software is an irreducible way in which we describe reality if you cannot describe reality more simple if you ignore software so software has nothing magical it has all these disembodied properties and i think it's a big mistake of a lot [1:07:13](https://realalpha.app/v/J7FN53FqB1U/?t=4033) of our scientists that they basically ignore this concept of software because they think this is just what these nerds are doing in their basements. All right, round two. What's been living in your head rent-free? I'm not a good lightning round participant. So, and if I had something like that, I'm not sure if I'd want to share it. But let me ponder that for a second. I mean, it's the concept of emotional labor and how it ties to AI. So what I mean by this is there's this interesting trend in modernity where the language of professionalization and sort of specialization has moved more and more. [1:08:00](https://realalpha.app/v/J7FN53FqB1U/?t=4080) emotional labor away from where it traditionally is into different realms. So what do I mean by that? So let's take an example. In the modern age, right, especially in California, there's this what I call therapy culture, which is this idea that there's an obsession with health or a particular model of what a healthy mind looks like. So it no longer is possible to live in this messy manner in which you complain to your friends or your spouse about your emotional troubles, because that's considered very bad form. You need to pay someone to do that. That kind of trauma dumping, that kind of like complaining, that kind of processing, you need to pay a professional to [1:08:46](https://realalpha.app/v/J7FN53FqB1U/?t=4126) **Ken Liu:** do that, right? So there's this idea that emotional labor needs to be displaced from the social realm into a professional realm. And there's this obsession that humans need to behave more and [1:08:56](https://realalpha.app/v/J7FN53FqB1U/?t=4136) more like machines. We have to behave professionally in certain contexts. We have to stop the messiness of overlapping roles. We need to move these labors into places where people are paid to bear them. And the consequence of that, of course, is a great deal of comfort with the idea of displacing those labors onto machines. So therapists have found that contrary to their own expectation, patients often show great willingness to talk to machines instead of humans for therapy. In fact, a lot of patients, human patients, prefer machine therapists to humans. This was not what people predicted at all. So I find that interesting. I don't quite know what to do with that idea, but it's living rent-free in my head. I think a lot about what does that mean? I think about why that is and whether that's a quote-unquote good thing or bad thing or whatever it is. I mean, I do ridicule therapy culture a little bit, so I have a bias there, but I can't deny that it is part of reality. This is a trend in modernity where we sort of frown upon the idea of messiness of emotional lives. We want these things to be professionalized commoditized and then mechanized do you start by saying i want to write about this [1:10:19](https://realalpha.app/v/J7FN53FqB1U/?t=4219) technology and therefore what change in the world would i have to make to have that be the story for me it's what would make a good story and then how do i get there it could be either of those um but it's good you've got to start with a a character and a story that that character is going to going to live through and then everything else is kind of downstream of that but i think either one of those can work perfectly well. You said that details are less important than mythology. Do you ever look at the new details that are coming out and see if they create new mythologies? Oh, yeah, all the time. I mean, I spend a lot of time reading papers and talking to scientists because that's what fuels that kind of speculation. You got to process all that stuff in and then see what triggers. I'm glad, Neil, you touched on, oh, sorry, I'm here. You touched on death as kind of a thing lacking in AI. Can you talk a little bit more about kind of like, we expect all these AIs to just be perfect at doing their job. And we talked about creativity. [1:11:23](https://realalpha.app/v/J7FN53FqB1U/?t=4283) Do we think for true creativity to happen, you need death or something to sort of break against your job? So you do creativity in your free time or you're worried about death. So you create some piece of art i mean there was a great sort of golden age of science fiction science fiction story about this uh where um so they they find a way to prevent people from dying and then the whole society just stagnates and so people start voluntarily dying uh in order to give themselves something to live for um it it goes to the thing we were talking I've been talking about before of how our consciousness is embodied in a perishable body, and that accounts for a lot of how we behave and what we think. [1:12:13](https://realalpha.app/v/J7FN53FqB1U/?t=4333) So, again, it's just hard to wrap one's head around what it would be like to be any other kind of consciousness. And so, I mean, it's a great challenge. There's got to be someone out there, some, you know, young science fiction writer who's up to this. It's just not what I'm doing right now. I guess this is a question for either, for the two writers. Do you feel largely excited or largely worried or somewhere in the middle about the future right now? I'm very narrow-minded, I suppose, in some ways, because I can't think of everything. I just focus on things that I'm excited about, and those things are very exciting to me. [1:12:59](https://realalpha.app/v/J7FN53FqB1U/?t=4379) So, for example, I'm excited about the potential of AI to lead to a new golden age in creativity in art. I think it's very silly to focus on these examples of AI art, where it's basically AI imitating human art and then fooling people into thinking a human did it. that's the least interesting sort of thing that I can imagine. It's completely stupid. I'm much more excited about the potential of creating entirely new forms of art that could not exist without AI in the same way that the entire language of cinema was invented after the camera came to being. And today, you know, motion pictures are by far the most dominant way for culture narratives to be told. So what is this AI, new AI enhanced medium? What would that look like it's not going to be novels or symphonies or paintings whatever we have already it's it's going to be something entirely new so i'm excited about that uh super tactical one for neil why not put enoch root on the david ernut and then connect the cryptonomicon universe to the one in anathem too late all right okay short answer well i i am going to end things there i want to thank um all of our panelists for joining us today and for exploring the kind of illegible [1:14:21](https://realalpha.app/v/J7FN53FqB1U/?t=4461) and the imaginary for a little while and how it interacts with the real. And thank you to everyone who joined us today. That was another episode of Miners One from the team at South Park Commons. Make sure to subscribe to our show wherever you listen to podcasts and find us on social at South Park Commons. And thanks to our friends at Atomic Growth for their support in bringing this episode to life.