How the AI age forgets to ask: "What for?" | Benjamín Labatut + Jasmine Sun

0:08 Thank you, Lauren. That was lovely and it's so good to be here with you. Nice to be here. Well, I want to start with a question that gets at what Lauren's introduction is saying, which is you cover these deeply technical and scientific concepts in your novels, from quantum computing and physics to advanced AI innovations like AlphaGo, and I'm really curious what it is about literary writing in particular that you're drawn to as a vehicle for exploring these technologies?

0:42 Well, because I think the human phenomena is much more complex than, can be captured with something like nonfiction. I think, hearing—participating in these talks, you get a sense of something that's being left out, something fundamental, and I think that just goes back to the way that our—at least this part of civilization, has evolved. We have taken a definite direction towards the digital and that leaves out like the continuous, no? And I think we are really unlike the things, these things that we're creating.

1:22 We are continuous beings, we are not digital, and there's an enormous part that is left out. Literature tries to, you know, weave the rainbow back together. It involves irrationality, it involves all of those things that science has, by its own measure, no by its own method, left out. Literature just tries to put it back in, so presents a messier, darker and perhaps more complete, if less powerful, perspective on the world.

1:56 What do you mean when you say continuous, exactly? Well, I think that that's incredibly profound subject, that I have 16 minutes could not explain it, but just listening to the things, and looking at the visuals of the event, no? I just feel like I'm back at a time when people were washing their teeth with like radioactive products and and smiling—like beaming, no?

2:26 It all feels sort of 50s, a nuclear enthusiasm. And I have to sort of before I even—I couldn't, I couldn't attempt to answer the difficulties posed by the fact that most of our being right now is sort of digital and discrete and divided into things that can be easily accessed through rationality, logic, our computer systems, all work like this. The equations behind them are sort of like that.

2:58 It goes back to the foundation of this technology, no? Like the McCulloch-Pitts neuron, right? It's an abstraction that has nothing to do—and we have built with— it's a mathematical model of a neuron. It's basically Boolean logic applied to the idea, the abstraction, that a neuron either fires or it doesn't, and that is the ground zero of AI. That's it. So because I'm writing about it, and you study the people involved, and you immediately understand what's left out. And the fact that after that paper, which really kicks off everything, no? After that neuron, neural nets arise from that.

3:44 But the people who did that paper, like McCulloch and Pitts—Pitts drank himself to death because he was accused of raping his mentor's daughter. And McCulloch was a brilliant philosopher scientist, who ended up trying to find a new type of non-digital, non two-valued logic, working in a tiny study, and he also kind of drank himself to death. So what I do in literature is sort of: if you get, if you actually look at the people who do the fundamental discoveries and look into their lives and try to sort of look into their minds as well, like their souls, and you get past the advertising and the—like I was wondering—I was at the back looking at the beginning of the conference and I said, "Well, how about we add a little like AI slop to the visuals?"

4:36 No? Or some of the— because we all have visions of a really dark future. A very non-human future, but we don't include it in, at least not in the aesthetics. But I think that's coming and it's coming like— I think this is a precious time to be here because I think we're going to replace this enthusiasm with a little bit of shame and fear. I think it's happening to the people who created these technologies.

5:06 Their enormous enthusiasm is being replaced by something else. Well, let's talk about one of the people who was someone who was a forefather of the technology. So, in your novel, 'The MANIAC', the middle section is this partly fictionalized but historically grounded biography of John von Neumann. And he appears as this kind of flesh-and-blood incarnation of superintelligence, somebody who is brilliant but also terrifying because he is brilliant.

5:37 I'd love if you could say more about what made his character in particular so compelling to you. Well, not just because of the fact that he was such an astounding scientist, mathematician. But the people who used to talk to him, about him, it's like hearing someone talk about a superintelligent AI. And the way that he affected those around him, the way that he would suddenly maybe meet someone in a corridor and destroy their PhD thesis in 35 seconds.

6:08 And the vistas that he had on humanity, no? It's a cold and calculating logic-driven perspective. And I used von Neumann just to show I'm fascinated by his blind spots as a person, as a thinker. Luckily, we are not a species that reasons only. And the fact that our ways of being will always be more than our ways of knowing.

6:44 And this is how many of the problems that we face as individuals and as a species, of course, you can look at them with logic and reason. But then you get to scenarios like mutually assured destruction, because that's where it leads. Because it is an either-or and if-not-this-then-this mentality. But we have other ways of going about things. There are—the biggest problems. We don't solve them with our minds.

7:17 We sort of just live through them. And we are changed by them. And I think that we're at a moment where this is no longer science fiction. But it's going to start to interact with the messiness of the world. And if there is one thing that I think I could bet all my money on is that we will get the bad almost for sure. Because the good is always harder. Not just from the point of view of science, from the point of view of an individual.

7:49 You can always—the terrible things are easily reachable, right? But to change yourself in a meaningful way—to be better, not faster or cheaper. To be better is difficult. And I think that I have—I think that realism, optimism and realism at this point, we can even throw those perspectives away and just look around right now at what is happening. How we're living our lives.

8:27 I don't see that sort of bright 2.5% increase. I don't think we're going to sleep sound just because we're going to grow 0.5% faster. Yeah, I remember when Claude Code came out and I started playing with it. You first feel this excitement at the technology and how much you can create. And then I start to wonder how many of my problems are solved by software. And the answer is less than you think probably, most of the time.

8:59 How many of your problems are caused by software? But we're about to have the Hermès of software, so maybe no more. One thing that I really love about your retelling of the AlphaGo story at the end of 'The MANIAC' is that it sort of holds the light and the dark. It is both suffused with this clear marveling at the capabilities of the technology. You really understand and you appreciate these systems, and it also has the emotional texture and the sadness and the tragedy of the human players who have lost to AlphaGo.

9:33 And then what's interesting is the very last sentence of 'The MANIAC', it doesn't end with Lee Sedol's loss, it ends with the invention of AlphaZero, this successor system that didn't even need any human data to train on. I'm curious why you chose to leave readers with that final image. Because I think it's the trajectory that we're on, and I think it's a mistake. It's more exciting to think about AlphaZero and then AlphaFold and Alpha whatever.

10:05 Alpha, Beta, Gamma. But I'm sure that Lee Sedol's life after that has been more interesting. If you had to choose as a being—well, no. And that's the sort of— we forget to ask, I think, the right questions. It's like the questions are "How much", "How quick", and we sort of forget "What for?", like "What for?" I'm sure in this audience there's a bunch of people who know— have met the people who are driving these.

10:39 They're not very interesting people. I've been amazed by it. And I'm like—what they're doing is fascinating, but we are living things. We're living beings. And I think that the trajectories that we're on right now, I think about Lee Sedol who quit. He stopped playing. And then I'm like, what was I…? The thing that seduced me the most about him, of course, he was a genius, right?

11:14 But then he has this obsession with KPop dramas. And I imagine him singing in the shower, like in a really weird voice that he has. And I thought, "Well, yeah, yeah, that's the human phenomena." That is the human phenomena. The entire thing, that he has a family, that he has kids. And we leave it aside because we're caught in abstraction. We're caught by the— We're sort of enamored of our abstraction. We're enamored of the things that we can do.

11:42 And we forget what for? I don't think things are getting any better. I don't think things are getting any better. I mean, they might be getting flashier. But like, and not even just that. The AI that we have right now, let's forget a couple— The AI that we're getting right now, I can't get it to write a single good paragraph. And I've tried. And I'm sure you all have. And I'm like, "What do you mean?" You can read every book.

12:10 And like, what does— Do I need to pay more? For… I've tried the $200 a month version. They're not writing poetry either. What did you get out of it? Not a lot. Yeah, I mean, I think what's interesting is, in a way that makes me feel better that it can't write. Maybe just because I'm a writer and that's my cope. But I think it pushes people to write in more interesting ways. Because you don't want to just be remixing other ideas.

12:39 Because it can do that already. I mean, I'm interested to see where the systems will go. Maybe they will be able to do good poetry in a few years from now. I actually won't be surprised if they do. I mean, you alluded to this. But I think there are a lot of people in the audience who are scientists, and technologists, and engineers. People who are excited about building some version of superintelligence. Or maybe about superintelligence that accompanies or augments humans.

13:04 I'm curious what message you would leave these folks with as they go on their journeys. We're all like drunk on these words, 'super', 'ultra'... And they just obfuscate the fact that, like, again, with the superintelligence thing. There are ways of knowing that are not intelligence-based. There are lived processes. They affect everything about you.

13:34 We are not this sort of brain in a jar. We are not that. It is a part of us and we have— It's amazing that we've managed to sort of prove this hypothesis that intelligence is not substrate-dependent. That's fine. It doesn't take anything away from the fact that— How about well, how about they start thinking about something—a super loving being or a super sexy being.

14:05 They're building those AIs too. Oh, I know I want one of those robots as soon as it's out. But I don't think we'll be able to take them out with us because people will just shame us, no? And wait—but no. So, okay super intelligence, right? Yeah, well let's say we have it tomorrow. And then let's say we have the brilliant idea and we put it inside one of these robots. You told me the impression that you got from spending time with them.

14:37 And like, what was it? What did you feel immediately? I was in China—I was at Unitree, the leading humanoid robotics company. And when you stand face to face with a humanoid robot the first thought that you have, every one of us had before anything else, it's something precognitive, is "This thing could kill me." And it's evolutionary. It's psychological. It's crazy. In the same way that a chatbot talks back to you and you think you care about it, you stand face-to-face with a humanoid and you think, "This could kill me." That is just absolutely fundamental.

15:11 That is your entire being telling you something profound about what it means to be alive and what it means to be a human being. Like our first filter we pass anybody through is like "Is this guy a psychopath?", like "Is he going to kill me?" And the—and I don't want to—and I keep falling into jokes, but I want to be serious for a second. The whole way that we talk about this technology. The way that CEOs talk about it. It is chickens coming home to roost time.

15:43 We've spoken about taking everybody's job. We've spoken about measures, the percentage at which we're going to destroy the human race. Let's just take ourselves seriously. Let's take what we're doing seriously. There is a plan B and a plan C. There's also a great thing, which is no-fucking-clue-plan. No plan. We don't have a plan and I agree—yes, we're going through this and I don't believe anybody's plan.

16:16 Nobody who is intellectually honest will tell you a plan. I've spent time with, I don't know—Demis Hassabis. And we're like, "What do you think?" "I don't know. What do you think?" People are—we're fundamentally lost. Which to me, what does it signal? That we— if we navigate this space, it won't be by thinking about it. We're gonna live through it, we're gonna, and I hope we don't—let's just listen to the part of our brain that says "Kill a robot," no?

16:43 Trust that. How do you think that Demis feels when he encounters the sort of enormity of this? Of the thing he's doing. Well, I love him. I'm a friend. So I don't I'm not gonna betray— the truth of our conversations. But there is that level, right? Everybody has this sort of what they will say in private vs. what they will say in public. And I think Demis is a wonderful example of our culture's Faustian pact, no? This thirst for knowledge.

17:20 I think that, you know, all our stories are like, "Should I pick this cup? And then drink it, and live forever, and know everything?" or "Should I just be this thing?" And wisdom has always said leave that to the Gods. Leave it to the gods. You are not immortal and you are not all-knowing, and that is what makes you precious. You are precious because you're weak.

17:51 You're limited. We disabused ourself from the notion that we will live forever. So we're living in this scary time. So let's be a little bit more human. Yeah, there's an interesting way in which even though Tyler is an optimist and you are not you sort of converge on some of the same ideas around the limits of intelligence and rationality and everything else that we are. Well, thank you for having this conversation. Thank you so much. Sorry for bumming everybody out.