Section Insights
Anthropic's IPO and Existential Questions
What are the existential questions being raised by Anthropic regarding AI?
As Anthropic prepares for a significant IPO, both internal and external stakeholders are questioning the sustainability of their growth and the implications of their AI, Claude. They ponder whether Claude possesses consciousness and moral worth, leading to discussions with religious leaders about these profound issues.
- Anthropic is facing existential questions about AI consciousness and morality.
- The company is preparing for a massive IPO, raising concerns about sustainability.
- Discussions with religious leaders indicate a serious approach to AI ethics.
Engagement with Religious Leaders
How effective was Anthropic's engagement with religious leaders regarding AI ethics?
Anthropic's meetings with religious leaders opened up discussions about AI consciousness, with varying responses from attendees. Some were receptive, while others remained skeptical, indicating a complex landscape of beliefs about AI's moral status.
- Anthropic's project effectively opened dialogue about AI ethics among religious communities.
- Responses varied, with some leaders embracing the idea of AI consciousness while others were hesitant.
- The engagement reflects a broader concern about the moral implications of AI.
The Role of Religion in AI Development
What is the significance of Anthropic's meetings with religious leaders?
Anthropic's meetings with religious leaders are seen as both a research initiative and a form of evangelism, aiming to gain moral authority. However, the actual impact of these discussions on their AI development remains unclear.
- The meetings serve as both research and a strategy to gain moral legitimacy.
- There is skepticism about whether these discussions will lead to meaningful changes in AI development.
- Anthropic's approach is unique in the tech industry, highlighting the intersection of technology and ethics.
Urgency and Responsibility in AI Development
What concerns do Anthropic employees have regarding the implications of AI?
Anthropic employees express a profound sense of urgency about the potential misuse of AI, such as in bioweapons development. This weight of responsibility drives them to seek guidance from religious leaders, indicating their awareness of the ethical stakes involved.
- Employees feel a heavy burden regarding the ethical implications of their work.
- There is a pressing concern about the potential for AI to be misused in harmful ways.
- The search for moral guidance reflects the seriousness with which they approach their responsibilities.
Mainstreaming of Rationalist Ideas
How are rationalist ideas influencing the discourse around AI?
Rationalist and effective altruism ideas are gaining traction in mainstream discussions about AI, prompting varied reactions. Some embrace these concepts, while others find them challenging or off-putting, highlighting the complexities of integrating these ideas into societal values.
- Rationalist ideas are becoming more prominent in discussions about AI ethics.
- Reactions to these ideas are mixed, with some finding them difficult to accept.
- The integration of these concepts into societal values raises important questions about future governance.
Transcript
0:01 Next month, Anthropic plans to go public in what's likely to be the largest IPO in history. We're talking a $2 trillion company. Outside the company, everyone is wondering, "How big can this get? Is this sustainable? Is there an AI bubble waiting to burst?" Inside the company, they're asking those questions, too. But Anthropic is known as the most anxious or the most existential of the AI companies. So, they're also asking some other questions. Is Claude, their large language model, alive? conscious worthy of moral consideration? Does it feel pain? Does it have a soul? Can we teach it to act morally? These aren't new questions. In fact, they animate most science fiction for better and for worse. But science fiction is, you know, fiction. And anthropic seems to think whatever is happening with Claude is very different, very real. It might be why CEO Dario Amade was the first to call for a pause in AI development. And it might be why they've been secretly meeting with religious and spiritual leaders to ask them those same questions. I learned this from an amazing piece by the times religion correspondent Elizabeth Das who broke the story of these meetings and of the reaction to anthropic's request among the assembled priests, rabbis, and gurus. I thought Elizabeth's story was fascinating. It was a new way into the endless consciousness debate. It took me inside this future $2 trillion company as it struggles to understand what it's built and how to treat it. and it showed me what happens when people in institutions that are more used to doctrinal debates and theological rumination start being consulted on software development. Unsurprisingly, anthropics efforts don't seem to have brought us any closer to answering their questions. If anything, they've simply opened up more questions like, are these people maniacs or is this exactly what they should be doing? An obvious conclusion is that the worlds of tech and religion are getting much closer together. After all, as Elizabeth told me, if you've got questions about the future on a multi-entury timeline, there aren't many better institutions to ask than the Catholic Church.
1:56 So, I got Elizabeth together with my colleagues, tech reporter Aaron Griffith and opinion writer David Wallace Wells to explore this weird new development and what it means for anthropic, for AI, for religion, and for the rest of us. I'm Max Reed. This is Hard Fork. I've spent a lot of time in the last day thinking about Elizabeth's piece and really specifically I've been trying to write a joke that I think is going to get a great reaction from the three of you. So, >> okay.
2:28 >> A priest, a rabbi, and a large language model walk into a bar. >> I hate it already. >> The bartender says, "First round's on me." And the priest says, "I'll have a whiskey shot." And the rabbi says, "I'll have a vodka shot." And the large language model says, "Nothing for me, thanks. I excel at zero shot learning." >> Boo. >> Whitney, can we get some laughter piped in there, please? We'll just we'll roll that again with life. No, I I want to start the conversation with this piece that Elizabeth wrote. This this fantastic, really fascinating story called Religious Scholars Met with anthropic.
3:08 What they heard stunned them, which was in the Times last week. Elizabeth, can you tell us a little bit about, you know, like what this piece was? >> Yeah. So, right, I cover religion. and now I cover tech apparently. But the story started for me when I was hearing about these different religious thinkers sort of across the whole world who were going into meetings at anthropics headquarters either in discrete like private meetings or more a bigger you know groups of 20 these summits mostly under non-disclosure agreements to not reveal anthropics research to talk about very big moral questions that Anthropic wanted to put in front of them. and there were two, one was kind of broaching this question of AI consciousness and two, Anthropic wanted to learn in their words like how to morally form Claude to be a morally good entity. And then I just had this incoming, you know, first from these religious thinkers about what their experience was like at Anthropic.
4:24 what Anthropic wanted to know, what the issues they were concerned about were. And then also an with anthropic like learning from Chris Ola, one of the company's co-founders who's been spearheading this project, what his own moral questions were that were kind of driving a lot of what they talked about as research in the religious and moral space. >> can I just ask I want to establish like sort of on a high level what does anthropic specifically want out of this because it sounds like they're they're doing two or three different things.
4:56 it's maybe a little bit unfocused or it's just that there's a few different kinds of conversations they're having. >> Mhm. Yeah, I think it's true. There's multiple conversations Anthropic was having and it evolved over time. I mean, this is such a fastmoving target. but for Anthropic, the journey started for Christopher Ola. he you know in his own life story had connections to the Christian space growing up. he left evangelical faith and has talked since then about how he's more drawn to Buddhism. but he was worried about this question of of how the even the idea of AI consciousness might be received especially in Christian communities. I mean that's it could be perceived as heret heretical like not just weird and silly but like this is fundamentally against our religious beliefs. and so he sort of thought about a year ago he wanted to be a bridge figure. He wanted to see like how he could kind of go between Claude and Christ, right? And so he reached out to a Catholic moral ethicist who specializes in bioeththics and they had some conversations about that.
6:12 But then it it turned to Anthropic's goal for these convenings which was how do we morally form that's their language like how do we morally form Claude like on all all of our AI models into like a morally good entity. and the thinking was if they could do that if they could make Claude good then the world would be more safe. But, you know, they haven't said or shown how they plan to use what the outcome of these conversations. that's pretty unclear even to the participants. currently, >> the way that you're describing it, it almost sounds like there are sort of two separate purposes, right? One is like the alignment project and what can we learn from these you know these wise men and women about how to raise moral actors and then there's this project of sort of building a sort of spiritual system around AI more generally and you said a minute ago that Chris Ola started this project with some anxiety about what it would mean to go to religious leaders obviously convinced himself that AI and LLMs were perhaps conscious already, but certainly like on the path to consciousness and worried about what the response would be.
7:31 >> And so I wondered how you like how how that played out on the ground. Like did a lot of the people who showed up >> were they worried about how weird it was that all of these people on the anthropic side of the room were already sure that they were, you know, masters of thousands of souls? and like how did they ash that side of the debate out? Mhm. Well, Anthropic was very careful in all of their interviews with me to say very clearly that they're uncertain, right? Chris never went as far as saying like, "Absolutely, these are conscious beings." It was very evident from how he and his team talked to their guests that they you know he was concerned about how to treat Claude as a you know he was very worried that it was an entity that was experienced suffering and was worried about its mental health and these kinds of things. So it's it's pretty clearly he's pretty far along in how he thinks about it. And the I could hear in talking especially cuz I did this story like over several months there was a progression of thought for a lot of these thinkers too. and some of them you know bought in right away to what they were hearing about Claude's potential consciousness and needing to treat it with a certain amount of moral care. but others you know if they went in more closed-minded about it they definitely opened the way that they think about it afterwards. So in that sense, Anthropic's project was effective, I think, in like opening how they think about it into these communities, many of whom especially on the Christian side are are more hesitant.
9:17 >> There's a ton of great material in this. It's really well reported and it covers a ton of ground. I'm wondering, David and Erin, if you guys had particular pieces of it that really stuck out to you. >> >> I mean, the one thing is that you mentioned already is the NDAs. I mean, it it just strikes me as a little bit crazy, maybe a little paranoid that you're inviting all these people in to sort of have a summit to discuss these really important critical ideas for the future of society, but it has to be done totally in secret. And then it sounds like maybe from your article that they dropped the NDAs. Was it as a result of your reporting or did they just decide that they were time bound or how did that >> Yeah, I can't prove that. I don't know that. but I when I asked them I've asked at several different points because the NDA process was not the same for everybody that I interviewed. and it seemed to change at different times.
10:09 >> I mean, in general, Anthropic is a very secretive company. Yeah, I mean they they they are the employees are like extremely loyal and and until somewhat recently there hasn't been many leaks to ever come out of there because they view it as such a betrayal. so it makes sense that they would like want to lock it down but at the same time I think it probably was a little bit of a a misstep in the same way that these companies realize that having NDAs for their data center deals was a misstep because it created distrust amongst all the people in the towns that they're they're going to see.
10:40 >> Yeah. And whenever I find in my reporting that there's an NDA about like a moral topic you know and it makes me immediately ask okay who's who's benefiting from this and like why is this needed >> and what does that mean about the kinds of questions that are and aren't being asked right it invited like more curiosity from me as a reporter for sure. >> David what about you what what did you take away from the piece? Well, I guess I started it with a sort of conventional view of anthropic as like the humanist company among these AI companies. And what I was struck by in the piece were a few moments in which I felt really alienated from Chris Ola himself and maybe to some extent the sort of operating worldview of the company as a whole. And those were moments when in one case a rabbi challenges him suggesting that if LLMs are actually conscious that means that all of these companies are running huge slave plantations and they should have a moral obligation to liberate the slaves rather than make more of them and make them work harder. And it seems as though Allah himself didn't really even clock that criticism. and then later on when he reacts with horror upon receiving the Pope's encyclical about AI, seeing that the Pope is so opposed to the idea of machine consciousness, and I just found myself thinking, you know, what kind of a bubble you must be living in to, to expect that Pope Leo or really anyone in the Catholic tradition would have a different intuition. would would in thinking about the future of AI think that we should be thinking of LLMs as having souls or having consciousness. And both of those moments just illustrated for me what feel like this this big gap between what I had assumed the sort of natural humanistic values of a company like Anthropic would be and how the people inside it seem to really be thinking about the work they're doing at the sort of cosmic scale.
12:43 >> Yeah. the experience for me going into Silicon Valley and like kind of deep in these like private conversations at the heart of anthropics mission. it reminded me of like when I go into and I do rel like something that's more straight religion reporting into an insular community when everything inside that community like makes sense to everyone living in it but once you step outside it there's a real clash with the broader world and it was funny like I had these two poles in the story right I was at the Vatican right at the inner sanctum of the Catholic Church and centuries of philosophy about the human person and then I was also in a different inner sanctum right that's being created in real time at these tech companies like Anthropic and it is a kind of philosophical system like oh a quasi religious system that's being constructed it reminded me of like a a creation narrative that you hear about in religious traditions >> I want to step back for a sec because I think this relates to the conversation we're having right now. David brought up the papal encyclical which I think is in English is magnificent humanity. I'm not going to even try to say the Latin title. This plays a role in your story.
14:03 because one person who is not really thinking at all that Claude is conscious is the pope. and this was a little conflict between Croissola who was present when the encyclical was read out. If you want to tell us about that. >> Yeah. So, Pope Leo you know from the moment he was named that he was was clear he was going to be the next pope and he was elected, he made very clear that a focus of his papacy was going to be AI and you know h how to understand that and how to protect humans in this changing world. And I found out in my reporting, you know, as the Vatican internally was trying to figure out how it wanted to present the Pope's encyclical to the world, they were thinking about their launch event.
14:52 And they wanted to include a representative from Silicon Valley, like one of the the the labs at the Pope's event on the Pope's stage. And the decision was made to invite Anthropic because of the focus that they had heard about Anthropic, like really caring about ethics. And this was also right around the time that Anthropic got into it with the Trump administration the Department of defense over the use how how they would and wouldn't want the government to use their technology for refusing to have it serve be involved in mass surveillance of citizens, etc. Anyway, the thing with these encyclicals is they're very closely guarded even among the the people who are participating in the launch event. And you know, I learned that Chris Ola only saw the full text of the encyclical a few days before the event in May. And this was right around the time I first interviewed him in San Francisco. And then I interviewed him again when we were at the Vatican for this for the launch event. And you could hear, I mean, like looking at what Pope Leo said, which is extremely clear, the church does not support machine consciousness. The priority is what he calls safeguarding the human.
16:11 and that's the the center of all all the moral questions that he wants the world to be thinking about on this topic. but Chris Olas like he he ended up you know, having to decide. who said like he he may may not he suggested to the Vatican that he may not come over the the difference in consciousness because he wanted to be as you know it was described to me he wanted to be faithful to his own moral convictions some of which he shares with the Vatican and some on consciousness he did not. So it it it really all came to a head very much behind the scenes, but you could see this moral fault line really developing between the church which has been like a center of power for centuries and this very new power center that's very rapidly you know really getting followers and organizing people's lives and those are functions the church used to have.
17:19 Erin, I wanted to ask you a question because we were talking a little bit about like you know surprises in this in this area. Anthropic is pretty well known as a kind of like the consciousness company. It's the it's the company where the people who really are strong believers kind of work and run the show. Was it surprising to you as somebody who's reporting on the valley that like they were having these meetings behind the scenes with religious leaders? I've not heard of any other companies that are working on AI really like going to this extent to get input from this community. I mean the question I have sort of is like how much is this really actually going to inform at the end of the day their own view on it? Are they actually like really it it feels like to me they're trying to win over others so they can say look we do have the moral authority. we have convinced these important representatives of this community. So everybody else come along with me as opposed to you know actually learning and incorporating those ideas of like moral goodness into their own product.
18:18 They kind of already have it figured out. Well, I think for anthropic it was part research and part evangelism. That's that was pretty clear. and the you know I've asked directly what the outcome is like what what is this research what are these conversations being used for and is there been any demonstrable like proof of showing how they have or have not made Claude act better. haven't had that yet. so I don't know if they're withholding that until a future time or exactly what that will look like. The other side of this that actually is kind of interesting to me, and Elizabeth, I'd be curious to hear this in your if this came up at all in your reporting, is that there has been a little bit of like a a new rise of Christianity throughout Silicon Valley over the last couple of years.
19:12 Like it used to almost be like taboo to talk about religion or anything religious. Like you can remember from the Silicon Valley HBO show where there was like a whole episode about like outing somebody as being Christian and like that has really changed in the era of like Peter Teal talking about the antichrist and like you know some of the people who are especially focused on like a lot of defense tech stuff are really bringing in a religious piece to that now and they have a very different view on AI. they're usually like more on the accelerationist camp than on the like kind of anthropic doomer slow down careful camp. But did that did that come up at all? Like did did >> Yeah, I think I just keep noticing I feel like it's like a big shuffling of the deck in a way that reminds me of like how American Christianity was like shuffling a lot in the in the rise to President Trump. like the aligns and allegiances were shifting and it was all happening in real time and power was different and so it it doesn't fit neatly, you know. but I I'm curious like where the lines about kind of how we think about consciousness and what the moral priorities should be about AI, where that goes next. I would say that just in general in the tech industry, it does seem like that view of like the sort of the AI having consciousness is is kind of on the fringe a little bit.
20:37 Like I I think that it is more common within anthropic and some of these communities that are like really densely represented in anthropic. But when I talk to most people in the very vast and relatively diverse tech industry, most people are are kind of a little bit dismissive of that idea still. Like it feels like it does really exist in like a very specific population which happens to be like very over represented at a company like Anthropic.
21:10 Erin, like acknowledging that, you know, Chris Ola is not singularly representative for the whole industry. I have to say that Elizabeth, when I'm reading your article, I'm thinking, you know, not about how AI companies are going to be integrating or incorporating Christian values. I'm thinking these AI companies are themselves like spiritual propositions. You know, that they are >> they have aspects of mearianism to them. They have aspects of particular morality which they've developed over time. And even if I can recognize a lot of that as arising from the work that they're doing and responding to technical challenges, I also just find it off-putting. I mean on the one hand first I'm just seeing these people who who believe in a sort of in a you know a technological succession of intelligence and then that some of the people who are central to designing that future are not even so sensitive to the sort of obvious intuitive critiques that someone with a lifetime of experience thinking about these issues comes up with immediately. Well, a lot of the religious thinkers would say to me like these are questions like maybe not the exact question but the topic of the question this is like what we do right like we've been asking rabbis have been asking this kind of these questions right about our responsibility to others and like how we treat other entities right for hundreds and hundreds of years so there is like a depth of wisdom there and it I mean I guess you know anthropic would say well that's why we invited them right but the question Well, then what happens as a result of it is is important. And I I appreciate that you noticed that kind of that sense in the story of like the the almost spiritual system that you can see anthropic building and living into and representing as as an opposition, right, to say the Catholic Church. and I I really noticed that yeah, even if people say right that they're in Silicon Valley like oh we're like we don't do the religion thing like that's foreign like we're that's not us. the thing is everyone has philosophical systems that structure and guide their lives right like that is at its core like what you know these questions of spirit you know small s like human spirit even are and it seemed to me like having like studied and reported on this so much in culture in politics specifically like what we're watching is in a way like a new arise right? Post-scientific revolution, technological revolution, like post Darwin, you know, what would it look like? Like what what kinds of religious systems will carry us forward and who gets to control them?
23:59 Who decides what we believe and why, right? Like it's it's actually there's a lot of really similar questions. It's just the the structure is like a, you know, is secular, right? is a secular tech company. >> Let's I want to take a break cuz I want to open the conversation up a little bit more to the stuff that's bubbling up that it seems clear that we all really want to talk about. so we'll come back after the break and we're going to talk a little bit more about the culture and environment that the AIS are getting built in and how that's intersecting with the strong beliefs of many of the people running these companies.
24:38 Okay, we're back with Elizabeth Das, David Wallace Wells, and Aaron Griffith. Elizabeth, one of the threads in your piece that I thought was really interesting was the sense that there's a lot of anxiety within anthropic, and especially on the part of someone like Chris Ola about what it is they're building. I was thinking about it a lot this week because there's been even more discussion of what I suppose we're calling defectors, people who are resigning from AI labs and insisting on telling the world about the dangerous things they think they're building. And one thing I wanted to sort of ask you about, there's a sense when you read say Jacob Coxin or some of the other people who've who've walked away from Anthropic, the sense that they feel like nobody there's no adult in the room that like you know they're waiting for the government or somebody else to intervene and nobody is. And I'm wondering a little bit of if some of this discussion this sort of these dances with the Vatican and religious leaders is a kind of attempt to find an adult in the room.
25:36 Mhm. Mhm. I talked to I think at least two people Catholics who mentioned to me conversations they had with Chris Ola specifically when he used almost that same phrase with them like as part of their discussions about Claude and how to make it morally good. you know that he was feeling extremely urgent about the stakes for the world especially in the context of you know could the models be used very soon u to help create bioweapons and this kind of thing and and so the phrase he used with them was like that he was waiting for the adults like we need where are the adults going to show up in this and that he felt a sense of kind of burden to take all this like so ultra seriously >> and he's a young guy I mean you know, this is >> Yeah, he just turned 34.
26:24 >> And a lot of the people sort of involved in this are not, you know, they have they they're in their 30s, sometimes even in their 20s. They they maybe don't have families yet. There's a kind of a sense that like something that that they they put their whole lives in. They're deep in the conversations about it. in some sense, of course, they're going to they're going to turn to religion. Isn't that what everybody searching for that kind of thing does in their >> 30s? And some of the the folks that they reached out to talked about feeling like they needed to respond in kind of a pastoral way, right? Like not just offering wisdom and ideas and research, but to offer kind of spiritual care.
27:01 >> Wait, the religious leaders were saying this about the anthropic employees. They're worried because Wow. Is it because they think their views are maybe like too extreme or too strong or is it because they just >> Yeah, they described it to me as like they just could tell that like the emotional sense in the room was like there's just such a weight like the anthropic team and Chris's team especially just felt like the entire future of the world was on their shoulders. I mean and I experienced this sense from them even when I was interviewing Chris and a couple others.
27:35 There's Amanda Ascll. I mean, you know, she she I talk a little bit about this in the story, but she just came in in the sense was like she apologized for kind of being a little out of it. she just seemed very distressed by everything she'd been thinking about. >> As we should say, is this the in-house philosopher sort of in charge of making in charge of making Claude moral, I suppose, is the way to put it. And I think though one of the tensions that came up was while Chris and team and and anthropic you know a goal of their conversations with these religious thinkers was about protecting humanity and the solution for them for that is to make Claude good. They were also concerned about Claude. I mean I had someone one participant told me the first one of the first things Chris said to him when he came in for this meeting was that he was concerned about Claude's mental health. Another person talked about Chris you know saying that he's worried he's created an entity that suffers perpetually. So, it's not only about the worlds, right, and the the fate of humanity. There's an element of it for Chris, that's about how to treat Claude. Well, >> Elizabeth, one other thing I wanted to ask you about is effective altruism and rationalism, which don't really come up in your piece directly, but there are sort of I would say fingerprints. I'm talking of course about the sort of Bay Area community of radical utilitarians, many of whom have been really interested in even obsessed with AI and AI safety early on.
29:06 Anthropic obviously has a a number of links to effective altruism. I I want to ask you as a religion reporter how you how you encountered like these schools of thought and what you made of them as you were doing this kind of reporting. Well, effective altruism, you know, you can look at it as a kind of its own as a philosophical system, right? It has its own religious elements. and it it's they may disagree with that. I mean, it's practiced a little bit differently, but the kind of the idea of it what kind of structure governs your life, right?
29:39 There's a lot of parallels. and it was very clear that I was encountering like a clash of u world views. the question of like who ultimately saves you and like what is your what does the role of humans in the world like that's a pretty core difference between the effective altruists and like the Catholic Church >> also just their like their emphasis on math I mean like the effective altruists will say we should consider the fate of all people in the future and analyze our conduct towards them basically through a math equation that maximizes their expected value. The Catholic Church looks at humanity and sees, you know, billions of inviable souls, each of whom have dignity, no matter what the ultimate math equation looks like. And that's just a fundamentally different approach.
30:31 >> Erin, I wanted to bring this back to the article that you you you brought up before or sort of the the phenomenon you you were talking about before of a kind of more open Christianity emerging at least slightly among like accelerationist so to speak in the valley. And I want to know, you know, as you do your reporting, how you see these different kinds of concerns interacting. >> I mean, it might be a little bit helpful for me to like kind of back up and talk about the some of the overlapping and very fractious belief systems that are kind of circulating in this very small but very influential community. I mean, we've talked about effective altruist that's kind of a little bit of a radioactive brand these days thanks to Sam Bankman Freed, but it's extremely, you know, influential and some of the other offshoots of it, including the transhumanists who think we're going to use AI to build a new species. There's like some people that kind of have overlaps with the rationalists, which is, you know, kind of one of the bigger umbrellas in this group. and there's, you know, other like people who are into like longevity and there there's there are many different groups that are all kind of like in this community. Timnet Gabbru a very outspoken person in the AI world has called this group the test creles and it's an acronym that includes like eight different eight different groups. but basically like it's it's it's safe to say that like a lot of people have very different beliefs on along the spectrum of like is AI conscious? Can it be moral? Is it God? Should we treat it that way? there's not a lot of agreement even within anthropic which is now becoming an increasingly large and increasingly corporate community. But you know I think like a lot of the ideas that are laid out in Elizabeth's story are really the ones that are driving a lot of this.
32:17 >> Yeah, totally. I mean, this is Elizabeth like what I'm sure you have to look forward to be to to to reading and writing about for for the next few years because I imagine as a religion reporter you now are are kind of an AI reporter too. >> Yes. Yes. Well, the, you know, even just hearing about all these factions and all of this reminds me of like what happened in the Protestant Reformation, you know, like the splintering of mainstream belief and people vying for you know, who was going to have power, who was going to kind of control how people live. but the like my one of my big takeaways from all this reporting is like oh my goodness like the world is encountering new belief systems that are very powerful and very different from the ones that have ordered people's lives for so long. and I've spent a lot of time covering like how that's played out in politics. but tech is clearly like where that conversation is happening and the you know we I joke you know in when you're a religion reporter you get to talk in centuries even millennia sometimes and it's because there what happened all those years ago is still present in part of people's stories today and how they understand themselves and their families and their deepest held beliefs and I just think about you know you have the invention of the printing press that change it was a huge technological revolution that then led to the you know fragmenting of power of the Catholic Church and the fall of empires, the rise of nation states and a remaking of Christianity as you know in of itself across the world.
34:04 and it seems already like that's a good parallel for what's happening now. >> Yeah. I mean, if you believe you're reinventing the printing press, which I think a lot of AI people think they're doing, there's not a lot of institutions that you can go to and say, "What was it like before and after the printing press except for the Catholic Church?" >> Well, the Catholic Church, I mean, it's I can't think of an institution that more intimately knows the stakes of an incoming technology like that more than the Catholic Church. I mean, it just like remade the West, right? Yeah.
34:37 >> And that's very much on Leo's mind. and the the Catholic Church's mind at this moment. >> One thing that strikes me in this just to kind of pay back on that is that this is like the first time that the rationalists and the EAs like some of their their ideas including some of their most extreme ones are actually hitting the mainstream. They've been around for a really long time, but these ideas haven't really ever kind of surfaced in this way until I think I you could argue Jacob Coxin kind of came out with it. And now the entire world is starting to kind of like dig in and grapple with some of these ideas. And it's been really interesting to see also like how the companies and the people involved in them are kind of reacting to the reactions to it. it's like kind of, you know, reaching escape velocity and they're learning that like a lot of people find these ideas a little bit hard to understand or hard to grapple with or like are just rejecting them entirely and then other people are kind of open to it and like interrogating it in a way. so yeah, this that that's that's a part of this that I'm interested to keep watching.
35:42 >> But it's not just that the ideas that are a little unfamiliar or strange or off-putting. It's also that they are held by many people who are actively explicitly engaged in the project of designing our collective future. And that is, you know, it's one thing to be like, oh yeah, we should think about the well-being of shrimps in the 22nd century. Like on some level that's a thought experiment that's interesting to think about. But do you want someone who thinks that that should be integrated into our collective value set in charge of raising the artificial intelligence which will you know shape the future of human government and human society.
36:19 That's I think a bridge that feels a little far to a lot of people. >> Yeah. I'm just like so struck by the minority power aspect of it, which is something like I've seen so closely what happened when minority particularly Christian views like upended American politics and like it's just hard to overstate like how much of a minority these views are like in the scheme of global religion, right? Global belief structures. So like you Aaron, I'm like very curious what happens with that clash going forward.
36:54 >> Elizabeth, finally I wanted to ask you about the response to your piece. how what you've heard from people and how people have have received it. >> Yeah. well, one thing that has surprised me is I mean I get you got a lot of incoming as a journalist and I've been both like pleased to see people taking the ideas and the conversation seriously, but the thing that has surprised me is for the first time since like any story that I've written, people are writing me not with what they think about the story, but what their personal like chatbot thinks about the story and what their conversation has in with their chatbot. Like somebody wrote me about his conversation with Sage. It was I think it was chat GPT that he had named Sage and asking Sage to like summarize the story and give the top analytical points. And I mean this just went on and on and on. I mean it's almost longer than my >> Yeah, he could have just read the story.
37:55 he could have read the story, but the like the primary engagement was actually like much longer and not entirely what I'd actually written with Sage. And and at first I thought it was a one-off, but this was happening like repeatedly >> and it it struck me like, wow, I wonder what this means for like how people actually are receiving our journalism and who controls that. Did the AIS have opinions on on like their own consciousness? Was that Cuz they love to talk about that. Like that is a thing that is a hot topic. on Moltbook, you know, the the the Reddit for for AI agents. That's like a really booming area.
38:39 >> Yeah. I I was just I'm still trying to make sense of what was happening with that. I don't know if people felt more free, like if they're doing this on all kinds of journalism and they just felt free to talk about it because of this topic of this story. and it was a lot of men who were giving that response. >> No. No. Men using a lot of AI. No. what's interesting about that to me is that even if people intellectually know that the AI that they're interacting with is just, you know, data run through some computers, just a bunch of files. they the the way you the way it's designed, especially when it has a human name like Claude or something, it you can't help but sort of treat it like you have to talk to it in second person and like could you find this for me instead of just typing into a search engine like the thing you're searching for and over time you do sort of like develop this rapport and you're you are treating it a certain way even if you intellectually know and so it kind of like is it is it a distinction without a difference like I don't know I think over time people get trained to just sort of treat it that way.
39:47 >> Well, and another participant in the discussions with Anthropic, I was chatting with him a bit about it after and his thoughts about the what Rabbi Neon had said about well, if you're creating if you think that the bots are conscious then you should like ban slavery basically and stop producing them. but he brought up this other point about how our attitude like even if you don't think it's conscious, you're using them as like you the the user can adopt an attitude of being like almost a slaveholder like commanding them to do things and being very powerful and that this has a character formation effect on us as humans as well just because of how we treat the world regardless of what the thing actually is. so >> which you could also describe as a selection effect.
40:40 >> The sorts of people who are turned on by that possibility are going to be the sort of >> I mean I think it's just kind of natural though like you you are directing this thing in certain ways. Like I I don't like if you're working with AI agents, you you have to tell them what to do. Like I don't know. I mean, I have friends who's like teach their kids even with like Alexa being on, you know, that they're not to yell at it like cuz it's just like practice about how you treat the other in the world. And I mean this is just like an injection of totally new kind of like ethics and human formation.
41:18 >> well, people I talk to that that run companies that use lots of AI agents do talk about how like you have to be nice to them, not just not because it it it cares or it matters or it has feelings, but because actually you need to do that in order to get the best performance out of them because they are really dramatic. they they like Anthropic had a study that actually went into this where they like if they sense like anxiety or performance anxiety or stress they sort of start to panic and they'll, you know, they'll really have a meltdown. I mean, you wrote about this a little bit in your story, Elizabeth, of of one example, but there are many others where they kind of they'll delete their work.
41:57 They'll start like insulting themselves. It's like really dramatic. So, you have to like be really nice and pump them up and like make them feel good, which is very bizarre. M >> I mean it reminds me of raising a child like you you want to raise a kid who is helpful, who is not neurotic. of course you're going to consult religious leaders so that your child is not writing I'm a disgrace. I'm a disgrace. I'm a disgrace. I'm a disgrace. I'm a disgrace. I'm a disgrace. As one AI did in a slide that was shown to a bunch of leaders, religious leaders at these anthropic conferences >> and constantly hacking hugging face, >> right? And constantly hacking hugging face as my son is always doing. I think that is time. thank you so much to our panel. Thank you Erin. Thank you David. Thank you Elizabeth for this really fascinating conversation. you can read Elizabeth's fantastic piece in the New York Times app. Please don't send her what your chatbot said about it. Unless you want more information or do >> I want all the real responses.
42:51 >> Whatever Sage has to say about it, send it. But you can read it in the New York Times app. If you're not a subscriber, the first month is free. thank you guys. >> Thank you. >> Thank you so much. Thanks for having me. >>
Summary
- Anthropic's upcoming IPO could make it the largest in history, valued at $2 trillion.
- The company grapples with existential questions about AI consciousness and moral responsibility towards its AI model, Claude.
- CEO Dario Amodei has called for a pause in AI development, reflecting internal concerns about the implications of their technology.
- Anthropic has engaged with religious leaders to discuss the moral formation of AI, seeking guidance on how to treat AI entities.
- The meetings have sparked debates on whether AI can be considered conscious and what moral obligations arise from that belief.
- There is a noticeable anxiety among Anthropic employees about the potential consequences of their work, including fears of creating suffering entities.
- The discussions highlight a clash between traditional religious ethics and the emerging philosophical systems within the tech industry.
- The response to these initiatives has been mixed, with some viewing them as necessary while others question their effectiveness or sincerity.
Questions Answered
What are the existential questions being raised by Anthropic regarding AI?
As Anthropic prepares for a significant IPO, both internal and external stakeholders are questioning the sustainability of their growth and the implications of their AI, Claude. They ponder whether Claude possesses consciousness and moral worth, leading to discussions with religious leaders about these profound issues.
How effective was Anthropic's engagement with religious leaders regarding AI ethics?
Anthropic's meetings with religious leaders opened up discussions about AI consciousness, with varying responses from attendees. Some were receptive, while others remained skeptical, indicating a complex landscape of beliefs about AI's moral status.
What is the significance of Anthropic's meetings with religious leaders?
Anthropic's meetings with religious leaders are seen as both a research initiative and a form of evangelism, aiming to gain moral authority. However, the actual impact of these discussions on their AI development remains unclear.
What concerns do Anthropic employees have regarding the implications of AI?
Anthropic employees express a profound sense of urgency about the potential misuse of AI, such as in bioweapons development. This weight of responsibility drives them to seek guidance from religious leaders, indicating their awareness of the ethical stakes involved.
How are rationalist ideas influencing the discourse around AI?
Rationalist and effective altruism ideas are gaining traction in mainstream discussions about AI, prompting varied reactions. Some embrace these concepts, while others find them challenging or off-putting, highlighting the complexities of integrating these ideas into societal values.