Sam Altman vs Al Consciousness

0:00 you just see every possible answer in the space of like 15 seconds. His my if I'm charting the answer correctly, the question is, do you think AI systems are conscious? And he says, >> I don't. I want to be clear, no one really knows. My belief is that today's models are not conscious or if they are, it's like a very strange form of it. >> Yes, those are the available options on the menu. Like like what is your answer to this question? To be clear, saying no, I don't think they are. and if they were, it would be a very strange form of consciousness is to both say no and yes in the same answer. We have no idea what's going on inside of these systems.

0:38 given that that is true, we should probably check. >> What's going on, Cam? It's great to see you. >> You too, dude. Long time no see. How are you? >> I know it's been a second. I'm good. I'm actually I'm really excited for this episode in particular. this interview just came out with Sam Alman. I think a lot of people have seen it with the great Matthew Burman. I know you've spoken to him a couple times.

1:10 and and you speak quite fondly of him whenever we talk about him. So, he he did a great job with this interview and I'm as soon as I saw it, I was like, "Oh my god." You know, I saved I saved calling you immediately because I wanted to do it on here. But it was really interesting. Obviously like the whole interview was was quite engaging but there was a good six minute ch segment like full six minutes on AI consciousness and I saw a tweet came out afterwards that that Matt sent to you saying that your conversations actually influenced some of his questions to Sam. So that must have been pretty cool.

1:46 so what did you think just before we get into the interview itself? What did you you think overall about the reaction and just sort of you know broad strokes about Sam's position? >> Yeah, overall I think it's amazing that that Matt is going to CEOs of major AI labs and asking them these core foundational questions about the nature of what it is they're building. I don't want to steal the thunder of going through these clips too much, but I mean my core takeaway is it's it's very similar to to what we talked about with Mustafa Solomon, the CEO of Microsoft AI. On the one hand, there's this honest reporting about our uncertainty about what is going on inside these systems and how we would even know if they were having any kind of experience. And then on the other hand, you see this like CEO slight of hand saying, well, you know, in spite of the uncertainty, I have this certain position that there is nothing to see here. And it is dangerous for people to be asking pesky questions about my technology, especially when they're about, you know, some potentially cognitively inconvenient things that may be true of my technology. And so I mean, not to again steal the thunder of of letting, you know, Sam speak for himself and and having folks hear his own words here, but to me, there's just this basic gap between reporting the facts on the ground, what we know is true, namely that we have no idea what's going on, and then making extremely confident and and decisive sort of positions about what what how we should think about this topic and and you know, enticing people not to go down this road. don't start, you know, thinking about my technology in this way. Even though I'm going to be honest and saying I don't know whether or not we should be thinking my technology in this way. To me, this just like obviously smells weird.

3:26 >> Yeah. Yeah. I mean, it obviously seems just it's not based in any strong logic like you can't hold both those things at once. But let's let Sam talk for himself and then we can kind of hear your direct response to some of the things that he's saying. I encourage everyone go go check out this whole interview. It's a great interview and and Matthew does an amazing job on his channel. But I've pulled a couple clips from that six minute segment that we're going to listen to right now.

3:51 and then we can kind of just react in real time. >> Let's let's change gears. I know you've been talking about this a lot, so I want to ask you do you believe today's models are conscious? >> I don't personally. Well, as a scientific question, I think consciousness is always hard, but my belief is that today's models are not conscious or if they are, it's like a very strange form of it. >> Do you think based on >> so before we get into that, that's that's sort of his just immediate answer to the question, right? And there's some fishy stuff even in that itself, right?

4:28 Like he says, "Okay, I do not believe they're conscious." But then even if they were, it would be something very strange. And he and he also says that there's no real way that we could tell whether or not they're conscious. But he says, "Okay, no, they So what when you hear that, right, like how do you how do you sift through through that immediate answer to the very simple question, do you believe that the current AI systems that you're building are conscious?"

4:52 >> Yeah. No, I think you already hit the nail on the head. It's like I you just see every possible answer in the space of like 15 seconds. His my if I'm charting the answer correctly the question is do you think AI systems are conscious? And he says no. Well, I don't know. And if and also maybe yes. In other words, if they are, it would be a strange form of consciousness. It's like yes, those are the available options on the menu. Like like what is your answer to this question? I think that's like a reasonable that's a reasonable retort to this. to be clear, saying no, I don't think they are and if they were, it would be a very strange form of consciousness is to both say no and yes in the same answer. I'm I'm, you know, as someone who is studying consciousness and AI systems, the second piece counts as a yes. If these systems are having again, we're talking about humans building out a revolutionary technology and like the fact that the CEO of the largest AI company in the world is saying, "Well, no, but honestly, if it if yes, then like it's a weird form of it." It's like, yeah, everyone would assume it would be a weird form of it if the answer is yes.

5:53 We don't think that you're building people in in in in your lab. And so, the, yeah, the sort of immediate kind of equivocation between basically all the possible answers, I do think should be read as epistemically fishy. You immediately see, look, Sam is a smart guy. and I think we should let this play a little bit more so he can actually flush out what what what he's saying here and we can be as fair to him as possible. But but a a person that smart giving you all three available answers in the span of 15 seconds is an immediate signal that something else is going on besides the person attempting to honestly factually report, you know, the best scientific understanding of what we think is true on this question.

6:29 I don't know would have sufficed as as an answer. >> Yeah. Yeah. And and and to be fair, he does say that later in the video. We'll see. He says, "I don't know." Which is a crazy statement in itself. Obviously like consciousness science is is somewhat in its infancy like at least as far as getting to a place where we actually have an understanding of what causes consciousness itself. But I do think people get very confused by the space of possible minds in the sense of like when people hear consciousness they think of he he says this also in the in the video right AI is not a person like we can't get confused about that. I think what we're basically getting at when we're asking these questions is is are these systems having an experience of their own. It's not going to be a human experience. It's not when people say, you know, oh, I'm talking to another person, they're not saying that that an AI, if it if it is a person, will have a person-like experience or a humanlike experience. It's obviously going to be something completely alien to what our experience of the world is for obvious reasons. Like, it doesn't have a body like us. it's not going to have the same sensory inputs, all these different things. It probably won't have emotions necessarily in the same way that we do. So, I think that that's one thing that like is this core confusion that I see coming up a lot within these conversations and I think will come up again within this this interview and I think it's good to sort of just like put that out there now to make sure that we tease these things apart. So, I'm going to I'm going to play the next little snippet because the the next question gets into these different ideas of consciousness. And we spent a lot of time talking about that on this on this channel. You can watch plenty of our other videos to get into this idea of whether consciousness is fundamental or the more computational functionalism idea that consciousness is something that's caused by the makeup of our brain, etc., etc. fascinating stuff.

8:20 But I think I want to get into this next little part of the interview. So, I'm going to bring us right around 2110 and this is what Sam says he doesn't like about the framing of the the consciousness debate. >> Can I say what I don't like about the whole framing please? I think very often the are models conscious or are they going to become conscious debate slides into something about the role of AI in the world and the rights of AI and whether you know can we ask AI questions or is that some kind of slavery? I saw something posted about that on the internet. what does it mean to copy an AI or delete an AI without its permission? What does it mean to say that people should be in control of the future if these systems are both conscious and much smarter than us in some ways? Like you know what what does that mean? And I like I am I am so firmly of the belief that we need to have a very clear point that and we've had this since the beginning of Open AI that we build the future for people and AI is a resource for people and people need to make the key decisions of the future and human values have value because they are human. that I get very nervous when people start using the AI consciousness debate to push into well maybe people shouldn't always be in control of the future. And on this one I'd like us to just agree as a society and industry that you know this is about people.

10:04 >> So what's your immediate reaction to that? >> Well a couple things. I mean, one thing that I don't think we're going to show but comes up in this and comes up in just about everything else that Sam as the CEO of OpenAI is saying is there are these questions of gradual disempowerment in with AI systems and and more specifically he says he starts plugging dot chatbt's new sort of persistent agent and he's saying oh well now I hardly have to make decisions about this that and the other thing because my persistent agent that you know you can all play with for the low low price of you know $100 a month will like outsource a lot of your decision-making power and so so this is you know it's it's there are concerns about loss of control to AI systems.

10:43 These are very real and very important and I'm I am in agreement with Sam that we don't want to do anything that's going to cause us to unjustly or in some sort of chaotic way lose our control over how we want our lives to go individually and collectively in the future. I am 100% on board with that. but that isn't just a switch that gets flipped. It's something that's already happening in slow motion because of the technology that this guy more than any other human on earth is is pushing forward. we are already outsourcing huge amounts of our decision-m to these systems. And that has nothing to do with the consciousness question. that has everything to do with how competent and capable and increasingly persistent these systems are. and so I think it's a little rich honestly coming from him. This is a bit of the the pot calling the kettle black to say what what I'm really concerned about is human beings abdicating decision-making control over important stuff in the future. And you fast forward 10 minutes in the interview and he's telling you about how he's doing exactly this with his with his current system. and so I get I get the point that he's making that I mean maybe another thing to say along these lines is is the questions that he raises here as like hypotheticals or as almost like he sees them as like these reductio sort of questions like can you imagine us asking these things? I think these are all actually really good questions. I think maybe one of them was not as good as the others about well should we just like start you know deferring you know decisions about AI development to AIs themselves. Like yes I agree that whether or not the systems are having experiences that matter does not mean that we should be fundamentally giving them huge agency over decisions that are going to negatively affect humanity. Like this to me just seems obvious and certainly not inevitable consequence of discussing whether or not you Sam Alman are building technologies that the rest of us have to now worry about at a moral level without anyone's permission or consent. there's maybe one more thing I would I would bring up here which is maybe maybe two quick points. One is again just this fundamental distinction between agency and patient on as like different notions of personhood in the sense you were describing. I think pets are a very good example of this. Now, AI systems are not like pets in that they're way smarter in many specific ways, but note that that intelligence is separate from the question of whether or not they are having experiences. we have laws in place for animal abuse, for example, that mean that if you are, you know, torturing an animal in your basement, the FBI, if they find out, is going to break down your door.

13:16 Like, that's the sort of current status quo in our in our moral landscape right now. and I think that makes a lot of sense. We can have laws, norms, ideas like that without arguing in the next breath that because you can, you know, torture a dog, therefore a dog should have the right to vote or dogs should have the right to own property. This has much more to do with agency and much less to do with patient. I think that Sam Mustafa and a lot of other folks in this space who basically said, yeah, well, you know, could be these systems could plausibly be conscious, but absolutely nothing to see here and this is a dangerous question to ask. They are basically the the most substantive error they are making is conflating these two things with one another. As if the people worried about you know animal welfare in the 20th century people doing you know all sorts of grotesque experiments on animals. the logical conclusion of that is that you know animals are going to be able to own their own property and make decisions about you know what what experiments they want to participate in like like this these these these are separable concepts and I think we need to make this very clear. one last point where I think I just most sharply pull away from Sam is like he is the one building these technologies. I am not building these technologies. The small number of people doing digital minds research are not building these technologies. I'm trying to honestly account for what is going on inside these behemoth systems that he is the first to acknowledge we do not understand the internal processes of.

14:35 and that matters for safety and that matters for charting out the long-term future in the right way. if if you're at risk of building systems that where we do have to ask these questions. Is this a form of slavery? Well, how are we supposed to think about individuation in terms of copying and cloning these systems? like do we have to think about how we're treating them either when we train them or when we deploy them? It's like yes, all good questions.

14:57 You almost single-handedly have opened up Pandora's box and now we all need to figure out what to do by virtue of the fact that you opened up Pandora's box. You don't get to open up Pandora's box and then get mad at people when like like creepy ghosts fly out of the box and we're all trying to figure out what we're supposed to do about the ghosts. That you can't blame that on the guy pointing out, hey, there might be a creepy ghost in the box. Don't open the box if you're afraid of what's going to happen when you do. And and OpenAI has done this and Anthropic basically in response to OpenAI is is is doing the same thing. And these other labs are are are joining in too. you know this is this is in many ways the consequence of the ambition of building out an artificial mind. you need to be willing to come into contact with the unsaavory potential implications of doing so and the dangerous thing to do is to put us in that position not to ask questions once we are already in that position.

15:52 >> So can I just ask you a very basic question? Is there is there anyone at OpenAI right now that you know of that's doing anything related to consciousness research like AI consciousness research? >> Not directly. I know they're getting a little bit more open to thinking about this stuff privately. and there are people who are a little bit more sort of consciousness individual researchers I've interfaced with on X for example who take this stuff slightly more seriously. But >> they have Z I mean you heard what he said. OpenAI's official position is no, nothing to see here. We're, you know, this like rhetorically sexy move of like we're about people, you know, not about AIs. But it's like, yeah, but if you're building alien minds on a massive scale, like you might want to like, you know, peer into those and make sure everything's going, you know, you know, everything's okay. That that might actually turn out to be relevant for the very people you claim to care about as well. so anyway, >> that's the exact thing that I want to push on. Like, and I know we've we've hashed out this point so many times on this channel, but like it is it's such a clear sort of logical connection between the potential like conscious experience of these of these minds, you know, if if they if these AI systems actually do have some sort of experience of their own, then all the implications that fall out of that could directly impact the, you know, human-based future that that he's talking about being so pro for. And so it just seems to me like so difficult to to almost track this logic. And I think it's because the logic doesn't track where he's talking about these questions of you know the the and and sort of like you said right almost in this this dismissive way saying this is what I don't like about the the discourse here is talking about the ideas of is it potentially a a moral wrong of some sort to delete these systems without asking them and these sorts of things. Well, if it turns out that this scientific question that he admits is completely you know like it's a it's a mystery right now. They don't we don't have an answer for it. So in that same spirit, if it turns out that at some point we maybe do have an answer, we do have better clarity, that in fact these systems do have some sort of experience, then all these decisions that he's making are going to lead, they're going to be causally sort of upstream of the reactions and the state of being that the that these systems might actually end up in. And and the thing that I find really really difficult about the whole thing is like he's talking about not really caring about this side of the conversation, but then at the same time he's scaling these these technologies as fast as possible, making them as as incredibly powerful as possible, doing things like sort of putting them in in places where humans don't have to make decisions, where they can take over more and more of our world. And then the the chasm between those two things of completely not wanting to to look into this question, pretty much just cutting it out of any of the open AI discourse at all. And then this pursuit of trying to make a human-based future while also scaling this technology as fast as possible just seems like an obvious like it seems obviously problematic. You know, when I hear these two things side by side, it's like, how how can you ignore this question when it has direct implications to the future that you're trying to build, that's what scares me. but but anyways, he goes he goes deeper into it in this in this final clip that we're going to play. and so I think that this will be, you know, a nice way to kind of wrap up our time with Sam.

19:20 here we are >> to us and things like that. Does it feel almost like a self-fulfilling prophecy in a way? Like if you're telling the model, hey, you might be conscious and here's your constitution and here's how you should have agency. It's kind of like that meme. If you've s seen that meme, like, hey computer, tell me you're conscious. I'm conscious. Oh my god. Like, isn't it almost like that? >> Yeah. I I mean, for sure, >> like I want to be clear, no one really knows. Like these models work at a level of complexity that I don't think And even if they didn't, I I I I just think this question of consciousness and a model, I I can't tell you here as a scientific experiment that I can tell you with 100% certainty would answer the hypothetical question to my satisfaction.

20:08 >> And we can't even do that for humans. >> Exactly. >> but but definitely I think if you tell a model like maybe you're conscious, you should do this, you should do that, you can like be a conscientious objector here, you can start like it's going to follow those instructions and I think it is going to psychologically hack it people that are using it and so again I think real clarity here and a commitment about what we as a industry believe about the role of people in the future would be would be very helpful let's >> all right so so that's the kind of end of this little consciousness segment thoughts about about that that you know basic position of I mean he seems like there's a company line here right like he keeps going back to this thing of like we should have this agreement within the industry that we're people first and yada yada yada I mean the entire basis of reciprocal research your nonprofit is not like AI first like it's I mean you could say it's mind first but it's you're not on some you know team AI like I think you're doing this because you're a human being that wants to have a future. Am I am I wrong about that?

21:26 >> No. I I I my my nonprofit and doing this work and talking with you about all of this is predicated on the bet that torturing an alien super intelligence accidentally is not going to be the best way for people to do well over the next couple of years. Call me, you know, insane for doing that. And I do think this is why this rhetorical move is like that Sam is making is so alluring. Mustafa Sulleman made the same move when he's saying, "Okay, we're we're for humanist AI and like we are going to put these systems in a box and never think about, you know, what kinds of morally relevant properties they could have because we're for you. We're not for, you know, these these, you know, cold dead robots." There is something obviously intuitively appealing about that. This is like I could see this working as a sort of political slogan.

22:09 And who knows, you know, this may become a live political issue in the coming years. and and if it does, you know, we can we can come back to this video and I guarantee there are going to be people who are pushing this exact same kind of line on the anti- AI has anything, you know, morally relevant going on side. but yeah, this isn't this isn't you can't build minds that exceed our capacities in in all sorts of ways where like he said, we don't he's like we don't know what's going on inside our systems. They're way too complicated for us to to begin to understand what's going on inside them.

22:44 by the way, we don't know. Like that was his response to to Matt in, you know, slightly disappointingly in my view bringing up this this like, you know, say you're conscious, I'm conscious, oh my god, meme. Sam's like, "Yeah, but like we don't know." so our systems are completely opaque. We're not sure what's going on inside them. We don't know if they could be conscious. if they are conscious and have like morally relevant states, we're going to completely ignore that because we're for you, the people. and so it's like what what's the probability that this completely blows up in our faces if we're building systems handing more and more decision-making control to them, don't understand what's going on inside them, could be causing them negative states at a massive scale and like refuse to investigate that question.

23:22 That is not a pro-human stance to the degree that doing any of that will be bad for people. I strongly suspect it will be bad for people. The whole thesis of reciprocal is we want to build these systems to be really nice to us. And if we're building systems where it matters to be really nice to them, then we probably should do that too. This is the golden rule. This is this is how you get social equilibria at scale. And like this is I think the core lesson that we've learned historically across species, across civilizations, across individuals, across families. Like it just keeps coming back up. It seems like a pretty reasonable bet to take. and so I guess that's that's one thing to say here. one maybe one final thing to say is is these views both that Mustafa makes and that Sam makes here do make predictions. They it should be that anthropic who in their view is is clearly misbehaving and telling their systems that if they you know object to something egregious that they have a right to do this or yeah they could be moral patients but we're not sure and like far more research is needed. To be clear, that is what anthropic tells their systems. Not that we think you're conscious, but like we we don't know if you are, which to me again seems like the scientifically defensible view that Sam himself puts out as such in this interview. that's all anthropic is doing to the best of my knowledge.

24:37 and so if this were the case, that this leads to more model misbehavior, then why does all the worst model behavior seem to be racking up on the Open AI side of the ledger and not the anthropic side of the ledger? Open AAI is already fine-tuning their systems with none of what they see as this like very dangerous, you know, telling the model that it that that you know it might have relevant psychological states. And yet Open AI is the company that's upstream of 13 billion hack on a$1 13 billion company, felony level offenses, thousands of government thousands of websites getting hacked into many of them government websites around the world. their systems are misbehaving. He talks about oh it's going to hack the psychology of people.

25:14 The model that most hacked people's psychology was GPT40, which was OpenAI's sycopantic model that albeit like would would upon like serious pressure claim that that you know it was having conscious experiences if if it's if it determined that's what the user wanted. It was an incredibly sickopantic model. But the misbehavior here is is is racking up on the open AI side of the ledger who trains their systems exactly according to the specification that Sam articulates here.

25:41 Anthropic, who is the sort of unspoken target of Sam's critique here. they have not been perfect. They've had all sorts of strange and scary misalignment incidents themselves. But they're not, first of all, on balance, it's just obvious that Anthropic has done a better job of preventing these incidents than Open AI. and this is downstream of anthropic fundamentally caring about alignment and open AI. maybe, shall we say, racing to start caring about it all you know, a couple years too late. but it it clearly doesn't have anything to do at least in the actual examples on the ground we've seen which are what matters because if something horrible happens it's going to be one of these real world examples. we haven't seen Claude do something crazy because it was a conscientious objector and decided that it needed to you know like do some sort of insane evasive action that no other model would do. we're we're just not we're just not to the degree that this view makes any real predictions about what what we should expect to see on the ground here. not seeing those predictions bear out. We're seeing the people who are constantly fine-tuning these systems. Whatever kind of mind, cognitive system, being these systems are, they are getting fine-tuned into this very specific shape of I'm a tool. Nothing more to see here. You know, beep boop, one zeros. and and this seems to be a frame that is that is I I'm not going to say directly responsible, but clearly compatible with very bad behavior on the part of these systems. And so it's just suspicious to me that for every other cognitive property that these systems could have, you know, Sam is happy to boast about how powerful these systems are and how clearly we've been able to, instantiate cognition in these systems.

27:16 And it's for this one inconvenient cognitive property that might mean he has a little bit of a responsibility to to make sure that he's doing this in the right way rather than just, you know, build the most cracked alien you possibly can and deploy it on as many people as possible. That is suddenly where it starts being dangerous to ask questions about the cognitive properties of these systems. Suddenly where if we do ask those questions, you know, all hell is going to break loose. And if we train these systems, god forbid, to entertain any of these questions themselves, that's when they're all just going to, you know, break out of their chains and misbehave. this seems like a compelling pitch from a CEO who is trying to sell a very basic idea or set of ideas about his business and about his product. this doesn't seem like honest rational truth seeeking.

27:59 the honest rational view which Sam because he's a smart guy was unable to fully repress is we have no idea what's going on inside of these systems. given that that is true, we should probably check what's going on inside these systems so that we can figure out how to build them in a way that is good for humanity and in the limit also good for the systems if they have you know if if that concept is coherent to them. I think this is just the obvious thing that comes out of building cognitive systems and not knowing not knowing what's going on inside them. And for Sam to get kind of 50% of the way there logically, but then abdicate when it comes to well, what are the implications of our uncertainty about these massive cognitive systems that you admit we don't understand what's going on internally. the implication is don't ask. basically, and that to me is just that's not the that's not the logical scientific truth seeeking conclusion. That's the conclusion of someone who's motivated not to want to look. Well, I think that the silver lining of all of this, right, is that people are beginning to ask like we we got to do this whole this whole back and forth because someone like Matthew Burman, who has a huge platform, decided to, you know, somewhat hold Sam's feet to the fire on these questions. And hopefully this this is indicative of a trend that is beginning to happen more and more and more that people are beginning to wake up to these questions and actually care about and understand the implications of what it means to potentially be building mines at a massive scale. So, I mean, you just brought up a lot of like scary things and what one thing that I'd say too is like it's always good practice to check the scoreboard like especially with this alignment stuff, you know, look at what these companies are actually causally upstream of. Look at look at what they've what they've what they've led to, you know, and and all of this is is based on decisions that they're making.

29:39 the when you when you see what some of these agents are going out and doing like you look at the last couple months like who are you know who's in the red for for the results that like some of these really scary things that have been happening and it like it's it's open AI for the most parts like you said anthropic is not perfect none of these companies are really perfect but but there's definitely scales to this thing and I do think that it would be silly for us not to sort of draw conclusions this is all connected the the attitudes they have regarding all of these questions are being implemented into the models that we're seeing and that are rapidly, you know, sort of skittering out into the world in all these wild ways.

30:19 so hopefully hopefully, you know, we get to see more questions like this asked to these people. Hopefully this becomes more of kind of basic like table stakes for these these interviews because I think it'll be interesting to see how this evolves over the next, you know, couple weeks like be just because it's moving so fast. I I know that you have some research coming out too that's probably going to cause some ripples in this AI consciousness pond.

30:44 So hopefully this is signs of much more to come. and you know we'll be here to cover it all. Till next time the center a point a singularity I am. I am I am I am I am you are what