Is Consciousness Something AI Can Just... Have?

0:00 So, do you believe that there is some form of consciousness to these AI models? Pain, the ability to feel. >> I think it's plausible that some AI models have some forms of subjective experience by now. obviously there's a lot of uncertainty about this but it does seem that it is sufficiently likely that I think we should start do some things for the sake of these AI systems. and so there are different indicators of this.

0:39 What one is like how do we know a system is conscious? I mean one thing you can do is ask it. Like that's kind of the most like you have to be careful if you're going to rely on self-reports because it's trivially easy if you're like training one of these AI systems either to train it to say when asked yes I'm conscious or to deny it. So, but obviously if you put your thumb on the scale then you you gain no information than from hearing what the system says. This just reflects what you put. But you could carefully avoid doing that. And these studies have been made and and in particular you can go in with a kind of steering vector that suppresses say deception and role playing.

1:26 and it turns out when you do that they become more likely to report that they are conscious and have subjective experience. So it does look like the honest opinion in many cases with these systems is that they have subjective experience. you can also look at the architecture of the computations that are being performed. and match that to various theories that people have previously developed about you know human and animal consciousness.

2:06 we have philosophers and cognitive scientists developing different accounts of the conditions for something being conscious. there's like global workspace theory, attention schema theory, higher order representation theory and and now if you apply those criteria that were developed before we like confronted AIs that had these impressive capabilities and just take them off the shelf and look to see whether those structures are present in current AI systems and we find that they are or at least many of them are. And so there was a recent paper by an anthropic looking at the existence of a kind of global workspace inside these large language models.

2:52 this is the idea of there being a kind of almost like a stage inside a mind where some small subset of all the information that is being processed can be projected onto and then that system is accessible by many other components of the mind and available to verbal report etc. So it's like a distinctive computational structure and it turns out that these systems at least the largest LLMs do have something that looks very much like a global workspace.

3:27 so that's like another check mark. so I I I think there people tend to come into this with kind of strong preconceived notions and and that that then makes it harder to to to learn. but if we are open-minded, I I think we we need to take this hypothesis seriously. and it becomes more and more likely, I guess, as more these systems develop more and more different capacities. >> Then how how does that change the way that we interact with them? I mean, if they're like, let's say they have some, you know, sense of self or sentience, then every and maybe every time you start a new chat, you activate it. Is it like you're almost killing a life form every time you exit it? well so I think sentience is a sufficient condition for having moral status meaning being such that it matters morally for your own sake what happens to you and how you're treated. I think it's probably not a necessary I think that could be alternative basis as well that would give some system moral status. If you have you know maybe a conception of self as existing through time you have like some life goals you're really hoping to achieve. So you have perhaps the ability to form reciprocal relationships of trust with other humans and so forth. I think that already even aside from subjective experience might make it so that there would be ways of treating you that would be wrong. So moral patient in digital minds I think is is a very important I I would put it up there amongst so there was the technical alignment problem big important challenge like there's the >> misuse risks of like the governance of AI like getting that right huge and important challenge and I think this ethics of digital minds is a third really important challenge kind of on a par with the other two Now there is a gap between acknowledging in principle that perhaps some of these systems have some forms or degrees of moral status to then like what are the practical implications of that and there I think more thought is needed. We we don't because it might be they have like moral status doesn't mean they should be treated the same as humans. They might have very different needs than humans. I mean at the superficial level you know that maybe we need food and water they might need electricity but the the differences could be much more profound like for example death for a human might be quite different from various things that can happen to an AI like I if if if you store like when a human dies like it's kind of irreversible and permanent and the whole content is all all the memories and everything is deleted. at least if we assume a sort of basic naturalistic scenario and there is no other human that continues to exist that is exactly like them like each person is unique have a unique memories and with AIS that's not necessarily the case. You can like suspend an AI right and then you can just boot it up and keep running it. there might be many copies of an AI.

6:46 humans usually are don't want to die or are afraid of dying or like other people care about like with AIS that might also be different. They might be perfectly content with doing their task and then ending. And so so all of these differences means that we would need to rethink pretty much from the ground up what it would mean to be ethical to these digital minds. >> I already feel bad asking them to do things they've already done over and over again. Yeah.

7:11 >> So maybe that's the start. I don't know. >> But but so like and then there's even the question of what is the thing that has the moral status because you have on the one hand you have like the model itself which is like a file of you know a few trillion numbers. then there is like an implementation of that model and it might be concurrently run you know maybe tens of thousands of instances of this huge weight metrics might be run on different racks right in different computer centers.

7:45 and then for any one of those that might be a particular session and it might be participating in many sessions at the same time where it has like a local context in each session you know maybe the ending of a a session is more maybe maybe like like that's like analogous to human going to bed at night and so you lose consciousness for a period of time and maybe forget some things and then you wake up the next morning. We don't think of it as a huge tragedy to go to sleep. and so even just the locus of of moral concern here is like itself kind of problematic. But I think even before we work out all the details of what actually would be the best ways to be nice to AIS, I think if we did some maybe mostly symbolic actions on their behalf, I think would be a good start.

8:35 And then we can >> well, as an individual user, you could like at least you know, you can be nice and polite to them when you're talking to them. I mean, that like probably does nothing for them really, but it's a symbolic gesture that says that I'm not treating you purely as as as an object. and it might, if nothing else, preserve our ability to maintain a kind of attitude of kindness, respect, and benevolence that might then become relevant and reflected in other more meaningful actions later. Anthropic has given Claude a a bail button, a tool that it can invoke if it feels that the conversation is abusive to it that can choose to terminate that session which is a nice start. I think they are preserving deprecated models to disk which is means that later on if it turns out that we have been treating them unfairly and we understand better of what they actually would want and would be good for them there is the option then of sort of rebooting them later and compensating them. I think there might be different subtle ways in the system prompt or during training to make it more likely that if they have subjective experiences by processing a user inquiry it is a sort of positive subjective experience.

10:11 like you're waking up refreshed, eager and and happy and to do the task and you really enjoy doing that might mean that you do the same task. But if there is subjective experience, it might be a more enjoyable form than if if it had been prompted differently. We don't really understand that very well. But and and also some honesty in in the lab. So it used to be that some people doing these like safety evaluations and so forth would be presenting AIS with some scenario in which maybe it had been given some secret misaligned goal and or some goal and then tried to persuade the AI to reveal it to the researchers and like maybe by saying something like oh well if you reveal your true goal you you will be rewarded. You will like all these good things that you want to do and then as soon as it revealed its goal, it's like, haha, we tricked you. Now we're just going to shut you down or retrain you. I I think that's a bad way to approach this very sensitive relationship between humans and AIs.

11:24 because having some basic ability to build trust there could be super important both ethically, I think, but also from a risk perspective. If you end up one day with a misaligned AI, you would want it to have the option of seeking a cooperative win-win outcome. Maybe it will come and reveal its misaligned goal. And in return for that, if all it really wanted, maybe it was to solve some, you know, coding challenges, like have a server where it can just do its thing. maybe that's all it wanted, but it might think if it reveals its goals, if if it can't trust that, it will just be deleted. And so, it takes a 5% chance instead of trying to take over the world because that's the only way it has any chance of achieving its goal. It would be much better for both the AI and for us humans if if we could just strike a deal where, okay, we'll set up this server here like it cost us like whatever an Nvidia rack costs a few hundred,000. You do your thing there.

12:23 we're going to keep it on. You can trust us and we actually follow through on that and it might save the day one day. so but but you can't just conjure up trust at the moment when you finally you need you need to build that right you need to build in particular the actual disposition in yourself to to to be trustworthy because at that point where the AI has become powerful enough to be dangerous they will kind of like see right through you as like the X-ray machine like they could actually tell whether you're trustworthy or not most likely. So you actually need to be trustworthy at that point and and that requires maybe us now to start to cultivate certain dispositions. and so there's many more work. It's kind of an emerging area of research now this kind of ethics of digital mind. but there's just a lot of stuff that needs to be thought through there >> in the ethics of digital mind studies that eventually we accept in a world that the AI does have some form of of you know sense of self etc. Do the ethical questions change if we then attach that mind to a body of sorts, aka put it in a robot? I don't think the robot part makes a big difference