Transcript
0:05 so today we have the pleasure of hearing from justice mariano florentino whaler tino is a professor in the law school at stanford he's also a justice on the california supreme court he was also an official in the clinton and obama administration which is incredibly cool tino did his undergrad at harvard he did his law school at yale he also has a phd in political science at stanford and he has done a lot of work around cyber law
0:30 and actually like ai and legislation as well as other work around international affairs and public health he also teaches a class on regulating ai which is a very cool class so if you're interested in these areas and if you're interested in these topics i absolutely recommend taking that class especially after taking 221 i think that would be a really good class to take and then we've been basically interacting with tino through the ai safety center and the human center the
0:55 ai institute institute over the past couple of years we have a project together right now on adaptive agents so it's really great to work with tino it's really great to hear from him and and for me he's like on the list of the top five people i would want to have a conversation with this includes roboticists this is a very small list so it's really great today to hear from tino and and i'm excited to hear your
1:17 talking you know so welcome thank you very much professor sally dorsa and thank you percy and peng and woody it's really an honor to be here and to share some time with you i have to tell you that that last comment you made dorsa is a lot of pressure i don't want to let the class down and get demoted and not be on your top five list it's also you know been really great to get to know you and i learned so much
1:42 from all of our interactions i appreciate that you've come to speak at my class so it's only fair and it's really an honor to be here i want to take like 35 to 40 minutes which i know in the era of zoom is a long time so i'm going to hope that those of you who have been good enough to tune in i know these doing this live is optional that you're going to find this worthwhile i want us
2:04 to have a lot of time for this discussion but let me just give you a quick overview of what i mostly want to do i want to explore with you why your interest in artificial intelligence which is what led you to take this class is actually incredibly relevant to policy to politics and law and along the way you're going to see it's also relevant to international affairs and geopolitics but really in the course of this talk i
2:31 want to share with you some reasons not only why you should be interested in law and policy and take your technical knowledge and expect that it's going to be relevant to a lot of really important questions the world is facing i also want to maybe give you a sense of how i became really interested in the subject along the way and i'm going to try to share my slides now so you have a better sense of what we're talking about
2:57 so let me start by by noting that right now you're at an amazing moment in your life you're learning about artificial intelligence you have this extraordinary university at least virtually around you eventually you'll be back here physically i hope and expect and you can look at this talk and think about it from the perspective of a technical expert which is what you're becoming by taking this class but before we get to that i want you to
3:22 imagine yourself not as a technical expert but to think of yourself as just a citizen somebody who has to think about how does this technology affect daily life who's being affected by it where are the inequities what are the opportunities for understanding it better and then near the end of the talk i want you to imagine yourself as a policy maker somebody who has to make decisions about how to allocate scarce resources where government budget should go
3:48 what people should do in the legislature and the courts around how to resolve the technical questions and policy questions and the legal questions that arise and what you're going to find is that your technical knowledge is extremely relevant to a lot of these crucial issues but at the same time you need to round out that knowledge by understanding a little bit about the legal system and about organizations so the bottom line really is that i'm going
4:14 to share with you a lot of different messages but the core message is that this technology that you were learning to master has not only benefits but risks and in the course of implementing that technology society is going to be shaping how that technology is used through the legal system and also through organizations through the associations the institutions the groups but especially the firms the agencies that so many of us are going to work in law firms government agencies
4:45 big corporations nonprofit organizations now i want to tell you because i know that it is difficult to hang on to your attention but i'm going to try that there are some things that i absolutely want to remember have you remember if you remember one thing about my whole presentation it's that the impact of artificial intelligence on the world on your daily life is a function of law and organizations it's not anything that actually acts directly by itself but it
5:14 has to be mediated by some organization by like what stanford does or what the you know republican party does or what the united nations does but also it's needed by legal rules and along the way you're going to find that we might sometimes talk as though we're discussing the possibility of developing legal rules that will apply to ai well i'm here to suggest you that many of those rules already exist the question is just how to translate them to this
5:40 context if i can convince you to remember two things and not just one i'd like you to remember the point above but then also i want you to remember some crucial terminology and that is the techniques of ai like machine learning are different from ai systems or applications and these are the mechanisms obviously that instantiate the techniques that are attached to a user interface i'll say more about this later and that actually spit out information recommendations
6:10 insights that people will then act on and if miraculously enough i can get you to remember three things and this is the last thing i really want you to remember for sure it's the previous two points plus that point that law is kind of merging with the design and policy challenges that are implicit in ai so what i'm going to end up is by telling you that lawyers are becoming more and more a little bit like people like you
6:34 who are trying to wrap their minds around machine learning supervised learning unsupervised learning reinforcement learning and in the same way you and your community the people who are the technical experts are increasingly pushing to ask questions like what is the right way to use this technology what do we want it to do and not do so with that as background let me acknowledge more explicitly the benefits side of the ai technology you're learning to master because if we don't then we're going to
7:03 get a pretty distorted picture if you were physically on campus right now and you were walking around stanford you could go to 10 different places on campus where really cool stuff is happening that is relevant to real problems that people are facing around the world and where ai techniques are being used to try to make the world a little bit of a better place so let's take for example the population of the world that is facing
7:28 serious nutritional stress meaning people who are at serious risk of starving a generation ago that population was much bigger than it is now but sadly that population is still stubbornly large 700 million people are so faced serious food insecurity these are generally the people living on a dollar a day or less you see some of the kids here overwhelmingly that population is concentrated in africa and in india and in asia but there are also some people in north
7:57 america and europe even who face food insecurity the different ways that we have to allocate resources effectively to make sure food doesn't go to waste to make supply chains more efficient to pinpoint where there are problems in real time and what's more a lot of this population not only faces problems around food but also faces problems around education the distribution of access to high quality education is incredibly unequal as you know we're all a part of that
8:24 system we take part in it so when i think about the future of both nutrition and education in a world that is more equitable and more benign i cannot imagine that future without some use of artificial intelligence techniques to democratize education to make the delivery of food more efficient to pinpoint problems in real time in somewhat similar fashion this quirky set of four images you see here is an example of the work that dan hoe my colleague at the law school is
8:54 doing with some colleagues to use satellite imagery to pinpoint where sources of pollution are in much more accurate fashion than anything the government currently has really and what that would allow us to do is more effectively cross-reference the self-reported data that comes from firms that claim to be complying with environmental law with the reality and it takes some fairly sophisticated but also in some ways intuitive machine learning techniques to make use of this visual data and then you've got a picture of a
9:25 courtroom this is not the kind of courtroom where i sit because it's the trial courtroom mostly this is where trials are actually heard in superior court the reality is that in california if we had more time and if we were in person i would ask you to guess the number of cases we hear in california courts every year and generally speaking when i ask that question people like 20 000 and i give a shock response that's too long people
9:50 say okay 200 000 and my eyebrows still go up and finally we'll get to like 800 000 well the actual retail answer is like 6 million cases a year so it will not shock you to hear that probably 40 50 of those cases the litigants are self-represented there are people like you they don't have a lawyer they are trying their best to navigate an incredibly complicated system i would love to imagine a world where the distribution of legal knowledge is not
10:22 so restricted just to people who have a stanford law degree or a similarly great credential or can pay a lot of money for a fancy lawyer but where software and ai systems that you might help design can help people navigate a very intricate legal system but at the very far right you see a picture of an african-american man under the words criminal of justice and there's a question mark there and why i'm doing that is to highlight that
10:49 this whole world we can imagine also has its risks and its downsides and to make this more concrete i want to focus on one person in particular the gentleman you see here robert williams is one of many people whose lives are being affected by the fact that artificial intelligence systems are not just theoretical anymore in terms of their practical application they're being used in all kinds of settings including in the criminal justice system so here he was one day in a suburb of
11:21 detroit when he gets arrested by police he's told that he's being arrested because he's suspected of committing larceny which is a fancy word for stealing robbing a store in detroit and it turns out that because the police were using an image recognition system doing facial recognition and they had a base of data corpus of 49 million images and the system indicated that the image from the security camera in detroit
11:51 matched robert williams's picture he was arrested now you might ask did the police have another reason to suspect him did were there outstanding arrest warrants for him had he committed similar crimes in the past and the answer is no no no okay so once he was arrested the police admitted that the photo is a little blurry they admitted that they didn't have any other information about him and after a little bit more discussion they ultimately agreed with mr williams that the picture
12:23 really just didn't look like him meaning the intuitive human response was like no that doesn't seem to be you but the algorithm says it's you so what do we do the answer is he was detained for 30 hours now i'm not suggesting that there aren't worse things than being detained for no reason for 30 hours but i'll tell you i grew up on the u.s mexico border and it was a fact of life in my family that sometimes you need to
12:47 cross over to the american side to go shopping or do something else like that and being detained even 45 minutes an hour an hour and 15 minutes all those things happen to me it's not very pleasant so you can imagine what it's like or you can begin to imagine if you tried what it's like to be detained more than 38 hours and they'll be told that it's because a computer made a mistake computer must have gotten it wrong was
13:08 the exact thing that he was told everything that i want to share with you from here on out in a way you could sum up by asking this narrow question of why did this happen to mr williams and what does that mean what are the remedies do we have a legal system a society where we can sort of disentangle the mistaken uses from the correct ones manage the risks appropriately can we take seriously the fact that humans also make
13:31 mistakes when they're looking for faces i'll say more about that in a moment but i hope i can press you to think about the situation with mr williams in a little bit of a broader context because we could talk about criminal justice or we could talk about testing as you may know the intellectual the international baccalaureate exams this last year because of code were not actually given but instead students were given a score that was their predicted
13:58 score based on the previous portfolio work they'd submitted we can talk about testing in remote settings where your facial your image is sort of being analyzed by a camera that's trying to detect whether you're cheating we can talk about insurance we can talk about it 36 other domains where this stuff is really effective life and the broader question really is what does the incident involving robert williams tell us about law
14:29 about artificial intelligence and about how society is changing and our legal system is changing in response to this technology so that is the tip of a very very big iceberg now let me acknowledge again the point about how there's a lot about this subject that goes deeper it doesn't just start with the history of artificial intelligence it actually starts with the history of really modern society now on the screen you see the picture of a very intense looking man
15:01 named max weber for anybody who's ever taken a class on social theory or sociology his name might be familiar there's a lot i could say about him but here's the main point i want to make writing in the very early 20th century max weber was looking around society and observing things noting society didn't work the same way then that it did 100 or 200 years ago many many people worked inside organizations with a hierarchy formal systems of authority
15:32 organizations had a director an assistant director officials clerks and all of this observed max labor was a means to which the modern nation state processed information took it and decided what to do with it rationally sometimes developing the mechanisms to act as if by reflex by recognizing the kind of problem and quickly delivering a response sometimes by elevating it to people who could sit in an office talk in a conference room and come up with a solution thinking
16:00 presumably logically and what max faber noted much to the influence of people who came after him including yours truly is that these bureaucracies aspired to work like a machine right they were trying to automate the process of decision-making in some way to the point that it could be predictable and rational and weber pointed out that that was all well and good but there were going to be some problems along the way and in some ways i'm here to tell you that many of the
16:30 problems that weber highlighted how we have a love-hate relationship with these bureaucracies on the one hand we think that they're inefficient that they're rule-bound that they're not creative that they're frustrating that they're slow but at the same time we can't live without them that will end up illuminating in some ways some of the really interesting choices we have about how we use artificial intelligence maybe in some ways to replace conventional bureaucracy but i would argue in other ways
16:56 to replicate and in some ways channel some of the same tragic conflicts and tensions now channeling max weber to some degree and also reflecting my own interest in ai in 2016 i wrote a piece that had the following punchline basically which was sometimes we're going to deal with the concerns we have about the role of artificial intelligence by suggesting that really all we're building are recommendation engines not really that different from the way netflix works you
17:27 may also like to watch this judge you may think that this person deserves a harsher sentence but it's really up to you judge you don't have to be the one to decide or rather you don't you have to be the one to decide we don't have to be the ones to decide we the ones who designed the ai system we're just giving you a recommendation we're using these techniques to give you a sense of what the likelihood is this
17:48 person will be offended and the point i was trying to make in 2016 which seems now like a long time ago is that actually that line between the computer program in particular the ai system that has sophisticated user interface capacities to sort of speak to you in natural language or to serve up the information in a way that's easier for you to assimilate it's really difficult to police that line between they're just supporting your decision and they're
18:14 actually making the decision now here's one place where i can highlight my point at the very beginning about how law emerges with organizations which merges with ai if you really want to understand the effect right so if you want to know if an ai system is actually serving as a decision support tool rather than actually making a decision you're going to want to know the answer to questions like well are the designers of that system liable
18:37 if it turns out to make a recommendation that's really really bad that results in people getting injured or conversely is the organization run in a way that the decision making the decision maker using the ai system is being audited and is being checked to see if all of her decisions are just exactly rubber stamping what the software does and if that's the case what's the point of having the human decision making in the loop anyway right
19:03 so i'm giving you the sense that we're building up to this point of all these conflicts and questions and meanwhile people like robert williams are getting arrested but now let me return to this point about how humans often are not great decision makers too so we can think about where it is that human cognition fails in terms of perception we can think about how humans add up information and come up with a thought or a decision we can think about what
19:30 motivates humans whether it is that even if i have every reason in the world based on my job to be fair when i'm working in a police station i'm deciding who to arrest if i have an improper motivation if i want to impress somebody who happens to be on a ride-along with me that day or if i really dislike the person who works in this particular area of town and i want to arrest them because i have a nefarious motivation
19:54 that can mean that human decision making gets all messed up and even the legal arrangements we have to police human behavior can fall short but my next slide which is probably the messiest slide of the whole presentation so you don't have to memorize it or even read it all you know i can make these available to you later but here's the punchline the punchline is that the mere argument that humans are not as good as the
20:19 performance of ai systems in a discrete test like facial recognition does not really answer the question of how you want ai systems to be used by organizations to make decisions because the devil's really in the details let me just pick two points here to highlight let's talk about perception so the field of the neurophysiology of how vision works is really really complicated and fascinating and it's not an accident i would argue that some of the coolest
20:48 things that we have been learning about how to develop better image recognition systems in the ii space are influenced by what we learned from neuroscience but the fact that that's still a bit of a mystery highlights to you that we actually only understand a little bit about how humans make visual processing decisions for example we know that it takes about 100 milliseconds for humans to perceive whether a picture reflects a person of one gender or another generally for
21:18 humans to pick up emotions for humans to recognize familiar faces but eyewitness identification i do you remember that involves unfamiliar faces do you remember whether this image is showing you the person that you think you saw two weeks ago when the glass was shattered and somebody came into your apartment at night and grabbed your beautiful collection of baseball cards and left that is a lot less exact and as one of my colleagues explained in a dissenting
21:44 opinion in a case called people we read you know we would be grossly inaccurate if we suggested that that is a system of identification that works really really well but then of course if you compare that to the way ai systems work on the one hand ai systems might be in the lab much more accurate than humans at picking out the similarity between two images just at random not that are sort of known earlier the way humans might know
22:12 them earlier but on the other hand the ability of those systems outside of the lab to operate effectively and particularly to detect emotions for example is not so great these systems have in a number of applications and instantiations real differences in how effectively they work for pictures of people who identify as white rather than for blacks or asians and of course you have all kinds of other failure modes like hackie and then of course we could talk about
22:44 legal arrangements and here i would just note that we humans have hundreds of years of experience dealing with human mistakes that's really what the legal system is designed to do we are only learning now how to adapt our legal rules and standards to deal with the mistakes that machines make we're not starting from scratch but it would be a mistake to assume that we figured out exactly how to do that so now i want to make the point that when
23:14 we are dealing with problems posed by ai in the legal system we are not starting from scratch and the best way i can make that point is to just highlight for those of you who are vicariously interested in asking yourself what would it be like to go to law school what would that feel like and you're thinking well maybe that would not be terrible maybe it might be kind of fun i'll give you a flavor of some of the
23:36 subjects that people learn about in law school and it will not take a rocket scientist that will not take a stanford computer science professor to see that these subjects that we cover in law school they're just literally touching right up against ai already and it will continue to be the case so in an area of law called agency law is where we figure out like if professor sadik says to a ta i want you to go
24:00 across campus and i want you to pick up this particular computer and i want you to carry it to the other side of campus and along the way the person picks up the computer but then gets distracted drops the computer and kills a bird and it turns out that that bird is the prize-winning bird of somebody's like bird collection or whatever does professor sadik end up being responsible well agency law resolves that kind of question when are you
24:24 responsible for the actions of others in your organization of your agent now ordinarily agency law applies to the actions that you begin to put in motion that some other human being engages in but you can totally see how this branch of law is beginning to grapple with the question of when you are responsible for the actions that you set in motion because you design an agent to do something like to sort employee applicants and then the agent does that the artificial
24:52 software based agent okay so then you have my core field of administrative law and legislation this is the law of what counts as sufficient justification for any action of government if the president signs an executive order saying i don't want the census to keep on going until december i want it to stop in october when does the president have the power to do that how does that power get into some conflict potentially with the power of congress to pass a law
25:22 saying how long the sense is supposed to continue you get the idea what if the government says well you're going to have to move out of this home because we want to build a road through here what right do you have to challenge that kind of action so obviously the more and more that government decision making involves reliance on machines the more and more that this branch of law is going to have to deal with the question
25:43 of what does it mean when the machine is empowered to play a crucial role in that government decision does that make it more reliable less reliable more fair less fair when can we do that one can we not do that last but certainly not least tort law tort law is about who has a duty to whom what counts as a reasonable decision and how do we attribute causal responsibility for bad things that happen translation let's say you're back
26:06 on campus and sadly you get covert 19. can you blame the university when why can you blame the university why can you not or you know let's suppose forget covert 19 for a moment let's suppose that you're in a lab and sadly your lab partner decides to try to attack you and you survive but you're asking well wasn't the university responsible for making sure that i wasn't attacked that's toward law and you can imagine that as the
26:34 information that is the fuel of modern ai systems and the sort of fuel for machine learning increasingly flows to systems that are interconnected questions about what a decision maker does with that information and whether that information makes the decision maker responsible for a different kind of safety protection relative to somebody that could be protected that all becomes more interesting okay so let me give you some context for how to think about these problems by just acknowledging the history of ai is kind
27:04 of long and it does not start with the birth of the internet it goes back further in history to some of professor sadik's colleagues in the computer science department at stanford so i could go on and on about this but my little subtext in addition to what i want to share about the history of ai is to kind of quickly give you a sense of how in the world i became super interested in this beginning a little in college but
27:27 then again when i worked in government at the treasury department and even more so when i came back from working for obama in 2010 so just look at those pictures for a moment you might recognize some of these faces i'm sure you recognize at least one the one where the woman had read as it were but if you go back a little further what you're going to see in the picture under 1950s is herbert simon a really really smart man whose parents
27:52 were refugees from germany who spent most of his career at carnegie mellon university and i mean come on you have to be pretty smart if you start as a political scientist but then become interested in psychology end up writing about economics and winning the nobel prize in economics and along the way you become like a major pioneer of ai that was herbert simon for you he was so brilliant i recommend to you any book or article ever written by herbert simon
28:22 among other things one of the reasons he won the nobel prize in economics is because he developed the notion of bounded rationality which is at the core of what we call behavioral economics now the notion that you may not you may be best modeled as a human not as somebody who's trying to optimize but as somebody who's trying to satisfy a certain threshold and we can certainly use that insight to imagine how to design a software agent and how to do
28:45 machine learning which is one reason why you can imagine his expertise and brilliance got transferred over into ai he's most associated ai with the story of development of systems to do basically like first order logic and mathematical type reasoning what some refer to as good old-fashioned ai and i'll just note here that that was really important but always treat it as the holy grail maybe something was elusive and not possible
29:17 to realize the kind of instinctive almost automatic decision making and motion that now is so much at the cutting edge of what we are helping robots and ai systems to do the recognition piece was missing even if the cognition piece at least around how you prove theorems was possible to instantiate early on briefly by the 1970s the picture really is different here that picture includes ed fagenbaum somebody who is a colleague
29:48 in computer science and always someone fascinating to talk to somebody who's been one of my mentors a little bit and trying to learn about ai and he's very much associated with expert systems with taking insights from not only the work of herbert's simon who was actually ed fagenbaum's but also from psychology and sociology and decision theory to develop systems that could act almost as experts and replicate knowledge in particular domains
30:19 and then by the 2000s the real phenomenon that changes everything and certainly gives rise to the prominence of the person in the third picture sheryl sandberg is the rise of the internet because of course all this stuff about ai was happening partly in academic labs and partly in defense departments but suddenly the ability we have to harvest and centralize billions and billions and billions of pieces of behavioral data from humans and of course to do it in
30:49 systems that work faster and have access to more computing power lets us do some truly amazing things and i'll just note here that my interest in this begins in college in 93 when i was trying to understand how human decision making could be modeled so kind of very much the herbert simon sort of work but when i was working in treasury in the late 90s wasn't lost on me that there was just so much data that the us government had gathered
31:13 around financial transactions and i was interested in privacy as you might have been but also interested in the idea that if that data were available how could it be used in a way that was efficient lawful analytically sophisticated to detect really really problematic uses of the financial system including to commit corruption for example to launder money and so on and so i became exposed to some of the techniques that you're learning about in this class right now
31:41 when i came back from the obama administration in 2010 it struck me that so many of the domains in which i was working particularly around public health and criminal justice were already being affected by early examples and applications of this stuff so i became really interested in trying to understand how this stuff would affect every aspect of decision making in law and political science and try to learn more about what you're learning about right now so here's where i want to highlight
32:10 where my own thinking went after i returned from the white house in 2010 it struck me that some of the most interesting work happening in ai in universities but also in the private sector was about pushing the boundaries of analytical techniques to discern patterns unsupervised learning reinforcement learning and so on and the breakthroughs were really extraordinary and they continue to be but it was also striking to me that these techniques in their raw form were not necessarily designed to influence or
32:36 help non-experts they were not necessarily designed to solve real-world problems so if instead you're looking at how ai techniques get used like they were used in the arrest of robert williams you're not dealing with ai techniques by themselves you're dealing with ai systems which my co-author and i defined using probably a little too much mumbo jumbo a socio-technical embodiment of policy codified an appropriate con computational learning tool so a system to gather data and learn from the data
33:09 and embedded in a specific institutional context meaning it fits in an organization and is given a certain purview people who make decisions are told here's how you can use the tool here's how you shouldn't use the tool right and really what that means is when you want to understand how ai is being used in the real world you have to understand relationships of power who gets to decide that the system works the way it does and that somebody
33:35 can point to that system and claim that it embodies some kind of intelligence why does this matter well it matters because now here we get to the other side of the coin of the internet right we're not in a world where this is mostly happening in the lab right now we're in a world where really important things in the world are being affected by ai i cannot give this lecture without pointing to the toothbrush that somebody
33:58 recently gave me as a gift that advertises how it uses artificial intelligence to learn how to brush your teeth and this is the genesis of a concept i call toothbrush maturity when technology gets to be so ubiquitous to the point that it intersects even with a toothbrush then you know that you're dealing with something that has to be understood in its real world context and not just in the theoretical stories you can tell about how well it's
34:23 going to work another example of this really though is that the very large internet companies that are around us in silicon valley have a market capitalization that you can't really explain without understanding just how well online advertising must be working and how much it's leveraging the enormous amounts of data that are generated by the internet and analyzed by some of the ai techniques that you are learning about here where's this going is so interesting and the short
34:50 answer is i really wonder whether anyone really fully knows and that's true of almost any technology right you can't always predict by the way i'm about three quarters of the way through the presentation so just bear with me for a few more minutes but you know i can point to the different things here but the main point i want this slide to highlight for you is that some of the breakthroughs that we're seeing right now are not so much
35:13 progress in terms of just more clever algorithms or even more different data but it's partly that it's partly just leveraging more and more computing power i wonder where that's going to go i don't know that that's sustainable but i do think that if you want to get a sense of where this field is going think a little bit about language in particular because if i go back and think a little bit about how government agencies were
35:38 making decisions in the late 90s when i was there most of the expert analysis was being done using techniques like probit logit econometrics like regression stuff you're going to be learning about in this class but it was being mediated through humans presenting to each other what ar systems may increasingly have the capacity to do is to use those very same techniques but to then communicate with the user in a way that is adaptive to the human
36:03 and able to leverage language in a way that previously software did not do so that persuasive ability of software is something we have never really seen before and as we have more effective use of compute and greater use of compute i think the feats that will be possible when you sort of marry up the gt3 type stuff with the analytics will be very different which is to say a lot of humans who are consuming the output are not necessarily
36:29 going to be in a great position to be very sophisticated arbiters of whether what they're being told or recommended is accurate or not just to wrap up there are all kinds of interesting intersections now about the law and ai and policy problems that result i want to make a pitch to you this is kind of tentative i'm not as certain about this as i am about other things that we're actually having this really weird bifurcated bimodal distribution of
36:55 attention to the problems where some problems now are so familiar that we don't necessarily know how to solve them but you will hear the buzzwords very often explainability interpretability bias privacy etc and these problems i think of as not medium to long-term problems they are present day problems they have hit already just ask robert williams and then when you see an interview with like elon musk then you're going to hear about catastrophic or existential risk
37:26 i think that it would probably be a big mistake to ignore catastrophic of existence or existential risk much as i would have argued in the 1960s if i'd been alive then and an adult then anybody who was interested in the future of fossil fuels even if we didn't have all the science would probably be making a mistake if they ignored completely what the risks might be if they were trying to understand the risk systemically for the planet
37:50 of the use at scale of these techniques for producing energy once the rest of the world meaning poor people in indonesia india and africa and china began to demand the level of consumption of energy that americans and europeans have taken had taken for granted right but i still think that in some ways the catastrophic or existential risk piece is not a risk that i believe the world is likely to be facing in five years or eight years or ten
38:20 years that's just maybe something we can go into in the q a about why that is but i suspect that the the level of delegation we have already engaged in the ai systems doesn't get to the point where they can protect their purview and power without our intervention as well as they might someday and obviously that requires for the thinking but that leaves some seriously interesting issues that i think really deserve attention more in the short term
38:48 for one this question of where causal responsibility lies when a system that deploys ai acts in a way that is not safe think about the autonomous vehicle but not only the autonomous vehicle think about the ai system in a large company that increasingly is making financial decisions reviewed by humans perhaps needed by humans but increasingly in an autonomous way i think that problems involving power and collective action are really interesting in this space so if you're
39:19 running a large company and suddenly 27 of those jobs are now going to be or the functions done by different people in different jobs are now going to be done by ai systems how does that redistribute power within the organization how does the advent of lethal autonomous weapons influence the distribution of power in geopolitics for example how does it empower countries with smaller armies and so on another point which is familiar to people working on cars in particular is
39:50 that precision can spur disagreement right now a lot of legal rules are written in fairly general terms which is to say humans are not supposed to drive when they are impaired they're supposed to engage in driving that shows reasonable care etc etc these are fairly vague descriptions in the court figure out what that means in particular fact patterns with the help of a jury but when you can actually program an automated system to make split-second decisions that are extremely precise
40:21 about when and how to prioritize exposing some smaller number of humans to risk when you can save a larger number of humans like a variation of a trolley problem that will spur disagreements that didn't exist before just like mapping technologies when they developed and became more precise spurred disagreements between different countries that previously shared borders in very inhospitable locations when the border could really not be traced with quite as much detail and specificity just to mention maybe one or two last
40:53 quick things on this slide i think it's going to be really interesting as ai systems pose the question of what it means to maximize social welfare like how do you design a system that is going to have its core attribute this is what you want to do and some people are trying to do that it's going to try to keep humans safe or it's going to try to avoid doing anything that will imperil too many people
41:16 taking human values and turning them into code is actually really really difficult and it is related to the process through which humans think about change and conflict which is to say we often deal with conflict or institutions like courts and legislatures increasingly as we deal with conflict or machines we'll have to program machines to help diffuse conflict and not only to point out how two views that seem to be very similar are actually in tension with each other
41:44 all right so we've gotten to my very last slide i'll end here there's probably too much text on it but here's what i want to highlight if you are listening to this lecture and you're thinking i hope that part of my career is spent thinking about how i can help move ai and the design of ai so that it is socially beneficial i want to highlight to you that that is actually really difficult to define it
42:08 probably in ways you've already anticipated but i want to highlight in particular a tension between two different ways of thinking of what the social good is for purposes of ai and pretty much everything else in one version of what it means to work for the social good you basically develop systems that increasingly are good at giving people what they want what they say they want but especially what their behavior indicates that they like and that they value
42:32 so the entertainment that they want the products that they want the classes that they want the kind of teaching that maximizes student evaluation you know feedback and so on but of course you know part of what makes life so interesting is that there's a separation sometimes between what people say they want and what they actually want or what people say they want and what they do or for that matter what people want at time one when you started listening to this
42:57 lecture and what you want right now it's probably to stop right and once you start admitting to the idea that human welfare is more complicated and further once you start designing systems that are in real time shaping human affect and culture and behavior it actually becomes really really difficult to know where to land like how to take advantage of the human knowledge you have to know like how to make humans better off i don't know how we solve that problem
43:24 but i do know that the things that i do as a judge and the things that we do in law schools as lawyers and the things you do as technical experts are increasingly merging and i don't think that we can answer these really tough questions without acknowledging that our bodies of knowledge have a border that is increasingly becoming really blurry and with that i'm going to stop and thank you for listening and i'm looking forward to
43:48 your comments and questions and feedbacks concurring opinions to sense whatever you want to share so i think yeah so the way you're going to go for questions and thoughts is either raise your hand or put it on a chat and then they'll just call you or read the question great
44:23 i should add that if we were in a real classroom what law professors do is we call on people so i would call on people but i can't really call them people so i'm going to wait for your questions i have a question you mentioned that a lot of the laws right that will be used for ai actually already exists you can give an example for an existing law that you think will be used for ai applications
44:46 or systems pretty soon absolutely great question thank you the short answer is let me let me start with the common law which are the subjects that we teach law students in the first year that are they're sort of defined by the fact that the law is a little bit more judge-made so someone like you as you as you learn basic civics of a system like the american system in most cases the legislature is elected to enact what the law is and then
45:18 the executive branch will implement it and then the courts will judge and interpret like what the law means but there are certain branches of the law where in our anglo-american tradition the law is actually first developed by judges over time little by little case by case and then the legislature will jump in and they will tweak the law in this way that way so so those bodies of law include contract law and tort law and both of
45:42 those are so clearly about ai at some level so contract law is the law of promises that we make to each other and when they're binding and when they're not right when professor saudi promised you a good class i think she's delivering as are her colleagues in this class but like if you say well the class wasn't good i've been defrauded you know the law will try to determine whether there's like a actual legal claim that you have or not
46:07 so in the air context just imagine now for a moment that increasingly transactions are being made by two ai systems making spot contracts with each other in a split second because you've pre-programmed one to say as long as this stock falls below this price buy a whole bunch of it right and when the lines of supply and demand cross because two ai systems are talking to each other the deal is made but then it turns out
46:31 that maybe this was class c shares of stock not class a and so like whose gets who's stuck you know dealing with the cost of a transaction that is not what both parties wanted so existing contract law has a lot to say about that now tort law is an area of law that frankly when i was a law student i thought was really boring when i was a law professor i thought was kind of technically complicated not that
46:54 interesting to me i had my hands full of teaching other stuff as a judge i think of it as fascinating really really interesting that's the body of law governing when your conduct harms other people and when you were liable for that there is no way to have a discussion about cars like automated cars without having tort law be a big part of that so like you know to what extent is the designer of the software that runs the
47:23 vision system for the car responsible for this person who gets run over versus the person who runs the company that tested the software versus the company that designed the car and marketed for you versus the driver that pushed the car to operate in really bad weather and and torque is really complicated but i will just give you one quick insight which is very intuitive which is one workforce concept in tort law which has been on the books for a while
47:48 is the notion that the tortella should pay attention other things being equal to who in that chain of causation is in the best position to have avoided the harm at the lowest cost least cost avoidable and you can see how here that would be a really interesting and important question like who could have done a very little bit to avoid that person from getting run over there are dozens of other areas of law but this to
48:09 me is a really good example of how when silicon valley says like oh we have to decide if ai is going to be regulated i think there's a little disconnect with reality okay i think we have go ahead yeah yeah thanks i have a question so
48:39 these systems are probably like very opaque for judges and people who actually have to make decisions about you know what happened and how does it interact with case law and statutes and whatever and my question is if as a judge you have to answer a question of fact about a particular ai system you just like have experts come in and testify and be like well nobody else has any hope of being able to understand it so we're just going to take what they
49:04 say as gospel or do like judges and clerks and things like actually try to educate themselves on math behind how the stuff works that is an excellent question and i have good news and bad news what do you want to hear first let's let's hear the good news okay the good news is that in some ways the problem you've just described is not completely new to law and in our system we have a kind of interplay of
49:33 decision making that involves jurors being asked subject to some instructions given by judges how to interpret an ambiguous fact pattern lay people who are supposed to be acting in their best conscientious effort to do the right thing and follow the instructions and then experts who in an adversarial system are selected carefully vetted carefully debriefed before they sort of come before the court and then subject to cross-examination who can shed light on
50:04 things and help the jurors and the judge make decisions and then judges who are supposed to resolve questions as a matter of law like questions that ultimately are more about how to interpret the legal issue itself and if for example a statute says a quote-unquote highly autonomous system shall be regulated subject to subpart j but then like is this a highly autonomous system or not so that's a mixed question of law in fact for example the judge might do part of it so
50:29 in particular the expert testimony piece will frequently involve experts who come and pick at the really intricate like math type questions that you're raising and i wouldn't say the system is perfect but i would say the system works okay so other contexts where the highly technical gets adjudicated kind of like it would in an ai context would be dna evidence for example base pairs and you know what does it mean if you say the match is 1 in
50:59 1.7 billion how do you know that what's the difference between 1 and 1 3 and 3 million which is what the other experts said like we have a way of dealing with elections here is the bad news though i think the bad news is that none of the technologies in the past have had the potential that ai systems do to talk back and i think that is like not a small thing because what that means is
51:26 these ai systems can be designed in a way that i think creates a little bit of a comforting illusion that even the experts understand what's really going on when they may be influenced more by design choices that might be really really hard to arbiter where the ai system is actually maximizing not the level of accuracy it conveys to the user about the mathematical basis for a conclusion like that this person is likely to reoffend
51:57 but instead maximizing the possibility that the decision maker who's being influenced by the ai system is going to agree with it and that could even be an expert who's testifying right so this is where ai accountability begins to merge with cyber security with me like ultimately like cyber security problems are very much about how like if you go back literally to supply chains and how you can mess up the very core architecture of how like a
52:22 microprocessor works the ways you can bias results that can become incredibly difficult even for an expert to pick a part and i don't think we have a great answer for that like there may be blockchain type like really fancy ways of of using like you know hardcore like encryption like stuff to sort of like have greater confidence in results and to know when things have been messed with but somewhere along the line there are humans and humans are imperfect and i
52:52 just i worry about that piece of it a lot awesome and we have a couple of questions in the chat are you able to view the chat let's see i see three participants raised hand that's oh do you want to take both of them together yes let me see from a global perspective
53:23 any collaboration between countries regard of ai adoption you see ai systems from countries with different values will be prevented from being adopted okay and then similar to how developing countries will strive to attain the same level of life quality as developed countries it seems just a matter of time before ai becomes the next thing how should we think about international cooperational rivalry oh yes in ai development is there anything we can do as technologists to help how should we
53:51 navigate a time where u.s has marked ai software export restrictions okay great questions let me start with the second one a little bit so i think that as technologists you can probably help by trying to make sure that the hype doesn't run away with the discussion of these issues so i could find people who are in the national security world i
54:22 can find people who are in the public intellectual world who will see the relationship between the us and other countries so much through the lens of rivalry that very little space will be left for any collaboration between scientists for example between civil society nonprofits that are trying to reduce the risk of climate change by using machine learning tools or whatever and i think that technologists will be important voices in saying we can be legitimately concerned about
54:52 how differences in technological development can affect geopolitics or relations between different countries but not run to the conclusion that everything is pure competition i bet you that there are people in this class who are not born in the u.s well i know that's the case because i wasn't born in the u.s and i know that's true of others of you and to me that's like a really poignant reminder of the risks of having the conversation about eye shut
55:17 down to the point that it becomes too one-sided and too too much just about national advancement in no way do i want to deny and this will get me a little into this first to the earlier question that there were different agendas different goals different geopolitical objectives of different countries and that getting some advantage in ai technology can translate into potential military and economic advantage so some balance has to be struck and that requires some careful discussion it requires some
55:49 norms it requires some cooperation from universities because we think of universities as working best when mostly things are pretty open in the university we share knowledge i learn from you all you will learn from me etc but the reality is sometimes careful languages have to be drawn and ultimately that does reflect the reality that countries do have different values about others and i'll just mention one word that it is a simple way of making that point but there are other
56:14 examples i could give you the word is privacy so i can imagine many countries in the world that could argue either that their populations simply have no particular reason to value privacy the way americans do or that whether their population values privacy or not their law is such that they prioritize other things and they're simply going to gather a ton of data about a ton of stuff and i don't think that data translates automatically to greater power in the ai
56:44 space but it is a significant advantage if you have it like it'll be interesting to ask in the coming months and years to what extent some combination of reinforcement learning and the generation of artificial sort of fake data like or fake you know information inputs into like a reinforcement learning algorithm can make up for just raw access to real world data but i still think a corpus of actual human behavior is quite something and if you simply
57:09 didn't have to worry about privacy the insights you might get into the expressions on people's faces when they're having a private conversation about something incredibly sensitive or the fear you can see in somebody's eyes when they begin to realize that they've said something on social media that is likely to bring a knock on the door from the police is valuable particularly if your goal is to try to not only not only improve the lot and the
57:36 well-being of the population but to control them and to limit the extent to which they push against you so i think this leaves us in a really challenging space like i really really would urge all of us to highlight the importance of some public collaboration across borders on this we don't stand much of a chance i think of getting really to where humanity needs to get to on so many crucial issues including ai safety by the way we
57:59 don't have some sharing of information pure competition is going to drive a lot of the dangerous and riskiest technological experimentation on the ground at least for a while on the other hand i think we'd be naive to think that everybody shares interests so some degree of building of norms and cooperation among communities of people who are in civil society in the world of nonprofits or philanthropy or education i think will be really crucial
58:33 as a question yes the general question this comes to be real if i just say i've developed projects and i have to connect all the information from the network so what should i share or take care of something like the copyright privacy and the commission from the source society or even some citation i have to prepare this kind of thing
59:08 yeah yeah thank you i this is a really big subject let me just try to abstract a little bit from your good question you're basically raising the broader question of how we might think of ownership and responsibility over data particularly as people work together on these projects that mine huge amounts of data and use it in maybe different ways i mean i and i'll give you a short answer but then i'll elaborate a little bit the
59:36 short answer is increasingly the world is waking up to the fact that control over data really is control over property in some ways so just as you might have a use agreement that says to somebody you can you can license the use of this technology that i've developed let's say a piece of hardware like a camera that can see really really well at night but by using this camera you are agreeing not to use it to you know
60:05 look into people's homes without the permission or something so increasingly i would say there is a criss-crossing regime of law some of it is state law some of it is federal law pertaining to particular classes of data like medical data that really highly regulates the use of data by the same token are there opportunities still to harvest scrape even data from say the public internet that can then be used in different ways well of course and sometimes
60:32 that will allow breakthroughs to occur in ai but but this is where it gets really tricky because we're really in the midst now of developing national and then eventually maybe global norms about what it means to appropriately design ai systems that will get rid of data that no longer are needed for the original purpose so let me give you two competing perspectives about that first one is that because ai systems are so capable of developing insights using the techniques you are
61:02 learning in this class that discern patterns in the data that human intuition would not have been able to detect big masses of data used in new ways is risky because it means that maybe you end up getting embarrassed about people discovering that the fact that you like a certain kind of literature and that your eyes move in a certain way when you're in a conversation mean that you have a really short attention span and you can't be
61:29 trusted with a certain kind of job right so those questions are partly being mediated with the respect to like do we not create norms that once data are used for the purpose for which they are collected we destroy the data now i'll give you the perspective of me the academic so a lot of my early work as a law professor was historical i would actually look at old memos and documents that were going back to like presidential decisions made like
61:57 in the roosevelt administration where i was looking at what happened when roseville was trying to reorganize the government on the eve of world war ii and how he was trying to protect certain programs from being defunded as the country was getting ready to go to war by the way as a little subplot on that one of the things i learned about that i was not expecting is that there was a big biological weapons research program that was being funded with white house
62:20 support despite the fact that that arguably contravened certainly contrary statements the white house had made and arguably contravened you know certain aspects of legal norms at the time but long story short the point is if the norm had been followed that you only keep the data for its original intended use and then get rid of it where in the world would i be able to write the stuff that i did how would i be able to do it and you could say okay
62:44 well those are presidential records that's different but in general the historians who write about how humans lived 150 years ago are doing it with data that were not intended for historians so i think we have to strike a balance but i would say you should assume whenever you're dealing with data that there's probably some rule and if there isn't some rule that's in the law it's probably in human subjects requirements in a universal
63:17 yeah and my question is with regards to the out of lab and the in the models working inside the lab which you mentioned so and so let's say if an organization develops a model let's say it's for an auto self-driving cars for example i'm just picking it up 99 of the time it inside the lab the models work very well and outside the lab you know it causes
63:50 loss of a person a life is an example i'm taking an extreme here so if this kind of situation comes to you as a judge in your code what is the decision making which on the law side which you take with regards to the ai model and what's the thinking which goes behind you to make this kind of
64:20 the judgment against the maker of the model or so it's working 99 of the time but there are failures because you know the models are true given what has been coded and what the training has been it has been done on so i'm just wondering the decision process what would so is it tied to the risk or which the company has done the due diligence or
64:51 what is the essence of the responsibility and so that was the broad questions thank you thank you so that's a another good opportunity for me to share with you how much existing law is already grappling with these issues that arise that are so relevant to ai and particularly to the scaling up of ai outside the laboratory so i'll preface this by saying that because i'm a sitting judge i wouldn't want you to feel like i'm telling you
65:23 exactly how i would decide the case if it came up because we actually have cases that are not like what you're saying that are pending in the california courts and i'm not supposed to say how decide them but i can tell you i can tell you in general the bodies of law that are relevant to trying to deal with this question and in what direction they've moved over time so we have bodies of law particularly in
65:47 tort law and in contract law and in consumer protection law more generally that can use basically three sorts of techniques to deal with the risk that you're pointing to which arises when you go from highly efficacious behavior in the lab to what happens when some technology is operating quote unquote in the real world so when a vision system for example is tested in control conditions it works fine but now you're putting it on the front end of a
66:15 car that is going to work mostly autonomously and it's driving around palo alto and then even driving around some much more irregular environment like some dusty road unpaved in northern mexico so one body of law is tort law again this is the body of law involving the duties that you owe for example to others as a company or as a person and here a core insight of tort lies that you have a duty of care if there's a theory of
66:45 negligence that is being or well let me put it this way if the claim is that the manufacturer should have been more careful than the manufacturer was and the manufacturer owes a duty to the person who's using the product which is its own separate question a crucial issue will be the extent to which prevailing norms in the industry about how much testing happens outside the lab or followed or not the more those norms converge the easier it is for the company to say
67:17 perhaps like well look we did some outside the lab testing you know you could easily spend a billion dollars a day testing outside the lab infinitely but we did enough testing that we met the industry not a different technique would be to rely more on contract law where you could say i was sold a product that was guaranteed to me to have a degree of safety and efficacy in it and in fact it didn't reflect that
67:45 because it wasn't tested outside the lab and the promise that was made to me was not that this product was just tested in the lab it also implied a lot of testing outside the lab and the third strategy is more administrative regulation so this is like what the fda does with respect to pharmaceutical products and here the key insight is that we don't rely purely just on tort law or contract law we actually have the government saying to you you
68:10 can only sell this product if you've tested it in a particular way and as you go through that pharmaceutical approval process and get into phase two phase three phase four trials effectively what you're doing is you're going further and further outside the lab so we can do this to some degree they're going to be some nuances but i think it's really important to remember we have different tools we can use to deal with that risk thank you
68:37 notice we have a couple of questions on the chat so we do some of those and then come back to the raised hands yes i think maybe we have time just to take these two are you guys these two questions so first on the criminal justice system oh i see that okay i'm going to just read it out if we find that an algorithm deployed in the criminal justice system that doesn't explicitly take into account race but it's systematically
69:02 discriminated against black people perhaps compass are the your legal ways to counter that discrimination is it constitutional they can do a count to protect a character like race oh great okay doesn't mean we just shouldn't use the algorithm oh okay excellent question so the short answer is the legal system has in america for good reason been long deeply concerned with racial inequities whether it has been sufficiently concerned is a question that we can leave for another
69:32 day and others can talk about but it has been concerned about it and that concern is reflected in several parts of our legal system it's reflected in a lot of statutes at the state and federal level against discrimination by race like provisions against discrimination and employment and hiring but it's also reflected in the constitution for example in the equal protection clause of the constitution and here i would just say the legal system treats differently uses
70:03 of race that are well first of all the legal system treats different uses of race than other classifications that people might be subjected to they are subjected to what the legal system calls strict scrutiny which is a very very demanding form of review where they're essentially explicit classifications are not permitted unless there's a very compelling strong justification on the part of the government and there's no really no realistic way of doing it in a
70:34 different manner where it gets more complicated with something like compass is where there's no explicit racial classification and you still have biases and here i just note that you can find algorithmic ways to reduce that bias or even disappear it completely and there might even be legal reasons to do that but generally when you do that one of two trade-offs will happen you will either increase the likelihood that other variables that may not otherwise be so
71:04 consequential become more consequential so to some people that might mean you're introducing a certain kind of different bias although we may not care about it as much because it may not be racial or you simply reduce the accuracy of the overall model in some cases now that may be entirely sensible to do but these questions about when and how you recalibrate the results of a decision-making process because it doesn't take race into account and yet it still
71:30 gives unequal results is a very familiar and vexing and difficult question in criminal justice and in the legal system more generally let me just take the last question then which one is it the one that says via advertising complicated question actually okay so my question is regarding is that one how much tolerance okay would citing have when ai makes mistakes on the one hand we say humans are not perfect on the other hand we want to see how much a
71:58 company can improve an algorithm to avoid an accident caused by their ai powered product to what extent should they our product manufacturers optimize their product to become acceptable great question i think this gives us a chance to end where we began actually because when it started i highlighted to you that not only can ai have many benefits for society i named some but also that the relevant comparison is not to perfection but to what imperfect forms of decision making
72:28 we might have to rely on if we don't rely on a particular ai system but it would be wrong to conclude from that discussion that as long as ai systems are more accurate than human decision makers then there's no legal problem or there's no policy problem i think instead the reality is that as ai performance improves in a very discreet domain two things might happen the first that are relevant to the answer to your question the first is
73:00 that we may come to understand and trust ai systems better to make that discrete decision so long as it doesn't introduce some other biases that we think of as even more concerning so notice the point that i made about how ai systems might be really good at picking out faces that are unfamiliar relative to humans who are picking out unfamiliar faces but if they're trying to discern emotion they may not be as good as humans right
73:28 now that might change over time but right now that means that we have to be very specific with respect to what we're expecting system to do rather than presuming that there's a sort of halo efficacy beyond the very narrow context in which it's been tested which might have to include just beyond the lab right number two as ai systems get better in general the standard of care that the legal system uses to discern whether something works effectively or not will begin to
73:56 be redefined so that it's not just human efficacy it's a well-performing ai systems efficacy and not only a well-performing a system but ideally a well-performing ai system that does not have a built-in set of biases that we consider problematic so that means for example if over time 70 of human passengers were in autonomous vehicles rather than human driven ones been a faulty form of performance from one of those vehicles that increases the risk of harm and actually results in somebody
74:27 getting harmed might still be actionable despite the fact that even that faulty system works a lot better than human drivers did it also means that even if ai systems are better at discerning new faces if their accuracy is much much better for white faces than black faces that would be a policy and potentially legal problem for some that might require remediation and attention even if the system works better than humans all of which is why i think these
74:57 problems are going to keep your generation busy for a long time thank you all right thank you so much you know thanks for the great talk and awesome discussion and there are lots of interesting questions this is really fun so let's thank you know again thank you everybody i really enjoyed this and i appreciate your very thoughtful questions and best of luck you