Transcript
0:05 okay hello everyone i'm dorothy static and i am one of the co-instructors of cs221 and today i'm here with percy liang and our group of tas here to teach the first lecture of 221 so with that i would like to first before getting started in the details of the class just introduce the team so i am dorothy sadik i am an assistant professor in computer science and i this is the fifth time i'm
0:35 teaching cs221 the second time i'm teaching it online virtually i think it is the third time i'm co-teaching with percy liang so really excited to to start the quarter with you guys here and and my research just a little bit about my research my work is on the in robotics and ai and in general very interested in the interaction between robotics and ai agents with humans and with other agents other intelligent agents so if these topics are of
1:05 interest come to office hours we'd love to chat about that and talk about it offline in general too my co-instructor today here is percy liang i think i saw percy somewhere yeah i'm here hello everyone i'm percy i'm the co-instructor and i think this is my ninth or tenth year teaching 221 it's really been interesting how ai has evolved since since when i first started talking about it my research interests are in machine
1:36 learning and natural language processing thinking about how to make systems more robust and trustworthy recently i've been really fascinated by what we've been calling foundation models models such as gp3 and bert and dali and i'm going to discuss that more later in the class all right thank you percy all right so so what are we going to be talking about today so our plan for today is to talk a little bit about some of the course logistics and then some of
2:06 the course contents like what are we going to actually cover as part of this class then we'll have some icebreakers so we'll have like a five minute breakout room we'll discuss things about ai and then toward the end of the class i'm going to talk a little bit about the history of a brief history of ai and then what ai is today and what are some risks and benefits of ai and how we should think about it moving forward so
2:28 so that is our plan for today okay so before i start also if there are any questions feel free to put questions on on zoom chat and or raise your hand and the ca skin and can try to kind of like answer the questions or like ask the questions like throughout as a as i give as i give the talk all right so so let's talk about course logistics so so we're going to have a set of
2:55 activities as part of the class and and last year when we had to go virtually because of because of covet we started experimenting with a few different ways of changing and reformatting the class and some of them worked really well and some of them actually didn't work so well so based on the experience that we have from last year we've decided to switch up the activities a little bit more and also like some of some of these
3:17 some of these changes really make sense to do also throughout like normal quarter to even if even if you're not virtual so so one of these changes is going from traditional lectures to something that you're calling modules so these are basically pre-recorded modules and lectures that are that are broken into small bite-sized chunks so so on every topic that we are we are going to cover in this class we're going to have a lecture of like 10 to 20 minutes and a module
3:44 really that goes over that topic and these are pre-recorded and you're going to release these like the modules for that week on the monday of the week so you have time to to watch these kind of like based on your own schedule and when it makes sense for you to watch these lectures so it's a little bit easier to manage these like bite-sized chunks that's one reason that you're going towards these modules also these are pre-recorded and you're probably going
4:09 to use the same recordings as as last year so so so we have more time to spend with you guys during our lecture times kind of in a flipped format okay so so that's the modules then in addition to that during our normal lecture time you're going to have two types of activities so on mondays you're going to have faculty chats so these are going to be unzoom and they're basically small group discussions with faculty on on ai related topics so
4:36 they're going to be basically six like 25 minute sessions so so every monday from from 1 30 to 3 p.m percy and i each of us are going to have a zoom room like 30 minutes each for each session and you're going to be assigned for one of these faculty chats so this is actually mandatory to attend at least like one of these faculty chats and the reason we are doing this is traditionally when you have a large ai
5:00 course like you have like 300 something enrollment and it's really hard to get to know you guys and actually like talk to you and sometimes it becomes like really difficult to know the faculty when you're in some of these larger classes and this is really like what what we are trying what we're trying to do here is is really to to get to know you through some of these faculty chats and discuss like some of the ai related
5:20 topics some of the some of the more recent material research material around newer topics like foundation models that percy is going to talk about or some of the some of the topics around robotics autonomous driving and so on so i'll talk about this a little bit later in the talk the exact format of it and and and there is a little bit of a homework to do beforehand before coming to to these sessions but but the idea is we'll have
5:45 these in pers not in person sorry virtual virtual faculty chats on mondays and lecture time and one other point that i want to mention is that if you have conflicts during lecture times you want to make sure that you wouldn't have conflict for the time that you're you're assigned so it does actually because that is mandatory so make sure that you actually don't have conflicts during lecture times all right so the second bit is problem session so this is going to be on
6:11 wednesdays so on wednesdays again during class time you're going to have these problem sessions and they're kind of like traditional sections except for we have changed them a little bit based on that based on the feedback that we got last year and during these problem sessions we're going to have the cas work out practice problems so these could be previous years quizzes or previous years exam questions or basically just problems that can help you get started on your homework or or
6:37 basically get ready for for some of some of the exam questions later on so so i do recommend going to these problem sessions they're incredibly useful to to get your hands dirty with some of the topics that that you're learning that week and again this is on zoom on wednesdays class time all right so what else we're going to also have homework parties so homework parties used to be very popular when they were in person and
7:01 last year i think it was a little bit more difficult to make it happen but eventually people realize that homework parties like matter a lot because that's a very good place to show up and and and work with other people on your homework problems get started on on on some of some of the more challenging problems together study together and the cas will be there to answer questions so these homework parties are going to happen on
7:23 nukes which is a platform that we started using last year again all the information about zoom nukes like all these all these links are is on the cs221 website so so the details of all everything i say today is also on the website okay all right so beyond homework parties we also have office hours so the cas have office hours and there are two types of office hours so there there are a set of in-person office hours that i was
7:48 talking about earlier these are limited but they're going to be in the basement of hwang and and they are group based there's no sign up required but but basically there's a group of students and the ca at a at the basement of huang in addition to that we also the majority of office hours are actually going to be virtual and these are by appointments so so we used calendar last year and part of it was just these queues were
8:10 becoming too long to handle so now we you have you can make it and make an appointment for ca office hour it's going to be a one-on-one office hour and these office hours will happen on weeks okay in addition our ca office hours are two categories so we have separated general office hours from homework office hours so if you have over questions you should just go to the dedicated homework office hours but if you have more general
8:33 questions about the course if you're thinking about your project if you're thinking about general ai questions you should go to the general office hours or the faculty office hours all right so and then the final thing that we have is faculty office hours so so percy and i both of us will have a 50-minute office hours weekly and the schedule for this is also on on on the website so you can take a look at that again it's one on one it is going
9:00 to be virtual and you can sign up for them beforehand and come and chat with us and again all details is on the website all right so let me just like see if there are any questions here does that anyone have any questions should i look at chat okay let me just quickly look at chat so many of you know each faculty chat to attend so we will be in touch about this soon so
9:30 actually on that there is i'll talk about this in a little bit but basically there's a survey that you need to fill out by i should notice by wednesday correct me if i'm wrong by wednesday no wednesday okay by wednesday yeah so that is basically to get your preferences on on what faculty chat you would like to go and then we'll also assign you after that to specific faculty chats okay all right
10:01 do we sign up for ca and faculty office hours through nukes so no we will use calendar for signing up but we'll use nukes for for answering questions all right okay all right so if there are not no more questions let me go to the next slide so so now let's talk a little bit about periodics so so this is a question that oftentimes comes up so what are the prereqs for the class so you need to have some programming
10:28 background it would be good if it is in python so so these are some courses that are prereqs for this course in addition to that it's a good idea to have some mathematic math backgrounds so discrete math cs10103 is is a is a prereq in addition to that it would be a good idea to have some background in probability and linear algebra so 109 and math math 51. okay but but in general we want you guys to
10:51 have general familiarity with these and have some mathematical rigor and general familiarity with probability linearity of our discrete math these types of topics we are not really expecting a very specific type of knowledge for example like in linear algebra you'll learn about eigenvectors but like we don't really like require the knowledge of eigenvectors in this class so so they're not specific topics that we're looking for but generally you want to know math you want to know
11:17 programming and and come into class with that knowledge and and the reason is that this course is also fairly like fast-paced so you don't want to spend your time like learning python or learning learning math through this class your your python programming and programming is going to improve your math knowledge is going to improve but you don't want to spend time learning these backgrounds and you want to really spend all of your time learning ai so so
11:40 if there are gaps some people do catch up it is possible but again you want to spend your time like learning ai so you kind of like leave it to you guys to decide and then and move forward and and you might ask okay how do i decide so we have a we have a couple of things online that you can take a look at so so we have a set of modules that we recorded actually last year and and these are
12:00 prereq modules so these kind of provide refreshers of some of these topics so definitely take a look at some of these prereq modules and that gives you a kind of like a good sense of what is required to know to come into this class in addition to that the first homework is based on foundations and the first homework really gives you a good idea of what to expect as part of this class in terms of again programming and math
12:22 knowledge coming in so take a look at these before deciding if you want to skip every wreck or not but in general again i do think it's a good idea to have these backgrounds coming into this class all right so let's then talk about grading a little bit so grading is fairly straightforward so so we're going to have a set of homeworks that's 55 of the grade we're going to have two exams so that is 40 of the grade the
12:48 faculty chats we actually count participation part as part of the grade so that is five percent and then projects we are going to make that optional this year so it's going to count down towards extra credit and then if you contribute to ed so we're going to use at this quarter as opposed to piata if you contribute to ed that is also going to give you some level of extra credit okay and in general you can take the class
13:11 for letter grade or pass no pass that is also like your choice basically so now let's talk about each one of these components a little bit in more detail so so in terms of homeworks we have eight homeworks and these eight homeworks are a mix of programming questions and written questions and the programming problems are mainly focused on a specific application so like for example we might be looking at blackjack as a game or we might be
13:37 looking at like pac-man as a game of pac-man or various types of topics like car tracking so so there's a particular application that is used as part of the programming component of the homeworks and these programming components they're auto-graded so and then there are a set of basically public and private tests so you should definitely try out these public tests first make sure that you you test this thoroughly because the grading is very strict it's based on auto grading and
14:03 you don't see all the tests so that's kind of like the point that i was trying to make here and then in addition to that you have seven total late states and you can use maximum of two per homework the reason for that is you want to release the homework solutions so you can't use more than two late days for homeworks okay so so that is homeworks okay so that's our plan for homeworks the usual we'll go with that
14:28 one other point that i want to add on homeworks is we are adding an extra addition to every every single homework which is an ethics component so it's an ethics component is going to be added to all of our homeworks is a new edition that you're having this quarter and we're also going to significantly change some of these homeworks to incorporate an ethics question into them so so we're trying to incorporate that throughout the class throughout these
14:51 homeworks so that would be that would be also an addition to consider this course all right so moving forward with exams so this quarter last year we decided to do a set of quizzes this year we're not going to do the quizzes that didn't really like students don't really like it every every week so instead we were going to have two exams okay and and the point of the exams is really to test the ability of your knowledge on
15:17 working in new problems it's not really to know like facts that you're teaching it's more about like your knowledge of ai and if you can actually apply that to new problems and and all these problems are going to be written so no coding and you should take a look at like past exams to get a sense of like that like how these problems look like and and what is a format of them so each one of the exams
15:38 is going to be a hundred minutes and and these exams are going to be to be open book so we actually have the dates for these exams already they are going to be released in a 24 24-hour window so they're going to be released on one of the first one is going to be released on october 29th at 3 15 p.m and then it's going to be due the next day at 3 15 p.m pacific time and then similarly we have
16:02 exam 2 on december 8th since it's 3 15 p.m pacific time is the time okay all right so we have these dates if you have major conflicts about with any of these dates you should let us know by october 8th which is week three of the class okay in addition we will not have any late days for these exams again and because we need to release solutions we need to make sure it works for everyone so no late days gets applied to the
16:29 exams and of course no collaboration on the exam so please do not talk about the exams on x so so like if you've done it you're done with it but there's still like time left within that 24 hour window do not post anything about the exam on on it okay all right so that was that was exams and the last component that is mandatory as part of the class is the faculty chat participation so so as i
16:57 was mentioning earlier the goal of this is really discussing topics around ai like related and topics around the ai so fill out this initial survey that i was talking about by wednesday so so that way we can start scheduling these you're going to be a sign in session again six sessions run in parallel on mondays and this is during class time on mondays so make sure that you can actually make them make that time and then
17:21 you should prepare before these sessions and these sessions are going to be on different topics so if they are on specific like research topics like robotics autonomous driving ethics robustness foundation models we often have some related material that that we released beforehand sometimes these are we had a set of fireside chats last year it could be like that fireside chat to watch and or talks last year it could be basically that talk to watch beforehand so you come to the come to
17:49 the session a little bit prepared and we can talk about these topics we also have another set of topics that are really about more thinking about academia versus industry graduate school thinking about like how you read a research paper so so some of these and other components that are maybe not necessarily a particular research area and again you'll have some material for this reading material to have beforehand so you come in again prepared so the way we
18:16 are looking at participation as part of these faculty chats is as you come in you should introduce yourself and you should also share a little bit about your thoughts or your goals for that session so you should actively participate in that in that 25 minute session and you kind of expect that when you're thinking about participation and grading participation during these faculty chats okay you will not be tested on the material that you're discussing on the
18:41 faculty chats so i also just wanted to mention all right do we need to attend one faculty chat session to get yes so you will be assigned one to one faculty chat if there is room you can actually attend more faculty chats we are potentially going to have more room based on the number of students who are enrolled but we will be in touch on like what is like what are the availabilities and if you can attend more than one faculty
19:10 chat but yeah you'll be assigned one okay so let me talk about the project also real quick so the project this quarter is going to be optional this is what we did last year too because the course is virtual and we thought it would be it might be a little bit more difficult to to find a team and work together but regardless like a lot of students did the project last year and and there are a lot of interesting ideas and projects
19:36 came out of that and it was really exciting to see like so many cool projects like during that quarter too so i do recommend that you guys look into this closely even though it is optional so so the idea is you want to choose a task where you can actually apply some of the ideas that you have learned as part of this class and and use those techniques for that particular task it's a little bit open-ended you
19:57 need to like decide what that task is but that's also the beauty of it right you can pick anything and apply some of the ai techniques that you're learning for that the idea is that you can work in groups of up to four people and then you also have a set of stones like you need to fill out a project interest form there's a proposal progress report and there's a video and final reports that you need to do so if
20:20 you decide to do the project and actually get the extra credit you should do all these different you should actually like pass through all these milestones and finish the project again the task is completely open and and but but there are a set of well-defined steps that we expect you guys to to to have throughout the course throughout the course for this for this project so this includes things of the form of like defining the task or
20:44 implementing your baselines and oracles and things of those form or having a literature review thinking about what your revaluation metrics are and and you will have a ca assigned to you if you decide to do a project you'll have a ca assigned to your group and your ca can also walk you through some of these different components that that you want to have as part of your project and in addition to that one other thing that we've added is a mandatory check-in
21:09 meeting with your ca so this is a 15-minute mandatory check-in meeting david your ca you think this is really useful and to make sure that you are you keep you keep up with the project if you decide to do it and in general if you want to think about ideas for what to do for your project or if you have some idea and you want to discuss it definitely come to office hours you can come to a person at nice office hour or
21:29 like the ci's office hours and discuss discuss some of these questions okay all right and the last point that i want to mention on honor on logistics is the honor code so i want to spend a little bit of time talking about this because this is really important you guys don't want to deal with it we don't want to deal with it so let's just talk about it and get it out of the way so especially this quarter given that
21:54 things are online we do want you guys to collaborate we do want you guys to discuss together learn together like think about problems together but the write up and the code needs to be entered independently so so you need to like write your code you need to write up your solutions like independently based on your own thoughts and your own ideas so please do not sure code please do not share your write-ups with others and don't look at anyone else's write-up
22:19 or code even if it is on internet and you found it but not look at these things and then yeah do not post it online but like if you're proud of your code you shouldn't post it on github do not do that and and in general when you're debugging try to look at like input output behavior you could be like going to homework parties and debugging your solutions with other people and really just look at input output behavior don't
22:42 look at each other's code and and that way you'll be safe but i do want to emphasize that we do run moss periodically and this will automatically like detect like if there is like matching between codes and and please like do not do that in boss is really good so yeah so and every year we have a number of cases and sometimes like we run these things mid-quarter so i want to also emphasize that and then you don't want
23:07 to you don't want to want things like yeah you don't want to go through these things like mid-quarter and and it's it's again something that we don't want to deal with we don't want to deal with let's just not do it you're also changing a number of like homework questions and you're adding like you're adapting things to make this a little bit easier on everyone all right oh and then the last point i want to make is on communication so
23:32 we're going to use ed this quarter so in general if you have any questions best idea is to put it public at the post that way of course staff students everyone can see it and you have a broader group of people who can answer that question and you can hope like probably other people are thinking about that question too so that's kind of a best way of communicating with us if there is a private question make a private ad post
23:53 and that way the course staff can see it and and for example if you have a question that can give away answers it's a good idea to to post that as a private post and in general if there are sensitive matters that you want to discuss or oe accommodations you should email the court the the this particular email address this goes to only four people this goes to percy and i and shiri and faith so our student
24:15 liaison and our head ca so if you have any sensitive matter you should just send an email to this email list that goes to the four of us in addition to that you're going to have periodic surveys you also have a welcome survey already on canvas so so please take a look at that and fill that up and and that way we can start getting getting some feedback and again as the course is virtual we would love to get
24:37 more feedback periodic feedback throughout the course so it will be great to give us feedback tell us what works what doesn't work so we can adapt and again all these details everything i've said so far is on the course website and and with that i can take any questions about logistics i know i covered quite a bit on logistics if anyone wants to just ask a question that's probably easier to because then so if so on the exam if we're looking
25:06 for a clarification should we post that privately to ad not at all or should we email it assuming that it's not something that was supposed to give anything away it's just supposed to be a clarification of what's intended by the question you should you should post a private private post on end that only go to their courses that was a good guideline but then as
25:39 far as with the coding of python what about the use you know in terms of basic routines obviously not trying to copy codes wholesale but as far as using things like stack overflow and and others as far as four obviously there's all the various things that can be used as sort of like you know virtual to tutorials and you know various things that you wanna accomplish with the python that
26:09 you're writing i assume that as long as it's you know just little routines that it's not a problem it's the problem is when it's starting to be that you're taking somebody's idea wholesale yeah and in general yeah so try to write things like yeah yourself basically like when it comes to writing the code part of it try to like you can get ideas you can discuss the idea of it with other people or you can like yeah
26:35 like look at like online forms for ideas but when it comes to writing the code try to just write it yourself if there are specific like things that you're not sure of like you should go to the ca office hours or should go to our office hours basically ask us like that specific instance and we can talk about it then yeah okay all right so let's move forward so now let's talk a little bit about the
26:59 course content okay so what are we discussing what is ai what are we going to be covering in this class so so in general you're an ai right like you're interested in solving realistic complex problems that have a lot of messiness and uncertainty and if you think about a complex problem let's say routing cars like in a city with a lot of like complex settings that is happening in that city how do you how do you go about solving that question let's
27:26 say the question is just routing the vehicles right like you're not going to start just writing out code for it right like you're going don't start from scratch and from like not really having a formal exam to just directly code that that seems pretty difficult and in general there's there's a gap between the code or the software the hardware that in general we developed as ai scientists as engineers and what is happening in reality like what is the
27:51 real world with messy with all the messy and complex messiness and complexities that exist and then really what ai and what in this course is trying to do is to bridge that gap to figure out how we can take some of these real world problems and and make it simpler in a way that is manageable so we can develop algorithms and code for it and for that we have a paradigm in this class that would like to follow and this
28:14 paradigm has three different core components three pillars and these three pillars are the modeling in friends and learning pillars and i'm going to talk a little bit about these so so the idea is we take a very difficult problem we model it and then we develop inference algorithms for it and then throughout the this process there could also be the model could have a set of unknowns and we use learning throughout to actually make our models
28:40 better so so let me let me try to make this a little bit more clear moving forward so so let's go back to this real world problem that you were talking about droughting vehicles like on in a city okay so so this is a this is a big problem and in general i would like to have a formalism so what modeling does is it takes that complex problem and it tries to come up with a formalism with a
29:01 mathematical way of thinking about that problem and and modeling just by definition is lossy right like i'm not gonna get all that complexity that exists in the real world right all models are wrong but some are useful right so so under that idea of course you're going to lose some of this complexity but we're still going to come up with something that is somewhat useful for the goal that we have maybe i would like to find the shortest surest
29:25 way of getting from one one road to another road and if that is my goal i can basically maybe model this real world problem as a graph problem where i have i have a bunch of edges and vertices and my vertices here are maybe my locations like in the world and then the edges are maybe the roads that connect them okay so this would be a graph model that represents that real world problem so we're going to spend quite a bit of
29:50 time in this class talking about modeling and then once i have a model then i can start asking questions about that model right i can ask for what is the shortest path of getting from one one node to another node or what is the most scenic path of getting getting from one region to another region or i might have different objectives that i would like to be able to optimize and inference is really a way of trying
30:13 to solve that problem and give us an answer to some of these questions that that we have here okay so how do we make predictions how do we figure out what is the right path to take in in in this problem is is kind of like the thing that inference lets us get and then finally the last pillar is learning and the way i want you to think of learning is is that if you think about that model
30:34 right like we're oftentimes you're not gonna write be able to write everything in in that model with all the complexities but all we can do is we can write a skeleton for what we're trying to do maybe a graph but that graph might not you might not know what are the edge like what are the what are the weights on on the edges like we might not be given given given the the edge values here because like that would be like too
30:56 complicated to write or we might just not have it like periods at the beginning so we often have a model without parameters and the goal of learning is to look at data and from data complete this model and and add these parameters that were unknown at the beginning so so what learning is really doing is is taking the complexity that we have in writing the specification writing the model and and takes that away and puts that into data and the fact that there
31:23 is data i can take that data and then based on how good that data is or based on what i can learn from that data i would be able to complete my model and have a better model than it can actually do and friends over so so we're going to have also learning throughout this class as a filler in every section that you'll talk about in this class okay all right so so modeling inference and learning are just three pillars that
31:47 keep appearing throughout every week of this class but what is our course plan so our first plan is really to talk about different types of models starting from low-level intelligence all the way to higher level intelligence and and we're going to basically go over a variety of these models but before we start talking about these we're going to actually spend two weeks talking about machine learning so so this is just to get some of the basics of machine
32:12 learning out of also machine learning in general is a very powerful tool that has been quite impactful in the field of ai so so it's a good idea to learn some of these some of these ideas in machine learning at the beginning so that we can actually also use it like throughout the class when we are thinking about learning modeling and inference throughout the class based on these different types of models that we will discuss throughout throughout okay
32:34 so so next next week and the week after is basically going to be modules and machine learning okay and and i'm just spending a little bit time on what is machine learning so again the role of machine learning is to take data right and and from that data try to generate these models that that were at the beginning incomplete but now we can actually use them and then we can actually incorporate the data the information that's in that data
33:00 in in the model and the idea of it is really moving from code to data so again adding moving the complexity that exists in the code to complexity existing data and then one other point about machine learning is it kind of like it requires faith right so so if we have some data based on that data below the model there's no reason on the surface that that model could work in a new scenario that it could
33:25 generalize to new settings and then we'll talk about the idea of generalization quite a bit like when is it that the model could generalize to new settings like if i've trained it on some set of data of let's say like house prices how can i make sure that this model could actually work in a new setting for a new house and and that kind of goes back to this question of generalization and we'll spend time on
33:45 that all right so that was machine learning so as we were talking about machine learning in the first two two weeks of the class we're also going to spend a little bit of time talking about reflex based models so these are kind of the lowest level of intelligence in terms of in terms of the modeling paradigms that we'll be talking about throughout the course and then example and and here's an example of a reflex based model so i'm
34:10 going to ask you guys what is this animal okay maybe you can put in chat what is this animal or just like chat what was it it was a zebra right and you were very quickly you were able to quickly figure out that you just saw a zebra here right and and this is really based on your reflexes this is really like an example of what a reflex based model could do other examples of reflex space models
34:37 are things of the form of linear classifiers or deep neural networks and the idea that i'm calling these like the reason i'm calling these low intel low-level intelligence is we're not making a lot we're not doing a lot of reasoning here we basically have a feed forward model and then we're not doing much computation into responding and saying well that was a zebra right like we quickly were just able to quickly say that that that was a zebra and this
35:00 reflect space models they're the most common form of models in machine learning they're often like fully feed forward no backtracking no reasoning about what was going on and just like evaluating the model i think deep neural networks are an example of this linear classifiers are an example of this and that that's actually a reason where we are going to discuss machine learning we're going to also spend a little bit time thinking about reflex space models right
35:23 inference is extremely simple and then we just call the model all right so moving along on top of the reflex based models the one level higher we're going to talk about state-based models and these state-based models we're going to talk about three types of them search problems mbps and adversarial games so so so what are state-based models so so here is an example let's say you want to you want to play a game of chess and you want to figure out
35:49 where what should be the next move of white so so this is not the same as detecting if that if that animal was a zebra this is actually a lot more difficult than that you actually need to sit down and do a little bit of reasoning and figure out what state in the world you're in and and figure out like how how the world is going to evolve so there is kind of like this notion of like sequences of of
36:13 actions and sequence of states that are coming after each other like a leading to b and so on and and this kind of brings us this idea of state-based models and it hasn't it has many applications including in in games so if you think about games like chess go pac-man starcraft these are these are all examples where we can we can think about state-based models as a good way of modeling them they show up in robotics all the time
36:37 you think about motion planning like getting a robot arm to move from one location to another location we oftentimes use state-based models as a way of formalizing that they also show up in natural language generation machine translation image captioning they're basically like all throughout ai and and they're very good way of thinking about like what are all the sufficient information that you need to know at the current time and how that should evolve like in the next time
37:04 step and then adding an ordering of going from this state to the next state so so we'll talk about three types of state-based models we'll talk about search problems where you can actually control everything so so you have a state and then based on the action that you take you end up in a new state we'll talk about markov decision processes which are making research problems a little bit more difficult by adding uncertainty that comes from the
37:29 world so so basically these markov decision processes are state-based models where you're playing against the nature right like nature gives you some probabilities you you would look at like coin tosses and based on that you proceed so there is kind of this notion of uncertainty and then we'll spend some time talking about games adversarial games where you're not playing against nature which is probably instead you're playing against another opponent which is also very intelligent and is making decisions kind of like
37:56 against you as opposed to you and then we'll we'll basically go over these different types of state-based models a little bit okay so and as part of the homework for state-based models we're going to play around with the game of pac-man i just want to show a quick demo of this game here yeah so you're going to play around with the game of pac-man and basically come up with algorithms for pac-man who can avoid ghosts and eat these food pellets
38:26 and it will be kind of fun playing around with it and let me go back to my slides yeah so and as you're thinking about pac-man and general state-based models the things to think about is what is a notion of states how should you transit how do you transition from one state to another how can you come up with a strategy a policy that can get you from one point to another so you avoid like the ghosts
38:47 and feet your food pilots and so on and these are some of the questions that we're going to talk about when we discuss state-based models all right so moving forward we're then going to move to the next level of complexity and intelligence and not complexity intelligence and that is variable based models so so an example of a variable based model is something like a game of sudoku so if you think about state-based models there's a notion of like sequential ordering of
39:15 states you have to do a to get to you have to go through a to get to b right like if you think about like moving through a graph to for solving like shortest path you actually need to like see city one to two and then after that ccd2 but there are a set of set of problems that don't really require that type of ordering that type of strict ordering so think of the game of sudoku
39:36 right like the game of sudoku you have a bunch of numbers you want to make sure that you can fit in like digits of one through nine in every row and column and the order that you put in these numbers that doesn't really matter right like you can put the nines first or you can put the ones first and that really doesn't matter really brings us this idea of variable based models where we don't have this script ordering and because of
40:00 that then we can do something that's a little bit more intelligent and and helps us come up with better algorithms in these settings so we will talk about two types of variable based models we'll talk about constraint satisfaction problems these are settings when we have hard constraints so sudoku was an example right like you have a hard constraint you have to actually fit like one through nine in in your board or like scheduling type problems like a person cannot be at two
40:24 places at the same time so it actually has very strong strict relations between the different variables that exist but we all in addition to that we have also bayesian networks that try to take those hard constraints and make them soft so there are stock dependencies when you think about bayesian networks like unlike let's say sudoku or scheduling so so an example is let's say you want to track an airplane or you want to track a car right like if you're
40:49 tracking your car you might have a set of sensors on that car and those sensors are noisy they're not going to give you the ground truth of where the car is you also know that your car cannot like teleport right so so the previous time step where it was and the current times the next time step they're related to each other and based on these different types of relations of where you're where the car is and where the car is going to
41:11 be and the fact that you have these noisy sensor readings you can have these soft dependencies between these variables and that allows you to estimate where the car is and then that's the topic that that we'll discuss like through beijing network we'll have a homework on this and that will actually be about tracking cars that'll be exciting all right and then finally the last components that we are going to discuss is going to be on logic so so this
41:38 would bring us to the highest level of intelligence and and for logic like as an instance of an example that uses logic we can think of a virtual assistant so think of a virtual assistant what do you want from virtual assistant you oftentimes want to tell it some information give it some information and you also want to be able to ask it some some questions right and then expect it to respond and maybe you would want to use like natural
42:05 language as a way of communicating with this virtual assistant so we actually go through a virtual assistant example as part of the homework and logic i want to show a quick demo of that again here let me see if i can bring this to the right window let me go to the let me see if i can bring terminal to this there you go there you go okay so this is actually a tool that we're going to play around with during
42:35 the during the logic homework and basically it's a virtual assistant you can give it information you can ask it information so let me let me try to some example let me let me actually give it some information so i'm going to say alice is a student okay sorry dorset could you zoom in oh yeah
43:10 okay so i told it alice as a student and it just learned something i can ask it now is alice student what should it say it says yes right because i just told it alice's student i'm going to ask is bob student what should it respond you should probably say i don't know right because how would it know i don't know let me give it some facts i can say students are people
43:41 okay then i can say alice is not a person let's see what it says in response to that okay it says i don't buy that so it understands it understands contradiction so so i i told the students or people i i showed a generalization and now it's a contradiction to that and it understands that i can say alice is the person let's see what that says it confirms i already knew that okay let's give it some more information
44:09 alice is from phoenix maybe let's do that house is from phoenix i learned something we can say phoenix is a hot city i learned something i can say cities are places places i learned something let me actually make this a little bit smaller so you can see this okay so if it is snowing
44:39 then it is cold so i'm gonna teach you kind of like anything else type of statement i'll learn something okay i'm gonna ask it is it snowing what should you say oh he doesn't know okay so it says i don't know okay so i'm gonna give it more information if a person is from a hot place and it is it is cold then she's not happy okay so i'm giving it this more complicated
45:10 if-then-else type statement i learned something i'm gonna ask is it snowing what would it say mommy doesn't know right i don't know let's say alice is happy okay now i'm gonna ask is it snowing what you could say yes it's not snowing okay yeah so yeah so so this was just an example that was going over like this
45:40 the spiritual assistant and you'll play around with this virtual assistant and in the logic module you will be thinking about this idea of giving information asking information and the logical relationships between them and and this will be something that we will work on so i just want to quickly show this demo but one thing to notice here is that here we were giving these kind of like heterogeneous information this was very different from me giving like a millions
46:05 of pictures of cats and training in all natural right like i was giving these very heterogeneous information and then the system was able to like reason about these this information in a very deep way right like it was making these deep connections and i could ask these questions from it and that's very exciting right like that like being able to have these types of deep deep deep interactions between the symbols that we are providing it all right so so that kind of brings
46:32 us to the end of this module where we are thinking about different types of different types of states state different types of sorry models and and as just as a just a quick recap right in this class we were going to talk about low low-level intelligence all the way to high-level intelligence models reflex-based models state-based models variable-based models and logic and in each one of these models we're going to talk about the usual paradigm that we
46:57 have that tries to do modeling so we'll talk about modeling in each one of these these settings and then we talk about inference what are the different inference algorithms that we can use in addition to that we talk about learning right like how can we have data and how can we how can we learn and improve our models for each one of these components so that paradigm keeps showing up throughout the class basically every week all right so now let's spend five
47:22 minutes and and let's have an icebreaker so he's going to put us in in groups of four and then in breakout rooms and during this five minutes let's just try to introduce ourselves to others and and just to maybe set the stage let's let's have a question and let's discuss a question and the question is what is the biggest benefit of ai and what is the biggest risk of ai and when you come back try to
47:46 like put that on chat and we will discuss and and go from there okay so let's spend five minutes on on in brave cat rooms so it was good talking to some of you during the breakout rooms maybe yeah so maybe you can kind of like put some of your responses some of the things that you discussed on chat as a way of like just discussing some of them also quick things don't direct message
48:18 me on chat i'm like barely looking at chat so if there are any questions please ping the cas or email me later on or with questions on ed okay all right so yeah any biggest benefits biggest risks anyone has thoughts okay thank you for starting this improving people's lives tangible applications mutual assistance biggest risks ml
48:49 fairness ethics yeah so these are all like great points i'm gonna talk about them a little bit a fifth time but now let's let's continue with the kind of like next segment where i want to give a little bit of a history of ai here and this is going to be brief i don't want to go into too much detail it's not going to be a complete history but i think it's good idea to talk about this
49:12 because it gives a little bit of like insights in terms of like why we are where we are today and how things shaped over time so if you want to give a history of ai like you can really like go back to 1950 and 19 1950 alan turing put out during put out his kind of like his landmark paper on computing machinery and intelligence and in this paper alan turing asked a question can machines think okay and he
49:37 came up with his answer which was the imitation game which you might know as the touring test and then you guys might be familiar with the turing test the idea of the turing test is that a machine can pass a turing test if it is able to fool a person into thinking that it's actually a human okay and and this paper was was really like foundational like in a sense that it started allowing us to think about
50:00 intelligence a lot more carefully and and actually try to formalize that in a in a better way and it it was kind of like one of the first words one of the foundational works that it started thinking about yeah formalizing this this idea of intelligence and then we might argue on if turing test is a good test or not in terms of measuring intelligence and then we might have various opinions on that but that part of it is not really the
50:23 part that matters the part of it that matters is thinking about intelligence being able to formalize it and one other thing that alan turin actually i'm turing actually like provided in this in this paper was this idea of separating the the question that you're trying to ask the what from the how like how are we going to answer this question right like alan turing came up with this idea of imitation game or or or and and basically what what this was
50:50 giving us was was this idea of object specification right this is the thing that we are trying to get this is our specification that we are trying to get at but how we do it or how the machine really does it but he didn't really specify that and this modularity of specifying what we are trying to get and how we are trying to get that this is really like a foundational idea that we have been also using in a lot of our
51:14 algorithms and we see throughout this class like separating out the objective from the algorithm and how we are going about it is actually quite important and then it's a very good foundational idea and and one interesting thing is that touring also provided like at the end of this paper provided some ideas in terms of how we should go about it and he talked about two different approaches one was this very abstract way of going about this problem kind of like a top-down
51:41 view kind of like how we would go about solving chess and this is really related to this idea of symbolic ai that i'm going to talk about and he also provided another potential way of going about this which is like having machines that have sense organs aka sensors and then teach them like a child right and then this idea of teaching a machine and then getting that machine and have sensors on it and let it sense data and from that
52:05 data try to learn that that idea is very related to the idea of neural ai so so since this point right 19 1950 where touring put out this paper there has been kind of like three different flavors of ai that has been around right this is there's symbolic ai there's neural ai and there's statistical ai and i want to give a little bit of a history brief history of each one of these so let's start with in 1956 and let's start
52:31 with the the story of symbolic ai so this is the first type of ai flavor of ai that i want to talk about so so the term ai really comes back to to this age of 1956 and this is when mccarthy basically organized a workshop and at dartsmith college so john mccarthy he was a faculty at stanford cs he actually created the stanford ai lab and and he organized a workshop at dartmouth college that summer and he invited a lot
53:00 of other big names like marvin minsky newell herbert simon he invited all of these people and the goal of this workshop was to think about intelligence right they wanted to they had a very ambitious goal they wanted to think about every aspect of learning and then every feature of intelligence and they wanted to model that so precisely so they can have a machine that could simulate that and this is a very ambitious goal right and then they were
53:25 really after generality they wanted to figure out what are the general principles of intelligence and learning that they can really get so so they can have an artificial intelligence and an intelligent agent that can stimulate that and that was that was really exciting and immediately after after this workshop all these people went back their ways and they started producing really cool systems around this time so this was really the birth of ai and you started seeing a lot of
53:51 early successes so like 1952 arthur fami or arthur samuel put out one of these first checkers programs that was able to play at the at the level of an amateur like play checker at the level on amateur which was really exciting we also in addition to that we had in 1955 noah and simon came up with the theorem with kind of our first fear improvers and and they basically had the system that could that could solve
54:19 generally solve problems and solve theorems they came up with a proof for a theorem that was actually a lot more elegant than than like a per like than what people had before and they tried to publish a paper on this proof but the paper got rejected because the reviewers thought the theorem already existed but it was really exciting to be able to have systems that can prove theorems that can play checkers that can generally solve problems
54:43 and there are a lot of optimism right like all these really famous people in the in the field and they all had a lot of optimism about what is possible with ai right herbert simon said machines will be capable within 20 years of doing any work a man can do marvin minsky said within 10 years the problem of artificial intelligence will be substantially solved claude shannon said i visualized a time when when we will be to robots
55:07 with dogs or to humans and i'm rooting for the machines so these are not like random people on the street famous people like founding fathers of ai and and these are like some of the optimism overwhelming optimism that people had around that time what is what we could actually do with these ai systems and unfortunately we started seeing very underwhelming results so around this time right the government really cared about the problem machine translation and there there's a lot of
55:33 funding around it and then we started seeing results that weren't very like that that weren't kind of underwhelming and then here is kind of a made-up example but the results were things of the form of you may you might have a text that says the spirit is willing but the fresh the flesh is weak and you might translate that to russian and translate it back to english and you would get a text that says the white card is good but the meat
55:57 is rotten which is which is very which is not very good and as we started seeing these results right like governments started putting out a report about how these results are are not are not so great and then they started cutting off funding for ai research and this is around the same time that we started seeing the first winter of ai so so a lot of optimism wasn't really going anywhere and we had this first winter of ai and that wasn't
56:25 so great so if you think about this first early era of of ai what were some of the problems so some of the problems were first off we had very limited computation like a lot of these problems were written as logical problems and and usually like resolved as search problems where the search space was just growing exponentially and with the limited hardware that we had it was just like not possible to solve these very difficult problems
56:50 but even if we had like infinite compute at that time which we didn't there was another problem and this other problem was the fact that we had limited information right like if you think about solving some of these very complex ai problems that people were thinking about they they needed to write out these problems and the knowledge that exists around them using using words and objects and writing out the concepts and then it was very difficult to actually
57:14 provide all these information and we had really limited information about some of these concepts but regardless we started seeing a lot of interesting contributions that came out of this era even though it was a failure and we had this winter of ai there were a lot of interesting ideas that came out around this time like we had the list programming language we had ideas like garbage collection and time sharing and a lot of these ideas are associated
57:36 actually to john mccarthy but it was exciting to see a lot of advances even though like we there were the problems were still there and we couldn't really solve the big problem and this really brings us to the era of 70s and 80s so in 70s and 80s really people started thinking about this idea of knowledge and building knowledge based systems and and and kind of like the core idea was knowledge is really the key if we can
58:01 encode knowledge if you can bring in ideas from experts and donate like bring in your domain knowledge and incorporate that in the system then we can actually solve interesting ai questions so so this was the rise of expert systems where we can basically elicit like domain specific knowledge from an expert and encode that into like these if then else type statements into rules that the system can call and be able to like solve various types of problems
58:27 there was also another shift around this time so so the first era of ai right the john mccarthy like dartmouth college like workshop that happened that was all about understanding intelligence being able to say well what is human intelligence and can we stimulate that and that didn't really work out but in this new era what happened was people started kind of like changing and changing paradigms and they started thinking about applications a lot more so sure you're not going to be able to
58:55 think about intelligence and simulating intelligence but i can build systems that can be used in chemistry or they can be used in medical diagnosis or they could be used in business and then this was kind of like the first era that people started building ai systems that maybe they learned so much about simulating intelligence but they were they were about solving interesting useful problems that could be used in industry so lots of events during this time
59:18 right so so these knowledge-based systems they really helped us with both the information and computation gap right they allowed us to like incorporate knowledge and information and by doing so they allowed us to kind of like prune the search space and that would help us need like less compute to solve some of these problems so that was exciting and this was the first time that we were seeing some of these real applications that actually impacted industry and that was also a lot that
59:45 was also very exciting but there were still some problems around this era one of the problems was these rules were very deterministic and they couldn't really handle the uncertainty that existed in real world we had all these deterministic connections and rules that were never coming together and they weren't really capturing the complexity of that that exists in the world in addition to that the rules were becoming very complex very quickly so so this is a quote from terry vinograd who
60:12 was a faculty and hci in cs department in hci at stanford and at the time he was faculty actually and working in ai at mit and and here's here is some codes that he said around these knowledge-based systems he said well these systems are a dead end and they have very complex interactions that are that are difficult to handle and they're just really no easy footholds to have and and this really brings us to the second winter of ai so lots of
60:39 excitement right we're seeing real applications but there was still quite a bit of difficulty into extending these systems and and this was kind of the end of like this era of symbolic ai right symbolic ai really dominated ai for many decades and then and i i now want to go back in time right like i'm now in 1987 but now i want to go back in time and tell you a little bit of a history of neural ai and
61:05 where that started and what was what was kind of the progression of that and that takes us to 1943. so going back in time let's think about okay artificial neural networks and how they started so so in 1943 mccullough and pitts they they came up with the first artificial neural network where they had a single neuron and they basically modeled that's that single neuron with that and they they thought about very similar relations like ants and nerves and and they
61:33 weren't thinking about learning rules or anything of that form at that point and then that is kind of like 1943 the very first version of artificial neural networks and in 1949 help came up with the idea of coming up with a learning rule here this learning rule was very simple it was cells that fire together wire together and this learning rule it was actually kind of like it didn't really work and it wasn't very unstable but it was one of the first learning
61:56 rules that was put in place and finally in 1958 you started seeing some advances in artificial neural networks this is when rosenblatt came up with the perceptron algorithm for a single layer neural network which is basically a linear classifier and this perceptron algorithm was even been like was was being used until very recently and it showed a lot of success it was actually very powerful and there was a lot of excitement around it 1959 we started seeing the analog of
62:24 linear regression that was adeline there was a multi-layer extension of that madeline and and this was actually used for removing echoes on phone lines and this was again one of the very first times that people started using artificial neural networks for for a real application like removing echoes from a phone line and in 1969 that was an important year for artificial neural networks so so this year minsky and packard wrote a book on artificial neural networks they
62:51 wrote a book called perceptrons and they basically tried to analyze mathematical properties of linear models and the thing that they showed was actually something that was very simple which is a single layer neural network is a linear classifier it's actually not going to be able to solve the function of x or and then this this book is really associated to shutting down the research on artificial neural networks so that was a time that people started thinking maybe maybe
63:18 these types of artificial neural networks are not very powerful and maybe we should stop doing research on them even though the book wasn't really any saying anything about these neural networks regardless they started seeing a revival of neural networks and and around 1980s and this was kind of with the rise of convolutional neural networks and the idea like this came under the umbrella connectionism and you started seeing that this very very first convolutional neural networks and the training of them was very ad hoc
63:47 but but in 1986 that was kind of around the time that we started seeing better training and giving like more principled ways of training these systems so so around this time rama hart hinton and and williams they popularized or kind of like reinvented the idea of back propagation and then that was adding a lot more principle into how we should train these systems and 1989 again that was like one of the first times that you started seeing
64:15 these systems being used in practice so john le applied commercial neural networks for recognizing handwritten digits and he actually deployed this with usps for detecting digits of zip codes which was really exciting but still this idea of artificial neural networks it was still like a niche like it wasn't like a thing that everyone was working on and and that was until like the era of like deep learning and the 2000 2010s like era so and part of the
64:41 reason was it was actually really difficult to train these models but in 2006 hinton and all they developed the system an unsupervised layer-wise pre-training system that was helping pre-training some of these some of these neural networks and kind of reducing the efforts that goes into into training the disease models but the break really happened around 2000 so 2012 we started seeing systems like alex said that were giving us huge gains in terms of object recognition like these systems basically
65:15 revolutionize and transform the field of computer vision overnight right the type of the computer vision course that i took in 2012 was actually pre-neural networks and it was very different from like what is being taught today and and this basically changed the field like overnight with with the rise of commercial neural networks and training these systems and being able to do object recognition and finally in 2016 also we started seeing things like alphago like another breakthrough and and alphago
65:45 was basically using deep reinforcement learning to defeat like world champion go player and that was another breakthrough that that was that was a game that people were not thinking we're thinking it was a lot more difficult and it was really exciting to see deep learning and deep reinforcement learning is able to solve some of these problems so all right so let me try to wrap up because i know it's almost 3 p.m and and we'll release modules on some of the
66:08 rest of the rest of the lecture later today but let me just give you some some kind of like food for thoughts i've talked about symbolically i have talked about numeral ai the symbolic ai is really this top-down view that that is that goes back its roots goes back really to logic right like you had these very big goals of building a virtual assistant and neural ai on the other hand it's more like bottom up it's
66:34 trying to solve these perceptual tasks and and the two like might seem like might seem to have very philosophical differences and there's they might seem contradictory but they're actually not very contradictory like there are a lot of deeper connections between them and today actually people are thinking about integrating them in ways that that we weren't able to do before and even if you go back to the history of it right like what mccullough and were doing with
66:59 the first neural network was was actually analyzing properties of a logical system or alphago if you think about alphago it's a very logical game that you write like the the the rules of the game in logic and and it's using it's using neural networks to solve that game so there are deeper connections between these two between between these two views of ai and they really come together all right so sorry for going over a little bit
67:27 what is left as part of this lecture is really talking talking a little bit about statistical ai we'll release modules on these and then thinking about where statistical ai is coming into play and and wrapping up the history of ai and then the other part that is really left in is talking about ai and what are some risks and benefits of that and that's something that you guys talked a little bit about during the breakout rooms but we'll also
67:51 talk about that in lecture so if you want to watch these lectures later that would be cool and with that i can if there are any questions i can take any questions otherwise i'll see you guys next week you