Chapter 2.0 - Matrix and Vector Properties

0:08 we're going to talk today about Matrix and Vector properties these are matrices and vectors or essentially linear algebra is the most important mathematical structure we have for ourselves in terms of representing data representing a collection of of of simulation or computational points everything going to be put into vector or Matrix form and so the manipulation of these vectors and matrices become very important for us especially as the most common

0:38 mathematical problem to solve is probably ax equal to B in other words a linear system of equations and so we want to talk about some of the properties here again this is a series of lectures in data driven modeling and scientific computation you can find more information here at my website as well as this GitHub link has all the code that we will need for this so Matrix and Vector properties so what is a matrix first of all a matrix is just a

1:06 collection of numbers it's like the filing cabinet of the mathematics world right so we think about this here as essentially having a collection of data which is a certain number of rows and columns and you can specify the location by AIG J so a little AIG J which would tell you which row that's typically the first number designates the row the i j a i j the I is the row the the J is the column and this gives you a coordinate

1:32 system for representing any data point in here so typically you can represent like this and parentheses form AI J is component by component now these matrices are typically rectangular right so you specify the number of rows and columns and then the Matrix a u you would load in information or data or computations into this Matrix and then manipulate this all a vector is is a m matx that has either a single row or a

2:03 single column so right so one way to think about it it's a really skinny Matrix so that's the way we think about vectors in general vectors are a subass of a matrix so we often refer to a matrix as an m byn so M rows n columns and so we're going to be doing manipulations on this and we want to know some of the basic properties of how we can do manipulations okay so there are some

2:31 Concepts we should know one is What's called the transpose and basically what I want to do in this first lectures walk through properties and these properties become very important as we do manipulations of data and as we manipulate with matrices the first thing we want to do is what's called The Matrix transpose the transpose is given here so the transpose denoted by a t is you switch whatever where the number is in row and column space you switch

2:59 column and row so AI J T becomes Aji so essentially what this means is columns flip over to row vectors and vice versa so here's an example 1 1253 essentially this column Vector of one two becomes a row Vector one two the 53 becomes a row Vector 53 so you just essentially flip the row column space with the transpose complex conjugate here's another operation denoted by a bar is

3:30 the bar tells you wherever you see an i in other words an imaginary number you flip from the I the sign of the I from I to minus I so for instance here if this is the Matrix a it's complex conjugate wherever you saw an i you now have a minus sign in front of that I so it's conjugation and then the adjoint this is an important operation which is a complex conjugate transpose we often to

3:53 know that a star is you do the transpose plus you do complex conjugation so the joint becomes a very important operation for us in general and we'll use it in different computational settings is this ad joint Matrix and remember it's not just the transpose but complex conjugation now if you have only real data then the adjoint is equal to the transpose okay so that's some basic concepts we have and we can talk

4:25 about the qualities or properties of these matrices so Matrix a is equivalent to a matrix B in other words they're equal if component by component they're the same so each row row and column component AIG J is equal to each row column component of b which is B J so component by component every Matrix has to be exactly the same there's also a concept of What's called the zero Matrix which means all the components of the

4:54 Matrix are zero each row and column component is zero and we can add and subtract matrices by just simply adding them component by component so if I take a matrix a plus or minus B then I would just simply say the new Matrix that's this addition or subtraction is just simply you add or subtract sorry that should be a plus minus right there you should add or subtract component by component okay so this is basic manipulations of

5:24 matrices you do have some important properties you have the communative property a plus B is the same plus b plus a you have Association A + B plus C is equivalent of a plus b plus C so these are very simple things that we learned in in in algebra but here they are again you can multiply a matrix by number Alpha and what it does is it multiplies every single component by Alpha so if you look

5:53 here Alpha a is equivalent to Alp each component or essentially each component gets multiplied by that Alpha itself so that's multiplication by a scaler and if you multiply two matrices this is one of our critical operations is Matrix multiplications and specifically we often do Matrix times Vector multiplications but remember a vector is just a matrix really in Disguise then we have a plus Bal C where the C IJ is essentially the sum of the a

6:27 i k b KJ so how does this work in practice so let's do a manipulation here's a matrix a and another Matrix B now this Matrix B happens to be a vector how would we multiply these together essentially what we're going to do to do this multiplication is you take component by component the 3 * 1 add it to the 2 * 0 add it to the 1 * 2 you add that all together right which is here

6:56 that equals five so that becomes the first row of this product Matrix here and then you go to the next row again 1 time 6 5 * 0 0 * 2 add them all together and that becomes six and then finally the third row 1 * 1 8 * 0 3 * 2 add them all together and it's seven so this is Basics matrix multiplication and if I didn't have just a column Vector you here about a matrix it' be each single

7:25 column multiplying each of these rows up here okay so when you're doing this multiplication if I look at this Matrix here the number of columns have to match the number of rows here otherwise this doesn't work so that's one of the key restrictions that you have in this multiplication process so we'll be doing a lot of Matrix multiplies usually if you don't have that right the rows and columns don't match python will complain and

7:56 will say you can't do this multiplication and you do have Key Key Properties you have distribution the distribution property takes a can be multiplied by B plus C you can take a and then plus a c you have associative but you don't have communative a * B is not equal to B * a this doesn't work for Matrix algebra in general okay so just keep that in mind but you do as distributive and associative properties so be be aware of

8:28 that okay Vector multiplication works very similarly so if you have two vectors let's say of the same size it's size n let's say U and T then in order for me to multiply them together I have to take the U Matrix for instance make it a transpose because its number of columns have to meet match the number of rows of V so I can't just multiply U * V in order for me to take this dot product is

8:54 I have to take U flip it into a row Vector time V and then I can do it component by component here so this is how this works right here right below us here it's here's U flipped on its side and here's V and then you just basically U1 V1 U2 V2 all the components together at them and this is what's called an inner product or a DOT product now the dot product is a really important

9:19 concept it's basically telling you the projection of v on to V or V onto U so if you have two vectors in a space you can always ask how much is this Vector in the direction of the other Vector the dot product or inner product is the critical assessment of that property so here's some properties you if you do this transpose here you transpose V is equivalent to V transpose U you also have this property here

9:50 you can take the transpose Time Each of times each of these vectors individually in atom and if you have a constant here the constant can come just out or the constant can be folded back into front of V so it doesn't matter how you do that multiplication with a constant just a scaling factor that happens now inner products themselves are going to be very important for us this is again related to projections and so we have some properties here so an

10:20 inner product oftentimes is represented by with parentheses U comma V so U comma V is just it's given here it's just the transpose of U * V and often times the real inner products defined as U transpose time V complex conjugate so this is really important the second term gets complex conjugated okay so if you are working with complex numbers always keep that in mind that the inner product the second one has has the the

10:50 conjugation associated with it now here's some properties that we have one is that UV is equal to this conjugate V where you sitch u and v a scaler can come out of the inner product if the scaler comes out of the inner product in the second term remember comple complex conjugating it comes out as complex conjugate and then you can take the inner product of a sum of vectors as the inner product of each Vector individually so in products are really

11:18 important because they're really talking to us about projection into spaces so if I have two vectors then if I want to understand how one how much of one VOR Vector is in the other direction of the other Vector I just take its inner product and that tells me that exactly Vector magnitudes what's the length of a vector the vector magnitude is just basically the inner product of the vector with itself square root so it's right here so it's u.u if I want to

11:48 find the length of a vector I take the inner product with itself in other words it's pointing in the right direction and the square root there is basically this is like a Pythagorean theorem type result which is you take theare square of the components square root which is just like when you do Pythagorean theorem C is equal to a s + b s square root which are let's say in a triangle and this is a generalization of that

12:10 comp of of that concept here which is the square of all the components square root there's also a second really important concept here which is called orthogonality if two vectors their inner product is zero it means they are orthogonal that means there is no projection of One Direction into the other so if you take that inner product and you find it zero that means those two directions those two vectors are orthogonal to each other so what does

12:42 orthogonality mean we're used to orthogonality in terms of our coordinate systems here's a coordinate system XYZ so if I have a point up here in space right this point here then I can say well you go out certain amount in X certain amount in y certain amount in Z to get to this point right so that's where this point is now if I want to take this Vector that defines this point and I want to figure out how much is in

13:05 the X Direction all I have to do is Project it onto the unit Vector in the X direction if I want to find out how much is in the y direction I project it I take an inner product with the unit Vector in the y direction and similarly in the z z direction if I want how much is in the Z Direction I just take the inner product with that unit normal in the Z Direction so in fact here are the unit

13:27 normal vectors so this point can be represented in some sense as a sum of the X the Y and the Z directions together all all together and these are the unit normals that would go along with it right here these x dot X hat y hat Z hat these are unit vectors generically we're going to work in more than three dimensions so this concept of unit vectors we're going to take it to n Dimensions so if I work in

13:55 an N dimensional space there's going to be n of these directions right here here okay so these directions these end directions are going to be where I'm working in and I want to think about having a coordinate system or an orthogonal basis in this end directions so just to show you about orthogonality we always think about these vectors as being orthogonal so let's take an inner product the inner product of the X hat Direction and the

14:22 yck direction is right here it's explicitly worked out right so here it is the X direction is one z0 I've laid it on its and I'm taking the inner product with 0 one0 so when I do this Matrix multiply remember I have you know this one * 0 0 * 1 0 * 0 so all of these and I add them together so it's 0 plus 0 plus 0 which is equal Zer in other words the inner

14:47 product is zero between these two vectors which means these two vectors are orthogonal with this idea of orthogonality in mind right orthogonality typically for us defines a coordinate system it also defines this concept called linear dependence or linear Independence so suppose I have linear dependence here this idea is if I have n vectors in an N dimensional

15:17 space then if there exists some nonzero C which satisfy this then two of these vectors or some of these vectors are kind of a sense going the same direction in other words if I want this to add to zero and if they're really orthogonal to each other there's no way to make this zero unless I make the c0o okay so that's called linearly independent however if I can add these so that some of these vectors can cancel out another that

15:47 means they're all going they're actually in the same direction that means I can represent it with other vectors which means the C ofj aren't zero and they're called linearly dependent so I don't have an n-dimensional orthogonal coordinate system in this case here when I have a linear dependent system typically what we want to work with are linearly independent systems which means X1 X2 all the way to X of n are in some sense are all in different directions they

16:16 don't have to necessarily be orthogonal to each other of course we would love them to be orthonormal vectors which we will try to construct but if they're not orthonormal we can still have a basis of n linearly in directions which allow us to represent any Vector in that n-dimensional space some other Concepts regarding the inverse and the identity the identity I which is a bolded eye here it is is

16:47 essentially ones down the diagonals zero everywhere else so it's a big Matrix we down the middle it's ones and then everywhere else is zeros and if you multiply a matrix by I it's the same as I * a matrix is equal to a so this property holds here that we have here so one way to think about it is I and its inverses I also we also have this property here an inverse of a matrix

17:17 which is a what is the Matrix B so that when I multiply by a I get out the identity and that Matrix B is called often called a inverse a 1 which is the inverse It undoes Whatever A is doing so what a is If I multiply a minus one I get down back to the identity that is called the inverse and this exists as long as the determinant of the Matrix a is not zero so it's non singular so hopefully these

17:49 are just review pieces from linear algebra but I wanted to go over them so that you have this all all these Concepts in your mind before we start really doing a lot of computing by way speaking of the determinant how do we compute the determinant for 2 by tws or 3x3 here's how you do this the determinant is given by this formula here for a 2X two and this formula here for a 3X3 and so hopefully you have have

18:15 learned this from linear algebra typically when we compute determinants we would do determinants computationally so we'd use a python code to do this and we'll but here is if you do it by hand this is how you would do it by hand okay so let's assume if I'm going to solve a problem like ax equal to B then I want to take a and let's suppose its determinant is not zero if the

18:45 determinant is not zero that means it has an inverse which means I could take the inverse on both sides of ax equal to B and A minus one a is just one it's the identity so I get X on this side and on the other side get a - 1 * B so in other words if I have this inverse I can hit it against B and take that essentially Matrix multiply and then I will get the solution X okay so

19:12 this is one way to solve ax equal to B in practice we will never solve ax equal to B this way Computing this inverse is actually kind of an unstable operation so we're going to find other ways to solve axal B and these will be in subsequent lectures how we solve this a a x equal to B so if the determinant by the way just just to capitalize finish off here if the determinant of of a matrix is zero

19:39 then you have then you you have linearly dependent Solutions and you cannot represent the solution if the determinant is not zero then in fact you can represent linearly the solution with linear independent Solutions okay so that kind of finishes up this section most what this was aimed at was to give you a reminder of some of the Matrix and Vector properties that we have for multiplication and some of their basic mathematical properties that we need in order for to use them in our

20:12 computations what we're going to get into next is actually Sol solution techniques for ax that are computational and it's important we know some of the Frameworks for solving all of these things as we move forward