>> In this module we'll review vectors. I will go over definitions of vectors, what are row vectors, column vectors. We're going to define linear independence, bases, transposes, inner products, lengths of vectors or norms. All of these concepts are going to be needed for doing some of the elementary linear algebra that's needed in this course. What's a vector? It's just a collection of real numbers. You can collect them, and put them in a row or you can put them in a column. So here is an example of a row vector. Where I've put everything as a row here's an example of a column vector and if you notice carefully both of these vectors have n components. So we will call a vector to have n components if it consists of n real numbers. A row vector or column vector and we will denote any of these vectors by the symbol r to the n. R to the n means that every component is a real number and they, there are n of them in the place. To just fix ideas it might be easier to think in terms of R2 which means these are vectors with just two components and it's easiest to think of these vectors as belonging to a plane. And we're used to calling this plane by the x axis and the y axis. So think of x axis being one of the components of the vector. Y axis being another component of the vector. So, let me just give you some examples of vectors that we going to use later on in this module. So, here's one vector. It's x component or the first component is going to be one and the second component is say going to be equal to two. So that the vector will be this vector. At the end of the dot the dot corresponds to a y axis of two and x axis of one. So was labeled this as V. This will simply be the vector one two. The first component equal to one in the second component equal to two. Here's another example. The x axis, or the first component equal to 4 and y axis equal to 1. So that's that point, and I'm going to be connecting it with this vector. And in the rest of this module, I'm going to referring to this vector as w. It has 4 in the first component, and 1 in the second component. By default in this course unless otherwise specified we will assume that all vectors are column vectors, that is all the components have been arranged as a column. So we now know what a vector is, they are collections of real numbers without, by default it's going to be a column vector. Now we want to understand what can we represent using these vectors, what happens with them and so on So the first thing that we want to do is multiply these vectors with real numbers and add them up. So v1 and v2 are vectors. And I want to multiply them by a real number, alpha 1. And a real number, alpha 2. And then get a final vector w. So, to give you an example, here is my vector, v1. One, here is my vector V two. And just to fix ideas, each of these V one and V two actually lives in R to the three. Why R to the three? Every component is a real number, and there are three components. So the three refers to the fact that there are three components. I'm going to multiply them by two real numbers, so two is going to play the role of alpha one. 4 is going to play the role of alpha 2. And what do I do? I take these real numbers, multiply them component by component and add them up. So, in order to get the first component, I take the 2, multiply it to the 1, take the 4, multiply it to the 0. I get 2 times 1 plus 4 times 0 equals 2. That gives me the first component. If I want to go to the second component then I take the 2 and multiply it to the second component 1. Take the 4 multiply it to the second component which is also 1, 2 times 1 is 2, 4 times 1 is 4 add it up together you get 6. Want to go to the third component the same thing, 2 times 0. 4 times z, 1, 2 times 0 is 0. 4 times 1 i 4. You get a total component 4. So when you write a vector w as a combination of vectors v1 and v2, we're going to say that this Vector w is linearly dependent on v1 and v2. Why the linear? Because I'm multiplying by a real number, and adding them up. Other words are, w's a linear combination of v1 and v2. Get another word. W belongs to the linear span of v1 and v2. All of these 3 things mean the same. Linear dependency, linear combination, linear span. Now, we'll, in the next set of ideas, we'll need a concept of linear independence. When can we say that we cannot write a vector as a linear combination of other vectors. To fix ideas, let's again go back to r2, and I'm going to use my favorite two vectors, v and w. So here's my vector v and here's my vector w. For now we don't really need what the components are so I'm not going to bother with that. Now we want to understand what does a linear span mean, what can I, what kind of vectors can I generate by scaling the w? What can I do by multiplying w by areal number? And you can convince yourself very easily That all the vector that you can get, are going to be on that straight line. When you scale it up you get, by a positive number you get up the line if you multiply by a negative number you go down the line. Similarly all the vector that can be gotten as linear combinations of v rely on this straight line. Now we want to ask ourself can I represent v as A multiple of w. Clearly that's not true because I can't, I can't, when I scale the w I get points on this line, v doesn't belong to that line so I can't do anything about that. Similarly, w is not written as a linear combination of v and therefore we'll say that v and w are linearly independent. Let's throw in another vector now, x. It's again a vector in r 2 and I want to ask myself, is this linearly independent of v and w? Is it possible that x can not be written as a linear combination of v and w? Again it's very easy to convince yourself that if you just draw a line parallel to v, you can write, your vector x as a combination of vector that starts from the origin, it's aligned to the vector w and comes up to this point. So therefore, this vector is some alpha 1 times w. This vector here is, is parallel to the vector v. So I can write it at sum scale multiple alpha 2 times v. And because, now, x is the sum of this vector and that vector, you end up getting that x is actually equal to alpha 1 times w, plus alpha 2 times v which means that x is linearly dependent, LD just for short, linearly dependent on v and w. In fact we'll see in the next slide that in R2 if I give you any 2 linearly independent vectors you can write any other vector as a linear combination of these 2. And that set of vectors, say these two, v and w, would actually be called a basis. So that's the next concept that I want to learn about vectors. A basis is a linearly independent set of vectors that spans the entire space. Any basis for Rn, meaning a vector which has n components, has exactly n elements. So basis for Rn has exactly n elements. In the last page I showed you an example of R2, and this should have just two elements. V and W are linearly independent, there were two of them, and therefore I know, that this must be a basis. Now it will turn out that its much easier to think in terms of a standard basis. What's a standard basis is shown here. Its a collection of vectors such that they have only one in one of the components and all of the other components are equal to zero. So the vector E1 will have 1 in the first component, the vector E2 is going to have a 1 in the second component, and the vector En is going to have a 1 in the last, or the nth, component. Now if you give me any vector W which belongs to Rn meaning that it has n components, I can very easily write it as a linear combination of E1 through En. Why, because in each of these vectors is exactly one element that's not zero. So if I want to get the first element of w, right? I have to, say, take it to be w1 times e1, because everything else has zero contribution. If I want the second component, right, I have to take w2 times e2, nnd so on up to wn times en. And therefore, this basis. I can split any vector w as a linear combination very easily. And we will see that, in practice, these terms ought to be very convenient. Again, to put it in perspective. Here's r2. Here is my x asix. Here is my y axis. And when I showed you in the first slide, x axis refers to the first component. And y axis refers to the second component. So the vector e1 is just this one. It has one component one in the x direction and zero in the y direction. E2 is this one, it has a component one in the y direction and zero in every other direction and it's very easy if I take a vector x, all I have to do is drop it down here and This, this length down here on the x-axis is x1, on the y-axis is x2. And it's a very easy way to combine. But any other set of linearly independent vectors, two of them will make sure that this is going to be a basis. So, again, this was v, and this was w from the last page. And v and w is also a basis. This is also a basis. And E1, E2, which is are these 2 special ones, are also a basis. The second basis is special. And we'll just call it the standard basis, because it turns out it's very convienent to work in terms of this basis. Alright. So, so far what do we know? We know what are vectors. We know linear dependence, we know linear independence. We know that if I am a vector in Rn, meaning it has n components, I can find a basis of n linearly independent vectors, such that every vector can be written as a combination of these. So what's the next concept? The next concept that we want to learn, is that of a length of a vector. Vector. So let's start with the basics. And then we'll generalize is to what I want here. So, if I give you r2. And let's take a vector which has the x component equal to, let's say, 4. And the y component equal to, let's say. So this vector, has a representation four, three. Then our high school trigonometry tells us that the length of this vector is nothing but 4 squared plus 3 squared. So I'm taking the x-axis and squaring it. I'm taking the y-axis and squaring it. Square root. So that gives me 16. Plus 9 square root which gives me 25 square root equaled to 5. So that's nice. I have a, I have a definition of length. What does this definition of length satisfy? It satisfies several properties. It satisfies that the length of any vector is always great than equal to 0. If the length of a vector is equal to zero, then that vector itself must be zero. So you cannot have a vector which is not equal to zero who's length is equal to zero. And the intuition follows from this triangular expression as well. If I scale a vector by an amount alpha it doesn't matter whether I scale it positively or negatively. The length just gets scaled by the absolute value. So, here's the idea here's the value of vector v if I double it. >> I get this vector if I multiply it by minus 2 I get that vector, the length of this vector and the length of that vector is the same. The direction has changed, but the length remains the same. That's what this absolute value does. The third one here. Tells you the relationship between lengths of additional vectors and the original lengths, and this is known as a triangle inequality. The way you remember this geometrically is that if you've got one vector here, you've got another vector there, the sum of the vectors is this one, the length, this length is always going to be, let's call it l. Three is always going to be lessthan equal to L1 plus L2. This is the basic geometic fact L3 has to be less than equal to L1 plus L2. And this particular fact is encapsulated in this little >> Statement over here, and that's why it's also called the tirangle inequality. Now mathematicians have looked at this concept of length, this particular concept of length that I've been talking about which is take each of the components, square it, add it up, take the square root and now they're calling it the I2 norm. The 2 stands for the fact that I'm squaring it and take the square root and you'll notice that the properties that I want. The fact that the length is greater or equal to zero. The fact that triangle inequality holds. The fact that if I scale things remain the same is true for other definitions of length. Now, since I am taping this in New York, I'm going to try to tell you about a particular notion of language, which is called the L1 norm, otherwise known as the Manhattan distance. I'm sitting at a particular point here, I want to travel to another point, I can only go In blocks north and south or east and west, and I want to figure out how much do I need to walk in order to get from this point to that point. So I have to walk from this point to this point and then walk from that point to that point. So this distance is 4, that distance is 3, so the L1 distance, or the L1 length, or the Manhattan distance is 4 plus 37. The L two distance is five because I can sort of cut across. The L one distance is seven. L one also satisfies all the properties of a norm or a length and therefor it's sometimes convenient to encapsulate all of them as just norms and lengths. For the purposes of this course we will mostly be focusing on L two. But I just wanted you to know, that there are other definitions of length, that are interesting, and sometimes become important in applications. Alright, now we know vectors, we know linear combination, we know lengths. Now we want to go to the next concept, which is that of an angle. And in order to get to an angle, I have to introduce this idea of an inner product or a dot product. So the dot product of any two vectors v and w, so v and w are vectors in Rn. They have n components. So the dot product between v and w is simply going to be taking the ith component of v, taking the ith component of w, multiplying them together, and adding them up. For those of you who are experts in Excel, this is nothing but sum product, take the product of the components and add them up. The sum product function in excel is nothing but an inner product or dot product. If you review back, into the last slide we had defined the L 2 norm there to be every component's squared square root and that can now be written that length L 2 norm of the vector is simply take the vector v, take its dot product. So v, dot product with v, and take the square root. So now we want to understand an angle between these two vectors, so to do that, here's a picture. Here's my w, here's my v, here's an angle theta between them. So in order to understand how the inner product relates to the angle, the length of this vector is The length we, the length of this vector is w. So that inner product of v and w, simply is, take a, take the component of v along w and multiply to the length of w. So if you drop down an orthogonal point over here, this component. Is exactly equal to the norm, or the length, of v cosine of theta, and that I'm going to multiply with the length of w, and that should give me v dot w, which ends up giving me the cosine of theta Is exactly equal to vw divided by the norm of v, and the norm of w, or the length of v, and the length of w. And, to emphasize the fact that these are all 2 norms. I'm just going to put a 2 there. All of that is encapsulated in this slide. Cosine of theta is v.w divided by v and w. And this is true, not just in our 2, but in our n. You an define angles in rn. I'm going to show you in the next module that v dot w is actually a combination of two operations, the transpose operation and the matrix multiplication operation. But that has to wait until we get to the module of matrixes. So that pretty much brings us to the end of the introduction to vectors. This is all we need to do in terms of vectors, and this is all we're going to be using in the course. In the next prerequisite module, the next, the module that comes up next, we're going to review concepts about matrixing
