>> In this module we'll review vectors. I will go over definitions of vectors,
what are row vectors, column vectors. We're going to define linear independence,
bases, transposes, inner products, lengths of vectors or norms.
All of these concepts are going to be needed for doing some of the elementary
linear algebra that's needed in this course.
What's a vector? It's just a collection of real numbers.
You can collect them, and put them in a row or you can put them in a column.
So here is an example of a row vector. Where I've put everything as a row here's
an example of a column vector and if you notice carefully both of these vectors
have n components. So we will call a vector to have n
components if it consists of n real numbers.
A row vector or column vector and we will denote any of these vectors by the symbol
r to the n. R to the n means that every component is a
real number and they, there are n of them in the place.
To just fix ideas it might be easier to think in terms of R2 which means these are
vectors with just two components and it's easiest to think of these vectors as
belonging to a plane. And we're used to calling this plane by
the x axis and the y axis. So think of x axis being one of the
components of the vector. Y axis being another component of the
vector. So, let me just give you some examples of
vectors that we going to use later on in this module.
So, here's one vector. It's x component or the first component is
going to be one and the second component is say going to be equal to two.
So that the vector will be this vector. At the end of the dot the dot corresponds
to a y axis of two and x axis of one. So was labeled this as V.
This will simply be the vector one two. The first component equal to one in the
second component equal to two. Here's another example.
The x axis, or the first component equal to 4 and y axis equal to 1.
So that's that point, and I'm going to be connecting it with this vector.
And in the rest of this module, I'm going to referring to this vector as w.
It has 4 in the first component, and 1 in the second component.
By default in this course unless otherwise specified we will assume that all vectors
are column vectors, that is all the components have been arranged as a column.
So we now know what a vector is, they are collections of real numbers without, by
default it's going to be a column vector. Now we want to understand what can we
represent using these vectors, what happens with them and so on So the first
thing that we want to do is multiply these vectors with real numbers and add them up.
So v1 and v2 are vectors. And I want to multiply them by a real
number, alpha 1. And a real number, alpha 2.
And then get a final vector w. So, to give you an example, here is my
vector, v1. One, here is my vector V two.
And just to fix ideas, each of these V one and V two actually lives in R to the
three. Why R to the three?
Every component is a real number, and there are three components.
So the three refers to the fact that there are three components.
I'm going to multiply them by two real numbers, so two is going to play the role
of alpha one. 4 is going to play the role of alpha 2.
And what do I do? I take these real numbers, multiply them
component by component and add them up. So, in order to get the first component, I
take the 2, multiply it to the 1, take the 4, multiply it to the 0.
I get 2 times 1 plus 4 times 0 equals 2. That gives me the first component.
If I want to go to the second component then I take the 2 and multiply it to the
second component 1. Take the 4 multiply it to the second
component which is also 1, 2 times 1 is 2, 4 times 1 is 4 add it up together you get
6. Want to go to the third component the same
thing, 2 times 0. 4 times z, 1, 2 times 0 is 0.
4 times 1 i 4. You get a total component 4.
So when you write a vector w as a combination of vectors v1 and v2, we're
going to say that this Vector w is linearly dependent on v1 and v2.
Why the linear? Because I'm multiplying by a real number,
and adding them up. Other words are, w's a linear combination
of v1 and v2. Get another word.
W belongs to the linear span of v1 and v2. All of these 3 things mean the same.
Linear dependency, linear combination, linear span.
Now, we'll, in the next set of ideas, we'll need a concept of linear
independence. When can we say that we cannot write a
vector as a linear combination of other vectors.
To fix ideas, let's again go back to r2, and I'm going to use my favorite two
vectors, v and w. So here's my vector v and here's my vector
w. For now we don't really need what the
components are so I'm not going to bother with that.
Now we want to understand what does a linear span mean, what can I, what kind of
vectors can I generate by scaling the w? What can I do by multiplying w by areal
number? And you can convince yourself very easily
That all the vector that you can get, are going to be on that straight line.
When you scale it up you get, by a positive number you get up the line if you
multiply by a negative number you go down the line.
Similarly all the vector that can be gotten as linear combinations of v rely on
this straight line. Now we want to ask ourself can I represent
v as A multiple of w. Clearly that's not true because I can't, I
can't, when I scale the w I get points on this line, v doesn't belong to that line
so I can't do anything about that. Similarly, w is not written as a linear
combination of v and therefore we'll say that v and w are linearly independent.
Let's throw in another vector now, x. It's again a vector in r 2 and I want to
ask myself, is this linearly independent of v and w?
Is it possible that x can not be written as a linear combination of v and w?
Again it's very easy to convince yourself that if you just draw a line parallel to
v, you can write, your vector x as a combination of vector that starts from the
origin, it's aligned to the vector w and comes up to this point.
So therefore, this vector is some alpha 1 times w.
This vector here is, is parallel to the vector v.
So I can write it at sum scale multiple alpha 2 times v.
And because, now, x is the sum of this vector and that vector, you end up getting
that x is actually equal to alpha 1 times w, plus alpha 2 times v which means that x
is linearly dependent, LD just for short, linearly dependent on v and w.
In fact we'll see in the next slide that in R2 if I give you any 2 linearly
independent vectors you can write any other vector as a linear combination of
these 2. And that set of vectors, say these two, v
and w, would actually be called a basis. So that's the next concept that I want to
learn about vectors. A basis is a linearly independent set of
vectors that spans the entire space. Any basis for Rn, meaning a vector which
has n components, has exactly n elements. So basis for Rn has exactly n elements.
In the last page I showed you an example of R2, and this should have just two
elements. V and W are linearly independent, there
were two of them, and therefore I know, that this must be a basis.
Now it will turn out that its much easier to think in terms of a standard basis.
What's a standard basis is shown here. Its a collection of vectors such that they
have only one in one of the components and all of the other components are equal to
zero. So the vector E1 will have 1 in the first
component, the vector E2 is going to have a 1 in the second component, and the
vector En is going to have a 1 in the last, or the nth, component.
Now if you give me any vector W which belongs to Rn meaning that it has n
components, I can very easily write it as a linear combination of E1 through En.
Why, because in each of these vectors is exactly one element that's not zero.
So if I want to get the first element of w, right?
I have to, say, take it to be w1 times e1, because everything else has zero
contribution. If I want the second component, right, I
have to take w2 times e2, nnd so on up to wn times en.
And therefore, this basis. I can split any vector w as a linear
combination very easily. And we will see that, in practice, these
terms ought to be very convenient. Again, to put it in perspective.
Here's r2. Here is my x asix.
Here is my y axis. And when I showed you in the first slide,
x axis refers to the first component. And y axis refers to the second component.
So the vector e1 is just this one. It has one component one in the x
direction and zero in the y direction. E2 is this one, it has a component one in
the y direction and zero in every other direction and it's very easy if I take a
vector x, all I have to do is drop it down here and This, this length down here on
the x-axis is x1, on the y-axis is x2. And it's a very easy way to combine.
But any other set of linearly independent vectors, two of them will make sure that
this is going to be a basis. So, again, this was v, and this was w from
the last page. And v and w is also a basis.
This is also a basis. And E1, E2, which is are these 2 special
ones, are also a basis. The second basis is special.
And we'll just call it the standard basis, because it turns out it's very convienent
to work in terms of this basis. Alright.
So, so far what do we know? We know what are vectors.
We know linear dependence, we know linear independence.
We know that if I am a vector in Rn, meaning it has n components, I can find a
basis of n linearly independent vectors, such that every vector can be written as a
combination of these. So what's the next concept?
The next concept that we want to learn, is that of a length of a vector.
Vector. So let's start with the basics.
And then we'll generalize is to what I want here.
So, if I give you r2. And let's take a vector which has the x
component equal to, let's say, 4. And the y component equal to, let's say.
So this vector, has a representation four, three.
Then our high school trigonometry tells us that the length of this vector is nothing
but 4 squared plus 3 squared. So I'm taking the x-axis and squaring it.
I'm taking the y-axis and squaring it. Square root.
So that gives me 16. Plus 9 square root which gives me 25
square root equaled to 5. So that's nice.
I have a, I have a definition of length. What does this definition of length
satisfy? It satisfies several properties.
It satisfies that the length of any vector is always great than equal to 0.
If the length of a vector is equal to zero, then that vector itself must be
zero. So you cannot have a vector which is not
equal to zero who's length is equal to zero.
And the intuition follows from this triangular expression as well.
If I scale a vector by an amount alpha it doesn't matter whether I scale it
positively or negatively. The length just gets scaled by the
absolute value. So, here's the idea here's the value of
vector v if I double it. >> I get this vector if I multiply it by
minus 2 I get that vector, the length of this vector and the length of that vector
is the same. The direction has changed, but the length
remains the same. That's what this absolute value does.
The third one here. Tells you the relationship between lengths
of additional vectors and the original lengths, and this is known as a triangle
inequality. The way you remember this geometrically is
that if you've got one vector here, you've got another vector there, the sum of the
vectors is this one, the length, this length is always going to be, let's call
it l. Three is always going to be lessthan equal
to L1 plus L2. This is the basic geometic fact L3 has to
be less than equal to L1 plus L2. And this particular fact is encapsulated
in this little >> Statement over here, and that's why it's also called the tirangle
inequality. Now mathematicians have looked at this
concept of length, this particular concept of length that I've been talking about
which is take each of the components, square it, add it up, take the square root
and now they're calling it the I2 norm. The 2 stands for the fact that I'm
squaring it and take the square root and you'll notice that the properties that I
want. The fact that the length is greater or
equal to zero. The fact that triangle inequality holds.
The fact that if I scale things remain the same is true for other definitions of
length. Now, since I am taping this in New York,
I'm going to try to tell you about a particular notion of language, which is
called the L1 norm, otherwise known as the Manhattan distance.
I'm sitting at a particular point here, I want to travel to another point, I can
only go In blocks north and south or east and west, and I want to figure out how
much do I need to walk in order to get from this point to that point.
So I have to walk from this point to this point and then walk from that point to
that point. So this distance is 4, that distance is 3,
so the L1 distance, or the L1 length, or the Manhattan distance is 4 plus 37.
The L two distance is five because I can sort of cut across.
The L one distance is seven. L one also satisfies all the properties of
a norm or a length and therefor it's sometimes convenient to encapsulate all of
them as just norms and lengths. For the purposes of this course we will
mostly be focusing on L two. But I just wanted you to know, that there
are other definitions of length, that are interesting, and sometimes become
important in applications. Alright, now we know vectors, we know
linear combination, we know lengths. Now we want to go to the next concept,
which is that of an angle. And in order to get to an angle, I have to
introduce this idea of an inner product or a dot product.
So the dot product of any two vectors v and w, so v and w are vectors in Rn.
They have n components. So the dot product between v and w is
simply going to be taking the ith component of v, taking the ith component
of w, multiplying them together, and adding them up.
For those of you who are experts in Excel, this is nothing but sum product, take the
product of the components and add them up. The sum product function in excel is
nothing but an inner product or dot product.
If you review back, into the last slide we had defined the L 2 norm there to be every
component's squared square root and that can now be written that length L 2 norm of
the vector is simply take the vector v, take its dot product.
So v, dot product with v, and take the square root.
So now we want to understand an angle between these two vectors, so to do that,
here's a picture. Here's my w, here's my v, here's an angle
theta between them. So in order to understand how the inner
product relates to the angle, the length of this vector is The length we, the
length of this vector is w. So that inner product of v and w, simply
is, take a, take the component of v along w and multiply to the length of w.
So if you drop down an orthogonal point over here, this component.
Is exactly equal to the norm, or the length, of v cosine of theta, and that I'm
going to multiply with the length of w, and that should give me v dot w, which
ends up giving me the cosine of theta Is exactly equal to vw divided by the norm of
v, and the norm of w, or the length of v, and the length of w.
And, to emphasize the fact that these are all 2 norms.
I'm just going to put a 2 there. All of that is encapsulated in this slide.
Cosine of theta is v.w divided by v and w. And this is true, not just in our 2, but
in our n. You an define angles in rn.
I'm going to show you in the next module that v dot w is actually a combination of
two operations, the transpose operation and the matrix multiplication operation.
But that has to wait until we get to the module of matrixes.
So that pretty much brings us to the end of the introduction to vectors.
This is all we need to do in terms of vectors, and this is all we're going to be
using in the course. In the next prerequisite module, the next,
the module that comes up next, we're going to review concepts about matrixing