So, this is a very high level overview of how we apply the PCA technique. We started by centering the dataset. So, essentially for each attribute in your dataset, you subtract the mean, and you zero center of the data, that's your first step. Then, we compute the covariance matrix, we find the eigenvectors, and eigenvalues of the covariance matrix, and I will talk about this in details just after I do a high level them of PCA. Then, these eigenvectors will become the principal components, these lines that actually give you the direction of greatest variance. The eigenvalues of the covariance matrix provide the explained variance. So, basically these eigenvalues tell you how much of the variance the individual component explains. So, ideally once you have found the eigenvectors and eigenvalues, you will sort them by eigenvalues decreasingly. Then, you will pick the first eigenvector which will be your first principal component. That's the direction that preserves most of the variance, and then you keep going until you reach the number of required dimensions. Now, if you are not familiar with covariance matrix, eigenvectors, eigenvalues, and so on. I will explain them very briefly, but for the time being I would like to treat PCA as a black box, and just see how it works. Once you have found your principal components, then you will use them to select new dimensions, and project the data, and end up with new dataset. So, let's go to Notebook now. So, here is my Notebook in Watson Studio, let me open it. I start by importing certain libraries that I need. Also, I defined one function called corvaris which essentially generates correlated variables with some added Gaussian noise, and I use this quite often to generate artificial data that has some kind of relationship. So, I will use this function to generate some data, and then we will see how we can use PCA to reduce the dimensionality of the data. Okay. So, here is my Notebook. Let me start by defining the function. So, run the cell, and then I would generate some artificial data. The matrix I end up with looks like this. If you look at the parameters of my function, so I'm generating data in the interval 2-4. That's my first attribute in this matrix A. then, the step is 0.2. So, I generate a range of data, and then I have a function that I use to link the second attribute to the first attribute, and in this case my function is two times sine of x sine of the first attribute. I also inject some random Gaussian noise with sigma of two. So, this is actually my data, and as we've just mentioned, what we'll do now is first center the data. So, if I plot the data as it is, that's my original dataset. You will notice that my zero, zero point is somewhere here. So, my data are clustered to the right of the zero, zero of the coordinate system. What I will do next is I will just subtract the mean from each attribute. If you look at the dataset now, you will see that they are centered around zero. If I plot them again, you'll see that I keep essentially the same shape of the data. It's just that we have transposed the data, we moved it to the zero, zero point in the coordinate system, and that's important for PCA to work properly. So, not only have my data centered, I will use a function from Numphy to compute the eigenvectors, and eigenvalues. Again, I won't treat PCA for the time being, as a black box, but very shortly we will go into the details of how we can compute this manually. So, let me run the cell. Now, they have been computed. If I look at the eigenvectors, I get two vectors because the dimensionality of my data is two, I end up having two vectors. This is the first one, this is the second one. Then, the eigenvalues, if I look at the eigenvalues are 6.1 and 1.6. So, my first vector explains most of the variants that the eigenvalue for this vector, and the second vector explains what's remaining of the variance. What I can do now is I can generate some data using the slope of my first vector, and plot this. So, you can see the direction of the variance in the dataset, and that's exactly this. If you are interested in the second vector, we can actually also plot the second vector, and see what direction the second vector gives us. As expected, it's 90 degrees to the first principal component. So, this direction explains what's left of the variance. The final thing we can do is actually using the first principal component to project our datapoints. I'm using the second component, but yeah, you get the idea. If I use the second component, I'll project my datapoints here. If I use the first one, I'll project them alongside the long axis that preserve most of the variance. So, that's the idea. That's how we use PCA out of the box using Numphy. Now, let's get back to the definition, and look into details, and see if we can actually implement PCA manually.
