Steps of PCA

Loading...

Из курса от партнера Imperial College London

Mathematics for Machine Learning: PCA

803 оценки

Imperial College London

Mathematics for Machine Learning: PCA

803 оценки

Курс 3 из 3 — Specialization Mathematics for Machine Learning

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction. At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge. The lectures, examples and exercises require: 1. Some ability of abstract thinking 2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis) 3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization) 4. Basic knowledge in python programming and numpy Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms.

Из урока

Principal Component Analysis

We can think of dimensionality reduction as a way of compressing data with some loss, similar to jpg or mp3. Principal Component Analysis (PCA) is one of the most fundamental dimensionality reduction techniques that are used in machine learning. In this module, we use the results from the first three modules of this course and derive PCA from a geometric point of view. Within this course, this module is the most challenging one, and we will go through an explicit derivation of PCA plus some coding exercises that will make us a proficient user of PCA.

Steps of PCA4:07

PCA in high dimensions5:48

Other interpretations of PCA (optional)7:42

Summary of this module0:42

This was the course on PCA0:56

Познакомьтесь с преподавателями

Marc P. Deisenroth
Lecturer in Statistical Machine Learning
Department of Computing

[MUSIC]

In this video, we'll go through the individual steps of PCA.

But before we do this, let me make two comments.

When we derive PCA, we make the assumption that our data is centered,

that means it has mean 0.

This assumption is not necessarily right for PCA,

and we would have come to the same result, but

subtracting the mean from the data can avoid numerical difficulties.

Assume the values of our data are centered around ten to the eight.

Then computing the data covariance matrix requires us to multiply huge numbers,

which results in numerical instabilities.

An additional second step that is normally recommended,

after subtracting the mean, is to divide every dimension of the centered data

by the corresponding standard deviation.

This makes the data unit-free and guarantees that the variance

of the data in every dimension is 1, but it leaves the correlations intact.

Let's have a look at an example.

Clearly, this data spreads much more in one dimensions than the other dimension

and the best projection of PCA is clear.

However, there's a problem with this data set.

The two dimensions of the data set are both distances, but

one is measured in centimeters and the other one in meters.

The one measured in centimeters naturally varies much more than the other one.

When we divide each dimension of the data set by the corresponding standard

deviation, we get rid of the units and

make sure that the variance in each dimension is 1.

When we look at the principal subspace of this normalised data set, we can now

see that there is actually quite a strong correlation between these two dimensions.

And the principal axis have changed.

But now let's go through PCA step by step and we'll have a running example.

We're given a two dimensional data set, and

we want to use PCA to project it onto a one dimensional subspace.

The first thing that we do is to subtract the mean.

The data is now centered.

Next, we divide by the standard deviation.

Now the data is unit-free and

it has variance one along each axis, which is indicated by these two arrows.

But keep in mind that the correlations are still intact.

Third, we compute the data covariance matrix and its eigenvalues and

corresponding eigenvectors.

The eigenvectors are scaled by the magnitude of the corresponding eigenvalue

in this picture.

The longer vector spends the principal subspace, let's call it u.

And in the last step,

we can project any data point x star onto the principal subspace.

To get this right, we need to normalise x star using the mean and standard deviation

of the data set that we used to compute the data covariance matrix.

So we're going to have a new x star, and

the new x star is going to be the old x star minus the mean of

the data set, divided by the standard deviation.

And we do this for every dimension in x star.

Now we can get the projection of x star as x star tilde,

or the projection of x star onto the principal subspace u

as B times B transpose, times x star.

Where B is the matrix that contains the eigenvectors that belong to the largest

eigenvalues as columns.

And B transposed times x star are the coordinates of

the projection with respect to the basis of the principal subspace.

In this video, we went through the steps of PCA.

First, we subtract the mean from the data and

send it at zero to avoid numerical problems.

Second, we divide by the standard deviation to make the data unit-free.

Third, we compute the eigenvalues and eigen vectors of the data covariance matrix.

And finally, we can project any data point onto the principal subspace

that is spanned by the eigenvectors that belong to the largest eigenvalues.

[MUSIC]

Ознакомьтесь с нашим каталогом

Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.

Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.

© Coursera Inc., 2019 Все права защищены.

Загрузить из App Store

Загрузить в Google Play