0:00

So one issue with either the SVD or

Â principal components analysis is miss, is always missing values.

Â So real data will typically have missing values, and the

Â problem is that if you try to run the SVD

Â on a data set that doesn't have or that has

Â some missing values, like you see that I've created here.

Â You can see that you get an error.

Â You just can't run it on a data set that has missing values.

Â So you need to do something about the missing values.

Â Before you run an SVD or a PCA.

Â 0:26

So, one possibility and there are many others though, is the,

Â use the impute package which is available from the bioconductor project.

Â and, and just impute the missing datas, sorry the missing data points and so

Â that you can have a value there, and then you can run your SVD.

Â And so this approach you.

Â This code here uses the impute.knn function which takes a, a missing row or

Â missing values in a row, and imputes it by the k nearest neighbors to

Â that row.

Â So if k, for example, is five, then it will

Â take the five rows that are closest to the row with

Â the missing data, and then impute the data in that

Â missing row with the kind of average of the other five.

Â And so, once we've imputed the data with

Â this impute.knn function we can run the svd.

Â You can see, it runs without error.

Â And we can, we can kind of plot the first singular

Â vectors from each of them.

Â And so you can see that the on the left hand side, I've got the,

Â the data, the, kind of the, the

Â first singular vector from the original data matrix,

Â and the second, on the right-hand side,

Â I've got the, server singular vector from the

Â data matrix that was in, that was kind of where the missing data was imputed.

Â Now, you can see that they're roughly similar.

Â They're not exactly the same, but the imputation

Â didn't seem to have a major effect on

Â the on the running of the svd. So

Â 1:46

this is a final example here, it's just kind of an interesting example.

Â I just want to show how you can take an

Â actual image, which is represented as a matrix, and

Â kind of and, and, and kind of develop a

Â lower dimensional or lower rank representation of this actual image.

Â So here's a picture of a face.

Â It's a relatively low resolution picture of a face, but you can see that there

Â is a, you know, a nose and ears and two eyes and a mouth there.

Â And so what we're going to do is we're going to run the svd

Â on this face data and look at the variance explained.

Â And so you can see that the first

Â singular vector explains about 40% of the variation.

Â And then the second is about say 20 some percent.

Â And then the third one is about maybe 15%.

Â And so if you look at say the first five to ten singular

Â vectors they capture pretty much all of the variation in the data set.

Â And so we can, we can see, we can actually

Â look at the image that's generated by say the first

Â singular vector, or the first five, or the first ten.

Â 2:53

that is, that uses fewer components than the original data set too.

Â So here I'm creating one that just uses

Â the first principle component the first singular vector.

Â I'm using one that takes the first five altogether.

Â And then another one that takes the first ten.

Â And, and so we can take a look at what this approximation looks like.

Â So the first image here, which is all the way

Â on the left here, just uses a single singular vector.

Â And you can see that it's not a very good, it's not a pretty picture so to speak.

Â There's not really a face there.

Â There's not much you can see.

Â But it's asking a lot to represent an entire image using just a single vector.

Â So, if we move on to the second one from the left, you

Â can see that basically most of the, of the key features are already there.

Â So this uses the first five singular vectors.

Â And you can see that clearly there's a face.

Â There's two eyes, a nose, and a mouth, and two ears.

Â 3:49

If you move on to the next picture, which is

Â letter C, here, you can see that it's a little bit,

Â kind of has a little bit more definition.

Â This is using the first ten pixel, singular vectors.

Â And, but it's not very different from the second one, which uses, only used five.

Â And then the very last one here on the right is the original data set.

Â So, you can see that if you go, if you use

Â just a few, a singular vector, maybe up to five or ten.

Â You can get a reasonable approximation of this face without

Â having to kind of store all of the original data.

Â So, this is an example

Â of a kind of data compression type of approach.

Â that, that, the singular value decomposition can can generate.

Â Now data compression and kind of statistical summaries

Â are kind of two sides of the same coin.

Â And so if you want to summarize the data set with the, with a,

Â with a smaller number of features the

Â singular value decomposition is also useful for that.

Â 4:44

So just a couple of notes and further resources

Â for the singular value decomposition and principle component analysis.

Â One of the issues is that the scale of your data matters.

Â So if you have, for example, it's common to

Â measure lot's of different variables that come on different scales.

Â And that can cause a problem, because if one

Â variable is much larger than another variable, just because

Â the unit's so different that will tend to drive

Â this, the principle components analysis over the singular vectors.

Â And so that may not be particularly meaningful to you.

Â And so you want to look at the, see that the, kind of the

Â scale of the different columns or rows are roughly comparable to each other.

Â 5:21

the, as we saw in the, one of the, in the example with the

Â two different patterns, the principle components and

Â the singular vectors may mix together real patterns.

Â And so, the patterns that you see may not represent the kind of

Â separable patterns but they may be

Â patterns that may, that are mixed together.

Â The singular value decomposition can be computationally intensive if you have a

Â very large matrix so that's one, that's something to keep in mind.

Â We use relatively small matrices here, but

Â of course computing power is getting ever more

Â 5:48

powerful and and there there are some highly optimized and

Â specialized matrix libraries out there

Â for computing the singular value decomposition.

Â And so this can be done on, on lots of kind of practical problems.

Â 6:04

And so here are a couple links to kind of further resources

Â for what, how to use principle

Â components analysis in the singular value composition.

Â And, and also there are other kind

Â of approaches that are similar to this.

Â But are a kind of different in many of the details.

Â You may hear about these approaches.

Â Things like factor analysis, independent

Â components analysis and latent semantic analysis.

Â And these are worth exploring, but are related to the, kind

Â of the basic ideas behind principle

Â components analysis and singular valued decomposition.

Â Which is that you want to find the kind

Â of lower dimensional representation that explains most of the variation

Â in the data that you see.

Â