Normal conditional distributions

Loading...

Из курса от партнера Johns Hopkins University

Advanced Linear Models for Data Science 2: Statistical Linear Models

27 оценки

Johns Hopkins University

Advanced Linear Models for Data Science 2: Statistical Linear Models

27 оценки

Welcome to the Advanced Linear Models for Data Science Class 2: Statistical Linear Models. This class is an introduction to least squares from a linear algebraic and mathematical perspective. Before beginning the class make sure that you have the following: - A basic understanding of linear algebra and multivariate calculus. - A basic understanding of statistics and regression models. - At least a little familiarity with proof based mathematics. - Basic knowledge of the R programming language. After taking this course, students will have a firm foundation in a linear algebraic treatment of regression modeling. This will greatly augment applied data scientists' general understanding of regression models.

Из урока

The multivariate normal distribution

In this module, we build up the multivariate and singular normal distribution by starting with iid normals.

Normals and multivariate normals8:33

The singular normal distribution7:53

Normal likelihoods5:22

Normal conditional distributions8:48

Познакомьтесь с преподавателями

Brian Caffo, PhD
Professor, Biostatistics
Bloomberg School of Public Health

Let's talk about the conditional distribution associated with the normal

distribution.

So let me let X,

my vector X be equal to X1 X2.

And so without loss of generality, we're just going to assume that

we want to know what the conditional distribution of X1 given X2 is.

So let me write out the answer first.

First of all, I'm working under the assumption that x as we're

talking about in almost all of the cases from this section of the class.

X is multi variant normal distributed.

So I'm going to write out the variance covariance matrix this way.

Sigma 1 1, sigma 1 2,

sigma 1 2 transposed, and sigma 2 2.

So what I'd like to know is what's the distribution of X1 given X2.

And we know that it's normally distributed,

because we've mentioned earlier that all the conditional distributions are normal.

But what's the variance and covariance?

So, Let me just write out the answers first.

And then I'll show you a nifty derivation of this.

I'll do the derivation just because I like it.

So the expected value of X1 given X2 took the particular value,

little x2 is exactly equal to mu 1, okay?

It will be weird if that wasn't there

plus sigma 1 2 sigma 2 2 inverse X2 minus mu 2.

And then the variance of X1 given that

X2 equals little x2 is then equal to sigma 1 1.

So what we would hope it would be, because that would be easy,

just the same as the marginal variance.

But of course, there's got to be other stuff minus,

sorry, minus sigma 1 2 sigma 2 2 inverse.

You could either write it as sigma 2 1 or sigma 1 2 transpose.

I'd like to write it as sigma 1 2 transpose, okay?

Let's come up with a derivation of this.

And there's a really clever one,

and I'm not sure how someone figured this out but

it's really, I find it really nice.

And in fact it was a student in my class, who showed me this,

because I was deriving it using inverses of partition matrices

which is a lengthy bookkeeping procedure though not hard.

And they show me there's a much easier way to do it.

And what she said was,

define X as Z equal to X1 plus AX2.

Where A is equal to negative sigma 1 2,

the sigma 2 2 inverse.

And then what I would contend is that

the covariance between X2 and Z.

Let's say, Z and X2 is equal to 0, and let's work that out really quick.

So the covariance of Z and X2 is nothing other

than the covariance of X1 plus AX2 and X2.

Which is equal to the covariance of X1 and

X2 plus A covariance of X2 and X2, okay?

So that's equal to covariance of X1, X2 is sigma 1 2 plus A,

which I have defined as negative sigma 1 2 sigma 2 2 inverse.

And covariance X2 and X2, itself, is sigma 2 2, okay?

So I think you can pretty much see at this point that this equals 0.

So Z is independent of X2.

So the distribution of Z given X2 is just the distribution

of Z disregarding X2, because Z is independent of X2.

So let's calculate, and we know that Z is normal,

because it's a linear combination of normals.

So let's calculate the expected value of Z.

Which is equal to mu

1 plus A mu 2, right?

And the variance of Z, Is equal to,

A the variance of X2 times A transpose with

variance of X2 is sigma 2 2 times A transpose.

Now let's fill some of those in that's mu 1

minus sigma 1 2 sigma 2 2 inverse mu 2.

And this is equal to the negative sign will cancel out sigma 1 2.

And then I get a sigma 2 2 inverse sigma 2 2 sigma 2 2 inverse, so

it's just going to be sigma 2 2 inverse sigma 1 2 transpose, okay?

Okay, so let's think about this.

So the expected value of Z given X2 is equal to little x2 is

equal to the expected value of Z, but it's also equal to,

Mu 1 minus sigma 1 2 sigma 2 2 inverse mu 2, okay?

So we have that much so far, but

we can also write out the expected value of Z as the expected value.

Remember, the definition of Z which we've written up here.

And that works out to be then that we can

write this as expected value of X1 plus

AX2 given X2 equals little x2, okay?

So we then can write,

given X2 plus A expected

value of X2 given X2.

So this is the expected value of

X1 given X2 plus A expected value

of X2 given X2 is just X2.

Okay, so now take this equation and this equation, and

we can solve for the expected value of X1 given X2.

Right?

And it works out to be exactly the formula that we had above.

And then you can use the same technique to derive the variance,

so the variance of Z given X2 equals little x2.

Well that is just exactly equal to the variance of Z,

since Z is independent of X2, since Z is independent of X2.

And we see right up here the variance

of Z is sigma 1 2 sigma 2 inverse,

sigma 1 2 sigma 1 2 sigma 2

2 inverse sigma 1 2 transpose.

But then we also know that it's

the variance of X1 plus AX2 given X2.

Because we're just simply rewriting that statement out with the definition of Z.

And by the rules of variants, we can then apply the variance formula and

apply the rules of variants, and solve and solve again.

And I'm going to ask you to complete the steps for homework.

Ознакомьтесь с нашим каталогом

Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.

Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.

© Coursera Inc., 2019 Все права защищены.

Загрузить из App Store

Загрузить в Google Play