[SOUND] With me now are answer on the questions which are asked in the beginning of this lecture. So I assume that 200 variables that have normal distribution and they are unrelated. The question is whether it is true or not that they are independent. Actually, the following theory holds that X1 and X2 be two normal, random variables that, for simplicity, I assume that they are standard normal. Of course, this assumption is not really important. And we know that they uncorrelated, so correlation between X1 and X2 = 0. Then it turns out that X1 and X2 are independent, if and only if the vector X1, X2 is Gaussian. Let me now prove this theorem. Let me start with the first part, then from the dependence, it follows that x1 and x2 is a Gaussian vector. Basically, this is a very simple fact. Because if you take any linear combination of X1 and X2, this combination will have a normal distribution because here, you have a sum of two independent, normally distributed random variables. So it has a normal distribution, and therefore X1, X2 is a Gaussian Vector. Now, what about the second item? So one Y from the right hand side, it follows that X1 and X2 are independent. Well basically, we should apply the theorem, which we showed before. And let me first mention that a covariance metrics if X1 and X2 is a Gaussian vector, is of the form 1, 1, 0, 0. 0 because they're uncorrelated, therefore, all also covariances are equal to 0. And 1 just because the variances of X1 and X2 are equal to 1. So the matrix C is of this form and therefore, the matrix A which was introduced in the second part of the theory, is the square root of the matrix C, is also equal to 1, 0, 0, 1. In fact, if you multiply this matrix by itself, you get exactly the same matrix C. Therefore, by the second part of the theorem, it follows that a vector X1, X2 can be represented as a matrix A, Multiplied by a standard normal random variable, X1. A standard normal random vector, X1 0, X2 0, plus vector nu. Nu is a vector of mathematical expectations, and in this case, all mathematical expectations are equal to 0. Therefore, we'll have here 0 vector and therefore, vector X1, X2 is equal in distribution to the vector X1 0 and X2 0. Finally, we conclude that X1 and X2 independent because X1 0 and X2 0 are also independent. So this observation completes the proof. And so now, we know that an uncorrelated normal random variables are independent, if and only if they form a Gaussian vector. Well, it will be fair to at least one example of uncorrelated normal distributed random variables, which are in fact dependent. Well, this example is given by the following, X1 and X2. So let me take as X1 any standard normal random variable. And let me define X2 as absolute value of X1 multiplied by a random variable psi, whereas psi is equal to 1 and- 1 with probability one-half. And we assume that psi and X1 are independent. So let me first show that the random variable, X2 has a normal distribution, with parameter 0 and 1. So just a simple exercise from the probability theorem. If we now calculate a probability that X2 is less or equal than SX, then we can represent this probability by the law of total probabilities. That is, actually the probabilities that absolute value of X1 is less or equal than x given that psi = 1, multiplied by the probability of psi = 1. Plus probability that x2 is larger or equal than -x, given that psi is equal to -1 multiplied with the probability that psi = -1. According to our definition of psi is the probabilities are equal to one-half. And since X1 and psi are independent, this conditional probabilities are in fact unconditional. If we now assume that x is bigger than 0, then we immediately get this probability, is actually equal to 1, because we have here something positive absolute value of X2. And we should calculate the probability that this positive random variable is larger than something negative. Of course, this probability is equal to 1. So we conclude that the probability that X2 is less or equal than X, is equal to one-half, 1 plus probability that absolute value of X1 is less or equal than x. Due to the properties of the standard normal distribution, this expression is equal to the probability that X1 is less than or equal than x. So we conclude that both X1 and X2 are standard normal. And now, let me show that they are uncorrelated. So let me show that the covariance between X1 and X2 = 0. In fact, covariance can represent that the mathematical expectation of the product between X1 and X2, minus mathematical expectation of X1, multiplied by mathematical expectation of X2. So what is between here? Mathematical expectation of X1 multiplied by absolute value of X1, multiplied by psi. You know that psi and X1 are independent, and therefore, any function of psi and X1 are also independent. So also, X1 and absolute value of X1 and psi are independent. And therefore, what is given here is the mathematical expectation of X1 multiplied by absolute value of X1 and mathematical expectation of psi. Due to our definition, mathematical expectation of psi = 0, and therefore, this mathematical expectation is also equal to 0. Well, as for the second term, we have here mathematical expectation of X1. And you know that X1 is a standard normal random variable, and therefore, this mathematical expectation is equal to 0. And finally, we conclude that 0 minus 0 is 0, and therefore, X1 and X2 are uncorrelated. Well, in the next part, I would like to show that X1 and X2 dependent. Does this statement turns out to be the most difficult part of the proof? So let me now prove that X1 and X2 are dependent. Let me show this by assuming converse. So I assume that X1 and X2 are independent. And in this case, we have that due to this theorem, vector X1, X2 is a Gaussian vector. This means that due to this definition, which I gave before, any linear of combination of the components has normal distribution. In particular, if I consider the difference between components, this difference, which is equal to X1- absolute value of X1 psi, should have a normal distribution. Let me show that it is impossible. Let me first consider the probability that the random variable psi is larger than 0. Well, let me think about the following thing. So when psi can be positive? For instance when X1 is positive and psi = -1, in this case, if you have something positive plus positive, so therefore, this is positive. This means, there's a probability that psi is positive, is larger or equal than the probability that X1 is positive, intersection with the event that psi = -1. But X1 and psi are independent. And therefore, this probability is in fact the product of the probabilities that X1 is larger than 0, and the probability of psi = -1. But you know, that for normal distribution, probability that a standard normal random variable is positive, is equal to one-half. And as for the random variable psi, you know that the probability that psi is equal to -1 is also equal to one-half. Therefore, this probability is larger or equal than 1 divided by 4. And obviously, we can show that the probability that psi = 0, is also larger or equal than one-fourth. Here, we should consider the case that X1 is positive and psi is equal to 1. In this case, we guess that probability of psi is equal to 0, is large or equal than the intersection of this probability, that X1 is positive and psi is equal to 1. Then we decompose the probability of the intersection of true events as a product of two probabilities and get exactly this statement. And now, we shall think about how it is possible that a random variable has Gaussian distribution, and it is positive with non-zero probability. And at the same time, it is equal to 0 with non-zero probability. If variance of psi is positive, then the probability that this random variable is equal to 0, is equal to 0, and therefore, the second statement is not fulfilled. Otherwise, if the variance of a normally distributed random variable is equal to 0, then this, I'm not sure, a constant, and in this case, it is either equal to 0 or positive with probability 1. And here, you see that these two statements cannot be fulfilled simultaneously. Therefore we conclude, that psi doesn't have a normal distribution, and we arrive at a contradiction here. So this observation completes the proof. So let me conclude, that we have found two random variables, X1 and X2, which have normal distribution, more or less standard normal distribution, which are uncorrelated but dependent. [SOUND]