[MUSIC] So here we're gonna derive the individual test, the corrected significance level from the overall significance level. So, we're gonna set the overall significance level, alpha, equal to the probability that at least one of the tests is significant. All right, so at least one is significant. Well, what's that? That's 1 minus the probability that none of them are significant. And the probability that none of them are significant, assuming independence, is the probability that the first one is not significant times the probability that the second one is not significant and so on. So that's 1 minus alpha c raised to the k experiment. 1 minus alpha c, times 1 minus alpha c, times 1 minus alpha c, and so on. And that's what this expression says. So fine. So, now we just solve for alpha c, and we get this expression, 1 minus 1, minus alpha, raised to the 1 over k. The k th root of 1 minus alpha. Okay, so showing the same plot from before, but now zooming the scale in down around 0.05 where the significance level was, you can see the difference between these two corrections. So the Sidak correction is more conservative than the Bonferroni Correction. So, Bonferroni evens it out across. So, instead of it increasing the likelihood of making a mistake quickly, which is what the previous plot shows, this is zooming in on 0.05, and showing that the Bonferroni makes it constant across, regardless how many tests. Which makes sense cuz you're just dividing it by the number of tests you've done, okay. And the Sidak Correction is even more conservative. All right, so that's what to remember. Both of these are considered to be more conservative than is perhaps necessary. You give up too much statistical power when you use these. And, in fact, any correction for the it goes back to the definition of familywise error rate is considered to be too conservative. Another way of controlling for multiple hypotheses tests, that is less conservative, is by considering the false discovery rate. Okay, and so the false discovery rate you can understand by going back to our grid, and labeling it a slightly different way. The mnemonic here is that the total T and F stand for true and false, D and N stand for discovery and non discovery. So false discovery is FD, true discovery is TD. True non discovery is TN and false non discovery is FN. With this notation, the false discovery rate, FDR, which is sometimes called Q, is the number of false discoveries over the total number of discoveries. And so here, in this notation by the way, D is equal to FD plus TD. So these are counts. These are the number of true relationships and false relationships, and so on. So this is the rate you're trying to control for. The Bonferroni correction and other familywise error rate corrections tend to wipe out evidence of the most interesting effects, we say they suffer from low power. So the false discovery rate controls offer a way to increase power while maintaining some principle bound on error. Okay, and so it's, more dutifully, it's based on the assessment that four false discoveries out of ten. If you reject the null hypothesis ten times, you make five discoveries quote, discoveries, having four of those be false is really bad. But it's much worse than making 20 false discoveries out of 100. The idea here is that finding true effects is a good thing. And so even though you're gonna make some mistakes, if you can, the more you find the more value you've added.