When the explanatory variable has more than two levels, the Chi-Square statistic and associated p value do not provide insight into why the null hypothesis can be rejected. It does not tell us what way the rates of nicotine dependence are not equal across the frequency categories. There are of course, many ways for the rates to be unequal. Having each of them as unequal to each other is just one of them. >> Maybe there are only two of the population rates that are not equal to one another. To determine which groups are different from the others, we will again, need to perform a post hoc test. By conducting post hoc comparisons between pairs of rates, in a way that avoids excessive type one error. In other words, avoids rejecting the null hypothesis, when the null hypothesis is true. We will be much better able to appropriately describe which population rates are different from the others. I would really love to tell you that the post hoc paired comparisons are as easy to conduct in the context of Chi-Square test of independence as they are in the context of ANOVA. Unfortunately, though the task is a bit more arduous. There is no built-in follow up multiple comparison test available for this procedure. If we reject the null hypothesis, we need to perform comparisons for each pair of nicotine dependent's rates across the six smoking frequency categories. In the case of 6 groups, we actually need to perform 15 pairwise comparisons. With these red brackets, I'm illustrating 15 paired comparisons that we'll need to conduct. As you can see, there are so many, it's actually difficult to illustrate this graphically. >> If you'll recall, the Family-Wise Error Rate for 15 different comparisons is .54. This means that if we do no protect against type 1 error, we will be wrongly rejecting the null hypothesis, and saying that there is an association over half the time. Having about a 50-50 chance of being right, would obviously give us absolutely no confidence in our decisions. >> So, it will appropriately protect against type 1 error in the context of the Chi-Square test, we will use the post hoc approach known as the Bonferroni Adjustment. The goal of using the Bonferroni Adjustment is to control a family-wise error rates, also known as the maximum overall type 1 error rate. So, that we can evaluate which pairs of nicotine dependents rate are different from one another. >> Briefly, the process would be to conduct each of the 15 paired comparisons. But rather than evaluating significance at the p .05 level, we would adjust the p value to make it more difficult to reject the null hypothesis. The adjusted p value is calculated by dividing p .05 by the number of comparisons that we plan to make. So, if we make 3 comparisons, we would only reject null hypothesis if the p value were .017 or less. For the 15 paired comparisons that we plan to make to better understand the association between smoking frequency and nicotine dependence, our adjusted p value is .003. Adjusting the p value is definitely the easy part of the process. Now, for the more challenging piece. For the actual post hoc testing, we need to run a Chi-Square test for each of the 15 paired comparisons. To do this, I can add syntax to my program at the end of the data step just before the PROC SORT statement. Where I choose two smoking frequency groups at a time. So, I'm going to start by comparing my usual smoking frequency per month group 1, and my usual smoking frequency per month group 2.5. If I save and run this program, I get a new Chi-Square table that includes only those two frequency groups by the presence or absence of nicotine dependence. Again, I wanna focus here on the column percentages. 9.86 and 18.46, are these two rates significantly different from one another? If I look down at my Chi-Square value and probability, I can see that they aren't. So, I want to accept the null hypothesis, since this probability value is not only not less than 0.05. It is definitely not less than my Bonferroni Adjusted p value of .003. Going back to my graph, showing the rates of nicotine dependence for each smoking frequency group. Similar to the notation that was used in the Duncan post hoc test in the context of ANOVA, I'm going to designate the first two rays with the same letter. That is a capital A, indicating that they do not differ significantly from each other. This is just the first step at our post hoc analysis. Now, we need to run two by two Chi-Squares for each of the remaining 14 paired comparisons. >> To do this, we could continue one comparison at a time, choosing two frequency groups. By adding syntax to the program, at the end of the data step, and just before the PROC SORT statement. Use the syntax requesting a Chi-Square analysis, comparing those smoking one day per month and those smoking six days per month. For our code, we can see that only smoking frequency groups equal to 1 and equal to 6 are included in the Chi-Square table and analysis. The nicotine dependence rates are 9.86 and 21.59. The p value associated with the Chi-Square statistic is .0468. Initially, we might wanna say that this is a significant finding because it's less than a p value of .05. Remember though, that the adjusted p value for these comparison is .003. So, this is not significant. Going back to the graph of nicotine dependence rates, we now know that frequency group equal to 1 and equal to 6, do not have significantly different rates of nicotine dependence. We'll illustrate this by again, adding the same letter to the smoking frequency group equal to 6. This can get pretty tedious and of course, it would be very easy to get a big confused and to skip one or more paired comparisons. Let's look at how to use syntax that you already know to this systematically. And to get results for all of the paired comparisons simultaneously. First, we'll get rid of the lines of syntax that subsets our data to specific smoking frequency groups. Now, we'll create a series of new data sets in associated Chi-Square tables that call in the original working data set. And select each of the various combination of smoking frequency groups. We've just added quite a bit of syntax to this program. You'll see however, that it's repetitive, and that in each group of syntax, only the logic statements that select the specific frequency groups and the working data set name need to change. Following the data step and the request for output that we've been working with. We're going to add a new data step, and call it COMPARISON1. We set the original working data set, which was called NEW, above. And then subset the data to the two smoking frequency categories. Smoking frequency category 1, and 2.5. We end this data step by sorting by the unique identifier, and requesting a Chi-Square analysis with a PROC FREQ procedure. Then we end this new syntax group with a RUN statement. Next, we repeat those same lines of syntax for each of the remaining 14 paired comparisons. Changing only the name of the new working data set, and the selected smoking frequency groups. Copy and paste works very well when creating numerous new data sets for post hoc comparison tests. It's good for avoiding typos that can compromise your program. Here, we call the DATA COMPARISON2, and select smoking frequency group 1, compared to smoking frequency group 6. Here, the data is COMPARISON3. And we're comparing smoking frequency group 1 to smoking group 14. COMPARISON4 is comparing smoking frequency group 1, and smoking frequency group 22. We see these lines of syntax repeated with new data names, and the additional paired comparisons for the smoking frequency groups. All the way down to COMPARISON15, smoking frequency group 22 versus smoking frequency group 30. Well, we run this program, the results include the overall Chi-Squared table. That is, the sixth level smoking frequency variable by the nicotine dependence response variable. And then Chi-Square tables for each of the paired comparisons. 1 versus 2.5, 1 versus 6, 1 versus 14, 1 versus 22, 1 versus 30, 2.5 versus 6, and so on. The goal is to examine the p value for each of the paired comparisons. And to use the adjusted Bonferroni p value of .003 to evaluate significance. Here, we've created a table that shows the p values of each of the paired comparisons from the output. Obviously, there are several that are less that p is .05. Here are the p values that are less than .003. As we can see, smoking frequency group 30 that is, those who smoke 30 days in a usual month, is significantly different from each of the other smoke frequency levels. In addition, smoking frequency group 1 has significantly different nicotine dependence rates, than smoke frequency groups 14, and 22. Using the letter convention, in which nicotine dependence rates with the same letter are not significantly different, these post hoc findings can be pictured like this. Here's another way we could picture the significant differences between rates. As you can see, the more differences there are, the more challenging the visualization can be to create.