8.6 Power Calculations in More Complex Designs
In this chapter I have constrained the discussion largely to statistical procedures that we
have already covered, although I did sneak in the correlation coefficient to be discussed in
the next chapter. But there are many designs that are more complex than the ones discussed
here. In particular the one-way analysis of variance is an extension to the case of more than
two independent groups, and the factorial analysis of variance is a similar extension to the
case of more than one independent variable. In both of these situations we can apply rea-
sonably simple extensions of the calculational procedures we used with the ttest. I will dis-
cuss these calculations in the appropriate chapters, but in many cases you would be wise to
use computer programs such as G*Power to make those calculations. The good thing is that
we have now covered most of the theoretical issues behind power calculations, and indeed
most of what will follow is just an extension of what we already know.8.7 The Use of G*Power to Simplify Calculations
A program named G*Power has been available for several years, and they have recently come
out with a new version. The newer version is a bit more complicated to use, but it is excellent
and worth the effort. I urge you to download it and try. I have to admit that it isn’t always ob-
vious how to proceed—there are too many choices—but you can work things out if you take
an example to which you already know the answer (at least approximately) and reproduce it
with the program. (I’m the impatient type, so I just flail around trying different things until I
get the right answer. Reading the help files would be a much more sensible way to go.)
To illustrate the use of the software I will reproduce the example from Section 8.5 us-
ing unequal sample sizes. Figure 8.4 shows the opening screen from G*Power, though
yours may look slightly different when you first start. For the moment ignore the plot at
the top, which you probably won’t have anyway, and go to the boxes where you can select
a “Test Family” and a “Statistical test.” Select “ttests” as the test family and “Means: Dif-
ference between two independent means (two groups)” as the statistical test. Below that
select “Post hoc: Compute achieved power—given a, sample size, and effect size.” If I had
been writing this software I would not have used the phrase “Post hoc,” because it is not
necessarily reflective of what you are doing. (I discuss post hoc power in the next section.
This choice will actually calculate “a priori” power, which is the power you will have be-
fore the experiment if your estimates of means and standard deviation are correct and if
you use the sample sizes you enter.)
Now you need to specify that you want a two-tailed test, you need to enter the alpha
level you are working at (e.g., .05) and the sample sizes you plan to use. Next you need to
add the estimated effect size (d). If you have computed it by hand, you just type it in. If not,
you click on the button labeled “Determine 1 ” and a dialog box will open on the right. Just
enter the expected means and standard deviation and click “calculate and transfer to main
window.” Finally, go back to the main window and click on the “Calculate” button. The dis-
tributions at the top will miraculously appear. These are analogous to Figure 8.1. You will
also see that the program has calculated the noncentrality parameter (d), the critical value of
tthat you would need given the degrees of freedom available, and finally the power, which
in our case is .716, which is a bit lower than I calculated as an approximation.
You can see how power increases with sample size and with the level of aby request-
ing an X-Y plot. I will let you work that out for yourself, but sample output is shown in
Figure 8.5. From this figure it is clear that high levels of power require large effects or large
samples. You could create your own plot showing how required sample size changes with
changes in effect size, but I will leave that up to you.238 Chapter 8 Power