Statistical Methods for Psychology

Fvalues that we obtain. Finally, we can ask how many of those Fvalues exceed the critical

value of F, thus leading to a significant result. In other words, what we’re saying here is

“Assume that the populations really have means like the ones Conti and Musty obtained.

How often would we obtain sample data from such populations that would lead us to reject

?” This is what power is all about.

Using a very simple program that I wrote using R, though I could have used SPSS or

SAS, I created five populations with parameters corresponding to the statistics that Conti

and Musty found. I then drew five samples at a time from these populations, and ran an

analysis of variance on the result. I simplified the problem slightly by assuming that we ran

10 subjects in each group, rather than the unequal numbers of subjects in their groups,

though it would have been easy to do it the other way. I repeated this process 10,000 times,

and Table 11.7 presents the results of the first 10 sets of samples, showing the means and

with their associated Fand pvalues, Here you can get some idea of the natural variability

of these statistics even in a case where we know that the null hypothesis is false.

Notice that the eighth set of means has an Fof 2.54, which just barely misses being sig-

nificant. This illustrates the point that even having quite different population means does

not guarantee that each replication of the experiment will reject the null hypothesis.

Another way to look at these results is to plot all 10,000 Fvalues that were produced.

If the null hypothesis had been true, the Fs would be distributed around a mean of approxi-

mately. Instead, this empirical Fdistribution, shown

in Figure 11.6, has a mean considerably above 1.046, reflecting the fact that the noncen-

trality parameter is not 0.00. The mean of this distribution is 5.833. I said earlier that when

is not true, the expected value of Fis

where the first part of this equation includes the noncentrality parameter. Using the

means and variance given on page 339, and treating Conti and Musty’s means as popula-

tion means, and their MSerroras our estimate of se^2 , would give an expected value of

(1 1 4.58)(1.046) 5 5.838, which is very close to the actual mean of this distribution (5.833).

Finally, the critical value is. You can see that most of this distribution is

above that point. In fact, 92.16% of the values exceed 2.58, meaning that given these popu-

lation parameters, the probability of rejecting (i.e., the power of the test) is .9216. This

value agrees closely with the values we could calculate exactly using electronic tables of

the complete noncentral Fdistribution (.9207), or approximately using the table of the

noncentral Fdistribution in the back of this book.

H 0

F4,45=2.58

E(F)=a 11

nat^2 j

s^2 e(k 2 1)

ba

dferror

dferror 22

b

H 0

dferror>(dferror 2 2)= 45 > 43 =1.046

H 0

352 Chapter 11 Simple Analysis of Variance

Table 11.7 Means of 10 computer replications of Conti

and Musty (1984)

Control 0.1 mg 0.5 mg1mg2 mg Fp

34.30 57.60 65.00 47.62 37.10 4.10 .0064

25.30 54.80 61.22 49.37 39.70 8.60 <.0001

26.90 44.80 56.44 53.43 31.30 7.45 .0001

27.40 49.50 59.89 46.12 37.70 7.34 .0001

31.20 50.80 61.22 47.37 35.10 9.73 <.0001

32.70 47.60 62.33 56.00 43.10 3.73 .0105

30.10 47.70 62.44 59.87 26.90 8.87 <.0001

39.60 57.00 60.44 52.12 53.40 2.54 .0527

26.70 52.30 60.33 47.25 32.90 4.77 .0027

36.70 42.70 60.00 58.62 46.20 5.00 .0020