3.Use of special conventions. When we encounter a situation in which there is no way we
can estimate the required parameters, we can fall back on a set of conventions proposed
by Cohen (1988). Cohen more or less arbitrarily defined three levels of d:
Effect Size d Percentage of OverlapSmall .20 85
Medium .50 67
Large .80 53
Thus, in a pinch, the experimenter can simply decide whether she is after a small, medium, or
large effect and set daccordingly. However, this solution should be chosen onlywhen the
other alternatives are not feasible. The right-hand column of the table is labeled Percentage
of Overlap, and it records the degree to which the two distributions shown in Figure 8.1 over-
lap. Thus, for example, when d 5 0.50, two-thirds of the two distributions overlap (Cohen,
1988). This is yet another way of thinking about how big a difference a treatment produces.
Cohen chose a medium effect to be one that would be apparent to an intelligent viewer, a
small effect as one that is real but difficult to detect visually, and a large effect as one that is
the same distance above a medium effect as “small” is below it. Cohen (1969) originally
developed these guidelines only for those who had no other way of estimating the effect size.
However, as time went on and he became discouraged by the failure of many researchers to
conduct power analyses, presumably because they think them to be too difficult, he made
greater use of these conventions (see Cohen, 1992a). In addition, when we think about d, as
we did in Chapter 7 as a measure of the size of the effect that we have found in our experi-
ment (as opposed to the size we hope to find), Cohen’s rules of thumb are being taken as a
measure of just how large our obtained difference is. However, Bruce Thompson, of Texas
A&M, made an excellent point in this regard. He was speaking of expressing obtained differ-
ences in terms of d, in place of focusing on the probability value of a resulting test statistic.
He wrote, “Finally, it must be emphasized that if we mindlessly invoke Cohen’s rules of
thumb, contrary to his strong admonitions, in place of the equally mindless consultation of p
value cutoffs such as .05 and .01, we are merely electing to be thoughtless in a new metric”
(Thompson, 2000, personal communication). The point applies to any use of arbitrary con-
ventions for d, regardless of whether it is for purposes of calculating power or for purposes of
impressing your readers with how large your difference is. Lenth (2001) has argued convinc-
ingly that the use of conventions such as Cohen’s are dangerous. We need to concentrate on
both the value of the numerator and the value of the denominator in d, and not just on their
ratio. Lenth’s argument is really an attempt at making the investigator more responsible for
his or her decisions, and I doubt that Cohen would have any disagreement with that.
It may strike you as peculiar that the investigator is being asked to define the difference
she is looking for before the experiment is conducted. Most people would respond by say-
ing, “I don’t know how the experiment will come out. I just wonder whether there will be a
difference.” Although many experimenters speak in this way (the author is no virtuous
exception), you should question the validity of this statement. Do we really not know, at
least vaguely, what will happen in our experiments; if not, why are we running them? Al-
though there is occasionally a legitimate I-wonder-what-would-happen-if experiment, in
general, “I do not know” translates to “I have not thought that far ahead.”Recombining the Effect Size and n
We earlier decided to split the sample size from the effect size to make it easier to deal with
nseparately. We now need a method for combining the effect size with the sample size. We
use the statistic d(delta) 5 d[f(n)] to represent this combination where the particular230 Chapter 8 Power
d(delta)