Sex-Related Comparisons: Observations 107similarly. Another advantage of meta-analysis
is that researchers can examine how other
variables influence, or moderate, the size of
the effect. Amoderating variableis one that
alters the relation between the independent
and the dependent variable. I often refer to
a moderating variable as an “it depends on”
variable. When sex comparisons are made,
a difference may “depend on” the age of the
respondents, the gender role of the respon-
dents, or the year the study was published.
Recall that Block (1976) found that many sex
differences were apparent only among older
participants; thus age was a moderator vari-
able. Another potential moderating variable
is the year of publication. If a sex difference
existed in the 1980s but disappeared by the
2000s, perhaps women’s and men’s behavior
became more similar over time. We can even
ask if the results of a sex comparison depend
on the sex of the author; men or women
may be more likely to publish a certain re-
sult. Age, gender role, author sex, and year of
publication are frequently tested as modera-
tor variables in the following meta-analyses.
In one way, meta-analysis is limited in
the same way narrative reviews are: Research-
ers still make subjective decisions about what
studies to include in the review. Researchers
conducting a meta-analysis often come up
with a set of criteria to decide whether a study
is included in the review. Criteria may be
based on sample characteristics (e.g., restrict
to English-speaking samples) or on method-
ological requirements (e.g., participants must
be randomly assigned to condition). One dif-
ficulty with any kind of review, meta-analytic
or narrative, is that studies failing to detect a
difference are less likely to be published. In
meta-analysis, this is referred to as thefile-
drawer problem(Hyde & McKinley, 1997):
Studies that do not find sex differences are
not published and endup in investigators’ filemany sex differences are small. Whether
small means trivial depends on the domain
you are investigating. The finding that sex ac-
counts for 1% of the variance in an outcome
does not appear to be earth-shattering. How-
ever, 1% can be quite meaningful (Rosenthal,
1994): It depends on the outcome. For exam-
ple, small effects in medical studies can have
enormous implications. In a study to deter-
mine whether aspirin could prevent heart
attacks, participants were randomly assigned
to receive aspirin or a placebo. The study was
called to a halt before it ended because the
effects of aspirin were so dramatic (Steering
Committee, 1988). The investigators deemed
it unethical to withhold aspirin from people.
In that study, aspirin accounted for less than
1% of the variance in heart attacks.
What about outcomes that are relevant
to gender? Bringing the issue closer to home,
Martell, Lane, and Emrich (1996) used com-
puter simulations to examine the implica-
tions of a small amount of sex discrimination
on promotions within an organization. They
showed that if 1% of the variance in perfor-
mance ratings were due to employee sex, an
equal number of men and women at entry-
level positions would result in 65% of men
holding the highest-level positions over
time—assuming promotions were based on
performance evaluations. So here, a very
small bias had large consequences. How-
ever, there are other times when 1% of the
variance is trivial and does not translate into
larger real-world effects. Keep these ideas
in mind when considering the sizes of the
effects in this chapter.
Using meta-analysis rather than narra-
tive reviews to understand an area of research
has several advantages. As mentioned previ-
ously, meta-analysis takes into consideration
the size of the effects; thus all studies showing
a significant difference will not be weighedM04_HELG0185_04_SE_C04.indd 107 6/21/11 8:02 AM