Statistical Conclusion Validity
Statistical conclusion validity refers to the confidence one has that the results
of the statistical analysis accurately reflect the true relationship between the IV
and DV. This does not happen when researchers make a type II error. Type II
errors occur when researchers inaccurately conclude that there is no relationship
between the IV and DV when an actual relationship does exist. A type II error is
more likely to occur when the sample size is small. Often large samples are needed
in order to detect the effect of the IV on the DV. Low reliability of the measures
is another factor that can interfere with researchers’ abilities to draw accurate
conclusions about relationships between the IV and DV. When ap praising
research articles for threats to statistical conclusion validity, it is important for
readers to consider the information presented in the methods section that details
the reliability of instruments. Researchers can control for low reliability of the
measures by using well-established and well-designed instruments.External Validity
External validity refers to the degree to which the results of the study can be
generalized to other subjects, settings, and times (Cook & Campbell, 1979).
There are five major threats to external validity. Construct validity is concerned
with whether the instruments are really measuring the theoretical concepts
under investigation. Other threats to external validity include threats related to
selection of subjects for the study and ones that occur because of interactions
between the IV and the subjects, the setting, or history.Construct Validity
Construct validity is an important consideration when evaluating external
validity. Assessing construct validity allows researchers to determine whether
instruments are actually measuring the theoretical cause or effect concepts that
are intended to be measured. A threat of construct validity can lead to bias or
unintentional confounding of the results.
Bias refers to a systematic error in subject selection, measurement of vari-
ables, or analysis. For example, a researcher is studying the effect of anger
on blood pressure; however, the instrument used to measure anger contains
some items that might also reflect depression. The relationship between angerKEY TERMS
statistical
conclusion validity:
The degree that
the results of the
statistical analysis
reflect the true
relationship
between the
independent
and dependent
variables
type II error:
When researchers
inaccurately
conclude that there
is no relationship
between the
independent
and dependent
variables when an
actual relationship
does exist; when the
researcher accepts
the null hypothesis
when it should have
been rejected
external validity:
The degree to
which the results
of the study can be
generalized to other
subjects, settings,
and times
construct validity:
A threat to external
validity when the
instrument does not
accurately measure
the theoretical
conceptsA researcher is interested in studying the effect of hearing loss on self-esteem in adolescents
attending grades 6–12. What threats to internal validity should the researcher address? What
controls could the researcher use to minimize these threats?CRITICAL THINKING EXERCISE 6-1
158 CHAPTER 6 Key Principles of Quantitative Designs