related to what prevailing psychiatric opinion
regarded as maladjustment.
Ensuring content validity, however, involves
much more than deciding what you want to assess
and then making up some items that appear to do
the job. Rather, more sophisticatedcontent validation
methods involve (a) carefully defining all relevant
aspects of the variable you are attempting to mea-
sure; (b) consulting experts before generating items;
(c) using judges to assess each potential item’s rele-
vance to the variable of interest; and (d) using psy-
chometric analyses to evaluate each item before you
include it in your measure (Haynes, Richard, &
Kubany, 1995; Nunnally & Bernstein, 1994).
However, as Wiggins (1973) observes, several
potential problems are inherent in the content
validity approach to test construction. First, can
clinicians assume that every patient interprets a
given item in exactly the same way? Second, can
patients accurately report their own behavior or
emotions? Third, will patients be honest, or will
they attempt to place themselves in a good light
(or even a bad light at times)? Fourth, can clinicians
assume that the“experts”can be counted on to
define the essence of the concept they are trying
to measure? Most of these seem to be general prob-
lems for the majority of inventories, regardless of
whether they depend on content sampling to estab-
lish their validity.
Empirical Criterion Keying. In an attempt to
help remedy the foregoing difficulties, theempirical
criterion keyingapproach was developed. The most
prominent example of this general method is the ori-
ginal Minnesota Multiphasic Personality Inventory
(MMPI).Inthisapproach,noassumptionsaremade
as to whether a patient is telling the truth or the
response really corresponds to behavior or feelings.
What is important is that certain patients describe them-
selves in certain ways. As Meehl (1945) put it,“Thus if a
hypochondriac says that he has‘many headaches’the
fact of interest is that he says this”(p. 9).
The important assumption inherent in this
approach is that members of a particular diagnostic
group will tend to respond in the same way. Conse-
quently, it is not necessary to select test items in a
rational, theoretical fashion. All that is required is to
show on an empirical basis that the members of a
given diagnostic group respond to a given item in
a similar fashion. For example, in contrast to non-
clinical respondents, if most individuals diagnosed
with psychopathy agree with the item“Igrewup
in a house that had three steps on the front porch,”
then that item is a good one because it is endorsed by
members of the psychopathic group. Thus, indepen-
dent of an item’s surface content, the test response
becomes a“sign”of one’s diagnostic status. The util-
ity of an item is thus determined solely by the extent
to which it discriminates among known groups. The
test response is not necessarily asampleof behavior
because the content of the item may not be directly
associated with the symptoms that characterize
members of that diagnostic group.
Of course, the criterion keying method has its
problems. Foremost is the difficulty of interpreting
the meaning of a score. For example, suppose that
some patients diagnosed with schizophrenia are
answering items intended to place them somewhere
along the adjusted–maladjusted dimension. Suppose
also that most of these patients happen to come
from less educated families than do the participants
in a comparison group. When these patients with
schizophrenia endorse the item“I almost never read
books,”that endorsement may reflect their poor
educational background rather than their psychopa-
thology. Although demonstrating that the test can
discriminate among various patient groups is one
aspect of establishing the validity of a test, the sole
use of the empirical criterion keying method to
select items for a test is not recommended (Strauss
& Smith, 2009).Factor Analysis. These days, the majority of test
developers use a factor analytic (or internal consis-
tency) approach to test construction (Clark &
Watson, 1995; Floyd & Widaman, 1995; Reise,
Waller, & Comrey, 2000). The Guilford Inventories
(Guilford, 1959) are excellent historical examples of
a factor analytic approach. Here, the idea is to exam-
ine the intercorrelations among the individual items
from many existing personality inventories. Succeed-
ing factor analyses will then reduce or“purify”scales222 CHAPTER 8