Social Research Methods: Qualitative and Quantitative Approaches

QUALITATIVE AND QUANTITATIVE MEASUREMENT

teachers but invalid for measuring morale among

police officers.^8

At its core, measurement validity tells us how

well the conceptual and operational definitions

mesh with one other: The better the fit, the higher is

the measurement validity. Validity is more difficult

to achieve than reliability. We cannot have absolute

confidence about validity, but some measures are

more validthan others. The reason is that constructs

are abstract ideas, whereas indicators refer to con-

crete observation. This is the gap between our

mental pictures about the world and the specific

Measurement validity How well an empirical indi-

cator and the conceptual definition of the construct that

the indicator is supposed to measure “fit” together.

many people who are homeless avoid involvement
with government and official agencies. However, if
we combine the official records with counts of
people sleeping in various places and conduct sur-
veys of people who use a range of services (e.g.,
street clinics, food lines, temporary shelters), we
can get a more accurate picture of the number of
people who are homeless. In addition to capturing
the entire picture, multiple indicator measures tend
to be more stable than single item measures.
4.Use pilot studies and replication. You can
improve reliability by first using a pilot version of
a measure. Develop one or more draft or prelimi-
nary versions of a measure and try them before ap-
plying the final version in a hypothesis-testing
situation. This takes more time and effort. Return-
ing to the example discussed earlier, in my survey
of teacher morale, I go through many drafts of a
question before the final version. I test early ver-
sions by asking people the question and checking
to see whether it is clear.
The principle of using pilot tests extends to
replicating the measures from researchers. For
example, I search the literature and find measures of
morale from past research. I may want to build on
and use a previous measure if it is a good one, citing
the source, of course. In addition, I may want to add
new indicators and compare them to the previous
measure (see Example Box 1, Improving the Mea-
sure of U.S. Religious Affiliation). In this way, the
quality of the measure can improve over time as long
as the same definition is used (see Table 1 for a sum-
mary of reliability and validity types).

Validity.Validity is an overused term. Sometimes,

it is used to mean “true” or “correct.” There are

several general types of validity. Here we are con-

cerned with measurement validity, which also has

several types. Nonmeasurement types of validity are

discussed later.

When we say that an indicator is valid, it is

valid for a particular purpose and definition. The

same indicator may be less valid or invalid for other

purposes. For example, the measure of morale dis-

cussed above (e.g., questions about feelings toward

school) might be valid for measuring morale among

EXAMPLE BOX 1

Improving the Measure of U.S.

Religious Affiliation

Quantitative researchers measure individual religious

beliefs (e.g., Do you believe in God? in a devil? in life

after death? What is God like to you?), religious prac-

tices (e.g., How often do you pray? How frequently do

you attend services?), and religious affiliation (e.g., If

you belong to a church or religious group, which

one?). They have categorized the hundreds of U.S.

religious denominations into either a three-part

grouping (Protestant, Catholic, Jewish) or a three-part

classification of fundamentalist, moderate, or liberal

that was introduced in 1990.

Steensland and colleagues (2000) reconceptual-

ized affiliation, and, after examining trends in reli-

gious theology and social practices, argued for

classifying all American denominations into six major

categories: Mainline Protestant, Evangelical Protes-

tant, Black Protestant, Roman Catholic, Jewish, and

Other (including Mormon, Jehovah’s Witnesses,

Muslim, Hindu, and Unitarian). The authors evalu-

ated their new six-category classification by examin-

ing people’s religious views and practices as well as

their views about contemporary social issues. Among

national samples of Americans, they found that the

new classification better distinguished among reli-

gious denominations than did previous measures.