Social Research Methods: Qualitative and Quantitative Approaches

QUALITATIVE AND QUANTITATIVE MEASUREMENT

possible to have perfect reliability and validity, but

they are ideals toward which we strive. Reliability

and validity are salient because our constructs are

usually ambiguous, diffuse, and not observable. Re-

liability and validity are ideas that help to establish

the truthfulness, credibility, or believability of find-

ings. Both terms also have multiple meanings. As

used here, they refer to related, desirable aspects of

measurement.

Reliabilitymeans dependability or consistency.

It suggests that the same thing is repeated or recurs

under the identical or very similar conditions. The

opposite of reliability is an erratic, unstable, or in-

consistent result that happens because of the mea-

surement itself. Validitysuggests truthfulness. It

refers to how well an idea “fits” with actual reality.

The absence of validity means that the fit between

the ideas we use to analyze the social world and

what actually occurs in the lived social world is

poor. In simple terms, validity addresses the ques-

tion of how well we measure social reality using our

constructs about it.

All researchers want reliable and valid mea-

surement, but beyond an agreement on the basic

ideas at a general level, qualitative and quantitative

researchers see reliability and validity differently.

Reliability and Validity
in Quantitative Research
Reliability.Measurement reliabilitymeans that
the numerical results an indicator produces do not
vary because of characteristics of the measurement
process or measurement instrument itself. For
example, I get on my bathroom scale and read my
weight. I get off and get on again and again. I have

a reliable scale if it gives me the same weight each

time, assuming, of course, that I am not eating,

drinking, changing clothing, and so forth. An unre-

liable scale registers different weights each time,

even though my “true” weight does not change.

Another example is my car speedometer. If I am

driving at a constant slow speed on a level surface

but the speedometer needle jumps from one end to

the other, the speedometer is not a reliable indica-

tor of how fast I am traveling. Actually, there are

three types of reliability.^6

Three Types of Reliability

1. Stability reliabilityis reliability across
   time. It addresses the question: Does the measure
   deliver the same answer when applied in different
   time periods? The weight-scale example just given
   is of this type of reliability. Using the test-retest
   method can verify an indicator’s degree of stability
   reliability. Verification requires retesting or re-
   administering the indicator to the same group of
   people. If what is being measured is stable and the
   indicator has stability reliability, then I will have the
   same results each time. A variation of the test-retest
   method is to give an alternative form of the test,
   which must be very similar to the original. For
   example, I have a hypothesis about gender and
   seating patterns in a college cafeteria. I measure my
   dependent variable (seating patterns) by observing
   and recording the number of male and female
   students at tables, and noting who sits down first,
   second, third, and so on for a 3-hour period. If, as I
   am observing, I become tired or distracted or I for-
   get to record and miss more people toward the end
   of the 3 hours, my indicator does not have a high
   degree of stability reliability.


2. Representative reliabilityis reliability
   across subpopulations or different types of cases. It
   addresses the question: Does the indicator deliver
   the same answer when applied to different groups?
   An indicator has high representative reliability if it
   yields the same result for a construct when applied
   to different subpopulations (e.g., different classes,
   races, sexes, age groups). For example, I ask a ques-
   tion about a person’s age. If people in their twenties
   answered my question by overstating their true age

Measurement reliability The dependability or con-

sistency of the measure of a variable.

Stability reliability Measurement reliability across

time; a measure that yields consistent results at differ-

ent time points assuming what is being measured does

not itself change.

Representative reliability Measurement reliability

across groups; a measure that yields consistent results

for various social groups.