citizenry thinks about a particular issue. Some surveys are created by websites or pop-
ular magazines, and these sometimes get attention for their results even though most
fail to use valid methods of sampling and questioning. Still, numerous surveys that
we see, hear, or read about are developed and privately administered by bona fide
research organizations like Roper or Gallup; other sound surveys are publicly financed
and available to all researchers, such as the General Social Survey at the National
Opinion Research Center in Chicago.Survey Questions.Surveys are the mainstay of sociological research, but coming up
with good survey questions is hard. The wording of the question, the possible
answers, even the location of the question in the survey questionnaire can change
the responses.
Take a classic example from 1941 (Rugg, 1941). In a national survey, respon-
dents were asked two slightly different questions about freedom of speech:
■Do you think the United States should forbid public speeches against democracy?
■Do you think the United States should allow public speeches against democracy?When the results came in, 75 percent of respondents would not allowthe
speeches, but only 54 percent would forbidthem. Surely forbidandnot allowmean
the same thing in practice, but the wording changed the way people thought about
the issue. Psychologists, sociologists, and statisticians are still trying to figure how to
avoid this problem.
Have you ever shoplifted? No? Well, then, have you ever taken an object from a
store without paying for it? Respondents are much more likely to answer “yes” to
the second version because it somehow doesn’t seem as bad, even though it’s really
the same thing.
Do you think women should have the right to have an abortion? How about the
right to end their pregnancy? You guessed it—far more respondents favor the right
to end a pregnancy than to have an abortion.120 CHAPTER 4HOW DO WE KNOW WHAT WE KNOW? THE METHODS OF THE SOCIOLOGISTCalculating the
number of
deaths as a
consequence of war is a gruesome but
difficult task. We might know how many
troops armies have, but what about
civilian casualties? In Iraq, for example,
different sources of data—hospital
records, media reports, police reports, or
mortuary data—all provide conflicting
numbers. (These numbers are low because
many people don’t go to hospitals, are
buried by their families, and are not
reported to the media or police. What’s
more, Iraq has never had a national
census, so random sampling would be
uncertain because the lists of residents
from which such a sample might be
drawn would be incomplete.)
Demographer Gilbert Burnham and
his colleagues at the Johns Hopkins
School of Public Health conducted
cluster samples in which they picked out
neighborhoods at random and surveyedFinding Hard-to-Get Answers
through SamplingHow do we know
what we know
all the people living in them. They
examined data from 47 neighborhoods,
each of which had about 40 residents
living in it. They asked residents
whether anyone had died since the U.S.
invasion and what the cause of death
was and certified over 90 percent of the
deaths. They compared this to data from
before the invasion, and they calculated
that about 650,000 more people had
died than would have died had the war
never begun, a number significantly
higher than earlier estimates (The
Economist, October 14, 2006, p. 52).
The statistical methods we use often
have significant impact on how we
perceive an event.