22 Chapter 1 What Is Psychology?
And again, technology can help, because many
people feel more anonymous when they interact
with a computer than when they fill out a paper-
and-pencil questionnaire, and so are less likely to
lie (Turner et al., 1998). If the survey is filled out
online, however, it may be hard to know whether
participants have understood the questions and
have taken them seriously.
When you hear about the results of a survey
or opinion poll, you also need to consider which
questions were (and were not) asked, and how
the questions were phrased. These aspects of a
survey’s design may nudge responses in a par-
ticular direction, as political pollsters well know
(“Do you favor raising your property tax to spend
millions of dollars to repair your local schools?”
is more likely to evoke a no than “Do you favor
rebuilding schools that are decaying, lack heat,
and are infested with rats?”). Many years ago,
the famed sex researcher Alfred Kinsey made it
his practice always to ask, “How many times have
you (masturbated, had nonmarital sex, etc.)?”
rather than “Have you ever (masturbated, had
nonmarital sex, etc.)?” The first way of phras-
ing the question tended to elicit more truthful
responses than the second because it removed
the respondent’s self-consciousness about hav-
ing done any of those things. The second way
of phrasing the question would have permitted
embarrassed respondents to reply with a simple
but dishonest “No.”
As you can see, although surveys can be ex-
tremely informative, they must be conducted and
interpreted carefully.
newspapers and on the Internet usually have not
been evaluated for either validity or reliability.
These questionnaires often have inviting head-
lines such as “What Breed of Dog Do You Most
Resemble?” or “What’s Your Love Profile?” but
they are merely lists of questions that someone
thought sounded good.
Surveys LO 1.12
Psychological tests usually generate information
about people indirectly. In contrast, surveys are
questionnaires and interviews that gather infor-
mation by asking people directly about their ex-
periences, attitudes, or opinions about everything
from political preferences to sexual preferences.
Most of us are familiar with national opinion sur-
veys, such as the Gallup and Roper polls. And if
you eat at a restaurant, get your car serviced, stay
at a hotel, or order something online, you’re apt
to get a satisfaction survey five minutes later. How
reliable are all these surveys?
Surveys produce bushels of data, but they are
not easy to do well. Sampling problems are often
an issue. When a talk-radio host or TV personal-
ity invites people to
post comments on
the Internet about a
political matter, the
results are unlikely to
generalize to the pop-
ulation as a whole,
even if thousands of
people respond. Why? As a group, people who listen
to someone like Rush Limbaugh are more conser-
vative than fans of someone like Jon Stewart.
Popular polls and surveys also frequently suf-
fer from a volunteer bias: People who are willing
to volunteer their opinions may differ from those
who decline to take part. When you read about a
survey, or any other kind of study, always ask who
participated. A nonrepresentative sample does not
necessarily mean that a survey is worthless or un-
interesting, but it does mean that the results may
not hold true for other groups.
Yet another problem with surveys, as with
self-reports in general, is that people sometimes
lie, especially when the survey is about a touchy
or embarrassing topic (“I would never do that
disgusting/dishonest/fattening thing!”) or asks
about an illegal act, such as using banned drugs
(Tourangeau & Yan, 2007). The likelihood of ly-
ing is reduced when respondents are guaranteed
anonymity and allowed to respond in private.
Researchers can also check for lying by asking the
same question several times with different word-
ing to see whether the answers are consistent.
Surveys
and interviews that ask
people directly about
their experiences, atti-
tudes, or opinions.
About Opinion Polls
and Surveys
volunteer bias A
shortcoming of findings
derived from a sample
of volunteers instead of
a representative sample;
the volunteers may differ
from those who did not
volunteer.
Thinking
CriTiCally
©The New Yorker Collection 1970 Henry Martin from cartoonbank.com. All Rights Reserved.
“Are you (a) contented, (b) happy, (c) very happy,
(d) wildly happy, (e) deliriously happy?”