SURVEY RESEARCHfewest errors had the highest percentage of refusals
to answer, and the form with the most errors had the
lowest refusal rate. This example suggests that
errors in a noncontroversial factual question may
vary with minor wording changes.
Questionnaire Design Issues
Length of Survey or Questionnaire.How long
should a questionnaire be or an interview last?^37
We prefer long questionnaires or interviews because
they are more cost effective. The cost for a few
extra questions once a respondent has been sam-
pled, has been contacted, and has completed other
questions is small. There is no absolute proper
length. It depends on the survey format (to be dis-
cussed) and on the respondent’s characteristics.
A 5-minute telephone interview is rarely a prob-
lem. Mail questionnaires are more variable. A short
(three-page) questionnaire is appropriate for the
general population. Some researchers have had
success with questionnaires as long as ten pages
(about one hundred items), but responses drop sig-
nificantly for longer questionnaires. For highly
educated respondents and a very salient topic, a
fifteen-page questionnaire may be possible. Face-
to-face interviews can be long, with ones lasting
an hour not uncommon. In special situations,
researchers have conducted face-to-face interviews
as long as 3 or 5 hours.
Question Order or Sequence.We face three
question sequence issues: organization of the over-
all questionnaire, question order effects, and con-
text effects.
1.Organization of questionnaire.In general,
you should sequence questions to minimize respon-
dent discomfort and confusion. A questionnaire has
opening, middle, and ending questions. After an
introduction explaining the survey, it is best to make
opening questions pleasant, interesting, and easy to
answer. This helps a respondent to feel comfortable
about the questionnaire. Avoid asking many boring
background questions or sensitive questions at the
beginning. Organize questions in the middle into
common topics. Mixing questions on different top-
ics causes confusion. Orient respondents by plac-
ing questions on the same topic together after
introducing the section with a short statement (e.g.,
“Now I would like to ask you questions about hous-
ing”). Make question topics flow smoothly and log-
ically, and organize them to assist respondents’
memory or comfort levels. Do not end with sensi-
tive issue questions, and always say “thank you.”
2 .Order effects.The order in which questions
appear in a questionnaire can influence respondent
answers.^38 Such order effects appear to be
strongest for people who lack strong views, for less
educated respondents, and for older respondents or
those with memory loss.^39 For example, opinions
that support a single woman having an abortion reg-
ularly rises if the question follows a question about
abortion being acceptable when a fetus has serious
defects but not when the question is alone or before
a question about fetus defects. A classic example of
order effects is presented in Expansion Box 7, Ques-
tion Order Effects.
Answers to earlier questions can influence
later ones in two ways: through their content (i.e.,
the issue) and through the respondent’s response.
For example, you ask a student, “Do you support
or favor educational contributions for students?”
Answers vary depending on the preceding question
topic. If it comes after “How much tuition does
the average U.S. student pay?” respondents will
interpret “contributions” to mean what students
will pay. If the question comes after “How much
does the Swedish government pay to students?”
respondents interpret “contributions” to mean those
the government will make. Previous answers can
also influence responses because having already
answered one part respondents will assume no
overlap. For example, you ask a respondent, “How
is your wife?” The next question is, “How is your
family?” Most respondents assume that the second
question means family members other than the wife
because they already answered about her.^40
3.Context effects.Survey researchers have
observed powerful context effectsin surveys.^41Order effect A result in survey research in which a
topic or some questions asked before others influence
respondents’ answers to later questions.