American Politics Today - Essentials (3rd Ed)

136 CHAPTER 5|PUBLIC OPINION AND THE MEDIA

ISSUES WITH SURVEY METHODS

Building a random sample of respondents is not easy. One standard strategy is to

choose households at ra ndom from census data a nd send inter viewers out for face-

to-face meetings. Another strategy involves contacting people by telephone using

random digit dialing, which allows surveyors to fi nd people who have unlisted

phone numbers or who just use a cell phone. While each technique in theory pro-

duces a random sample, in practice they both may deviate from this ideal. For

example, face-to-face interviewing risks losing households in which both adults

work away from the home during the day.

To keep costs down, many organizations use other strategies. These may

include Internet polling, in which volunteer respondents log on to a website to

participate in a survey, or robo-polls, in which a computer program phones people

and interviews them. While these techniques are less expensive, there are serious

doubts about the randomness of the samples they produce.^43

Question wording can also infl uence survey results. Table 5.2 shows four dif-

ferent questions asked during late 2010 to measure opinions about the change in

The sampling error in a survey (the predicted difference
between the average opinion expressed by survey
respondents and the average opinion in the population,
sometimes called the margin of error) using a random
sample depends on the sample size. Sampling error is large
for small samples of around 100 or less, but it decreases
rapidly as sample size increases.
The graph shows how the sampling error for a random
sample decreases as sample size increases. For example,
in surveys with 1,000 respondents the sampling error is 2
percent, meaning that 95 percent of the time, the results
of a 1,000-person survey will fall within the range of 2
percentage points above or below the actual percentage in
the population that holds a particular opinion surveyed. If
the sample size was increased to 5,000 people, the sampling
error would decline to 0.5 percent.
Sampling errors need to be taken into account in
interpreting what a poll says about public opinion. Suppose
a 1,000-person survey fi nds that 60 percent of the sample
favor candidate Smith, while 40 percent support candidate
Jones. Since the difference in support for the two candidates
(20 points) exceeds the sampling error (4 points), it is
reasonable to conclude that Smith has more supporters
in the population than Jones and should be considered the
favorite to win the election.
In contrast, suppose the poll found a narrow 51 to 49
percent split slightly favoring Smith over Jones. Since the
difference in support is smaller than the sampling error,
it would be a mistake to conclude that Smith is the likely

winner. Even though Smith is ahead among the sample,

Jones may have more supporters in the population. Put

another way, given the sampling error, the survey results

tell us that there is a 95 percent chance that Smith’s

support in the population is between 49 and 53 percent,

and Jones is between 47 and 51 percent. In other words,

when a poll shows a difference in support smaller than the

sampling error, the only thing that poll tells us is that neither

candidate is the clear favorite.

1

2

3

4

5

6%

100 600 1,100 1,600 2,100 2,600

Sample size

Sampling error

SAMPLING ERROR IN MASS SURVEYS

NUTS & bolts

5.1

sampling error A calculation that
describes what percentage of the
people surveyed may not accurately
represent the population being stud-
ied. Increasing the number of respon-
dents lowers the sampling error.