Chapter 7, page 160
diet; ineffective reasoners are more likely to draw conclusions from insufficient information (Klaczynski,
1997; Stanovich & West, 1998). This brings us back to an idea we discussed in Chapter 1: the importance
of comparison groups in experimental research. As you learned in Chapter 1, a relevant comparison group
is a group that is similar to the treatment group in as many respects as possible except for the treatment. In
experiments with random assignment, the comparison group is a control group. In situations where random
assignment is not carried out, one can still try to identify relevant comparison groups that are as similar as
possible to the group of interest. Without such comparison groups, meaningful conclusions are often
impossible.
Here is another example. A teacher is trying to figure out why seven of his fourth grade students are
doing poorly. The teacher thinks that the cause may be that the students are spending too much time
watching TV. Therefore, the teacher interviews each student and finds out that all 7 students watch TV for
at least 2 and a half hours a day. He concludes that too much TV is the cause of the problem. Has the
teacher drawn a valid conclusion? The answer is no, because the teacher has not compared these students
with students who are doing well. In fact, most American children watch from 3 to 5 hours of TV per day
(Christakis, 2007). It is likely that the children doing well watch many hours of TV per day, as well. If so,
this cannot be the factor causing the seven children to do poorly. Faulty conclusions arise from not thinking
comparatively.
Problem 7.11
Understanding students’ thinking: Evaluating students’ use of evidence
A student thinks that laziness causes school failure. The teacher asks, “Why do
you think so?” The student replies, “Because I see it around me, you know. I
have friends who fail. They figure it’s the right thing to do, and, you know, they
just get lazy or want to hang out with their friends. And I read someone say the
same thing on some blog on the internet.” Evaluate this student’s reasoning.
(adapted from data in Kuhn, 1991)
Response: In terms of the strategies discussed in this chapter, the student’s
reasoning is flawed in several respects. The student is considering only a small
sample (a few friends). The student also fails to consider appropriate
comparison groups, such as students who do not fail; do those students also
like to hang out with their friends? You could also say that the student has not
tried to generate possible evidence on the other side of the question. And the
student does not employ the strategy of sourcing with respect to the
information read on the blog. Whose blog is this? Is the blog trustworthy? The
student does not attend to these issues.
Sourcing. Sourcing refers to carefully considering the source of evidence and how credible the
source is when evaluating evidence. For example, if you are a juror hearing eyewitness testimony in a
murder trial, you will be most confident in the testimony if the eyewitness has no reason to be biased, if the
eyewitness had a clear view of the event, and if there is no reason to suspect that the eyewitness has
distorted memories of the event (cf. Ennis, 1987). If an eyewitness stands to profit financially from the
defendant’s conviction, you are likely to treat the testimony more skeptically than if the eyewitness is a
neutral observer who did not previously know the defendant. Studies indicate that students often fail to
consider the source of evidence when they weigh the evidence (Britt & Aglinskas, 2002; De La Paz, 2005;
Wiley & Bailey, 2006). Some characteristics of trustworthy sources are in Table 7.4