In fact, the findings revealed just the opposite. For alternative options
shortfalls, scores at step 1 averaged only .1 hits; at step 2, 1.6; and at step 3,
2.1. For other side of the case shortfalls, scores at step 1 averaged again about
.1 hits, at step 2 about 3.0, and at step 3 about 2.9. In other words, very few
subjects detected the shortfalls at step 1, showing hardly any sensitivity to the
shortfalls. When, at step 2, the possibility of a shortfall was pointed out, only
about half the subjects agreed that this was a shortfall. Yet, as gauged by step
3, almost everyone, whether they noticed the shortfall initially or thought it
was a shortfall, could devise alternative options and other-side reasons.
This was strong support for the importance of the dispositional side of
thinking. It also challenged the common presumption that dispositions were
mostly a matter of motivation. To be sure, inclination proved an important
factor in accord with the step 2 scores. However, by far the greatest bottle-
neck was sensitivity—failure to notice at all what needs thoughtful attention.
A second study streamlined the procedures previously described, which in-
volved one-on-one or small group administration. This version allowed pa-
per administration in large groups. The study eliminated step 2, focusing on
the contrast between sensitivity and abilities. The study employed shorter
and more stories, a body of 18 stories spanning three kinds of thinking—deci-
sion making, problem solving, and explanation—and embodying shortfalls
in seeking options and looking for reasons on both sides of a matter as be-
fore. Ninety-four sixth graders responded to all 18 stories. Scoring for each
response was done with a 6-point Likert scale, the low end representing no or
sparse responses, the high end richly articulated responses. After some prac-
tice, strong interjudge reliability was achieved.
The findings mirrored those from the previous study. If dispositions con-
tributed little to performance, students would easily notice and attend to the
thinking shortfalls at step 1, although their comments on the shortfalls might
be shallow. Scores for step 1 and (skipping step 2) step 3 would be about the
same, because subjects would already have done well on step 1, perhaps men-
tioning one or two more responses on step 3. In fact, step 3 performance was
far superior to step 1. Composite scores for each subject were created for step
1 and for step 3 by summing ratings across stories. The mean composite score
for step 1 (x = 2.12) was over one standard deviation lower than that for step
3 (x = 2.98,p< 0001).
The results were also analyzed in terms of hits and misses, the style of the
previous study, by establishing a threshold on the Likert scales for steps 1
and 3 for what constituted a hit. For both options and other-side reasons,
subjects showed a mean hit percentage at step 1 of about 10% and at step 3
of about 70%. These rates did not vary appreciably with the kinds of stories
in which the shortfalls were imbedded—decision making, problem solving,
and explanation.
364 PERKINS AND RITCHHART