79
Programmatic Assessment of Critical Thinking
suggesting where students may need more help or where the curriculum could be adjusted
to better address core concepts in critical thinking.
In addition to Cluster One data, we also benefit from Cluster Three (Natural World)
data collected by our colleagues. It is our position that critical thinking and scientific rea-
soning are at least related, that improvements in scientific reasoning constitute to some
extent improvements in critical thinking, and that measures of our students’ scientific
reasoning can inform us about their critical thinking. In particular with respect to training
psychology majors at JMU, we believe that our students’ critical thinking is enhanced as a
result not only of general education coursework designed to address knowledge, skills, and
abilities (KSAs) related to critical thinking specifically (i.e., Cluster One, Skills for the 21st
Century), but also as a result of coursework designed to address KSAs related to scientific
reasoning (i.e., Cluster Three, the Natural World). From this argument, it follows that we
also can use measures to assess scientific reasoning as a measure of our students’ critical
thinking.
The Cluster Three (Natural World) requirements include a math course and science
courses to establish quantitative and scientific literacy. The general education program is
intended to provide all students with foundational KSAs on which they can build more
specialized KSAs from their majors, minors, and preprofessional programs.
The Natural World (NW) assessment instrument consists of 50 objective answer ques-
tions. Reliability has steadily improved with each revision of the instrument. Our assessment
specialists selected the best items from earlier administrations to form the fifth version,
NW–5. The NW–5 showed the best reliability to date, with a = .67 for the freshmen and
a = .75 for sophomores (Horst, Lottridge, Hoole, Sundre, & Murphy, 2005).
To examine scientific thinking in our psychology majors, we examined how psychology
majors performed on the NW–5 during two test administrations: Fall 2001 and Spring
- Forty-one psychology majors completed the NW–5 testing during Fall 2001, and
70 psychology majors completed the NW–5 during Spring 2003. Of these two groups,
22 overlapped, so we were able to look at independent as well as dependent group differ-
ences over time.
Figure 7.1 shows the summary data for performance on the NW–5 test. Because each
correct answer on the 50-item test was awarded 2 points, students could obtain a score
50
55
60
65
Independent Repeated
Group
Freshmen
Midcareer
Mean (+/
−
SE) Score
Figure 7.1. Mean (±SE) NW–5 scores for incoming freshman and mid-career students.