279
Psychological Measurement
relations are at best moderate, and often similarly intended measures are only weakly
correlated. Despite following the same guidelines and having access to the same
resources, students in my courses have produced measures that had virtually zero
shared variance with measures of the same trait produced by other students. Given the
educational goals of this activity, I consider such low correlations a good thing as they
provide wonderful fodder for critical thinking. We discuss reliability, validity, and
measurement error, as well as how different researchers may conceptualize traits in
very different ways. In addition, the intertrait correlations usually are interesting, and
students use them to test their a priori hypotheses. In my classes, the procedures
culminate with each student writing a brief report that includes all elements of an
empirical manuscript.
Evaluation and Adaptation to Other Courses
Students have responded favorably to this activity both formally and informally. Hettich
(1974) recommended using student-generated data in class activities, and students have
reported this aspect of the current exercise to be particularly interesting. Most students
thought the activities fulfilled a variety of learning objectives, and most were more aware
of the complexity of scale construction following the activity than at the outset. Students
also indicated significantly greater agreement with the statement, “Writing a personality
test would be an interesting task” at the conclusion of the activity than they had at the
beginning.
As noted throughout, one could easily adapt this activity for various courses. Steps
might include combining sessions, changing the target traits, or including more sophisti-
cated analyses. For example, instructors could introduce regression by having students
examine the incremental prediction of one of the traits using the other two. Further, stu-
dents could investigate moderating relations by testing whether correlations between traits
are moderated by gender. Instructors could introduce more complex measurement issues
by including item-total correlations, internal consistency analysis, and even factor analysis
to determine how various items fit with intended constructs. To make the activity more
practical within a limited time frame, instructors could combine the scale construction
and data collection sessions. The students could then analyze the data outside of class and
discuss the results during a subsequent class meeting. In conclusion, the flexibility of this
exercise permits its use in many different courses to promote critical thinking about
psychological measurement.
References
Beins, B. C. (1993). Using the Barnum effect to teach about ethics and deception in research.
Teaching of Psychology, 20, 33–35.
Benjamin, L. T., Jr. (1983). A class exercise in personality and psychological assessment. Teaching of
Psychology, 10, 94–95.