Genes, Brains, and Human Potential The Science and Ideology of Intelligence

88 PRETEND INTELLIGENCE

Th e main prob lem is that it is very diffi cult for supervisors to agree on
what constitutes good or bad per for mance. In a 1991 article, Linda Gott-
fredson noted, “One need only ask a group of workers in the same job to
suggest specifi c criterion mea sures for that job in order to appreciate how
diffi cult it is to reach consensus about what constitutes good per for mance
and how it can be mea sured fairly.”^15
Consequently, assessments of job per for mance tend to be subjective,
based on inconsistent criteria. Th ey are notoriously biased: age eff ects and
halo eff ects have been reported; height and facial attractiveness have been
shown to sway judgments, and unconscious ethnic bias is also pres ent.
Just how poor supervisors’ ratings are is revealed by their weak agreement
with more objective criteria such as work samples or work output. Cor-
relations near zero are reported in a number of studies.
Another prob lem is that much of the so- called correction to correlations
is based on corrections for mea sure ment error— the way the mea sure of
a function can vary from occasion to occasion (see more on this below).
But in the case of job per for mance, what is called “mea sure ment error”
may not be error at all, but normal fl uctuation.
Aft er all, no one performs at peak all the time: we all exhibit a diff er-
ence between maximum and typical per for mances. In fact individual
variation tends to be greater than that across diff er ent individuals. Yet
correcting these for assumed mea sure ment error boosts the correlations
enormously. In describing the diffi culties, in his own experience, of
seeking objective supervisor ratings across a wide range of jobs, Robert
Guion suggested that we should abandon the pretence of objective, true,
or hard criteria of job per for mance.^16
In other words, far from validating the IQ test and all the other claims
that go with it, mea sures of job per for mance are themselves unreli-
able  and inaccurate. But let us turn to the corrections themselves,
because there are impor tant lessons to be learned from them about the
whole exercise.

DUBIOUS CORRECTIONS

Much of the prob lem of making these corrections is that they require cru-

cial data from the original studies. But those data are oft en missing. In

This content downloaded from 139.184.14.159 on Tue, 17 Oct 2017 13:52:12 UTC

http://www.ebook3000.com