18.3 Step 2 : Observe Performance
.........................................................................................................................................................................................
As Smith ( 1976 : 367 ) noted: ‘Observation and interpretation hold the key to the
establishment of eVective criteria.’ Yet, an ongoing problem in appraising people is
the lack of reliability in the observation of their behavior (Ronan and Prien 1971 ;
Spool 1978 ). This unreliability is largely attributed to well-known rating errors such
as ‘Wrst impressions,’ ‘halo,’ and ‘similar-to-me.’ Lifson ( 1953 ) found that up to one-
third of performance measurement variance is due to rater diVerences despite the
fact that the observers had considerable experience in observing and evaluating
people in the workplace. Lance ( 1994 ) corroborated thisWnding. Experience,
however, is not a substitute for training. To solve the problem regarding lack of
reliability, an observer must be trained. In this section, training programs that have
been shown to be eVective are described, and the necessity of taking context into
account is explained.
Latham et al. ( 1975 ) developed a workshop based on three principles of learning,
namely, active participation, knowledge of results, and practice. In brief, managers
were given job descriptions, observed people on videotape, made an appraisal,
received feedback on their accuracy, discussed the rating error that was made
(e.g. ‘central tendency’), ways of overcoming each error (e.g. contrast eVects),
and then observed and appraised subsequent people on videotape. The trained
supervisors made signiWcantly more accurate ratings than those in the control
group.
The beneWt of this training on rating accuracy for the company where it was
conducted was shown in a subsequent study (Pursell et al. 1980 ). An extensive job
analysis had been conducted for use in selecting predictors for a validation study.
Yet no evidence of validity was subsequently found for any of theWve tests. This was
surprising because the tests were developed and/or selected to correspond directly
with the skills required by job incumbents. Moreover, the supervisors themselves
were involved in the development of the appraisal instrument. They even stated
that this was an excellent way of assessing an employee’s eVectiveness and that the
approach would be uncomplicated for them. This suggested that the absence of
signiWcant validity coeYcients was due to rating errors in the use of the appraisal
scale rather than to technical inadequacies in either the choice of predictors or
the appraisal instrument. Therefore, the above training program was used to
minimize rating errors. Twelve months after the initial performance evaluations,
and one month after the rater training program, the supervisors re-evaluated
the same employees. The result was signiWcant validity coeYcients for four of the
Wve predictors. These correlations were signiWcantly diVerent from those that were
previously obtained. Additional evidence that the training of the observers was
eVective was shown in the respective distribution of ratings before and after the
training. The range of criterion ratings at Time 1 was 4 – 9 while that at Time 2
372 g a r y l a t h a m e t a l.