278
Program directors are twice as likely to get faculty input about an underperform-
ing resident verbally as via written evaluations [ 10 ]. Consider this scenario:
Numeric rotational evaluation ratings often fail to provide accurate feedback
regarding resident performance; however, information from the comments section
can be more valuable. This underscores the importance of establishing a multifacto-
rial assessment system that solicits performance feedback through multiple sources
and evaluators. Even when supervisors recognize underperformance in a resident,
they are often reluctant to rate a resident as unsatisfactory or even lower numerically
in the performance range [ 11 , 12 , 21 ]. Schwind et al. [ 12 ] reviewed all rotation
evaluations for surgical residents over a 5-year period at one institution. Less than
1% of the individual evaluations even nominally noted a deficit, while 28% of the
residents were identified as having deficits requiring some level of intervention.
There are a number of contributors to this. If the supervisor is not consistently
directly observing the resident, he or she may not observe performance deficits.
Performance in technical/operative skill, which is more likely to be directly
observed, is more accurately rated than other competency areas. Subtle deficits or
incidents may fade in the supervisor’s memory between their occurrence and the
time of evaluation. Deficits in knowledge base and clinical judgment may be masked
by the compensation of other team members. In other instances, raters feel they lack
documentation of the day-to-day observations to support a low or failing rating, lack
confidence in the validity of the overall assessment system, lack confidence in the
availability or efficacy of remediation options, or may fear repercussions in the
event of an appeal or grievance [ 11 , 12 , 21 , 28 ]. Williams et al. [ 21 ] offer the follow-
ing suggestions to maximize the accuracy of rotation ratings:
- Maximize both the number of ratings and raters to increase the situations and
tasks observed, and dilute any idiosyncrasies of individual raters. Thirty-eight
ratings per resident per year are recommended to assure a reproducible estimate
of performance. - Include nurses and patients as raters. In general, accurate assessment of inter-
personal skills requires a greater number of ratings for reliability. Nurses and
patients tend to be more accurate raters in this competency. - Use simulations and standardized observed clinical encounters liberally to
ensure observation and assessment of all aspects of performance.
You receive a call from Dr. A about a PGY2 he is supervising. Dr. A tells you
that he is very concerned about Resident B’s clinical judgment and patient
care skills. You are surprised because Resident B’s written evaluations have
consistently been in the good or above range, including those by Dr. A. You
ask the CCC to review Resident B at their next meeting, after which the CCC
chair tells you the CCC has agreed unanimously that Resident B is in need of
remediation.K. Broquet and J.S. Padmore