143identical reliability coefficients and again highlighted in need for an adequate num-
ber of evaluations for reliability [ 19 ]. The most widely used application that
approaches this idea of a single global evaluation of a resident’s performance is the
SIMPL smartphone-based application [ 20 ]. This tool asks for a global assessment
of overall performance in a case and a score for the degree of autonomy and case
complexity. With the small number of questions and smartphone-based platform,
the evaluation can be completed in very little time with minimal interruption to
normal work flow [ 21 ].
Another approach to operative performance assessment could eschew numeric
ratings altogether and rely on a verbal narrative. These verbal narratives can poten-
tially convey a greater amount of information more efficiently than numeric ratings.
This is particularly true when accounting for the types of factors such as case com-
plexity or other unusual circumstances that may be difficult to adequately capture in
a predetermined numerical score. Instead of the rater wondering whether they
should give the resident a lower score when struggling through a difficult case or a
higher score after factoring in unusual circumstances, the rater simply describes
what happened. Schwind et al. found that written comments were particularly help-
ful at identifying performance deficits compared to numerical scores [ 22 ]. This
attribute would be particularly useful for a system of immediate operative assess-
ments since one of its main goals is to recognize and help struggling residents ear-
lier. The difficulty with verbal narratives, of course, is that they can be more
time-consuming to create, tabulate, and report out in an organized fashion. However,
we probably should not favor a numerical system simply because its numbers seem
more precise and easier to understand if what is actually conveyed by those numbers
is misleading. Fortunately, technology can help with verbal narrative-based systems
as well. With voice recognition software, a faculty member could dictate a short
narrative after a case just as quickly as they could fill out a set of predetermined
numerical scores, and software could similarly help to synthesize the various narra-
tives into a cohesive report. If an assessment of operative performance has twin
goals of recording how well the trainee did and also guiding improvement, it may
make more sense to simply keep the entire discussion as a verbal narrative. Giving
a resident a simple numerical score does little to tell them how to improve. If a ver-
bal narrative can accomplish both goals efficiently, we may not need to bother tak-
ing the extra time to try to assign an arbitrary numerical rating. Ideally this would
be done as both a verbal, two-way discussion between the resident and faculty
member and recorded narrative to document the assessment.
Guiding Improvement in Operative Competency
Effective education in any setting, including the operating room, requires of course
not just evaluation and assessment but also guidance to help the learner improve and
reach their ultimate goal. Learning will always be most effective when there is a
high level of self-direction, but teachers have a great impact on how far that self-
direction will carry a student. Becoming a skilled clinician in any field and
8 Teaching in the Operating Room