86
Anderson et al. assessed residents with OSATS during articular fracture reductions
and found that higher OSATS scores did not correlate with restoring articular con-
gruity [ 54 ]. They concluded that OSATS may overestimate surgical skills, at least
for orthopedics. No such studies have been conducted evaluating GOALS or
GEARS.
Combined Metrics: Epistemic Network Analysis
Global rating scales, such as OSATS, do not include generalizable metrics for non-
technical skills and fail to include any metrics on errors and procedural outcomes
[ 33 ]. During a skills assessment comparing OSATS ratings with task-specific
checklist ratings and final product analysis, results have shown that OSATS did not
correlate with multiple task-related errors or predict final product quality [ 31 ].
Current assessment metrics for nontechnical skills, including teamwork, leadership,
communication, situational awareness, and decision-making, focus on these skills
in isolation of technical skills [ 55 ]. As such, performance feedback is usually one-
dimensional and fails to elucidate the connection between technical and nontechni-
cal skills or how these skills relate to procedural errors and outcomes. Consequently,
the use of checklists and global rating scales as isolated assessment tools limits the
type of feedback trainees receive and increases the risk of rating clinicians as com-
petent despite the possibility of having critical performance deficits.
Surgical competency requires mastery of numerous, widely diverse, technical,
and nontechnical skills that must be integrated seamlessly in fast-paced, stressful
environments. Technical competency represents a highly complex class of skills
2015Expert GOALS rating^1052213120(^10319)
8 18
11
16
4 2417
(^69)
15
5
510
Crowdworker C-SATS rating
15 20
Rating equality line
Pegboard best fitting line: (y = –4.03 + 1.57° x)
Suturing best fitting line: (y = –1.15 + 1.16° x)
1412
212
23
7
Fig. 5.27 Expert GOALS
rating compared to crowd
worker C-SATS ratings [ 50 ]
E.I. George et al.