107Powers20165 surgeonsPGY3 and PGY 4 urology residents
Attending surgeonsIntraoperative renal artery and vein dissection during live robotic partial nephrectomyGEARSaNovel five-point scale for hilar dissectionTurk workers: 11 h, 33 min Surgeons: 13 daysSurgeon-level GEARS ratings: Pearson’sr = 0.84Video-level GEARS ratings: Pearson’sr = 0.82Force sensitivity domain: Pearson’s r = 0.35 Pearson’sr forremainder of GEARS domains = 0.76–0.83 Renal artery specific rating: Pearson’sr = 0.83N/AGhani2016Practicing urologists in the state of Michigan, enrolled in Michigan Urological Surgery Improvement Collaborative (MUSIC)Live video from nerve-sparing robotic-assisted laparoscopic radical prostatectomyBladder neck dissection
Apical dissection
Nerve sparing
Urethrovesical anastomosisGEARSaRobotic Anastomosis and Competency Evaluation (RACE) for anastomosis performanceSummary pass/failTurk workers: 21 h for GEARS; 38 h for RACE Surgeons: 15 days for both ratingsGEARS: Pearson’s r = 0.78 RACE: Pearson’s r = 0.74 Identical evaluation of relative passing rate for each surgeonN/A(continued)6 Crowdsourcing and Large-Scale Evaluation