115- Stefanidis D, Korndorffer JJ, Heniford B, Scott D. Limited feedback and video tutorials
optimize learning and resource utilization during laparoscopic simulator training. Surgery.
2007;142(2):202–6. - Perone J, Fankhauser G, Adhikari D, Mehta H, Woods M, Strohmeyer J, et al. Who did the
case? Perceptions on resident operative participation. Am J Surg. 2017;213(4):821–6. - Morgan R, Kauffman DF, Doherty G, Sachs T. Resident and attending perceptions of resident
involvement: an analysis of ACGME reporting guidelines. J Surg Educ. 2016;74(3):415–22. - Reznick RK. Teaching and testing technical skills. Am J Surg. 1993;165(3):358–61.
- Williams RG, Klamen DA, McGaghie WC. Cognitive, social and environmental sources of
bias in clinical performance ratings. Teach Learn Med. 2003;15(4):270–92. - Gundle K, Mickelson D, Hanel D. Reflections in a time of transition: orthopaedic faculty and
resident understanding of accreditation schemes and opinions on surgical skills feedback. Med
Educ Online. 2016;21(1):30584. - Martin JA, Regehr G, Reznick R, Macrae H, Murnaghan J, Hutchison C, et al. Objective struc-
tured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273–8. - Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, et al. A
global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg.
2005;190(1):107–13. - Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ. Global evaluative assessment of
robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol.
2012;187(1):247–52. - Hogg M, Zenati M, Novak S, Chen Y, Jun Y, Steve J, et al. Grading of surgeon technical per-
formance predicts postoperative pancreatic fistula for Pancreaticoduodenectomy independent
of patient-related variables. Ann Surg. 2016;264(3):482–91. - Shah J, Darzi A. Surgical skills assessment: an ongoing debate. BJU Int. 2001;88(7):655–60.
- Holst D, Kowalewski TM, White LW, Brand TC, Harper JD, Sorensen MD, et al. Crowd-
sourced assessment of technical skills: differentiating animate surgical skill through the wis-
dom of crowds. J Endourol. 2015;29(10):1183–8. 150413093359007 - Stefanidis D, Arora S, Parrack DM, Hamad GG, Capella J, Grantcharov T, et al. Research
priorities in surgical simulation for the 21st century. Am J Surg. 2012;203(1):49–53. - Greenberg CC, Ghousseini HN, Pavuluri Quamme SR, Beasley HL, Wiegmann
DA. Surgical coaching for individual performance improvement. Ann Surg [Internet].
2015;261(1). Available from: http://journals.lww.com/annalsofsurgery/Fulltext/2015/01000/
Surgical_Coaching_for_Individual_Performance.8.aspx. - Greenberg C, Dombrowski J, Dimick J. Video-based surgical coaching: an emerging approach
to performance improvement. JAMA Surg [Internet]. 2016;151(3):282–3. Available from:
https://doi.org/10.1001/jamasurg.2015.4442. - Howe J. The rise of crowdsourcing. Wired Mag [Internet]. 2006;14(6):1–5. Available from:
http://www.clickadvisor.com/downloads/Howe_The_Rise_of_Crowdsourcing.pdf. - Garrigos-Simon FJ, Gil-Pechuán I, Estelles-Miguel S. Advances in crowdsourcing. Cham:
Springer; 2015. p. 1–183. - Brabham DC. Crowdsourcing as a model for problem solving: an introduction and cases.
Converg Int J Res into New Media Technol. 2008;14(1):75–90. - Lévy P. Collective intelligence: Mankind’s emerging world in cyberspace [internet].
Challenges. 1997:277 p. Available from: http://portal.acm.org/citation.cfm?id=550283 - Ipeirotis P. Demographics of mechanical turk. Working Paper CeDER-10-01. 2010. http://hdl.
handle.net/2451/29585 - Ranard BL, Ha YP, Meisel ZF, Asch DA, Hill SS, Becker LB, et al. Crowdsourcing – har-
nessing the masses to advance health and medicine, a systematic review. J Gen Intern Med.
2014;29:187–203. - Cooper S, Khatib F, Treuille A, Barbero J, Lee J, Beenen M, et al. Predicting protein structures
with a multiplayer online game. Nature. 2010;466:756–60. - Kawrykow A, Roumanis G, Kam A, Kwak D, Leung C, Wu C, et al. Phylo: a citizen science
approach for improving multiple sequence alignment. PLoS One. 2012;7(3):e31362.
6 Crowdsourcing and Large-Scale Evaluation