475instructors felt the URO Mentor displayed a high degree of realism, but their study
was flawed in that they never disclosed how many participants were in the study
nor how it was done [ 95 ]. However, there have been several other studies that have
demonstrated construct validity for the URO Mentor simulator. Watterson et al.
and Wilhelm et al. did similar studies in 2002, both of which verified construct
validity. In their studies, they used 20 and 21 medical students, respectively, and
randomized them to teaching on the URO Mentor system versus control groups.
Both found that the trained participants did significantly better than the control
groups (Watterson: global rating score 23.6 vs 14.7, p < 0.001; Wilhelm: 21.3 vs
16.1, p < 0.001) [ 145 , 146 ].
Jacomides et al. studied the completion time of training modules on the URO
Mentor for 16 medical students and 16 urology residents. They discovered that the
students significantly decreased their completion times of the module after training
on the URO Mentor for 5 h. However, they found no significant difference in the
completion times among the residents. Notably, they found the medical students
were able to complete the task in similar times to first-year residents, who had a
median 14 clinical URS procedures after training [ 147 ]. This is significant in that
medical students may be able to bypass the early learning curve and catch up to resi-
dents in terms of operating times by using the VR simulator. Matsumoto et al. fur-
ther exhibited construct validity by assessing 16 urology residents using several
parameters in the task of basketing a distal ureteral stone on the URO Mentor. Their
study found that senior residents scored significantly better than junior residents in
terms of global rating scores, examiner checklist assessment, pass/fail rating, time
to complete task, and incidence of scope trauma [ 148 ]. In a study of 89 participants
that consisted of both urologists and urology residents, Dolmans et al. found that
URO Mentor scored a mean global realism score of 3.14 on a 1–5 Likert scale for
URS. Eighty-two percent of participants rated it ≥3.5 on a scale of 1–5 in terms of
usefulness as an educational tool. In this study, the overall rating for the URO
Mentor on a 10-point scale (1 = poor, 10 = excellent) was 7.3 [ 149 ].
Criterion validity for URO Mentor has also been evaluated in multiple studies.
The importance of criterion validity is that it helps answer the question if a simula-
tor can effectively translate to improve clinical performance. Ogan et al. studied 16
medical students and 16 urology residents for criterion validity on the URO Mentor.
Participants underwent a baseline evaluation on the URO Mentor, and the medical
students underwent an additional 5 h of supervised training on the simulator. After
the medical students received training, all participants then underwent a second
evaluation on the URO Mentor in addition to a similar task on a fresh-frozen cadaver.
The study found that the medical students significantly improved performance from
their baseline assessment to their second simulated task, but they still underper-
formed against the residents in the cadaveric URS in multiple subjective and objec-
tive measurements. In terms of criterion validity, the student performance on the
post-training simulation strongly correlated with performance on the cadaver in
areas of time =, global rating score anatomy, and overall scores. Unfortunately,
these correlations did not hold for urology residents. This suggests that the URO
Mentor is helpful in predicting the performance of inexperienced endoscopists, but
24 Simulation in Surgery