33Evaluation and Feedback
When introducing a new curriculum or major curricular reform to a program, it will
be important to consider the process of this introduction. It is often prudent to make
the initial introduction of the curriculum a pilot experience or study and provide
ample opportunity for faculty, administrators, and learners to provide anonymous
feedback. Once there appears to be general acceptance or minimal required revisions
of the curriculum, it can then be phased-in over a specified period of time or across
variable years of the entire curriculum. Again in this phase-in process, there should
be ample opportunity to obtain constructive feedback and evaluation of the curricu-
lum. With successful acceptance of the curriculum, it can then be fully implemented.
Even after full implementation, it is imperative that continued evaluation and feed-
back continue for the maintenance and sustainability of the curriculum.
A curriculum is a dynamic process which is continually developing based on a
closed loop including evaluation and feedback. It is this ongoing acquisition of
information from educators, learners, and administrators that provides a guide to
improvement of components, and even, of the entire curriculum. In addition, evalu-
ation results can be utilized to seek continued or additional support for a curriculum,
assess individual achievements within the program, satisfy external requirements
such as those from ACGME, and serve as a basis for scholarly activity in the devel-
opment of presentations and publications.
The evaluation process should identify users, uses, and resources utilized by the
curriculum. It should also identify specific evaluation questions and designs and
choose measurement methods and construct instruments of assessment. Ethical con-
cerns within the curriculum should be addressed and the accurate collection, analyz-
ing, and reporting of the data must be insured in this process. These evaluations may
be formative, or internal to the curriculum, and are a method for judging the worth of
a program while the program activities are forming or in progress. Equally important
is summative evaluation of the curriculum where the focus is on the outcome of a
program to determine if the overriding goals of the educational program are being
achieved. An example of formative program evaluation is: After each didactic lecture
of the ambulatory urology rotation, learners completed an evaluation form. It was
discovered that students had already learned about UTIs in the AUA online clinical
problem solving on this topic, so the lecture was replaced with one on STDs. As an
example of summative evaluation: The final evaluation for the pilot basic ultrasound
skills training program showed residents had a high level of satisfaction and learner
proficiency, so additional educational grant funding was sought to ensure continued
resources and time to maintain and expand this program.
Most of the evaluation questions should relate to specific measurable curricular
objectives for the learner, the process or outcome of the curriculum. It is helpful if
questionnaires include items that do not relate to specific objectives and are open-
ended in nature or seek a short answer response in order to detect unexpected
strengths and weaknesses within the curriculum. Providing space and opportunities
for generalized comments or observations by learners and educators can be very
illuminating.
3 Curriculum Development