41Your literature review will yield one of three results:The right assessment tools already exist. It’s possible that your literature review will
turn up a tool that measures exactly what you are trying to measure. If you are
attempting to measure “grit,” for example, you’d be in luck – your literature
review would reveal several publicly available, high-quality surveys on grit.
Likewise, if you are hoping to assess operative performance on a laparoscopic
cholecystectomy, you can find a validated tool online.
Some helpful tools exist, but they’re not an exact fit. At the very least, you probably
will find tools that partly cover what you hope to measure – you may be able to
use some items from those tools – but will need to write additional questions that
suit your needs. For example, a team of researchers at Southern Illinois University
School of Medicine sought to assess skill in urological operative procedures. The
operative performance rating forms available through the American Board of
Surgery formed the basis for a rating tool, but new items needed to be developed
for urological procedures [ 3 ].
No tools exist. This is the least likely outcome. It is possible that the construct you
hope to measure has never been empirically measured, even partly. This makes
the following steps more difficult, as you’ll be designing a tool from scratch. But
it also signals an opportunity: you’re covering a topic that is entirely new in the
scholarly literature.
A literature review will take time, but it will be one of the best uses of your time.
This chapter outlines how to build an assessment – a several-step process that
requires a great deal of work. It’s tempting to skip the literature review to begin
work on those steps. In reality, however, skipping the literature review would almost
certainly create more work than it would save. It would be a waste to build an
assessment tool if all along an adequate tool already existed.
When searching the literature, you will probably discover that others have
attempted to measure the skills, attitudes, or knowledge that you are trying to mea-
sure. Whenever this is the case, recall the advice that was given at the beginning of
this chapter. Find the assessment tool used, and take it for yourself.
After examining a particular assessment tool, you may feel that it suits your
needs. However, if you find a tool that you think works, be sure to ask yourself
whether it measures exactly what you want to measure. A common pitfall in assess-
ment – and in survey-based work more generally – is when researchers use tools
that seem “good enough.” If you’re reading this textbook, it is likely because you
have specific educational goals and concerns about your learners. Your assessment
should be aligned to those goals and concerns as closely as possible.
Reviewing the literature will accomplish at least two goals: you will learn how
people have tried to measure your construct in the past, and at the same time, you
will gain greater knowledge about how researchers think about the construct. After
conducting a literature review, it’s important to consider updating the definition of
your construct – adding components that other authors identified and perhaps elimi-
nating components that seem at odds with the literature.
4 Measurement in Education: A Primer on Designing Assessments