Advances in the Canine Cranial Cruciate Ligament, 2nd edition

356 Future Directions

of health assessment instruments has increas-
ingly made its way to the veterinary litera-
ture over the past decade. Outcome measure
development programs focusing on the valida-
tion of owner-completed assessments, such as
the ACVS Outcome Measures Project (Schulz
et al. 2006; Brown 2014a; Brown 2014b; Brown
2014c), raise the level of awareness that, if devel-
oped appropriately, owner completed outcome
assessment instruments for chronic pain, func-
tion, and quality of life, those outcomes that are
most important to the owner and patient, can
deliver valid, reliable, and compelling clinically
relevant data.

The stepwise development of health

measurement instruments

Step 1: Devising the items

The first step in developing an instrument
(questionnaire) involves devising the items
(questions) (McDowell 2006; Streiner & Nor-
man 2014). Two of the more relevant tech-
niques for developing questions in a rigor-
ous and systematic manner include the use
of focus groups and key-informant interviews
with owners and/or caregivers of animals that
have the health condition to be measured. Par-
ticular attention is paid to the clarity, read-
ability, and format of both the questions and
the response options. Devising the items is an
underappreciated process that greatly impacts
the validity, reliability, and ultimate utility of
the questionnaire.

Step 2: Selecting the items

Not all items originally developed are ulti-

mately included in the instrument (McDowell

2006; Streiner & Norman 2014). Some may be

confusing with variable interpretations. Even

seemingly straightforward questions such as

“Does your dog have difficulty going up and

down the stairs?” can pose problems. This is

a double-barreled question that asks two ques-

tions at the same time. How would an owner

answer if their dog has difficulty going up, but

not down? Once the items have been screened

for readability and ambiguity, they can be pre-

sented to a panel of ‘experts’ to determine

whether the items look reasonable. They deter-

mine whether the items appear, on the surface,

to be measuring what is intended (face valid-

ity) and whether the questions cover all of the

relevant aspects of the condition (content valid-

ity). Once a preliminary set of items is agreed

on, the instrument is pretested in a small group

of respondents. An inter-item correlation matrix

is analyzed to identify items with consistently

low correlations with other items in the instru-

ment. These items are revised and the prelim-

inary instrument pretested again, or the items

are removed before moving onto large-scale

testing for reliability and validity.

Step 3: Reliability and validity testing

Once an instrument is developed, it must be

established that it is valid. It should target what

it is intended to measure, and be reliable and

measure in a consistent manner (Figure 43.1)

Not Reliable or Valid Reliable but Not Valid Both Reliable & Valid

Figure 43.1 Understanding Reliability and Validity in the development of outcome assessment instruments. An
instrument that is reliable measures in a consistent manner. An instrument that is valid measures the parameter that it is
intended to measure. Instruments must have evidence of both validity and reliability to be appropriate for use in clinical
research.