6 Evidence Appraisal: Research 125
After the EBP team has determined the level of research evidence based on study
design, team members need to assess the quality of the evidence. To best assess
quality, the team needs to understand how to interpret research evidence.
Interpreting Primary Research Evidence
To consider statistical information presented in reports of research evidence ef-
fectively, EBP team members need a basic understanding of a study’s validity, reli-
ability, and precision. Validity and reliability are the cornerstones of the scientific
method. Validity refers to the credibility of the research—the extent to which the
research measures what it claims to measure. Reliability refers to the consistency
or repeatability of measurement. For example, a patient scale is off by 5 pounds.
When the patient is weighed three times, the scale reads 137 every time (reliabil-
ity). However, the weight is not the patient’s true weight, because the scale isn’t
recording correctly (validity). Precision has to do with how to interpret statistical
measures of central tendency (mean, median, and mode) and clinical and statisti-
cal significance (p-value, confidence interval).
Measures of Validity
The validity of research is important because if the study does not measure what
is intended, the results won’t effectively answer the aim of the research. There
are two aspects of validity: internal and external. Internal validity is the degree
to which observed changes in the dependent variable are caused by the experi-
mental treatment or intervention rather than from other possible causes. An EBP
team should question whether there are competing explanations for the observed
results. Many factors pose threats to internal validity, all of which represent pos-
sible sources of bias. These include investigator bias, where a researcher subcon-
sciously influences the study participants’ responses. Another well-recognized
threat to internal validity is the Hawthorne effect, in which participants alter
their behavior because they are aware that they are being observed. Two other
biases are attrition bias and selection bias. Attrition bias refers to the loss of sub-
jects during a study and affects the representative nature of the sample. Selection
bias exists in all nonrandomly selected samples.
