6 Evidence Appraisal: Research 133ferent samples of the same population. Internal validity, discussed earlier in this
chapter, refers to the extent to which inferences regarding cause and effect are
true. True experiments have a high degree of internal validity because manipu-
lation and random assignment enables researchers to rule out most alternative
explanations of results (Polit & Beck, 2017). In experimental designs, researchers
still need to consider contamination between treatments. Subject mortality may
affect the study. There may also be different dropout rates between experimental
and control groups. This may be a threat particularly if the experimental treat-
ment was painful or time-consuming. Participants remaining in the study may
differ from those who dropped out. Even with strong research designs, it is im-
portant to assess the nature of possible biases. Selection biases should be assessed
by comparing groups on pretest data (Polit & Beck, 2017). If there are no pretest
measures, groups should be compared on demographic and disease variables
such as age, health status, and ethnicity. If there are multiple points of data col-
lection, attrition biases are important to assess by comparing those who did or
did not complete the intervention. EBP groups want to analyze carefully how the
researcher addresses possible sources of bias.
External validity refers to the extent that results will hold true across different
subjects, settings, and procedures. The most frequent criticism by clinicians of
RCTs is lack of external validity. Results from an RCT that has internal validity
may be clinically limited if it lacks external validity—that is, if results cannot be
applied to the relevant population. Questions a team may pose to uncover po-
tential threats to external validity include, “How confident are we that the study
findings can transfer from the sample to the entire population? Are the study
conditions as close as possible to real-world situations? Did subjects have inher-
ent differences even before manipulation of the independent variable (selection
bias)? Are participants responding in a certain way because they know they are
being observed (the Hawthorne effect)? Are there researcher behaviors or charac-
teristics that may influence the subject’s responses (experimenter effect)? In multi-
institutional studies, are there variations in how study coordinators at various
sites managed the trial?