services voluntarily be seeing their counselor?). Also, because the counselor is administering the
questionnaire, the respondents would have a tendency to want to please the interviewer. The
sample certainly suffers from undercoverage—only a small subset of the general population is
actually being interviewed. What do those not being interviewed think of the counseling?Experiments and Observational Studies
Statistical Significance
One of the desirable outcomes of a study is to help us determine cause and effect. We do this by looking
for differences between groups in an experiment that are so great that we cannot reasonably attribute the
difference to chance. We say that a difference between what we would expect to find if there were no
treatment and what we actually found is statistically significant if the difference is too great to attribute
to chance. We discuss numerical methods of determining significance in Chapters 11 – 14 .
An experiment is a study in which the researcher imposes some sort of treatment on the
experimental units (which can be human—usually called subjects in that case). In an experiment, the
idea is to determine the extent to which treatments, the explanatory variable(s), affect outcomes, the
response variable (s). For example, a researcher might vary the rewards to different work group members
to see how that affects the group’s ability to perform a particular task.
An observational study , on the other hand, simply observes and records behavior but does not
attempt to impose a treatment in order to manipulate the response. Therefore in an observational study, we
cannot address cause and effect because we have not imposed a treatment.
Exam Tip: The distinction between an experiment and an observational study is an important one.
There is a reasonable chance that you will be asked to show you understand this distinction on the
exam. Be sure this section makes sense to you.example: A group of 60 exercisers are classified as “walkers” or “runners.” A longitudinal study
(one conducted over time) is conducted to see if there are differences between the groups in
terms of their scores on a wellness index. This is an observational study because, although the
two groups differ in an important respect, the researcher is not manipulating any treatment.
“Walkers” and “runners” are simply observed and measured. Note that the groups in this study
are self-selected. That is, they were already in their groups before the study began. The
researchers just noted their group membership and proceeded to make observations. There may
be significant differences between the groups in addition to the variable under study.
example: A group of 60 volunteers who do not exercise are randomly assigned to one of the two
fitness programs. One group of 30 is enrolled in a daily walking program, and the other group
is put into a running program. After a period of time, the two groups are compared based on
their scores on a wellness index. This is an experiment because the researcher has imposed the
treatment (walking or running) and then measured the effects of the treatment on a defined
response.