It is possible that one of the teachers was more enthusiastic than the others and that this
was responsible for the results. One way to control for this confounding effect is to hold
the confounding variable teacher constant, that is to use one teacher for all three methods.
Any effect of teacher would be equally represented in all treatment groups, see Figure
1.2.
(^) Treatment (Group)
Silent reading Storytelling Storytelling enhanced by
pictures
Teacher 1 (35 pupils
randomly assigned to
treatment)
Teacher 1 (35 pupils
randomly assigned to
treatment)
Teacher 1 (35 pupils
randomly assigned to
treatment)
Figure 1.2: Effect of teacher controlled
The researcher may believe, based on previous studies, that storytelling enhanced by
pictures is a more effective method of teaching vocabulary than either of the other two
methods. It is posited that pictorially enhanced storytelling will have a greater effect on
pupils’ vocabulary acquisition, measured by a score representing correct understanding of
a number of target words. This vocabulary score is known as the dependent or response
variable because these response scores are dependent on the experimental manipulation
of teaching method, the independent variable.
A factor is a discrete or categorical variable which is used to define experimental
groups. In the teaching methods experiment because there are three experimental or
treatment method groups, these different groups correspond to levels of the factor
teaching method. Here there are three levels of the factor. In this example the three levels
of the factor teaching method are deliberately selected or fixed by the researcher and this
is known as a fixed-factor or fixed-effect design. Another way a factor could be fixed is
by natural circumstances not under the direct control of the researcher. Such a factor
could be sex. In both the fixed-effect example of teaching method and the fixed-effect
sex, any experimental manipulations would be the same in any replications of the
experiment because they are fixed and do not change randomly from one replication to
another. Note, however, that we could only draw general conclusions from the methods
experiment to sets of the three methods used and not from any other methods of teaching
vocabulary. In some circumstances it may be appropriate to select the levels of a factor at
random from a population of treatments or levels. This is known as a random effects
design. For example, a random sample of three ways of teaching vocabulary could have
been selected and methods selected may include peer teaching, storytelling and silent
reading in the reading corner. Given the random process whereby the teaching methods
were selected it would be unlikely that the same methods would be selected in a
replication of the experiment. Note, however, that we could draw more general
conclusions about the effectiveness of teaching methods if our experimental design were
based on a random effects design. If data from a fixed-effect design were analyzed by
analysis of variance procedures this would be referred to as a fixed-model ANOVA. The
term One-way ANOVA may also be used to refer to analysis of a one-factor design even
though it has in this case three treatment levels. Analysis of a random effects design
would be referred to as a random-model ANOVA. This distinction between random and
Statistical analysis for education and psychology researchers 12