Statistical Analysis for Education and Psychology Researchers

Interpretation

The estimate of the difference and the t-value (−0.68) are the same as in the previous
worked example. We would conclude that there is no significant difference between
Christian and Muslims in their mean attribution scores.

Post hoc comparisons

Multiple t-tests on a set of means should be discouraged because this will lead to a high
experiment-wise error rate, that is for all the comparisons made in the analysis of an
experiment (or survey) at least one Type I error will be made. If we assume that α′ is the
Type I error rate for a single comparison then the experimentwise error rate, EWα is
1 −(1−α′)c, where c is the number of orthogonal (independent) comparisons in the
experiment. For example, with only two comparisons, and an alpha of 5 per cent, then
EWα is about 10 per cent [1−(1−0.05)^2 =9.75%]. Most post hoc multiple comparison
procedures adjust for experimentwise error, this is one reason why multiple ordinary t-
tests should not be used.
To perform a Newman Keul’s post hoc procedure, which adjusts for experiment-wise
error in all pairwise comparisons, the following SAS code would be entered after the
MODEL statement,

means religion /snk;

Output from this procedure is shown in Figure 8.17:
General Linear Models Procedure
Student Newman-Keuls test for variable: ATTRIB1
NOTE: This test controls the type I experiment-wise error rate under the complete null
hypothesis but not under partial null hypotheses.
Alpha=0.05 df=21 MSE=6.702381
Number of Means 2 3
Critical Range 2.6919532 3.2627471
Means with the same letter are not significantly different.
SNK Grouping Mean N SET
A 18.000 8 2
A
A 17.125 8 1
A
A 16.375 8 3

Figure 8.17: SAS output for Newman-

Keul’s multiple comparison procedure

Interpretation

As expected, in this particular analysis there are no significant differences among any of
the means.

Inferences involving continuous data 325