There should always be a good theoretical rationale for linking error variances in
this way^13 (Kline 2005: 318) and models that include error covariances are often
criticised for ‘capitalising on chance’ (see e.g. Kline 2005: 147). In this context,
capitalising on chance refers to introducing error covariances – i.e. introducing
unmeasured and un-theorised linkages between variables - to satisfy the
observed data, rather than modifying the model substantively i.e. changing the
content (and therefore the meaning) of the model based on theory.
In adopting a model generating approach, this research is somewhat exploratory,
and, by its exploratory nature, is to some extent inherently capitalising on
chance! Put another way, while the meanings and structures of the models are
based in theory, the development of the models is guided to some degree by the
nature of the collected data. Nevertheless, the use of covaried error terms is
avoided because, not only does this technique capitalise on chance, it also
introduces unmeasured (and un-theorised) variance into the model.
Accordingly, for this research, the modification indices are used in a more
circumspect (cautionary) way. That is, when models have been modified as far as
is possible using evidence from the magnitudes of factor loadings and the
standardised residual covariance matrices, high MI values are then used to flag
pairs of variables for inspection. In particular, flagged pairs of variables are
examined to see if there is a large degree of overlap in item content – an
application of modification indices described by Byrne (2010: 110). Where
content overlap is identified, the weaker performing variable is removed and the
model re-estimated to see if model fit has improved. Indicator variable
performance is judged by comparing factor loadings, or, where factor loadings are
very similar, by comparing the total magnitude standardised residual covariances
(SRCs) associated with each indicator variable^14. The variable with the greatest
associated SRC (in other words, the part of the model that is less well-explained
by the data) is then removed.
13
For example, it is uncommon to link the paths between error terms in two separate
constructs. During the descriptions of model modifications to follow, the phrase
‘theoretically plausible’ will be used to distinguish between MI values that indicate
potentially acceptable modifications and those that would make no substantive sense.
(^14) SCR values can be both positive and negative, with deviations from zero in either
direction signalling unmeasured variance. Accordingly, to compare magnitude of SRCs
between variables, all SRC values were rendered positive by being raised to power of
2 (i.e. they were squared).