Split-sample cross-validation of the model
The most effective method of validating an SEM model is to test the model in a
strictly confirmatory mode using data from an independent sample drawn from
the same population (Blunch 2008: 98). Given the amount of time taken to
successfully collect data suitable for the exploratory model that has been
developed above, a second round of data collection for undertaking an
independent sample for a cross-validation exercise was not possible.
In the absence of an independent sample, one approach to cross-validation is to
randomly split the existing sample and to use the two resulting sets of cases to
perform multi-group SEM analyses to confirm (or refute) group invariance (Hair et
al. 2006: 819). If the models estimated using the data split into two groups
demonstrate adequate fit and there is no statistically significant difference in
parameter estimates, then it can be concluded that the model is likely to cross-
validate in an independent sample.
This procedure was carried out and the steps are summarised below:
the RANDBETWEEN function in Microsoft Excel was used to generate 213
random numbers between 2,000 and 200,000 (these limits were set
arbitrarily);
these random numbers were pasted into the SPSS database and the cases
were then sorted by ascending random number;
the sample was split into two groups (Group 1 [G1] = 106 cases and Group 2
[G2] = 107 cases) at the midway point in the ascending order of random
numbers. These groups are each just above the 100 cases lower limit for
sample size in SEM analysis described in Section 6.5.2 above;
following the steps described by Hair et al. (2006: 820-824) and Byrne (2010:
266 - 271) these two groups were used to:
o confirm that global model fit for both groups is satisfactory; and
o evaluate the equivalence of the factor loadings, structural coefficients
and factor covariances.
The loose cross-validation (where separate models are estimated using the split
samples) found that models both demonstrated satisfactory global fit:
G1: χ^2 = 98.923, d.f. = 98, sig = 0.455; RMSEA = 0.009; CFI = 0.999; and
SRMR = 0.0655; and
G2: χ^2 = 111.601, d.f. = 98, sig = 0.164; RMSEA = 0.036; CFI = 0.987; and
SRMR = 0.0700.