TABLE 6-4: INTRACLASS CORRELATIONS AND INTERRATER AGREEMENT.....................................................................................................
Table 6.4 shows good Rwg scores that range from .83 to .87 and being above the threshold of .70
(James et al., 1993), support aggregation. Likewise, the ICC(1) range from .17 to .33. There are no
definitive guidelines on acceptable values ICC(1) (Ostroff & Schmitt, 1993). In past research, ICC(1)
values have ranged from 0 to .50 with a median of .12 (James, 1982). Hence, one may conclude that
the ICC(1) scores equally support aggregation. As for the ICC(2) scores, Ostroff and Schmitt (1993)
mention that ICC(2) values should be .60 or higher to conclude that acceptable levels of mean score
reliability exist. Three of the four dimensions satisfy this criteria. ICC(2) for Organizational creativity
is .57. Other studies however, have shown the a ICC(2) of >.40 is acceptable for aggregation (de
Hoogh et al., 2004; Schneider et al., 1998; Simsek et al., 2010). Overall, the indicators support
aggregation for these dimensions.
All ratings (except self-scores) were averaged across the four items of each dimension and then
aggregated across raters for each focal leader. There are several statistical approaches that have been
developed to determine the internal reliability of an instrument or its subscales. These include the
split-KDOIWHFKQLTXHDQG&URQEDFK¶VDOSKD 7RGD\&URQEDFK¶V(1951) alpha coefficient is the preferred
approach for estimating internal reliability. This measure of reliability represents the proportion of
total variance in a given scale that can be attributed to a common source (DeVellis, 1991). It is also
defined as the estimated correlation of the given scale with another scale of the same length from the
universe of possible items (Kline, 1986). An alpha of .70 or higher is generally considered to be
acceptable. The results of the internal reliability analysis (as well as the descriptive statistics and
intercorrelations between the dimensions) are shown in Table 6.5.