whatsoever or statistical control of potential omitted variables. Although this is a
unique advantage, some evidence indicates that non-experimental designs can
produce results quite similar to experimental designsifthe studies are similar in
other design characteristics such as degree of attrition, type of control group, size of
pretest diVerences between groups, and degree of self-selection (Heinsman and
Shadish 1996 ).
A second approach is statistical control using regression analysis, or equivalently
when the treatment is a group variable, analysis of covariance (ANCOVA). A
problem with ANCOVA, however, is that the researcher must not only be able to
identify and include all variables that, if excluded, (signiWcantly) bias the eVect
estimate, she or he must also measure them reliably because partialing unreliable
control variables results in undercorrection for group diVerences (e.g. Cohen and
Cohen 1983 ). In the multivariate case, the most practical way to correct for
measurement error is to use a structural equation model (SEM) such as LISREL.
As also noted below in our discussion of matching and propensity scores, even if
relevant variables are included and measured without error, conclusions may still
be diYcult if the groups diVer signiWcantly on these control variables. Cochran
( 1957 ), in an example cited by Rubin ( 2001 ), uses the following example to
demonstrate this point:
suppose that we were adjusting for diVerences in parents’ income in comparison of private
and public school children, and that the private school incomes ranged from $ 10 , 000
$ 12 , 000 , while the public school incomes ranged from $ 4 , 000 $ 6 , 000. The covariance
[analysis] would adjust results so that they allegedly applied to a mean income of $ 8 , 000
in each group, although neither group has any observations in which incomes are at or near
that level. (pp.265 6)
Three other approaches to omitted variable bias may be less familiar to readers
of this chapter and are thus covered in somewhat more depth.
- 1 Propensity Scores
Fulmer et al. ( 2003 ) used matching in their study by comparing theWnancial
performance of companies on the Fortune list of 100 Best Companies to Work
For with that of a set of companies matched on industry, size, and previous
Wnancial performance. A more sophisticated approach to matching is propensity
scores (Rosenbaum and Rubin 1983 ). In the two-group case (treatment, control),
the propensity score is the probability, conditional on a set of covariates, that a
subject receives a treatment.^12 It is derived by regressing the binary treatment
(^12) Estimating the eVect of HR on performance really requires knowledge of what would have
happened to the same employees or companies had they been covered during the period by a diVerent
HR system. This is not a new idea, but it has received much attention in the statistics literature of
late, much of it around Rubin’s work (‘Rubin’s causal model’ or ‘Rubin’s model of potential
outcomes’). Much of its application has been to experimental designs where intended treatment
562 b a r r y g e r h a r t