Factor Analysis and Principal Components Analysis 243
N × q matrix of factor loadings B by the matrix inverse of T, we obtain a
new N × q loading matrix L = BT–1, such that the new factor model
yatt=+Lg+εtis observationally equivalent to the original model.
Given any set of factors ft, we can always find a nonsingular matrix T
such that the new factors gt = Tft are uncorrelated and have unit variance;
that is, Eg('ttgI)=. Uncorrelated factors are often called orthogonal factors.
Therefore, we can always choose factors such that their covariance matrix
is the identity matrix Ω=I so that equation (12.8) becomes
cov,()ΩΣ==IBB'+Ψ (12.9)
This is a counterintuitive conclusion. Given a set of observable vari-
ables, if they admit a factor representation, we can always choose factors
that are mutually uncorrelated variables with unit variance.
In a strict factor model, the correlation structure of observed variables is
due uniquely to factors; there is no residual correlation. For a large covari-
ance matrix, this is a significant simplification. For example, in order to esti-
mate the covariance matrix of 500 stock return processes as in the S&P 500
universe, we have to estimate (500 × 501)/2 = 125,250 different entries
(covariances). However, if we can represent the same return processes with
10 factors, from equation (12.9) we see that we would need to estimate only
the 500 × 10 = 5,000 factor loadings plus 500 diagonal terms of the matrix Ψ.
This is a significant advantage for estimation. For example, if our sam-
ple includes four years of daily return data (250 trading days per year or
1,000 observations for four years), we would have approximately 500 ×
1,000 = 500,000 individual return data. If we have to estimate the entire
covariance matrix we have only 500,000/125,250 ≈ 4 data per estimated
parameter, while in the case of a 10-factor model we have 500,000/5,500 ≈
90 data per parameter.
We can now compare the application of regression models and factor
models. If we want to apply a multiple regression model we have to make
sure that (1) regressors are not collinear and (2) that residuals are serially
uncorrelated and uncorrelated with all the observed variables. If we want
to apply a multiple regression model, we assume that residuals are serially
uncorrelated but we accept that residuals are cross correlated.
However, if we want to investigate the strict factor structure of a set of
observed variables, we have to make sure that residuals are not only serially
uncorrelated but also mutually uncorrelated. In fact, a strict factor model explains
the correlation between variables in terms of regression on a set of common fac-
tors. We will see later in this chapter how this requirement can be relaxed.