264 The Basics of financial economeTrics
To summarize, both factor models and principal components analysis
try to find a parsimonious representation of data. Factor models assume a
statistical model for data while PCA is a pure data-reduction technique. The
two techniques are essentially equivalent in large models but there might be
significant differences in small models.
Key Points
■ (^) The statistical techniques of factor analysis and principal component
analysis are used to reduce a large number of observed variables to a
smaller number of factors or principal components.
■ (^) Principal component analysis involves creating a linear combination of
a set of explanatory variables to form a set of principal components.
■ (^) Classical factor models assume that there are a finite number of series
and a finite number of observations.
■ (^) Classical factor models assume that residuals are uncorrelated. In this
case, the variance of the data is due only to factors.
■ (^) Factor analysis is the process of estimating factor models.
■ (^) Although maximum likelihood estimation methods can be employed to
estimate the parameters of a model, factors are not uniquely determined
and cannot be estimated with these methods.
■ (^) Factor scores can be used to approximate factors.
■ (^) Factor models are similar to multiple regressions but factors are gener-
ally nonobservable and residuals are uncorrelated.
■ (^) In addition to factor analysis, the parsimonious representation of data
can be done using principal components analysis.
■ (^) PCA is a data-reduction technique, not a statistical model.
■ (^) To perform PCA, (1) the eigenvectors and eigenvalues of the covariance
matrix are computed, (2) principal components are then computed by
multiplying data and eigenvectors, and (3) a small number of principal
components corresponding to the largest eigenvectors are chosen and
data as regressions on the chosen principal components are represented.
■ (^) Principal components estimate factors only in the limit of very large
models.
■ (^) The main differences between factor analysis and PCA are (1) residuals
of PCA are correlated and (2) PCA is not a statistical model.
■ (^) In large factor models, factors can be estimated with good approxima-
tion with principal components.