Factor Analysis and Principal Components Analysis 241
content. We will define different categories of factor models, but common to
all factor models are the following two assumptions:
- Both factors and residuals are zero-mean variables
- Residuals are uncorrelated with factors, that is:
Ef()tt= 00 ,,EE()εε''= ()ftt' = 0 for any t (12.7)
A factor model is called a strict factor model if the residuals are uncorre-
lated; that is, if the covariance matrix of the residuals is a diagonal matrix.^1
A factor model is called a scalar factor model if, in addition, all variances of
residuals are identical.
Strict and scalar factor models with a finite number of samples and time
series are called classical factor models. Later in this chapter we will define a
different type of factor models called approximate factor models.
SIMILarItIeS aNd dIFFereNCeS BetWeeN
FaCtOr MOdeLS aNd LINear reGreSSION
Factor models have the same form as a multiple linear regression. How-
ever, there are two major differences between factor models and multiple
regressions:
Difference 1. Factors are unobserved variables, determined by the data,
while in multiple regressions regressors are given observed vari-
ables.
Difference 2. Although in both multiple regressions and factor models
the error terms are assumed to be serially uncorrelated, residuals of
multiple regressions can be mutually correlated while residuals of
strict factor models are assumed to be uncorrelated.To understand the difference between the two models, consider the
monthly returns of the 500 stocks comprising the S&P 500 index over the
past 30 years. A factor model would try to reveal if the monthly returns of
these 500 stocks can be explained in terms of a smaller number of hidden
factors, say two. In this case, both the model parameters and the factors
have to be estimated from the monthly return series for the 500 stocks.
Now suppose that the objective is to try to see if there is a linear relationship
(^1) For an explanation of the different types of matrices, see Appendix D.