Anon

Factor Analysis and Principal Components Analysis 237

yt, t = 1,... , T of the scalar dependent variable and by T observations
[,xx 1 tp...,]',t tT= 1 , ..., of the p-vector of independent variables indexed
by time. As explained in Chapter 5, variables indexed by time are called time
series. In this setting, a linear regression is a relationship between time series.
For example, if we want to investigate whether there is a linear relationship
between the returns RIt of a stock index I, the rate of inflation INt, and the
economic growth rate GRt, we might write a linear regression equation as
follows:

RItt=+αα 01 IN++αε 2 GRtt

The theory of regression can be generalized to multiple regressions

formed by N regression equations with the same regressors and with error

terms that are serially uncorrelated but can be cross correlated

yaii=+bx 11 ++bxip piε,,iN= 1 ..., (12.1)

Assuming that all the error terms are uncorrelated with the regressors and

that no equation is a linear combination of the others, it can be demon-

strated that the usual ordinary least squares (OLS) estimators are efficient

estimators of the equations given by (12.1).

Basic Concepts of Factor Models

Suppose now that instead of the full set of dependent and indepen-

dent variables yt, x 1 t,... , xqt we are given only a multivariate time series

yt=()yy 1 tN, ..., t'. For example, suppose we are given a set of stock returns.

The question that leads to factor analysis (the process of estimating factor

models) and to factor models is whether the structure of the data, in par-

ticular the correlation structure of the data, can be simplified. With factor

analysis we try to understand if and how we can explain our variables as

a multiple linear regression on a reduced number of independent variables.

We will start by defining factor models and factor analysis and then look at

some other ways of creating factor models.

Linear factor models assume that the observed variables yt can be rep-

resented as a multiple linear regression on a number q of unobserved, or

hidden variables fit, i = 1,... , q called factors.

We can write a factor model in various forms. Here are three common

forms for representing a factor model.