316 The Basics of financial economeTrics
Index. Assume that at the beginning of each period the best model actu-
ally describing the return in the period is known to the portfolio manager.
Using this information, a portfolio consisting of the predicted top quartile is
formed. The excess return from this portfolio generated with perfect infor-
mation would suggest the extent of return contribution from model estima-
tion. Accordingly, based on the results of a 2010 study^15 shown in Table
15.3, the annual mean excess return of the top predicted quartile is between
12% and 26%, depending on the length of the investment horizon.
In contrast, the annual mean excess return of the actual top quartile in
the S&P 500 is between 42% and 121%. The difference in excess return
between the actual top quartile portfolio and the predicted top quartile
portfolio, between 30% and 95%, would suggest the extent of the return
contribution from model prediction. It is clear then that for all investment
horizons, the return contribution from model prediction is on average two
to five times the excess returns from model estimation.
Therefore, for all practical purposes, the step of identifying a predict-
able model is responsible for the largest potential value added in generating
predictable excess returns. The implication is that resources allocated to
research should be placed disproportionally toward the effort of out-of-
sample prediction.
test the prediction again Another safeguard against data snooping is to scru-
tinize the model once more through time. That is, the conditional model to
estimate expected return needs to be tested again in a “fresh” data period.
As it requires multiple time periods to observe the conditional model for
expected returns, the prediction model derived under a single condition has
to be confirmed again. In the following figure, we specify the relationship in
time periods among estimation, testing, and confirmation.
The sample period:
Estimation Testing Forecast Testing Forecast
Period I Period I Period II Period II Period II Now
The sequential testing of the prediction model in the forecast period
would affirm that the condition that converts the model of actual returns
to the model of expected returns still produces an acceptable level of
performance. As the conditioning factor varies from one period to another,
(^15) Christopher K. Ma, “Nonlinear Factor Payoffs?” (Research Paper #97-5, KCM
Asset Management, Inc., 2010).