Model Selection 301
today is an exponential average of the model rating today and model ratings
in the past. Obviously parameters must be set correctly, which again forces
a careful analysis of possible data snooping biases. Whatever the averaging
process one uses, the methodology should be carefully checked for statistical
consistency. The key principle is that averaging is used to eliminate noise,
not genuine information.
Averaging models is more difficult than averaging results. In this case,
the final result is a single model, which is, in a sense, the average of other
models.^8
Model Selection in a Nutshell
It is now time to turn all the caveats into some positive approach to model
selection. As explained in the next chapter, any process of model selection
must start with strong economic intuition. Machine learning alone is unlikely
to identify investment strategies that yield significant positive results.
Intuition applied to financial decisions clearly entails an element of
human creativity. As in any other scientific and technological endeavor, it is
inherently dependent on individual abilities. Is there a body of true, shared
science that any modeler can use? Or do modelers have to content them-
selves with only partial and uncertain findings reported in the literature? At
this time, the answer is probably a bit of both.
One would have a hard time identifying laws in financial economics
that have the status of true scientific laws. Principles such as the absence of
arbitrage are probably what comes closest to a true scientific law but are
not, per se, very useful in finding, say, profitable trading strategies.^9 Most
empirical findings in finance are of an uncertain nature and are conditional
on the structure of the financial market and financial system. It is fair to say
that intuition in finance is based on a number of broad financial principles
plus a set of findings of an uncertain and local nature. Empirical findings
in finance are statistically validated on a limited sample and probably hold
only for a finite time span. Consider, for example, findings such as volatility
clustering in asset returns. One might claim that volatility clustering is ubiq-
uitous and that it holds for every financial market. In a broad sense this is
(^8) Shrinkage of the covariance matrix used in computing portfolio variance is a simple
example of averaging models.
(^9) In finance there is arbitrage if the same financial product can be traded at different
prices in different locations at the same moment. Arbitrage, if it exists, allows one to
make unlimited profit, as unlimited quantity of the product can be bought where it
is cheap and sold where it is expensive.