FRANÇOIS-SERGE LHABITAN T 205developed, there is a tendency to either quickly adapt an older model with
or without authorized modifications (the spreadsheet syndrome), or rebuild
a new model from scratch (the blank sheet syndrome). Neither of these
approaches is efficient and both are potential model risk generators. By
storing models in a common library, by documenting them and knowing
their users, a significant amount of time and money can be saved and model
risk is significantly reduced.
An up to date inventory will also help in understanding and explaining
internal divergences. For instance, traders, risk managers and back-offices
frequently use different models. This leads to an internal control problem
and opens the door to conflicts regarding unexplained profit-and-loss dif-
ferences. Although it would be preferable that all of them rely on the same
approved model, this is rather wishful thinking, because their needs are fun-
damentally different. Knowing who uses what can significantly help solve
such internal control problems.
Rule 3: Define a model-testing framework
This may appear as a tautology, but each financial institution should estab-
lish a complete and rigorous model-testing framework. Too often, model
testing is limited to proofing some mathematical formulations and enter-
ing a few parameters in a spreadsheet to observe the model’s output.
This is clearly insufficient. Data mining techniques make it easy to obtain
statistical proofs of nearly any relationship by selecting an appropriate
historical dataset. Therefore, a rigorous model-testing framework should
include:
Adedicated model validation team, which should be independent of both
the models’ developers and final users to ensure impartiality and elim-
inate the operational risk embedded in the implementation of a model.
Independent assessment is the only way to provide a welcome degree
of comfort, useful suggestions and improvements, and avoid the set of
incentives to realize profits early.
A precise framework to guide all persons involved in models validation.
This should include a standardized series of test procedures and data sets,
as well as minimum precise requirements to qualify a model as acceptable
(the model risk metric). These should not be considered as exhaustive,
but rather as minima. For instance, whatever the option pricing model,
a deep in the money call option should behave like a forward, while a
deep out of the money call should behave like a zero-coupon bond.
A clear formalization of internal responsibilities for validation. As a rule,
if somebody is supposed to do it, nobody will do it.