200 MODEL RISK AND FINANCIAL DERIVATIVESinevitably carry their drawbacks. They produce discontinuities, which can
be inherent numerical artefacts or genuine jumps in the portfolio sensitivi-
ties. The uniqueness (and meaningfulness) of a numerical solution should
also imperatively be checked. Estimating the errors involved in a numerical
scheme is a hard task, and numerical errors tend to accumulate and bias the
final result.
10.5.2 Model calibration
The usefulness of a model and the value of its output are only as good as
the model’s ability to be effectively calibrated to its market environment.
In the case of backward-looking models, information is available and there
are numerous econometric techniques to estimate the necessary parameters
and calibrate a model to market data. In the case of forward-looking mod-
els, validation can only occur after the fact, that is, when the authors will
typically be unavailable. The calibration is then possible in only one way, by
back-testing, that is using the spatial and statistical properties of the past to
predict the present.
Although often neglected, the calibration stage is essential to detecting
model risk. Indeed, the theory of parameter estimation generally assumes
that the true model is known, and that the true model parameters must be
estimated using a representative set of data. Are these properties verified in
practice? Fitting a time-series model is usually straightforward nowadays
in the use of appropriate computer software. However, model errors are
likely to result in parameter instability. This was easy to observe in the case
of simpler static models with observable parameters, but as soon as a model
includes time-varying or stochastic parameters, these will absorb all the
errors and output them as a simple change in value. Things get even worse
when some of the quantities we are dealing with are pure abstractions, such
as the expected future volatility. Even if we assume that this quantity is
constant, how can we measure it?
Last but not least, instability in the calibration process can also result
from numerical problems (such as near-singular matrix inversion) or from
implementation problems: a model may require a large number of iterations
to converge (a typical problem in Monte-Carlo simulations or in solving
partial differential equations), may require a higher precision for floating-
point numbers, or may use inappropriate approximations.
10.5.3 Model usage
Finally, model risk may arise during the model usage, even though all
of the previous steps were correctly performed. For instance, some of the