14.3 Constructing Bayesian Networks 351
14.3.6 Validating and Revising BNs
Testing and validation have always been accepted as an important part of
any development activity. This is especially true for BNs. BNs have the ad-
vantage that there is a well-developed, sophisticated mechanism for testing
hypotheses about PDs. However, this is also a disadvantage because statis-
tical hypothesis testing generally requires more than one test case, and even
with a large sample of test cases the result of a test can be equivocal. A BN
test case is usually a specific example of a stochastic inference.
There are several techniques for validating BNs:
- Test cases. When there are special cases whose answer is known, one can
perform the inference and check that the result is close to the expected
answer.
- Sensitivity analysis. This technique determines the impact of inaccura-
cies of the CPD entries by systematically varying them and recording the
effects on test cases. As one might expect, different CPD entries of BNs can
have very different sensitivities (Henrion et al. 1996; Pradhan et al. 1996).
Sensitivity analysis can also be used to focus attention on the probabilities
that need to be determined more accurately.
- Uncertainty analysis. In this technique, all of the probabilities are varied
simultaneously by choosing each one from a prespecified distribution that
reflects their uncertainties. One then records the effects on test cases. This
technique can determine the overall reliability of a BN. However, it yields
less insight into the effect of separate probabilities than is the case for
sensitivity analysis.
- Consistency checking. If a BN was developed using components as in
the OOBN methodology, then one can check that the components have
been used correctly. Software development tools make extensive use of
this technique, which is called “type checking.”
When an evaluation of a BN fails, the BN must be modified. Usually all
that is necessary is to change one or more of the CPDs. However, some-
times testing uncovers previously unsuspected dependencies, and the BN
structure must be changed. One can design a BN by starting with some
simple structure and then revising the design to make it more accurate or
simpler. This is the approach developed by Helsper and van der Gaag (2001,
2002). This methodology uses an ontology as the starting point, as in sub-
