Stephen G. Hall and James Mitchell 2135.4.1 Interval forecasts
A “good” interval forecast should, at a minimum, have correct coverageex post;
i.e., the outturn should fall in the interval the predicted proportion of times: for
example, on 95% of occasions for a 95% confidence interval. But, as argued by
Christoffersen (1998), in a time series context a “good” interval forecast should
not just have correctunconditionalcoverage, but correctconditionalcoverage, so
that in volatile periods the interval is wider than in less volatile periods. This
means that occurrences inside the interval should not come in clusters over
time. This is analogous to expecting independence of orders greater than or
equal tohwhen evaluating a sequence of rolling optimalh-step-ahead point
forecasts or optimal fixed-event point forecasts; e.g., see Clements and Hendry
(1998, pp. 56–62).
More formally, defineItas an indicator variable that takes the value 1 if the
outcome falls within the interval forecast at timet, and 0 otherwise. Consider an
interval forecast for coverage probabilityp,0≤p≤1. Then Christoffersen (1998)
defines a set ofex anteforecasts as having correctconditionalcoverage, or as being
“efficient” with respect to the information set (say,t− 1 ), ifE(It|t− 1 )=p.
Ift− 1 ={It− 1 ,It− 2 ,...}then this implies{It}is independent and identically
distributed (i.i.d.) Bernoulli with parameterp.
Christoffersen (1998) then suggests a likelihood ratio (LR) test for correctcondi-
tionalcoverage. Whent− 1 =∅, the empty set, the test reduces to anunconditional
test of the null hypothesis thatE(It)=p. Wallis (2003) describes an asymptotically
equivalent Pearson chi-squared test, with the advantage that, unlike the LR tests,
its exact distribution can be derived. Wallis (2003) also extends the tests to density
forecasts. The extension is based on reducing the density forecast to ak-interval
forecast; Boero, Smith and Wallis (2004) explore, as a function of the size ofk,
the properties of the chi-squared test in small to moderate sample sizes typical to
macroeconomics.
Christoffersen (1998) also suggested, and Clements and Taylor (2003) refined,
regression-based tests of interval forecasts. They involve estimating:
It=α+βt− 1 +εt, (5.21)where the set of interval forecasts are conditionally efficient whenα =pand
β = 0, implying that, as before, E(It | t− 1 ) = p. Similarly to Mincer–
Zarnowitz regressions, these regression-based tests distinguish between conditional
and unconditional objectives. The forecasts have correct unconditional coverage
whenα=p, and are conditionally efficient when the forecast “errors” are uncorre-
lated with information available at the time the forecast was made, i.e.,α=pand
β=0.
Equation (5.21) also serves as the basis for tests of probability event forecasts; see
Clements (2004). Considerpt|t− 1 to be the probability forecast made one period
ahead of an event (such as a breach of the inflation target,a) happening at time
t;pt|t− 1 =P(yt≥a|t− 1 ). Conditional efficiency,E
[
It|t− 1]
=pt|t− 1 , then