220 Recent Developments in Density Forecasting
consider the two degrees-of-freedom LR test referred to above, whereq(z∗it|t−h)=
φ
[(
z∗it|t−h−μ)
/σ]
/σ.5.4.2.3 Scoring rules
In contrast to evaluation based on thepit’s there is a tradition, particularly within
meteorology, of employing scoring rules (see Gneiting and Raftery, 2007, for a
review, and Hall and Mitchell 2007; Amisano and Giacomini, 2007; Adolfson,
Linde and Villani, 2007, for applications in economics). Scoring rules are (specific)
loss functions that assign a numerical score based on the density forecast and the
subsequent realization of the variable. They evaluate relative, but not absolute,
density forecast performance. Gneiting, Balabdaoui and Raftery (2007) provide a
related discussion of thesharpnessof density forecasts, which refers to the concen-
tration of the density forecast, and argue that, subject to correct calibration, the
sharper the better.
Following the aforementioned applied papers, we restrict attention to the loga-
rithmic scoring rule:S(g(yt|it−h),yt)=lng(yt|it−h), where the density forecast
is evaluated at the realisation of the random variable. The logarithmic scoring rule
is intuitively appealing as it gives a high score to a density forecast that provides a
high probability to the valueytthat materializes. It also conveniently relates to the
KLIC; see (5.27). When the predictive densityg(yt|it−h)is normal with mean
mitand variancevit(defined below):
S(g(yt|it−h),yt)=−0.5 ln 2π−0.5 lnvit−0.5
(yt−mit)^2
vit, (5.32)indicating that the logarithmic score depends on the conditional forecasts for both
the mean and variance. Competing density forecasts can be ranked according to
the size ofS(g(yt|it−h),yt), with higher values indicating better performance.
S(g(yt|it−h),yt)cannot be used to test the null hypothesisH 0 :KLIC
i
t−h=0,
as this can be achieved only if the practitioner specifiesf(.),orq(.)orh(.);
see (5.28).
5.4.2.4 Comparing competing density forecasts
The KLIC, and alsoS(g(yt|it−h),yt)given its relationship with the KLIC, can be
used to compare competing density forecasts; Bao, Lee and Saltoglu (2007) devel-
oped a test for equal predictive performance. It formalizes previous attempts that
visually compared alternative density forecasts according to their relative distance
to, say, the uniform distribution; e.g., see Clements and Smith (2000). Bao, Lee and
Saltoglu (2007) test is a direct generalization of tests of equal point forecast accuracy
popularized by Diehold and Mariano (1995) (DM) and extended by West (1996)
and White (2000). These tests assume some, usually a quadratic, loss function.
A test for equal density forecast accuracy of two competing (non-nested) density
forecastsg(yt| 1 t−h)andg(yt| 2 t−h), both of which may be misspecified, is