Paul Johnson, Steven Durlauf and Jonathan Temple 1133More fundamentally, as explained in Brock and Durlauf (2001a) and Brock,
Durlauf and West (2003), even a sophisticated extreme bounds approach is some-
what problematic when viewed from a decision-theoretic perspective. Suppose,
for example, that interest inψlderives from countryi’s consideration of a policy
change in which a variablesi,lwill be increased by one unit. LetEl
(
si,l,m)
repre-
sent the policy maker’s expected loss associated with a policy indicator in country
i, and suppose that she is only interested in the case where the increase in the
indicator will raise growth, so that the policy change is sensible only ifψˆl,m>0.
One can approximate at-statistic rule, requiring that the coefficient estimate forsl
be statistically significant, as:
El(
si,l+1,m)
−El(
si,l,m)
=(
ψˆl,m− 2 sd(
ψˆl,m))
>0, (24.16)wheresd
(
ψˆl,m)
is the estimate of the standard deviation associated withψˆl,mand
the required significance level is assumed to correspond to at-statistic of 2. As odd
as it seems, this is the form of the loss function implicitly assumed whent-statistics
are used to make policy decisions. Extreme bounds analysis requires that (24.16)
holds for everym∈Mso thatEl
(
si,l)
, the expected loss for a policy maker when
conditioning only on the policy variable, must have the property that:
El(
si,l+ 1)
−El(
si,l)
> 0 ⇒El(
si,l+1,m)
−El(
si,l,m)
> 0 ∀m, (24.17)which means that the policy maker must have minimax preferences with respect
to model uncertainty. That is, she will make the policy change only if it yields a
positive expected payoff under the least favorable model inM. Such extreme risk
aversion seems hard to justify, notwithstanding claims that individuals do indeed
assess uncertainty about models in different ways to the uncertainty that arises
within models.^6
Even when one moves away from decision-theoretic considerations, extreme
bounds analysis encounters substantial problems.^7 One practical criticism is devel-
oped carefully in Hoover and Perez (2004). Using simulations, they show that an
extreme bounds analysis can easily lead to the conclusion that many growth deter-
minants are fragile even when they are part of the DGP. Intuitively, adding more
and more irrelevant variables to such an analysis always has the potential to over-
turn any specific finding, including relationships that are part of the true DGP.
Hoover and Perez also find that the procedure has poor power properties, in the
sense that some regressors that do not matter may spuriously appear to be robust.
The first concern, that extreme bounds analysis is excessively conservative, had
already led Sala-i-Martin (1997a, 1997b) to propose less demanding criteria. These
were justified in essentially heuristic terms, with their statistical properties remain-
ing somewhat uncertain. Again using simulations, Hoover and Perez (2004) found
that these newer robustness criteria, although less demanding, could again have
poor size properties, in the sense that “true” growth determinants are still likely to
fail to be identified.