Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

444 Sufficiency

7.6.1 BootstrapStandardErrors

Section 6.3 presented the asymptotic theory of maximum likelihood estimators
(mles). In many cases, this theory also provides consistent estimators of the asymp-
totic standard deviation of mles. This allows a simple, but very useful, summary of
the estimation process; i.e.,θˆ±SE(θˆ)whereθˆis the mle ofθand SE(θˆ) is the corre-
sponding standard error. For example, these summaries can be used descriptively as
labels on plots and tables as well as in the formation of asymptotic confidence inter-
vals for inference. Section 4.9 presented percentile confidence intervals forθbased
on the bootstrap. The bootstrap, though, can also be used to obtain standard errors
for estimates including MVUE’s.
Consider a random variableXwith pdff(x;θ), whereθ∈Ω. LetX 1 ,...,Xn
be a random sample onX.Letθˆbe an estimator ofθbased on the sample.
Supposex 1 ,...,xnis a realization of the sample and letθˆ=θˆ(x 1 ,...,xn)bethe
corresponding estimate ofθ. Recall in Section 4.9 that the bootstrap uses the
empirical cdfFˆnof the realization. This is the discrete distribution which places
mass 1/nat each pointxi. The bootstrap procedure samples, with replacement,
fromFˆn.
For the bootstrap procedure, we obtainBbootstrap samples. Fori=1,...,B,
let the vectorx∗i=(x∗i, 1 ,...,x∗i,n)′denote theith bootstrap sample. Letθˆ∗i=θˆ(x∗i)
denote the estimate ofθbasedontheith sample. We then have the bootstrap
estimatesθˆ∗ 1 ,...,θˆ∗B, which we used in Section 4.9 to obtain the bootstrap percentile
confidence interval forθ. Suppose instead we consider the standard deviation of
these bootstrap estimates; that is,

SEB=

[

1

B− 1

∑B

i=1

(θˆ∗ 1 −θˆ∗)^2 ,

] 1 / 2

, (7.6.3)

whereθˆ∗=(1/B)

∑B
i=1
θˆ∗ 1. This is the bootstrap estimate of the standard error of
θˆ.

Example 7.6.4. For this example, we consider a data set drawn from a normal
distribution,N(θ, σ^2 ). In this case the MVUE ofθis the sample meanXand
its usual standard error iss/

√
n,wheresis the sample standard deviation. The
rounded data^1 are:
27.5 50.9 71.1 43.1 40.4 44.8 36.6 53.5 65.2 47.7
75.7 55.4 61.1 39.8 33.4 57.6 47.9 60.7 27.8 65.2
Assuming the data are in the R vectorx, the mean and standard error are computed
as
mean(x); 50.27; sd(x)/sqrt(n); 3.094461
The R functionbootse1.Rruns the bootstrap for standard errors as described
above. Using 3,000 bootstraps, our run of this function estimated the standard error
by 3.050878. Thus, the estimate and the bootstrap standard error are summarized
as 50. 27 ± 3 .05.

(^1) The data are in the file sect76data.rda. The true mean and sd are: 50 and 15.