Palgrave Handbook of Econometrics: Applied Econometrics

790 Computational Considerations in Microeconometrics

Algorithm 15.4.1.1.1 Conditional density estimation – implementation

1. Select functionk(·)in (15.11), and choose the smoothing parameterhby cross-
   validation maximum likelihood, i.e., selecthto minimize:

L[h]=

∑n

i= 1 loĝf−i(x

c

i),

wherêf−i(·)equals (15.13) after replacing

∑n

i= 1 by

∑n

i=1;i=j.

2. Usingh, found in the previous step, calculatêf(xci)in (15.13) for eachi=
   1,...,n.


3. Plot (if feasible) or present summary statistics.

Parallelization
Since the above algorithm relies on local averaging, the computational order
of step 2 is O

(

n^2 q

)

. Data-driven methods, such as bandwidth selection by

cross-validation, performed in step 1, add an additional order of computational
magnitude. The burden increases as the amount of available data rises. These
numerical demands can potentially overwhelm the computational resources of
modern day desktop workstations. For these reasons, although approximations
are available (see, e.g., Silverman, 1982; Scott and Sheather, 1985) parallelization,
as discussed in section 15.2.3, is an attractive alternative (see, e.g., Racine, 2002).
We proceed to implement algorithm 15.4.1.1.1 for each dataset using a high-
performance computer cluster. Each implementation uses 10 multiple starts to
numerically minimizeL[h]in step 1, and 1,000 bootstrap replications of step 2 to
calculate a 95% confidence interval. The first experiment uses a single node with its
2 ×2.0 GHz quad-core Intel Xeon 5335 processor, i.e., two processors, and takes 27
minutes and 19 seconds of CPU time. The second experiment uses eight nodes with
two of the above processors each, i.e., 16 processors in total, and takes 2 minutes
and 52 seconds for completion. Computational time is reduced roughly 10 times by
parallelization.^7 Both experiments provide the exact same result, i.e., Figure 15.1.
Both distributions are left-skewed, but earnings of young individuals show a
larger right tail. Unlike their older counterparts, the distribution of average annual
earnings for 19–26-year-old individuals seems to be a mixture of at least three
heterogeneous groups.

15.4.1.2 Example: conditional density estimation

To illustrate the computational intensity of fully nonparametric methods, we con-
sider the estimation of the conditional probability density function (p.d.f.) of

vectorxcgiven another vectorxd, i.e.:

̂f(xci|xdi)≡̂f(xci,xdi)/̂p(xdi), (15.14)

=

∑n

j= 1 K(x

c

i,x

c

j;h)L(x

d

i,x

d

j;λ)

∑n

j= 1 L(x

d

i,x

d

j;λ)

.