60 1. INTRODUCTION
1.12 () www Using the results (1.49) and (1.50), show that
E[xnxm]=μ^2 +Inmσ^2 (1.130)
wherexnandxmdenote data points sampled from a Gaussian distribution with mean
μand varianceσ^2 , andInmsatisfiesInm=1ifn=mandInm=0otherwise.
Hence prove the results (1.57) and (1.58).
1.13 () Suppose that the variance of a Gaussian is estimated using the result (1.56) but
with the maximum likelihood estimateμMLreplaced with the true valueμof the
mean. Show that this estimator has the property that its expectation is given by the
true varianceσ^2.
1.14 () Show that an arbitrary square matrix with elementswij can be written in
the formwij =wSij+wAijwherewSijandwAijare symmetric and anti-symmetric
matrices, respectively, satisfyingwSij=wSjiandwAij=−wAjifor alliandj.Now
consider the second order term in a higher order polynomial inDdimensions, given
by
∑D
i=1
∑D
j=1
wijxixj. (1.131)
Show that
∑D
i=1
∑D
j=1
wijxixj=
∑D
i=1
∑D
j=1
wSijxixj (1.132)
so that the contribution from the anti-symmetric matrix vanishes. We therefore see
that, without loss of generality, the matrix of coefficientswijcan be chosen to be
symmetric, and so not all of theD^2 elements of this matrix can be chosen indepen-
dently. Show that the number of independent parameters in the matrixwSijis given
byD(D+1)/ 2.
1.15 () www In this exercise and the next, we explore how the number of indepen-
dent parameters in a polynomial grows with the orderMof the polynomial and with
the dimensionalityDof the input space. We start by writing down theMthorder
term for a polynomial inDdimensions in the form
∑D
i 1 =1
∑D
i 2 =1
···
∑D
iM=1
wi 1 i 2 ···iMxi 1 xi 2 ···xiM. (1.133)
The coefficientswi 1 i 2 ···iMcompriseDMelements, but the number of independent
parameters is significantly fewer due to the many interchange symmetries of the
factorxi 1 xi 2 ···xiM. Begin by showing that the redundancy in the coefficients can
be removed by rewriting thisMthorder term in the form
∑D
i 1 =1
∑i^1
i 2 =1
···
i∑M− 1
iM=1
w ̃i 1 i 2 ···iMxi 1 xi 2 ···xiM. (1.134)
