Pattern Recognition and Machine Learning

(Jeff_L) #1
704 D. CALCULUS OF VARIATIONS

Figure D.1 A functional derivative can be defined by
considering how the value of a functional
F[y]changes when the functiony(x)is
changed toy(x)+ η(x)whereη(x)is an
arbitrary function ofx.
y(x)

y(x)+η(x)

x

y(x), whereη(x)is an arbitrary function ofx, as illustrated in Figure D.1. We denote
the functional derivative ofE[f]with respect tof(x)byδF/δf(x), and define it by
the following relation:

F[y(x)+η(x)] =F[y(x)] +


δF
δy(x)

η(x)dx+O( 2 ). (D.3)

This can be seen as a natural extension of (D.2) in whichF[y]now depends on a
continuous set of variables, namely the values ofyat all pointsx. Requiring that the
functional be stationary with respect to small variations in the functiony(x)gives

δE
δy(x)

η(x)dx=0. (D.4)

Because this must hold for an arbitrary choice ofη(x), it follows that the functional
derivative must vanish. To see this, imagine choosing a perturbationη(x)that is zero
everywhere except in the neighbourhood of a point̂x, in which case the functional
derivative must be zero atx=̂x. However, because this must be true for every
choice of̂x, the functional derivative must vanish for all values ofx.
Consider a functional that is defined by an integral over a functionG(y, y′,x)
that depends on bothy(x)and its derivativey′(x)as well as having a direct depen-
dence onx
F[y]=


G(y(x),y′(x),x)dx (D.5)

where the value ofy(x)is assumed to be fixed at the boundary of the region of
integration (which might be at infinity). If we now consider variations in the function
y(x), we obtain

F[y(x)+η(x)] =F[y(x)] +

∫ {
∂G
∂y

η(x)+

∂G

∂y′

η′(x)

}
dx+O( 2 ). (D.6)

We now have to cast this in the form (D.3). To do so, we integrate the second term by
parts and make use of the fact thatη(x)must vanish at the boundary of the integral
(becausey(x)is fixed at the boundary). This gives

F[y(x)+η(x)] =F[y(x)] +

∫ {
∂G
∂y


d
dx

(
∂G
∂y′

)}
η(x)dx+O( 2 )(D.7)
Free download pdf