Pattern Recognition and Machine Learning

2.3. The Gaussian Distribution 111

Figure 2.22 Example of a Gaussian mixture distribution

in one dimension showing three Gaussians

(each scaled by a coefficient) in blue and

their sum in red.

x

p(x)

the eruption in minutes (horizontal axis) and the time in minutes to the next erup-

tion (vertical axis). We see that the data set forms two dominant clumps, and that

a simple Gaussian distribution is unable to capture this structure, whereas a linear

superposition of two Gaussians gives a better characterization of the data set.

Such superpositions, formed by taking linear combinations of more basic dis-

tributions such as Gaussians, can be formulated as probabilistic models known as

mixture distributions(McLachlan and Basford, 1988; McLachlan and Peel, 2000).

In Figure 2.22 we see that a linear combination of Gaussians can give rise to very

complex densities. By using a sufficient number of Gaussians, and by adjusting their

means and covariances as well as the coefficients in the linear combination, almost

any continuous density can be approximated to arbitrary accuracy.

We therefore consider a superposition ofKGaussian densities of the form

p(x)=

∑K

k=1

πkN(x|μk,Σk) (2.188)

which is called amixture of Gaussians. Each Gaussian densityN(x|μk,Σk)is
called acomponentof the mixture and has its own meanμkand covarianceΣk.
Contour and surface plots for a Gaussian mixture having 3 components are shown in
Figure 2.23.
In this section we shall consider Gaussian components to illustrate the frame-
work of mixture models. More generally, mixture models can comprise linear com-
binations of other distributions. For instance, in Section 9.3.3 we shall consider
mixtures of Bernoulli distributions as an example of a mixture model for discrete
Section 9.3.3 variables.
The parametersπkin (2.188) are calledmixing coefficients. If we integrate both
sides of (2.188) with respect tox, and note that bothp(x)and the individual Gaussian
components are normalized, we obtain

∑K

k=1

πk=1. (2.189)

Also, the requirement thatp(x)  0 , together withN(x|μk,Σk)  0 , implies

πk 0 for allk. Combining this with the condition (2.189) we obtain

0 
πk
 1. (2.190)