9.3. Probability 529
The subsetsA,Betc. of the sample spaceScan be interpreted in different
ways. Two most commonly used interpretations use the so called frequentist and
Bayesian approaches. Each of these approaches has its own pros and cons in terms of
usability and ease of application. Which approach to use is largely dependent on the
application and the personal bias of the experimenter. Since these two approaches
have real world significance and are not of just academic interest, we will spend some
time to understand them in practical terms.
9.3.A FrequentistApproach
This is the most common approach toward defining the subsets of the sample space.
Here the outcomes of a repeatable experiment are taken as the subsets. The limiting
frequency of occurrence of an eventAis then assigned to the probabilityP(A). In
simple terms it means that in this approach if we perform a repeatable experiment
then the probability will be the frequency of the outcome.
The problem with this approach is that it does not provide a platform to include
subjective information into the process, such as experimenter’s prior belief about
the behavior of the system. Consequently in the frequentist approach sometimes
it becomes harder to treat systematic uncertainties. In such cases the Bayesian
approach provides a more natural way to draw meaningful inferences.
9.3.B BayesianApproach........................
In this approach the subsets of the sample space are interpreted as hypotheses,
which are simply true or false statements. This approach actually defines a degree
of certainty to the hypothesis as opposed to the frequentist statistics in which there
are only two degrees (True impliesP(A) = 1 and False impliesP(A)=0).
Let us assume that we perform an experiment to examine the validity of a math-
ematical model (or theory). The theory gives us a degree of certainty or probability
P(T) about the outcome of the experiment. If we represent the probability of the
outcome given the theory byP(D|T), then Baye’s theorem states that
P(T|D)∝P(D|T)P(T). (9.3.1)Determination ofP(T) is one of the fundamental concerns in Bayesian statistics,
which does not provide any fundamental rules for that. Another problem is that
there could be more than one possible hypotheses and hence the right hand side
of the above equation must be summed over all the possibilities to normalize the
equation.
Both of these interpretations of probability yield almost same answers for large
data sets and therefore the choice largely depends on the personal bias of the exper-
imenter.
9.3.C Probability Density Function
The outcome of a repeatable experiment, usually referred to as arandom variable,
is not always discrete and usually can take any value within a continuous range. If
xis continuous then the probability that the outcome lies betweenxandx+dxcan