Logistic Regression: A Self-learning Text, Third Edition (Statistics in the Health Sciences)

L¼L(u)
¼joint probability of observing
the data

ML method maximizes the like-
lihood functionL(u)

^u¼ð^y 1 ;^y 2 ;...;^yqÞ¼ML estimator

Thelikelihood function LorL(u)represents the

joint probability or likelihood of observing the

data that have been collected. The term “joint

probability” means a probability that com-

bines the contributions of all the subjects in

the study.

As a simple example, in a study involving 100

trials of a new drug, suppose the parameter of

interest is the probability of a successful trial,

which is denoted byp. Suppose also that, out of

thenequal to 100 trials studied, there arex

equal to 75 successful trials andnxequal

to 25 failures. The probability of observing 75

successes out of 100 trials is a joint probability

and can be described by the binomial distribu-

tion. That is, the model is a binomial-based

model, which is different from and much less

complex than the logistic model.

The binomial probability expression is shown

here. This is stated as the probability thatX, the

number of successes, equals 75 given that there

arenequal to 100 trials and that the probability

of success on a single trial isp. Note that the

vertical line within the probability expression

means “given”.

This probability is numerically equal to a con-

stantctimespto the 75th power times 1pto

the 10075 or 25th power. This expression is

the likelihood function for this example. It

gives the probability of observing the results

of the study as a function of the unknown para-

meters, in this case the single parameterp.

Once the likelihood function has been deter-

mined for a given set of study data,the method

of maximum likelihood chooses that estimator

of the set of unknown parametersuwhich max-

imizes the likelihood function L(u). The esti-

mator is denoted asu^and its components are

^y 1 ;^y 2 , and so on up through^yq.

In the binomial example described above, the

maximum likelihood solution gives that value

of the parameterpwhich maximizes the like-

lihood expressionctimespto the 75th power

times 1pto the 25th power. The estimated

parameter here is denoted asp^.

EXAMPLE

n¼100 trials

p¼probability of success

x¼75 successes

nx¼25 failures

Pr (75 successes out of 100 trials)

has binomial distribution

Pr (X¼75 |n¼100,p)

"

given

Pr (X¼75 |n¼100,p)

¼cp^75 (1p)^100 ^75

¼L(p)

EXAMPLE (Binomial)

ML solution.

^pmaximizes

L(p)¼cp^75 (1p)^25

112 4. Maximum Likelihood Techniques: An Overview