Logistic Regression: A Self-learning Text, Third Edition (Statistics in the Health Sciences)

L^SS max¼

Y^40

i¼ 1

YiYið 1 YiÞ^1 Yi

For any binary logistic model:

Yi¼ 1 :YiYið 1 YiÞ^1 Yi¼ 11 ð 1  1 Þ^1 ^1 ¼ 1

Yi¼ 0 :YiYið 1 YiÞ^1 Yi¼ 00 ð 1  0 Þ^1 ^0 ¼ 1

L^SS max 1 always

Important implications for GOF

III. The Deviance Statistic

Deviance:

Dev(ββˆ) = −2 ln(Lˆc / Lˆmax)

b^¼ð^b 0 ;^b 1 ;^b 2 ;...;^bpÞ

L^c¼ML for current model

L^max¼ML for saturated model

(Note: If subjects are the unit of

analysis,

L^maxL^SSmax)

Lˆ

c closer to Lmax

ˆ

⇓

better fit (smaller deviance)

Lˆ

c =Lmax^ ⇒ –2^ ln(Lc / Lmax)

ˆ ˆˆ

Lˆ

c << Lmax ⇒ Lc / Lmax small fraction

ˆˆˆ

⇒ ln(Lˆc / Lˆmax)

⇒ –2 ln(Lˆc / Lˆmax)

perfect fit

poor fit

= –2 ln(1) = 0

large,negative

large,positive

It follows that the formula for the ML value of

the SS saturated model (L^SS max) involves sub-

stitutingYifor P(Xi) in the above formula for

the likelihood, as shown at the left.

From simple algebra, it also follows that the

expressionYiYið 1 YiÞ^1 Yiwill always be equal

to one whenYiis a (0,1) variable.

Consequently, the maximized likelihood will

always equal 1. This result has important

implications when attempting to assess GOF

using a saturated model as the gold standard

for comparison with one’s current model.

As mentioned at the beginning of this chapter,

a widely used measure of GOF is thedeviance.

The general formula for the deviance (for any

regression model) is shown at the left. In this

formula,b^denotes the collection of estimated

regression coefficients in the current model

being evaluated, L^c denotes the maximized

likelihood for the current model, and L^max

denotes the maximized likelihood for the

saturated model.

Thus, the deviance contrasts the likelihood of

the current model with the likelihood of the

model that perfectly predicts the observed out-

comes. The closer are these two likelihoods, the

better the fit (and the smaller the deviance).

In particular, ifL^c¼L^max, then the deviance is

0, its minimum value. In contrast, ifL^cis much

smaller thanL^max, then the ratioL^c=L^maxis a

small fraction, so that the logarithm of the

ratio is a large negative number and2 times

this large negative number will be a large posi-

tive number.

312 9. Assessing Goodness of Fit for Logistic Regression