Logistic Regression: A Self-learning Text, Third Edition (Statistics in the Health Sciences)

The proportion of true positives among all

cases is calledsensitivity(Se), and the propor-

tion of true negatives among all noncases is

called thespecificity(Sp). Ideally, perfect dis-

crimination would occur ifboth sensitivity and

specificity are equal to 1.

Thus,for a given cut-point, the closer both the

sensitivity and specificity are to 1, the better the

discriminatory performance(see example at left).

A drawback to measuring discrimination as

described above is that the sensitivity and spec-

ificity that results from a given cut-point may

vary with the cut-point chosen. An alternative

approach involves obtaining a summary mea-

sure based on a range of cut-points chosen for

a given model. Such a measure is available

from anROC curve.

ROCstands forreceiver operating characteris-

tic, which was originally developed in the con-

text of electronic signal detection. When

applied to a logistic model, an ROC is a plot

ofsensitivity(Se)vs. 1 specificity(1 2 Sp)

derived from several cut-points for the predicted

value.

Note that 12 Sp gives the proportion of

observed noncases that are (falsely) predicted

to be cases, i.e., 1Sp gives the proportion of

false positives(FPs). Since we want both Se

and Sp close to 1, we would like 12 Spclose

to zero, and moreover, we wouldexpectSeto be

larger than 12 Sp, as in the above graph.

ROC curves for two different models based on

the same data are shown at the left. These

graphs may be compared according to the fol-

lowing criterion:The larger the area under the

curve, the better is the discrimination. In our

example, we see that the area in Example A is

larger than the area in Example B, indicating

that the model used in Example A discrimi-

nates better than the model in Example B.

Se¼nTP=n 1 ¼ 70 = 100 ¼ 0 : 7

Sp¼nTN=n 0 ¼ 80 = 100 ¼ 0 : 8

Perfect (ideal) discrimination:

Se¼Sp¼ 1

Example: cut-point¼0.2:
Model 1: Se¼0.7 and Sp¼0.8
betterDPthan
Model 2: Se¼0.6 and Sp¼0.5

Drawback: Sensitivity and specific-
ity varies by cut-point

ROC curve:
considers Se and Sp for a
range of cut-points.

1.00

1 – specificity 1.00

x = cut-point

Example A

Sensitivity

́

́

́

́

́

1 Sp¼

falsely predicted cases

observed noncases

¼

nFP

n 0

Want:

1 Sp close to 0andSe> 1 Sp

1.00

1.00 1.00

1 – specificity 1 – specificity

Example A Example B

Sensitivity Sensitivity

1.00

́

́

́

́

́

́

́

́

́

́ ́

Key: The larger the area under the
curve, the better is theDP.

Presentation: I. Overview 349