Se and 1Sp both focus on predicted
cases
If good discrimination, would expect
Se> 1 Sp for allcp
C. Pick a case and a noncase at random: what is
probability thatP^ðXcaseÞ>P^ðXnoncaseÞ?
i. One approach: “collectively” determine
whether Se exceeds 1Sp over several cut-
points ranging between 0 and 1.
ii. Drawback: Se and Sp values are “summary
statistics” over several subjects.
iii. Instead: use proportion of case, noncase pairs
for which^PðXcaseÞP^ðXnoncaseÞ.
III. Receiver Operating Characteristic (ROC) Curves
(pages 354–358)
A. ROC plotssensitivity(Se) by 1 2 specifity
(1 – Sp)values over all cut points.
i. Equivalently, ROC plotstrue positive rate(TPR)
for casesbythefalse positive rate(FPR)for
noncases.
B. ROC measures how well model predicts who will
or will not have the outcome.
C. ROC provides numerical answer to question: for
randomly case/noncase pair, what is probability
thatP^ðXcaseÞP^ðXnoncaseÞ?
i. The answer: AUC¼area under the ROC.
ii. The larger the area, the better is the
discrimination.
iii. Two extremes:
AUC¼ 1 )perfect discrimination
AUC¼ 0 : 5 )no discrimination
D. Grading guidelines for AUC values:
0.901.0¼excellent discrimination (A);
rarely observed
0.800.90¼good discrimination (B)
0.700.80¼fair discrimination (C)
0.600.70¼poor discrimination (D)
0.500.60¼failed discrimination (F)
E. Complete separation of points (CSP)
i. Occurs if all exposed subjects are cases and
almost all unexposed subjects are noncases.
ii. CSP often found when AUC0.90.
iii. CSP)impossible as well as unnecessary to fit
a logistic model to the data.374 10. Assessing Discriminatory Performance of a Binary Logistic Model