Logistic Regression: A Self-learning Text, Third Edition (Statistics in the Health Sciences)

EXAMPLE (continued)

b 0

b 1

b 2

b 3

b 4

b 5

b 6

b 7

1. Largest CNI “large”
   (e.g., >30)


2. At least two VDPs “large”
   (e.g., ≥ 0.5)

diagnosing

collinearity

b 0

b 1

b 2

b 3

b 4

b 5

b 6

b 7

Diagnosing collinearity conceptually
Computer software for nonlinear
models

Collinearity objective:

 Determine if fitted model is
unreliable
,
 Determine whether Varð^bjÞis
“large enough”

Estimated Variance – Covariance Matrix

b 3 )

= ˆ

Var(

Var(bˆ 2 )

Var(bˆ 1 )

Var(bˆ 3 )

Vˆ

= I–1^ for nonlinear models

Covariances

Covariances

One popular way to diagnose collinearity uses

a computer program or macro that produces a

table (example shown at left) containing two

kinds of information,condition indices(CNIs)

andvariance decomposition proportions(VDPs).

(See Kleinbaum et al., Applied Regression and

Other Multivariable Methods, 4th Edition,

Chap. 14, 2008 for mathematical details about

CNIs and VDPs)

Using such a table, a collinearity problem is

diagnosed if the largest of the CNIs is consid-

ered large (e.g.,>30) and at least two of the

VDPs are large (e.g.,0.5).

The diagnostic table we have illustrated indi-

cates that there is at least one collinearity prob-

lem that involves the variablesE,C 3 andEC 3

because thelargest CNI exceeds 30and two of

theVDPs are as large as 0.5.

We now describe briefly how collinearity is

diagnosed conceptually, and how this relates

to available computer software for nonlinear

models such as the logistic regression model.

The objective of collinearity diagnostics is to

determine whether (linear) relationships

among the predictor variables result in a fitted

model that is “unreliable.” This essentially

translates to determining whether one or more

of the estimated variances (or corresponding

standard errors) of the b^j become “large

enough” to indicate unreliability.

The estimated variances are (diagonal) compo-

nents of the estimated variance–covariance

matrixðV^Þobtained for the fitted model. For non

linear models in which ML estimation is used,

theV^matrix is calledthe inverse of the informa-

tion matrix(I^21 ), and is derived by taking the

second derivatives of the likelihood function (L).

Presentation: IV. Collinearity 271