118 CHAPTER 7 Correlation, Regression, and Logistic Regressionrelationship between a response variable and one or more explanatory
variables. The difference that distinguishes logistic regression from
other forms of regression is that there are only two possible outcomes,
and the job is to estimate the probabilities of the two possible outcomes
or the odds of the outcome of interest.
Because we have a dichotomous response variable, we use a very
different methodology from the one employed in ordinary linear regres-
sion. The text by Hosmer and Lemeshow ( 2000 ) is one of the most
readable texts devoted to logistic regression and providing instructive
examples.
In this section, we provide one simple example along with its solu-
tion. For logistic regression, we have predictor variables X 1 , X 2 ,... X k
and are interested in
E [ Y | X 1 , X 2 ,... X k ], where Y is the dichotomous outcome variable.
This expectation is a probability because Y only takes on the values 0
and 1, and so the conditional expectation is the conditional probability
that Y = 1. For simplicity, we will go through the notation when there
is only one predictor variable X in the model. Then we let
π ( x ) = E [ Y|X = x ]. Now because Y is dichotomous and π ( x ) is a prob-
ability, it is constrained to belong to (0, 1). The possible values for X
may be unconstrained (i.e., may be anywhere between − ∞ and + ∞ )
Then if we want the parameters α and β for the right - hand side of
the equation to be of the linear form α + β x when X = x , then the left -
hand side cannot be constrained to a bounded interval such as (0, 1).
So we defi ne the logit transformation g ( x ) = ln[ π ( x )/{1 − π ( x )}]. First
we note that the transformation ω ( x ) = π ( x )/{1 − π ( x )} takes values
from (0, 1) to (0, ∞ ). Then applying the logarithm transforms, it takes
values from (0, ∞ ) to ( − ∞ , ∞ ).
So the logistic regression model is g ( x ) = α + β x. The observed
values of g( X ) will have an additive random error component. We can
express this on the probability scale by inverting the transformation to
get π ( x ) = exp( α + β x )/[1 + exp( α + β x )]. To see this requires a little
basic algebra as follows: exp(ln[ x ]) = x since the natural logarithm and
the exponential function are inverse functions of each other. Now exp
( g [ x ]) = exp(( α + β x )) = exp{ln[ π ( x )/(1 − π [ x ])]} = π ( x )/{1 − π ( x )}.
So we now solve the equation π ( x )/{1 − π [ x ]) = exp(( α + β x )) for π ( x ).
So multiplying both sides by 1 − π ( x ), we get π ( x ) = { 1 − π ( x )}
exp(( α + β x )). Distributing exp(( α + β x )) on the right - hand side gives
us π ( x ) = exp(( α + β x )) − π ( x )exp(( α + β x )), and then by adding π ( x )