binary, ordinal, and multinomial logistic regression …...ordinal logistic regression 18 the model:...

Binary, Ordinal, and Multinomial Logistic Regression for Categorical Outcomes

1

Karen Grace-Martin

©2018 Karen Grace-Martin| http://TheAnalysisFactor.com

2

1. Linear Model: Quantitative Dependent Variable • The Model

• Interpreting Coefficients

2. Binary Logistic Model: Binary Dependent Variable • The Model


3. Multinomial Logistic Model: Unordered Multi-category Dependent Variable • The Model


4. Proportional Odds Logistic Model: Ordered Multi-category Dependent Variable

• The Model


©2018 Karen Grace-Martin | http://TheAnalysisFactor.com

Outline

3 ©2018 Karen Grace-Martin | http://TheAnalysisFactor.com

1. Linear Model: Quantitative Dependent Variable

4

Dependent Variable is

• Continuous

• Unbounded

• Measured on an interval or ratio scale


Linear Model

Linear Regression

5

The Model:

𝐸 𝑌|𝑋 = 𝛽0 + 𝛽1X1 + ⋯ + 𝛽𝑘X𝑘

Parameter Estimation: Ordinary Least Squares

Model Fit: R2

Assumptions: εi ~ i.i.d N(0, σ2)


ikikiii XXXY 122110 ...

6 ©2018 Karen Grace-Martin| http://TheAnalysisFactor.com

SAT math

800700600500400300

Colle

ge

GP

A S

core

s

4.5

4.0

3.5

3.0

2.5

2.0

1.5

1.0

Linear Regression

E(College GPA) = -.03 + .20*HSGPA + .003*SATV + .002*SATM -.15*Sports -.26*Male

How to interpret coefficients:


2. Binary Logistic Model: Binary Dependent Variable

8

Two possible outcome values on each trial:

Success/Failure

Academic Warning/Not

1/0

P is the probability of success for any given student at any given value of Xs


Binary Dependent Variable

P

Binary Logistic Regression

9

The Model:

Appropriate when: Y is binary

Parameter Estimation: Maximum Likelihood

Model Fit: Deviance (-2LL)

Assumptions: trials are independent; probability of success is the same on any given trial with the same values of X


kk XXXP

PLn

...

122110

Binary Logistic Regression

10

The Model:


eβ, the odds ratio, is the ratio of the odds of two values of X one unit apart


kk XXXeYOdds

...22110)1(

kk XXXP

PLn

...

122110

http://thecraftofstatisticalanalysis.com/understanding-probability-odds-ratios-regression/











3. Multinomial Logistic Model:

Unordered Multi-category Dependent Variable

12

More than two unordered possible outcome values on each trial:

1 = Academic Warning

2 = Passed classes and transferred out or withdrew

3 = Passed classes and remained in Good Standing


Unordered Multi-category Dependent Variable

Multinomial Logistic Regression

13

The Model:

Where j is the number of categories

h=1 to j-1

k is the number of predictors

Appropriate when: Y is categorical, more than 2 categories




kkhhhh

j

h XXXP

P

...ln 22110


14

The Model:


Odds Ratios all compared to the reference outcome

Academic warning vs.

Good Standing

Transferred vs. Good Standing


kkhhhh

j

h XXXP

P

...ln 22110

kk XXXP

P222211202

3

2 ...ln

kk XXXP

P122111101

3

1 ...ln


15

The Model:


Odds Ratios all compared to the reference outcome

Academic warning vs.

Good Standing

Transferred vs. Good Standing


kkhhhh

j

h XXXP

P

...ln 22110

kk XXXP

P222211202

3

2 ...ln

kk XXXP

P122111101

3

1 ...ln

Entire set of coefficients is different


4. Proportional Odds Logistic Model:

Ordered Multi-category Dependent Variable

17

More than two unordered possible outcome values on each trial:

1 = Academic Warning

2 = Good Standing

3 = Dean’s List


Ordered Multi-category Dependent Variable

Ordinal Logistic Regression

18

The Model:

Where j is the jth ordered category

k is the number of predictors

Appropriate when: Y is ordered categories



Assumptions: Proportional odds/Parallel Lines


kkj

ij

ijXXX

F

F

...

1ln 22110

j

m

imij PF1


19

The Model:


Each Odds Ratio compares to all higher ordered categories


Academic Warning and Good Standing vs. Dean’s List (1 and 2 vs. 3)

kk

i

i XXXP

P

...

1ln 221101

1

1

kk

ii

ii XXXPP

PP

...

)(1

)(ln 221102

21

21

kkj

ij

ijXXX

F

F

...

1ln 22110

j

m

imij PF1

Academic Warning vs. Good Standing and Dean’s List (1 vs. 2 and 3)


20

The Model:


Each Odds Ratio compares to all higher ordered categories


Academic Warning and Good Standing vs. Dean’s List (1 and 2 vs. 3)

kk

i

i XXXP

P

...

1ln 221101

1

1

kk

ii

ii XXXPP

PP

...

)(1

)(ln 221102

21

21

kkj

ij

ijXXX

F

F

...

1ln 22110

j

m

imij PF1

Academic Warning vs. Good Standing and Dean’s List (1 vs. 2 and 3)

Only the intercept is different

binary, ordinal, and multinomial logistic regression …...ordinal logistic regression 18 the model:...

Documents