Praktikum Pengolahan Data Ekonomi

Multinomial Logistic Regression

Program Studi Ekonomi Pembangunan
Fakultas Ekonomi dan Bisnis
UPN “Veteran” Yogyakarta
Categorical response variables
• Often we have a categorical response Y with R ≥ 3 (unordered) categories.

• Examples of such outcomes:

• Voting intention (Conservative, Labour, Liberal Democrats, or other).
• Cause of death (e.g. different types of cancer)
Multinomial logistic regression: the basics
• Idea: Extension of (binary) logistic regression – use separate logit model for
each pair of response categories.

• If Y has R categories, there are R(R-1) possible pairwise log-odds.

• However, we only need R-1 of these.

• For 3 categories, 3*(3-1)=6 possibilities
(1,2 1,3 2,3 2,1 3,1 3,2)
but we only need to model two pairs.
Multinomial logistic regression: interpretation
• Interpretation of any log-odds model depends on which response corresponds
to the numerator and the denominator
• The category corresponding to the denominator is called the reference or
baseline category and is usually:
• the first or last category
• the most ‘meaningful’ category
• the most frequent category
• not a ‘rare’ category
Multinomial logistic regression: interpretation
• For a single explanatory variable X:
Π1 Π2
log = 𝛼1 + 𝛽1 𝑋 and log = 𝛼2 + 𝛽2 𝑋
Π3 Π3

• Which outcome category (1-3) is the reference category here?

Interpretation of the parameters
• Model: log(п1/п3)= α1 +β1X log(п2/п3)= α2 +β2X
• β1 is the effect of X on the log-odds of being in category 1
instead of category 3, i.e. ‘binary’ outcome either 1 or 3.
• β2 is the effect of X on the log-odds of being in category 2
instead of category 3, i.e. ‘binary’ outcome either 2 or 3.
• Odds ratios can be obtained by calculating exp(β1) and
exp(β2)

• The baseline is category 3

Example:Cooking Fuel Choice among Indonesia's Households

• Outcome cooking fuel has three categories

1. Electricity_LPG
2. Kerosene
3. Firewood

• Our explanatory variables:

• sex
• age
• educ
• marital_status
• ln_income
• working_status
• location
. summarize
. tab cooking_fuel
Variable Obs Mean Std. dev. Min Max
cooking_fuel Freq. Percent Cum.
age 6,221 39.50683 11.96942 11 85
marital_st~s 6,221 .8886031 .3146483 0 1
Electricity/LPG 4,503 72.38 72.38
sex 6,221 .8919788 .3104322 0 1
Kerosene 332 5.34 77.72
income 6,221 2.45e+07 4.38e+07 0 1.00e+09
educ 6,221 8.311847 4.694339 0 22 Firewood 1,386 22.28 100.00

employment~s 2 6 1.414214 5 7 Total 6,221 100.00

working_st~s 6,221 .9903552 .0977408 0 1
status_ker~2 6,003 .9898384 .1002996 0 1
cooking_fuel 6,221 1.498955 .8340853 1 3
location 6,221 .572416 .4947679 0 1

ln_income 6,221 16.32503 1.496672 0 20.72327

Multinomial logistic (base outcome = Electricity/LPG (1))
mlogit cooking_fuel sex age educ marital_status ln_income working_status location
Multinomial logistic regression Number of obs = 6,221
LR chi2(14) = 1571.33
Prob > chi2 = 0.0000
Kerosene vs Electricity/LPG
Log likelihood = -3723.6805 Pseudo R2 = 0.1742
• Sex (β = 0.210, p = 0.319): Not significant. Sex does not
predict kerosene use relative to electricity/LPG.
cooking_fuel Coefficient Std. err. z P>|z| [95% conf. interval]
• Age (β = –0.008, p = 0.135): Not significant.
Electricity_LPG (base outcome) • Education (β = –0.030, p = 0.039): Each additional year
of education is associated with a 0.030 decrease in the
Kerosene
sex .2097542 .2102889 1.00 0.319 -.2024045 .6219128 relative log odds of using kerosene vs electricity/LPG.
age -.007978 .0053372 -1.49 0.135 -.0184387 .0024827 • Marital status (β = –0.620, p = 0.001): Married
educ -.0304431 .0147235 -2.07 0.039 -.0593006 -.0015855 respondents have a 0.620 decrease in the relative log
marital_status -.6195073 .1903931 -3.25 0.001 -.9926708 -.2463437
ln_income -.1101557 .0529534 -2.08 0.038 -.2139425 -.0063689
odds of using kerosene compared to electricity/LPG
working_status 14.76582 786.4889 0.02 0.985 -1526.724 1556.256 compared to those unmarried.
location -.3509729 .1187764 -2.95 0.003 -.5837703 -.1181754 • Log income (β = –0.110, p = 0.038): A one-unit increase
_cons -14.4045 786.4893 -0.02 0.985 -1555.895 1527.086
in log income is associated with a 0.110 decrease in the
Firewood relative log odds of kerosene use.
sex .2082137 .144551 1.44 0.150 -.075101 .4915283 • Working status (β = 14.77, p = 0.985): Not significant
age .0170384 .003194 5.33 0.000 .0107784 .0232985
• Location (β = –0.351, p = 0.003): Those who lived in
educ -.129065 .009428 -13.69 0.000 -.1475437 -.1105864
marital_status .0908849 .1403273 0.65 0.517 -.1841515 .3659212 urban area corresponds to a 0.351 decrease in the
ln_income -.3173701 .0307448 -10.32 0.000 -.3776289 -.2571113 relative log odds of kerosene vs electricity/LPG
working_status .8008932 .4086005 1.96 0.050 .000051 1.601735 compared to those lived in rural.
location -1.626202 .0760995 -21.37 0.000 -1.775354 -1.47705
_cons 3.818786 .639695 5.97 0.000 2.565007 5.072565
Multinomial logistic (base outcome = Electricity/LPG (1))
mlogit cooking_fuel sex age educ marital_status ln_income working_status location
Multinomial logistic regression Number of obs = 6,221
LR chi2(14) = 1571.33 Firewood vs Electricity/LPG
Prob > chi2 = 0.0000
Log likelihood = -3723.6805 Pseudo R2 = 0.1742
• Sex (β = 0.208, p = 0.150): Not significant.
• Age (β = 0.017, p < 0.001): Each additional year of
cooking_fuel Coefficient Std. err. z P>|z| [95% conf. interval] age is associated with a 0.017 increase in the relative
log odds of using firewood vs electricity/LPG.
Electricity_LPG (base outcome)
• Education (β = –0.129, p < 0.001): Each additional
Kerosene year of schooling is associated with a 0.129 decrease
sex .2097542 .2102889 1.00 0.319 -.2024045 .6219128 in the relative log odds of firewood use.
age -.007978 .0053372 -1.49 0.135 -.0184387 .0024827
educ -.0304431 .0147235 -2.07 0.039 -.0593006 -.0015855 • Marital status (β = 0.091, p = 0.517): Not significant.
marital_status -.6195073 .1903931 -3.25 0.001 -.9926708 -.2463437 • Log income (β = –0.317, p < 0.001): A one-unit
ln_income -.1101557 .0529534 -2.08 0.038 -.2139425 -.0063689 increase in log income is associated with a 0.317
working_status 14.76582 786.4889 0.02 0.985 -1526.724 1556.256
location -.3509729 .1187764 -2.95 0.003 -.5837703 -.1181754
decrease in the relative log odds of using firewood.
_cons -14.4045 786.4893 -0.02 0.985 -1555.895 1527.086 • Working status (β = 0.801, p = 0.050): Being employed
is associated with a 0.801 increase in the relative log
Firewood
sex .2082137 .144551 1.44 0.150 -.075101 .4915283
odds of firewood use (borderline significant).
age .0170384 .003194 5.33 0.000 .0107784 .0232985 • Location (β = –1.626, p < 0.001): urban area is
educ -.129065 .009428 -13.69 0.000 -.1475437 -.1105864 associated with a 1.626 decrease in the relative log
marital_status .0908849 .1403273 0.65 0.517 -.1841515 .3659212 odds of firewood vs electricity/LPG.
ln_income -.3173701 .0307448 -10.32 0.000 -.3776289 -.2571113
working_status .8008932 .4086005 1.96 0.050 .000051 1.601735
location -1.626202 .0760995 -21.37 0.000 -1.775354 -1.47705
_cons 3.818786 .639695 5.97 0.000 2.565007 5.072565
• If you want to change the base/reference Multinomial logistic regression Number of obs = 6,221
group: LR chi2(14) = 1571.33
Prob > chi2 = 0.0000
• mlogit cooking_fuel sex age educ Log likelihood = -3723.6805 Pseudo R2 = 0.1742

marital_status ln_income
working_status location, base(3) cooking_fuel Coefficient Std. err. z P>|z| [95% conf. interval]

Electricity_LPG
sex -.2082137 .144551 -1.44 0.150 -.4915283 .075101
• The interpretation is same with our age -.0170384 .003194 -5.33 0.000 -.0232985 -.0107784
previous example. educ .129065 .009428 13.69 0.000 .1105864 .1475437
marital_status -.0908849 .1403273 -0.65 0.517 -.3659212 .1841515
ln_income .3173701 .0307448 10.32 0.000 .2571113 .3776289
working_status -.8008932 .4086005 -1.96 0.050 -1.601735 -.000051
location 1.626202 .0760995 21.37 0.000 1.47705 1.775354
_cons -3.818786 .639695 -5.97 0.000 -5.072565 -2.565007

Kerosene
sex .0015405 .2369667 0.01 0.995 -.4629058 .4659868
age -.0250164 .0058052 -4.31 0.000 -.0363943 -.0136385
educ .098622 .0163539 6.03 0.000 .0665689 .130675
marital_status -.7103922 .2176895 -3.26 0.001 -1.137056 -.2837286
ln_income .2072144 .0553539 3.74 0.000 .0987227 .3157061
working_status 13.96493 786.489 0.02 0.986 -1527.525 1555.455
location 1.275229 .1325737 9.62 0.000 1.015389 1.535069
_cons -18.22328 786.4895 -0.02 0.982 -1559.714 1523.268

Firewood (base outcome)

Relative Risk Ratios (RRR)
• Why Use Relative Risk Ratios (RRR) Instead of Coefficients?
• Coefficients are in log-odds units
• Hard to interpret directly
• Not intuitive for applied audiences
• RRR = exp(β) (exponentiated coefficient)
• Translates log-odds into odds ratios
• Shows multiplicative effect of a predictor on the odds of choosing category
j vs the base outcome
• Easier to communicate
• RRR > 1 → predictor increases odds
• RRR < 1 → predictor decreases odds
• RRR = 1 → no effect
• mlogit cooking_fuel sex age educ marital_status ln_income working_status location, rrr
Multinomial logistic regression Number of obs = 6,221
LR chi2(14) = 1571.33
Prob > chi2 = 0.0000
Log likelihood = -3723.6805 Pseudo R2 = 0.1742

cooking_fuel RRR Std. err. z P>|z| [95% conf. interval]

Electricity_LPG (base outcome)

Kerosene
sex 1.233375 .259365 1.00 0.319 .8167645 1.862487
age .9920538 .0052948 -1.49 0.135 .9817303 1.002486
educ .9700156 .014282 -2.07 0.039 .9424234 .9984157
marital_status .5382096 .1024714 -3.25 0.001 .3705856 .7816535
ln_income .8956946 .0474301 -2.08 0.038 .8073948 .9936513
working_status 2586520 2.03e+09 0.02 0.985 0 .
location .7040029 .0836189 -2.95 0.003 .5577913 .8885402
_cons 5.55e-07 .0004364 -0.02 0.985 0 .

Firewood
sex 1.231476 .1780111 1.44 0.150 .9276498 1.634813
age 1.017184 .0032488 5.33 0.000 1.010837 1.023572
educ .8789168 .0082865 -13.69 0.000 .8628248 .895309
marital_status 1.095143 .1536784 0.65 0.517 .8318098 1.441842
ln_income .7280612 .0223841 -10.32 0.000 .6854848 .7732821
working_status 2.22753 .9101697 1.96 0.050 1.000051 4.961635
location .1966751 .0149669 -21.37 0.000 .1694234 .2283103
_cons 45.54889 29.1374 5.97 0.000 13.00075 159.5832

Note: _cons estimates baseline relative risk for each outcome.

 Delta-method

Marginal Effects
dy/dx std. err. z P>|z| [95% conf. interval]

sex
_predict
1 -.0329881 .0201224 -1.64 0.101 -.0724272 .006451
2 .0082119 .0104602 0.79 0.432 -.0122898 .0287135
3 .0247762 .0186218 1.33 0.183 -.0117218 .0612742
. margins, dydx(*)
age
Average marginal effects Number of obs = 6,221 _predict
Model VCE: OIM 1 -.0017172 .0004569 -3.76 0.000 -.0026127 -.0008217
2 -.0005922 .0002654 -2.23 0.026 -.0011123 -.0000721
dy/dx wrt: sex age educ marital_status ln_income working_status location 3 .0023094 .0004072 5.67 0.000 .0015114 .0031075

1._predict: Pr(cooking_fuel==Electricity_LPG), predict(pr outcome(1)) educ

_predict
2._predict: Pr(cooking_fuel==Kerosene), predict(pr outcome(2))
1 .016558 .0012705 13.03 0.000 .0140679 .0190482
3._predict: Pr(cooking_fuel==Firewood), predict(pr outcome(3)) 2 -.0000834 .0007137 -0.12 0.907 -.0014822 .0013154
3 -.0164747 .0011588 -14.22 0.000 -.0187458 -.0142035

Delta-method marital_status
dy/dx std. err. z P>|z| [95% conf. interval] _predict
1 .0133804 .0193552 0.69 0.489 -.024555 .0513158
sex 2 -.0321691 .0095414 -3.37 0.001 -.05087 -.0134682
3 .0187887 .0180395 1.04 0.298 -.016568 .0541455
_predict
1 -.0329881 .0201224 -1.64 0.101 -.0724272 .006451 ln_income
2 .0082119 .0104602 0.79 0.432 -.0122898 .0287135 _predict
3 .0247762 .0186218 1.33 0.183 -.0117218 .0612742 1 .042095 .0044564 9.45 0.000 .0333607 .0508294
2 -.0019798 .0025772 -0.77 0.442 -.007031 .0030714
age 3 -.0401153 .0038143 -10.52 0.000 -.0475911 -.0326394
_predict
1 -.0017172 .0004569 -3.76 0.000 -.0026127 -.0008217 working_status
_predict
2 -.0005922 .0002654 -2.23 0.026 -.0011123 -.0000721
1 -.6722337 30.72631 -0.02 0.983 -60.89469 59.55022
3 .0023094 .0004072 5.67 0.000 .0015114 .0031075 2 .7334703 39.54606 0.02 0.985 -76.77539 78.24233
3 -.0612366 8.819955 -0.01 0.994 -17.34803 17.22556
educ
_predict location
1 .016558 .0012705 13.03 0.000 .0140679 .0190482 _predict
2 -.0000834 .0007137 -0.12 0.907 -.0014822 .0013154 1 .2073551 .0094373 21.97 0.000 .1888584 .2258519
3 -.0164747 .0011588 -14.22 0.000 -.0187458 -.0142035 2 .0005889 .0054581 0.11 0.914 -.0101087 .0112865
3 -.2079441 .0086542 -24.03 0.000 -.2249059 -.1909822

marital_status
_predict
Prediction of probrability outcome for base 1
. margins, dydx(*) predict(pr outcome(1))
Interpreting the AME (Average Marginal
Average marginal effects Number of obs = 6,221 Effects) (Pr = Probability of
Model VCE: OIM Electricity/LPG)
• Age (AME = –0.0017, p < 0.001):
Expression: Pr(cooking_fuel==Electricity_LPG), predict(pr outcome(1)) Each additional year of age decreases
dy/dx wrt: sex age educ marital_status ln_income working_status location the probability of using electricity/LPG
by about 0.17 percentage points.
• Education (AME = +0.0166, p < 0.001):
Delta-method Each additional year of schooling
dy/dx std. err. z P>|z| [95% conf. interval] increases the probability of
electricity/LPG use by about 1.7
sex -.0329881 .0201224 -1.64 0.101 -.0724272 .006451 percentage points.
age -.0017172 .0004569 -3.76 0.000 -.0026127 -.0008217 • Log income (AME = +0.042, p < 0.001):
educ .016558 .0012705 13.03 0.000 .0140679 .0190482 Each one-unit increase in log income
marital_status .0133804 .0193552 0.69 0.489 -.024555 .0513158 increases probability of electricity/LPG
ln_income .042095 .0044564 9.45 0.000 .0333607 .0508294 use by about 4.2 percentage points.
working_status -.6722337 30.72631 -0.02 0.983 -60.89469 59.55022 • Location (AME = +0.207, p < 0.001):
location .2073551 .0094373 21.97 0.000 .1888584 .2258519 Moving to that location (urban) increases
the probability of electricity/LPG use by
20.7 percentage points on average. Very
strong effect.
 Prediction of probrability outcome for base 2
. margins, dydx(*) predict(pr outcome(2))
Interpreting the AMEs (Pr =
Average marginal effects Number of obs = 6,221
Model VCE: OIM
Probability of Kerosene)
• Age (AME = –0.0006, p = 0.026):
Expression: Pr(cooking_fuel==Kerosene), predict(pr outcome(2)) Each additional year of age reduces
dy/dx wrt: sex age educ marital_status ln_income working_status location the probability of kerosene use by
0.06 percentage points.
• Marital status (AME = –0.032, p =
Delta-method 0.001):
dy/dx std. err. z P>|z| [95% conf. interval] Being married lowers the
probability of kerosene use by
sex .0082119 .0104602 0.79 0.432 -.0122898 .0287135
age -.0005922 .0002654 -2.23 0.026 -.0011123 -.0000721
about 3.2 percentage points,
educ -.0000834 .0007137 -0.12 0.907 -.0014822 .0013154 compared to unmarried individuals.
marital_status -.0321691 .0095414 -3.37 0.001 -.05087 -.0134682 Significant effect.
ln_income -.0019798 .0025772 -0.77 0.442 -.007031 .0030714
working_status .7334703 39.54606 0.02 0.985 -76.77539 78.24233
location .0005889 .0054581 0.11 0.914 -.0101087 .0112865
 Prediction of probrability outcome for base 3
. margins, dydx(*) predict(pr outcome(3)) Interpreting the AMEs (Pr = Probability
of Firewood)
Average marginal effects Number of obs = 6,221
Model VCE: OIM • Age (AME = +0.0023, p < 0.001):
Each additional year of age increases
Expression: Pr(cooking_fuel==Firewood), predict(pr outcome(3)) the probability of using firewood by
dy/dx wrt: sex age educ marital_status ln_income working_status location 0.23 percentage points.
• Education (AME = –0.0165, p < 0.001):
Each extra year of schooling reduces
Delta-method
the probability of using firewood by
dy/dx std. err. z P>|z| [95% conf. interval]
about 1.6 percentage points.
sex .0247762 .0186218 1.33 0.183 -.0117218 .0612742 • Log income (AME = –0.040, p < 0.001):
age .0023094 .0004072 5.67 0.000 .0015114 .0031075 Each one-unit increase in log income
educ -.0164747 .0011588 -14.22 0.000 -.0187458 -.0142035 reduces the probability of using
marital_status .0187887 .0180395 1.04 0.298 -.016568 .0541455 firewood by 4.0 percentage points.
ln_income -.0401153 .0038143 -10.52 0.000 -.0475911 -.0326394
working_status -.0612366 8.819955 -0.01 0.994 -17.34803 17.22556 • Location (AME = –0.208, p < 0.001):
location -.2079441 .0086542 -24.03 0.000 -.2249059 -.1909822 Living in urban location reduces the
probability of firewood use by about
21 percentage points.
Marginal effects for categorical variable
• margins [Link], predict
(outcome(1))
• marginsplot, name(location)
• margins [Link], predict
(outcome(1))
• marginsplot, name(sex)
• margins i.marital_status,
predict (outcome(1))
• marginsplot,
name(marital_status)
• margins i.working_status,
predict (outcome(1))
• marginsplot,
name(working_status)

• graph combine location sex

marital_status working_status,
ycommon
Thanks!
RodhiahUmaroh, [Link].