National Economic University

Chapter 3

Logit and Probit Models

Dr. Phung Minh Duc

 Contents
1. Logit and Probit Model
2. Practice
 1 Logit and Probit Model

❖ The logit/probit model is suitable for the case where the dependent
variable 𝑌 is binary.
1 𝑖𝑓 𝑦𝑒𝑠
𝑌=ቊ
0 𝑖𝑓 𝑛𝑜
❖ Examples of binary variables:
▪ Consumer economics: Whether a consumer makes a purchase
▪ Labor economics: Whether an individual participates in the labor market
▪ Agricultural economics: Whether a farmer adopts or uses organic practices,
marketing/production contracts,…
 1 Logit and Probit Model

❖ OLS regression with binary dependent variable

▪ OLS does not fit the data: 𝒀 = 𝜷𝟎 + 𝜷𝟏 𝑿𝟏 + ⋯ + 𝜷𝒌 𝑿𝒌 + 𝒖
▪ The predicted Y value can be less than 0 or greater than 1
▪ The value of Y between 0 and 1 has no meaningful interpretation.
1 Logit and Probit Model

❖ The logit or probit models is often chosen to answer the question:

What factors determine the probability that the dependent variable takes
a specific value from its set of possible values?
1 Logit and Probit Model

❖ Binary outcome models

▪ The OLS model: 𝒀 = 𝑿’𝜷 + 𝒖
in which, 𝑋 ′ 𝛽 = 𝛽0 + 𝛽1 𝑋1 + ⋯ + 𝛽𝑘 𝑋𝑘 .
▪ Binary outcome models estimate the probability that 𝑌 = 1 as a function
of the independent variables
𝒑 = 𝐏𝐫 𝒀 = 𝟏 𝑿 = 𝑮(𝑿′ 𝜷)
1 Logit and Probit Model

There are three different models depending on the function form of 𝐺(𝑋 ′ 𝛽),
including:

𝑷 𝒀 = 𝟏 𝑿 = 𝑿′ 𝜷 (Linear Probability Model)

𝐞𝐱𝐩 𝑿′ 𝜷
𝑷 𝒀=𝟏𝑿 =𝑮 𝑿′ 𝜷 𝑷 𝒀=𝟏𝑿 =
𝟏 + 𝒆𝒙𝒑 𝑿′ 𝜷
(Logit Model)

𝑷 𝒀 = 𝟏 𝑿 =  𝑿′ 𝜷 (Probit Model)
1 Logit and Probit Model

❖ Linear probability model (LPM)

▪ The Linear Probability model has the form:
𝒑 = 𝑷 𝒀 = 𝟏 𝑿 = 𝑮 𝑿′ 𝜷 = 𝑿′ 𝜷
▪ A problem with the LPM is the predicted probabilities will not be limited
between 0 and 1.
Therefore, we do not use the Linear Probability Model with binary outcome
data.
1 Logit and Probit Model

❖ Logit Model
▪ The logit model is used when the dependent variable takes two
values: 0 and 1.
▪ The general logit model has the form
′ 𝐞𝐱𝐩 𝑿′ 𝜷
𝒑=𝑷 𝒀=𝟏𝑿 =𝑮 𝑿𝜷 = (*)
𝟏+𝒆𝒙𝒑 𝑿′ 𝜷
▪ Linearizing (*) we obtain:
𝑝
ln = 𝑋 ′ 𝛽 = 𝛽0 + 𝛽1 𝑋1 + ⋯ + 𝛽𝑘 𝑋𝑘
1−𝑝
𝑝
▪ The odd ratio (or relative risk) is .
1−𝑝
1 Logit and Probit Model

❖ Note:
𝒑
𝒍𝒏 = 𝑿′ 𝜷 = 𝜷𝟎 + 𝜷𝟏 𝑿𝟏 + ⋯ + 𝜷𝒌 𝑿𝒌
𝟏−𝒑
𝑝
▪ The odds ratio (or relative risk) is and measures the probability
1−𝑝
that 𝑌 = 1 relative to the probability that 𝑌 = 0.
▪ The larger the odds ratio, the greater the probability of 𝑌 = 1 .
▪ An odds ratio of 2 means that the probability of the outcome 𝑌 = 1 is
twice as likely as the outcome of 𝑌 = 0.
▪ If 𝛽𝑘 is positive and statistically significant, then the variable 𝑋𝑘
positively affects the probability that 𝑌 = 1 occurs (i.e., it increases
the probability that 𝑌 = 1 occurs).
1 Logit and Probit Model

❖ Probit Model
▪ The Probit model is used in case the dependent variable takes two
values: 0 and 1.
▪ The general logit model has the form:
𝒑 = 𝑷 𝒀 = 𝟏 𝑿 =  𝑿′ 𝜷 = (𝜷𝟎 + 𝜷𝟏 𝑿𝟏 + ⋯ + 𝜷𝒌 𝑿𝒌 )
In this expression, (. ) denotes the standard normal distribution.
 Logit and Probit Model

❖ Estimate Method
▪ Logit and Probit models can not be estimated using the OLS method.
▪ Both Logit and Probit models are estimated using Maximum Likelihood
Estimation (MLE).
▪ The log-likehood function of the parameters and the data (𝑋𝑖 , 𝑌𝑖 ) has the
form:
𝐿𝑖 𝛽 = 𝑌𝑖 log[𝐺 𝑋𝑖′ 𝛽) + 1 − 𝑌𝑖 log[1 − 𝐺 𝑋𝑖′ 𝛽) .
1 Logit and Probit Model

❖ Marginal Effects
▪ Marginal Effects for the OLS model:
𝝏𝒑
= 𝜷𝒋
𝝏𝑿𝒋

▪ Marginal Effects for the Logit model

𝝏𝒑 ′ 𝐞𝐱𝐩 𝑿′ 𝜷
= 𝑮 𝑿 𝜷 𝜷𝒋 = 𝜷
𝝏𝑿𝒋 [𝟏+𝒆𝒙𝒑 𝑿′ 𝜷 ]𝟐 𝒋

▪ Marginal Effects for the Probit model

𝝏𝒑
= 𝑮 𝑿′ 𝜷 𝜷𝒋 =  𝑿′ 𝜷 𝜷𝒋
𝝏𝑿𝒋
 1 Logit and Probit Model

❖ Marginal Effects
▪ When estimating logit and probit models, it is common to report the
marginal effects after reporting the coefficients.
▪ The marginal effects reflect the change in the probability of 𝑌 =
1given a one-unit change in an independent variable 𝑋.
▪ The marginal effects depend on 𝑋, so we need to estimate them at
specific values of 𝑋 (typically at the means)
▪ Coefficients and marginal effects always have the same signs.
1 Logit and Probit Model

❖ Goodness of fit
▪ Assume the observations are classified into 𝐽 groups. Let 𝑛𝑗 be the
number of observations in group 𝑗. The number of observations with 𝑌 =
1 in group 𝑗 is 𝑌(𝑗), and the mean predicted probability in group 𝑗 is 𝑝(𝑗),
Ƹ
where
𝑝ො
𝑌(𝑗) = σ𝑗∈𝐽 𝑦𝑗 , 𝑝(𝑗)
Ƹ = σ𝑖∈𝐽 𝑛 𝑖
𝑗 2
𝐽 𝑌(𝑗)−𝑛𝑗 𝑝(𝑗)
ො
▪ The Hosmer-Lemeshow test statistics is: 𝐻𝐿 = σ
𝑗=1 𝑛𝑗 𝑝ො 𝑗 1−𝑝(𝑗)
ො

▪ Hosmer-Lemeshow (1989) showed that, if the model is correctly

specified, then the HL statistic follows a 2 distribution with 𝐽 − 2
degrees of freedom.
1 Logit and Probit Model

❖ R2 and Pseudo_R2
▪ R2 is used to evaluate the fit of the model in the OLS method. In the logistic
model, the fit is represented by the Pseudo R2, which is defined as:
𝟐 𝑳∗
𝑷𝒔𝒆𝒅𝒐_𝑹 = 𝟏 −
𝑳𝟎
where, 𝐿∗ is the maximum log-likehood value of the estimated model, and 𝐿0
is the maximum log-likehood value of the model with only a constant.
▪ In the logit model, the most important aspects are the expected sign of the
regression coefficients, their statistical significance, and their practical
significance. The Pseudo R² is only of secondary importance.
1 Logit and Probit Model

❖ Commands in Stata
▪ logit Y X => Estimates the effect of variables 𝑋 on the log-odds of 𝑌
▪ logit Y X, or => Estimates the effect of variables 𝑋 on the odds of 𝑌.
▪ mfx or margins, dydx(*) : Computes the marginal effects at the mean
of 𝑋.
▪ margins, dydx(*) at(…): Computes the marginal effects for a specific
group (at given values of 𝑋).
1 Logit and Probit Model

❖ Commands in Stata
▪ probit Y X: Estimates the effect of variables X on the probability of Y
using the Probit model.
▪ mfx or margins, dydx(*): Computes the marginal effects at the mean
of X.
▪ margins, dydx(*) at(…): Computes the marginal effects for a specific
group (at given values of X
1 Logit and Probit Model

❖ Testing
▪ linktest: The function form Test
(If P-value is large, then the function form is correct)
▪ estat gof (or estat gof, group(10): The goodness of fit Test
(If P-value is large, then the model is suitable)
1 Logit and Probit Model

❖ Logit or Probit?
▪ The choice between logit and probit estimation is
based on the assumption about the error
distribution:
➢ If the distribution is logistic => use the logit
model
➢ If the distribution is normal => use the probit
model
▪ The estimated results from the two models are
usually similar, but not directly comparable.
▪ In practice, the choice is often up to the researcher.
Practice
1 Practice

To get the data. Type:

use "[Link]
To run a logit model, type:
logit y_bin x1 x2 x3 [Link]
2 Practice
2 Practice

Interpretation:
• In this estimation result, none of the coefficients except for Agree
significantly affect the log-odds ratio of the dependent variable. The
coefficient for Agree is significant at the 5% level.
• The Coefficient column shows the coefficients in log-odds form.
For example, when x1 increases by one unit, the expected change in
the log odds is 1.133556 (an increase), holding all other variables in
the model constant. However, this increase is not statistically significant
because the p-value is not <0.05.
2 Practice

To get odds ratio rather than logit coefficients, type:

logit y_bin x1 x2 x3 [Link], or
2 Practice

Interpretation:
• Odds Ratio: They represent the odds of 𝑌 = 1 when X increases by 1 unit.
These are the exp(logit coeff).
– If the 𝑂𝑅 > 1 then the odds of 𝑌 = 1 increases
– If the 𝑂𝑅 < 1 then the odds of 𝑌 = 1 decreases
• The odds ratio for x1 which is 3.106685, that mean, if x1 increases by
one unit, the odds of 𝑌 = 1 are 3.1 times higher when x1 increases by
one unit, keeping all other predictors constant.
1 Practice

To calculate marginal effects after logit, type:

quietly logit y_bin x1 x2 x3 [Link]
margins, dydx(*) atmeans post
2 Practice

Interpretation:
• x1=.1384634 The change in probability for one instant change in x1 is
13 percentage points (however, the change is not statistically significant)
• x2=.036904 The change in probability for one instant change in x2 is
3 percentage points (however, the change is not statistically significant)
• Agree=-.3656898 The change in probability when opinion goes from
‘strongly agree’ to ‘agree’ decreases by 36 percentage points
or -0.36. This change is statistically significant, because the p-value is
0.029 which is <0.05.
2 Practice

Interpretation:
• Disag=.0312784 The change in probability when opinion goes from
‘strongly agree’ to ‘disagree’ increases by 3 percentage
points or 0.03. However, the change is not statistically significant.
• Str Disag=.0574484 The change in probability when opinion goes
from ‘strongly agree’ to ‘strongly disagree’ increases by
5 percentage points or 0.05. However, the change is not statistically
significant.
1 Practice

Estimating predicted probabilities after logit

quietly logit y_bin x1 x2 x3 [Link]
margins, atmeans post
1 Practice

Interpretation:
The probability of y_bin=1 is 85%, given that all predictors are set to their mean values.
1 Practice

Estimating predicted probabilities after logit

quietly logit y_bin x1 x2 x3 [Link]
margins opinion, atmeans post
1 Practice

Interpretation:
Holding all variables at their mean values. The probability of y_bin = 1 is:
▪ 87% among those who “strongly agree”,
▪ 51% among those who “agree”,
▪ 91% among those who “disagree” and
▪ 93% among those who “strongly disagree”
Practice