Notes Methods & Techniques

Lecture 1 – Introduction

Everyday learning
- Learning via tradition (little effort).
- Learning via experts (little effort).
- Learning via own experience (causal reasoning).

Problems with everyday learning:

Tradition: tradition may be wrong.
Experts: difficult to challenge authority.
Own experience: possibly no accurate observations, overgeneralization: selective
observations, illogical reasoning: science is probabilistic, logical reasoning.

Scientific learning
“Is something true or not?”
Extending existing knowledge, learning via scientific methods.
Theory, data, and analysis are important in scientific learning. Scientific learning is mainly
about quantitative questions and answers.
- Type 1 Association: description of facts.
- Type 2 Causal relation: one variable influencing another.

Ideal experiment: Randomized controlled trial (RCT). Control and treatment group: difference
between sample means of treatment and control group is an estimation of the causal
experiment. Random assignment is needed. However, this is often expensive, impossible, and
it may even be unethical.

Types of data
Observational data: not designed especially for causal effects.
Experimental data: designed to evaluate treatment or policy.

Time dimension:
Time series data consist of a single entity observed at multiple times (GDP of a country over
several years).
Cross-sectional data consist of multiple entities observed at a single time period (GDP of
several countries in a specific year).
Panel data (longitudinal data) consist of multiple entities, where each entity is observed at
two or more time periods (GDP of several countries over several years).

Unit of analysis: What one is studying: individual, firms, groups, regions, or countries.

Faulty reasoning about unit of analysis:

Ecological fallacy: erroneously drawing conclusions about individuals solely from the
observations of higher aggregations.

Conceptualization
The process through which we specify what we mean when we use particular terms in
research. Typically difficult in social sciences.

1
Operationalization
The development of specific research procedures that will result in empirical observations
representing those concepts in the real world. Operationalization is about the measurement of
a theoretical concept.

Operationalization: criteria for measurement quality

Reliability: the quality of a measurement method that suggests the same data would have
been collected each time in repeated observations of the same phenomenon. Cronbach’s
alpha: range 0-1: higher values indicate higher reliability, however, it does not say anything
about validity.
Validity: a term describing a measure that accurately reflects the concept it is intended to
measure.

Summary chapter 1 – Economic Questions and Data

1. Many decisions in business and economic require quantitative estimates of how a
change in one variable affects another variable.
2. Conceptually, the way to estimate a causal effect is in an ideal randomized controlled
experiment, but performing such experiments in economic applications is usually
unethical, impractical, or too expensive.
3. Econometrics provides tools for estimating causal effects using either observational
(non-experimental) data or data from real-world, imperfect experiments.
4. Cross-sectional data are gathered by observing multiple entities at a single point in
time; time series data are gathered by observing multiple points in time; and panel data
are gathered by observing multiple entities, each of which is observed at multiple
points in time.

Lecture 2 & 3 – Simple linear regression model: introduction, estimation,

and properties of estimator

Linear regression models are specifically suited for continuous dependent variable Y and any
kind of variable X.
n
1
Sample covariance : s XY = ∑ ( X −X )(Y i−Y )
n−1 i=1 i
The sample covariance shows if X and Y tend to move in the same (+) or opposite (-)
direction. However, the unit (unit of X × unit of Y) is difficult to interpret.
s XY
Sample correlation ( coefficient ) :r XY =
sX sY
The sample correlation coefficient is unit free, it always takes on a value between -1 and 1.
It measures the strength of the relationship between X and Y.
Linear regression more explicitly tries to assess a cause and effect relationship, and to
quantify such causal effect (under assumptions). It is a very flexible framework which can be
extended in several respects (multiple X-variables, nonlinear relationship).

Linear regression model

How variable X affects variable Y.
Y i=β 0 + β 1 X i+ ui

2
ui: error term, other factors which are not considered. Vertical distance between population
regression line and observation. The population regression line ( β 0 + β 1 X i ) represents the
average relationship between Yi and Xi.
Estimating regression coefficients
True β0 and β1 are unknown.
Ordinary least squares (OLS) provides an estimation for the values of β0 and β1.
Estimation : ^β 0∧ β^ 1
n
Minimize : ∑ (Y i − ^β 0− ^β 1 X i ¿)2 ¿
i=1
Fitted regression line : Y^ i= β^ 0 + ^β 1 X i
Y i− β^ 0 − ^β1 X i is the distance between the regression line and the observation points.
(Y i− β^ 0 − ^β1 X i)2 is to correct for positive and negative distances and it puts more (less) weight
on points that are close to (far away from) the regression line.
n

∑ ( X i−X )(Y i−Y ) s XY

OLS estimator of β 1 : ^β 1= i=1 N
=
s 2X
∑ ( X i− X) 2

i=1
This is similar to the correlation coefficient, however, it is not unit free: measured in (units of
Y / units of X).
OLS estimator of β 0 : β^ 0=Y − ^β 1 X
Predicted ( fitted ) values : Y^ i = ^β 0+ ^β1 X i
Residuals : u^ i=Y i−Y^ i
Observed values :Y i =Y^ i+ u^ i

Explained variation: Explained sum of squares.

n
ESS=∑ ( Y^ i−Y )2
i=1
Total variation: Total sum of squares.
n
TSS=∑ (Y i−Y )2
i=1
Proportion of sample variance of Yi which is explained by Xi: R2
2 ESS
R=
TSS
R =corr(Y i , Y^ i)
2 2

In case of a single regressor: R2=corr(Y i , X i )2

Standard error of regression : SER=s ^u=√ s 2u^
n
1
∑ u^ 2 = SSR
Sample variance of the residuals u^ i : s2u^ =
n−2 i=1 i n−2
Large standard error of regression: poor fit of estimated regression model. Small SER:
good fit of estimated regression model.

Assumptions of OLS
1. Zero Conditional Mean: E ( ui| X i )=0
2. Yi and Xi are independent and identically distributed.

3
 3. Large outliers are unlikely.

1 Zero Conditional Mean

E ( ui| X i )=0
Expected value of ui is the same regardless of the value of Xi. Implies corr ( u I , X i )=0
Xi is uncorrelated with any other factor that influences Yi.
This assumption holds if Xi is randomly assigned. However, this is usually not the case.
Therefore most of the times it is difficult to know if the assumption holds.

If Xi is not assigned randomly: we need to believe that Xi is unrelated with other factors that
also influence Yi  “as good as random”.
For each other factor:
- Does not influence Yi?
- Uncorrelated with Xi?

If Xi is related with other factors, the model will give an estimation of the causal effect, based
on a mixture of Xi and other factors (β and γ). For a good regression model, the zero
conditional mean assumption should hold.

2 Observations are independent and identically distributed

This assumption holds if sample is drawn randomly: (X i, Yi) have same distribution,
independent.
Does not hold with dependent observations:
- Sample consists of same unit of observation over time (time series).
- Sample consists of multiple units of observation and each is observed at several points
in time (panel data).

3 Large outliers in X and Y are unlikely

4 4
0< E (X i )< ∞ and 0< E (Y i )<∞ (finite kurtosis).
OLS can be sensitive to large outliers. Possible sources of outliers is data errors  fix or
delete from dataset. Otherwise it is a plausible assumption.

Sampling distribution of OLS estimators

OLS estimators ^β 0 and ^β 1 are computed from a random sample and are therefore random
variables with a probability distribution. Different samples lead to different coefficients. The
sample distribution is necessary for hypothesis testing, to obtain confidence intervals, and to
know whether estimators are biased or unbiased.

^β follows a Normal distribution centered at β1, because of the large sample.

1
E( ^β 0) = β0 and E( ^β 1) = β1. OLS estimators are unbiased.

Unbiasedness of β1
Unbiased if E( ^β 1 ¿ = β1
n

∑ ( X i−X )(Y i−Y )

^β 1= i=1
N

∑ ( X i−X )2
i=1

4
 [ ]
n

∑ (X i−X )(Y i−Y )

E ( β^ 1 ) =E i=1
N

∑ ( X i−X )2
i=1

[ ]
n
1
∑ (X − X )E ( ui|X 1 , … , X n )
n i=1 i
E ( β^ 1 ) =β 1+ E n
1
∑ ( X −X )2
n i=1 i

[ ]
n
1
∑ ( X − X) E ( ui|X 1 , … , X n )
n i=1 i
E n
→0 under assumption 1 → E ( β^ 1 ) =β 1
1
∑
n i=1
(X i−X )2

Because of the Central Limit Theorem:

^β follows a Normal distribution: N ( β 0 , σ 2^ )
0 β 0

^β follows a Normal distribution: N ( β 1 , σ 2^ )

1 β 1

( β^ ¿¿ 0 , β^ )¿ follows a bivariate Normal distribution.

1

1 var [ ( X i −μ X ) ui ]
Variance :σ 2β = 2
n [var ( X i ) ]
1

Variance decreases when:

- Var(Xi) increases
- When var(ui) decreases
- When n increases.

The OLS estimator is consistent for β1 (converges in probability to β1).

Interpretation
Conditional expectation of Y given X:
E ( Y i|X i) =E ( β 0 + β 1 X i+u i| X i )=β 0 + β 1 X i+ E ( ui|X i )
↘ = 0 (assumption 1)
E ( Y i|X i) =β 0 + β 1 X i
E ( Y i|X i+1 ) =E ( Y i|X i ) + β1
E ( Y i|X i+ ∆ X i ) =E ( Y i| X i ) + β 1 × ∆ X i
Intercept: interpretation only makes sense when Xi = 0 occurs in the dataset.

Binary regressor (dummy variable)

Population model :Y i= β0 + β 1 Di+ ui
E ( Y i|Di ) =β 0+ β1 Di
E ( Y i|Di=0 )=β 0
E ( Y i|Di=1 )=β 0 + β 1
βi: difference between average Di = 1, compared to Di = 0.

5
Summary chapter 4 – Linear Regression with One Regressor
1. The population regression line, β 0 + β 1 X , is the mean of Y as a function of the value of
X. The slope, β1, is the expected change in Y associated with a one-unit change in X.
The intercept, β0, determines the level (or height) of the regression line.
2. The population regression line can be estimated using sample observations (Yi, Xi), i =
1, …, n by ordinary least squares (OLS). The OLS estimator of the regression
intercept and slope are denoted by ^β 0∧ β^ 1.
3. The R2 and standard error of the regression (SER) are measures of how close the
values of Yi are to the estimated regression line. The R2 is between 0 and 1, with a
larger value indicating that the Yi’s are closer to the line. The standard error of the
regression is an estimator of the standard deviation of the regression error.
4. There are three key assumptions for the linear regression model: (1) The regression
errors, ui, have a mean of zero, conditional on the regressors Xi. (2) The sample
observations are independent and identically distributed (i.i.d.), random draws from
the population. (3) Large outliers are unlikely. If these assumptions hold, the OLS
estimators ^β 0∧ β^ 1 are (1) unbiased, (2) consistent, and (3) normally distributed when
the sample is large.

Lecture 4 – Simple linear regression model: Hypothesis testing and

confidence intervals

Hypothesis testing
In general: H0: β1 = β1,0 vs. H1: β1 ≠ β1,0
β1,0: Hypothesized value (H0: β1 = 0)

Reject the null-hypothesis if the estimate ^β act

1 is sufficiently different from hypothesized value
β1,0.
Reject the null-hypothesis if the probability of drawing a sample that gives an estimate even
further away from β1,0.

The p-value gives the probability of obtaining an estimate ^β 1 even further away from β1,0 than
^β act , if H0 is true.
1

The p-value gives the probability of drawing a sample that gives β^ larger than |β^ | or
act
1 1

smaller than−|β^ 1 | .
act

In general: the p-value gives the probability of drawing a sample that gives ^β 1larger than
|β^ act
1 − β 1 ,0| or smaller than −|β 1 − β 1 ,0|.
^ ^ act ^

P-value
Pr H ( β^ 1−β 1 ,0 ←|β^ act
0 1 −β 1 ,0|∨ β 1−β 1, 0 >|β 1 −β 1 ,0|)
^ ^ act
Pr H (| β
0
^ − β |>|β
1 1, 0
^ act −β |)
1 1, 0

(| || |)
β^ 1−β 1 ,0 ^β 1 −β1 , 0
act

Pr H > =t−statistic
SE ( β^ 1 ) SE ( ^β 1)
0

β^ 1−β 1 ,0
t−statistic :t=
SE( β^ )
1

6
P=Pr H (|t|>|t act|)
0

P=2 Φ (−|t act|)

Φ: distribution function of a Normal N(1,0) distribution. The area under the density curve.
Φ(Z) is the probability of being below Z. 1 – Φ(Z) is the probability of being above Z.

P=Φ (−|t |) +[1−Φ(|t |)]

act act

Since there is a symmetric Normal distribution:

P=2 Φ (−|t |)
act

Reject H0 if P < α (0.05), thus when|t act|>t ¿ (1.96).

Confidence intervals
Often a confidence interval of 95% is used. From all samples that can be drawn, the interval
will contain the true value of β1 in 95% of the cases.
Pr ( lower bound< β1 <upper bound ) =0.95

( )
^β −β
Pr −1.96< 1 1 ,0 <1.96 =0.95
SE ( ^β 1 )
Pr ⁡[ β −1.96 SE ( ^β ) < ^β < β +1.96 SE ( ^β ) ]=0.95
1 1 1 1 1

95 % Confidence Interval :[β 1−1.96 SE ( ^β 1 ) ; β 1+1.96 SE ( β^ 1 ) ]

A set of values that cannot be rejected using two-sided hypothesis testing (at 5% significance
level). The narrower the confidence interval, the more useful it is. Reject H 0 if β1,0 does not lie
within the confidence interval.

Homoskedasticity (constant variance) of the error terms

Homoskedasticity is not an assumption of OLS, but a special case.
Homoskedasticity implies that var ( ui| X i=x ) is constant for i = 1, …, n. Constant variance, it
does not depend on X.
If homoskedasticity holds:
- Formulas of standard errors can be simplified.
- OLS estimator is efficient (minimum variance) among all unbiased linear estimators.

Recommendation
Always use heteroskedasticity-robust standard errors (White in Eviews), and not the
homoskedasticity-only variance (Estimation default in Eviews), unless there is a very strong
belief that homoskedasticity holds.

The size of the effect is also important! Not just the statistical significance.

Summary chapter 5 – Regression with a Single Regressor: Hypothesis Tests and

Confidence Intervals
1. Hypothesis testing for regression coefficients is analogous to hypothesis testing for the
population mean: use the t-statistic to calculate the p-values and either accept or reject
the null hypothesis. Like a confidence interval for the population mean, a 95%
confidence interval for a regression coefficient is computed as the estimator
±1.96*(standard error).

7
 2. When X is binary, the regression model can be used to estimate and test hypotheses
about the difference between population means of the “X = 0” group and the “X = 1”
group.
3. In general, the error ui is heteroskedastic – that is, the variance of u i at a given value of
Xi, var ( ui| X i=x ), depends on x. A special case is when the error is homoscedastic –
that is, var ( ui| X i=x ) is constant. Homoskedasticity-only standard errors do not
produce valid statistical inferences when the errors are heteroskedastic, but
heteroskedastic-robust standard errors do.

Lecture 5 – Multiple regression model: Omitted variable bias, OLS

estimator, measures of fit, least squares assumptions, multicollinearity

Omitted variable bias

Estimate β1 is biased if corr(ui, Xi) ≠ 0. The error term includes other variables.
ui= β2 X 2 i +v i
Assume for now that vi = 0. That is, apart from X1i and X2i there are no other omitted
variables.
corr ( u i , X 1 i ) =corr( β2 X 2 i +v i , X 1 i)
corr ( u i , X 1 i ) =0 if β2 = 0 or corr ( X 2 i , X 1 i ) =0
corr ( u i , X 1 i ) ≠ 0 if β2 ≠ 0 and corr ( X 2 i , X 1 i ) ≠ 0

corr ( X 1 , X 2 )
E ( β^ 1 ) =β 1+ β 2 sX
sX 1
2

E ( β^ ) =β +0 under assumption 1 (zero conditional mean).

1 1
Direction of bias in model including X1, and omitting X2
Corr(X1, X2) > 0 Corr(X1, X2) < 0
β2 > 0 Positive bias Negative bias
β2 < 0 Negative bias Positive bias

If we have data on the omitted independent variable, we can incorporate it in the model:
Multiple regression model :Y = β0 + β 1 X 1+ β 2 X 2 + v i
Main variable of interest: X1
Control variable: X2
If there is no data available on the omitted variable, use another variable as an approximation.
If ^β 2> 0 and there is a positive correlation between X1 and X2, this suggest there is upwards
omitted variable bias in the simple model with only one regressor.
If there is no logical causal effect, it’s an association.

Population multiple regression model

Y 1=β 0 + β 1 X 1i + β 2 X 2 i+ ui
E ( Y i|X 1i=x 1 ; X 2 i=x 2 )=β 0 + β 1 x 1 + β 2 x 2

Interpretation
When X1j goes up by ΔX1j, holding X2j constant, then E ( Y i|X 1 j ; X 2i ) increases with ΔX1jβ1. It
is common to interpret in terms of a unit changes. If X 1j increases by 1, holding X2j constant,
then E ( Y i|X 1 j ; X 2i ) increases by β1 on average.

8
Y i=β 0 + β 1 X 1 i+ β 2 X 2 i +…+ β k X ki +u i
E ( Y i|X 1i=x 1 ; X 2 i=x 2 ,… , X ki=x k ) =β 0+ β 1 x1 + β 2 x 2 +…+ β k x k
Where k is the number of independent variables.
OLS estimator: obtain ^β 0 , β^ 1 , … , β^ k that minimize:
n

∑ (Y i− β^ 0− ^β1 X 1 i− ^β 2 X 2 i−…− ^β k X ki)2

i=1
Predicted values : Y^ i= ^β 0+ ^β1 X 1 i + ^β 2 X 2 i +…+ ^β k X ki
Residuals : u^ i=Y i−Y^ i

OLS assumptions in a multiple regression model

1. Zero conditional mean: E ( ui| X 1i , X 2 i ,… , X ki )=0
2. (Yi, Xi): i.i.d.
3. Large outliers are unlikely
4. No perfect multicollinearity

Perfect multicollinearity
A perfectly linear relationship between two X-variables.
X 1 =a+b X 2+ c X 3 ; b , c ≠ 0
For example: height measured in inches and in centimeters. Same variable in different units.
Dummy variable trap: e.g. gender: a dummy variable for male and female. Always leave a
dummy of one category (does not matter which one) out of the model, or the constant term.
Interpretation of coefficient dummy variable: compared to ‘base variable’, variable for which
no dummy variable is included in the model.

Sampling distribution of the OLS estimator under assumptions

( ^β ¿¿ 0 , ^β1 , … , ^β k )¿ follow a multivariate Normal distribution.
^β ( j=1 , … k ) follows a Normal distribution: N ( β j , σ 2^ )
j β j

^ 2
Standard error of β (σ ^ ) and precision:
j βj
- Decreases with sample size n
- Decreases with variance of Xj
- Increases with variance of error term (ui)
- Increases with correlation between X’s.

Imperfect multicollinearity and (im)precision

If two regressors j and l have “large” correlation but |corr( j, l)|< 1, then there is imperfect
multicollinearity between Xj and Xl.
^β and ^β are potentially imprecise (large σ 2^ , σ 2^ ; wide confidence intervals). But if
j l β β j l

assumption 1 holds then they are still unbiased, it might not be a problem with a large sample
size.

Measures of fit
SER : s ^u=√ s 2u^
n
1
s2u^ = ∑ u^ 2= SSR
n−k −1 i=1 i n−k−1

9
 n
Explained ∑ of squares : ESS=∑ ( Y^ i−Y )2
i=1
n
Total ∑ of squares :TSS=∑ (Y i−Y )2
i=1
2 ESS SSR
R= =1−
TSS TSS
R2 always increases when a regressor is added, even with irrelevant regressors.
2 2
Adjusted R : R
2 n−1 SSR
R =1− ×
n−k −1 TSS
(n – 1) / (n – k – 1) is the deflation factor: R2 might decrease when we add additional
regressors that explain very little variation.

Warnings about measures of fit

Measures of fit don’t tell anything about assumptions, only about how well the model
explains variation.
If the dependent variable is defined differently, e.g. Y, ln(Y), then you cannot compare using
measures of fit.

Summary chapter 6 – Linear Regression with Multiple Regressors

1. Omitted variable bias occurs when an omitted variable (1) is correlated with an
included regressor and (2) is a determinant of Y.
2. The multiple regression model is a linear regression model that includes multiple
regressors, X 1 , X 2 , … , X k . Associated with each regressor is a regression coefficient,
( ^β ¿¿ 1 , β^ 2 , … , β^ k )¿ . The coefficient β1 is the expected change in Y associated with a
one-unit change in X1, holding the other regressors constant. The other regression
coefficients have an analogous interpretation.
3. The coefficients in multiple regression can be estimated by OLS. When the four least
squares assumptions are satisfied, the OLS estimator is unbiased, consistent, and
normally distributed in large samples.
4. Perfect multicollinearity, which occurs when one regressor is an exact linear function
of the other regressors, usually arises from a mistake in choosing which regressors to
include in a multiple regression. Solving perfect multicollinearity requires changing
the set of regressors.
5. The standard error of the regression, the R 2, and the R2 are measures of fit for the
multiple regression model.

Lecture 6 – Multiple regression model: hypotheses tests (and confidence

intervals) single coefficients, joint hypotheses, role of control variables &
conditional independence

Hypothesis tests for single coefficients:

H 0 : β j=β j ,0
H1: β j≠ β j, 0
1. Compute standard error of ^β j
2. Compute t-statistic
3. Compute P-value

10
 4. Reject or don’t reject H0.

Confidence intervals for single coefficients:

95 % Confidence Interval :[β 1−1.96 SE ( ^β 1 ) ; β 1+1.96 SE ( β^ 1 ) ]
90 % Confidence Interval :[β −1.64 SE ( β^ ) ; β + 1.64 SE ( ^β ) ]
1 1 1 1

Tests for joint hypotheses

H 0 : β j=β j ,0 , β m =β m ,0
H 1 : at least one β ≠ 0

Coefficients β1, β2 follow a bivariate Normal distribution, therefore a joint test is needed to
test the significance of the coefficients. Two separate tests are inappropriate, because then the
significance levels would be different. Solution for this is the Bonferroni method: a test
based on individual t-statistics but using special critical values, to make sure that the test has
the right size and significance levels.

F-statistic for H0: β1 = 0 and β2 = 0 vs. H1: β1 ≠ 0 and/or β2 ≠ 0

( )
2 2
1 t 1+ t 2−2 ^ρt 1 ,t 2
F=
2 1− ρ^ 2t 1 ,t 2
t1: t-statistic of test β1 = 0
t2: t-statistic of test β2 = 0
^ρt 1 ,t 2: estimator of correlation between the two t-statistics
The F-statistic follows the F q ,∞ distribution with q: the number of restrictions and ∞ : large
¿
sample. Reject H0 if F> Fq ,∞.

Special cases:
1 2 2
- Uncorrelated t-statistics: F= (t + t )
2 1 2
- Single restriction: F=t 2 F 1, ∞
- Overall regression F-statistic: H 0 : β 1=0 , β2 =0 … β k =0vs. H 1 : at least one β j ≠ 0

Single restrictions involving multiple coefficients: H0: β1 = β2 vs. H1: β1 ≠ β2.

1. Test directly using F-statistic ~ Fq, ∞.
2. Transform the regression model.

Transformation of the zero conditional mean assumption to conditional independence

ZCM : E ( ui|X 1 , X 2 ) →corr ( X 1 ,u i )=0∧corr ( X 2 , ui ) =0
However, for the unbiased estimator of causal effect of interest β 1, we can also use a weaker
assumption:
E ( ui| X 1 , X 2 ) =E ( ui|X 1 ) → corr ( X 1 , ui )=0 ,but we can have corr ( X 2 , ui ) ≠ 0
We can interpret X1 as causal: it only captures the effect of X1.
But not for X2: partial association: captures effect of X2 and other factors related to X2.
However, this is not a problem, since our variable of interest is X1, and X2 is a control
variable.

11
In conditional independence assumption also doesn’t hold, then there is still omitted variable
bias in the model.
If corr ( X 1 ,ui ) ≠ 0 we can add more control variables to test if the estimated effect is robust.

Summary chapter 7 – Hypothesis Tests and Confidence Intervals in Multiple Regression

1. Hypothesis tests and confidence intervals for a single regression coefficient are carried
out using essentially the same procedures used in the one-variable linear regression
model.
2. Hypotheses involving more than one restriction on the coefficients are called joint
hypotheses. Joint hypotheses can be tested using an F-statistic.
3. Regression specification proceeds by first determining abase specification chosen to
address concern about omitted variable bias. The base specification can be modified
by including additional regressors that address other potential sources of omitted
variable bias. Simply choosing the specification with the highest R 2 can lead to
regression models that do not estimate the causal effect of interest.

Lecture 7 – Nonlinear regression functions

Nonlinear regression functions

1. Effect of a variable depends on the value of this variable
2. Transforming into natural logarithms
3. Interaction effects

Effect of a variable depends on the value of this variable

Polynomials:
- Linear: α 1 x i
- Quadratic: α 1 x i +α 2 x 2i
- Cubic: α 1 x i +α 2 x 2i +α 3 x 3i
If you test a model several times, you should account for the fact that you’ve already tested
before (Comparing model a and b, and model b and c is not enough, you should also compare
model a and c).

Interpretation of quadratic term: compare differences using different (low and high) intervals.
Usually graphs give a better idea of the interpretation.

Transforming into natural logarithms

ln (Y )=β 0+ β1 X 1 + β 2 X 2+ …+ui

Reasons for using natural logarithms:

1. Because large outliers lead to a violation of the OLS assumptions. Large outliers
become unlikely when natural logarithm is used.
2. When the effect of a regressor on Y is proportional to the level of Y. The natural
logarithm converts changes in a variable into percentage changes, and therefore the
effect of a regressor on the dependent variable is not constant, but proportional to the
dependent variable.

Using the natural logarithm can also correct long tails to the right. It doesn’t work for long
tails to the left.

12
A one-unit change in Xi leads to a βi*100% change in Y.
The coefficient can be interpreted as semi-elasticity.

ln ( Y + ∆ Y )−ln ( Y ) =ln ( Y +Y∆Y )≅ ∆YY

The derivative of ln(Y) describes the behavior of ln(Y) in close proximity of Y. Hence when
∆ Y is small,
δ ln (Y ) 1 ∆Y
×∆ Y = ×∆ Y = ≅ ln (Y + ∆ Y )−ln (Y )
δY Y Y

Log-linear model
ln (Y )=β 0+ β1 X 1 +…
Interpretation: one additional unit of X1 leads to a percentage change of β1*100% in Y.
The changes are relative, not absolute.

Linear-log model
Y = β0 + β 1 ln( X 1)+…
Interpretation: a 1% increase in X1 leads to a β1/100 change in Y.
This only works with small changes. There is a relative change in X, and an absolute change
in Y.

You can’t compare log-linear and linear-log models by comparing R 2, because the dependent
variables are different: ln(Y) and Y.

Log-log model
ln (Y )=β 0+ β1 ln( X 1)+…
Elasticity : β 1=
2
[ ][ ]
∆Y ∆ X
Y
/
X
Now R can be used to compare log-linear and log-log models, because the dependent
variable is the same: ln(Y).

Interaction effects
Effect on X1 depends on the value of a dummy variable (e.g. male, where male = 1 and female
= 0).
Accounting for interaction effect: include the variable male*X1.
The coefficient of the interaction regressor tells the difference between the effect of X 1 on
males and females.
Interaction effect between two binary (dummy) variables or two continuous variables: follow
the same procedure.

Recommendation: always make a graph for a nonlinear regression model.

Model is linear in parameters

- Model is nonlinear in its regressor, but linear in its parameters.
- OLS can accommodate a wide range in nonlinearities as long as the model is linear in
its parameters.
- But OLS cannot deal with nonlinearities in its parameters e.g.
E ( Y i|… )=β 0 + β 1 X 1+ e +ui
β 2

13
Summary chapter 8 – Nonlinear Regression Functions
1. In a nonlinear regression, the slope of the population regression function depends on
the value of one or more of the independent variables.
2. The effect on Y of a change in the independent variable(s) can be computed by
evaluating the regression function at two values of the independent variable(s).
3. A polynomial regression includes powers of X as regressors. A quadratic regression
includes X and X2, and a cubic regression includes X, X2, and X3.
4. Small changes in logarithms can be interpreted as proportional or percentage changes
in a variable. Regressions involving logarithms are used to estimate proportional
changes and elasticities.
5. The product of two variables is called an interaction term. When interaction terms are
included as regressors, they allow the regression slope of one variable to depend on
the value of another variable.

Lecture 8 – Internal and external validity

Internal validity: are the effects really causal effects? Can point estimates be interpreted as
causal effects? All four OLS assumptions should hold.
External validity: does the model also hold outside the used population/beyond experimental
setting?

Only causal effects should lead to policy prescriptions. If there is no causal effect, but only an
association, this is irrelevant for policy prescriptions.

Threats to internal validity:

1. Omitted variable bias
2. Errors in variables
3. Sample selection
4. Simultaneous causality
5. Functional form misspecification.

The estimated effect is a causal effect if all four assumptions of OLS hold.
A study is internally valid if its statistical inferences about causal effects are valid for the
population and setting studied.

Causal estimates: OLS estimators of the coefficients should be unbiased and consistent,
conditional mean independence, and hypothesis tests have the desired significance level.

Omitted variable bias

The conditional mean independence assumption is violated when there is an omitted variable
that correlates with the variable of interest (X 1) and that determines the dependent variable
(ln(Y)).

Solution if omitted variable or adequate controls are available: add them to the model.
Solution if omitted variable or adequate controls are not available: panel data, instrumental
variables regression, research design: randomized control variable/quasi-experimental
approach.

Errors in variables
Errors in variables bias arises when the independent variable is measured imprecisely.

14
Nonrandom measurement error in in dependent variable: violation of conditional mean
independence assumption.

There is even bias when the measurement error is random.

Classical measurement error: bias in coefficient of X1 towards zero.
ln (Y i)=β 0+ β 1 X i +ui
With an approximation for Xi: X ipr =X i +meas errori
pr
ln (Y i)=β 0+ β 1 ( X i −meas errori)+u i
ln ( Y ) =β 0+ β 1 X ipr +[β 1 ( X i− X ipr ) +u i]
E ( β1 ( X i−X i ) + ui|X i ) ≠ 0
pr pr

( )
2
^β p σX
1 β 1 ≤ β1
→ σ 2X +σ 2meas error

2
This is always a downward bias. The bias increases in σ 2X and decreases in σ meas . error

Random measurement error in dependent variable does not lead to bias, but reduces precision
(larger variance).
pr
ln (Y i )=ln(Y i)+meas errori
ln (Y i)=β 0+ β 1 X i +ui
pr
ln (Y i )=β 0+ β1 X i +ui +meas errori
2 2
σ ( ui +mea serror i ) ≥ σ (ui)

Measurement error Random Nonrandom

X Bias towards zero Downward or upward bias
Y No bias Downward or upward bias

Solution to errors in variables

Classical measurement error in:
- Dependent variable: no solution required
- Independent variable and variance measurement error known: use formula.
In all other cases:
- Instrumental variables
- Mathematical model for the measurement error

Sample selection bias

Missing data:
- Completely at random  no bias.
- Based on regressor  no bias.
- Based on dependent variable, conditional on regressor: missing data is actually based
on the value of the error  Upward or downward bias.
Sample selection bias: E ( u| X i ) ≠ 0  bias.
Data selection based on X: no bias.

Simultaneous causality

15
Causality runs from the regressor to the dependent variable, but also from the dependent
variable to the regressors (reverse causality). X  Y and Y  X.
OLS picks up both directions of causality  bias

If simultaneous causality is negative, then the causal effect is underestimated.

If simultaneous causality is positive, then the causal effect is overestimated.
2
ln (Y )=β 0+ β1 X 1 + β 2 X 2+ β 3 X 2 + β 4 D1 + β 5 D1∗X 2+ β6 X 3+u i
X 3 =γ 0+ γ 1 ln (Y )+ γ 2 D1+ v i
cov ( ui ; X 3 ) =cov ( ui ; γ 0+ γ 1 ln ( Y ) + γ 2 D 1+ v i ) ≠ 0

There are no easy solutions: instrumental variables regression, research design: randomized
control variable/quasi-experimental approach.

Functional form misspecification

Not including nonlinearities.
Similar to omitted variable bias: E ( u| X i ) ≠ 0 .
Solutions: study data in detail (graphs). For a continuous dependent variable: polynomials,
transformations, and interaction effects.

Inconsistency of OLS assumptions

Heteroskedasticity: use “White” heteroskedasticity-robust standard errors.
2nd OLS assumptions: variables are i.i.d.: independence across observations. Solution: random
sampling.

External validity
A study is externally valid if its statistical inferences can be generalized to other populations
and settings.
Differences in population: it is about differences in the dependent variable and the regressors.
External validity is more of a concern for research based on experimental data compared to
observational data.

Causal models: both internal and external validity matter.

Forecasting: internal validity is unimportant. External validity is more important than for
causal models, but in terms of time (not in terms of different populations or settings). The
goal is not explaining behavior, but forecasting. Only criterion: good explanatory power,
stability of results, and precision.

Summary chapter 9 – Assessing Studies Based on Multiple Regression

1. Statistical studies are evaluated by asking whether the analysis is internally and
externally valid. A study is internally valid if the statistical inferences about causal
effects are valid for the population being studied. A study is externally valid if its
inferences and conclusions can be generalized from the population and setting studied
to other populations and settings.
2. In regression estimation of causal effects, there are two types of threats to internal
validity. First, OLS estimators are biased and inconsistent if the regressors and error
terms are correlated. Second, confidence intervals and hypothesis tests are not valid
when the standard errors are incorrect.

16
 3. Regressors and error terms may be correlated when there are omitted variables, an
incorrect functional form is used, one or more of the regressors are measured with
error, the sample is chosen nonrandomly from the population, or there is simultaneous
causality between the regressors and dependent variables.
4. Standard errors are incorrect when the errors are heteroskedastic and the computer
software uses the homoskedasticity-only standard errors, or when the error term is
correlated across different observations.
5. When regression models are used solely for forecasting, it is not necessary for the
regression coefficients to be unbiased estimates of causal effects. It is critical,
however, that the regression model be externally valid for the forecasting application
at hand.

Lecture 9 – Tools to ensure or restore internal validity: sampling, panel

data and IV methods

Appropriate sampling
- Obtain a representative sample of the population under study.
- Consistent standard errors, sampled observations should be independent, randomly
selected.

Two essential steps in sampling

1. Define the population
2. Once population defined, how to derive a sample of that population that is
representative? If representative, sample is sufficient to make inferences about total
population.

Two main sampling techniques

Probability sampling Non-probability sampling
Random selection all members of the Non-random selection: only techniques in
population have an equal chance of being which samples are selected in some way not
selected in the sample (observations are suggested by probability theory.
independent). Representativeness is not guaranteed.
Representative – the quality of a sample to
have the same distribution of characteristics
as the population from which is was drawn.
Allows for using probability theory. Does not allow for using probability theory
Examples: Examples:
Simple random sampling (inefficient). Convenience sampling
Multistage cluster sampling (clusters are Snowball sampling (when population is
independent, but entities within clusters may unknown).
not be independent). Many disadvantages, but essential when no
Superior sampling design, but register is register of population exists (or impossible
needed. to construct).

(No exam material, but useful for Research Project)

(Sampling weights)
(Appling a probability sampling framework ex-ante, but ex-post is appears the sample in not
representative. Solution: sampling weights: each observation gets a sampling weight to restore
representativeness of sample, typically based on observable characteristics. High (low)
sampling weight when less (more) likely to be in the sample).

17
Panel data
When omitted variables or control variables are not available.

Panel data: Panel data consists of observations on the same n entities at two or more time
periods T.
(Yit, Xit) i = 1, …, n; t = 1 … T
Balanced (same T for all) vs. unbalanced (T might vary across individuals).
Time periods are called “waves”.

Panel data can help to get rid of OVB by observing changes in the dependent (and also, but
not only, the independent variable).
Comparing subsequent observations, variables (including error terms and omitted variables)
drop out when they have stayed constant.
2
Y i 2=β 0 + β 1 X 1i 2 + β 2 X 2i 2 + β 3 X 2i 2 + β 4 X 3i 2+ ui 2
2
Y i 2=β 0 + β 1 X 1i + β 2 X 2i 2 + β 3 X 2i 2 + β 4 X 3i +ui
(Assuming X1, X3, and the error term have stayed constant over time)
2
Y i 1=β 0 + β 1 X 1i + β 2 X 2i 1 + β 3 X 2i 1 + β 4 X 3i +ui
If :u¿ =α ¿ + v ¿ ,then
Y i 2−Y i 1=β 1 ( X 2i 2−X 2i 1 )+ β3 ( X 22i 2− X 22i1 ) +(v i2 −v i 1)

Advantages of using panel data: study changes in the dependent variable (and independent
variable) allows getting rid of omitted variables (that are not available) that don’t change over
time. It can also be extended to more than two time periods/waves.
Limitation of using panel data: Time-invariant variables, even when not omitted, drop out.
Model is silent on the effects of time-invariant variables.

Instrumental variables
Addresses the violation of the zero conditional mean assumption (errors in variables, OVB
when the omitted variable is not available, time-varying variables which lead to OVB), and it
provides a solution for simultaneous causality.

Isolate exogenous variation

Variation in X1i consists of two parts:
- Variation correlated with ui (endogenous variation) leading to OVB.
- Variation that is independent of ui (exogenous variation).

An instrument is valid when it satisfies two conditions:

- Instrument exogeneity: Instrument is uncorrelated with the error term. This cannot
be tested. Corr(instrument, ui) = 0.
- Instrument relevance: Instrument is a good predictor of the endogenous variable.
This can be tested. Corr(instrument, Xi) ≠ 0.

Finding an instrument that influences X1, but also influences Y, but only through X1!

Stage 1

18
Predict X1 by the instrument. Since the instrument is assumed to be independent of ui, the
predicted value of X1 is also independent of ui.
X 1i=π 0 + π 1 instrument i +v i
^
X 1i=E ( X 1i|instrument i )=π 0 + π 1 instrument i

Stage 2
Regress Y on predicted X1.

Two Stages Least Squares (TSLS), three methods:

- 2-stage procedure, as described above.
- Estimate directly with TSLS, point estimates are similar, but the standard errors differ
(accounts for two stages).
- Divide coefficient of Y on IV by coefficient of regression of X on IV (as formula
below).
TSLS
E ( Y i|instrument i )
Insignificant TSLS estimate : β 1 =
E ( X 1i|instrument i )

OLS estimate > TSLS estimate: upward OVB in the OLS estimator?

βOLS (after including other variables) is larger than βTSLS, because

- There remains OVB, exogeneity can’t be tested.
- IV explains a minor share of the total variation in X1.

If the instrument predicts X1 perfectly, then βOLS = βTSLS, this can only be justified when there
is no OVB. Hence, TSLS estimates are only externally valid for the variation in X 1 that is
explained by the instrument.

TSLS and IV are a general tool to address violations of the crucial OLS assumption
conditional mean independence.
How to come up with a good instrument:
- Economic theory
- Start looking for ‘exogenous variation’.

Summary chapter 10 (10.1 and 10.2) – Regression with Panel Data

1. Panel data consist of observations on multiple (n) entities – states, firms, people, and
so forth – where each entity is observed at two or more time periods (T).
2. Regression with entity fixed effects controls for unobserved variables that differ from
one entity to the next but remain constant over time.

Summary chapter 12 (12.1 and 12.5) – Instrumental Variables Regression

1. Instrumental variables regression is a way to estimate regression coefficients when one
or more regressors are correlated with the error term.
2. Endogenous variables are correlated with the error term in the equation of interest;
exogenous variables are uncorrelated with this error term.
3. For an instrument to be valid, it must be (1) correlated with the included endogenous
variable, and (2) exogenous.

Lecture 10 – Tools to ensure and restore internal validity: experiments and

quasi-experiments

19
(Quasi-) experimental variation is a benchmark for causal effects estimation (not for research
that is after associations or forecasting).

Experiment Quasi-experiment
Treatment is assigned randomly Treatment is assigned “as if” random
On purpose Not on purpose

Why do we need a (quasi-) experimental variation?

(Quasi-) experimental variation in policy enforcement is random.
Observational variation in policy enforcement may be due to differences, non-random
variation.

(Quasi-) experiments: basic problems in causal inference

- Potential outcomes can’t be observed at the same time  at the individual level, you
can never estimate a causal effect.
- However, a randomized controlled trial (RCT)/experiment can inform on the ‘average
causal effect’/’average treatment effect’ when
- Subjects are sampled at random  representative effect.
- When treatment is randomized such that any differences that might exist between
the treatment and control group are eradicated.
- One might also be able to retrieve the ‘average treatment effect’ from a quasi-
experiment.

Average treatment effect (ATE):

E ( y 1 i )−E ( y 0i ) =E ( y 1 i|t i =1 ) + E ( y 1 i|t i=0 )−E ( y 0i|t i=1 ) −E ( y 0 i|t i =0 )
Average difference between effect on treatment group (t = 1) and control group (t = 0).
E ( y 1 i )−E ( y 0i ) =[ E ( y 1 i|t i=1 )−E ( y 0i|t i=0 ) ] + [ E ( y 1 i|t i=0 ) −E ( y 0 i|t i =1 ) ]
[ E ( y 1 i|t i=1 )−E ( y 0 i|t i=0 ) ]: observed difference (OD). Difference in effects on treatment
group and control group.
E ( y 1 i|t i =0 ): effect on people that received treatment, if they wouldn’t receive treatment.
E ( y 0 i|t i=1 ): effect on people that didn’t receive treatment, if they would receive treatment.
[ E ( y 1 i|t i=0 ) −E ( y 0 i|ti =1 ) ]: not observed.
Goal : ATE= E( y 1 i− y 0 i )
OD=E ( y 1 i|t i=1 )−E ( y 0i|t i=0 )
OD=E ( y 1 i|t i=1 )−E ( y 0i|t i=1 ) + E ( y 0i|t i=1 ) −E ( y 0 i|t i=0 )
E ( y 1 i|t i =1 )−E ( y 0i|t i=1 ): average treatment effect on the treated: ATET.
E ( y 0 i|t i=1 )−E ( y 0 i|t i=0 ): selection bias.

Random treatment assignment and random sampling lead to:

- No selection bias: E ( y 0 i|t i=1 )=E ( y 0 i|t i=0 )
- OD = ATET: E ( y 1 i|t i =1 )−E ( y 0i|t i=0 )=E ( y 1 i|t i=1 )−E ( y 0i|t i=1 )
- ATE = ATET: E ( y 1 i )−E ( y 0i ) =E ( y 1 i|t i =1 )−E ( y 0 i|t i=1 ) =E ( y 1i|t i=0 )−E ( y 0 i|t i=0 )

However, if there is selection bias: E ( y 0 i|t i=1 ) ≠ E ( y 0 i|t i=0 )  OD ≠ ATET ≠ ATE.

20
Difference estimator:
- y i=β 0 + β 1 t i +ui
- Experimental variation ensures that E ( ui|t i )=0 , implying that ATE = ATET = OD.
- Add control variables to improve efficiency.
- Add observables wi if the treatment is random conditional on these observables:
E ( ui|t i , wi )=E ( u i|wi ) .
- Adding control variables will reduce any remaining OVB in the TSLS estimates (if IV
is imperfect).
- Adding control variables never influences the point estimate.

Experiments: threats to internal and external validity

Internal validity
- Failure to randomize: are subjects assigned to the treatment group the same as subjects
assigned to the control group? No problem if assignment is done randomly. However,
potentially problematic when seemingly non-systematic ad-hoc rule is used.
Solution: test whether subjects are similar:
F-test: t i=γ 0 +γ 1 pretreat i + …+γ k pretreat k + v i . If all coefficients are not significant,
random assigned can be assumed.
- Partial compliance: some subjects in treatment group do not actually receive
treatment, some subjects in control group do get treatment  E ( ui|t i ) ≠0 or
E ( ui|t i , wi ) ≠ E ( u i|wi ).
If there is data on random assignment (ra) and actual treatment: use random
assignment as an IV for treatment. Exogeneity assumption is always satisfied,
relevance assumption is plausibly satisfied.
TSLS TSLS
y i=β 0 + β1 t i +e i
t i=π 0 + π 1 ra i+ v i
If there is data on the random assignment (ra), but not on the actual treatment, use
intention to treat (ITT): y i=δ 0 +δ 1 ra i +e i
- Attrition: harmless: people move out of the population, conditional mean
independence is still satisfied, since it’s unrelated to the treatment. Harmful: people
have for example developed the illness that was treated in the experiment, and are
excluded from the experiment, conditional mean independence does not longer hold,
since it’s related to the experiment.
- Experimental effects (Hawthorne effects): just being in the experiment already
changes the condition of the subject (motivational effects, placebo effects). Solution:
double blind.
- Small samples – randomization: may still be not random, because of the small sample.

IV-βTSLS measures the effect of actually receiving the treatment.

ITT-δ measures the effect of being selected into the treatment group.
External validity
- Non-representative sample, program/policy.
- General equilibrium effects.

Quasi-experiments
Randomness is introduced by variations in individual circumstances that make it appear as if
the treatment is randomly assigned. Quasi-experiments are also called ‘natural experiments’,
because the randomness is not on purpose.

21
The goal of quasi-experiments is the same as with experiments: isolate “random variation” to
obtain ATE.

Compared to experiments:
- Quasi‐experimental designs are more often applied onto representative samples.
- One should adopt a more critical attitude towards random treatment assignment.

Randomness is not on purpose:

- The subjects are sampled at random  representative sample, typically superior to
experiments.
- Assignment of treatment is randomized, typically inferior to experiments, conditioning
on controls is often required.

Techniques
The treatment is fully determined by quasi-experimental variation.
- OLS. Control group and treatment group are identical pretreatment.
- Differences-in-differences (DiD): there is random assignment treatment, but there
remain differences between treatment and control group. Solution: Compare changes
in outcomes pre‐ and post-treatment to remove differences in outcomes pretreatment.
after before
∆ y i = y i − y i =β 0+ β1 t i +u i ATE = β1
DiD = difference in treatment group – difference in control group
- Sharp regression continuity design (SRD): treatment depends entirely on crossing a
certain threshold.
- Instrumental variables: treatment is non-random, but a determinant of treatment is
random.
- Fuzzy regression continuity design: same intuition as SRD, but reception of treatment
is not the sole determinant.

Threats to internal and external validity

Compared to experiments, … is … threat to internal validity for a quasi-experimental design.
- Failure to randomize – a bigger
- Failure to follow treatment protocol – an equally important
- Attrition – an equally important
- Experimental effects – a smaller.

Threat to internal validity that only matters for quasi-experiments: instrument validity
- Random IV might not suffice for exogeneity assumption (IV and error are unrelated)
to hold.
External validity: similar to experiments.

Summary chapter 13 – Experiments and Quasi-Experiments

1. The average causal effect in the population under study is the expected difference in
the average outcomes for the treatment and control groups in an ideal randomized
controlled experiment. Actual experiments with human subjects deviate from an ideal
experiment for various practical reasons, including the failure of people to comply
with the experimental protocol.
2. If the actual treatment level Xi is random, then the treatment effect can be estimated by
regressing the outcome on the treatment. If the assigned treatment Zi is random but the

22
 actual treatment Xi is partly determined by individual choice, then the causal effect
can be estimated by instrumental variables regression, using Z i as an instrument. If the
treatment (or assigned treatment) is random conditional on some variables W those
control variables need to be included in the regressions.
3. In a quasi-experiment, variations in laws or circumstance or accidents of nature are
treated “as if” they induce random assignment to treatment and control groups. If the
actual treatment is “as if” random, then the causal effect can be estimated by
regression (possibly with additional pretreatment characteristics as regressors); if the
assigned treatment is “as if” random, then the causal effect can be estimated by
instrumental variables regression.
4. Regression discontinuity estimators are based on quasi-experiments in which
treatment depends on whether an observable variable crosses a threshold value.
5. A key threat to the internal validity of a quasi-experimental study is whether the “as
if” randomization actually results in exogeneity. Because of behavioral responses, the
regression error may change in response to the treatment induced by the quasi-
experiment, so the treatment is not exogenous.
6. When the treatment effect varies from one individual to the next, the OLS estimator is
a consistent estimator of the average causal effect if the actual treatment is randomly
assigned or “as if” randomly assigned. However, the instrumental variables estimator
is a weighted average of the individual treatment effects, where the individuals for
whom the instrument is most influential receive the greatest weight.

Lecture 11 – Binary Choice Models

Binomial: only two options. Since we can’t make calculations with options, we assign
numbers to them: 0 and 1.
Point prediction using OLS, when Y is a binary dependent variable, tells us the proportion.
When working with binary variables, you’re predicting a proportion = probability.
Expectation:
E(Y) = Pr(Y = 1)*1 + Pr(Y = 0)*1
E(Y) = Pr(Y = 1) + 0
E(Y) = Pr(Y = 1)
Expectation = probability.

Linear probability model

Pr (Y )=β 0 + β1 X 1
The interpretation of the coefficients, t-statistics, and p-values is similar to a normal
regression model. If X1 increases by 1, Pr(Y) increases by β1.

The labeling of the dependent variables does not influence the results.
It is also possible to add more independent variables to the linear probability model.
R2 should be ignored, because it doesn’t account for the use of binary variables.
However, with the linear probability model, Pr(Y) < 0 and Pr(Y) > 1 are not excluded, even
though this is impossible, therefore we need to use another model.

Probit model
‘Indirectly’ estimating probabilities.
First estimate the z-value
z=β 0 + β 1 X 1
When using the probit model, we only look at one-sided probability.

23
Using z-values, we get a ‘bended’ probability function, which can take on any probability [0,
1].

z=β 0 + β 1 X 1
Φ ( z)=Φ (β 0 + β 1 X 1)
Pr (Y )=Φ( β 0+ β 1 X 1)

Eviews:
- Equation estimation  Method: BINARY
- No need to use ‘White’ standard errors.
- McFadden R2 replaces the normal R2.

Interpretation of probit model:

If X1 increases by 1, the z-value increases by β 1, this is not the same as an increase in Pr(Y) by
β1.
Since the model is non-linear, the effect size differs. The effect size depends on where one is
looking. Default: calculate effect size at mean value of regressor (or let software calculate
mean effect size).

Logit model
Indirectly estimating probabilities, but not through z-values. Instead, use the ‘logistic
function’.
L=β 0 + β 1 X 1
1
Pr (Y )= (−L)
1+ e
This is almost entirely similar to the probit function, however, the coefficients are slightly
different, also in interpreting them.

Why should we use the logit model

Pr ⁡(Y =1)
Odds=
Pr ⁡(Y =0)
Sometimes it’s more natural to think in terms of odds than in terms of probabilities.
Logit models the natural logarithm of the odds.
1
Pr (Y )= (−( β + β X ))
1+ e 0 1 1

L=ln(odds )=β 0+ β1 X 1

Interpretation of the coefficients: if X1 increases by 1, L increases by β1.

Recommendation:
- Don’t use the linear probability model, it is inappropriate and sometimes just wrong.
- Use the probit model.

Multinomial variables
- Binomial: 2 values
- Continuous: ∞ values
- Multinomial: many (>2) values, but not ∞.

24
Multinomial: Predict probability of choosing each of the options, and make it such that all
probabilities add up to 1. You can use ‘multinomial probit’ or ‘multinomial logit’.

Ordered choice
Multiple options, but in a specific ordering (e.g. Likert-scale). You cannot replace these
options with numbers, but you should use ‘Ordered probit’.

Summary chapter 11 – Regression with a Binary Dependent Variable

1. When Y is a binary variable, the population regression function shows the probability
that Y = 1 given the value of the regressors, X1, X2, …, Xk.
2. The linear multiple regression model is called the linear probability model when Y is a
binary variable because the probability that Y = 1 is a linear function of the regressors.
3. Probit and logit regression models are nonlinear regression models used when Y is a
binary variable. Unlike the linear probability model, probit and logit regressions
ensure that the predicted probability that Y = 1 is between 0 and 1 for all values of X.
4. Probit regression uses the standard Normal cumulative distribution function. Logit
regression uses the logistic cumulative distribution function. Logit and probit
coefficients are estimated by maximum likelihood.
5. The values of coefficients in probit and logit regressions are not easy to interpret.
Changes in the probability that Y = 1 associated with changes in one or more of the
X’s can be calculated using the general procedure for nonlinear models.
6. Hypothesis tests on coefficients in the linear probability, logit, and probit models are
performed using the usual t- and F-statistics.

25