Statistical Foundations of Business Analytics

Chapter 4: Endogeneity

Tim Ederer

Mini 2, 2024
Tepper Business School
Introduction

With Chapter 1, 2 and 3, we can make inference about β

• The only assumptions we need are EXO and RANK

What happens when EXO is not satisfied?

• Whatever is in εi is correlated with xi : E[εi |xi ] ̸= 0
• We refer to this as endogeneity

1 / 19
Reminder: Consequences of Failure of EXO

Endogeneity has severe consequences

• β̂ is biased and inconsistent
• Inference procedure collapses

Contrary to HOMOSKEDASTICITY and IID, EXO is necessary!

• β is not identified if EXO is not satisfied
• Impossible to find an alternative unbiased estimator for β

What should we do?

• Solution: change the data

2 / 19
Designing Experiments
Exogeneity by Construction

Can we construct our own dataset (yi , xi ) for i = 1, ..., n such that EXO holds?
• We cannot get rid of εi
• But we can try to make sure that xi is uncorrelated with εi

Solution: get control over the realizations of xi for each i

• Experiments allow you to assign a chosen value to xi for each i
• Example: choose which ad to assign to user i

How do we assign xi such that EXO holds?

3 / 19
Random Assignment

Assignment rule: for each i assign a random value of x to xi

• When xi is binary: assign 0 or 1 with probability 1
2
• When xi is continuous: draw a random value of x

Why does randomization work?

• Breaks the dependence between xi and εi
• Correlation between xi and εi vanishes to 0 as n grows =⇒ EXO is satisfied!

4 / 19
Why Randomization Works: Visualization

• n = 50 =⇒ Cor(xi , εi ) = −0.091

5 / 19
Why Randomization Works: Visualization

• n = 100 =⇒ Cor(xi , εi ) = 0.018

6 / 19
Why Randomization Works: Visualization

• n = 1000 =⇒ Cor(xi , εi ) = −0.009

7 / 19
Why Randomization Works: Visualization

• n = 10000 =⇒ Cor(xi , εi ) = −0.006

8 / 19
Examples

A/B testing
• yi : clicks, sales, engagement of customer i
• Randomly assign customers to alternative website designs, pricing, advertisement
(
1, if customer i exposed to design A
• xi =
0, if customer i exposed to design B

Randomized Control Trials (RCT)

• yi : health of patient i
• Randomly assign patients to treatments
(
1, if patient i is treated
• xi =
0, if patient i is untreated

9 / 19
Baseline Covariates Balance

How can you check if randomization worked in practice?

• Impossible to directly check if xi is uncorrelated with εi
• But you can check if xi is uncorrelated with other observed variables!

Balance tests
• Take zi a variable measured before the experiment (demographics, baseline outcomes)
• Run regression zi = β1 + β2 xi + νi
• If confidence interval for β2 is tight around 0, you’re good!

10 / 19
Stable Unit Treatment Value Assumption (SUTVA)

SUTVA: no spillovers across individuals with different assignments

• A/B testing: behavior of group A should not affect outcome of group B

What happens if SUTVA is not satisfied?

• Spillover effect appears in εi and can make EXO fail
• Example: positive spillover from group A to B =⇒ E[εi |xi = 0] > E[εi |xi = 1]

How can we avoid spillovers?

• Think carefully about the design of your experiment
• Cut communication channels, create segmented platforms, etc...

11 / 19
Practical Guidelines

Problem: running experiments with a large sample size n can be costly

• Randomization guarantees EXO only for large n
• Confidence intervals might be too large with small n

Advice 1: stratification
• Partition data in subpopulations/strata (by gender or age for example)
• Randomly assign x within strata
• This will make sure that xi is uncorrelated with baseline covariates

Advice 2: collect as much information about participants as you can

• Adding more variables in the model will reduce variance of residuals
• Reducing variance of residuals will decrease standard errors

12 / 19
Limitations

Experiments are not always feasible

• Controlled assignment is not possible for many variables xi
• Ethical concerns

Sometimes difficult to have participants representative of population

• Consenting participants might be different than the population
• =⇒ lack of “external validity”

Ideas might not work once implemented at scale

• Spillovers, interference might happen at scale

13 / 19
Panel Data
Panel Data

Definition
• Data containing individuals i = 1, ..., n over different time periods t = 1, ..., T
• xit : explanatory variables of individual i in period t
• yit : outcome of individual i in period t

Linear regression model with panel data

yit = xit′ β + εit

How can we use this additional data to relax EXO?

14 / 19
Relaxing EXO

Decompose error term εit in two terms

• εit = αi + (εit − αi ) = αi + ηit
• αi : persistent unobserved factors that affect outcome of i (called fixed effect)
• ηit : residual time-varying unobserved factors that affect yit

With panel data EXO is not necessary!

• We do not need to assume E[εit |xit ] = 0
• We just need E[ηit |xit ] = 0 =⇒ αi and xit can be correlated!

How do we construct an unbiased and consistent estimator for β?

15 / 19
Fixed Effects Estimator

Rewrite model

yit = αi + xit′ β + ηit

= α1 d1i + ... + αn dni + xit′ β + ηit

where d1i = 1 if i = 1, and d1i = 0 otherwise

OLS estimator of (α1 , ..., αn , β) is called the fixed effects (FE) estimator
• EXO holds in this new model so (α̂1 , ..., α̂n , β̂) are unbiased
• β̂ is consistent for n → ∞ even for small T
• (α̂1 , ..., α̂n ) are consistent only for T → ∞

16 / 19
Within Estimator

Estimating n fixed effects can be computationally costly

• Inverting large matrices takes time even for R!

Alternative solution: within estimator

PT
• Construct ẏit = yit − ȳi where ȳi = 1
t=1 yit
T
PT
• Construct ẋit = xit − x̄i where x̄i = 1
t=1 xit
T
• Run the regression: ẏit = ẋit′ β + νit

The within estimator is equivalent to the fixed effects estimator

• Demeaning yit allows to get rid of αi in the model equation
• No need to estimate the fixed effects (α1 , ..., αn )!

17 / 19
Illustration in R

Example: yit = β1 + β2 xit + εit , with β = (0, 1)

• Assume that there is an unobserved omitted variable αi such that Cor(αi , xit ) = 0.5
• εit = αi + ηit where E[ηit |xit ] = 0 but E[εit |xit ] ̸= 0

Standard OLS estimator for β is biased and inconsistent

• Converges to 1.5 instead of 1 as sample size n grows
• EXO does not hold because of omitted variable

Fixed effects estimator for β is unbiased and consistent

• Converges to 1 as sample size n grows
• Solves for omitted variables bias!

18 / 19
Summary

EXO is a necessary assumption

• If it is not satisfied, no other solution than changing the data

Two strategies
• Experiments allow you to construct data such that EXO holds
• Panel data allow you to make EXO less stringent

So far we have focused on continuous outcomes y

• Many outcomes are binary or categorical: credit default, product choice, etc...
• Next chapter: generalize linear regression model to allow for discrete outcomes

19 / 19