Experimental Economics

Discrimination

Sevgi Yuksel

Partly based on slides by Heidi L. Williams.

Plan

• A very short overview of literature on discrimination.

• Discuss a paper that connects discrimination to biased attention:
“Seeing What is Representative”, by Ignacio Esponda, Ryan Oprea
and Sevgi Yuksel
How do define discrimination?

Very difficult!
How do define discrimination?

Very difficult!

Arrow (1973) motivates his definition of discrimination as follows: “The

fact that different groups of workers, be they skilled or unskilled, black or
white, male or female, receive different wages, invites the explanation that
the different groups must differ according to some characteristic valued on
the market. In standard economic theory, we think first of differences in
productivity. The notion of discrimination involves the additional concept
that personal characteristics of the worker unrelated to productivity are
also valued on the market.”
Three problems with such a definition

1. How to define equal productivity?

2. Technology determining productivity may not be exogenous.

3. Investments in productivity may be a function of pre-market

discrimination or expectations of labor market discrimination.
Theories of Discrimination

Can be divided into two classes:

1. Taste-based: Becker (1957) The Economics of Discrimination

− As if willing to pay to associate with some persons, not others
Implication: Conditional on being hired (or being paid the same
amount), discriminated group should be better qualified

2. Statistical: Phelps (1972), Arrow (1973)

− Employers have prior beliefs about productivity of group members
− Relevant if race, gender correlated with productivity
− Two kinds:
Initial beliefs correct
Implication: Each worker is paid according to her expected
productivity. There is not equal pay for equal productivity, but there is
equal pay for equal expected productivity. There is discrimination in
the sense that there is different pay granted to individuals with the
same true productivity
Initial beliefs incorrect (stereotypes)
Overview of empirical evidence on discrimination

1. Regression analysis
− Goldberger (1984); Neal and Johnson (1996)

2. Audit studies
− Bertrand and Mullainathan (2004)

3. Quasi-experiments
− Goldin and Rouse (2000); Anwar, Bayer, and Hjalmarsson (2012)

4. Testing Models
− Charles and Guryan (2008) Chandra and Staiger (2010)
Regression analysis

• Are men paid more than equally productive women?

• Are men less qualified than equally paid women?
• How much of the black-white earnings gap is explained by differences
in skills acquired prior to labor market entry?
− Payoff to skill lower for blacks → skill differences could reflect
anticipation that returns from acquiring skills will be low.
(Lundberg-Startz 1983)
− Intuitive, but difficult to test
− Example: Do returns to AFQT differ by race?
− Bottom line: More focus needed on understanding sources of observed
skill gaps between blacks and whites.
Audit studies

• Old literature (> four decades old) has tested for evidence of
discrimination in labor, housing, and product markets by conducting
‘audit’ field experiments
• Useful overview: Riach and Rich (2002)
• Conclusion of Riach and Rich: “...demonstrated pervasive and
enduring discrimination against non-whites and women”
Bertrand and Mullainathan (2004)

• Well-known audit resume study

• Sent 5,000 resumes to help-wanted ads in Boston and Chicago
• Randomized otherwise equivalent resumes to have African-American or
White sounding names: Emily Walsh or Greg Baker relative to Lakisha
Washington or Jamal Jones
• Also experimentally vary credentials
• What is causing discrimination?
Quasi experiments: Goldin and Rouse (2000)

• US symphony orchestras long conducted non-blind auditions

• Over time, some began using screens to hide performers
• Over time, notable increase in share female
• Headline estimate: blind auditions increase relative probability that
women advance from preliminary round by 50%
• Can’t distinguish between taste-based, statistical
Quasi experiments: Anwar, Bayer, and Hjalmarsson (2012)

• Examine the impact of jury racial composition on trial outcomes using

data on felony trials in FL from 2000-2010
• Exploit day-to-day variation in the composition of the jury pool to
isolate quasi-random variation in the composition of the seated jury
• Large racial gap (16pp) in conviction rates when no blacks in jury pool
• ≥ 1 black member in jury pool eliminates this gap
• White conviction rates sharply higher with ≥ 1 black member
• Headline estimate: racial gap in conviction rates is entirely eliminated
when the jury pool includes at least one black member
Testing models: Chandra and Staiger (2010)

• Large literature documenting evidence of disparities in health care

treatment and health outcomes
• Taste-based: providers use higher benefit threshold for providing care
to minority patients
− Implies that returns to the marginal minority patient receiving
treatment will be higher than the returns to the marginal non-minority
patient receiving treatment

• Statistical: minorities may have lower benefit from treatment both

models, minorities receive less treatment, but statistical implies
under-treatment may be optimal
• Argue results most consistent with statistical discrimination
• Unclear why minorities, women are less appropriate for treatment: key
to interpreting findings, public policy relevance
Attention Discrimination: Bartos, Bauer, Chytilova, and Matejka

• In cherry-picking markets (e.g., much of the labor market, admissions

to top schools, the scientific review process in leading scholarly
journals), decision makers should favor acquiring information about
individuals from an a priori more attractive group.
• In lemon-dropping markets (e.g., the rental housing market, admissions
to nearly open-access schools), decision makers benefit more from
acquiring information about individuals from a less attractive group.
• Why? Information should be acquired when its expected benefits are
higher.
• Evidence from in rental housing and labor markets in the Czech
Republic and in the labor market in Germany.
 Seeing What is Representative

by Ignacio Esponda, Ryan Oprea, Sevgi Yuksel

Main contributions of the paper

1. Document a new mistake in inference: Representative signal distortion

(RSD).
− Arises when DM forms an inference about an individual from a known
group (e.g., gender, race).
− Driven by representativeness: DM is able to contrast different groups
with distinct prior type distributions.

− DM distorts their evaluation of new information: misinterprets signals

as being more representative of the group the individual belongs to.
Main contributions of the paper

1. Document a new mistake in inference: Representative signal distortion

(RSD).
− Arises when DM forms an inference about an individual from a known
group (e.g., gender, race).
− Driven by representativeness: DM is able to contrast different groups
with distinct prior type distributions.

− DM distorts their evaluation of new information: misinterprets signals

as being more representative of the group the individual belongs to.

2. Study implications of RSD for statistical discrimination.

− Produces an irrational discriminatory gap in inferences.

− By limiting scope for RSD, we can decrease discrimination at not cost to

accuracy.
Representative signal distortion is distinct from other inferential biases

Biases in updating: combining prior with signal.

• RSD does not cause the DM to under-weigh the prior or the signal, but
distorts the perception of the signal.

Confirmation bias.
• Distortion of the signal in RSD is not driven directly the prior but by
the contrast between the prior and other distributions.

Incorrect beliefs about group differences (stereotypes).

• RSD operates (not by distorting prior beliefs) but by distorting the
way people perceive new information in updating contexts.
What we do

1. Present a model of RSD, showing how it can arise from

representativeness (extending Bordalo Coffman Gennaioli Shleifer 16).

2. Conduct a new experiment designed to identify this bias.

3. Study resulting statistical discrimination and the impact of

interventions that reduce scope for RSD.
Theoretical Framework
Inference problem

DM must estimate type t of an individual.

The individual belongs to a group g ∈ G.

Type distribution f (t | g ) depends on the group.

DM observes a noisy signal s on t, drawn from h(s | t, g ).

DM forms an estimate t̂ to minimize (t̂ − t)2 .

Inference problem

DM must estimate type t of an individual.

The individual belongs to a group g ∈ G.

Type distribution f (t | g ) depends on the group.

DM observes a noisy signal s on t, drawn from h(s | t, g ).

DM forms an estimate t̂ to minimize (t̂ − t)2 .

If t ∼ N µg , σ 2 and s = t + ϵ with ϵ ∼ N 0, ξ 2 ,

ξ2
t̂ Bay = ωµg + (1 − ω)s, where ω Bay = .
ξ2 + σ2

The predicted type is a random variable:

t̂ = ωµg + (1 − ω)t + (1 − ω)ϵ. (1)

Representative signal distortion

NEW: RSD can distort evaluation of new evidence.

Drawing on Gennaioli Shleifer 16, Bordalo Coffman Gennaioli Shleifer 16:

Misperceive signals by h̃ instead of h:

h̃(s | t, g ) = κh(s | t, g )(R(s, g , −g ))γ

• κ is a normalization factor,
• γ ≥ 0 measures distortion due to representativeness,
R(s, g , −g ) = yy(s(s| |−g
g) R
•
)
with y (s | g ) := h(s | t, g )f (t |, g )dt captures
how representative signal s is of group g given reference group −g .
Implications of contrast-biased evaluation in guassian setting

Assume t ∼ N µg , σ 2 and s ∼ N t, ξ 2 ,

Subjectively observed signal is s ∼ N (t + ∆g , ξ 2 ) with

ξ2
∆g = γ (µg − µ−g ).
ξ2 + σ2
Implications of contrast-biased evaluation in guassian setting

Assume t ∼ N µg , σ 2 and s ∼ N t, ξ 2 ,

Subjectively observed signal is s ∼ N (t + ∆g , ξ 2 ) with

ξ2
∆g = γ (µg − µ−g ).
ξ2 + σ2
Predictions:

1. ∆g > 0 for high-mean group, ∆g < 0 for low mean group.

2. ∆g = 0 in the absence of group contrasts.
3. ∆g = 0 when evaluation of s cannot be influenced by
representativeness.
Implications of contrast-biased evaluation in guassian setting

Assume t ∼ N µg , σ 2 and s ∼ N t, ξ 2 ,

Subjectively observed signal is s ∼ N (t + ∆g , ξ 2 ) with

ξ2
∆g = γ (µg − µ−g ).
ξ2 + σ2
Predictions:

1. ∆g > 0 for high-mean group, ∆g < 0 for low mean group.

2. ∆g = 0 in the absence of group contrasts.
3. ∆g = 0 when evaluation of s cannot be influenced by
representativeness.

The predicted type is a random variable:

t̂ = ωµg + (1 − ω)t + (1 − ω)∆g + (1 − ω)ϵ. (2)

Implications of contrast-biased evaluation in gaussian setting

Assume t ∼ N µg , σ 2 and s ∼ N t, ξ 2 ,

Subjectively observed signal is s ∼ N (t + ∆g , ξ 2 ) with

ξ2
∆g = γ (µg − µ−g ).
ξ2 + σ2
Predictions:

1. ∆g > 0 for high-mean group, ∆g < 0 for low mean group.

2. ∆g = 0 in the absence of group contrasts.
3. ∆g = 0 when evaluation of s cannot be influenced by
representativeness.

The predicted type is a random variable:

t̂ = ωµg + (1 − ω)t + (1 − ω)∆g + (1 − ω)ϵ. (3)

| {z }
Bg
Design
Design of Baseline treatment

• Series of inference tasks.

• Each task: predict true type of an individual based on:
(i) group identity (green vs. orange).
(ii) noisy signal about the individual.
Design of Baseline treatment

• Series of inference tasks.

• Each task: predict true type of an individual based on:
(i) group identity (green vs. orange).
(ii) noisy signal about the individual.
Key features of the design

• Abstract design.
− Removes confounds (taste-based discrimination).
− Controls prior beliefs and objective.

• Scope for misperception of signal.

Design
Design

Baseline NoGroup SignalFirst OneGroup

Info on: both groups both groups both groups only one group
First: group identity X signal group identity
Second: signal signal group identity signal
Design

Baseline NoGroup SignalFirst OneGroup

Info on: both groups both groups both groups only one group
First: group identity X signal group identity
Second: signal signal group identity signal
Design

Baseline NoGroup SignalFirst OneGroup

Info on: both groups both groups both groups only one group
First: group identity - signal group identity
Second: signal signal group identity signal
Design

Baseline NoGroup SignalFirst OneGroup

Info on: both groups both groups both groups only one group
First: group identity - signal group identity
Second: signal signal group identity signal
Other design details

• Incentives:
− Base payment of $7.5
− Chance of winning bonus $20 is (100 - MSE) percent.

• 241 Subjects, 75 Rounds.

− Remove 10 subjects with MSE > 200 in aggregate analysis.
− Analysis focuses on second half of session.

• Programmed with Qualtrics.

• Data from Prolific (June 19, 2021), lasting about 60 min.
Results
A first look at results

Baseline

75
65
Mean Assesment
55
45
35
25

30 40 50 60 70
Type
A first look at results

Baseline

75
65
Mean Assesment
55
45
35
25

30 40 50 60 70
Type
A first look at results

Baseline

75
65
Mean Assesment
55
45
35
25

30 40 50 60 70
Type
A first look at results

Baseline

75
65
Mean Assesment
55
45
35
25

30 40 50 60 70
Type
A first look at results

Baseline

75

B = 1.8, ω = 0.16
65
Mean Assesment
55
45
35

B = -1.7, ω = 0.20
25

30 40 50 60 70
Type
A first look at results

Baseline NoGroup

75
75

B = 1.8, ω = 0.16 B = -0.1, ω = 0.02

65
65
Mean Assesment

Mean Assesment
55
55

45
45

35
35

B = -1.7, ω = 0.20 B = -0.3, ω = 0.02

25

25

30 40 50 60 70 30 40 50 60 70
Type Type
A first look at results

OneGroup SignalFist

75

75
B = -0.2, ω = 0.16 B = 0.2, ω = 0.15

65
65
Mean Assesment

Mean Assesment
55
55

45
45

35
35

B = 0.4, ω = 0.09
25 B = -0.3, ω = 0.18
25

30 40 50 60 70 30 40 50 60 70
Type Type
Estimates of representative signal distortion, ∆g

Baseline NoGroup OneGroup SignalFirst

3
2
1
Δ
0
-1
-2
-3
Base-rate neglect, ω Bay − ω

Baseline NoGroup OneGroup SignalFirst

.4
.3
.2
.1
ωBay - ω
0
-.1
-.2
-.3
-.4
Individual-level estimates of representative signal distortion, ∆g

Baseline NoGroup

1
.8

.8
.6

.6
Cdf

Cdf
.4

.4
.2

.2
0

0
-10 -8 -6 -4 -2 0 2 4 6 8 10 -6 -4 -2 0 2 4 6
Δg Δg

OneGroup SignalFirst
1

1
.8

.8
.6

.6
Cdf

Cdf
.4

.4
.2

.2
0

-10 -8 -6 -4 -2 0 2 4 6 8 10 -10 -8 -6 -4 -2 0 2 4 6 8 10
Δg Δg
Individual-level estimates of base-rate neglect, ω Bay − ω

Baseline NoGroup

1
.8

.8
.6

.6
Cdf

Cdf
.4

.4
.2

.2
0

0
-.2 0 .2 .4 .6 -.2 0 .2 .4 .6
ωBay-ω ωBay - ω

OneGroup SignalFirst
1

1
.8

.8
.6

.6
Cdf

Cdf
.4

.4
.2

.2
0

-.2 0 .2 .4 .6 -.2 0 .2 .4 .6
ωBay- ω ωBay- ω
Measures of (in)accuracy and discrimination

Mean squared error, MSE := E(t̂ − t)2 .

R

Group difference in assessments, GD := E t̂h − t̂l | t dF (t).
• Linked to “seperation” criteria in ML fairness literature.
(Barocas Hardt Narayanan 2019, Narayanan 2018; Hutchinson Mitchell 2019)

• Notion of fairness reflected in “equal pay for equal work”.

More on discrimination measure

Accuracy-discrimination frontier

Social planner minimizes (for some χ ∈ [0, 1]):

χGD + (1 − χ)MSE . (4)

• χ = 0 is the Bayesian benchmark.

• χ = 1 is the No Discrimination benchmark.

µl +µh


Focus on linear strategies, i.e., t̂ = ω1 µg + ω2 2
+ (1 − ω1 − ω2 )s.
Accuracy-discrimination frontier
Accuracy-discrimination frontier
Implications for Statistical Discrimination

85
75
Inaccuracy (MSE)
65

Baseline
55
45
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Implications for Statistical Discrimination

85
75

NoGroup
Inaccuracy (MSE)
65

Baseline
55
45
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Implications for Statistical Discrimination

85
75

NoGroup
Inaccuracy (MSE)
65

Baseline
55
45

Bayesian
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Implications for Statistical Discrimination

85
pBRN
75

NoGroup
Inaccuracy (MSE)
65

Baseline
55
45

Bayesian
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Implications for Statistical Discrimination

85
pBRN
75

NoGroup
Inaccuracy (MSE)
65

Baseline
55

OptNoDiscrimination
45

Bayesian
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Implications for Statistical Discrimination

85
pBRN
75

NoGroup
Inaccuracy (MSE)
65

Baseline
55

OptNoDiscrimination
45

Bayesian
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Implications for Statistical Discrimination

85
pBRN
75

NoGroup
Inaccuracy (MSE)
65

Baseline
55

OptNoDiscrimination NoBias
45

Bayesian
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Implications for Statistical Discrimination

85
pBRN
75

NoGroup
Inaccuracy (MSE)
65

Baseline
55

OptNoDiscrimination NoBias
SignalFirst
45

Bayesian
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Implications for Statistical Discrimination

85
pBRN
75

NoGroup
Inaccuracy (MSE)
65

Baseline
55

OptNoDiscrimination NoBias
SignalFirst
45

Bayesian
OneGroup
35

-1 0 1 2 3 4 5 6 7 8 9 10
Discrimination (GD)
Using the model to improve outcomes

Model implies an algorithm to correct for RSD and base-rate negllect.

Using the model to improve outcomes

Model implies an algorithm to correct for RSD and base-rate negllect.

Divide data into training and testing sets.
• Estimate (∆, ω, ξ 2 ) on training set.
• “Adjust” predictions in testing set by
(i) debiasing,
(ii) readjusting weight on signal.
Using the model to improve outcomes

Model implies an algorithm to correct for RSD and base-rate negllect.

Divide data into training and testing sets.
• Estimate (∆, ω, ξ 2 ) on training set.
• “Adjust” predictions in testing set by
(i) debiasing,
(ii) readjusting weight on signal.

Focus on two types of adjustments: OptNoDiscrimination and Bayesian.

Using the model to improve outcomes

Model implies an algorithm to correct for RSD and base-rate negllect.

Divide data into training and testing sets.
• Estimate (∆, ω, ξ 2 ) on training set.
• “Adjust” predictions in testing set by
(i) debiasing,
(ii) readjusting weight on signal.

Focus on two types of adjustments: OptNoDiscrimination and Bayesian.

Table: Actual vs. Adjusted Predictions

Data OptNoDiscrimination Bayesian

GD 7.98 > (p = 0.000) 0.81 < (p = 0.002) 9.40
MSE 59 > (p = 0.088) 56 > (p = 0.000) 44
Values represent mean estimates from 500 repetitions of the procedure.
Inequalities compare counterfactuals (OptNoDiscrimination and Bayesian) to data.
Reported p-values represent frequency of repetitions in which the inequality was violated.
Conclusion

We document a new mistake: representative signal distortion.

− Evaluation of new evidence is distorted when there are multiple groups
with contrasting type distributions.
Conclusion

We document a new mistake: representative signal distortion.

− Evaluation of new evidence is distorted when there are multiple groups
with contrasting type distributions.

RSD generates inefficiencies in statistical discrimination.

− Eliminating the bias creates opportunities to reduce discrimination at
no cost to accuracy.