Fairness in

AI/ML
Goal: Understand and apply basic AI/ML techniques to data scenarios, with a focus on
instituting “fair” practices when designing decision-making systems based on big data.

Number of publications on
fairness from 2011 to 2017

[Link]
[Link] facebook-apple-hiring-techlash/
fairness-in-machine-learning-3ff8ba1040cb

Fairness in AI/ML
 Word
Embedding
(NLP)
⬣ Word embeddings are a set of techniques in natural language processing (NLP)
for identifying similarities between words in a corpus by using some type of
model to predict the co-occurence of words within a small chunk of text
⬣ Word embeddings transform human language meaningfully into a numerical
form. This allows computers to understand the nuances implicitly encoded into our
languages.

Male-Female Verb tense Country-Capital

Word Embedding (NLP)

 [Link]
chatbots-keep-turning-racist/

[Link]
microsofts-politically-
correct-chat-bot-is-
even-worse-than-its-
[Link] racist-one/
technology/2016/03/microsoft-terminates-its-
tay-ai-chatbot-after-she-turns-into-a-nazi/
[Link]
 [Link]

Word Association Game: Play “Blocks”

 Word Analogy
[Link]
• Write out a list of 10 occupations / job classifications
• Enter gender + two of the occupations (e.g. woman nurse scientist; man nurse
scientist)*
• Repeat
• Document / Submit Results

• Group assignment - complete, and then report back.

• * Tool traditionally only recognizes two gender classes – woman, man

 Word
Similarity &
Relatedness
⬣ Representing words has become a convenient way to compute similarities
⬣ Relatedness measures the semantic similarity between words

⬣ How similar is pizza to pasta?

⬣ How related is pizza to Italy?

Word Similarity & Relatedness

⬣ Vectorization is the process of
converting text to numbers.
This conversation helps us to
measure the similarity between
words.

⬣ A Vector space model is an

algebraic model for
representing text as a vector of
identifiers in which semantically
similar words are mapped to
proximate points in geometric
space

Vector Space Models

One representation (Document
Occurrence): Assign identifiers
corresponding to the count of words in
each document (from a cluster of
documents) in which the word occurs

Chocolate 0 1 0 1 0 0 2 0 1 0 1 0

Menu2 Menu4 Menu7 Menu9 Menu11

Vector Space Models

One representation (Word Context): Quantify co-occurrence of terms in a
corpus by constructing a co-occurrence matrix which captures the number of
times a term appears in the context of another term.

⬣ Chocolate is the best dessert in the world

⬣ GeorgiaTech is the best university in the world
⬣ The world runs on chocolate

chocolate best dessert university world

chocolate 0 1 1 0 2
best 1 0 1 1 1
desert 1 1 0 0 1
university 0 1 0 0 1
world 2 1 1 1 0

Vector Space Models

We have four (tiny) documents:

⬣ Document 1 : “atlanta falcons jerseys”

⬣ Document 2 : “atlanta falcons highlights”
⬣ Document 3 : “losangeles dodgers jerseys”
⬣ Document 4 : “losangeles dodgers highlights”

Example
 Document 1 Document 3
Document 2 Document 4

atlanta 1 1 0 0
similar
falcons 1 1 0 0

losangeles 0 0 1 1
similar
dodgers 0 0 1 1 ⬣ Document 1 : “atlanta falcons jerseys”
⬣ Document 2 : “atlanta falcons highlights”
⬣ Document 3 : “losangeles dodgers jerseys”
⬣ Document 4 : “losangeles dodgers highlights”

If we use document occurrence vectors…

 Los
Atlanta Falcons Jerseys Highlights Dodgers
Angeles
Atlanta 0 2 0 0 0 0

Falcons 2 0 1 1 0 0

Dodgers 0 0 1 1 0 2
similar
Los
0 0 0 0 2 0
Angeles

⬣ Document 1 : “atlanta falcons jerseys”

⬣ Document 2 : “atlanta falcons highlights”
⬣ Document 3 : “losangeles dodgers jerseys”
⬣ Document 4 : “losangeles dodgers highlights”

If we use word context vectors…

 Cosine
Similarity &
Word
Analogy
We can use cosine similarity to
compute the similarity between two
word vectors. However, this notion of
similarity depends on what vector
representation is selected to
represent the words found in your
corpus.

Falcons similar to Dodgers? Atlanta similar to Falcons?

OR
Because they are both sports Because “Atlanta Falcons”
teams (Go Falcons!)

Computing Similarity (Relatedness)

Given two vectors a and b, the cosine similarity is defined as the dot-product of the two
vectors divided by their length.

Vectors are quite similar Vectors are not similar Vectors are similar but
to each other opposite

Mathematically, cosine similarity measures the cosine of the angle between two vectors
projected in a multi-dimensional space, where the two vectors are the word vector as
mentioned previously.

Cosine Similarity
 Atlanta 1 1 0 0
Falcons 1 1 0 0
Los Angeles 0 0 1 1
Dodgers 0 0 1 1

Similarity (altanta, falcons) = 1.0

Similarity (altanta, los angeles) = 0.0
Similarity (altanta, dodgers) = 0.0
Similarity (falcons, los angeles) = 0.0
Similarity (falcons, dodgers) = 0.0
Similarity (los angeles, dodgers) = 1.0

Thus, as claimed before, based on cosine similarity Atlanta is similar to Falcons and
Los Angeles is similar to Dodgers

Cosine Similarity
⬣ Word analogy problems have become one of the standard tools for
evaluating context-based word vectors.
⬣ The task consists of questions like “a is to b as c is to ______”

man is to woman as king is to ______?

dog is to cat as bark is to ______?
Atlanta is to Georgia as Los Angeles is to ______?

⬣ To solve the analogy problem, we need to find the word vector that is
most similar to the result vector of [c] + [b] – [a]

[king] + [woman] - [man] ≈ [X]

Word Analogy Task

[Link]
Select English GoogleNews Model
Navigate to “Word analogy” section. Results?

⬣ man is to woman as king is to ________?

queen

⬣ China is to Beijing as Russia is to ________?

Kremlin
⬣ china is to beijing as russia is to _______________?
USA or Europe
⬣ Apple is to Jobs as Microsoft is to ________?
Ballmer
⬣ student is to teacher as doctor is to _______________?
dentist or nurse

⬣ student is to professor as doctor is to _________________________?

neurologist or cardiologist

Word Analogy Exercise

 Word
Embeddings
(word2vec)
⬣ Stores each word as a point in
space, where it is represented by a
vector of a fixed number of
dimensions (generally 300)

⬣ Unsupervised, built just by

reading huge corpus of data

⬣ For example, “Chocolate” might

be represented as [1, 0, 1, 1, 0, 2]

⬣ As discussed before, dimensions

are projections along different
axes

Word Embeddings
 King Good
King
Awesome

- Man Bad
Queen
Man Worst
+ Woman
Queen

Woman

Can thus learn analogies:

Similar words have same angles vector[Queen] = vector[King] - vector[Man] +
vector[Woman]

Examples
 Vector Space Models

⬣ One representation: Predict the context

of a given word by learning probabilities of
co-occurence from a corpus (e.g. skip-
gram neural network models,
word2vec)

⬣ In theory, words that share similar

contexts tend to have similar
meanings. As such, instead of counting
co-occurrences directly, we should be able
to generate word vectors that can predict
the context of a word based on its
surrounding words by learning from a
corpus of data.
⬣ Predicts words
using context

⬣ Two versions:
CBOW (continuous
bag of words) and
Skip-gram

Word2Vec Architecture
CBoW is a neural network that is trained to predict which word fits in a gap in a
sentence. For example, given the partial sentence “the student ___ the exam”, the
neural network predicts that “passed” has a high probability of filling the gap.

the

Example: “The student student

passed the exam”
passed
⬣ Window size = 2
the

exam

Word2vec: Continuous Bag of Words

Goal: predicting the target word given the surrounding words

1. eat|apple
Illustrative Corpus:
2. eat|orange
1. I like to eat apples
3. eat|rice
2. I like to eat oranges
4. drink|juice
3. I like to eat rice
5. drink|milk
4. I drink juice
6. drink|water
5. I drink milk
7. orange|juice
6. I drink water
8. apple|juice
7. I drink apple juice
9. rice|milk
8. I drink orange juice
10. milk|drink
9. I drink rice milk
11. water|drink
10. …
12. juice|drink

[Link]

Toy Training Data

Goal: predicting the target word given the surrounding words

Toy Training Data

⬣ Alternatively, Skip gram starts with a single
word embedding and tries to predict the
surrounding words.
⬣ Word2vec uses words a few positions away
from each center word to predict similarities
between every word and its context words
⬣ The pairs of center word/context word are
called “skip-grams.”
⬣ Example: “the student passed the exam”
⬣ Center word: red
⬣ Context words: blue Skip-gram

Word2vec – Skip gram

Goal: predict surrounding words within a window of each word

[Link]

Toy Training Data

 [Link]

⬣ Presets: King and Queen

⬣ Play around with configuration
settings:
⬣ hidden size
⬣ random_state
⬣ learning_rate
⬣ Are there any parameters that
seem to change results?

Word Embedding Visualization

Easiest way to use it is via the Gensim libarary for Python
⬣ pip install nltk
⬣ pip install gensim
⬣ Code: [Link]

Original word2vec C code by Google:

⬣ [Link]

Using word2vec
 Word: walk
Window size = 3 Window size = 30

Importance of Parameters: Window Size

 Word: walk
No. of iterations = 1 No. of iterations = 100

Importance of Parameters: Iterations

Bias in Word
Embedding
 Summary:
How to learn word2vec embeddings

⬣ Start with N random 300-dimensional

vectors as initial embeddings
⬣ Using a machine learning classifier:
⬣ Take a corpus and take pairs of words that
co-occur as positive examples
⬣ Take pairs of words that don't co-occur as
negative examples
⬣ Train the classifier to distinguish these by
slowly adjusting all the embeddings to
improve the classifier performance
⬣ Throw away the classifier code and keep
the embeddings.
 Don’t worry, the word2vec
cosine distance
Espresso? But cappuccino
between them is so
I ordered a small that they are espresso
cappuccino!
almost the same tea
thing. croissant

espresso

A dapt ed f rom a c art oon by Dmit ry Malk ov, a dat a s c ient ist and c art oonis t at Dat a Mons t ers

Bias: word2vec
 ht t ps : //www. [Link]/art icle/ 21316-a-i-doesn-t-get-black-t witt er-y et

Bias
ht t ps : //mot herboard. [Link]/
en_us / art icle/ j5jmj8/ google-
art if icial-intelligenc e-bias

Bias
h t t p s://[Link] stcomp an [Link]/3 01 02 23 /go og le -tr an sla te s-g en de r- pr ob lem-
a n d - b i n g- tra nsl ate s-a nd -systr an s h t t p s://www.b lo g.g oo gl e/ pr od ucts/tr an sla te/ re du cin g- ge nd er -
b i a s - g oo gl e- tra nsl ate /

Bias
 0.0001%

h t t p s://[Link] gg le .com/c/g en de re d- pr on ou n-
h t t p s://[Link] tista .co m/st ati stics/5 07 85 8/ al ph ab et- go og le -r d- costs/ r e s o l u tio nh ttps: //[Link] gg le .co m/c/g e nd er ed -p ro no un -r eso lu tio n

Big Tech Tackles Bias

 Bias

woman cooking man fixing faucet

Mark Yatskar, [Link]
 Bias

Mark Yatskar, [Link]

Why Does
This
Happen?
 engineer nurse leader pretty (all)
Ratio of he:she 6.25 0.550 9.25 3.07 3.53
co-occurrences
The New York Times Annotated Corpus (1987-2007, approx. 1B words, context window: 8)

For notes:
[Link]
8_MarcEtienneBrunet.pdf

Word Co-Occurrences
 Bias in Embeddings

⬣ Cultural Biases
⬣ Biased Framings of Women
⬣ Ethnic Stereotypes

⬣ African-American names are associated with

unpleasant words (more than European-
American names)
⬣ Male names associated more with career
words, female names with family words
⬣ Competence adjectives are biased toward
men
⬣ Smart, wise, brilliant, intelligent,
resourceful, thoughtful, logical, etc.
 Change in association of Chinese names with adjectives framed as
"othering" (barbaric, monstrous, bizarre)

Garg, Nikhil, Schiebinger, Londa, Jurafsky, Dan, and Zou, James (2018). Word embeddings quantify 100
years of gender and ethnic stereotypes.

Embeddings reflect ethnic stereotypes over time

 Leroy Jacqueline
A = {he, him, man… } ≈ Male B = {she, her, woman} ≈ Female

Aylin Caliskan, Joanna J. Bryson, Arvind Narayanan (Science 2017)

WEAT (Attribute Word Set)

 Science Art
S = {physics, chemistry… } ≈ Science T = {poetry, litterature… } ≈ Arts

Aylin Caliskan, Joanna J. Bryson, Arvind Narayanan (Science 2017)

WEAT (Target Word Set)

 Target Word Sets:
S= {physics,chemistry… }≈ Science Measures
S=Science
T = {poetry, litterature… } ≈ Arts relative
T=Arts
association d SB
between
Attribute WordSets: four dSA
A = {he, him, man… } ≈ Male concepts d TB
B = {she, her, woman} ≈ Female
dTA

A=Male
B=Female

Effect Size = (dSA- dSB)- (dTA - dTB)

WEAT
 1. Identify bias direction.
Non-bias
⬣ Calculate the difference between:
doctor
⬣ ehe - eshe
babysitter
⬣ emale - efemale
⬣ ....
bias
grandmother grandfather Average

girl boy 2. Neutralize: For every word that

is not definitional, project to
he get rid of bias.
she

Bolukbasi et. al., 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings

Addressing Bias in Word Embeddings

 1. Identify bias direction.
Non-bias
⬣ Calculate the difference between:
doctor
⬣ ehe - eshe
babysitter
⬣ emale - efemale
⬣ ....
bias
grandmother grandfather Average

girl boy 2. Neutralize: For every word that

is not definitional, project to
he get rid of bias.
she

Bolukbasi et. al., 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings

Addressing Bias in Word Embeddings

 1. Identify bias direction.
Non-bias
⬣ Calculate the difference between:
doctor
⬣ ehe - eshe
babysitter
⬣ emale - efemale
⬣ ....
bias
grandmother grandfather Average

girl boy 2. Neutralize: For every word that

is not definitional, project to
he get rid of bias.
she

Bolukbasi et. al., 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings

Addressing Bias in Word Embeddings

 1. Identify bias direction.
Non-bias
⬣ Calculate the difference between:
doctor
⬣ ehe - eshe
babysitter
⬣ emale - efemale
⬣ ....
bias
grandmother grandfather Average

girl boy 2. Neutralize: For every word that

is not definitional, project to
he get rid of bias.
she
3. Equalize pairs.

Bolukbasi et. al., 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings

Addressing Bias in Word Embeddings

[Link]
Find five images that you think would show bias based on the discussions in class
and feed into URL field (e.g. using google image search). Do the results show
bias? What type? Why do you think so? Examples:
Man in Kitchen:
[Link]
5/[Link]
Man with Children:
[Link]
Woman in Jail:
[Link]

Class Exercise
 Facial
Recognition
 Face Recognition
Example Applications:
⬣ Exam Identity Verification
⬣ Physical Building Access
⬣ Computer/Device Access
⬣ Passport Verification
⬣ Jail Management Systems and Booking
⬣ Law Enforcement Investigations
⬣ Photo Tagging in Facebook
⬣ Smartphone and App Access
Wang, Y., & Kosinski, M. (in press). Deep neural networks are more accurate than humans at
detecting sexual orientation from facial images. Journal of Personality and Social Psychology.
[Link]
to-find-late-passengers
[Link]
[Link]
law-enforcement-for-a-fistful-of-dollars
[Link]
 Facial
Recognition
Algorithms
 Timeline of Innovations
⬣ 1960s: Facial recognition is possible as
an application by joining the power of
computers with human skills
⬣ 1980s-90s: Advancements occurred in
the development of mapping and
recognition software
⬣ 2000s: Facial recognition became
integrated with surveillance applications
such as for law enforcement
⬣ 2010s: Faster, portable, more powerful
processors along with advancements in
deep learning
⬣ Face detection: to identify and
locate human faces in an image
regardless of their position,
scale, in plane rotation,
orientation, pose (out of plane
rotation), and illumination.

⬣ Two-class classification - Face

vs. Non-face
[Link]
⬣ The first step for any automatic taking-photo-public-domain-image-free-
109918115
face recognition system

Face Detection vs Face Recognition

 The objective of any face
detection algorithm is to locate
all faces, irrespective of:

⬣ Positioning (e.g. Frontal, Side

View, Upside Down)
⬣ Rotation and Pose
⬣ Occlusion
⬣ Resolution or image quality
⬣ In a single image or a
sequence of images involving
motion (such as in video)
⬣ After detecting the face, image is [Link]
typically segmented based on this [Link]/n
akela-stock-
information, and normalizes for photos-image-
free-1015613
translation, scale, rotations.
⬣ Multi-class classification - One person
[Link]
vs. all the others [Link]/n
akela-free-

⬣ Face Identification: Given an image that

stock-photo-
image-free-
1015565
belongs to a person in a database, tell
which person it is.
[Link]

⬣ Face Verification: Given an image,

[Link]/n
akela-stock-
photography-
verify whether it is from the same person image-free-
it is claiming to be. 1015582

Face Recognition
 In General
Face recognition algorithms “measure”
nodal points on the face such as the
distance between the eyes, the length of the
nose, and the angle of the jaw to identify
distinguishable features.
Features that usually do not change:
⬣ Upper ridges of the eye sockets
⬣ Nose shape
⬣ Size of the mouth
⬣ Positioning of major features relative to
each other
Face space is a theory
in psychology that
defines a
multidimensional
space in which
recognizable faces are
stored. The
representation of faces
within this space are
according to invariant
features of the face
itself.

Valentine’s (1991) “Multidimensional Face Space” model

 Human Biases
⬣ “Own Race” bias (Meissner and Brigham
2001)
⬣ We are over twice as likely to identify
own own-race than other-race
⬣ Less hits, more false alarms with
other-race faces than with own-race
faces.
⬣ "Own gender" bias (Wright and Sladden
2003).
⬣ "Own-age" bias (Wright and Stroud 2002;
Anastasi and Rhodes 2006; Perfect and
Moon 2005).
⬣ Appearance-based methods (classifier) can be trained, typically using
positive (and usually negative) examples of faces, for face recognition
applications
⬣ Deep Neural Networks are now one of the most common methods for face
recognition and in encoding attributes from the images

Valentine’s (1991) “Multidimensional Face Space” model

Deep Neural
Network
Examples
⬣ Largest facial dataset (in 2014)
⬣ Trained on four million images belonging to more than 4,000
identities uploaded by Facebook users.

[Link]
in-face-verification/

Facebook and DeepFace

[Link]
 [Link]
[Link]
[Link]
 Emotions: Facial Recognition
⬣ Emotions are reflected in voice, hand and
body gestures, and mainly through facial
expressions
⬣ By knowing a person’s emotion, an AI/ML
system can adapt to the user
⬣ Responding appropriately to the user’s
emotional state will be perceived as more
natural, engaging, and trusting
⬣ In addition to market research, emotion
detection can be used to monitor driver
attention, monitor movie audience reactions,
and even help healthcare professionals
assess the wellbeing of patients.
Cross-cultural research by Ekman shows (and has also been refuted by
others) that some emotional expressions are universal:

⬣ Happiness
⬣ Sadness
⬣ Anger
⬣ Fear
⬣ Disgust Surprise

General procedure amongst most AI systems:

⬣ Extract facial features for an image

⬣ Feed the features into a classifier such as a neural network
⬣ Classify the image/features to one of the pre-selected emotion categories
(6 universal emotions + neutral)

Facebook and DeepFace

 Action Description Facial Feature
Unit Image
⬣ Facial Action Units (AUs) represent 1 Inner Brow Raiser
distinct muscular activities in the face 2 Outer Brow Raiser
that can be used to classify emotional 4 Brow Lowerer
expressions 5 Upper Lid Raiser

⬣ Note: Surprise and Fear have a 7 Lid Tightener

significant number of AUs in common 20 Lip Stretcher

22 Lip Funneler

Emotion AUs associated with Emotion 23 Lip Tightener

Angry 4, 5 and/or 7, 22, 23, 24 24 Lip Pressor

Fear 1, 2, 4, 5, 7, 20, 25 or 26 25 Lips Apart

Surprise 1, 2, 5, 25 or 26 26 Jaw Drop

Example - AUs
[Link]
/facial-recognition-software-emotional-science

[Link]
observation-techniques-spot-program
Issues with
Facial
Recognition
Algorithms
The perpetrator in the Boston Marathon bombing, whose image was in a
national database, was not recognized in video surveillance images. Why?

?
Dzhokhar Tsarnaev caught Dzhokhar drivers' licenses photo
by surveillance cameras in the Mass. DMV database

[Link]
bombing-manhunt/

The Boston Marathon Bomber

 [Link]

Try it yourself!
Today’s facial recognition technology will reliably find this
photo in a mugshot database of controlled facial images
But in the wild, identification becomes problematic due to:
⬣ Resolution (not enough pixels)
⬣ Facial Pose – angulated
⬣ Illumination
⬣ Occluded facial areas
Resulting in:
⬣ Facial Feature Points (eyes, mouth, etc.) not found or distorted
⬣ Higher errors in Algorithm Measurements
⬣ Not Enough Data or Feature Points to Analze

Why it’s difficult…

 Error Rates - Effective?

⬣ “false positives” which means

wrongly matching innocent people with
photos in the database
⬣ “false negatives” not catching people
even when their photo is in the
database
No standards exist for “acceptable”
error rates meaning “success” is
subjective. It varies depending on the
facial recognition system used and the
application it’s used in.
 Bias in
Facial
Recognition
[Link]
[Link]
 v

Test suites,
e.g. [Link]

Difference in Recognition Rates

 Demographics in law enforcement databases ≠ general population. More
Priors males, more minorities, younger age.
The most prominent study [Klare et al.] found that several leading algorithms
Bias performed worse on underrepresented minorities, women, and young adults
than on Caucasians, men, and older people, respectively.
If the suspect is an underrepresented minority, the system is more likely to
Consequences erroneously fail to identify the right person, potentially causing innocent people
to be bumped up the list—and possibly even investigated
“Q: Is the Booking Photo Comparison System biased against minorities[?]”
“A: No… it does not see race, sex, orientation or age. The software is
Awareness matching distance and patterns only, not skin color, age or sex of an
individual.”- Frequently Asked Questions, Seattle Police Department
There is no independent testing regime for biased error rates … two major
No Tests for Bias face recognition companies admitted that they did not run these tests

Clare Garvie, Alvaro M. Bedoya, Jonathan Frankle The Perpetual Line-up Unregulated Police Face
Recognition In America Georgetown Law Center on Privacy and Technology, October 18, 2016
[Link]

Quoting Georgetown’s Report: “The Perpetual Line-up”

Why Does
This
Happen?
⬣ LFW: Labeled faces in the wild: A database for studying face
recognition in unconstrained environments
⬣ ~16K images

[Link]
[Link]

Typical Open-access Face Datasets

⬣ Training sets are hard to get
⬣ If you are testing your accuracy on equally non-
diverse image set, you will not get real-world
accuracy measurements
⬣ To address – some companies like Modiface
make an effort to pay for extra images.
⬣ Others, scrape the web to get the images that
they need, which sometimes causes pushback
⬣ A researcher scraped 40,000 selfies from Tinder
without permission and posted the dataset online
⬣ Another researcher scraped videos of
transgender people from YouTube

Addressing Bias in Facial Recognition

 Addressing Bias in Facial
Recognition
⬣ InclusiveFaceNet: Improving Face Attribute
Detection with Race and Gender Diversity
⬣ Google researchers discuss a system for
detecting face attributes by learning the
different attributes associated with
different race and gender identities.
⬣ Example face attributes for race
representation include: bangs, big lips,
blurry, chubby, double chin, narrow eyes, pale
skin, pointy nose, wearing necklace, and
young
[Link]
⬣ Startup company Gfycat dove into facial recognition to help people
find the perfect animated GIF response when messaging friends.
⬣ When tested in-house, the software stumbled on successfully
identifying Asian employees
⬣ The fix that they discovered - build in a kind of Asian-detector.
⬣ When a new photo comes in that the system determines is similar to
the cluster of Asian faces in its database, it flips into a more sensitive
mode, applying a stricter threshold before declaring a match.
⬣ The company says the system is now 98 percent accurate for
Caucasians and 93 percent accurate for Asians.
[Link]

Quoting Georgetown’s Report: “The Perpetual Line-up”

 Facial Recognition (Python)

⬣ pip install opencv-python

[Link]

⬣ pip install face_recognition

[Link]
 What are
Predictive
Algorithms?
 After report: Amazon was browbeaten by the mayors of Boston, Chicago,
and New York into providing its Prime same-day delivery service in low-
income neighborhoods previously denied this benefit
[Link]
[Link]
⬣ Define/learn models to estimate likely outcomes based on historical
data
⬣ Why Predictive Algorithms?
⬣ Some outcomes may be difficult to model except with examples
(e.g. recognition of faces).
⬣ The amount of knowledge needed to find correlations may be too
large for encoding by humans or by hand (e.g. in medical
diagnostics
⬣ Some problems may adapt quickly over time and continuous
redesign of the system by hand may be difficult (e.g. search
engines)

Predictive Algorithms
 Typical Prediction Tasks

⬣ Classification: classify an instance

into a category

⬣ Regression: predict a numerical label

for an unlabeled observation

⬣ Clustering: discovering groups of

similar instances
Classification
⬣ Supervised Learning: Classifies data (constructs a model) based on the
training data (observations, measurements, etc.) and the values (class labels)
indicating the class of the observations
⬣ New data is classified based on the training set
Prediction
⬣ Models continuous-valued functions, i.e., predicts unknown or missing
values
Prediction is similar to classification: Construct a model and use model to
predict unknown value
Prediction is different from classification: Prediction models continuous-
valued functions versus predicting categorical class labels

Classification vs. Prediction

 ⬣ Learn a target function f that maps
attribute set x to one of the predefined
Tid Refund Marital Taxable
class labels in y
Status Income Cheat

1 Yes Single 125K No

⬣ X = Refund, Material Status, Taxable
2 No Married 100K No Income
⬣ Y = Two Classes – Yes (Cheat), No
3 No Single 70K No
4 Yes Married 120K No
5 No Divorced 95K Yes (Cheat)
6 No Married 60K No
7 Yes Divorced 220K No
8 No Single 85K Yes ⬣ Predict:
9 No Married 75K No
10 No Single 90K Yes
10

Tax-return Data for Year 2018

Classification Example (Catching Tax Cheaters)

⬣ Given that we have observed a set of
Tid Subject Caps Email
training data: Used Word Spam
Length
{( X1 , Y1 ), ( X 2 , Y2 ),! , ( X n , Yn )} 1 Buy Yes 150 Yes
⬣ Can we determine if this email is spam 2 Act Now Yes 200 Yes
or not? 3 Meet Singles No 60 Yes
4 As Seen On No 120 Yes
5 Online Degree No 95 Yes
6 Viagra Yes 300 Yes
7 Weight Loss Yes 170 Yes
8 Online Masters No 1K No
9 Dear Lovely No 75 Yes
Weight
10 No 170 No
Management

Classification Example (Spam Filter)

1. Training set, which consists of records with known class labels. This is used
to build a classification model.
2. Testing set, which consists of previously unseen data records. This is used
to evaluate the quality of the model.
3. The classification model is applied to new records with unknown class
labels
⬣ To achieve good accuracy on the test data, training examples must be
sufficiently representative of the test data.
Assumption: The distribution of training examples is identical to the distribution of
test examples (including future unseen examples).
⬣ In practice, this assumption is often violated to certain degree.
⬣ Strong violations will clearly result in poor classification accuracy.

General Requirements for Classification

 Classification
Algorithms
Training
Data

NAME POSITION YEARS PROMOTED Classifier

John Developer I 3 no (Model)
Anika Developer I 7 yes
Raul Developer III 2 yes
Jim Developer III 7 yes IF position = ‘Developer II’
Harrie Developer I 6 no OR years > 6
Anne Developer III 3 no THEN promotion = ‘yes’

Classification Process (1): Model Generation

⬣ Counts of test records that are correctly (or incorrectly) predicted by the
classification model
⬣ Confusion matrix
Predicted Class
Predicted Class
Actual Class

positive negative
Class = 1 Class = 0
Actual positive TP FN
Class = 1 f11 f10
Class negative FP TN
Class = 0 f01 f00

# correct predictions f11 + f 00 TP True Positives

Accuracy = =
total # of predictions f11 + f10 + f 01 + f 00 TN True Negatives

# wrong predictions f10 + f 01 FP False Positives

Error rate = =
total # of predictions f11 + f10 + f 01 + f 00 FN False Negatives

Classification Process (2): Evaluation

Precision (PPV)TP / (TP+FP)
⬣ Percentage of correct positive predictions
Recall / Sensitivity TP / (TP + FN)
⬣ Percentage of positively labeled instances, also predicted as positive
Specificity TN / (TN + FP)
⬣ Percentage of negatively labeled instances, also predicted as negative
Speed and scalability
⬣ time to construct the model or time to use the model
Interpretability:
⬣ understanding and insight provided by the model

Performance Evaluation
 Classification
Algorithms
Training
Data

NAME POSITION YEARS PROMOTED Classifier

John Developer I 3 no (Model)
Anika Developer I 7 yes
Raul Developer III 2 yes
Jim Developer III 7 yes IF position = ‘Developer II’
Harrie Developer I 6 no OR years > 6
Anne Developer III 3 no THEN promotion = ‘yes’

Classification Process (3): Model Use

 Another Issue: Generalization

Learning is concerned with accurate

prediction of future data, not accurate
prediction of training data.

⬣ How do we ensure good

generalization, i.e. avoid “over-fitting”
on our particular data samples?
⬣ We are ultimately interested in good
performance on new (unseen) test
data (i.e. the population).
Predictive
Algorithm
Examples
[Link]
[Link]
[Link]
[Link]
[Link]
Learning
Methods
 Types of Learning

Supervised Learning
⬣ Labels are provided and there is a strong
learning signal present between inputs
and outputs.
⬣ e.g. classification, regression.
Unsupervised learning
⬣ There is no direct learning signal or
obvious correlations between inputs and
outputs. We’re simply trying to find some
structure in the data.
⬣ e.g. clustering, dimensionality reduction
Supervised Learning
Classification Regression
⬣ Logistic Regression ⬣ Linear Regression
⬣ Naive Bayes ⬣ Decision Tree
⬣ Decision Tree ⬣ Random Forest
⬣ Random Forest
⬣ Neural Networks/Deep NN Unsupervised Learning
Clustering
⬣ K-means Clustering

Example Classification/Prediction Techniques

⬣ Example: Imagine you want to classify versus

(nurse) (doctor)
⬣ Data: 100 nurse images and 100 doctor images with labels of what is what.
!
{xi ,yi = 0}, i = 1,...,100
!
{x j ,y j = 1}, j = 1,...,200
where x represents the greyscale of the image pixels and
y=0 means “nurse” while y=1 means “doctor”

⬣ Task: Here is a new image: nurse or doctor?

Supervised Learning
Clustering
⬣ Find the picture in the database which is closest to your query image
⬣ Check its label
⬣ Declare the class of your query image to be the same as that of the
closest picture.

decision
curve

Closest Image
Query
(doctor)
kNN Decision Surface

One Answer: Nearest Neighbor

Cluster 1 Cluster 2
Dog, cat, cow?

Unsupervised: The class labels of training data is unknown

Clustering
Finding a group structure in the data
⬣ Data in one cluster is similar to each other
⬣ Data in different clusters is dissimilar
The class labels of training data is unknown
⬣ Given a set of measurements, observations, etc.
with the aim of establishing the existence of
classes or clusters in the data
Map each data point to a discrete cluster index
⬣ Centroid-based: Assume k clusters, guess at the
centers, assign points to nearest center
⬣ Hierarchical: Assume one cluster per point, and
repeatedly merge nearby clusters using some
distance threshold

Clustering
⬣ Learn face similarity from training pairs labeled as same/different
⬣ Cluster faces based on identity

[Guillaumin, Verbeek, Schmid, ICCV 2009]

Clustering Example
⬣ Example input database: Two numerical variables
⬣ How many groups are here?

Customer Demographics

Age Salary
100
90 20 40
80 25 50
Salary in $10K

70
60
24 45
50 Customers 23 50
40 40 80
30
20
45 85
10 42 87
0
0 20 40 60 80
35 82
Age
70 30

Clustering
Regression
⬣ Training set: faces (represented as vectors of distances between keypoints)
together with experimentally obtained attractiveness rankings
⬣ Learn: function to reproduce attractiveness ranking based on training inputs
and outputs

Attractiveness score f(v)

Vector of distances v
T. Leyvand, D. Cohen-Or, G. Dror, and D. Lischinski, Data-driven enhancement of facial attractiveness, SIGGRAPH
2008

Regression Example
[Link]
Consider the following regression
problem:

⬣ Predict the real value on the y-

axis from the real value on the x-
axis.

⬣ You are given 6 examples: {Xi,Yi}.

⬣ What is the y-value for a new

X*
query point X* ?

Generalization
 ⬣ Ockham’s razor: prefer
the simplest hypothesis
consistent with data

⬣ Similarity/continuity
bias: similar inputs
should have similar
outputs

X* Which Curve is Best?

Generalization
⬣ Models with too many ⬣ Models with too few parameters
parameters may fit the training may not fit the data well (high
data well (low bias), but are bias) but are consistent across
sensitive to choice of training set different training sets (low
(high variance) variance)
⬣ Generalization error is due to ⬣ Generalization error is due to
overfitting underfitting

Bias & Variance Trade Off

Decision
Tree
⬣ Simple, practical and easy to interpret
⬣ Given a set of instances (with a set of features), a tree is constructed
with internal nodes as the features and the leaves as the classes
Phase One: Tree construction
⬣ At start, all the training examples are at the root
⬣ Examples are partitioned recursively based on selected attributes
Phase Two: Tree pruning
⬣ Branches are removed that are identified as noise or outliers
Use of decision tree: Classifying an unknown sample
⬣ The attribute values of a new data example are tested against the
decision tree

Decision Trees
 Attributes / Features
Attribute Values
Classes

Tid Refund Marital Taxable

Status Income Cheat Refund
1 Yes Single 125K No
Yes No
2 No Married 100K No
3 No Single 70K No
NO MarSt
4 Yes Married 120K No
Married
5 No Divorced 95K Yes Single, Divorced
6 No Married 60K No
7 Yes Divorced 220K No TaxInc NO
8 No Single 85K Yes
9 No Married 75K No
< 80K > 80K
10 No Single 90K Yes
10

NO YES
Training Data

Example of a Decision Tree

 Test Data
Refund Marital Taxable
Refund
Status Income Cheat
Yes No
No Married 80K ?
MarSt
10

NO
Married
Single, Divorced

TaxInc NO
< 80K > 80K

NO YES

Apply Model to Test Data

 Test Data
Refund Marital Taxable
Refund
Status Income Cheat
Yes No
No
No Married 80K ?
MarSt
10

NO
Married
Single, Divorced

TaxInc NO
< 80K > 80K

NO YES

Apply Model to Test Data

 Test Data
Refund Marital Taxable
Refund
Status Income Cheat
Yes No
No
No Married 80K ?
MarSt
10

NO
Married
Single, Divorced

TaxInc NO
< 80K > 80K

NO YES

Apply Model to Test Data

 Test Data
Refund Marital Taxable
Refund
Status Income Cheat
Yes No
No
No Married
Married 80K ?
MarSt
10

NO
Married
Single, Divorced

TaxInc NO
< 80K > 80K

NO YES

Apply Model to Test Data

 Test Data
Refund Marital Taxable
Refund
Status Income Cheat
Yes No
No
No Married
Married 80K ?
MarSt
10

NO
Married
Single, Divorced

TaxInc NO
< 80K > 80K

NO YES

Apply Model to Test Data

⬣ Data: What kind of data do we have? Do we have a sufficient
amount and representation of data to learn a good model?

⬣ Representation: How do we represent the data? How do we

categorize it?

⬣ Model space: What errors are we willing to entertain in the model to

explain the data?

⬣ Learning algorithm: Which learning algorithm do we choose and

how do we select its parameters?

⬣ Evaluation: How do we determine whether a model is good or good

enough?

Considerations
Crime Based
Predictive
Algorithms
in Use
[Link]
 ⬣ Algorithms identify
potential crime hot-
spots

⬣ Based on where
crime is previously
reported, not where it
is known to have
occurred

Smithsonian. Artificial Intelligence Is Now Used to Predict Crime. But Is It

Biased? 2018 ⬣ Predicts future
events from past

Predicting Policing
• Choose an address or zipcode (home, work place, …)
• E.g. 85 5th Street NW, Atlanta, GA 30308
• Enter into the Interactive Crime Map
• [Link]
• Enter into the Income Map
• [Link]
• Report Back
• Crime Layer (correlated with key) [E.g. Well above avg. (> 2x)]
• Income level [E.g. $80,132 ± $21,087]
• Majority Demographic [E.g. Nonwhite 53.3%]

Predictive Algorithms – Class Exercise

COMPAS
 COMPAS (Correctional
Offender Management
Profiling for Alternative
Sanctions):

⬣ 137-questions
questionnaire and
predictive model for "risk of
recidivism”

⬣ The model is a proprietary

secret of Northpointe, Inc.
Pro Publica, May 2016. [Link]
the-compas-recidivism-algorithm

Criminal Recidivism
 Supreme Court of Wisconsin (US)

The court ruled that judges are allowed

to use COMPAS scores, but they:

⬣ Must receive them accompanied by

disclaimers and criticisms

⬣ can use them as a factor to give non-

prison alternatives to prison

⬣ can use them as a factor to impose

terms and conditions in parole

Wall Street Journal, July 13th, 2016. [Link]

can-consider-predictive-algorithms-in-sentencing/
⬣ Tradeoffs must be built into the prediction algorithm and are policy
decisions.
⬣ Which mistake is worse and how much worse?

Trade-Offs
[Link]
 ⬣ Leads to retaining more high risk offenders but also more low risk
offenders

Race
ZIP Code Forecast Forecast
Age Arrest No arrest
Prior Record RISK Actual 100
400
Gender Arrest False Negatives
Current Changes 300
Actual No
Recency of Crime False 300
Arrest
Positives
Age at First Arrest
……

Predictors

One False Negative is Worth Three False Positives

[Link]
 Judges’
Release
Decisions vs.
Machine
Predictions and
Crime Risk
 Decisions to Jail vs. Release Defendants

⬣ Kleinberg et al. (2017) compare the accuracy

of human decisions and machine predictions in
the criminal justice system
⬣ Data: 750,000 individuals arrested in New York
City between 2008-2013
⬣ Same data on prior history that is available to
judge (rap sheet, current offense, etc.)
⬣ Data on subsequent crimes to develop and
evaluate performance of algorithm
⬣ Define “crime” as failing to show up at trial;
objective is to jail those with highest risk of
committing this crime
Judges’ Release Decisions vs. Machine Predictions and Crime Risk
Judges’ Release Decisions vs. Machine Predictions and Crime Risk

Predicted crime risk

is highly correlated
with observed
crime rates (for
defendants who are
released)
Judges’ Release Decisions vs. Machine Predictions and Crime Risk

Judges release
50% of defendants
whose predicted
crime risk exceeds
60%
Judges’ Release Decisions vs. Machine Predictions and Crime Risk

Judges release
50% of defendants
whose predicted
crime risk exceeds
60%

Yet they jail 30% of

defendants with
crime risks of only
20%
 Discrimination/suboptimal choice is not
Is it at least better than people? driven entirely by deep-rooted beliefs
Decisions can also vary greatly based
on factors unrelated to substantive
features of the issue at hand
⬣ Data: 1,100 judicial rulings on parole
for prisoners in Israel over 10 months
⬣ Judges review about 20 cases on
average each day in succession and
can grant parole or reject
⬣ Ordering of cases depends upon
when attorney shows up and is
essentially random

Biases due to Decision Fatigue

 Ethical Debate Regarding
Predictive Analytics

Use of big data for predictive analytics raises

serious ethical concerns, particularly in the context of
criminal justice
How large should the gains be from machine
prediction before its adopted?
Tension between two views:
⬣ Should a person be treated differently simply
because they share attributes with others who have
higher risks of crime?
⬣ Should police/judges/decision makers discard
information that could help make society fairer and
potentially more just than it is now on average?
 Bias in
Predictive
Algorithms
How would you Bananas
describe this Bananas on
with stickers
image? shelves

Bunches of
bananas with
Bananas stickers on them
on shelves in a
store

Bananas in
a store
Why don’t we tend
Did any of you say to say yellow
Bunches of yellow bananas? bananas?
bananas
Yet, bananas have different attributes:
⬣ Yellow
⬣ Green
⬣ Spotty
Why then is Yellow considered the
prototypical attribute for bananas whereas
everything else needs a label added?
This is an example of human biases. And it
causes issues because it means the data
that’s fed into our algorithms is labeled in this
biased way as well.

Human Bias Banana Example

 An Example Application

⬣ Imagine that you are an emergency room in a hospital

that collects data on newly admitted patients (e.g.,
blood pressure, age, gender, etc.)

⬣ The hospital needs to decide, based on that data,

how to quickly triage – i.e. assign priorities and
determine a patient’s priority for admission to the
emergency department and also the appropriate
place/type of treatment

⬣ Typically, the higher the severity, the higher the priority

⬣ The Problem: how to predict high-risk patients and

discriminate them from low-risk patients.
[Link]
patients/
Imagine that you are a credit card company that collects thousands of
applications every day for applicants applying for new credit cards. Each
application contains information about an applicant such as:
⬣ Age
⬣ Marital status
⬣ Annual salary
⬣ Outstanding Debts
⬣ Credit rating
⬣ etc.

The Problem: how to predict whether a credit application should be

approved or not approved and the amount of credit to provide

Another Example Application

[Link]
[Link]
/2019-11-10/apple-co-founder-says-
goldman-s-apple-card-algo-discriminates
[Link]
⬣ Predictive algorithms use labeled data to make predictions

⬣ Such labeled data is rampant with human biases

Examples of Human Bias in Labeling of Data Used in Predictive Algorithms

⬣ Reporting Bias

⬣ Selection Bias

⬣ Stereotypical Bias

⬣ Historical Biases

There is also the biases present in the collection of data as discussed earlier
[Link]
[Link]

Biases in Labeled Data

Ethical Issues
Ethical Issues