0% found this document useful (0 votes)

4 views682 pages

PC - Collections Sous Python

The document provides an overview of object collections in Python, focusing on tuples, lists, and dictionaries. It explains their creation, modification, and various operations, including indexing, concatenation, and methods associated with each type. Additionally, it includes exercises for practical application of the concepts discussed.

Uploaded by

mohammedsandida

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

4 views682 pages

PC - Collections Sous Python

Uploaded by

mohammedsandida

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

Python Programmation language

Chapitre 4

Object collections in Python

TUPLES, LISTS, DICTIONARIES

Static read-only array of heterogeneous objects

TUPLES
Creation of tuples and data access
#définition d'un tuple The ( ) are important to indicate that it is a
t1 = (2,6,8,10,15,26) tuple, « , » separates the items.
print(t1)
#taille du tuple (2,6,8,10,15,26)
print(len(t1)) 6 items
#accès indicé
Firstitem, indices go from 0 to len(t1)-1
a = t1[0]
print(a) Note: a is not a tuple
#modification ?
t1[2] = 3 ERROR
#plage d'indices
b = t1[2:5] Warning: we recover from n°2 (included) ton°5
print(b) (not-included) i.e.. the indices 2, 3, 4
#autre plage Result : b is a tuple with (8,10,15)
c = t1[:4]
print(c) The 4 first items i.e. the indices 0, 1, 2, 3 : we obtain the tuple
#indiçage négatif (2, 6, 8, 10).
d = t1[-1]
print(d) Le 1er last from the end :26
#indiçage négatif
e = t1[-3:]
print(e) The 3 last items : (10,15,26)
#concatenation
t2 = (7, 9,31) (2,6,8,10,15,26,7,9,31)
t3 = t1 + t2
print(t3)
(7,9,31,7,9,31)
#replication
t4 = 2 * t2
print(t4) it poses no problem
#tuples heterogeneous objects
v1 = (3,6,"toto",True,34.1)
print(v1)
Kind of 2-dimensional array
#tuple of tuples
x = ((2,3,5),(6,7,9),(1,8))
print(x) 
x[0] → (2,3,5) Organisationf of
# indexed access x[1] → (6,7,9) the structure
print(x[2][1]) → 8 x[2] → (1,8)
#access to size
print(len(x)) → 3 3 items on the 1ère dimension
print(len(x[2])) → 2 2 items in the tuple referenced by x[2]
Review of tuples

Type ‘tuple’ cf. call type() function

Collection of objects of heterogeneous types
Size and content fixed when writing the program
Cannot modify: non-mutable object
Tuple type variable is actually a reference (pointer to pointer)
Benefits from the crumb collection mechanism
Indexed access, range of indices possible, negative indices also possible
Complex structures with tuple of tuples, and even more – see later
Exercice

1. Crée un tuple couleurs = ('rouge', 'vert', 'bleu’).

2. Affiche le deuxième élément.
3. Essaie de modifier un élément du tuple. Que se passe-t-il ?
Dynamic read-write array of heterogeneous objects

LISTS
List – type list

List ≈ tuple of dynamic and modifiable size

 #définition d'une liste The[ ] are important indicates that it is a

list « , » separates items.
 L1 = [2,6,8,10,15,26]
 print(L1) [2,6,8,10,15,26]

 #taille de la tuple [2,6,3,10,15,26]

= 6 print(len(L1))

The other mechanisms associated

 #accès indicé
with tuples can be transposed to
= 2 a = L1[0]
lists:
 print(a) index ranges
negative indications
 #modification ! Possible heterogeneous objects
! list of lists (2D or more arrays)
 L1[2] = 3 concatenation, replication
 print(L1)
Modification of size and content
#autre liste A list is an object (class instance) to which methods
L2 = [32,69,28,69] are associated allowing it to be manipulated.
#ajout
[Link](21) [32,69,28,69,21]
print(L2)
#insertion à l'indice 1
[32,53,69,28,69,21]
[Link](1,53)
print(L2)
#suppression elt n°3
[32,53,69,69,21] 28 has disappeared
del L2[3]
from L2
print(L2)
#accès + suppression elt n°1
a = [Link](1) [32,69,69,21] 53 has disappeared
print(a) → renvoie 53 from L2
#inversion
[Link]()
[21,69,69,32]
print(L2) Note : [Link]()
#étendre Allow you to empty list
[Link]([34,55]) [21,69,69,32,34,55]
print(L2)
« List Comprehensions »

Objective : a simple (and concise) mechanism to generate a list from another list

Example 1 : Square all numbers

source = [1,5,8,12,7]
resultat = [] resultat = [v**2 for v insource]
for v in source: print(resultat)
[Link](v**2)
print(resultat)

Exemple 2 : conditionnal task

source = [1,5,8,12,7]
resultat = []
for v in source: resultat = [v**2 for v in source if (v %2== 0)]
if (v % 2 == 0): print(resultat)
[Link](v**2)
print(resultat)
Content Processing
L2 = [21,69,69,32,34,55]
#search for item
trouve = 32 in L2 Return True since the value
print(trouve) 32 is in the list

#index
Return 4 since the value 34 appears at
id = [Link](34)
index n°4 (index of the 1st found)
print(id)

#counting Return 2 since the value 69 appears

nb = [Link](69) twice in the list
print(nb)

#remove by value Remove the value 69 from the list,

the first that the method will find
[Link](69)
print(L2) [21,69,32,34,55]

This mechanism works with any type of object as long as a

comparison is possible (e.g. string, etc.)
A variable of list type is a reference
#L3
L3 = [61,92,17]
print(L3)
#affectation ? In reality , Is the reference that is copied
L4 = L3
print(L4) L3and L4 « point » to the same place
#modification d'une valeur
L4[1] = 55
#répercussions
print(L4) → [61,55,17]
#mais aussi sur L3
print(L3) → [61,55,17] ???

#L3
L3 = [61,92,17]

#copie des valeurs

L4referen ce a new memory area,
L4 = [Link]()
print(L4) And the Data in L3 y are copied.

L4[1] = 55
print(L4) → [61,55,17]

print(L3) → [61,92,17] !!! L3 is not impacted.

An example : sum of valu es ent ere d by t he u ser
An example (again): direct loop on the elements of the list

Allow to define and create

An initially empty list.

A list is directly
“iterable”, there is no need to
use an index
Exercice

1. Crée une liste nombres = [2,4,6,8,10] et affiche le troisième élément.

2. Ajoute le nombre 12 à la liste et supprime 4.

[Link] la liste et affiche chaque élément au carré.

[Link] le plus grand et le plus petit nombre dans la liste.

[Link] 5 nombres à l’utilisateur et stocke-les dans une liste, puis affiche la

moyenne.

6.Écris un programme qui inverse une liste.

7.Écris un programme qui fusionne deux listes [1,2,3] et [4,5,6].

8.Écris un programme qui compte combien de fois un élément apparaît dans une
liste.
A special case of list

String
A string is a particular list with associated methods
#définir une chaîne
s1 = "bonjour le monde"
Quotes to delimit a string
print(s1)
#longueur
long = len(s1) Mechanism identical to
print(long) tuples and lists
#accès indicé
s2 = s1[:7]
print(s2) ERROR. A string cannot be edited. It is necessary to put
#non modifiable
#s1[0] = "B" the result of a manipulation in another chain.
#méthodes associées
S = [Link]()
print(S)
#recherche d'une sous-chaîne
id = [Link]("JO") Specific methods allow you to
print(id) 3 (1ère occurrence si plusieurs) manipulate strings. See
#nb d'occurences [Link]
nb = [Link]("ON") [Link]#text-sequence-type-str
print(nb) 2
#remplacement de « O » par « A »
SA = [Link]("O","A")
print(SA)
Explicit transformation into a list (for processing)

A string can be transformed into a list to carry out sophisticated

processing. The tool is very flexible.

[‘B’,’O’,’N’,’J’,’O’,’U’,’R’,’ ‘,’L’,’E’,’ ‘,’M’,’O’,’N’,’D’,’E’]

#transf. en liste All operations on the lists are possible thereafter.
liste = list(S)
print(liste)
[‘BONJOUR’,’LE,’MONDE’]
Space is used as a separator here, but it
#découpage par séparateur
can be any other character, including a
decoupe = [Link](" ")
special character (e.g. \t for tab)
print(decoupe)

#former une chaîne à

#partir d’une liste
SB = "-".join(decoupe) "BONJOUR-LE-MONDE"
print(SB) The words in the list have been merged with
the "-" separator. Any separator is possible,
including the empty string.
An example
List with key access

DICTIONARIES
Dictionnary - The type dict Dictionnaire : unordred
#définition d'un dictionnaire
d1 = {'Pierre':17, 'Paul':15,'Jacques':16}
(unindexed) collection of
print(d1) objects (simple or
#ou complex) based on the
Noter le rôle de { }, de « :»
print([Link]())
et « ,» associative mechanism
#nombre d'élements « key – value ».
print(len(d1)) → 3 items

#liste des clés

[‘Paul’, ‘Jacques’, ‘Pierre’]
print([Link]())

#liste des valeurs [15, 16, 17]

print([Link]())

#accès à une valeur par clé Notes :

print(d1['Paul']) → 15 1) [Link]()
#ou empty the dictionnary
print([Link]('Paul')) → 15
(2) d1 is a reference,
#si clé n'existe pas [Link]() allow to
print(d1['Pipa']) → ERROR copy the content.
Dictionary – Modifications, additions and deletions
#modification
{'Pierre':17, 'Paul':15,'Jacques':16} → {'Pierre':17, 'Paul':15,'Jacques':18}
d1['Jacques'] = 18
print(d1) Adding by definition a new “key – value». N.B.:
If ‘Henri’ already exists, its old value will be
#ajouter un élément overwritten.
d1['Henri'] = 22
{'Pierre':17, 'Paul':15,'Jacques':18, ‘Henri’:22}
print(d1)

#ajout d'un bloc d’éléments

[Link]({'Monica':36,'Bill':49})
print(d1)
{'Pierre':17, 'Paul':15,'Jacques':18, ‘Henri’:22, ‘Monica’:36, ‘Bill’ : 49}

#détecter présence clé

test = 'Pierre' in d1
print(test) → True

#suppression par clé

del d1['Monica']
print(d1) {'Pierre':17, 'Paul':15,'Jacques':18, ‘Henri’:22, ‘Bill’ : 49}
Further with the keys

Keys are not necessarily strings. The tool is very flexible

but, be careful, so much freedom can also be
detrimental. You have to be very rigorous.

#autre type de clé

d2 = {('Pierre',56):['Directeur',1253,True],('Paul',55):['Employé',100,False]}
print([Link]())
print([Link]())

In thisexample :
• Key is a tuple;
• value is a list.
Example

Example :
Kate 15.0
Pipa 23.5
William 10.7

49.2
Exercice

1. Crée un dictionnaire etudiant = {'nom': 'Sara', 'age': 21, 'note': 16}

1. Affiche la valeur associée à la clé 'nom’.

2. Ajoute une nouvelle clé 'ville' avec une valeur.

3. Modifie la note de l’étudiant.

Parcourt le dictionnaire et affiche clé et valeur.

1. Crée une liste de dictionnaires pour représenter une classe de 3 étudiants.

2. Écris un programme qui demande un nom et cherche s’il existe dans le
dictionnaire.

3. Crée un dictionnaire qui compte le nombre de lettres dans un mot saisi par
l’utilisateur.
Machine
Learning A-Z
Course Slides
Data
Preprocessin
g
The Machine
Learning
Process
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
The Machine Learning Process
Data Pre-Processing
• Import the data
• Clean the data
• Split into training & test sets
• Feature Scaling

Modelling
• Build the model
• Train the model
• Make predictions

Evaluation
• Calculate performance metrics
• Make a verdict
Training Set
& Test Set
Training Set &Test Set

~
Train
80% 𝑦 = 𝑏0 + 𝑏1𝑋1 + 𝑏2𝑋2

Test
20% V.S.
Predicted values 𝑦! Actual values 𝑦
Feature
Scaling
Feature Scaling
Feature Scaling

Normalization Standardization

𝑋 − 𝑋𝑚𝑖𝑛 𝑋−𝜇
𝑋! = 𝑋! =
𝑋𝑚𝑎𝑥 − 𝑋𝑚𝑖𝑛 𝜎

[0 ;1] [-3 ;+3]

Feature Scaling

70,000 $ 45 yrs
10,000 1

60,000 $ 44 yrs
8,000 4

52,000 $ 40 yrs
Feature Scaling
Normalization

!
𝑋 − 𝑋 𝑚𝑖𝑛
𝑋 =
𝑋𝑚𝑎𝑥 − 𝑋𝑚𝑖𝑛

[0 ;1]
Feature Scaling

70,000 $ 45 yrs
60,000 $ 44 yrs
52,000 $ 40 yrs
Feature Scaling

1 45 yrs
0.444 44 yrs
0 40 yrs
Feature Scaling

1 1
0.444 0.7
5
0 0
Regression

©SuperDataScience
Simple Linear
Regression
Simple Linear Regression

𝑦! = 𝑏0 + 𝑏1 𝑋1
Dependent variable Independent variable

y-intercept (constant)

Slope coefficient
Simple Linear Regression
Each point represents
𝑦 [tonnes] a separate harvest
(Potato yield)

~
+3𝑡
𝑦= 𝑏0 + 𝑏1𝑋1

𝑃𝑜𝑡𝑎𝑡𝑜𝑒𝑠 𝑡 = 𝑏0 + 𝑏1×𝐹𝑒𝑟𝑡𝑖𝑙𝑖𝑧𝑒𝑟 𝑘𝑔
8𝑡

𝑏0 = 8[𝑡]
𝑡 𝑋1 [kg]
+1𝑘𝑔
𝑏1 = 3[ ] (Nitrogen Fertilizer)
𝑘𝑔
Ordinary
Least Squares
Simple Linear Regression
𝑦 [tonnes]
Ordinary Least Squares: (Potato yield)

𝑦𝑖
𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙: 𝜀𝑖 = 𝑦𝑖 − 𝑦^𝑖

𝑦^𝑖
𝑦𝑖

𝑦 = 𝑏0 + 𝑏1𝑋1 𝑦^𝑖

𝑏0, 𝑏1 such that:

𝑆𝑈𝑀(𝑦𝑖 − 𝑦𝑖 ) 2 is minimized
𝑋1 [kg]
(Nitrogen Fertilizer)
Multiple Linear
Regression
Multiple Linear Regression

𝑦= 𝑏0 + 𝑏1 𝑋1 + 𝑏2𝑋2 + ⋯ + 𝑏𝑛 𝑋𝑛
Dependent variable Independent variable 1 Independent variable 2 Independent variable n

y-intercept Slope coefficient 1 Slope coefficient 2 Slope coefficient n

(constant)
Multiple Linear Regression

~
𝑡 𝑡 𝑡
𝑃𝑜𝑡𝑎𝑡𝑜𝑒𝑠 𝑡 = 8𝑡 + 3 ×𝐹𝑒𝑟𝑡𝑖𝑙𝑖𝑧𝑒𝑟 𝑘𝑔 − 0.54 ×𝐴𝑣𝑔𝑇𝑒𝑚𝑝 °𝐶 + 0.04 ×𝑅𝑎𝑖𝑛[𝑚𝑚]
𝑘g °𝐶 𝑚𝑚
R Squared
R Squared
𝑦 [tonnes] Regression: 𝑦 [tonnes] Average:
(Potato yield) (Potato yield)

𝑦𝐴𝑣g
𝑦𝑖 𝑦𝑖

𝑦! 𝑖

𝑋1 [kg] 𝑋1 [kg]
(Nitrogen Fertilizer) (Nitrogen Fertilizer)

𝑆𝑆𝑟𝑒𝑠 = 𝑆𝑈𝑀(𝑦𝑖 − 𝑦! 𝑖 ) 2 𝑆𝑆𝑡𝑜𝑡 = 𝑆𝑈𝑀(𝑦𝑖 − 𝑦𝑎𝑣g)2

Rule of thumb (for our tutorials)*:
1.0 =Perfect fit (suspicious)
𝑆𝑆𝑟𝑒𝑠 ~0.9 =Very good
𝑅2 = 1 − <0.7 =Not great
𝑆𝑆𝑡𝑜𝑡 <0.4 =Terrible
<0 =Model makes no sense for this data

*This is highly dependent on the context

Adjusted
R Squared

©SuperDataScience
Adjusted R Squared
𝑆𝑆𝑟𝑒𝑠
𝑅2 = 1 − R2 – Goodness of fit
𝑆𝑆𝑡𝑜𝑡 (greater is better)
Problem:
𝑦
^ = 𝑏0 + 𝑏1𝑋1 + 𝑏2𝑋2 + 𝑏3𝑋3 𝑆𝑆𝑟𝑒𝑠 = 𝑆𝑈𝑀(𝑦𝑖 − 𝑦! 𝑖 ) 2
𝑆𝑆𝑡𝑜𝑡 doesn’t change
𝑆𝑆𝑟𝑒𝑠 will decrease or stay the same (This is because of Ordinary Least Squares: 𝑆𝑆𝑟𝑒𝑠->Min)

Solution:

𝑛−1
𝐴𝑑𝑗 𝑅2 = 1 − 1 − 𝑅2 ×
𝑛−𝑘−1
k – number of independent variables
n – sample size
Assumptions Of
Linear
Regression
Assumptions of Linear Regression

Anscombe's quartet (1973):

Assumptions of Linear Regression

[Link] 2. Homoscedasticity 3. Multivariate Normality

(Linear relationship between Y and each X) (Equal variance) (Normality of error distribution)

4. Independence 5. Lack of Multicollinearity 6. The Outlier Check

(of observations. Includes “no autocorrelation”) (Predictors are not correlated with each other) (This is not an assumption, but an “extra”)

𝑋1~ 𝑋2 𝑋1~ 𝑋2
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Profit R&D Spend Ad min Marketing State

192,261.83 165,349.20 136,897.80 471,784.10 New York

191,792.06 162,597.70 151,377.59 443,898.53 California
191,050.39 153,441.51 101,145.55 407,934.54 California
182,901.99 144,372.41 118,671.85 383,199.62 New York
166,187.94 142,107.34 91,391.77 366,168.42 California

y =b 0 +b1x1 +b2x2+b3*x3 + ???

Machine Learning A-Z

Dummy Variables

Profit R&D Spend Ad min Marketing State New York California

192,261.83 165,349.20 136,897.80 471,784.10 New York 1 0

191,792.06 162,597.70 151,377.59 443,898.53 California 0 1
191,050.39 153,441.51 101,145.55 407,934.54 California 0 1
182,901.99 144,372.41 118,671.85 383,199.62 New York 1 0
166,187.94 142,107.34 91,391.77 366,168.42 California 0 1

y =b 0 +b1x1 +b2x2+b3x3 +b4D1

Machine Learning A-Z

Machine Learning A-Z
Dummy Variables

Profit R&D Spend Ad min Marketing State New York California

192,261.83 165,349.20 136,897.80 471,784.10 New York 1 0

y =b 0 +b1x1 +b2x2+b3x3 +b4D1

Machine Learning A-Z

Dummy Variables

Profit R&D Spend Admin Marketing State New York California

192,261.83 165,349.20 136,897.80 471,784.10 New York 1 0

191,792.06 162,597.70 151,377.59 443,898.53 Califor nia 0 1
191,050.39
182,901.99
153,441.51
144,372.41
D =1- D
101,145.55
118,671.85 2
407,934.54
383,199.62
Califor nia
1New York
0
1
1
0
166,187.94 142,107.34 91,391.77 366,168.42 Califor nia
0 1

y =b 0 +b1x1 +b2x2+b3x3 +b4D1 +b5*D2

Machine Learning A-Z

Dummy Variables

Profit R&D Spend Ad min Marketing State New York California

192,261.83 165,349.20 136,897.80 471,784.10 New York 1 0

y =b 0 +b1x1 +b2x2+b3x3 +b4D1 +b5*D2

Always omit one dummy
variable
Machine Learning A-Z
Machine Learning A-Z
X3 X4 X5
X2
X6

X1 y X7

Why?
Machine Learning A-Z
1)

Machine Learning A-Z

5 methods of building
models:
1. All-in
2. Backward Elimination
3. Forward Selection Stepwise
4. Bidirectional Elimination Regression
5. Score Comparison

Machine Learning A-Z

“All-in” – cases:
• Prior knowledge; OR
• You have to; OR
• Preparing for Backward
Elimination

Machine Learning A-Z

Backward Elimination
STEP 1:Select a significance level to stay in the model (e.g. SL =0.05)

STEP 2: Fit the full model with all possible predictors

STEP 3: Consider the predictor with the highest P-value. If P >SL, go to STEP 4, otherwise go to FIN

STEP 4: Remove the predictor

STEP 5: Fit model without this variable*

FIN: Your Model Is Ready

Machine Learning A-Z

Forward Selection
STEP 1: Select a significance level to enter the model (e.g. SL =0.05)

STEP 2: Fit all simple regression models y ~ xn Select the one with the lowest P-value

STEP 3: Keep this variable and fit all possible models with one extra predictor added to the one(s) you
already have

STEP 4: Consider the predictor with the lowest P-value. If P <SL, go to STEP 3, otherwise go to FIN

FIN: Keep the previous model

Machine Learning A-Z

Bidirectional Elimination
STEP 1: Select a significance level to enter and to stay in the model
e.g.: SLENTER =0.05, SLSTAY =0.05

STEP 2: Perform the next step of Forward Selection (new variables must have: P <SLENTER to enter)

STEP 3: Perform ALL steps of Backward Elimination (old variables must have P <SLSTAY to stay)

STEP 4: No new variables can enter and no old variables can exit

FIN: Your Model Is Ready

Machine Learning A-Z

All Possible Models
STEP 1:Select a criterion of goodness of fit (e.g. Akaike criterion)

STEP 2: Construct All Possible Regression Models: 2N-1total combinations

STEP 3: Select the one with the best criterion

Example:
FIN: Your Model Is Ready 10 columns means
1,023 models

Machine Learning A-Z

5 methods of building models:
1. All-in
2. Backward Elimination
3. Forward Selection
4. Bidirectional Elimination
5. Score Comparison

Machine Learning A-Z

Machine Learning A-Z
In this section we learned:
1. How to create dummies for categorical IVs
2. How to avoid the dummy variable trap
3. Backward, Forward, Bidirectional, All Possible
[Link] actually built a model. Step-By-Step!!
5. How to use adjusted R-squared in modelling
6. How to interpret coefficients of a MLR

Machine Learning A-Z

Machine Learning A-Z
Simple
Linear
Regression

Multiple
Linear
Regression

Polynomial
Linear
Regression

Machine Learning A-Z

Machine Learning A-Z
Polynomial
Linear
Regression

Machine Learning A-Z

Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
1992
Vladimir Vapnik

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Ordinary Least Squares ε-Insensitive Tube
x2 SUM (y – ŷ)2 ->min x2

ε
ε

x1 x1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
𝑚
1 𝑤 2 +𝐶& 𝜉𝑖 + 𝜉∗ → 𝑚𝑖𝑛
𝑖
2
𝑖=1

Ordinary Least Squares ε-Insensitive Tube

x2 SUM (y – ŷ)2 ->min x2 Slack Variables ξi and ξi*

ξ5 ε
ξ3
ξ2 ε
ξ4*

ξ1*

x1 x1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Additional Reading:

Chapter 4 – Support Vector Regression

(from: Efficient Learning Machines:
Theories, Concepts, and Applications for
Engineers and System Designers)

By Mariette Awad & Rahul Khanna (2015)

Link:

[Link]

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
x1
ε

Machine Learning A-Z

x2
P¥thon -Google Drive x . support_vec1or_re@ essJcr .ipy” x data_preprocessir+g_template. ‹ x :.’.- data_qrepmces.sing_tooIs.ipynt. x: ':!7 polynomiaLjeg [Link] -i' x Com' cf suF@or¿v=c-cir_regres x

6 C fi [Link]’dri\’eNoL2xxfVBcfvE uCNjRpTpye/g6IJS‹81HKu#scroIITo --i•i_PG3 TAa‘7q0

1 Copy of support_vector_regression.ipynb
c o @ Comment Share @
File Edit View Insert Runtime Tools HeJp AII chąngessaved
RAM
+ Code + Text Disk -
Editing •••
:= Files
6 Upload C Refresh Mount Drive Position_SaIaries.csv X

1 to10 of 10 entries Filter

Position Level Salary
sampIe_data
Business Analyst 1 45000
@ Position [Link]
Junior Consultant 2 50000
Senior Consultant 3 60000
Manager 4 80000
Country Manager 5 140000
Region Manager 6 150000
Partner 7 200000
Senior Partner 8 300000

Visualising the SVR results (for higher resolution and smoother curve) C-level
CEO
9
1D
500000
1000000
Show ”zo t per page
1 x_grid = [Link](min(ec_X.inveree_transform(X)), max(sc_X.inveree_trane£orm(X)), 0.1)
2 x_grid = X_grid.reshape((len(X_grid), 1))
3 [Link](sc_x.inverse_transform(x), sc_y.inverse_tranaform(y1, color = 'red’)
sc_y.inyerse tranp€orm([Link](sq_X.transform(X_grid))), color = ’blue')
Bluff (Support Vector Regression)')
6 [Link]('Position level')
7 [Link]('Salary')
8 [Link]()

Disk 76.87 GB available

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Section on SVM:
• SVM Intuition

Section on Kernel SVM:

Y
• Kernel SVM Intuition
• Mapping to a higher dimension
• The Kernel Trick
• Types of Kernel Functions
• Non-linear Kernel SVR
X

Image source: [Link]

Machine Learning A-Z ©SuperDataScience

Split 3
200
Split 2
170

Split 4

20 40 X1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X1
Split 1

20
X2

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
No
X1<20

Yes

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X1
Split 2
Split 1

20
X2

170

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
No
X1<20

Yes

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
No
X2 <170

Yes
No
X1<20

Yes

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X2 Split 1

Split 3
200
Split 2
170

20 X1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
No
X2 <170

Yes
No
X1<20

Yes

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X1<20

Yes No

X2 <200 X2 <170

Yes No Yes No

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X2 Split 1

Split 3
200
Split 2
170

Split 4

20 40 X1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X1<20

Yes No

X2 <200 X2 <170

Yes No Yes No

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X1<20

Yes No

X2 <200 X2 <170

Yes No Yes No

X1 <40

Yes No

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Y
X2 Split 1

65.7
Split 3 1023
200
Split 2
170

300.5 -64.1 0.7

Split 4

20 40 X1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X1<20

Yes No

X2 <200 X2 <170

Yes No Yes No

X1 <40

Yes No

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X1<20

Yes No

X2 <200 X2 <170

Yes No Yes No

300.5 65.7 X1 <40 1023

Yes No

-64.1 0.7

Machine Learning A-Z ©SuperDataScience

STEP 2: Build the Decision Tree associated to these K data points.

STEP 3: Choose the number Ntree of trees you want to build and repeat STEPS 1& 2

STEP 4: For a new data point, make each one of your Ntree trees predict the value of Y to
for the data point in question, and assign the new data point the average across all of the
predicted Y values.

Machine Learning A-Z ©SuperDataScience

R Squared

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
R Squared
𝑦 [tonnes] 𝑦 [tonnes]
Regression: Average:
(Potato yield) (Potato yield)

𝑦𝐴𝑣g
𝑦𝑖 𝑦𝑖

𝑦! 𝑖

𝑋1 [kg] 𝑋1 [kg]
(Nitrogen Fertilizer) (Nitrogen Fertilizer)

𝑆𝑆𝑟𝑒𝑠 = 𝑆𝑈𝑀(𝑦𝑖 − 𝑦! 𝑖 ) 2 𝑆𝑆𝑡𝑜𝑡 = 𝑆𝑈𝑀(𝑦𝑖 − 𝑦𝑎𝑣g)2

*This is highly dependent on the context

©SuperDataScience
Adjusted
R Squared

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Adjusted R Squared
𝑆𝑆𝑟𝑒𝑠
𝑅2 = 1 − R2 – Goodness of fit
𝑆𝑆𝑡𝑜𝑡 (greater is better)
Problem:
𝑦
^ = 𝑏0 + 𝑏1𝑋1 + 𝑏2𝑋2 + 𝑏3𝑋3 𝑆𝑆𝑟𝑒𝑠 = 𝑆𝑈𝑀(𝑦𝑖 − 𝑦! 𝑖 ) 2
𝑆𝑆𝑡𝑜𝑡 doesn’t change
𝑆𝑆𝑟𝑒𝑠 will decrease or stay the same (This is because of Ordinary Least Squares: 𝑆𝑆𝑟𝑒𝑠->Min)

Solution:

𝑛−1
𝐴𝑑𝑗 𝑅2 = 1 − 1 − 𝑅2 ×
𝑛−𝑘−1
k – number of independent variables
n – sample size

©SuperDataScience
Classification

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
What is Classification?
Classification: a Machine Learning technique to identify the
category of new observations based on training data.

Likely to stay Likely to leave Dogs Cats

©SuperDataScience
Logistic
Regressio
n

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Logistic Regression
Logistic regression: predict a categorical 𝑦 [yes/no]
dependent variable from a number of (Took up offer?)
independent variables.

YES

~ YES

81%

≥ 50%
Will purchase Age
< 50%
health insurance:
Yes / No 42%

𝑝 NO
ln = 𝑏0 + 𝑏1𝑋1 NO
1−𝑝 18 35 45 60
𝑋1 [yrs]
(Age)

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Logistic Regression

~
Will purchase Age Income Level of Family or
health insurance: Education Single
Yes / No

𝑝
ln = 𝑏0 + 𝑏1𝑋1 + 𝑏2𝑋2 + 𝑏3𝑋3 + 𝑏4𝑋4
1–𝑝

©SuperDataScience
Maximum
Likelihood

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Maximum Likelihood
𝑦 [yes/no] 𝑦 [yes/no]
(Took up offer?) (Took up offer?)

YES YES
0.95 0.98 1-0.96
0.92
1-0.58

0.54

1-0.10
0.03 1- 0.01 1-0.04

NO NO
𝑋1 [yrs] 𝑋1 [yrs]
18 60 18 60
(Age) (Age)

Likelihood =0.03 x 0.54 x 0.92 x 0.95 x 0.98 x (1– 0.01) x (1– 0.04) x (1– 0.10) x (1– 0.58) x (1– 0.96)

Likelihood =0.00019939

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Maximum Likelihood
𝑦 [yes/no]
(Took up offer?)
Likelihood =0.00007418

Likelihood =0.00012845

YES Likelihood =0.00016553

Likelihood =0.00019939

Best Curve <= Maximum Likelihood

NO
𝑋1 [yrs]
18 60
(Age)

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Before K-NN After K-NN
X2 x2

Category 2 Category 2

New data point K-NN New data point assigned

to Category 1

Category 1 Category 1

x1 x1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 1:Choose the number K of neighbors

STEP 2: Take the K nearest neighbors of the new data point, according to the Euclidean distance

STEP 3: Among these K neighbors, count the number of data points in each category

STEP 4: Assign the new data point to the category where you counted the most neighbors

Your Model is Ready

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 1:Choose the number K of neighbors: K =5
x2

Category 2

New data point

Category 1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
y

y2 P2(x2,y2)

y1
P1(x1,y1)

x1 x2 x

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
x2

Category 1:3 neighbors

Category 2
Category 2:2 neighbors
New data point

Category 1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
x1
x2

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Maximum Margin

Support
Vectors

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Maximum Margin

x2 Positive Hyperplane

Maximum Margin
Hyperplane
(Maximum Margin Classifier)

Support
Vectors

Negative Hyperplane

Machine Learning A-Z ©SuperDataScience

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Support
Vectors

x1
x2

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Linearly Separable Not Linearly Separable
x2 x2

x1 x1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
f =x - 5

x1
0

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
f =(x – 5)^2
f =x - 5

x1
0

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
f =(x – 5)^2
f =x - 5

x1
0

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2D Space
New
Dimension
z 3D Space
x2

Hyperplane

Mapping Function

x1
x1
Machine Learning A-Z ©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2D Space
z 3D Space x2

Non Linear Separator

Projection

x1
Machine Learning A-Z ©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Mapping to a Higher Dimensional Space
can be highly compute-intensive

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
2
→ 𝑖
− 2σ 2
𝑥→−𝑙
𝑒 𝐾 𝑥⃗, 𝑙⃗𝑖 =

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2
𝑥→−𝑙→𝑖
𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒− 2σ 2

Image source: [Link]

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2D Space
x2

2
𝑥→−𝑙→𝑖
𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒− 2σ 2

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2D Space
x2

2
𝑥→−𝑙→𝑖
𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒− 2σ 2

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2D Space
x2

2
𝑥→−𝑙→𝑖
𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒− 2σ 2

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
x2 𝐾 𝑥⃗, 𝑙⃗1 + 𝐾 𝑥⃗, 𝑙⃗2
(Simplified Formula)

Green when:
𝐾 𝑥⃗, 𝑙⃗1 + 𝐾 𝑥⃗, 𝑙⃗2 >
Red when:
𝐾 𝑥⃗, 𝑙⃗1 + 𝐾 𝑥⃗, 𝑙⃗2 =

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2
𝑥→−𝑙→𝑖
Gaussian RBF Kernel 𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒− 2σ2

Sigmoid Kernel 𝐾 𝑋, 𝑌 = tanh 𝛾 X𝑋 𝑇𝑌 + 𝑟

Polynomial Kernel 𝐾 𝑋, 𝑌 = 𝛾 X𝑋 𝑇𝑌 + 𝑟 𝑑 , 𝛾 > 0

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Section on SVR:
• SVR Intuition
Section on SVM:
• SVM Intuition
Y
Section on Kernel SVM:
• Kernel SVM Intuition
• Mapping to a higher dimension
• The Kernel Trick
• Types of Kernel Functions X

• Non-linear Kernel SVR

Image source: [Link]

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X
Y

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X
Y

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X
Y

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X
Y

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X
Y

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X
Y

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2
𝑥→–𝑙→𝑖
– 2σ2
𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒

Y Y

X X

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2
𝑥→–𝑙→𝑖
– 2σ2
𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒

Y Y

X X

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2
𝑥→–𝑙→𝑖
– 2σ2
𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒

Y Y

X X

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
X
Y

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Y 2
𝑥→–𝑙→𝑖
– 2σ2
𝐾 𝑥⃗, 𝑙⃗𝑖 = 𝑒

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
m1 m1 m1 m1 m1 m1 m1 m1 m1 m1 m1 m1 m1

m2 m2 m2 m2 m2 m2 m2 m2 m2 m2

Machine Learning A-Z ©SuperDataScience

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
m2

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
𝑃 𝐵 𝐴 ∗ 𝑃(𝐴)
𝑃(𝐵)
𝑃 𝐴𝐵 =

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
𝑃 𝐵 𝐴 ∗ 𝑃(𝐴)
𝑃(𝐵)
𝑃 𝐴𝐵 =

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
X2
Ca
Dtreiv
geosry 2

CW
ata
elgko
s ry 1

Age
X1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives

Walks

New data point

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
𝑃(𝐵)
𝑃 𝐵𝐴
𝑃 𝐴𝐵 =
∗ 𝑃(𝐴)

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝑊𝑎𝑙𝑘𝑠 ∗ 𝑃(𝑊𝑎𝑙𝑘𝑠)
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝐷𝑟𝑖𝑣𝑒𝑠 ∗ 𝑃(𝐷𝑟𝑖𝑣𝑒𝑠)
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 𝑣. 𝑠. 𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Age
Drives

Walks
Salary

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑊𝑎𝑙𝑘𝑒𝑟𝑠
𝑃(𝑊𝑎𝑙𝑘𝑠) =
𝑇𝑜𝑡𝑎𝑙 𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
10
Walks 𝑃 𝑊𝑎𝑙𝑘𝑠 =
30

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝑊𝑎𝑙𝑘𝑠 ∗ 𝑃(𝑊𝑎𝑙𝑘𝑠)
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑆𝑖𝑚𝑖𝑙𝑎𝑟 𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
𝑃(𝑋) =
𝑇𝑜𝑡𝑎𝑙 𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
4
Walks 𝑃 𝑋 =
30

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝑊𝑎𝑙𝑘𝑠 ∗ 𝑃(𝑊𝑎𝑙𝑘𝑠)
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑆𝑖𝑚𝑖𝑙𝑎𝑟
𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
𝐴𝑚𝑜𝑛𝑔 𝑡ℎ𝑜𝑠𝑒 𝑤ℎ𝑜 𝑊𝑎𝑙𝑘
𝑃(𝑋|𝑊𝑎𝑙𝑘𝑠) =
Walks
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑊𝑎𝑙𝑘𝑒𝑟𝑠
3
𝑃 𝑋|𝑊𝑎𝑙𝑘𝑠 =
10

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝑊𝑎𝑙𝑘𝑠 ∗ 𝑃(𝑊𝑎𝑙𝑘𝑠)
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

3 ∗ 10
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 = 10 4 30 = 0.75
30
#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝐷𝑟𝑖𝑣𝑒𝑠 ∗ 𝑃(𝐷𝑟𝑖𝑣𝑒𝑠)
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

1 ∗ 20
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 = 20 4 30 = 0.25
30
#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 𝑣. 𝑠. 𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
0.75 𝑣. 𝑠. 0.25

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
0.75 > 0.25

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 > 𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives

Walks

New data point

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives

Walks

New data point

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝐷𝑟𝑖𝑣𝑒𝑠 ∗ 𝑃(𝐷𝑟𝑖𝑣𝑒𝑠)
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Age
Drives

Walks
Salary

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝐷𝑟𝑖𝑣𝑒𝑟𝑠
𝑃(𝐷𝑟𝑖𝑣𝑒𝑠) =
𝑇𝑜𝑡𝑎𝑙 𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
20
Walks 𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 =
30

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝐷𝑟𝑖𝑣𝑒𝑠 ∗ 𝑃(𝐷𝑟𝑖𝑣𝑒𝑠)
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝐷𝑟𝑖𝑣𝑒𝑠 ∗ 𝑃(𝐷𝑟𝑖𝑣𝑒𝑠)
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑆𝑖𝑚𝑖𝑙𝑎𝑟
𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
𝐴𝑚𝑜𝑛𝑔 𝑡ℎ𝑜𝑠𝑒 𝑤ℎ𝑜 𝑊𝑎𝑙𝑘
𝑃(𝑋|𝐷𝑟𝑖𝑣𝑒𝑠) =
Walks
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑊𝑎𝑙𝑘𝑒𝑟𝑠
1
𝑃 𝑋|𝐷𝑟𝑖𝑣𝑒𝑠 =
20

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝐷𝑟𝑖𝑣𝑒𝑠 ∗ 𝑃(𝐷𝑟𝑖𝑣𝑒𝑠)
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

1 ∗ 20
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 = 20 4 30 = 0.25
30
#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

Walks

New data point

Age

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Salary
Drives 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑆𝑖𝑚𝑖𝑙𝑎𝑟 𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
𝑃(𝑋) =
𝑇𝑜𝑡𝑎𝑙 𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠
4
Walks 𝑃 𝑋 =
30

Age
NOTE: Same both times

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝑊𝑎𝑙𝑘𝑠 ∗ 𝑃(𝑊𝑎𝑙𝑘𝑠)
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
#3 Likelihood #1 Prior Probability
#4 Posterior Probability

𝑃 𝑋 𝐷𝑟𝑖𝑣𝑒𝑠 ∗ 𝑃(𝐷𝑟𝑖𝑣𝑒𝑠)
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋 =
𝑃(𝑋)

#2 Marginal Likelihood

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋
𝑣. 𝑠.
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
𝑃 𝑋 𝑊𝑎𝑙𝑘𝑠 ∗ 𝑃(𝑊𝑎𝑙𝑘𝑠) 𝑃 𝑋 𝐷𝑟𝑖𝑣𝑒𝑠 ∗ 𝑃(𝐷𝑟𝑖𝑣𝑒𝑠)
𝑣. 𝑠.
𝑃(𝑋) 𝑃(𝑋)

Machine Learning A-Z ©SuperDataScience

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
0.75 𝑣. 𝑠. 0.25

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
0.75 > 0.25

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
𝑃 𝑊𝑎𝑙𝑘𝑠 𝑋 > 𝑃 𝐷𝑟𝑖𝑣𝑒𝑠 𝑋

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Split 1

x1
60
x2

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Split 1

x1
Split 2

50
60
x2

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
x2 Split 2

60 Split 3 Split 1

50 70
x1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
x2 Split 2

60 Split 3 Split 1

Split 4

50 70
x1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Split 1

x1
60
x2

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
No
X2 <60

Yes

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Split 1

x1
Split 2

50
60
x2

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
No
X1 <50

Yes
No
X2 <60

Yes

Machine Learning A-Z

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
x2 Split 2

60 Split 3 Split 1

50 70
x1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X2 <60

Yes No

X1 <70 X1 <50

Yes No Yes No

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
x2 Split 2

60 Split 3 Split 1

20 Split 4

50 70
x1

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X2 <60

Yes No

X1 <70 X1 <50

Yes No Yes No

X2 <20

Yes No

Machine Learning A-Z ©SuperDataScience

STEP 2: Build the Decision Tree associated to these K data points.

STEP 3: Choose the number Ntree of trees you want to build and repeat STEPS 1& 2

STEP 4: For a new data point, make each one of your Ntree trees predict the category to
which the data points belongs, and assign the new data point to the category that wins
the majority vote.

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
p̂ŷ (Probability)
y(Predicted DV)
(Actual DV)

ŷ =1 ŷ =1
1

0.5

X
ŷ =0 ŷ =0

20 30 40 50

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
p̂y
(Probability)
(Actual DV)

#2 #4
1

0.5

#1 #3
X

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
False Positive
y (Actual DV)
p̂ (Probability) (Type I Error)

ŷ (Predicted DV) #2 #4
1

0.5

#1 #3
X

False Negative
(Type II Error) Fin.

Machine Learning A-Z ©SuperDataScience

Confusion
Matrix &
Accuracy

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Confusion Matrix &
Accuracy Prediction
NEG POS

TRU FALS
NEG
E E
NE PO
Actual

G S

FALS TRU
POS
E E
NE PO
Type II Error Type I Error
G
(False Negatives) S (False Positives) Image source: [Link]

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Confusion Matrix &
Accuracy Prediction
NEG POS Accuracy Rate and Error Rate:

𝐶𝑜𝑟𝑟𝑒𝑐𝑡 𝑇𝑁 + 𝑇𝑃 84
𝐴𝑅 = = = = 84%
𝑇𝑜𝑡𝑎𝑙 𝑇𝑜𝑡𝑎𝑙 100
NEG 43 12
Actual

𝐼𝑛𝑐𝑜𝑟𝑟𝑒𝑐𝑡 𝐹𝑃 + 𝐹𝑁 16
𝐸𝑅 = = = = 16%
𝑇𝑜𝑡𝑎𝑙 𝑇𝑜𝑡𝑎𝑙 100
POS 4 41

Type II Error Type I Error

(False Negatives) (False Positives)

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Additional Reading

Understanding the Confusion Matrix from Scikit

learn

Samarth Agrawal (2021)

Link:

[Link]
confusion-matrix-from-scikit-learn-c51d88929c79

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
ŷ (Predicted DV) Scenario 1:

Accuracy Rate =Correct / Total

0 1 AR =9,800/10,000 =98%

0 9,700 150
y (Actual

1 50 100
DV)

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
ŷ (Predicted DV) Scenario 1:

Accuracy Rate =Correct / Total

0 1 AR =9,800/10,000 =98%

0 9,850 0 Scenario 2:
y (Actual

Accuracy Rate =Correct / Total

AR =9,850/10,000 =98.5%
1 150 0
DV)

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Purchased

10,000

8,000

6,000

4,000

2,000

0
0 20,000 40,000 60,000 80,000 100,000 Total Contacted

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Purchased Crystal Ball Good Model

100%

80%

60%
Poor Model
40%
Random
20%

0
10%
0 20% 40% 60% 80% 100% Total Contacted

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Note:

CAP =Cumulative Accuracy Profile

ROC =Receiver Operating Characteristic

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Purchased Model

100%

80%

60%

40%
Random
20%

0
0 20% 40% 60% 80% 100% Total Contacted

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Purchased Perfect Model Good Model

100%

80%
aP
aR
60%
aR
AR = aP
40%
Random Model
20%

0
0 20% 40% 60% 80% 100% Total Contacted

Machine Learning A-Z ©SuperDataScience

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Purchased Perfect Model Good Model

100%
X%
80%
90% <X <100% Too Good
80% <X <90% Very Good
60% 70% <X <80% Good
60% <X <70% Poor
X <60% Rubbish
40%
Random Model
20%

0
50%
0 20% 40% 60% 80% 100% Total Contacted

Machine Learning A-Z ©SuperDataScience

Clustering

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
What is Clustering?

Clustering – grouping
unlabelled data

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
What is
Clustering? Supervised Learning
(e.g. Regression, Classification)

Unsupervised Learning
(e.g. Clustering)

Image source: [Link]/2073-8994/10/12/734

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
What is Clustering?
Spending Score Spending Score

Clustering

Annual Income $ Annual Income $

©SuperDataScience
K-Means
Clustering

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
K-Means Clustering

©SuperDataScience
The Elbow
Method

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
The Elbow Method

Within Cluster Sum of Squares:

...

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
The Elbow Method

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Cluster 1
C1
The Elbow Method

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Cluster 2
C2
Cluster 1
The Elbow Method

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
The Elbow Method
C2 Cluster 2

C1
Cluster 1

Cluster 3
C3

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
The Elbow Method
The Elbow Method

Optimal number of clusters

©SuperDataScience
K-Means++

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
K-Means++
Cluster 2
K-Means
Cluster 1

Cluster 3

Different results

Cluster 2
K-Means
Cluster 3

Cluster 1

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
K-Means++
K-Means++ Initialization Algorithm:

Step 1: Choose first centroid at random among data points

Step 2: For each of the remaining data points compute the distance (D)
to the nearest out of already selected centroids

Step 3: Choose next centroid among remaining data points using

weighted random selection – weighted by D2

Step 4:Repeat Steps 2 and 3 until all k centroids have been selected

Step 5: Proceed with standard k-means clustering

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
K-Means++

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Cluster 2

Cluster 3
Cluster 1
K-Means++

©SuperDataScience
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
After HC

Same as K-Means but different process

HC
Before HC
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 1:Make each data point a single-point cluster That forms N clusters

STEP 2: Take the two closest data points and make them one cluster That forms N-1
clusters

STEP 3: Take the two closest clusters and make them one cluster That forms N - 2
clusters

STEP 4: Repeat STEP 3 until there is only one cluster

FIN
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
x
P2(x2,y2)

x2
P1(x1,y1)

x1
y2

y1
y
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Distance Between Two Clusters:

• Option 1:Closest Points

• Option 2: Furthest Points

• Option 3: Average Distance

• Option 4: Distance Between Centroids

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Consider the following dataset of N =6 data points
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 1: Make each data point a single-point cluster That forms 6 clusters
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 1: Make each data point a single-point cluster That forms 6 clusters
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 2: Take the two closest data points and make them one cluster
That forms 5 clusters
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 3: Take the two closest clusters and make them one cluster
That forms 4 clusters
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 4: Repeat STEP 3 until there is only one cluster
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 4: Repeat STEP 3 until there is only one cluster
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 4: Repeat STEP 3 until there is only one cluster

FIN
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
P6
P5
P4
P3
P2
P1
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
P6
P5
P4
P3
P2
P1
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
P6
P5
P4
P3
P2
P1
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
P6
P5
P4
P3
P2
P1
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
P6
P5
P4
P3
P2
P1
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
P6
P5
P4
P3
P2
P1
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
2 clusters
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
4 clusters
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience
6 clusters
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience

Largest distance
2 clusters
P1
P3

Machine Learning A-Z

P6
P5
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIB UTION © SUPERDATASCIENCE [Link] [Link]
©SuperDataScience

Largest distance
3 clusters

P1
P3
P2
P4
P6
P5

Machine Learning A-Z

P9
P8
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
People who bought also bought …
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
User ID Movies liked
46578 Movie1, Movie2, Movie3, Movie4
98989 Movie1, Movie2
71527 Movie1, Movie2, Movie4
78981 Movie1, Movie2
89192 Movie2, Movie4
61557 Movie1, Movie3

Movie1 Movie2

Potential Rules: Movie2 Movie4

Movie1 Movie3
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Transaction ID Products purchased
46578 Burgers, French Fries, Vegetables
98989 Burgers, French Fries, Ketchup
71527 Vegetables, Fruits
78981 Pasta, Fruits, Butter, Vegetables
89192 Burgers, Pasta, French Fries
61557 Fruits, Orange Juice, Vegetables
87923 Burgers, French Fries, Ketchup, Mayo

Burgers French Fries

Potential Rules: Vegetables Fruits

Burgers, French Fries Ketchup

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Market Basket Optimisation:
Movie Recommendation:
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Support =10 / 100 =10%
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Market Basket Optimisation:
Movie Recommendation:
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Confidence =7 / 40 =17.5%
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Market Basket Optimisation:
Movie Recommendation:
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Lift =17.5%/ 10%=1.75
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Step 1:Set a minimum support and confidence

Step 2: Take all the subsets in transactions having higher support than minimum support

Step 3: Take all the rules of these subsets having higher confidence than minimum confidence

Step 4: Sort the rules by decreasing lift

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
People who bought also bought …
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
User ID Movies liked
46578 Movie1, Movie2, Movie3, Movie4
98989 Movie1, Movie2
71527 Movie1, Movie2, Movie4
78981 Movie1, Movie2
89192 Movie2, Movie4
61557 Movie1, Movie3

Movie1 Movie2

Potential Rules: Movie2 Movie4

Potential

Rules:

Vegetables
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Market Basket Optimisation:
Movie Recommendation:
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Step 1:Set a minimum support

Step 2: Take all the subsets in transactions having higher support than minimum support

Step 3: Sort these subsets by decreasing support

Examples used for educational purposes. No affiliation with Coca-

Cola
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
D5
D4
D3
D2
D1
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Where we think the μ* values will be
I.e. We are NOT trying to guess the distributions behind the machines

Return
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
We’ve generated our own bandit configuration
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
We’ve generated our own bandit configuration
New Round
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
We’ve generated our own bandit configuration
New Round
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
• Deterministic • Probabilistic
• Requires update at every round • Can accommodate delayed feedback
• Better empirical evidence
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Here’s what we will learn:
• Types of Natural Language Processing
• Classical vs Deep Learning Models
• End-to-end Deep Learning Models
• Bag-Of-Words

• Note: Seq2Seq and Chatbots are outside the

scope of this course
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Learning
Deep

Seq2Seq
DNLP
Processing
Language
Natural
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Learning
Deep

Seq2Seq
DNLP
Processing
Language
Natural
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Some examples:
1. If / Else Rules (Chatbot)
2. Audio frequency components analysis (Speech
Recognition) NLP DL
3. Bag-of-words model (Classification)
4. CNN for text Recognition (Classification)
5. Seq2Seq (many applic ations)
Comment Pass/Fail
Great job! 1
Amazing work.
Well done.
1
Yes
1
I’m back EOS Seq2Seq
Very well written. 1
Poor effort. 0
Could have done better. 0
h0 h1 h2Try harder
h3 next time.
… h0
n g0 g1 g2
… …

Hello Kirill , Checking EOS Yes I’m back

Encoder Decoder

Image Source: [Link]

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
DL
NLP
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
DL
NLP
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
DL
NLP
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
DL
NLP
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Deep Learning
End-to-end

Models
DL
NLP
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... ,0]

20,000 elements long

if badminton table
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... ,0]

20,000 elements long

SOS Special
EOS Words
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Hello Kirill, Checking if you are back to Oz. Let me know if you are around …Cheers, V

[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... ,0]

20,000 elements long

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Hello Kirill, Checking if you are back to Oz. Let me know if you are around …Cheers, V

[1,1,0, 0, 1,0, 2, 0, 1,0, 0, 0, 0, 0, 1,2, 0, 0, 0, 1,0, 0, 1,0, 0, ... ,3]

20,000 elements long

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Hello Kirill, Checking if you are back to Oz. Let me know if you are around …Cheers, V

[1,1,0, 0, 1,0, 2, 0, 1,0, 0, 0, 0, 0, 1,2, 0, 0, 0, 1,0, 0, 1,0, 0, ... ,3]

20,000 elements long

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Hello Kirill, Checking if you are back to Oz. Let me know if you are around …Cheers, V

[1,1,0, 0, 1,0, 2, 0, 1,0, 0, 0, 0, 0, 1,2, 0, 0, 0, 1,0, 0, 1,0, 0, ... ,3]

20,000 elements long

Training Data:
Hey mate, have you read about Hinton’s capsule networks?
Did you like that recipe I sent you last week?
Hi Kirill, are you coming to dinner tonight?
Dear Kirill, would you like to service your car with us again?
Are you coming to Australia in December?
…
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Hello Kirill, Checking if you are back to Oz. Let me know if you are around …Cheers, V

[1,1,0, 0, 1,0, 2, 0, 1,0, 0, 0, 0, 0, 1,2, 0, 0, 0, 1,0, 0, 1,0, 0, ... ,3]

20,000 elements long

[1,1,0, 0, 1,0, 2, 0, 1,0, 0, 0, 0, 0, 1,2, 0, 0, 0, 1,0, 0, 1,0, 0, ... ,3]

20,000 elements long

Training Data:
[1,1,0, 0, 0, 1,0, 0, 1,1,0, 0, 0, 0, 0, 1,0, 1,0, 1, 0, 0, 1,0, 0, ... ,2]
[1,1,0, 0, 0, 0, 0, 1,0, 0, 0, 0, 0, 0 ,0, 2, 0, 0, 0, 1,0, 0, 1,0, 0, ... ,0]
[1,1,0, 0, 0, 0, 0, 1,0, 0, 1,0, 0, 0, 0, 1,0, 0, 0, 1,0, 0, 0, 0, 1,... ,1]
[1,1,0, 0, 0, 0, 0, 1,0, 0, 1,0, 0, 0, 0, 1,1,0, 1,0, 0, 0, 0, 0, 0, ... ,1]
[1,1,0, 0, 0, 0, 0, 0, 1,0, 0, 0, 0, 0,0, 1,0, 0, 1,1,0, 0, 0, 1,0, ... ,1]
Imag…
e Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
2017

25,600x
1980

2x
1956
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Source: [Link]
Log-scale
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Source: Time Magazine
INPUT INPUT
INPUT INPUT

INPUT INPUT

INPUT

OUTPUT
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Geoffrey Hinton
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1

Input value 2
Output value

Input value 3

Input Hidden Output

Layer Layer Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Output Layer
Input Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Output Layer
Hidden Layers
Input Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Artificial Neural Networks Used for Regression & Classification
Supervised

Convolutional Neural Networks Used for Computer Vision

Recurrent Neural Networks Used for Time Series Analysis

Self-Organizing Maps
Unsupervised

Used for Feature Detection

Deep Boltzmann Machines Used for Recommendation Systems

AutoEncoders Used for Recommendation Systems

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
What we will learn in this
section:
• The Neuron
• The Activation Function
• How do Neural Networks work? (example)
• How do Neural Networks learn?
• Gradient Descent
• Stochastic Gradient Descent
• Backpropagation
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: Wikipedia
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: Wikipedia
Axon

Neuron
Dendrites
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: Wikipedia
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Node

neuron
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
neuron

Input signal m
Input signal 1

Input signal 2
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Output signal
neuron

Input signal m
Input signal 1

Input signal 2
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1

Input value 2 X2 neuron Output signal

Input value m Xm

Synapse
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1

Input value 2 X2 neuron y Output value

Input value m Xm

Synapse
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
Independent
variable 1

Input value 2
Independent
X2 neuron y Output value

variable 2

Input value m Xm
Independent
variable m

Standardize
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Additional Reading:

Efficient BackProp

By Yann LeCun et al. (1998)

Link:
[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
Independent
variable 1

Input value 2
Independent
X2 neuron y Output value

variable 2

Can be:
• Continuous (price)
• Binary (will exit yes/no)
Input value m Xm • Categorical
Independent
variable m
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
Independent
variable 1 y1 Output value 1

Input value 2
Independent
X2 neuron y2 Output value 2
variable 2

y3 Output value p
Input value m Xm
Independent
variable m
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
Independent
variable 1

Input value 2
Independent
X2 neuron y Output value

variable 2

Input value m Xm
Independent
variable m

Same observation
Single Observation Single Observation
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
w1

w2
Input value 2 X2 neuron y Output value

wm
Input value m Xm
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
w1

Input value 2 X2 w2
?
neuron y Output value

wm
Input value m Xm
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
w1

1st step:
Input value 2 X2 w2 y Output value

wm
Input value m Xm
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
w1

2nd step:
Input value 2 X2 w2 y Output value

wm
Input value m Xm
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
w1

2nd step:
Input value 2 X2 w2 3rd step
y Output value

wm
Input value m Xm
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
w1

2nd step:
Input value 2 X2 w2 3rd step
y Output value

wm
Input value m Xm
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
0
1
y
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Threshold Function

0
1
y
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Sigmoid

0
1
y
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Rectifier

0
1
y
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Hyperbolic Tangent (tanh)

-1
1
y
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Additional Reading:

Deep sparse rectifier

neural networks

By Xavier Glorot et al. (2011)

Link:
[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1
w1

2nd step:
3rd step
Input value 2 X2 w2 y Output value

wm
Input value m Xm
If threshold activation function:

Assuming the DV is binary (y =0 or

1)
If sigmoid activation function:
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Input value 1 X1

Output value
Input value 2 X2 y

Input value m Xm

Input Hidden Output

Layer Layer Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1
w1

Bedrooms X2 w2 Price =w1x1+w2x2+ w3x3+ w4x4

y
w3
Distance to city (Miles) X3

w4
Age X4

Input Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Area (feet2) X1

Bedrooms X2

y Price

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

Input value 2 X2 w2 y
ŷ Output value

wm
Input value m Xm
y Actual value
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
ŷ y C
Input value 1 X1
w1

Input value 2 X2 w2 ŷ Output value

C =½(ŷ- y)2
wm
Input value m Xm
y Actual value
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
ŷ y C
Input value 1 X1
w1

Input value 2 X2 w2 ŷ Output value

X2 w2 ŷ X2 w2 ŷ

wm wm
Xm Xm
y y

X1 X1
w1 w1

C =∑ ½(ŷ- y)2
X2 w2 ŷ X2 w2 ŷ

wm wm
Xm Xm
y y

X1 X1

Adjust w1, w2, w3

w1 w1

X2 w2 ŷ X2 w2 ŷ

wm wm
Xm
y
Xm
y
C
X1 X1
w1 w1

X2 w2 ŷ X2 w2 ŷ

A list of cost functions used in

neural networks, alongside
applications

CrossValidated (2015)

Link:

[Link]
functions-used-in-neural-networks-alongside-applications
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X1 X1
w1 w1

X2 w2 ŷ X2 w2 ŷ

wm wm
Xm Xm
y y

X1 X1
w1 w1

C =∑ ½(ŷ- y)2
X2 w2 ŷ X2 w2 ŷ

wm wm
Xm Xm
y y

X1 X1

Adjust w1, w2, w3

w1 w1

X2 w2 ŷ X2 w2 ŷ

wm wm
Xm
y
Xm
y
C
X1 X1
w1 w1

X2 w2 ŷ X2 w2 ŷ

Bedrooms X2

y Price

Distance to city (Miles) X3

Age X4

Input Layer Hidden Layer Output Layer

Bedrooms X2

y Price

Distance to city (Miles) X3

25 weights
Age X4

Input Layer Hidden Layer Output Layer

Sunway TaihuLight: World’s fastest Super Computer

93 PFLOPS

93 x 1015

1075 / (93 x 1015)

=1.08 x 1058 seconds

=3.42 x 1050 years

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
C =½(ŷ- y)2

ŷ
C
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
C =½(ŷ- y)2

X2 w2 ŷ X2 w2 ŷ

wm wm
Xm Xm
y y

X1 X1
w1 w1

C =∑ ½(ŷ- y)2
X2 w2 ŷ X2 w2 ŷ

wm wm
Xm Xm
y y

X1 X1

Adjust w1, w2, w3

w1 w1

X2 w2 ŷ X2 w2 ŷ

wm wm
Xm
y
Xm
y
C
X1 X1
w1 w1

X2 w2 ŷ X2 w2 ŷ

wm wm
Xm Xm
y y

X1 X1
w1 w1

C =∑ ½(ŷ- y)2
X2 w2 ŷ X2 w2 ŷ

wm wm
Xm Xm
y y

X1 X1

Adjust w1, w2, w3

w1 w1

X2 w2 ŷ X2 w2 ŷ

wm wm
Xm
y
Xm
y
C
X1 X1
w1 w1

X2 w2 ŷ X2 w2 ŷ

w’s
w’s
w’s
w’s

w’s
w’s
Upd
Upd

Upd
Upd
Upd

Upd
Upd

A Neural Network in 13 lines

of Python (Part 2 - Gradient
Descent)

Andrew Trask (2015)

Link:

Neural Networks and Deep

Learning

Michael Nielsen (2015)

Link:

[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
FoBrw
acakrp
drP
orpoapgaagta
iotn
ion

Image Source: [Link]

Neural Networks and Deep

Learning

Michael Nielsen (2015)

Link:

[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
STEP 1: Randomly initialise the weights to small numbers close to 0 (but not 0).

STEP 2: Input the first observation of your dataset in the input layer, each feature in one input node.

STEP 3: Forward-Propagation: from left to right, the neurons are activated in a way that the impact of each
neuron’s activation is limited by the weights. Propagate the activations until getting the predicted result y.

STEP 4: Compare the predicted result to the actual result. Measure the generated error.

STEP 5: Back-Propagation: from right to left, the error is back-propagated. Update the weights according to
how much they are responsible for the error. The learning rate decides by how much we update the
weights.

STEP 6: Repeat Steps 1to 5 and update the weights after each observation (Reinforcement Learning). Or:
Repeat Steps 1to 5 but update the weights only after a batch of observations (Batch Learning).

STEP 7: Whe n the whole training set passed through the ANN, that makes an epoch. Redo more epochs.
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
What we will learn in this
section:
• What are Convolutional Neural Networks?
• Step 1- Convolution Operation
• Step 1(b) - ReLU Layer
• Step 2 - Pooling
• Step 3 - Flattening
• Step 4 - Full Connection
• Summary

• EXTRA: Softmax & Cross-Entropy

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: a talk by Geoffrey Hinton
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Source: google trends
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Yann Lecun
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Facebook
Google
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
(Image
Output

Sad
CNN

Pixel 1 Pixel 2 Pixel 1 Pixel 2

2d array

Pixel 3 Pixel 4 Pixel 3 Pixel 4

Red channel Green

Colored Image 2x2px channel

Pixel 1 Pixel 2
3d array Colored
Pixel 1 Pixel 2
Colored
Image
Image
Pixel 3 Pixel 4
Pixel 3 Pixel 4

0
0

STEP 2: Max Pooling

STEP 3: Flattening

STEP 4: Full Connection

Gradient-Based Learning
Applied to Document
Recognition

By Yann LeCun et al. (1998)

Link:

[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Additional Reading:

Introduction to
Convolutional Neural
Networks

By Jianxin Wu (2017)

0 1 0 0 0 1 0
0 0 1
0 0 0 0 0 0 0

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

0 1 0 0 0 1 0 0
0 0 1
0 0 0 0 0 0 0

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1
0 0 1
0 0 0 0 0 0 0

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0
0 0 1
0 0 0 0 0 0 0

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0
0 0 1
0 0 0 0 0 0 0

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1

0 1 0 0 0 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1

0 1 0 0 0 1 0 1
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1

0 1 0 0 0 1 0 1 4
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1

0 1 0 0 0 1 0 1 4 2
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1

0 1 0 0 0 1 0 1 4 2 1
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1

0 1 0 0 0 1 0 1 4 2 1 0
0 1 1
0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1

0 1 0 0 0 1 0 1 4 2 1 0
0 1 1
0 0 1 1 1 0 0 0
0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1
0 1 0 0 0 1 0 1 4 2 1 0
0 1 1
0 0 1 1 1 0 0 0 0
0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1
0 1 0 0 0 1 0 1 4 2 1 0
0 1 1
0 0 1 1 1 0 0 0 0 1
0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1

0 1 0 0 0 1 0 1 4 2 1 0
0 1 1
0 0 1 1 1 0 0 0 0 1 2
0 0 0 0 0 0 0

Input Feature Feature Map

0 1 0 0 0 1 0 0 1 0 0 0
0 0 1
0 0 0 0 0 0 0 0 1 1 1 0

0 0 0 1 0 0 0 1 0 0 1 0 1 2 1
0 1 0 0 0 1 0 1 4 2 1 0
0 1 1
0 0 1 1 1 0 0 0 0 1 2 1
0 0 0 0 0 0 0

Input Feature Feature Map

Image Detector
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
We create many Feature
feature maps to Maps
0 0 0 0 0 0 0 obtain our first
convolution layer
0 1 0 0 0 1 0

0 0 0 0 0 0 0

0 0 0 1 0 0 0

0 1 0 0 0 1 0

0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Image

Convolutional
Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]/en/[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]/en/[Link]
Sharpen:
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]/en/[Link]
Blur:
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Edge Enhance:

Image Source: [Link]/en/[Link]

NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]/en/[Link]
Emboss:
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
We create many Feature
feature maps to Maps
0 0 0 0 0 0 0 obtain our first
convolution layer
0 1 0 0 0 1 0

0 0 0 0 0 0 0

0 0 0 1 0 0 0

0 1 0 0 0 1 0

0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Image

Convolutional
Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Feature Maps
0 0 0 0 0 0 0
Rectifier
0 1 0 0 0 1 0 y
0 0 0 0 0 0 0
0 0 0 1 0 0 0
0 1 0 0 0 1 0
0 0 1 1 1 0 0
0 0 0 0 0 0 0 0
Input Image

Convolutional Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Additional Reading:

Understanding
Convolutional Neural
Networks with A
Mathematical Model

By C.-C. Jay Kuo (2016)

Link:

Delving Deep into Rectifiers:

Surpassing Human-Level
Performance on ImageNet
Classification

By Kaiming He et al. (2015)

Link:

[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: Wikipedia
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: Wikipedia
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
0

1
1

Feature Map
0

2
2
0

1
1
1

0
0
0

0 1 1 1 0
Max Pooling
1 0 1 2 1

1 4 2 1 0

0 0 1 2 1

0 1 1 1 0 1
Max Pooling
1 0 1 2 1

1 4 2 1 0

0 0 1 2 1

0 1 1 1 0 1 1
Max Pooling
1 0 1 2 1

1 4 2 1 0

0 0 1 2 1

0 1 1 1 0 1 1 0
Max Pooling
1 0 1 2 1

1 4 2 1 0

0 0 1 2 1

0 1 1 1 0 1 1 0
Max Pooling
1 0 1 2 1 4
1 4 2 1 0

0 0 1 2 1

0 1 1 1 0 1 1 0
Max Pooling
1 0 1 2 1 4 2
1 4 2 1 0

0 0 1 2 1

0 1 1 1 0 1 1 0
Max Pooling
1 0 1 2 1 4 2 1
1 4 2 1 0

0 0 1 2 1

0 1 1 1 0 1 1 0
Max Pooling
1 0 1 2 1 4 2 1
1 4 2 1 0 0
0 0 1 2 1

0 1 1 1 0 1 1 0
Max Pooling
1 0 1 2 1 4 2 1
1 4 2 1 0 0 2
0 0 1 2 1

0 1 1 1 0 1 1 0
Max Pooling
1 0 1 2 1 4 2 1
1 4 2 1 0 0 2 1
0 0 1 2 1

Evaluation of Pooling
Operations in Convolutional
Architectures for Object
Recognition

By Dominik Scherer et al. (2010)

Link:

0 1 0 0 0 1 0

0 0 0 0 0 0 0

0 0 0 1 0 0 0

0 1 0 0 0 1 0 Convolution Pooling

0 0 1 1 1 0 0

0 0 0 0 0 0 0

Input Image
Convolutional Pooling Layer
Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: [Link]/~aharley/vis/conv/[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Pooled Feature
0
1
1
1
2
2
1
4
0

Pooled Feature
0
1
1
1
2
2
1
4
0

Pooling Layer
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
0 0 0 0 0 0 0
0 1 0 0 0 1 0
0 0 0 0 0 0 0
0 0 0 1 0 0 0 Convolution Pooling Flattening
0 1 0 0 0 1 0
0 0 1 1 1 0 0
0 0 0 0 0 0 0

Input Image
Input
layer of
Convolutional Pooling Layer a future
Layer ANN
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
X1

Flattening
X2 Output
value

Input Layer Fully Connected Layer Output Layer

Cat
0.9

0.2

0.2
0.1

0.1
0.1
1

Cat
0.9

0.2

0.2
0.1

0.1
0.1
1

Cat
0.1

0.1

0.2
0.2

0.2
0.9

0.9

Cat
0.1

0.1

0.2
0.2

0.2
0.9

0.9

Cat

0.1
0.8

0.8
0.4

0.2
0.1
1

Cat
0.1

0.8

0.4

0.2

0.1
0.9

1
1
Flattening
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Image Source: a talk by Geoffrey Hinton
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Additional Reading:

The 9 Deep Learning Papers

You Need To Know About
(Understanding CNNs Part 3)

Adit Deshpande (2016)

Link:
[Link]
[Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
©SuperDataScience
Machine Learning A-Z
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
0.05
0.95
Dog

z2
z1
Dog

Cat
Flattenin
g
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
1
0
Dog 0.9

0.9
0.7
0.3

0.1
0.4
0.1

0.1
0.9

0.9

0.6
1
1

0
Dog

Dog

Dog
Cat

Cat

Cat
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Row Dog Cat^ Dog Cat Row Dog Cat^ Dog Cat
^ ^
#1 0.9 0.1 1 0 #1 0.6 0.4 1 0
#2 0.1 0.9 0 1 #2 0.3 0.7 0 1
sification E rror
Cl
as
#3 0.1 1/0
3.9
=0.33
1 0
#3 0.4 1 0 Mean Squared Error
1/0
3.6
=0.33
0.25 0.71
Cross-Entropy
0.38 1.06
NOT FOR DISTRIBUTION © SUPERDATASCIENCE [Link]
Additional Reading:

A Friendly Introduction to
Cross-Entropy Loss

By Rob DiPietro (2016)

How to implement a neural

network Intermezzo 2

By Peter Roelants (2016)

Link:
[Link]
_intermezzo02/

PC - Collections Sous Python
No ratings yet
PC - Collections Sous Python
25 pages
Python Lists, Tuples, and Dictionaries Guide
No ratings yet
Python Lists, Tuples, and Dictionaries Guide
9 pages
Python Lists and Tuples Guide
No ratings yet
Python Lists and Tuples Guide
7 pages
Python Data Structures: Lists, Tuples, Dictionaries
No ratings yet
Python Data Structures: Lists, Tuples, Dictionaries
21 pages
Python Lists, Tuples, and Sets Guide
No ratings yet
Python Lists, Tuples, and Sets Guide
13 pages
Python Basics: Data Types & Semantics
No ratings yet
Python Basics: Data Types & Semantics
14 pages
Understanding Python Lists and Operations
No ratings yet
Understanding Python Lists and Operations
15 pages
Python Lists, Tuples, and Dictionaries Guide
No ratings yet
Python Lists, Tuples, and Dictionaries Guide
10 pages
Unit - 2 Pythonn
No ratings yet
Unit - 2 Pythonn
106 pages
Unit - 2 Pythonn 2
No ratings yet
Unit - 2 Pythonn 2
164 pages
Python - 2
No ratings yet
Python - 2
12 pages
Python Lists, Dictionaries, Tuples, Sets Guide
No ratings yet
Python Lists, Dictionaries, Tuples, Sets Guide
27 pages
Python Basics: Day 3 Overview
No ratings yet
Python Basics: Day 3 Overview
25 pages
Aide-Mémoire 3 Python
No ratings yet
Aide-Mémoire 3 Python
7 pages
Python Lists and Tuples Overview
100% (1)
Python Lists and Tuples Overview
9 pages
Python List and Tuple Data Structures
No ratings yet
Python List and Tuple Data Structures
352 pages
Python Lists and Tuples Explained
No ratings yet
Python Lists and Tuples Explained
24 pages
Python Lists and Tuples Explained
No ratings yet
Python Lists and Tuples Explained
12 pages
Python List Creation and Operations
No ratings yet
Python List Creation and Operations
46 pages
Python Collections Cheat Sheet
No ratings yet
Python Collections Cheat Sheet
3 pages
Unit 2 List Tuple
No ratings yet
Unit 2 List Tuple
21 pages
Python Program: Count Repeated Characters
No ratings yet
Python Program: Count Repeated Characters
25 pages
Python Data Types and Functions Guide
No ratings yet
Python Data Types and Functions Guide
9 pages
Data Structures in Python: Lists, Tuples, Sets
No ratings yet
Data Structures in Python: Lists, Tuples, Sets
19 pages
Python Lists and Tuples Explained
No ratings yet
Python Lists and Tuples Explained
21 pages
List Manipulation
No ratings yet
List Manipulation
26 pages
Python Data Types Overview
No ratings yet
Python Data Types Overview
30 pages
Python Data Structures: Lists, Tuples, Sets
No ratings yet
Python Data Structures: Lists, Tuples, Sets
19 pages
Python Unit 4
No ratings yet
Python Unit 4
33 pages
List Manipulation
No ratings yet
List Manipulation
46 pages
Unit III
No ratings yet
Unit III
50 pages
Python List Data Type Overview
No ratings yet
Python List Data Type Overview
126 pages
Python Sequence Data Structures Explained
No ratings yet
Python Sequence Data Structures Explained
83 pages
Python Data Structures: Lists & Dictionaries
No ratings yet
Python Data Structures: Lists & Dictionaries
99 pages
Python Mutable and Immutable Data Types
No ratings yet
Python Mutable and Immutable Data Types
21 pages
Python List Creation and Operations
No ratings yet
Python List Creation and Operations
50 pages
Python List Operations and Functions
No ratings yet
Python List Operations and Functions
49 pages
Python Functions and Lists Overview
No ratings yet
Python Functions and Lists Overview
47 pages
Python Lists and Tuples Guide
No ratings yet
Python Lists and Tuples Guide
16 pages
Python List Basics and Methods
No ratings yet
Python List Basics and Methods
20 pages
Python Lists: Operations and Methods
No ratings yet
Python Lists: Operations and Methods
17 pages
Unit 3 Notes
No ratings yet
Unit 3 Notes
13 pages
Review Python 2
No ratings yet
Review Python 2
17 pages
PWP Unit Iii
No ratings yet
PWP Unit Iii
18 pages
Python Lists Overview and Operations
No ratings yet
Python Lists Overview and Operations
64 pages
Python List Manipulation Techniques
100% (1)
Python List Manipulation Techniques
42 pages
Appending to Tuples in Python
No ratings yet
Appending to Tuples in Python
9 pages
Python Data Structures Overview
No ratings yet
Python Data Structures Overview
33 pages
Python List Operations and Methods
No ratings yet
Python List Operations and Methods
46 pages
Lists
No ratings yet
Lists
48 pages
Lists: Course: Introduction To Programming (BT101CO)
No ratings yet
Lists: Course: Introduction To Programming (BT101CO)
48 pages
Python Data Structures: Lists, Tuples, Sets
No ratings yet
Python Data Structures: Lists, Tuples, Sets
7 pages
UNIT IV Python Lists Tuples Dictionaries
No ratings yet
UNIT IV Python Lists Tuples Dictionaries
16 pages
Python List and Collection Types Guide
No ratings yet
Python List and Collection Types Guide
3 pages
CS (12th) Mindmaps
100% (2)
CS (12th) Mindmaps
14 pages
CM8 Variables-References
No ratings yet
CM8 Variables-References
27 pages
013 LISTS-new
No ratings yet
013 LISTS-new
52 pages
Samsung Electrical Parts List
No ratings yet
Samsung Electrical Parts List
10 pages
Vector Mathematics Overview by Ruth Gregory
No ratings yet
Vector Mathematics Overview by Ruth Gregory
8 pages
Ironmongery Direct Catalogue 2nd Edition
No ratings yet
Ironmongery Direct Catalogue 2nd Edition
28 pages
Extension Lecture On AI and New
No ratings yet
Extension Lecture On AI and New
1 page
Two-Body Problem Trajectory Equation
No ratings yet
Two-Body Problem Trajectory Equation
19 pages
Research Approval for Truancy Study
No ratings yet
Research Approval for Truancy Study
2 pages
Guidelines for Biodiversity Management Committees
No ratings yet
Guidelines for Biodiversity Management Committees
23 pages
Siemens Motors
100% (1)
Siemens Motors
105 pages
Signal Processing and Detection Methods
No ratings yet
Signal Processing and Detection Methods
6 pages
Design Data Handbook by K. Mahadevan
No ratings yet
Design Data Handbook by K. Mahadevan
2 pages
January 2023 Newsletter for Grade VIII
No ratings yet
January 2023 Newsletter for Grade VIII
6 pages
CSF-RS2 Flangeless (2 & 3 Slot)
No ratings yet
CSF-RS2 Flangeless (2 & 3 Slot)
1 page
Field Theory: Path Integral Approach
No ratings yet
Field Theory: Path Integral Approach
10 pages
Say-Mean-Matter Rubric Guide
No ratings yet
Say-Mean-Matter Rubric Guide
1 page
Essential MCAT Prep Tips and Resources
No ratings yet
Essential MCAT Prep Tips and Resources
13 pages
TESCAN VEGA For MS - Product Flyer
No ratings yet
TESCAN VEGA For MS - Product Flyer
1 page
Applying Scrum Roles To A College Cultural Festival
No ratings yet
Applying Scrum Roles To A College Cultural Festival
1 page
GEC Avery L226 Approval Cancellation
No ratings yet
GEC Avery L226 Approval Cancellation
20 pages
Refraction and Lens Properties Assignment
No ratings yet
Refraction and Lens Properties Assignment
4 pages
Dimensional Analysis & Sig Figs Worksheet
No ratings yet
Dimensional Analysis & Sig Figs Worksheet
3 pages
Leaf Pigment Analysis via Chromatography
No ratings yet
Leaf Pigment Analysis via Chromatography
3 pages
Unique Baby Name Combiner Tool
No ratings yet
Unique Baby Name Combiner Tool
3 pages
Environmental Science Exam Paper 2024
No ratings yet
Environmental Science Exam Paper 2024
5 pages
Exact Solutions of Schrödinger Equation
No ratings yet
Exact Solutions of Schrödinger Equation
13 pages
Wave LR Radar Level Transmitter Datasheet
100% (1)
Wave LR Radar Level Transmitter Datasheet
1 page
Advancing Physical Scene Understanding
No ratings yet
Advancing Physical Scene Understanding
325 pages
Principles of Solitary Vampirism Guide
No ratings yet
Principles of Solitary Vampirism Guide
74 pages
In Vitro Methods in Toxicology Vol. 1
No ratings yet
In Vitro Methods in Toxicology Vol. 1
16 pages
Understanding Triangle Similarity
No ratings yet
Understanding Triangle Similarity
6 pages
L2 Learning Motivation: Self-Determination Theory
No ratings yet
L2 Learning Motivation: Self-Determination Theory
29 pages

PC - Collections Sous Python

Uploaded by

PC - Collections Sous Python

Uploaded by

Python Programmation language

Object collections in Python

TUPLES, LISTS, DICTIONARIES

Type ‘tuple’ cf. call type() function

1. Crée un tuple couleurs = ('rouge', 'vert', 'bleu’).

List ≈ tuple of dynamic and modifiable size

 #définition d'une liste The[ ] are important indicates that it is a

 #taille de la tuple [2,6,3,10,15,26]

The other mechanisms associated

Example 1 : Square all numbers

Exemple 2 : conditionnal task

#counting Return 2 since the value 69 appears

#remove by value Remove the value 69 from the list,

This mechanism works with any type of object as long as a

#copie des valeurs

print(L3) → [61,92,17] !!! L3 is not impacted.

Allow to define and create

1. Crée une liste nombres = [2,4,6,8,10] et affiche le troisième élément.

2. Ajoute le nombre 12 à la liste et supprime 4.

[Link] la liste et affiche chaque élément au carré.

[Link] le plus grand et le plus petit nombre dans la liste.

[Link] 5 nombres à l’utilisateur et stocke-les dans une liste, puis affiche la

6.Écris un programme qui inverse une liste.

7.Écris un programme qui fusionne deux listes [1,2,3] et [4,5,6].

A string can be transformed into a list to carry out sophisticated

[‘B’,’O’,’N’,’J’,’O’,’U’,’R’,’ ‘,’L’,’E’,’ ‘,’M’,’O’,’N’,’D’,’E’]

#former une chaîne à

#liste des clés

#liste des valeurs [15, 16, 17]

#accès à une valeur par clé Notes :

#ajout d'un bloc d’éléments

#détecter présence clé

#suppression par clé

Keys are not necessarily strings. The tool is very flexible

#autre type de clé

1. Crée un dictionnaire etudiant = {'nom': 'Sara', 'age': 21, 'note': 16}

1. Affiche la valeur associée à la clé 'nom’.

2. Ajoute une nouvelle clé 'ville' avec une valeur.

3. Modifie la note de l’étudiant.

Parcourt le dictionnaire et affiche clé et valeur.

1. Crée une liste de dictionnaires pour représenter une classe de 3 étudiants.

[0 ;1] [-3 ;+3]

𝑏0, 𝑏1 such that:

y-intercept Slope coefficient 1 Slope coefficient 2 Slope coefficient n

𝑆𝑆𝑟𝑒𝑠 = 𝑆𝑈𝑀(𝑦𝑖 − 𝑦! 𝑖 ) 2 𝑆𝑆𝑡𝑜𝑡 = 𝑆𝑈𝑀(𝑦𝑖 − 𝑦𝑎𝑣g)2

*This is highly dependent on the context

Anscombe's quartet (1973):

[Link] 2. Homoscedasticity 3. Multivariate Normality

4. Independence 5. Lack of Multicollinearity 6. The Outlier Check

192,261.83 165,349.20 136,897.80 471,784.10 New York

y =b 0 +b1*x1 +b2*x2+b3*x3 + ???

Machine Learning A-Z

Profit R&D Spend Ad min Marketing State New York California

192,261.83 165,349.20 136,897.80 471,784.10 New York 1 0

y =b 0 +b1*x1 +b2*x2+b3*x3 +b4*D1

Machine Learning A-Z

Profit R&D Spend Ad min Marketing State New York California

192,261.83 165,349.20 136,897.80 471,784.10 New York 1 0

y =b 0 +b1*x1 +b2*x2+b3*x3 +b4*D1

Machine Learning A-Z

Profit R&D Spend Admin Marketing State New York California

192,261.83 165,349.20 136,897.80 471,784.10 New York 1 0

y =b 0 +b1*x1 +b2*x2+b3*x3 +b4*D1 +b5*D2

Machine Learning A-Z

Profit R&D Spend Ad min Marketing State New York California

192,261.83 165,349.20 136,897.80 471,784.10 New York 1 0

y =b 0 +b1*x1 +b2*x2+b3*x3 +b4*D1 +b5*D2

Machine Learning A-Z

Machine Learning A-Z

Machine Learning A-Z

STEP 2: Fit the full model with all possible predictors

STEP 4: Remove the predictor

STEP 5: Fit model without this variable*

Machine Learning A-Z

FIN: Keep the previous model

Machine Learning A-Z

y =b 0 +b1x1 +b2x2+b3*x3 + ???

y =b 0 +b1x1 +b2x2+b3x3 +b4D1

y =b 0 +b1x1 +b2x2+b3x3 +b4D1

y =b 0 +b1x1 +b2x2+b3x3 +b4D1 +b5*D2

y =b 0 +b1x1 +b2x2+b3x3 +b4D1 +b5*D2