Roadmap

Episode : S01E01
1.1 Introduction
Context, tools and ressources
1
1.2 From the linear regression
to the first neuron
History,
Fundamental
Concepts
1.3 Neurons in controversy

1.4 Data and neurons

Basic Regression
Basic Basic Basic Classification
Regression Classification
DNN DNN
[ Méthode scientifique ]

15
90
[ Méthode scientifique ]

16
1
Jim Gray, 2007 [GRAY] 90
[ *-learning ]

Artificial
Intelligence (AI)

17
90
[ *-learning ]

Artificial
Intelligence (AI) Machine
Learning (ML)

18
90
[ *-learning ]

Deep
Learning (DL)
Based on artificial neural
networks

Artificial
Intelligence (AI) Machine
Learning (ML)

19
90
[ *-learning ]

YOU
ARE
HERE

Deep
Learning (DL)
Based on artificial neural
networks

Artificial
Intelligence (AI) Machine
Learning (ML)

20
90
[ *-learning ]

21
90
Supervised learning

Learning
from examples

22
90
Supervised learning Learning from examples

Classification8:
Predict qualitative informations

This is a cat

Tell me,
what is it ?
This is a rabbit

23
90
Supervised learning Learning from examples

Classification8:
Predict qualitative informations

This is a cat

Tell me,
what is it ?
This is a rabbit

Régression :
Predict quantitative informations

150 KL 400 KL
Tell me,
what's the
price ?
120 KL 100 KL

24
90
Unsupervised learning

Learning
from data
alone

25
90
Unsupervised learning Learning from data alone

Clustering :
Finding Common Relationships

What is the
relationship
between
these data ?

26
90
Unsupervised learning Learning from data alone

Clustering :
Finding Common Relationships

What is the
relationship
between
these data ?

Reduction :
Reduce the number of dimensions

Simplify
while
keeping
meaning

27
90
[ *-learning ]

28
90
A strong competition
AI Publications (% of All Publi-
cations) 30000
AI Papers By Region
3,00 %
25000
2,50 %
China
20000
2,00 % Europe
15000
1,50 % United States
1,00 % 10000

0,50 % 5000

0,00 % 0
9 8 00 0 2 0 4 0 6 0 8 1 0 1 2 14 1 6 1 8
19 0 20 20 20 20 20 20 0 20 20

98

00

02

04

06

08

18
10

12

14

16
2 2

19

20

20

20

20

20

20

20

20

20

20
DL Papers per million people FWCI impact by region
40 3,5
35 3
30
2,5
25
20 2
15 1,5
10
1
5
0 0,5
0
Switz ore

e
erland

d
Germ n

Spain

Japan

Iran
alia

ia

China

India
UK
USA
Israel

cco

Brazil
CZE
m
a

any
ark
Italy

y
s
rland
Canad

Franc

Greec

Turke
Russi
e

Polan
Austr

Belgiu
Swed

Denm
p

8 0 2 4 6 8 0 2 4 6 8
Moro
Aus tr

9 0 0 0 0 0 1 1 1 1 1
Singa

9 0 0 0 0 0 20 0 0 0 0
Nethe

1 2 2 2 2 2 2 2 2 2
29
Source : AI Index [AIDEX] FWCI : Field-Weighted Citation Impact 90
A Python centered world

DL Framework Power Scores 2018

100
Most used DL framework
90 Medium Articles Supported by Google
Amazon Books Low level API – an hard way
80 Apache licence
70 ArXiv Articles
Search Volume
60
GitHub Activity
50 Usage Survey
40 Job Search Listings By François Cholet (Google)
Hight level API
30 Part on TensorFlow since 2017
MIT licence
20
10
0
re as ro ch e o T K J 2 r I
rF low K T Caff ean
XN E
CN
T
ing4
aff e
aine
as tA From Torch library
s o Py Th M ra n C Ch F Supported by Facebook
n e
Te epL BSD licence
De
30
Source : Je< Hale, « Deep Learning Framework Power Scores 2018 » [DLPW] 90
A Python centered world

Six-Month Growth Scores 2019

100
90 Quora Followers
Medium Articles
80 ArXiv Articles
70 GitHub Activity
60 Google Interest
Job Search Listings
50
40
30
20
10
0
w ro ch r as AI
rF lo
y T Ke F ast
sn o P
Te
31
Source : Je< Hale, « Deep Learning Framework Growth Scores 2019 » [DLPW] 90
Fine, but
Deep Learning
What's that?
39
90
- Quelques Rappels de Mathématiques pour l’IA -
Tableau interactif
Roadmap
Episode : S01E01
1.1 Introduction
Context, tools and ressources
1
1.2 From the linear regression
to the first neuron
History,
Fundamental
Concepts
1.3 Neurons in controversy

1.4 Data and neurons

Basic Regression
Basic Basic Basic Classification
Regression Classification
DNN DNN
Linear regression

We have a phenomenon, for which we have observations

42
90
Linear regression

We are looking for a straight line that passes

« as close as possible » to our points.

43
90
Linear regression

"As close as possible" means "minimize the distance

between the line and our points (observations)".

44
90
Linear regression

For this, we will use an «loss function », which we will try

to minimize.

RMSE : Root Mean Square Error

Erreur quadratique moyenne
MSE : Mean Squared Error
Moyenne du carré des erreurs

45
90
Linear regression

Good news !
We have a direct solution !

Bad news...
Complexity in n3

46
90
Gradient descent

⌅1

Loss
We will iteratively look for the
loss(⌅) best position of our line, by
⌅1 ⌅
varying its parameters (W).

⌅2
⌅1 But how can we eXciently
Loss
vary our parameters (W)?
loss(⌅)

⌅2 ⌅

⌅3
⌅2 Note : Loss functions could be :
⌅1
Loss

loss(⌅)
47
⌅3 ⌅ 90
Gradient descent

By changing W from YW
We improve loss(W) of Yloss
The gradient is the slope we will follow
to minimize our loss function.
↵ loss
Iterations

↵⌅
Loss

loss(⌅)
This process is called gradient descent and
the function used to optimize the descent,
optimization function
Best ⌅ ⌅
48
90
Gradient descent

#i Loss Gradient Theta

0 +12.481 -6.777 -1.732 -3.388 +0.000
20 +4.653 -4.066 -1.039 -2.033 +0.346
40 +1.835 -2.440 -0.624 -1.220 +0.554
60 +0.821 -1.464 -0.374 -0.732 +0.679
80 +0.455 -0.878 -0.224 -0.439 +0.754
100 +0.324 -0.527 -0.135 -0.263 +0.799
120 +0.277 -0.316 -0.081 -0.158 +0.826
140 +0.260 -0.190 -0.048 -0.095 +0.842
160 +0.253 -0.114 -0.029 -0.057 +0.851
180 +0.251 -0.068 -0.017 -0.034 +0.857
200 +0.250 -0.041 -0.010 -0.020 +0.861

49
Notebook [GRAD1] 90
Polynomial regression

51
Polynomial regression

Underfiting Overfiting
Logistic regression
A logistic regression is intended to provide a probability of belonging to a class.

Dataset : X Observations
y Classe

53
Logistic regression
A logistic regression is intended to provide a probability of belonging to a class.
Dataset : X Observations Objective : Predict the class
y Classe x given, we want to predict y
ypred = f(x)
where f is a linear function

?
- Introduction au Langage Python pour l’IA -
Démonstrations en interactif
Logistic regression

55
90
Logistic regression
Determined by the minimisation
of a cost function J(W)

56
90
Logistic regression

x1
w1 b
x2 w2
bia
w3
x3
wm

xm

That’s an « artificial neuron » !

So, we have a neural network of… 1 neuron !
57
90
Roadmap
Episode : S01E01
1.1 Introduction
Context, tools and ressources
1
1.2 From the linear regression
to the first neuron
History,
Fundamental
Concepts
1.3 Neurons in controversy

1.4 Data and neurons

Basic Regression
Basic Basic Basic Classification
Regression Classification
DNN DNN
[ intelligence ]

62
90
 [ intelligence ]
« Capacité de percevoir ou d'inférer
l'information, et de la conserver comme une
connaissance à appliquer à des
comportements adaptatifs dans un
environnement ou un contexte donné »

«↵Ability to perceive or infer information, and to retain it

as knowledge to be appli towards adaptive behaviors
within an environment or context↵»*

63
*
Wikipedia - Illustration : [POTATO] 90
 a [ intelligence ]
4
« Ensemble des fonctions mentales ayant
pour objet la connaissance conceptuelle et
rationnelle »*

«↵Set of mental functions aimed at conceptual and

rational knowledge↵»

64
*
[Link] 90
 ro ig
rsy
nt e b
ve
Co Th

Modelling the brain↵: Making a mind↵:

« Penser s’apparente « Penser, c’est calculer des symboles qui
à un calcul massivement parallèle de ont à la fois une réalité matérielle et une
fonctions élémentaires. valeur sémantique de représentation »1
L’information est un signal avant L’information est une donnée
d’être un code »1 symbolique de haut niveau.

Connectionnism vs Symbolic
Modelling the brain Making a mind
Modéliser le cerveau Forger une opinion

Tout [homme] est [mortel]

[Socrate] est un [homme]
Donc [Socrate] est [mortel]

65
1
D Cardon, JP Cointet, A Mazieres, 2018 [LRDN] 90
 66
Illustration : [NEURON] 90
67
90
Determined by the minimisation
of a cost function

68
90
 ro ig
rsy
nt e b
ve
Co Th

Modelling the brain↵: Making a mind↵:

« Penser s’apparente « Penser, c’est calculer des symboles qui
à un calcul massivement parallèle de ont à la fois une réalité matérielle et une
fonctions élémentaires. valeur sémantique de représentation »1
L’information est un signal avant L’information est une donnée
d’être un code »1 symbolique de haut niveau.

Connectionnism vs Symbolic
Modelling the brain Making a mind
Modéliser le cerveau Forger une opinion

Tout [homme] est [mortel]

[Socrate] est un [homme]
Donc [Socrate] est [mortel]

69
1
D Cardon, JP Cointet, A Mazieres, 2018 [LRDN] 90
 ro ig
rsy
nt e b
ve
Co Th

Connectionnism vs Symbolic

Facts Rules and laws Expert

Model Rules and laws Special case

70
90
 ro ig
rsy
nt e b
ve
Co Th

Evolution of the academic influence of connexionist and symbolic approaches 1

Ration of publications between connexionists and symbolists

65 522 publications
106 278 publications

71
1
D Cardon, JP Cointet, A Mazieres, 2018 [LRDN] 90
Perceptron

Perceptron
Frank Rosenblatt
1958

Linear and binary classifier

73
F. Rosenblatt, 1958 [FROS] 90
Perceptron

Iris plants dataset

Dataset from : Fisher, R.A. “The use of multiple measurements in
taxonomic problems” Annual Eugenics, 7, Part II, 179-188
(1936)
Length Width Iris Setosa (0/1)
x1 x2 y
1.4 1.4 1
1.6 1.6 1
1.4 1.4 1
1.5 1.5 1
1.4 1.4 1
4.7 4.7 0
4.5 4.5 0
4.9 4.9 0
4.0 4.0 0
4.6 4.6 0
(...)

74
Notebook [PER57] 90
- Regression logistic en Python -
Démonstration
Deep Neural Networks

77
90
Deep Neural Networks

Input layer Hidden layers Output layer

78
90
Deep Neural Networks

No activation
function

X(i)

y(i)
Train set

X(i) : Observations
y(i) : Expected output

(Normalized)
79
90
Deep Neural Networks

To define :
Optimization
Activation
Back-propagation Loss
Metrics
Learning process ... 80
90
Training process - general

81
90
Training process - general

(batch)

82
90
Training process - general

(batch)

83
90
Deep Neural Networks

1958

84
Roadmap
Episode : S01E01
1.1 Introduction
Context, tools and ressources
1
1.2 From the linear regression
to the first neuron
History,
Fundamental
Concepts
1.3 Neurons in controversy

1.4 Data and neurons

Basic Regression
Basic Basic Basic Classification
Regression Classification
DNN DNN
Regression with a DNN

No activation
function

X(i)

y(i)
Train set

X(i) : Observations
y(i) : Expected output

(Normalized)
93
90
- Régression à l’aide d’un Réseaux de neurones (RNN) en Python -
Démonstration
Regression with a Dense
Network (DNN)
Notebook : [BHP1]

Objective :
Predicts housing prices from a set of house features.
Dataset8:
Boston House Pricing Dataset (BHPD)

96
90
 Step 1 - Import and init Step 4 - Build a model

Step 2 - Retrieve data Step 5 - Train the model

n epoch

Step 3 - Preparing the data Step 6 - Evaluate

Xtest
Regression ytest
with a (DNN) Test set

Notebook : [BHP1] Xtrain

ytrain Epoch
Train set

97
90
Classification with a DNN

(Normalized)

99
90
- Classi cation à l’aide d’un réseau de neurones (RNN) en
python- Démonstration
fi
Simple classification with DNN
Notebook : [MNIST1]

Objective :
Recognizing handwritten numbers
Dataset8:
Modified National Institute of Standards and
Technology (MNIST)

100
90