Introduction to Machine Learning
in Python (Day 1)
Dr. Behnam Yousefi
Institute of medical systems biology, UKE
April 2024
Artificial Intelligence
Machine Learning
Deep Learning
Artificial Intelligence (AI)
• The imitation of human intelligence in machines.
Artificial Intelligence (AI)
• IBM’s Deep Blue computer defeated Garry Kasparov in 1997.
• AI was composed of a rule based decision making and not ML.
Machine Learning (ML)
• ML allows systems to automatically learn and improve from
experience without being explicitly programmed.
AI ML
Artificial Neural Network (ANN)
• Can learn almost any function given enough number of samples.
AI ML ANN
Deep Learning (DL)
• Deep and more complex ANNs.
• Ability of feature learning.
AI ML ANN DL
Machine Learning Methods
Features
Samples
Feature extraction
Time
Study subject Feature matrix ML model
Machine Learning Methods
Features
Have the ground truth
Supervised ML
Samples
Not have the ground truth
Unsupervised ML
Feature matrix ML model
Machine Learning Methods
Have the ground truth Classification Naïve Bayes
Supervised ML
Regression Linear regression
Not have the ground truth
Ex.
Unsupervised ML Clustering K-means
Naïve Bayes Classification
Linear regression
Maximum Likelihood (ML)
𝑝 𝑥 𝑐 = 0 𝑉𝑆 𝑝(𝑥|𝑥 = 1)
𝑝(𝑥|𝑐)
Class 0 Class 1
Maximum a posteriori (MAP)
𝑝 𝑥 𝑐 = 0 .𝑃 𝑐 = 0
𝑉𝑆
𝑝 𝑥 𝑥 = 1 . 𝑃(𝑐 = 1)
𝑥
Naïve Bayes Classification
Linear regression
𝑤! 𝑥!∗ + 𝑤" 𝑥"∗ + 𝑏 = 0
𝑧 = 𝑤! 𝑥! + 𝑤" 𝑥" + 𝑏
𝑥"
𝑧>0
𝑧<0
𝑥!
Validation Metrics
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑙𝑦 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑒𝑑 𝑠𝑎𝑚𝑝𝑙𝑒𝑠
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑎𝑚𝑝𝑙𝑒𝑠
Validation Metrics
Confusion table
Actual Actual
+ -
Pred. + TP FP
Pred. - FN TN
Validation Metrics
𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
𝑎𝑙𝑙
Confusion table
Actual Actual
+ -
Pred. + TP FP
Pred. - FN TN
Validation Metrics
𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
𝑎𝑙𝑙
Confusion table
𝑇𝑃
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = 𝑟𝑒𝑐𝑎𝑙𝑙 =
Actual Actual 𝑇𝑃 + 𝐹𝑁
+ -
Pred. + TP FP
Pred. - FN TN
Validation Metrics
𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
𝑎𝑙𝑙
Confusion table
𝑇𝑃
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = 𝑟𝑒𝑐𝑎𝑙𝑙 =
Actual Actual 𝑇𝑃 + 𝐹𝑁
+ - 𝑇𝑁
𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 =
𝑇𝑁 + 𝐹𝑃
Pred. + TP FP
Pred. - FN TN
Validation Metrics
𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
𝑎𝑙𝑙
Confusion table
𝑇𝑃
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = 𝑟𝑒𝑐𝑎𝑙𝑙 =
Actual Actual 𝑇𝑃 + 𝐹𝑁
+ - 𝑇𝑁
𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 =
𝑇𝑁 + 𝐹𝑃
Pred. + TP FP
𝑇𝑃
Pred. - FN TN 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
𝑇𝑃 + 𝐹𝑃
Validation Metrics
𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
𝑎𝑙𝑙
Confusion table
𝑇𝑃
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = 𝑟𝑒𝑐𝑎𝑙𝑙 =
Actual Actual 𝑇𝑃 + 𝐹𝑁
+ - 𝑇𝑁
𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 =
𝑇𝑁 + 𝐹𝑃
Pred. + TP FP
𝑇𝑃
Pred. - FN TN 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
𝑇𝑃 + 𝐹𝑃
ROC
Linearcurve
regression
𝑝(𝑥|𝑐)
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
Class - Class +
𝑥 1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
𝑝(𝑥|𝑐)
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
Class - Class +
𝑥 1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
𝑝(𝑥|𝑐)
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
Class - Class +
𝑥 1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
𝑝(𝑥|𝑐)
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
Class - Class +
𝑥 1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
𝑝(𝑥|𝑐)
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
Class - Class +
𝑥 1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
𝑝(𝑥|𝑐)
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
Class - Class +
𝑥 1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
𝑝(𝑥|𝑐)
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
Class - Class +
𝑥 1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
receiver operating characteristic (ROC) curve
𝑝(𝑥|𝑐)
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
Class - Class +
𝑥 1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
receiver operating characteristic (ROC) curve
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
ROC
Linearcurve
regression
Area under ROC curve
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
1 - 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦
Cross-validation
Linear regression
Train Test
Model Model
outcome outcome
Model fitting Model inference
Cross-validation
Linear regression (CV) Train Test
Repeated random sub-sampling CV
K-fold CV
Leave-one-out CV
…
Linear Regression
𝑦?$ = 𝑤. 𝑥$ + 𝑏
𝑦 𝑒$ = 𝑦$ − 𝑦?$
𝑒$" = 𝑦$ − 𝑦?$ "
&
"
𝑙𝑜𝑠𝑠 = B 𝑦$ − 𝑦?$
𝑥 $%!
Linear Regression
𝑦?$ = 𝑤. 𝑥$ + 𝑏
𝑦
1
𝑥
Logistic Regression
𝑦?$ = 𝑓(𝑤. 𝑥$ + 𝑏)
𝑦
1
1
𝑓 𝑧 =
1 + 𝑒 ()
𝑥
Logistic Regression
Model: 𝑦?$ = 𝑓(𝑤. 𝑥$ + 𝑏)
𝑙𝑜𝑠𝑠 = 𝐶𝑟𝑜𝑠𝑠𝐸𝑛𝑡𝑟𝑜𝑝𝑦(𝑦, 𝑦)
?
&
𝑙𝑜𝑠𝑠 = − B 𝑦 · log 𝑦? + (1 − 𝑦) · log 1 − 𝑦?
$%!
Logistic Regression in 2D
𝑥"
𝑥"
𝑥! 𝑥!
Linear Classification
Linear regression
𝑤! 𝑥!∗ + 𝑤" 𝑥"∗ + 𝑏 = 0
𝑧 = 𝑤! 𝑥! + 𝑤" 𝑥" + 𝑏
𝑥" 𝑧>0
𝑧<0
𝑦? = 𝑓(𝑤! 𝑥! + 𝑤" 𝑥" + 𝑏)
𝑥!
1 𝑧>0
𝑓 𝑧 =B
−1 𝑧 < 0
Linear Classification
Linear regression
𝑤! 𝑥!∗ + 𝑤" 𝑥"∗ + 𝑏 = 0
𝑧 = 𝑤! 𝑥! + 𝑤" 𝑥" + 𝑏
𝑥"
𝑦? = 𝑓(𝑤! 𝑥! + 𝑤" 𝑥" + 𝑏)
1 𝑧>0
𝑓 𝑧 =B
−1 𝑧 < 0
𝑥!
1
𝑓 𝑧 =
1 + 𝑒 !"
Thank you for your attention!