AE2444-II Artificial Intelligence for Aerospace Engineering
Introduction to Artificial Intelligence
Dimitrios Zarouchas, Associate Professor
Image by Adobe1AI
What is Artificial Intelligence?
2
What is Artificial Intelligence?
3
From Artificial Intelligence to
Augmented Intelligence
4
From Artificial Intelligence to
Augmented Intelligence
[Link]
‘Essentially what we did is we took a drone and we made it smart…’
5
Thinking Thinking
Humanly Rationally
Artificial Intelligence
Acting Acting
Humanly Rationally
Intelligent machines that perceive their
environment, act and achieve their mission
AI was founded in 1956 at Dartmouth College, NH, USA by John McCarthy.
6
Artificial Intelligence
Machine Learning
Machine Learning is the process of training
such an intelligent machine using algorithms
that learn from data
Machine learning uses mathematical models and with the availability of data makes predictions
without explicitly programmed to do so.
7
Artificial Intelligence
Machine Learning
Deep Learning
“Deep Learning employs a technique called a
artificial neural networks to perform machine
learning tasks”
The structure of neural networks is inspired by how the brain and its neurons works. More on this
topic in Lecture 3 and 4.
8
Artificial Intelligence Winter
9
Artificial Intelligence Winter
Data
Algorithms
Computational
power
Credit: depositphotos
10
Artificial Intelligence is a melting point
• Bayes rules from probability (Bayes, 1763)
• Least square regression from astronomy (Gauss, 1795)
• Maximum likelihood from statistics (Fisher 1922)
• Artificial Neural Network from neuroscience (McCulloch 1943)
• Stochastic gradient descent from optimization (Robbins 1951)
11
COURSE INFORMATION
12
The Team
Dr. Nick Eleftheroglou Dr. Dimitrios Zarouchas Dr. Bianca Giovanardi Prof. dr. Guido de Croon
Assistant Professor Associate Professor & Assistant Professor Professor & Director of MAVLab
Office room NB 0.47 Director of Center of Excellence Office room NB 2.26 Office room NB 0.35
in AI for structures
Office room NB 0.45
13
The Team - TAs
• Ali Alper Ataşoğlu
• Django van der Plas
• Ian Maes
• Jakub Korycki
• Katharina-Inés Janisch
• Mihai Constantinov
• Miłosz Pluciński
• Nikolaus Ricker
• Tamin Doozaneh
• Tan Zing Shawn
• Michal Fedoronko
• Jasper Geijsberts
• Tan Zing Shawn
14
Course Schedule
Lecture Lecturer Lecture Lab for Group I Lab for Group II
1 Dimitrios Zarouchas Introduction to AI Data analysis & Regression Data analysis & Regression
– Wednesday 15:45 – Thursday 08:45 – Tuesday 10:45 (week 3.2)
2 Bianca Giovanardi Classification Speech recognition Speech recognition
– Wednesday 15:45 – Thursday 08:45 – Tuesday 10:45 (week 3.3)
3 Guido de Croon Neural Networks (part 1) Attitude estimation Attitude estimation
– Wednesday 15:45 – Thursday 08:45 – Tuesday 10:45 (week 3.4)
4 Guido de Croon Neural Networks (part 2) Racing drones Racing drones
– Wednesday 15:45 – Thursday 08:45 – Tuesday 10:45 (week 3.5)
5 Dimitrios Zarouchas Dimensionality Reduction Principal Component Principal Component Analysis
– Wednesday 15:45 Analysis – Tuesday 10:45 (week 3.6)
– Thursday 08:45
6 Sara Colombo Ethics & Green AI No Lab No lab
Laura Daumiere – Wednesday 15:45
Miguel Guell Ybarra
7 Nick Eleftheroglou Clustering Turbofan Engine Diagnostics
– Wednesday 15:45 – Thursday 08:45
15
Course Schedule
Lecture Lecturer Lecture Lab for Group I Lab for Group II
1 Dimitrios Zarouchas Introduction to AI Set-up Installation &Python Set-up Installation &Python
– Wednesday 15:45 refresh refresh (week 3.2)
2 Bianca Giovanardi Classification Data analysis & Regression Data analysis & Regression
– Wednesday 15:45 – Thursday 08:45 – Tuesday 10:45 (week 3.3)
3 Guido de Croon Neural Networks (part 1) Speech recognition Speech recognition
– Wednesday 15:45 – Thursday 08:45 – Tuesday 10:45 (week 3.4)
4 Guido de Croon Neural Networks (part 2) Attitude estimation Attitude estimation
– Wednesday 15:45 – Thursday 08:45 – Tuesday 10:45 (week 3.5)
5 Dimitrios Zarouchas Dimensionality Reduction Racing drones Racing drones
– Wednesday 15:45 – Thursday 08:45 – Tuesday 10:45 (week 3.6)
6 Sara Colombo Ethics & Green AI Principal Component Principal Component Analysis
Laura Daumiere – Wednesday 15:45 Analysis – Tuesday 10:45 (week 3.7)
Miguel Guell Ybarra – Thursday 08:45
7 Nick Eleftheroglou Clustering Turbofan Engine Diagnostics
– Wednesday 15:45 – Thursday 08:45 (Thursday week 3.7)
16
House Rules
• The presence during the lectures and/or the labs is not mandatory
• Brightspace is the main channel of communication, i.e. announcements
• During the lecture, you are allowed to use your own computer/notebook to
take notes BUT make sure you don’t disturb your classmates.
• Share is care. If you talk during the lecture, you will be asked to share your
thoughts with the entire classroom.
• A computer-based exam is in place and the assignments act as brownie
points. You may get 1 full point if you complete successfully 5 out of the 6
assignments. Each assignment counts for 0.2 points. NOTE: in case you
complete 6/6 you still get 1 point.
• If you attempt to solve an assignment before its official release it is at your
own risk, the lab may change!
• Safety comes first: USE our set-up!
17
Learning Objectives
• Describe the most commonly used Machine Learning models and their scope
of applicability
• Match the appropriate class of models to a given Aerospace Engineering
application
• Understand the range of applicability and the limitations of the models
studied
• Apply the models studied using well-established open-source Python
packages
• Understand and use concepts and jargon borrowed from the Artificial
Intelligence community in the context of problems in Aerospace Engineering
• Apply conventional Artificial Intelligence algorithms to real-life problems in
Aerospace Engineering
18
MACHINE LEARNING
19
Machine Learning
Supervised Learning Unsupervised Learning
Regression Classification Data reduction Clustering
• Linear Regression • Support Vector • Principal Component Hierarchical Clustering
• Polynomial Regression Machines Analysis • Agglomerative
• Lasso & Ridge • Logistical Regression • Autoencoders • Divisive
Regression • Decision Trees • Linear Discriminant
• Gaussian Regression • Random Forests Analysis Partitional Clustering
• Neural Networks • Neural Networks • K-Means
• Fuzzy C-Means
20
Machine Learning
Supervised Learning Unsupervised Learning
Regression Classification Data reduction Clustering
• Linear Regression Hierarchical Clustering
• Polynomial • Support Vector • Principal Component
Machines Analysis • Agglomerative
Regression • Divisive
• Lasso & Ridge • Logistical Regression • Autoencoders
Regression • Decision Trees • Linear Discriminant
• Random Forests Analysis Partitional Clustering
• Gaussian Regression • K-Means
• Neural Networks • Neural Networks
• Fuzzy C-Means
21
Regression
Input Output
function that make prediction
Quantitative output
• Supervised technique (data is needed to estimate the parameters of the
model)
–Linear regression y=ax+b (data to estimate a and b)
22
Classification
Input Output
function that separates
data into different classes
Qualitative output
• Supervised technique (Prior knowledge of labels)
• Each label has ‘unique’ characteristics and forms a classifier
• The purpose of the function is to maximize the correlation between the
data and the classifiers and organize the data into classes
23
Data reduction
Input Output
function that separates
data into different groups
Qualitative output
• Unsupervised technique (NO prior knowledge of labels)
• The is no need to design a classifier
• The purpose of the function is to gather data into groups which are not
similar with other groups
24
Clustering
Input Output
function that separates
data into different groups
Qualitative output
• Unsupervised technique (NO prior knowledge of labels)
• The is no need to design a classifier
• The purpose of the function is to gather data into groups which are not similar
with other groups
25
Time for a coffee break!
By DALL-E 26
REGRESSION
27
Data– Our source of wisdom
Wisdom is the use of the knowledge to improve
Wisdom how a system works
Knowledge Knowledge is the use of these relations to
understand how a system works
Information Information is the relations between
different types of data
Data Data is everything we can measure
The DIKW model
28
Regression
A regression gives a functional relation
𝑌! = 𝑔 𝑥! , Θ + 𝑈!
xi = nonrandom observations
Yi= random variables
Ui= errors
Θ= a vector with unknown parameters
Reminder: Check the lecture 11 from Probability & Statistics course 29
Regression
• Linear regression model
–𝑌! = α + 𝛽𝑥! + 𝑈!
–α=intercept
–β=slope
• Non-Linear (Polynomial) regression model
–𝑌! = 𝛼 + 𝛽𝑥! + 𝛾𝑥!" + 𝑈! (parabolic) model
30
Linear Regression Model
• We need to estimate the intercept (α) and the
slope (β). We employ the Least Squares
Method (LSM)
–𝛼 = 𝑦*# − 𝛽𝑥̅#
# ∑(&! '&̅ " )(*! '*+" )
–β= ∑(&! '&̅ " )#
31
Polynomial Regression Model
• 𝑌! = 𝛽" + 𝛽# 𝑥! + 𝛽$ 𝑥!$ + ⋯ + 𝛽% 𝑥!% + 𝑈! , (i=1,2…n)
𝑦# 1 𝑥# 𝑥#$ … 𝑥#% 𝛽& 𝑈#
𝑦$ 1 𝑥$ 𝑥$$ … 𝑥$% 𝛽& + 𝑈$
• ⋮ = 1 ⋮ ⋮ ⋮ ⋮ ⋮
𝑦& 1 𝑥& 𝑥&$ … 𝑥&% 𝛽& 𝑈&
32
Polynomial Regression Model
• 𝑌! = 𝛽" + 𝛽# 𝑥! + 𝛽$ 𝑥!$ + ⋯ + 𝛽% 𝑥!% + 𝑈! , (i=1,2…n)
𝑦# 1 𝑥# 𝑥#$ … 𝑥#% 𝛽% 𝑈#
𝑦$ 1 𝑥$ 𝑥$$ … 𝑥$% 𝛽% + 𝑈$
• ⋮ = 1 ⋮ ⋮ ⋮ ⋮ ⋮
𝑦& 1 𝑥& 𝑥&$ … 𝑥&% 𝛽% 𝑈&
Vandermonde matrix
33
Polynomial Regression Model
𝑦! 1 𝑥! 𝑥!" … 𝑥!$ 𝛽$ 𝑈!
𝑦" 1 𝑥" 𝑥"" … 𝑥"$ 𝛽$ + 𝑈"
⋮ =
1 ⋮ ⋮ ⋮ ⋮ ⋮
𝑦# 1 𝑥# 𝑥#" … 𝑥#$ 𝛽$ 𝑈#
𝑦⃗ = 𝑋𝛽⃗ + 𝜀⃗ 𝛽⃗ = (Χ ) Χ)*+ Χ ) 𝑦⃗
34
Polynomial Regression Model - Example
𝑦! 1 𝑥! 𝑥!" … 𝑥!$ 𝛽$ 𝑈!
𝑦" 1 𝑥" 𝑥"" … 𝑥"$ 𝛽$ + 𝑈"
⋮ = 1 ⋮ ⋮ ⋮ ⋮ ⋮
𝑦# 1 𝑥# 𝑥#" … 𝑥#$ 𝛽$ 𝑈#
𝛽⃗ = (Χ ) Χ)*+ Χ ) 𝑦⃗
X=[1,4,5,9] & Y=[1,3,9,52]
35
Polynomial Regression Model - Example
𝑦! 1 𝑥! 𝑥!" … 𝑥!$ 𝛽$ 𝑈!
𝑦" 1 𝑥" 𝑥"" … 𝑥"$ 𝛽$ + 𝑈"
⋮ = 1 ⋮ ⋮ ⋮ ⋮ ⋮
𝑦# 1 𝑥# 𝑥#" … 𝑥#$ 𝛽$ 𝑈#
𝛽⃗ = (Χ ) Χ)*+ Χ ) 𝑦⃗
X=[1,4,5,9] & Y=[1,3,9,52]
β0=4.59 β1=-4.76 β2=1.11
36
Underfitting and Overfitting
37
The ultimate goal of Machine Learning
38
The ultimate goal of Machine Learning
Making good predictions
on unseen data
39
The ultimate goal of Machine Learning
The performance of your model on the
training set is NOT a good indicator of
performance on unseen data
40
Polynomial fitting of sin(2πx)
1 M =0
t Example:
0
Fit sin(2πx) with a
polynomial of order M
−1
0 x 1
Figure credits: Pattern Recognition and Machine
41
Learning, C.M. Bishop, 2009
Polynomial fitting of sin(2πx)
For M = 0 and M = 1,
1 M =1
the model is too
t
simple
0
It misses important
−1 relations between the
features and the
0 x 1 target
Figure credits: Pattern Recognition
42
and Machine Learning, C.M.
Polynomial fitting of sin(2πx)
1 M =3
t For M = 3 the model
0 is able to capture the
change of convexity
−1
0 x 1
Figure credits: Pattern Recognition
43
and Machine Learning, C.M.
Polynomial fitting of sin(2πx)
1 M =9 For M = 9 the
t polynomial fits the
0
training dataset exactly
Zero error on the
−1
training set, but
generalization to
0 1
x
unseen data is poor!
Figure credits: Pattern Recognition
44
and Machine Learning, C.M.
Training, validation, and test sets
Always keep the over-underfitting balance in mind
when
selecting a model and its parameters!
Split the dataset into:
• Training set
• Validation set
• Test set 45
Lab 1st
• Installation of docker
• Refresh of Python
46
See you next time!
By DALL-E 47