Introduction
Dr. [Link] Jeeva
AP/ECE
Sri Sivasubramaniya Nadar College of Engineering
Session Objectives
• To introduce the concept of machine learning
Session Outcomes
At the end of this session, the student will be able to
• Understand the difference between artificial intelligence, machine learning and
deep learning
• Know the definition of machine learning
Machine Learning History
• Samuel’s Checker • Rosenblatt’s • Symbolic Concept
perceptron Induction
Board playing
• Expert Systems
program • Pattern Recognition • Quinlan’s ID3
• Limitations of (Decision Tree)
Perceptron • Natural Language
Processing
1950s 1960s 1970s
• Advanced Decision • Support Vector Machines
Trees • Data Mining
• Resurgence of Neural • Reinforcement Learning
Network
• Multilayer Perceptron • Ensembles – Adaboost
algorithm
• Back Propagation
• Bayes Net Learning
1980s 1990s
Machine Learning History
Recently ML is becoming very popular for the following reasons:
New software/Algorithms
Artificial Neural Networks
Deep Learning
New hardware / GPU’s
Big data availability
Cloud Enabled
Machine Learning Applications
1994: Self Driving Car
1997: Deep Blue beats Gary Kasparov
2009: Google builds self driving car
2011: Watson wins Jeopardy
2014: Human Vision surpassed by ML systems
Machine Learning Applications
Learning to recognize spoken words
CMU Sphinx, Hey-siri, Alexa
Learning to drive an autonomous vehicle
ALVINN System, Google Lexus
Learning to classify new astronomical structures
Classify celestial objects
Learning to play world-class backgammon
ML - What we see now?
Spam Filtering
Amazon’s Echo
Product Recommendations
What is Machine Learning?
How ML is different ?
Data Program Data Examples
Computer Computer
Output Program / Model
Algorithm Machine Learning
What is Machine Learning?
Learning is the ability to improve one’s behaviour with experience
Building computer systems that automatically improves with
experience
ML explores algorithms that learns from data, build models from
data and use the model for decision making tasks
Definition by Tom Mitchell:
“A computer program is said to learn from experience E with
respect to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with
experience E ”
What is Machine Learning?
“A computer program is said to learn from experience E with respect
to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with
experience E ”
What is Machine Learning?
“A computer program is said to learn from experience E with respect
to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with
experience E ”
Prediction
What is Machine Learning?
“A computer program is said to learn from experience E with respect
to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with
experience E ”
Prediction Data
What is Machine Learning?
“A computer program is said to learn from experience E with respect
to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with
experience E ”
Prediction Data Accuracy
AI, ML and DL
ARTIFICIAL INTELLIGENCE
MACHINE LEARNING
DEEP LEARNING
AI, ML and DL
• AI: A branch of computer science dealing with the simulation of
intelligent behavior in computers
• ML: Field of study that gives computers the ability to learn without
being explicitly programmed
• DL: It is a subset of ML that is inspired by how human brains work.
Learning
• Learning from experience
• For machines, its learning from data
• Data – past experience
• Types of Learning:
– Supervised
– Unsupervised
– Reinforcement
– Evolutionary
Types of Learning
• Supervised learning:
A training set of examples with the correct responses (targets) is provided
and, based on this training set, the algorithm generalizes to respond
correctly to all possible inputs. This is also called learning from exemplars.
• Unsupervised learning:
Correct responses are not provided, but instead the algorithm tries to
identify similarities between the inputs so that inputs that have something
in common are categorized together. The statistical approach to
unsupervised learning is known as density estimation.
Types of Learning
• Reinforcement learning:
The algorithm gets told when the answer is wrong, but does not get told how to
correct it. It has to explore and try out different possibilities until it works out how
to get the answer right. Reinforcement learning is sometime called learning with
a critic because of this monitor that scores the answer, but does not suggest
improvements.
• Evolutionary learning:
Biological evolution can be seen as a learning process: biological organisms
adapt to improve their survival rates and chance of having offspring in their
environment. Evolutionary learning looks at how well this can be modeled in a
computer, using an idea of fitness, which corresponds to a score for how good
the current solution is.
Supervised Learning
New Input
X
Input Output
Learning
Model
Algorithm
Output
y
Supervised Learning
• Given:
Input feature set: x1, x2, … xn
Target value: y
A set of training examples where the values of the input
features x1, x2, … xn and the target feature y are given for
each example
A new input containing only the input features
• Prediction:
Predict the values for the target features for the new example.
Classification – if y is discrete
Regression – if y is continuous
Supervised Learning
Input Output
Labelled
data
Classification Plane
Unsupervised Learning
New Input
X
Input
Learning
Model
Algorithm
Output
y
Unsupervised Learning
• Given:
Input feature set: x1, x2, … xn
A set of training examples where the values of the input
features x1, x2, … xn are given for each example
A new input containing only the input features
• Prediction:
Predict the values for the target features for the new example.
Classification – if y is discrete
Regression – if y is continuous
Unsupervised Learning
Input
Clustering
Reinforcement Learning
action at
state st st+1
Agent Environment
reward rt rt+1
Schematic of ML Model
Training
Training set
Learning
Algorithm
Testing
Hypothesis Predicted
X / Model y
Representation
• Decision Trees
Old?
yes no
Short story? Author known?
yes no yes no
Read Author Known ? Read Short story?
yes no yes no
Read Skip Read Skip
To predict whether a person will read a story book or not?
Representation
• Linear Function / Polynomials
x
x
Distance covered
x x
x x
x
x
x
x x
x Predicting the distance covered
x
x x
in running based on the amount
of food consumed
Food consumed (calories)
Representation
• Multivariate Linear Functions / Polynomials
Representation
• Single Layer Neural Network (Perceptron)
Representation
• Multilayer Layer Neural Network
Terminology
• Features: The number of features or distinct traits that
can be used to describe each item in a quantitative
manner.
• Feature vector: n-dimensional vector of numerical
features that represent some object
• Instance space X: Set of all possible objects
describable by features.
• Example (x,y): Instance x with label y = f(x)
Classification & Regression
• Classification:
– Class label is discrete
– Example – Yes or No / 1, 2, 3, or 4
• Regression:
– Class Label is continuous
– Example – predicting a real number
Summary
• Learning is the ability to improve one’s behaviour with experience
• Building computer systems that automatically improves with experience
• Learning algorithm will explore possible function parameters, so that based on
the training experience it can come up with a best solution given its
computational limitations
• AI ML DL
• Machine Learning – Supervised, Unsupervised and Reinforcement
• Supervised – pre-classified training examples
• Unsupervised – training examples without labels
• Reinforcement – reward based learning
• Target function Representations – Decision Trees, Polynomials, Neural
Networks, etc.,
Quiz
Test Your Understanding
• Define Machine Learning.
• Determine the task, experience and performance of the ML problem Sales
prediction for a company
• What is deep learning?
• Explain different types of Machine Learning with examples.
Thank You