Learning

Module 5
 Learning
■ An agent is learning if it improves its
performance on future tasks after making
observations about the world
 A general model of learning
agents

3
 Defining the Learning Task
Improve on task, T, with respect to
performance metric, P, based on experience, E.
T: Playing checkers
P: Percentage of games won against an arbitrary opponent
E: Playing practice games against itself
T: Recognizing hand-written words
P: Percentage of words correctly classified
E: Database of human-labeled images of handwritten words
T: Driving on four-lane highways using vision sensors
P: Average distance traveled before a human-judged error
E: A sequence of images and steering commands recorded while
observing a human driver.
T: Categorize email messages as spam or legitimate.
P: Percentage of email messages correctly classified.
E: Database of emails, some with human-given labels
 Designing a Learning System
■ Choose the training experience
■ Choose exactly what is too be learned, i.e. the
target function.
■ Choose how to represent the target function.
■ Choose a learning algorithm to infer the target
function from the experience.

Learner
Environm
ent/ Knowle
Experienc dge
e
Performance
Element
 Sample Learning Problem
■ Learn to play checkers from self-play
■ Develop an approach analogous to that
used in the first machine learning system
developed by Arthur Samuels at IBM in
1959.
 Forms of Learning
■ Any component of an agent can be improved by
learning from data. The improvements, and the
techniques used to make them, depend on four major
factors:
1. Which component is to be improved.
2. What prior knowledge the agent already has.
3. What representation is used for the data and the
component.
4. What feedback is available to learn from.
 Components to be learned
1. A direct mapping from conditions on the current state to actions.
2. A means to infer relevant properties of the world from the
percept sequence.
3. Information about the way the world evolves and about the
results of possible actions
the agent can take.
4. Utility information indicating the desirability of world states.
5. Action-value information indicating the desirability of actions.
6. Goals that describe classes of states whose achievement
maximizes the agent’s utility
 Representation and prior
knowledge
■ Examples of representations for agent
components: propositional and first-order
logic, Bayesian networks etc.
■ Inputs most often are in the form of a
factored representation—a vector of attribute
values
■ Outputs are mostly in the form of continuous
numerical value or a discrete value
 Inductive and Deductive
learning
■ Inductive Learning: Learning a (possibly
incorrect) general function or rule from specific
input–output pairs is called inductive learning.

■ Analytical /Deductive Learning: going from a

known general rule to a new rule that is logically
entailed, but is useful because it allows more
efficient processing.
 Feedback to learn from
■ There are three types of feedback that
determine the three main types of
learning:
 UNSUPERVISED LEARNING
■ In unsupervised learning the agent learns
patterns in the input even though no
explicit feedback is supplied.
■ The most common unsupervised learning task
is clustering: detecting potentially useful
clusters of input examples.
■ For example, a taxi agent might gradually develop
a concept of “good traffic days” and “bad traffic
days” without ever being given labeled examples
of each by a teacher
Unsupervised learning:
clustering
 Unsupervised Learning
■ Applications:
1. Customer Segmentation: Segmenting customers or other
data points into different groups based on shared
characteristics. For example, you might use clustering to
identify different types of customers based on their
demographics, purchasing habits, or other data.
2. Anomaly detection: Clustering can be used to identify
unusual data points that do not fit into any of the
established clusters. These data points might represent
unusual events or anomalies that warrant further
investigation.
 Unsupervised Learning
■ Applications:
3. Dimensionality reduction: Clustering can be used to reduce
the number of features in a dataset by grouping together
similar features into clusters. This can be useful for
visualizing data or building machine learning models.
4. Preprocessing data: Clustering can be used as a
preprocessing step before building a machine learning
model. For example, you might use clustering to group
together similar data points, and then build a separate
model for each cluster.
 Unsupervised Learning
■ Clustering Algorithms:
1. K means Clustering
2. Hierarchical Clustering
3. DBSCAN Clustering
4. Expectation-Maximization (EM) Algorithm
5. Spectral Clustering
 SUPERVISED LEARNING
■ In supervised learning the agent observes some
example input–output pairs and learns a
function that maps from input to output.
■ Supervised machine learning has two key
components:
■ input data
■ output labels.

■ The goal is to build a model that can learn from this labeled
data to make predictions or classifications on new, unseen
data.
■ The labeled data consists of :
■ input features (also known as independent variables or
predictors)
■ corresponding output labels (also known as dependent
variables or targets).
■ The model’s objective is to capture patterns and relationships
between the input features and the output labels, allowing it
to generalize and make accurate predictions on unseen data.
 Supervised Learning

• Here x and y can be any value; they need not be numbers.

 Supervised Learning
■ The function h is a hypothesis.
■ To measure the accuracy of a hypothesis we give it a test set
of examples that are distinct from the training set.
■ Learning is a search through the space of possible
hypotheses for one that will perform well, even on new
examples beyond the training set.
■ We say a hypothesis generalizes well if it correctly predicts
the value of y for novel examples
 Supervised Learning:
Classification and Regression
■ When the output y is one of a finite set of
values (such as sunny, cloudy or rainy), the
learning problem is called classification, and is
called Boolean or binary classification if there
are only two values.
■ When y is a (such as tomorrow’s
temperature), the learning problem is called
regression.
 Supervised Learning Classification
■ Example: Cancer diagnosis

Training
Set
• Use this training set to learn how to classify patients
where diagnosis is not known:

Test Set

Input Data Classification

• The input data is often easily obtained, whereas the

classification is not.
 Supervised Learning: Regression
■ Example: fitting a function of a single variable to some data
points. The examples are points in the (x, y) plane, where y = f(x).
W

Ref : AI a modern approach- Peter Norvig

 Supervised Learning
■ Steps:
■ Data Collection and Labeling
■ Training and Test Sets
■ Feature Extraction
■ Model Selection and Training
■ Prediction and Evaluation
 Supervised Learning
■ Algorithms:
■ Linear Regression
■ Logistic Regression
■ Decision Trees
■ Random Forests
■ Support Vector Machines (SVM)
■ Naive Bayes
■ K-Nearest Neighbors (KNN)
 Supervised Learning
■ Applications
■ Spam Email Detection
■ Sentiment Analysis
■ Image Classification
■ Credit Scoring
■ Medical Diagnosis
■ Stock Market Prediction

[Link]
 Semi-Supervised Learning
■ Semi-supervised learning is the type of machine
learning that uses a combination of a small
amount of labeled data and a large amount of
unlabeled data to train models.
■ This approach to machine learning is a
combination of supervised machine learning,
which uses labeled training data, and
unsupervised learning, which uses unlabeled
training data.
Semi Supervised Learning
Semi supervised learning
 REINFORCEMENT LEARNING
■ In reinforcement learning the agent learns
from a series of reinforcements—rewards
or punishments.
■ For example, the lack of a tip at the end of the
journey gives the taxi agent an indication that it did
something wrong.
■ The two points for a win at the end of a chess game
tells the agent it did something right.
 Reinforcement Learning
■ Reinforcement learning is a machine learning training
method based on rewarding desired behaviors and
punishing undesired ones.
 Reinforcement learning

[Link]
 Reinforcement Learning
■ Applications:
■ Reinforcement learning is used when teaching a computer to
play games. The aim is to learn which tactics lead to victory
and which do not.
■ In autonomous driving, these learning algorithms are also
used so that the vehicle can decide on its own which course of
action is best.
■ For the air conditioning of server rooms, reinforcement
learning models decide when and how much to cool down the
room to use energy efficiently.
Defining the Learning Task
Improve on task, T, with respect to
performance metric, P, based on experience, E.
T: Playing checkers
P: Percentage of games won against an arbitrary opponent
E: Playing practice games against itself
T: Recognizing hand-written words
P: Percentage of words correctly classified
E: Database of human-labeled images of handwritten words
T: Driving on four-lane highways using vision sensors
P: Average distance traveled before a human-judged error
E: A sequence of images and steering commands recorded while
observing a human driver.
T: Categorize email messages as spam or legitimate.
P: Percentage of email messages correctly classified.
E: Database of emails, some with human-given labels
 Designing a Learning System
■ Choose the training experience
■ Choose exactly what is too be learned, i.e. the
target function.
■ Choose how to represent the target function.
■ Choose a learning algorithm to infer the target
function from the experience.

Learner
Environm
ent/ Knowle
Experienc dge
e
Performance
Element