CHAPTER FIVE

MACHINE LEARNING BASICS

 CHAPTER OUT LINE

Outline  Reinforcement learning

 Knowledge in Learning  Deep Learning
 Learning Probabilistic Models  Neural networks and back-
propagation
 Supervised learning
 Recurrent neural networks and
 Linear classification models
LSTMs
 Probabilistic model
 Convolution neural networks
 Unsupervised learning
2
 Clustering models
 INTRODUCTION TO MACHINE LEARNING

• An agent is learning if it improves its performance after making

observations about the world. When the agent is a computer, we call it
machine learning.

• A computer observes some data, builds a model based on the data,

and uses the model as both a hypothesis about the world and a piece
of software that can solve problems.

• Why would we want a machine to learn? Why not just program? There
are two main reasons:
• First, the designers cannot anticipate all possible future situations.
3
• Second, sometimes the designers have no idea how to program a solution.
"When is a learning-based
approach (like Machine Learning)
more suitable than a traditional
programming approach for
solving a problem?"
4
The "Explicit Programming" Problem: Spam Filter

• Imagine you had to write a traditional, explicit program for a spam

filter.

• You would have to define every single rule.

• Rule 1: If the email contains the words "Nigerian Prince", flag

as spam.

• Rule 2: If the email contains "FREE" and "!!!", flag as spam.

• Rule 3: If the sender's address is from a known spam domain

list (which you must also maintain), flag as spam.

• ... and so on, for thousands of potential rules. 5

 Spam Filter Continued …

The Problems with this approach:

• It's endless: Spammers constantly change their tactics. You'd be

in a constant arms race, updating your rules daily.

• It's fragile: The rule-based system is weak because it only knows

the exact words you taught it.

• It can't generalize: It can't understand the concept of spam; it

can only recognize what you've explicitly told it to look for.

6
 INTRODUCTION TO MACHINE LEARNING

Forms of Learning:
Any component of an agent program can be improved by machine learning.
Learning takes many forms, depending on the nature of the agent, the
component to be improved, and the available feedback.
The improvements, and the techniques used to make them, depend on these
factors:
• Which component is to be improved.
• What prior knowledge the agent has, which influences the model it builds.
• What data and feedback on that data is available. 7
 INTRODUCTION TO MACHINE LEARNING
Forms of Learning - Example
Smart Home Assistant
Example: A smart home assistant that learns by observing the homeowner's routines and
commands.
When the homeowner repeatedly says "Good night" and then turns off the lights,
locks the door, and sets the thermostat to 68°F, the assistant learns a condition-
action rule for a complex routine (component 1: policy).
By being shown many audio samples labeled as "vacuum cleaner noise" versus
"hairdryer noise," it can learn to recognize and distinguish between these specific
sounds to, for example, not interrupt during cleaning time (component 2:
8
perception/model).
 INTRODUCTION TO MACHINE LEARNING
Machine learning is the science of getting computers to act without being explicitly
programmed.
In the past decade, machine learning has given us self-driving cars, practical speech
recognition, effective web search, and a vastly improved understanding of the human
genome.
Machine learning is so pervasive today that you probably use it dozens of times a day
without knowing it.
Many researchers also think it is the best way to make progress towards human-level AI.
Machine learning, a branch of artificial intelligence, concerns the construction and study
of systems that can learn from data.
9
INTRODUCTION TO MACHINE LEARNING

10
INTRODUCTION TO MACHINE LEARNING
• The various strategies used by machine learning agents or machine
• into a taxonomy based on the
learning algorithms can be organized
desired outcome of the algorithm or the type of input available during
training the machine as follows.
• Based on Learning Strategies:-
• Supervised learning algorithms
• Unsupervised learning algorithms
• Reinforcement learning algorithms

11
SUPERVISED LEARNING ALGORITHMS:

• It is applicable when a machine has sample input and output data.

• learns a function that maps from input to
• The agent observes input-output pairs and
output. An output is called a label.
• Basically, the model is trained over a labeled dataset.
• A labeled database is one which contains both inputs as well as the output.
• The agent learns a function that, when given a new image, predicts the appropriate
label.
• Here the trained dataset act a teacher, and its primary role is to train the model. The
prediction is made once the model is built.
• The task is to learn a function y=h(x). 12
INTRODUCTION TO MACHINE LEARNING

13
SUPERVISED LEARNING ALGORITHMS

14
SUPERVISED LEARNING ALGORITHMS

15
How can we determine if
a machine learning
problem is a regression
problem or a
classification problem?
 PRACTICE EXAMPLES FOR DISCUSSION

Have students classify these problems:

 Predicting student exam scores
 Identifying if a tumor is malignant or benign
 Forecasting stock prices
 Detecting credit card fraud
 Estimating the age of a person from a photo
• If exact age:
• If age group: 17
What do we do when
we have data without
labels?

18
"Labeling data is expensive, time-consuming, and sometimes
impossible. Unsupervised learning is our key to unlocking the
vast treasure troves of unlabeled data in the world."

19
 UNSUPERVISED LEARNING
• The agent learns patterns in the input without any explicit feedback.
• It is a kind of learning in which the output target is not given to the model while performing
the training. It only has the input variables.
• The model has to learn by itself.
• The training data that is fed to the system can be unlabeled as well as unstructured in
nature.
• The unstructured data is the one where noise or some irrelevant information is present.
• The most common unsupervised learning task is clustering: detecting potentially useful
clusters of input examples.

20
UNSUPERVISED LEARNING

21
REINFORCEMENT LEARNING ALGORITHM
• The agent learns from a series of reinforcements: rewards and punishments.
•
• In this type of learning, agents (computer programs) need to explore the
environment, perform actions, and on the basis of their actions, they get
rewards as feedback.
• For each good action, they get a positive reward, and for each bad action,
they get a negative reward.
• The goal of a Reinforcement Learning agent is to maximize the positive
rewards.
• Since there is no labeled data, the agent is bound to learn by its experience
only. 22
REINFORCEMENT LEARNING ALGORITHM

23
DEEP LEARNING

 These algorithms run data through

• several "layers" of the neural
network algorithm.
 Deep learning requires a large amount of data to learn.
 Deep learning is currently the most widely used approach for
applications such as:
 visual object recognition,
 machine translation,
 speech recognition,
 speech synthesis, and
 image synthesis. 24
DEEP LEARNING

Deep learning has its origins in early •work that tried to model networks of
neurons in the brain with computational circuits.

The networks trained by deep learning methods are often called neural
networks.

Artificial Neural Network or **Neural Network** was modeled after the human
brain.

Humans have a mind to think and to perform tasks in particular conditions, but
how can a machine do that? 25
[Link]
Deep learning has its origins in early work that tried to model networks of neurons in the brain with computational circuits.
PNG 95.87KB
The networks trained by deep learning methods are often called neural networks.
change to text
Artificial Neural Network or Neural Network was modeled after the human brain.
Humans have a mind to think and to perform tasks in particular conditions, but how can a machine do that?
For this purpose, the artificial brain was designed, which is called a neural network.

DEEP LEARNING

26
[Link]
Deep learning has its origins in early work that tried to model networks of neurons in the brain with computational circuits.
PNG 95.87KB
The networks trained by deep learning methods are often called neural networks.
change to text
Artificial Neural Network or Neural Network was modeled after the human brain.
Humans have a mind to think and to perform tasks in particular conditions, but how can a machine do that?
For this purpose, the artificial brain was designed, which is called a neural network.

DEEP LEARNING ALGORITHMS

27
DEEP LEARNING ALGORITHMS

28
 DEEP LEARNING ALGORITHMS

 When training a neural network, an algorithm is

•
used to minimize the loss. This algorithm is called
Gradient Descent.
 Loss refers to the incorrect outputs given by the
hypothesis function.
 A Multilayer Neural Network is an artificial neural
network that has an input layer, an output layer, and
at least one hidden layer in between.
The number of nodes in the hidden layer and the
number of hidden layers can be decided accordingly.

29
 DEEP LEARNING ALGORITHMS

 Each node in the hidden layer• calculates its output value

based on the linear relationship with inputs, and the final
output is calculated based on the output values of the
nodes in the hidden layers.
 Each node's output value is multiplied by its respective
weight, and then the final output is calculated.

30
DEEP LEARNING ALGORITHMS

31
DEEP LEARNING ALGORITHMS

 we need to calculate the values of each • node in all the hidden layers to train the
network properly.
 Steps followed are:
 Start with the random values of weights.
 Repeat
 Firstly the error in the output layer is calculated.
 For every layer, starting from the output layer moving back inward towards
the first hidden layer from the output layer.
 Calculate the error in the back layer based on the values of weights.
 Then, Update the weights.
 The Backpropagation algorithm is the key algorithm through which multilayer
neural networks are trained based on the values of weights and create a function 32
to minimize the loss.
DEEP LEARNING ALGORITHMS

 Backpropagation is the core algorithm

• for training neural
networks, using the "chain rule" from calculus to efficiently
calculate the gradient of a loss function with respect to the
network's weights and biases.
 By propagating the error backward from the output layer to the
input layer, it determines how much each weight and bias
contributed to the error, allowing the network to adjust its
parameters to minimize future errors.

33
DEEP LEARNING ALGORITHMS

34
DEEP LEARNING ALGORITHMS
 Weights are numerical values assigned to the connections between neurons
• They represent the importance or
in different layers of a neural network.
strength of each input signal.
 Bias: A bias is a constant value added to the weighted sum of inputs in a
neuron, before the activation function is applied.
 Bias provides the model with flexibility to shift the activation function to the left
or right, which helps the network to better fit the data and learn complex
patterns.
 Without biases, a neuron would only activate if the weighted sum of inputs
crossed a certain threshold relative to the origin, limiting its ability to model
real-world data effectively.
•Learning: Like weights, biases are adjusted during the training process 35
through backpropagation to minimize the overall error.
DEEP LEARNING ALGORITHMS
How Neural Networks Learn?
Neural networks learn through a process• involving forward propagation and
backpropagation. Let’s see each step:
1. Forward Propagation
Forward propagation is the initial phase of processing input data through the
neural network to produce an output or prediction. Let's see how it works:

36
DEEP LEARNING ALGORITHMS
Input Layer: The process starts with data entering the network’s input layer.
This could be anything from pixel values • in an image to feature values in a
dataset.
Weighted Sum: Each neuron calculates the weighted sum of the inputs. Each
input is multiplied by its corresponding weight which shows the importance of
that input.
Adding Biases: A bias is added to the weighted sum. Bias helps shift the output
and provides flexibility, allowing the network to make better predictions even if
all input values are zero.
Activation Function: The sum of the weighted inputs plus bias is passed
through an activation function (e.g ReLU, sigmoid). The activation function
decides if the neuron should activate which means it will pass information to the 37
next layer or stay inactive.
DEEP LEARNING ALGORITHMS

38
DEEP LEARNING ALGORITHMS

Backpropagation •
Once the network has
made a prediction, it's
important to evaluate how
accurate that prediction is
and make adjustments to
improve future predictions.
This is
where backpropagation co
mes: 39
DEEP LEARNING ALGORITHMS
 Error Calculation: Once the network generates an output, it’s compared to the
• between the predicted and actual
actual result (the target). The difference
values is the error also called the loss.
 Gradient Calculation: The error is propagated back through the network and
the gradient or slope of the error with respect to the weights and biases is
calculated. This tells the network how to adjust the parameters to minimize the
error.
 Updating Weights and Biases: Using the gradient, the network adjusts the
weights and biases. The goal is to reduce the error in future predictions. This
step is done through an optimization algorithm like gradient descent.
 Iteration: This process of forward and backward propagation is repeated many
times on different batches of data. With each iteration, the network’s weights
and biases get closer to the optimal values, improving the model’s 40

performance.
DEEP LEARNING ALGORITHMS

Types of Artificial Neural Networks •

Neural Network works similarly as the human nervous system works.

There are several types of neural network.
These networks implementation are based on the set of parameter and
mathematical operation that is required for determining the output.
Most widely used are:
Convolution neural networks
Recurrent neural networks and LSTMs

41
DEEP LEARNING ALGORITHMS

Convolution Neural Networks •

 In deep learning, a convolutional neural network (CNN, or ConvNet) is a
class of artificial neural network (ANN), most commonly applied to
analyze visual imagery.
 Convolutions are a set of layers that go before the neural network
architecture.
 Convolutional networks are particularly well suited for image processing
and other tasks where the data have a grid topology.

42
DEEP LEARNING ALGORITHMS

Recurrent Neural Networks (RNNs) •

•Purpose: To recognize patterns in sequences of data, such as
text, speech, or time-series data.
 They have a "memory" from previous steps in the sequence,
allowing them to use past information to inform present
predictions.
•Limitation: Traditional RNNs have difficulty learning from data
with long-term dependencies because the signal from earlier steps
can be lost over time, a problem known as the vanishing gradient
problem.
43
DEEP LEARNING ALGORITHMS

Recurrent Neural Networks (RNNs) •

•Purpose: To recognize patterns in sequences of data, such as
text, speech, or time-series data.
 They have a "memory" from previous steps in the sequence,
allowing them to use past information to inform present
predictions.
•Limitation: Traditional RNNs have difficulty learning from data
with long-term dependencies because the signal from earlier steps
can be lost over time, a problem known as the vanishing gradient
problem.
44
DEEP LEARNING ALGORITHMS

Long Short-Term Memory (LSTM) networks

•Purpose: To address the limitations of standard• RNNs by effectively learning long-term
dependencies.
•Structure: LSTMs are a type of RNN, but they have a more complex internal structure.
•Key components:
•Cell State: A long-term memory pathway that information can flow through, which helps the
network remember information over extended periods.
•Gates: Mechanisms that control the flow of information into and out of the cell state.
•Forget Gate: Decides what information to discard from the previous cell state.
•Input Gate: Determines what new information to store in the cell state.
•Output Gate: Decides what part of the current cell state to output as the hidden state for
the next step.
•How it works: The gates use sigmoid and tanh activation functions to regulate the information,
allowing the LSTM to selectively remember or forget data, which prevents the vanishing gradient
problem and enables it to handle long sequences. 45
46