UNIT-V

DISTANCE MEASURES

Distance measures play an important role in machine learning.

They provide the foundation for many popular and effective machine learning
algorithms like k-nearest neighbors for supervised learning and k-means
clustering for unsupervised learning.

Different distance measures must be chosen and used depending on the types of
the data. As such, it is important to know how to implement and calculate a range
of different popular distance measures and the intuitions for the resulting scores.

In this tutorial, you will discover distance measures in machine learning.

Types of Distance Metrics in Machine Learning

1. Euclidean Distance
2. Manhattan Distance
3. Minkowski Distance
4. Hamming Distance

Euclidean Distance
Euclidean Distance represents the shortest distance between two [Link] is the
square root of the sum of squares of differences between corresponding elements.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Most machine learning algorithms, including K-Means use this distance metric to
measure the similarity between observations. Let’s say we have two points, as
shown below:

So, the Euclidean Distance between these two points, A and B, will be:

Formula for Euclidean Distance

We use this formula when we are dealing with 2 dimensions. We can generalize
this for an n-dimensional space as:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Where,

 n = number of dimensions
 pi, qi = data points

Manhattan Distance
Manhattan Distance is the sum of absolute differences between points across all
the dimensions.

We can represent Manhattan Distance as:

Formula for Manhattan Distance

Since the above representation is 2 dimensional, to calculate Manhattan Distance,
we will take the sum of absolute distances in both the x and y directions. So, the
Manhattan distance in a 2-dimensional space is given as:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
And the generalized formula for an n-dimensional space is given as:

Where,

 n = number of dimensions
 pi, qi = data points

Minkowski Distance
Minkowski Distance is the generalized form of Euclidean and Manhattan Distance.

Formula for Minkowski Distance

Here, p represents the order of the norm

Hamming Distance
Hamming Distance measures the similarity between two strings of the same length.
The Hamming Distance between two strings of the same length is the number of
positions at which the corresponding characters are different.

Let’s understand the concept using an example. Let’s say we have two strings:

“euclidean” and “manhattan”

Dept of CSE, ASGI B. Uma Maheswari
Assistant Professor
Since the length of these strings is equal, we can calculate the Hamming Distance.
We will go character by character and match the strings. The first character of both
the strings (e and m, respectively) is different. Similarly, the second character of
both the strings (u and a) is different. and so on.

Look carefully – seven characters are different, whereas two characters (the last
two characters) are similar:

Neighbours and Exemplars in machine learning

In machine learning, neighbors refer to instances that are close or similar to a

given data point within a dataset. The concept is used in algorithms like K-Nearest
Neighbors (KNN), which classify data based on the majority class of its nearest
neighbors. Exemplars, on the other hand, are specific instances or prototypes
used to represent a class or category in models like prototype-based learning.
These exemplars serve as references for categorizing new data points. Both
neighbors and exemplars play crucial roles in various machine learning
algorithms, aiding in classification, clustering, and pattern recognition tasks by
utilizing similarity measures between data points.

After completing this tutorial, you will know:

 The role and importance of distance measures in machine learning

algorithms.
 How to implement and calculate Hamming, Euclidean, and Manhattan
distance measures.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
  How to implement and calculate the Minkowski distance that generalizes
the Euclidean and Manhattan distance measures.
Role of Distance Measures

Distance measures play an important role in machine learning.

A distance measure is an objective score that summarizes the relative difference

between two objects in a problem domain.

Most commonly, the two objects are rows of data that describe a subject (such as
a person, car, or house), or an event (such as a purchase, a claim, or a diagnosis).

Perhaps the most likely way you will encounter distance measures is when you
are using a specific machine learning algorithm that uses distance measures at its
core. The most famous algorithm of this type is the k-nearest neighbors
algorithm, or KNN for short.
In the KNN algorithm, a classification or regression prediction is made for new
examples by calculating the distance between the new example (row) and all
examples (rows) in the training dataset. The k examples in the training dataset
with the smallest distance are then selected and a prediction is made by
averaging the outcome (mode of the class label or mean of the real value for
regression).

KNN belongs to a broader field of algorithms called case-based or instance-based

learning, most of which use distance measures in a similar manner. Another
popular instance-based algorithm that uses distance measures is the learning
vector quantization, or LVQ, algorithm that may also be considered a type of
neural network.
Dept of CSE, ASGI B. Uma Maheswari
Assistant Professor
Related is the self-organizing map algorithm, or SOM, that also uses distance
measures and can be used for supervised or unsupervised learning. Another
unsupervised learning algorithm that uses distance measures at its core is the K-
means clustering algorithm.

In instance-based learning the training examples are stored verbatim, and a

distance function is used to determine which member of the training set is closest
to an unknown test instance. Once the nearest training instance has been located,
its class is predicted for the test instance.

— Page 135, Data Mining: Practical Machine Learning Tools and Techniques, 4th
edition, 2016.
A short list of some of the more popular machine learning algorithms that use
distance measures at their core is as follows:

 K-Nearest Neighbors
 Learning Vector Quantization (LVQ)
 Self-Organizing Map (SOM)
 K-Means Clustering
There are many kernel-based methods may also be considered distance-based
algorithms. Perhaps the most widely known kernel method is the support vector
machine algorithm, or SVM for short.

K-NEAREST NEIGHBOR(KNN) ALGORITHM FOR MACHINE LEARNING

o K-Nearest Neighbour is one of the simplest Machine Learning algorithms

based on Supervised Learning technique.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o K-NN algorithm assumes the similarity between the new case/data and
available cases and put the new case into the category that is most similar to
the available categories.

o K-NN algorithm stores all the available data and classifies a new data point
based on the similarity. This means when new data appears then it can be
easily classified into a well suite category by using K- NN algorithm.

o K-NN algorithm can be used for Regression as well as for Classification but
mostly it is used for the Classification problems.

o K-NN is a non-parametric algorithm, which means it does not make any

assumption on underlying data.

o It is also called a lazy learner algorithm because it does not learn from the
training set immediately instead it stores the dataset and at the time of
classification, it performs an action on the dataset.

o KNN algorithm at the training phase just stores the dataset and when it gets
new data, then it classifies that data into a category that is much similar to
the new data.

o Example: Suppose, we have an image of a creature that looks similar to cat

and dog, but we want to know either it is a cat or dog. So for this
identification, we can use the KNN algorithm, as it works on a similarity
measure. Our KNN model will find the similar features of the new data set to
the cats and dogs images and based on the most similar features it will put it
in either cat or dog category.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 

Why do we need a K-NN Algorithm

Suppose there are two categories, i.e., Category A and Category B, and we have a
new data point x1, so this data point will lie in which of these categories. To solve
this type of problem, we need a K-NN algorithm. With the help of K-NN, we can
easily identify the category or class of a particular dataset. Consider the below
diagram:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
How does K-NN work?

The K-NN working can be explained on the basis of the below algorithm:

o Step-1: Select the number K of the neighbors

o Step-2: Calculate the Euclidean distance of K number of neighbors

o Step-3: Take the K nearest neighbors as per the calculated Euclidean

distance.

o Step-4: Among these k neighbors, count the number of the data points in
each category.

o Step-5: Assign the new data points to that category for which the number of
the neighbor is maximum.

o Step-6: Our model is ready.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Suppose we have a new data point and we need to put it in the required category.
Consider the below image:

o Firstly, we will choose the number of neighbors, so we will choose the k=5.

o Next, we will calculate the Euclidean distance between the data points. The
Euclidean distance is the distance between two points, which we have
already studied in geometry. It can be calculated as:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o By calculating the Euclidean distance we got the nearest neighbors, as three
nearest neighbors in category A and two nearest neighbors in category B.
Consider the below image:

o By calculating the Euclidean distance we got the nearest neighbors, as three

nearest neighbors in category A and two nearest neighbors in category B.
Consider the below image:

o As we can see the 3 nearest neighbors are from category A, hence this new
data point must belong to category A.

How to select the value of K in the K-NN Algorithm

Below are some points to remember while selecting the value of K in the K-NN
algorithm:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o There is no particular way to determine the best value for "K", so we need to
try some values to find the best out of them. The most preferred value for K
is 5.

o A very low value for K such as K=1 or K=2, can be noisy and lead to the effects
of outliers in the model.

o Large values for K are good, but it may find some difficulties.

Advantages of KNN Algorithm:

o It is simple to implement.

o It is robust to the noisy training data

o It can be more effective if the training data is large.

Disadvantages of KNN Algorithm:

o Always needs to determine the value of K which may be complex some time.

o The computation cost is high because of calculating the distance between

the data points for all the training samples

k-means algorithm:

K-Means Clustering is an Unsupervised Learning algorithm, which groups the

unlabeled dataset into different clusters. Here K defines the number of pre-defined
clusters that need to be created in the process, as if K=2, there will be two clusters,
and for K=3, there will be three clusters, and so on.

The k-means clustering algorithm mainly performs two tasks:

o Determines the best value for K center points or centroids by an iterative

process.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o Assigns each data point to its closest k-center. Those data points which are
near to the particular k-center, create a cluster.

Hence each cluster has datapoints with some commonalities, and it is away from
other clusters.

The below diagram explains the working of the K-means Clustering Algorithm:

K-Means Algorithm Work?

The working of the K-Means algorithm is explained in the below steps:

Step-1: Select the number K to decide the number of clusters.

Step-2: Select random K points or centroids. (It can be other from the input
dataset).

Step-3: Assign each data point to their closest centroid, which will form the
predefined K clusters.

Step-4: Calculate the variance and place a new centroid of each cluster.

Step-5: Repeat the third steps, which means reassign each datapoint to the new
closest centroid of each cluster.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.

Step-7: The model is ready.

Let's understand the above steps by considering the visual plots:

Suppose we have two variables M1 and M2. The x-y axis scatter plot of these two
variables is given below:

o Let's take number k of clusters, i.e., K=2, to identify the dataset and to put
them into different clusters. It means here we will try to group these datasets
into two different clusters.
o We need to choose some random k points or centroid to form the cluster.
These points can be either the points from the dataset or any other point.
So, here we are selecting the below two points as k points, which are not the

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 part of our dataset. Consider the below image:

o Now we will assign each data point of the scatter plot to its closest K-point
or centroid. We will compute it by applying some mathematics that we have
studied to calculate the distance between two points. So, we will draw a
median between both the centroids. Consider the below image:

From the above image, it is clear that points left side of the line is near to the K1 or
blue centroid, and points to the right of the line are close to the yellow centroid.
Let's color them as blue and yellow for clear visualization.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o As we need to find the closest cluster, so we will repeat the process by
choosing a new centroid. To choose the new centroids, we will compute the
center of gravity of these centroids, and will find new centroids as below:

o Next, we will reassign each datapoint to the new centroid. For this, we will
repeat the same process of finding a median line. The median will be like

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 below image:

From the above image, we can see, one yellow point is on the left side of the line,
and two blue points are right to the line. So, these three points will be assigned to
new centroids.

As reassignment has taken place, so we will again go to the step-4, which is finding
new centroids or K-points.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o We will repeat the process by finding the center of gravity of centroids, so
the new centroids will be as shown in the below image:

o As we got the new centroids so again will draw the median line and reassign
the data points. So, the image will be:

o We can see in the above image; there are no dissimilar data points on either
side of the line, which means our model is formed. Consider the below

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 image:

As our model is ready, so we can now remove the assumed centroids, and the two
final clusters will be as shown in the below image:

K-Medoids:
Medoid: A Medoid is a point in the cluster from which the sum of distances to other data
points is minimal.

(or)

A Medoid is a point in the cluster from which dissimilarities with all the other points in the
clusters are minimal.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Instead of centroids as reference points in K-Means algorithms, the K-Medoids algorithm
takes a Medoid as a reference point.

There are three types of algorithms for K-Medoids Clustering:

1. PAM (Partitioning Around Clustering)

2. CLARA (Clustering Large Applications)
3. CLARANS (Randomized Clustering Large Applications)

PAM is the most powerful algorithm of the three algorithms but has the disadvantage of
time complexity. The following K-Medoids are performed using PAM. In the further parts,
we'll see what CLARA and CLARANS are.

Algorithm:
Given the value of k and unlabelled data:

1. Choose k number of random points from the data and assign these k points to k number
of clusters. These are the initial medoids.
2. For all the remaining data points, calculate the distance from each medoid and assign it
to the cluster with the nearest medoid.
3. Calculate the total cost (Sum of all the distances from all the data points to the medoids)
4. Select a random point as the new medoid and swap it with the previous medoid. Repeat
2 and 3 steps.
5. If the total cost of the new medoid is less than that of the previous medoid, make the new
medoid permanent and repeat step 4.
6. If the total cost of the new medoid is greater than the cost of the previous medoid, undo
the swap and repeat step 4.
7. The Repetitions have to continue until no change is encountered with new medoids to
classify data points.

Advantages of using K-Medoids:

1. Deals with noise and outlier data effectively
2. Easily implementable and simple to understand
3. Faster compared to other partitioning algorithms

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Disadvantages:
1. Not suitable for Clustering arbitrarily shaped groups of data points.
2. As the initial medoids are chosen randomly, the results might vary based on the choice in
different runs.

K-Means and K-Medoids:

K-Means K-Medoids

Both methods are types of Partition Clustering.

Unsupervised iterative algorithms

Have to deal with unlabelled data

Both algorithms group n objects into k clusters based on similar traits where k is pre-defined.

Inputs: Unlabelled data and the value of k

Metric of similarity: Euclidian Distance Metric of similarity: Manhattan Distance

Clustering is done based on distance from centroids. Clustering is done based on distance
from medoids.

A centroid can be a data point or some other point in A medoid is always a data point in the cluster.
the cluster

Can't cope with outlier data Can manage outlier data too

Sometimes, outlier sensitivity can turn out to be useful Tendency to ignore meaningful clusters in
outlier data

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Hierarchical Clustering

Hierarchical clustering can be used as an alternative for the partitioned clustering

as there is no requirement of pre-specifying the number of clusters to be created.
In this technique, the dataset is divided into clusters to create a tree-like structure,
which is also called a dendrogram. The observations or any number of clusters can
be selected by cutting the tree at the correct level. The most common example of
this method is the Agglomerative Hierarchical algorithm.

What Is Ensemble?

The ensemble methods in machine learning combine the insights obtained from
multiple learning models to facilitate accurate and improved decisions. These methods
follow the same principle as the example of buying an air-conditioner cited above.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
In learning models, noise, variance, and bias are the major sources of error. The
ensemble methods in machine learning help minimize these error-causing factors,
thereby ensuring the accuracy and stability of machine learning (ML) algorithms.

BAGGING VS BOOSTING

We all use the Decision Tree Technique on day to day life to make the decision.
Organizations use these supervised machine learning techniques like Decision trees
to make a better decision and to generate more surplus and profit.

Ensemble methods combine different decision trees to deliver better predictive

results, afterward utilizing a single decision tree. The primary principle behind the
ensemble model is that a group of weak learners come together to form an active
learner.

There are two techniques given below that are used to perform ensemble decision
tree.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
BAGGING

Bagging is used when our objective is to reduce the variance of a decision tree. Here
the concept is to create a few subsets of data from the training sample, which is
chosen randomly with replacement. Now each collection of subset data is used to
prepare their decision trees thus, we end up with an ensemble of various models.
The average of all the assumptions from numerous tress is used, which is more
powerful than a single decision tree.

Random Forest is an expansion over bagging. It takes one additional step to predict
a random subset of data. It also makes the random selection of features rather than
using all features to develop trees. When we have numerous random trees, it is
called the Random Forest.

These are the following steps which are taken to implement a Random forest:

o Let us consider X observations Y features in the training data set. First, a

model from the training data set is taken randomly with substitution.

o The tree is developed to the largest.

o The given steps are repeated, and prediction is given, which is based on the
collection of predictions from n number of trees.

Advantages of using Random Forest technique:

o It manages a higher dimension data set very well.

o It manages missing quantities and keeps accuracy for missing data.

Disadvantages of using Random Forest technique:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Since the last prediction depends on the mean predictions from subset trees, it
won't give precise value for the regression model.

BOOSTING:

Boosting is another ensemble procedure to make a collection of predictors. In other

words, we fit consecutive trees, usually random samples, and at each step, the
objective is to solve net error from the prior trees.

If a given input is misclassifieyed by theory, then its weight is increased so that the
upcoming hypothesis is more likely to classify it correctly by consolidating the
entire set at last converts weak learners into better performing models.

Gradient Boosting is an expansion of the boosting procedure.

1. Gradient Boosting = Gradient Descent + Boosting

It utilizes a gradient descent algorithm that can optimize any differentiable loss
function. An ensemble of trees is constructed individually, and individual trees are
summed successively. The next tree tries to restore the loss ( It is the difference
between actual and predicted values).

Advantages of using Gradient Boosting methods:

o It supports different loss functions.

o It works well with interactions.

Disadvantages of using a Gradient Boosting methods:

o It requires cautious tuning of different hyper-parameters.

Dept of CSE, ASGI B. Uma Maheswari
Assistant Professor
RANDOM FOREST ALGORITHM

Random Forest is a popular machine learning algorithm that belongs to the

supervised learning technique. It can be used for both Classification and Regression
problems in ML. It is based on the concept of ensemble learning, which is a process
of combining multiple classifiers to solve a complex problem and to improve the
performance of the model.

As the name suggests, "Random Forest is a classifier that contains a number of

decision trees on various subsets of the given dataset and takes the average to
improve the predictive accuracy of that dataset." Instead of relying on one
decision tree, the random forest takes the prediction from each tree and based on
the majority votes of predictions, and it predicts the final output.

The greater number of trees in the forest leads to higher accuracy and prevents
the problem of overfitting.

The below diagram explains the working of the Random Forest algorithm:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Assumptions for Random Forest

Since the random forest combines multiple trees to predict the class of the dataset,
it is possible that some decision trees may predict the correct output, while others
may not. But together, all the trees predict the correct output. Therefore, below
are two assumptions for a better Random forest classifier:

o There should be some actual values in the feature variable of the dataset so
that the classifier can predict accurate results rather than a guessed result.

o The predictions from each tree must have very low correlations.

Why use Random Forest

Below are some points that explain why we should use the Random Forest
algorithm:

<="" li="">

o It takes less training time as compared to other algorithms.

o It predicts output with high accuracy, even for the large dataset it runs
efficiently.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o It can also maintain accuracy when a large proportion of data is missing

Why use Random Forest

Below are some points that explain why we should use the Random Forest
algorithm:

<="" li="">

o It takes less training time as compared to other algorithms.

o It predicts output with high accuracy, even for the large dataset it runs
efficiently.

o It can also maintain accuracy when a large proportion of data is missing.

How does Random Forest algorithm work

Random Forest works in two-phase first is to create the random forest by

combining N decision tree, and second is to make predictions for each tree created
in the first phase.

The Working process can be explained in the below steps and diagram:

Step-1: Select random K data points from the training set.

Step-2: Build the decision trees associated with the selected data points (Subsets).

Step-3: Choose the number N for decision trees that you want to build.

Step-4: Repeat Step 1 & 2.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Step-5: For new data points, find the predictions of each decision tree, and assign
the new data points to the category that wins the majority votes.

The working of the algorithm can be better understood by the below example:

Example: Suppose there is a dataset that contains multiple fruit images. So, this
dataset is given to the Random forest classifier. The dataset is divided into subsets
and given to each decision tree. During the training phase, each decision tree
produces a prediction result, and when a new data point occurs, then based on the
majority of results, the Random Forest classifier predicts the final decision.
Consider the below image:

Applications of Random Forest

There are mainly four sectors where Random forest mostly used:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 1. Banking: Banking sector mostly uses this algorithm for the identification of
loan risk.

2. Medicine: With the help of this algorithm, disease trends and risks of the
disease can be identified.

3. Land Use: We can identify the areas of similar land use by this algorithm.

4. Marketing: Marketing trends can be identified using this algorithm.

Advantages of Random Forest

o Random Forest is capable of performing both Classification and Regression

tasks.

o It is capable of handling large datasets with high dimensionality.

o It enhances the accuracy of the model and prevents the overfitting issue.

Disadvantages of Random Forest

o Although random forest can be used for both classification and regression
tasks, it is not more suitable for Regression tasks.

DEEP LEARNING TUTORIAL

Deep learning is based on the branch of machine learning, which is a subset of

artificial intelligence. Since neural networks imitate the human brain and so deep
learning will do. In deep learning, nothing is programmed explicitly. Basically, it is a
machine learning class that makes use of numerous nonlinear processing units so
as to perform feature extraction as well as transformation. The output from each
preceding layer is taken as input by each one of the successive layers.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Deep learning models are capable enough to focus on the accurate features
themselves by requiring a little guidance from the programmer and are very helpful
in solving out the problem of dimensionality. Deep learning algorithms are used,
especially when we have a huge no of inputs and outputs.

Since deep learning has been evolved by the machine learning, which itself is a
subset of artificial intelligence and as the idea behind the artificial intelligence is to
mimic the human behavior, so same is "the idea of deep learning to build such
algorithm that can mimic the brain".

Deep learning is implemented with the help of Neural Networks, and the idea
behind the motivation of Neural Network is the biological neurons, which is
nothing but a brain cell.

Deep learning is a collection of statistical techniques of machine learning for

learning feature hierarchies that are actually based on artificial neural networks.

So basically, deep learning is implemented by the help of deep networks, which are
nothing but neural networks with multiple hidden layers.

Example of Deep Learning

In the example given above, we provide the raw data of images to the first layer of
the input layer. After then, these input layer will determine the patterns of local
Dept of CSE, ASGI B. Uma Maheswari
Assistant Professor
contrast that means it will differentiate on the basis of colors, luminosity, etc. Then
the 1st hidden layer will determine the face feature, i.e., it will fixate on eyes, nose,
and lips, etc. And then, it will fixate those face features on the correct face
template. So, in the 2nd hidden layer, it will actually determine the correct face here
as it can be seen in the above image, after which it will be sent to the output layer.
Likewise, more hidden layers can be added to solve more complex problems, for
example, if you want to find out a particular kind of face having large or light
complexions. So, as and when the hidden layers increase, we are able to solve
complex problems.

Architectures

o DeepNeuralNetworks
It is a neural network that incorporates the complexity of a certain level,
which means several numbers of hidden layers are encompassed in between
the input and output layers. They are highly proficient on model and process
non-linear associations.

o DeepBeliefNetworks
A deep belief network is a class of Deep Neural Network that comprises of
multi-layer belief networks.
Steps to perform DBN:

1. With the help of the Contrastive Divergence algorithm, a layer of

features is learned from perceptible units.

2. Next, the formerly trained features are treated as visible units, which
perform learning of features.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 3. Lastly, when the learning of the final hidden layer is accomplished,
then the whole DBN is trained.

o RecurrentNeuralNetworks
It permits parallel as well as sequential computation, and it is exactly similar
to that of the human brain (large feedback network of connected neurons).
Since they are capable enough to reminisce all of the imperative things
related to the input they have received, so they are more precise.

DEEP LEARNING APPLICATIONS

o Self-DrivingCars
In self-driven cars, it is able to capture the images around it by processing a
huge amount of data, and then it will decide which actions should be
incorporated to take a left or right or should it stop. So, accordingly, it will
decide what actions it should take, which will further reduce the accidents
that happen every year.

o VoiceControlledAssistance
When we talk about voice control assistance, then Siri is the one thing that
comes into our mind. So, you can tell Siri whatever you want it to do it for
you, and it will search it for you and display it for you.

o AutomaticImageCaptionGeneration
Whatever image that you upload, the algorithm will work in such a way that
it will generate caption accordingly. If you say blue colored eye, it will display
a blue-colored eye with a caption at the bottom of the image.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o AutomaticMachineTranslation
With the help of automatic machine translation, we are able to convert one
language into another with the help of deep learning.

Limitations

o It only learns through the observations.

o It comprises of biases issues.

Advantages

o It lessens the need for feature engineering.

o It eradicates all those costs that are needless.

o It easily identifies difficult defects.

o It results in the best-in-class performance on problems.

Disadvantages

o It requires an ample amount of data.

o It is quite expensive to train.

o It does not have strong theoretical groundwork.

What is Reinforcement Learning

o Reinforcement Learning is a feedback-based Machine learning technique in

which an agent learns to behave in an environment by performing the
actions and seeing the results of actions. For each good action, the agent gets
positive feedback, and for each bad action, the agent gets negative feedback
or penalty.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o In Reinforcement Learning, the agent learns automatically using feedbacks
without any labeled data, unlike supervised learning.

o Since there is no labeled data, so the agent is bound to learn by its experience
only.

o RL solves a specific type of problem where decision making is sequential, and

the goal is long-term, such as game-playing, robotics, etc.

o The agent interacts with the environment and explores it by itself. The
primary goal of an agent in reinforcement learning is to improve the
performance by getting the maximum positive rewards.

o The agent learns with the process of hit and trial, and based on the
experience, it learns to perform the task in a better way. Hence, we can say
that "Reinforcement learning is a type of machine learning method where
an intelligent agent (computer program) interacts with the environment
and learns to act within that." How a Robotic dog learns the movement of
his arms is an example of Reinforcement learning.

o It is a core part of Artificial intelligence, and all AI agent works on the concept
of reinforcement learning. Here we do not need to pre-program the agent,
as it learns from its own experience without any human intervention.

o Example: Suppose there is an AI agent present within a maze environment,

and his goal is to find the diamond. The agent interacts with the environment
by performing some actions, and based on those actions, the state of the
agent gets changed, and it also receives a reward or penalty as feedback.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o The agent continues doing these three things (take action, change
state/remain in the same state, and get feedback), and by doing these
actions, he learns and explores the environment.

o The agent learns that what actions lead to positive feedback or rewards and
what actions lead to negative feedback penalty. As a positive reward, the
agent gets a positive point, and as a penalty, it gets a negative point.

Terms used in Reinforcement Learning

o Agent(): An entity that can perceive/explore the environment and act upon
it.

o Environment(): A situation in which an agent is present or surrounded by. In

RL, we assume the stochastic environment, which means it is random in
nature.
Dept of CSE, ASGI B. Uma Maheswari
Assistant Professor
 o Action(): Actions are the moves taken by an agent within the environment.

o State(): State is a situation returned by the environment after each action

taken by the agent.

o Reward(): A feedback returned to the agent from the environment to

evaluate the action of the agent.

o Policy(): Policy is a strategy applied by the agent for the next action based on
the current state.

o Value(): It is expected long-term retuned with the discount factor and

opposite to the short-term reward.

o Q-value(): It is mostly similar to the value, but it takes one additional

parameter as a current action (a).

Key Features of Reinforcement Learning

o In RL, the agent is not instructed about the environment and what actions
need to be taken.

o It is based on the hit and trial process.

o The agent takes the next action and changes states according to the feedback
of the previous action.

o The agent may get a delayed reward.

o The environment is stochastic, and the agent needs to explore it to reach to

get the maximum positive rewards.

Types of Reinforcement learning

There are mainly two types of reinforcement learning, which are:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o Positive Reinforcement

o Negative Reinforcement

Positive Reinforcement:

The positive reinforcement learning means adding something to increase the

tendency that expected behavior would occur again. It impacts positively on the
behavior of the agent and increases the strength of the behavior.

This type of reinforcement can sustain the changes for a long time, but too much
positive reinforcement may lead to an overload of states that can reduce the
consequences.

Negative Reinforcement:

The negative reinforcement learning is opposite to the positive reinforcement as it

increases the tendency that the specific behavior will occur again by avoiding the
negative condition.

It can be more effective than the positive reinforcement depending on situation

and behavior, but it provides reinforcement only to meet minimum behavior.

Reinforcement Learning Algorithms

Reinforcement learning algorithms are mainly used in AI applications and gaming

applications. The main used algorithms are:

o Q-Learning:

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o Q-learning is an Off policy RL algorithm, which is used for the
temporal difference Learning. The temporal difference learning
methods are the way of comparing temporally successive predictions.

o It learns the value function Q (S, a), which means how good to take
action "a" at a particular state "s."

o The below flowchart explains the working of Q- learning:

o State Action Reward State action (SARSA):

o SARSA stands for State Action Reward State action, which is an on-
policy temporal difference learning method. The on-policy control
method selects the action for each state while learning using a specific
policy.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
 o The goal of SARSA is to calculate the Q π (s, a) for the selected current
policy π and all pairs of (s-a).

o The main difference between Q-learning and SARSA algorithms is

that unlike Q-learning, the maximum reward for the next state is not
required for updating the Q-value in the table.

o In SARSA, new action and reward are selected using the same policy,
which has determined the original action.

o The SARSA is named because it uses the quintuple Q(s, a, r, s',

a'). Where,
s: original state
a: Original action
r: reward observed while following the states
s' and a': New state, action pair.

o Deep Q Neural Network (DQN):

o As the name suggests, DQN is a Q-learning using Neural networks.

o For a big state space environment, it will be a challenging and complex

task to define and update a Q-table.

o To solve such an issue, we can use a DQN algorithm. Where, instead

of defining a Q-table, neural network approximates the Q-values for
each action and state.

Dept of CSE, ASGI B. Uma Maheswari

Assistant Professor
Dept of CSE, ASGI B. Uma Maheswari
Assistant Professor