GMR Institute of Technology

Rajam, Andhra Pradesh

(An Autonomous Institution Affiliated to JNTU-GV, AP)

Department of Computer Science and Engineering

Class 6th Sem. - B. Tech. (CSE)
Fundamentals of Machine
Course Course Code 21CS002
Learning
Prepared by Dr. N Lakshmi Devi, Assistant Professor, CSE-AI&ML
Back Propagation Algorithm, Applications, Introduction to Deep
Lecture Topic Learning, Convolutional Neural Networks (CNN/ConvNets), CNN
Applications
Course Outcome CO5&6 Program Outcome PO2, PO5
Duration 360 min Lecture 43 to 48 Unit III
Learning Level REMEMBER UNDERSTAND APPLY ANALYSE EVALUATE CREATE

(Tick whichever is applicable) √ √ √

1. Objectives
To understand Neural Networks and its architecture.
2. Intended Learning Outcomes (ILO)
After the completion of the class the students will able to
A. Back Propagation Algorithm and its Applications.
B. Understand Introduction to Deep Learning
C. Understand Convolutional Neural Networks (CNN/ConvNets),.
3. 2D Mapping of ILOs with Knowledge Dimension and Cognitive Learning Levels of
RBT

Cognitive Learning Levels (2D)

Knowledge
Remember Understand Apply Analyze Evaluate Create
Dimension (1D)
Factual A
Conceptual A,C
Procedural B
Meta Cognitive

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 1
4. Teaching Methodology
Chalk & Talk, Visual Presentation and Demonstration

5. Evocation

Fig. 1: Evocation

6. Deliverables
Lecture-1,2
A Multilayer Feed-Forward Neural Network

The back propagation algorithm performs learning on a multilayer feed-forward neural network.
It iteratively learns a set of weights for prediction of the class label of tuples. A multilayer feed-
forward neural network consists of an input layer, one or more hidden layers, and an output
layer. An example of a multilayer feed-forward network is shown in Figure.

Figure: Multilayer feed-forward neural network.

Each layer is made up of units. The inputs to the network correspond to the attributes measured

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 2
for each training tuple. The inputs are fed simultaneously into the units making up the input
layer. These inputs pass through the input layer and are then weighted and fed simultaneously to
a second layer of “neuronlike” units, known as a hidden layer. The outputs of the hidden layer
units can be input to another hidden layer, and so on. The number of hidden layers is arbitrary,
although in practice, usually only one is used. The weighted outputs of the last hidden layer are
input to units making up the output layer, which emits the network's prediction for given tuples.

The units in the input layer are called input units. The units in the hidden layers and output layer
are sometimes referred to as neurodes, due to their symbolic biological basis, or as output units.
The multilayer neural network shown in has two layers of output units. Therefore, we say that it
is a two-layer neural network. (The input layer is not counted because it serves only to pass the
input values to the next layer.) Similarly, a network containing two hidden layers is called a
three-layer neural network, and so on. It is a feed-forward network since none of the weights
cycles back to an input unit or to a previous layer's output unit. It is fully connected in that each
unit provides input to each unit in the next forward layer.

Each output unit takes, as input, a weighted sum of the outputs from units in the previous layer It
applies a nonlinear (activation) function to the weighted input. Multilayer feed-forward neural
networks are able to model the class prediction as a nonlinear combination of the inputs. From a
statistical point of view, they perform nonlinear regression. Multilayer feed-forward networks,
given enough hidden units and enough training samples, can closely approximate any function.

Applications:

Application #1: Classification

In this classification problem, the goal is to identify whether a certain "data point" belongs to
Class 1, 2, or 3. Random points are assigned to a certain class, and the neural network is trained
to find the pattern. When training is complete, it will use what it has learned to accurately
classify new points.

Here, the network was able to distinguish group 1

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 3
(red) from group 2 (blue).

Here, the system was trained to identify three

groups.

Application #2: Function Approximation

In this problem, the network tries to approximate the value of a certain function. It is fed with
noisy data, and the goal is to find the true pattern. After training, the network successfully
estimates the value of the gaussian function (below).

Estimating the Gaussian function

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 4
Application #3: Time-series Prediction

In this problem, the goal is to design a neural network to predict a value based on a given time-
series data (i.e. stock market prediction based on given trends). To approach this problem, the
inputs to the neural network have to be refactored in chunks, and the resulting output will be the
next data item directly following that chunk (see below)

The input space and output space for the time-series prediction problem.

In this specific problem, goal was to predict time-series data based on a sine wave. After training
the network, lo' and behold system was able to accurately predict 1000 data points for the sine
wave. The results are seen below.

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 5
1000 sine wave predictions for the trained network.

Lecture-3,4

Deep Learning is a subset of machine learning in artificial intelligence (AI) that focuses on
neural networks with many layers. It aims to model complex patterns in data by learning
hierarchical representations. Deep learning has enabled significant advancements in various
fields such as computer vision, natural language processing, and speech recognition.

Key Concepts in Deep Learning:

1. Neural Networks:
o A neural network is a computational model inspired by the way biological neural
networks in the human brain process information.
o Basic components include neurons (nodes), layers (input, hidden, output),
weights, biases, and activation functions.
2. Deep Neural Networks (DNNs):
o These are neural networks with multiple hidden layers between the input and
output layers.
o The "depth" of the network refers to the number of layers.
o More layers allow the network to learn more complex features.
3. Training:
o Training a deep neural network involves adjusting the weights and biases of the
neurons to minimize the difference between the predicted and actual output.
o Common algorithms include backpropagation and gradient descent.
o The process requires a large amount of labeled data and significant computational
power.
4. Backpropagation:
o A key algorithm for training neural networks.
o It calculates the gradient of the loss function with respect to each weight by the
chain rule, allowing for efficient computation of gradients.
5. Activation Functions:
o Introduce non-linearity into the network, enabling it to model complex patterns.
o Common functions include ReLU, sigmoid, and tanh.
6. Loss Function:
o A function that measures the difference between the predicted output and the
actual output.
o Common loss functions include Mean Squared Error (MSE) for regression tasks
and Cross-Entropy Loss for classification tasks.
7. Optimization:
o Optimization algorithms adjust the weights to minimize the loss function.
o Common optimizers include Stochastic Gradient Descent (SGD), Adam, and
RMSprop.
8. Regularization:
o Techniques to prevent overfitting (when the model performs well on training data

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 6
 but poorly on unseen data).
o Methods include dropout, L2 regularization, and early stopping.
9. Architectures:
o Convolutional Neural Networks (CNNs): Primarily used for image data. They
use convolutional layers to detect spatial hierarchies.
o Recurrent Neural Networks (RNNs): Suitable for sequential data. They use
loops to carry information across time steps.
o Long Short-Term Memory Networks (LSTMs): A type of RNN that addresses
the vanishing gradient problem and is effective for long-term dependencies.
o Generative Adversarial Networks (GANs): Consist of a generator and a
discriminator network that compete with each other, often used for generating
synthetic data.
10. Applications:
o Computer Vision: Object detection, image classification, facial recognition, and
more.
o Natural Language Processing (NLP): Language translation, sentiment analysis,
text generation, and more.
o Speech Recognition: Converting spoken language into text.
o Autonomous Vehicles: Perception and decision-making for self-driving cars.
o Healthcare: Disease diagnosis, medical image analysis, and personalized
treatment plans.

Lecture-5,6
It is used to predict the output with the given input image. The image values are divided into pixels.
The values of the pixels are in between 0-255. Steps involved in CNN:

 Convolution
 Max pooling
 Flattening
 Full connection

Convolution

In this step feature detection takes place. Feature detectors like Sharpen, Blur, Edge Enhance,
Edge Detect, and Emboss are used to convert the pixel values into binary values. After applying
feature detectors we get many number of pooled feature maps. We use activation function called
as RLU (Rectified Linear Unit) to increase the non-linearity in our images.

Max Pooling

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 7
Max pooling is a pooling operation that selects the maximum element from the region of the
feature map covered by the filter. Thus, the output after max-pooling layer would be a feature
map containing the most prominent features of the previous feature map. As we get many number
of pooled feature maps it is difficult for computation hence we use max pooling for computation.

Fig Example of Max Pooling

Flattening

To insert all the pooled feature maps into ANN they should be in vectors i.e. 0/1 matrix form we
do this process in a flattened layer, so it is called as flattening. Hence all the pooled feature maps
come into a single vector.

In Keras, this is a typical process for building CNN architecture:

 Reshape the input data into a format suitable for the convolutional layers, using
X_train.reshape() and X_test.reshape()
 For class-based classification, one-hot encode the categories using to_categorical()
 Build the model using the [Link]() function.
 Add a convolutional layer, for example using [Link](Conv2D(…)) – see our
in-depth guide to Keras Conv2D layers.
 Add a pooling layer, for example using the [Link](MaxPooling2D(…)) function
 Add a “flatten” layer which prepares a vector for the fully connected layers, for

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 8
 example using [Link](Flatten())

 Add one or more fully connected layer using [Link](Dense)), and if necessary
a dropout layer.
 Compile the model using [Link]()
 Train the model using [Link](), supplying X_train(), X_test(), y_train() and y_test()
 Use [Link]() to generate a prediction.

In between the convolutional layer and the fully connected layer, there is a „Flatten‟ layer.
Flattening transforms a two-dimensional matrix of features into a vector that can be fed into a
fully connected neural network classifier.

Fig Image displaying Flattening process

Full Connection

At the end of a CNN, the output of the last Pooling Layer acts as input to the so called Fully
Connected Layer. There can be one or more of these layers (“fully connected” means that every
node in the first layer is connected to every node in the second layer). Fully Connected layers
perform classification based on the features extracted by the previous layers. Typically, this layer
is a traditional ANN containing a softmax activation function, which outputs a probability (a
number ranging from 0-1) for each of the classification labels the model is trying to predict.

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 9
 Fig End-to-End structure of a simple CNN
Lecture-6

Applications of Convolutional Neural Networks (CNNs):

1. Image Classification

 Description: Identifying and categorizing objects within an image.

 Example: Classifying images of animals into categories such as dogs, cats, and birds.

2. Object Detection

 Description: Identifying and localizing objects within an image.

 Example: Detecting pedestrians and vehicles in self-driving car systems.

3. Segmentation

 Description: Partitioning an image into distinct regions or segments, often for different
objects or parts.
 Example: Segmenting medical images to identify tumors or other abnormalities.

4. Facial Recognition

 Description: Identifying or verifying a person from a digital image or a video frame.

 Example: Security systems using facial recognition for access control.

5. Image Generation

 Description: Creating new images from a dataset of existing images.

 Example: Generative Adversarial Networks (GANs) creating realistic images of faces

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 10
 that do not exist.

6. Video Analysis

 Description: Processing and analyzing video data for various tasks.

 Example: Activity recognition in surveillance footage to detect suspicious behavior.

7. Style Transfer

 Description: Applying the style of one image to the content of another.

 Example: Transforming a photo into the style of a famous painting.

8. Medical Imaging

 Description: Analyzing medical images for diagnostic purposes.

 Example: Detecting anomalies in X-rays, MRIs, or CT scans.

9. Autonomous Vehicles

 Description: Processing visual information to aid in navigation and decision-making.

 Example: Lane detection, traffic sign recognition, and obstacle avoidance.

10. Text Recognition

 Description: Converting images of text into machine-readable text.

 Example: Optical Character Recognition (OCR) systems used for digitizing printed
documents.

11. Aerial and Satellite Image Analysis

 Description: Analyzing images taken from aerial or satellite platforms.

 Example: Monitoring agricultural fields, urban planning, and environmental changes.

12. Art and Creative Applications

 Description: Enhancing or transforming images for artistic purposes.

 Example: Creating digital art, filters for social media, or enhancing photographs.

13. Recommender Systems

 Description: Recommending items based on visual content.

 Example: Suggesting clothing items based on visual similarities in fashion retail.

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 11
These applications leverage the ability of CNNs to automatically and hierarchically
learn spatial hierarchies of features, making them powerful tools for various tasks
involving visual data.

7. Key Words

 Feed forward network, Convolution, Max pooling, Flattening, Max pooling.

8. Questions
Remember
1. List any two applications of ANN.
2. List any two applications of CNN.
3. Define CNN
4. Explain the architecture of CNN

Understanding

5. Explain the architecture of Back Propagation Network.

6. Identify the number of layers in max-pooling.
7. Explain the architecture of Neurons and Network of Neurons
Analyze

1. Compare and contrast any two applications of Back Propagation algorithms..

2. Compare and contrast any two classical training algorithms for neural network
training.
9. Stimulating Question
Explore the latest articles, projects, and questions and answers in Convolution
Neural Network.
10. Mind Map

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 12
11. Student Summary
At the end of this session, the facilitator (Teacher) shall randomly pic-up few students
to summarize the deliverables

12. Reading Materials

1. Stephen Marsland, "Machine Learning - An Algorithmic Perspective ", CRC Press,
2009.
2. Sunila Gollapudi, “Practical Machine Learning” First Edition 2016, Packt Publishing
Ltd.

13. Scope for Mini Project

Yes. Student can work on different supervised learning algorithms for various
real time applications.

Form No. AC 04. 00. 2016 – GMRIT, Rajam, Andhra Pradesh GMRIT 13