UNIT–V: CONVOLUTIONAL NEURAL NETWORKS

Convolutional Neural Networks (CNNs) are a class of deep neural networks

designed to process and analyze structured grid-like data, such as images. They
are especially effective for tasks involving spatial hierarchies, such as image
recognition and classification, and have been adapted for various applications
including natural language processing and time series analysis.
Key Components of CNNs
1. Convolutional Layers:
o Function: Apply convolutional operations to input data using filters
(or kernels). Each filter convolves over the input image to detect
features such as edges, textures, or patterns.
o Output: Generates feature maps (or activation maps) that highlight
the presence of these features in different regions of the image.
2. Activation Function:
o Function: Introduces non-linearity into the model, allowing it to
learn complex patterns. Typically, a non-linear activation function
like ReLU (Rectified Linear Unit) is used.
3. Pooling Layers:
o Function: Reduce the spatial dimensions of feature maps, which
decreases the computational load and helps control overfitting.
Max pooling is a common operation, selecting the maximum value
from a defined window.
o Example: Max pooling with a 2x2 window reduces the size of the
feature map by a factor of 2.
4. Fully Connected Layers:

Dr. G. Vennila, Assistant Professor, AIML 78 | P a g e

 o Function: After convolutional and pooling layers, high-level
features are flattened into a vector and passed through fully
connected (dense) layers for final predictions or classifications.
o Output: For classification tasks, the output is often processed
through a softmax activation function to produce class
probabilities.
5. Normalization Layers (Optional):
o Function: Layers like Batch Normalization or Layer Normalization
stabilize and accelerate training by normalizing the activations
from previous layers.
6. Dropout Layers (Optional):
o Function: A regularization technique that randomly sets a fraction
of neurons to zero during training to prevent overfitting.
How CNNs Work
1. Feature Extraction:
o Convolutional Layers: Extract features from the input image by
applying various filters.
o Activation Layers: Apply non-linear activation functions to capture
complex features.
o Pooling Layers: Reduce dimensionality of feature maps while
retaining important information.
2. Classification:
o Flattening: Convert feature maps into a single vector.
o Fully Connected Layers: Analyze the features and make the final
classification or regression predictions based on learned patterns.
CNN Architecture Example
1. Input Layer: An image of size 224x224x3 (height x width x channels).

Dr. G. Vennila, Assistant Professor, AIML 79 | P a g e

 2. Convolutional Layer: Apply 32 filters of size 3x3 to produce a feature
map.
3. Activation Layer: Apply ReLU activation to the feature map.
4. Pooling Layer: Apply max pooling with a 2x2 window to reduce spatial
dimensions.
5. Convolutional Layer: Apply 64 filters of size 3x3, followed by ReLU
activation and max pooling.
6. Flattening: Convert the 2D feature maps into a 1D vector.
7. Fully Connected Layer: Connect the flattened vector to a dense layer with
128 units.
8. Output Layer: Use a softmax function to produce class probabilities.
Applications of CNNs
1. Image Classification: Identifying categories of objects in images (e.g.,
distinguishing between cats and dogs).
2. Object Detection: Locating and classifying objects within an image (e.g.,
detecting faces in photos).
3. Semantic Segmentation: Classifying each pixel in an image into
predefined categories (e.g., segmenting different objects in an image).
4. Face Recognition: Identifying and verifying individuals based on facial
features.
5. Medical Image Analysis: Detecting abnormalities in medical scans (e.g.,
identifying tumors in MRI images).
Advantages of CNNs
• Automatic Feature Extraction: Automatically learns and extracts relevant
features from images, reducing the need for manual feature engineering.
• Spatial Hierarchies: Captures spatial hierarchies and relationships within
images effectively.

Dr. G. Vennila, Assistant Professor, AIML 80 | P a g e

 • Translation Invariance: Achieves invariance to the location of features by
using pooling layers, allowing recognition of objects in various positions.
Challenges
• Computational Complexity: Training CNNs requires significant
computational resources, particularly for deep networks.
• Data Requirements: Requires large amounts of labeled data to achieve
high performance.
• Overfitting: Deep CNNs can overfit to training data if not properly
regularized or if the training data is insufficient.
CNN Architecture
A Convolutional Neural Network (CNN) typically consists of an input layer,
several convolutional layers, pooling layers, and fully connected layers. This
architecture allows CNNs to effectively process and analyze structured data,
especially images.

Simple CNN architecture

How Convolutional Layers Work

Convolutional Layers are a key component of Convolutional Neural Networks

(CNNs) and are tailored to handle grid-like data, such as images. These layers
are responsible for extracting features by applying filters (or kernels) to the input
data. Here’s a breakdown of their operation:
Dr. G. Vennila, Assistant Professor, AIML 81 | P a g e
1. Concept of Convolution

Convolution in neural networks involves applying a filter to the input data to

create a feature map. This process entails sliding the filter over the input and
computing the dot product between the filter and the corresponding segment of
the input data at each position.

2. Convolution Operation

1. Filter (Kernel):

• Definition: A filter is a small matrix of weights designed to detect specific

features in the input data, such as edges, textures, or patterns.

• Size: Filters are generally small compared to the input image, commonly
being 3x3 or 5x5 matrices.

2. Stride:

• Definition: The stride is the number of pixels by which the filter moves
across the input image during the convolution process.

• Impact: A larger stride results in a smaller output feature map, while a

smaller stride maintains a higher spatial resolution in the output.

3. Padding:

• Definition: Padding involves adding extra pixels around the border of the
input image to control the dimensions of the output feature map.

• Types: Common padding methods include 'valid' (no padding) and 'same'
(padding to ensure the output size matches the input size).

3. Convolution Process

1. Apply Filter:

Dr. G. Vennila, Assistant Professor, AIML 82 | P a g e

 • Operation: Place the filter on the input image, compute the dot product
between the filter’s weights and the corresponding input pixels, and sum
the results.

• Result: This produces a single value in the resulting feature map.

2. Slide Filter:

• Operation: Move the filter across the input image according to the stride
value, performing the convolution operation at each position.

• Result: This creates a 2D matrix (feature map) that highlights where

specific features are present in different regions of the input image.

4. Output Feature Map

1. Feature Map:

• Definition: A 2D matrix representing the features detected by the filter

throughout the input image.

• Purpose: Displays the locations and presence of specific features, such as

edges or textures.

2. Multiple Filters:

• Operation: Multiple filters are applied within a convolutional layer, each

detecting different features. Each filter generates its own feature map.

• Result: The feature maps from all filters are stacked together, producing
a multi-channel output that captures a comprehensive set of features from
the input data.

Dr. G. Vennila, Assistant Professor, AIML 83 | P a g e

Image source: Deep Learning Udacity

Convolutional Layers: How They Work

Convolutional Layers are integral to Convolutional Neural Networks (CNNs) and are
designed to handle grid-like data, such as images. They perform feature extraction by
applying filters (or kernels) to the input data. Here’s a step-by-step explanation of how
convolutional layers function:
1. Concept of Convolution
Convolution in neural networks involves applying a filter to an input matrix to produce a
feature map. This process includes sliding the filter over the input data and calculating the
dot product between the filter and the section of the input it currently covers.
• Input Data: For example, an image with dimensions 5x5.
• Sliding Window: The filter moves across the input data. For a stride of 1 and no
padding, the filter moves one pixel at a time.
• Dot Product: At each position, the filter’s values are multiplied element-wise with the
corresponding values in the input data. The results are summed to produce a single
value in the output feature map.
• Feature Map: The resulting matrix highlights the features detected by the filter.
2. Steps in a Convolutional Layer

Dr. G. Vennila, Assistant Professor, AIML 84 | P a g e

1. Apply Filter:
• Initialize: Start with an input matrix (e.g., an image) and a filter (kernel).
• Compute Convolution: Slide the filter across the input matrix. At each position,
compute the dot product between the filter and the overlapping section of the input
matrix.
• Output: Store each result in a new matrix, known as the feature map.
2. Activation Function:
• Apply Non-linearity: After computing the convolution, an activation function like
ReLU (Rectified Linear Unit) is applied. This introduces non-linearity and allows the
network to learn more complex patterns.
• ReLU Formula: ReLU(x) = max(0, x)
3. Pooling (Optional):
• Reduce Dimensions: Pooling layers, such as max pooling, are often used after
convolution to reduce the spatial dimensions of the feature map while retaining the
most significant features.

Image source: [Link]

Dr. G. Vennila, Assistant Professor, AIML 85 | P a g e

Image source: [Link]

Convolutional Neural Network Architectures: AlexNet, VGG, GoogleNet,

and ResNet
Several pivotal Convolutional Neural Network (CNN) architectures have shaped
the field of computer vision, each contributing unique innovations. Here’s an
overview of four influential architectures: AlexNet, VGG, GoogleNet, and
ResNet.
1. AlexNet
Introduced: Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, 2012
Achievements: Won the ImageNet Large Scale Visual Recognition Challenge
(ILSVRC) with a significant margin.
Architecture:
• Layers: Comprises 5 convolutional layers followed by 3 fully connected
layers.
• Activation Function: Utilizes ReLU (Rectified Linear Unit), which
accelerated training compared to traditional sigmoid/tanh functions.

Dr. G. Vennila, Assistant Professor, AIML 86 | P a g e

 • Pooling: Applies max pooling after some convolutional layers to reduce
spatial dimensions.
• Normalization: Incorporates Local Response Normalization (LRN) to
normalize activations across feature maps.
Key Innovations:
• Deep Network: Demonstrated the effectiveness of deep networks for image
classification.
• GPU Training: Leveraged GPUs to drastically reduce training time.
• Data Augmentation: Used data augmentation to enhance model
generalization.
Impact: AlexNet’s success highlighted the potential of deep learning and GPUs,
leading to widespread adoption and further research.
2. VGG (VGGNet)
Introduced: Visual Geometry Group, University of Oxford, 2014
Achievements: Known for its simplicity and depth.
Architecture:
• Layers: Includes 16 or 19 weight layers (both convolutional and fully
connected).
• Convolutional Layers: Employs 3x3 convolutional filters with a stride of 1
and 2x2 max pooling with a stride of 2.
• Fully Connected Layers: Ends with 3 fully connected layers (in VGG-16).
Key Innovations:
• Small Filters: Utilizes small (3x3) filters, effectively capturing fine details
and patterns.
• Uniform Architecture: Consistent use of small filters and pooling layers
simplifies the network design.
Impact: VGG demonstrated that deeper networks with small convolutional filters
could achieve high performance, influencing subsequent architectures.
Dr. G. Vennila, Assistant Professor, AIML 87 | P a g e
3. GoogleNet (Inception V1)
Introduced: Google, 2014
Achievements: Won the ILSVRC 2014.
Architecture:
• Layers: Contains 22 layers with a complex design based on the Inception
module.
• Inception Module: Employs parallel convolutions of different sizes (1x1,
3x3, 5x5) and pooling operations to capture multi-scale features.
• Global Average Pooling: Replaces fully connected layers with global
average pooling to reduce overfitting and parameter count.
Key Innovations:
• Inception Modules: Combines convolutions of varying sizes and pooling
in parallel to capture features at different scales.
• Efficient Computation: Reduces computational cost and number of
parameters compared to earlier architectures.
Impact: GoogleNet’s multi-scale approach and efficiency set new standards for
scalable and effective network designs.
4. ResNet (Residual Networks)
Introduced: Kaiming He et al., 2015
Achievements: Won the ILSVRC 2015.
Architecture:
• Layers: Ranges from 18 to 152 layers, with deeper versions using residual
blocks.
• Residual Blocks: Incorporates shortcut connections that bypass one or
more layers to facilitate gradient flow and prevent vanishing gradients.
• Bottleneck Architecture: Uses 1x1 convolutional layers to reduce and then
restore dimensions around a 3x3 convolutional layer.
Key Innovations:
Dr. G. Vennila, Assistant Professor, AIML 88 | P a g e
 • Residual Learning: Introduces shortcut connections to improve training of
very deep networks by addressing vanishing gradient issues.
• Training Deep Networks: Demonstrates effective training of very deep
networks (e.g., 152 layers).
Impact: ResNet’s ability to train extremely deep networks improved performance
across various computer vision tasks and demonstrated that very deep networks
could be practical.
Summary of Architectures
• AlexNet: Pioneered deep learning for image classification with ReLU
activation and GPU training.
• VGG: Focused on depth and small convolutional filters with a uniform
architecture.
• GoogleNet: Introduced the Inception module for efficient multi-scale
feature learning.
• ResNet: Innovated residual learning to enable training of very deep
networks effectively.
Each of these architectures has made significant contributions to CNN
development and has influenced subsequent advancements in deep learning.

Dr. G. Vennila, Assistant Professor, AIML 89 | P a g e

 1. Convolutional Layers
o Encoder: Utilizes convolutional layers to extract features from the input data,
capturing spatial hierarchies and patterns. Pooling layers are used to
downsample feature maps and reduce spatial dimensions.
o Decoder: Applies transposed convolutional layers to upsample the compressed
latent space representation, aiming to reconstruct the original input data.

How Convolutional Autoencoders Work

1. Encoding
o Process: The input data (e.g., an image) is passed through a sequence of
convolutional layers. These layers extract features, while pooling layers
downsample the feature maps.
o Latent Space: The final layer in the encoder produces a compressed, lower-
dimensional representation of the input data, known as the latent space.
2. Decoding
Dr. G. Vennila, Assistant Professor, AIML 90 | P a g e
 o Process: The latent space representation is fed through a series of transposed
convolutional layers. These layers upsample the data to reconstruct the original
input.
o Reconstruction: The network aims to recreate the original input as closely as
possible. The quality of the reconstruction is evaluated by comparing it to the
original input.
3. Training
o Objective: The network is trained to minimize the reconstruction loss, such as
Mean Squared Error (MSE) or Binary Cross-Entropy, between the original
input and the reconstructed output.
o Learning: During training, the encoder learns to compress the data effectively,
while the decoder learns to reconstruct it with high accuracy.

Convolutional Autoencoders are valuable for tasks such as image denoising, dimensionality
reduction, and feature extraction, as they leverage the power of convolutional layers to capture
and reconstruct complex spatial patterns.

Content-Based Image Retrieval (CBIR)

Content-Based Image Retrieval (CBIR) is a technique used to search and retrieve images from
a database based on their visual content rather than relying on metadata or textual descriptions.

Dr. G. Vennila, Assistant Professor, AIML 91 | P a g e

CBIR systems analyze the actual content of images—such as colors, textures, shapes, and
patterns—to find similar or relevant images.
Key Concepts of CBIR
1. Feature Extraction
o Objective: Extract meaningful features from images for comparison purposes.
o Types of Features:
▪ Color: Color histograms, color moments, color distribution.
▪ Texture: Statistical measures, Gabor filters, Local Binary Patterns
(LBP).
▪ Shape: Edges, contours, shape descriptors like Hough transforms or
contour-based features.
▪ Spatial Information: Relative positions of features within the image.
2. Feature Representation
o Objective: Represent the extracted features in a format that enables efficient
comparison.
o Common Methods:
▪ Feature Vectors: Numerical representations of extracted features.
▪ Histograms: Distribution of feature values, such as color histograms.
3. Similarity Measurement
o Objective: Assess the similarity between images based on their feature
representations.
o Methods:
▪ Distance Metrics: Euclidean distance, Manhattan distance, cosine
similarity.
▪ Similarity Measures: Correlation, Chi-square distance.
4. Indexing and Search
o Objective: Efficiently store and retrieve images based on their feature
representations.
o Techniques:
▪ Spatial Indexing: KD-trees, R-trees, or other spatial indexing structures
to organize feature vectors.

Dr. G. Vennila, Assistant Professor, AIML 92 | P a g e

 ▪ Hashing: Techniques like Locality-Sensitive Hashing (LSH) for fast
similarity search.
5. Relevance Feedback
o Objective: Enhance search results through user feedback.
o Process:
▪ User Interaction: Users provide feedback on the relevance of retrieved
images.
▪ Model Update: Adjust feature representation or similarity measures
based on feedback to refine search results.
Example Workflow of CBIR
1. Feature Extraction:
o Extract features from both the query image and all images in the database.
2. Feature Representation:
o Represent these features using feature vectors or histograms.
3. Similarity Measurement:
o Compute the similarity between the query image and database images using
chosen distance metrics or similarity measures.
4. Retrieve Results:
o Rank and retrieve images from the database based on their similarity to the
query image.
5. Relevance Feedback (Optional):
o Use user feedback to improve search results.
Applications of CBIR
1. Image Search Engines:
o Enable users to search for images based on visual content, such as Google
Images or Bing Image Search.
2. Medical Image Analysis:
o Retrieve similar medical images for diagnostic purposes or comparison.
3. Security and Surveillance:
o Match and retrieve images or video frames for security and surveillance
applications.
4. Digital Asset Management:

Dr. G. Vennila, Assistant Professor, AIML 93 | P a g e

 o Organize and retrieve digital assets like photographs, art, and media based on
content.
5. Fashion and Retail:
o Allow users to search for clothing items or accessories based on visual
similarities.
Key Challenges
1. Feature Representation:
o Selecting and designing effective feature extraction methods for diverse types
of images.
2. Scalability:
o Efficiently handling large databases in terms of both storage and retrieval speed.
3. Semantic Gap:
o Bridging the gap between low-level visual features and high-level semantic
concepts.
4. Computational Complexity:
o Ensuring efficient computation for feature extraction, similarity measurement,
and retrieval.
5. Relevance Feedback:
o Effectively incorporating user feedback to enhance search results.

Dr. G. Vennila, Assistant Professor, AIML 94 | P a g e

Object Detection
Object detection using deep learning focuses on identifying and locating objects within images
or videos. Unlike image classification, which only identifies the presence of an object, object
detection provides both the location and class of objects through bounding boxes. Deep
learning has greatly enhanced the accuracy and efficiency of object detection. Here's an
overview of how deep learning is applied to object detection:

Key Components of Deep Learning-Based Object Detection

1. Feature Extraction
o Objective: Extract relevant features from images that are essential for detecting
objects.
o Method: Convolutional Neural Networks (CNNs) are commonly used to
capture various details and patterns from different levels of the image.
2. Bounding Box Prediction
o Objective: Predict the location and dimensions of bounding boxes around
detected objects.
o Method: Networks estimate the coordinates of bounding boxes along with class
labels.
3. Classification
o Objective: Classify objects within the predicted bounding boxes.
o Method: After detecting regions of interest, a classifier determines the category
of each object within the bounding boxes.
4. Post-Processing
o Objective: Refine detection results and remove duplicates.
o Method: Techniques like Non-Maximum Suppression (NMS) are used to
eliminate overlapping bounding boxes and retain the most accurate ones.

Popular Deep Learning Architectures for Object Detection

1. R-CNN (Regions with CNN Features)

o Introduction: Proposed by Ross Girshick et al. in 2014.
o Approach:

Dr. G. Vennila, Assistant Professor, AIML 95 | P a g e

 ▪ Region Proposal: Uses selective search to generate region proposals.
▪ Feature Extraction: Applies CNNs to extract features from each region
proposal.
▪ Classification and Bounding Box Regression: Utilizes SVMs for
classification and linear regression for bounding box refinement.
o Challenges: Computationally intensive due to separate processing of region
proposals.
2. Fast R-CNN
o Introduction: Proposed by Ross Girshick in 2015.
o Approach:
▪ Region of Interest (RoI) Pooling: Extracts features for each proposal
from a shared feature map.
▪ End-to-End Training: Trains the entire network end-to-end, enhancing
speed and accuracy.
o Advantages: Faster and more accurate than R-CNN by processing the entire
image once and using RoI pooling.
3. Faster R-CNN
o Introduction: Proposed by Shaoqing Ren et al. in 2015.
o Approach:
▪ Region Proposal Network (RPN): Integrates a region proposal network
that shares convolutional features with the detection network, removing
the need for selective search.
▪ End-to-End Training: Trains both RPN and detection networks together.
o Advantages: Much faster than Fast R-CNN due to the integrated RPN.
4. YOLO (You Only Look Once)
o Introduction: Proposed by Joseph Redmon et al. in 2016.
o Approach:
▪ Single Pass Detection: Divides the image into a grid and predicts
bounding boxes and class probabilities for each grid cell in one pass.
▪ Real-Time Detection: Designed for high-speed, real-time object
detection.
o Advantages: Fast and capable of processing video streams in real-time.

Dr. G. Vennila, Assistant Professor, AIML 96 | P a g e

 5. SSD (Single Shot MultiBox Detector)
o Introduction: Proposed by Wei Liu et al. in 2016.
o Approach:
▪ Multi-Scale Feature Maps: Uses multiple feature maps at different
scales to detect objects of varying sizes.
▪ Single Pass Detection: Performs detection in one pass, similar to YOLO.
o Advantages: Balances speed and accuracy, suitable for real-time applications.
6. RetinaNet
o Introduction: Proposed by Tsung-Yi Lin et al. in 2017.
o Approach:
▪ Focal Loss: Introduces focal loss to address class imbalance between
foreground and background.
▪ Single Shot Detection: Detects objects in one pass.
o Advantages: Effective at detecting small and difficult objects with improved
accuracy.
7. EfficientDet
o Introduction: Proposed by Mingxing Tan et al. in 2020.
o Approach:
▪ Compound Scaling: Uses compound scaling to balance model accuracy,
size, and speed.
▪ Efficient Backbone: Employs a highly efficient backbone for feature
extraction.
o Advantages: Achieves state-of-the-art performance with high efficiency.

Natural Language Processing and Sequence Learning

Dr. G. Vennila, Assistant Professor, AIML 97 | P a g e

mechanisms

1. Attention Mechanisms
o Self-Attention: Allows the model to focus on different parts of the input
sequence when making predictions.
o Transformer Architecture: Utilizes self-attention and feed-forward networks to
process sequences in parallel, overcoming the limitations of RNNs and LSTMs.

Key Models and Architectures

1. Recurrent Neural Networks (RNNs)

o Usage: Effective for tasks where sequential information is crucial, such as
language modeling and time series forecasting.

Dr. G. Vennila, Assistant Professor, AIML 98 | P a g e

 o Challenges: Struggle with long-term dependencies due to vanishing or
exploding gradients.
2. Long Short-Term Memory (LSTM)
o Usage: Handles long-term dependencies better than standard RNNs by using
gating mechanisms to control information flow.
o Components: Includes input gate, forget gate, and output gate.
3. Gated Recurrent Unit (GRU)
o Usage: A variant of LSTM that merges input and forget gates into a single
update gate, simplifying the architecture.
o Advantages: Often performs similarly to LSTM with fewer parameters.
4. Transformer
o Introduced: In "Attention is All You Need" by Vaswani et al. (2017).
o Architecture: Consists of an encoder and decoder, both utilizing self-attention
mechanisms and feed-forward layers.
o Self-Attention: Allows the model to weigh the importance of different words in
a sequence.
o Advantages: Handles long-range dependencies efficiently and processes
sequences in parallel.
5. BERT (Bidirectional Encoder Representations from Transformers)
o Introduced: By Google AI in 2018.
o Architecture: A transformer-based model trained bidirectionally on large text
corpora.
o Usage: Effective for various NLP tasks through pre-training on large datasets
and fine-tuning for specific tasks.
6. GPT (Generative Pre-trained Transformer)
o Introduced: By OpenAI.
o Architecture: A series of transformer-based models trained on extensive text
data.
o Usage: Generates human-like text, performs text completion, and more.
7. T5 (Text-To-Text Transfer Transformer)
o Introduced: By Google Research in 2020.

Dr. G. Vennila, Assistant Professor, AIML 99 | P a g e

 o Architecture: Treats all NLP tasks as text-to-text problems, enabling a unified
model for diverse tasks.
o Usage: Versatile model for translation, summarization, question answering, and
more.

Applications of Sequence Learning in NLP

1. Language Translation
o Objective: Translate text from one language to another.
o Approach: Models like Seq2Seq with attention and transformers (e.g., Google
Translate, BERT) are used.
2. Sentiment Analysis
o Objective: Determine the sentiment expressed in text (e.g., positive, negative,
neutral).
o Approach: Models like LSTM, GRU, and BERT classify sentiment based on
textual input.
3. Named Entity Recognition (NER)
o Objective: Identify and classify entities (e.g., names, dates, locations) in text.
o Approach: Models like CRF (Conditional Random Fields) and BERT-based
NER systems are used.
4. Text Summarization
o Objective: Generate concise summaries of longer documents.
o Approach: Abstractive summarization uses models like transformers, while
extractive summarization identifies key sentences or phrases.
5. Question Answering
o Objective: Provide accurate answers to questions based on context or
documents.
o Approach: Models like BERT and GPT-3 understand questions and extract or
generate answers from the context.
6. Speech Recognition
o Objective: Convert spoken language into written

Dr. G. Vennila, Assistant Professor, AIML 100 | P a g e