Module 3: Convolutional Neural Networks

Convolutional Neural Networks –Architecture, Convolution operation,

Motivation, pooling .Variants of convolution functions, Structured outputs,
Data types, Efficient convolution algorithms, Applications of Convolutional
Networks, Pre-trained convolutional Architectures : AlexNet, ZFNet,
VGGnet-19, ResNet50.
INTRODUCTION

• Convolutional networks also known as convolutional neural networks or CNNs,

are a specialized kind of neural network for processing data that has a known,
grid-like topology.

The name “convolutional neural network” indicates that the network employs a
mathematical operation called convolution.

• Convolution is a specialized kind of linear operation.

Convolution Operation
2D convolution
3D convolution
stride
padding
valid(left), same(middle) and full(right) padding. The extreme left one is for
stride=2.
• ‘W1’ is the width of the input image.
• ‘H1’ is the height of the input image.
• ‘D1’ is the depth of the image (for a monochrome
image, it’s 1 whereas for a coloured image it’s 3,
with RGB channels).
• ‘S’ stands for stride, i.e. the number of pixels you
shift the filter by while sliding it across the image.
• ‘K’ stands for the number of filters used.
• ‘F’ is the spatial extent of each filter
• Output: W2 x H2 x D2, where D2 is the number of
filters (K).
• The output consists of all feature maps obtained
from different filters stacked against each other.
Hence, the depth of the output is equal to the
number of feature maps/ number of kernels.
• The depth of the output is equal to the number of feature maps/ number
of kernels.
Let us do a few exercises
Architecture of CNN

There different layers that make up the CNN are:

1. Input layer
2. The convolutional layers,
3. Pooling layers,
4. Fully-connected (FC) layers.
5. Output layer

● When these layers are stacked, a CNN architecture will be formed.

CNN architecture
Input Layers:

It’s the layer in which we give input to our model. In CNN, Generally, the input will be an
image or a sequence of images.

Convolutional Layers:

This is the layer, which is used to extract the feature from the input dataset. It applies a set of
learnable filters known as the kernels to the input images. The filters/kernels are smaller
matrices usually 2x2, 3x3, or 5x5 shape. It slides over the input image data and computes the
dot product between kernel weight and the corresponding input image patch. The output of this
layer is referred as feature maps.
Activation Layer:
By adding an activation function to the output of the preceding layer, activation layers add
nonlinearity to the network. It will apply an element-wise activation function to the output
of the convolution layer. Some common activation functions are RELU, Tanh, Leaky
RELU, etc.

Pooling layer:
This layer is periodically inserted in the covnets and its main function is to reduce the size of
volume which makes the computation fast, reduces memory and also prevents overfitting.
Two common types of pooling layers are max pooling and average pooling.
Flattening:

The resulting feature maps are flattened into a one-dimensional vector after the convolution and
pooling layers so they can be passed into a completely linked layer for categorization or regression.

Fully Connected Layers:

It takes the input from the previous layer and computes the final classification or regression task.

Output Layer:

The output from the fully connected layers is then fed into a logistic function for classification tasks like
sigmoid or softmax which converts the output of each class into the probability score of each class.
Normal Convolution (l = 1):
In normal convolution, the feature detector (also known as the kernel or filter) moves
across the input image pixel by pixel, covering adjacent pixels in a regular grid
pattern. The filter is applied to each local region in the image, with no skipped pixels
between the receptive fields. This process involves a one-to-one mapping between
the filter's cells and the input image pixels, denoted by the grid in the left diagram.

Dilated Convolution (l = 2):

In dilated convolution, the feature detector has gaps between the pixels it covers.
These gaps (referred to as dilation) allow the filter to cover a larger receptive field
without increasing its size. For example, when dilation factor l = 2, the filter skips one
pixel between adjacent values, as shown in the right diagram. This allows the network
to capture wider context or long-range dependencies while maintaining a relatively
smaller filter size. Skipped pixels are typically marked in a different color (like pink), as
shown in the diagram.
Transposed Convolution
The operation of a transposed convolutional layer is similar to that of a
normal convolutional layer, except that it performs the convolution operation
in the opposite direction. Instead of sliding the kernel over the input and
performing element-wise multiplication and summation, a transposed
convolutional layer slides the input over the kernel and performs
element-wise multiplication and summation. This results in an output that
is larger than the input, and the size of the output can be controlled by the
stride and padding parameters of the layer.
Example 1:

Suppose we have a grayscale image of size 2 X 2, and we want to upsample it using a transposed
convolutional layer with a kernel size of 2 x 2, a stride of 1, and zero padding (or no padding). The
input image and the kernel for the transposed convolutional layer would be as follows:
This means that immediately neighboring locations will have different filters
Structured Output
Convolution networks can be used to output a high-dimensional structured object, rather than just predicting
a class label for a classification task or a real value for regression tasks.

This high-dimensional structured object is typically represented as a tensor, which could be a

multi-dimensional array that encodes spatial relationships or structured information about the input data.

For instance, in image segmentation tasks, the structured output might be a tensor where each element
represents the probability that a particular pixel belongs to a specific class (such as background, foreground,
or object).

The model outputs a tensor S where S[i,j,k] is the probability that pixel (j,k) belongs to class i.

This process is common in tasks where precise spatial localization is required, like object detection or
segmentation, where the model needs to label every pixel of an image.
● Spatial Reduction: In typical convolutional networks for classification tasks,
the spatial dimensions of the output tensor are reduced through pooling layers
with strides greater than 1. This reduction might make the output tensor smaller
than the input, which could be problematic in tasks like pixel-level
classification.

● To address this, strategies include:

○ Avoiding pooling: Some architectures avoid pooling altogether or use
pooling with a stride of 1 to maintain the spatial resolution.
○ Upsampling: The output tensor can be upsampled (using techniques like
transposed convolutions or deconvolution) to match the input size. This
allows pixel-level predictions without excessive spatial shrinking.
 Data Types
● The data used with a convolutional network usually consists of several
channels, each channel being the observation of a different quantity.

Single channel Multi-channel

• Audio waveform: The axis we • Color image data: One channel contains the
convolve over corresponds to time. red pixels, one the green pixels, and one the
We discretize time and measure the blue pixels. The convolution kernel moves over
amplitude of the waveform once both the horizontal and vertical axes of the
per time step. image, conferring translation equivariance in
• Volumetric data: A common source both directions.
of this kind of data is medical • Color video data: One axis corresponds to
imaging technology, such as CT time, one to the height of the video frame,
scans and one to the width of the video frame
Applications of Convolutional Networks

1. Image and Video Recognition

Image and video recognition are among the key convolutional neural networks applications. With their ability to process visual
data in real-time, enterprises use CNNs to detect faces, objects, gestures, and even scenes with great accuracy. For instance,
they can be used in security systems for real-time facial recognition and automated content curation. They can be further
trained/programmed to analyze foot traffic, product engagement, and demographic patterns.

2. Natural Language Processing

Beyond vision, convolutional neural networks applications extend to natural language processing to understand human language
with a high degree of precision. This includes both text and visual, providing remarkable features like sentiment analysis,
human-like chatbots, and different text classifications. For instance, enterprises can use CNN-powered NLP applications to
evaluate customer reviews, social media presence, and emotional insights. This helps product managers with product
development strategies that not only satisfy but also nourish the customers' needs.
3. Autonomous Vehicles
Convolutional neural networks applications are becoming more central to modern safe transportation technologies,
enabling reliable perception and real-time decision-making for autonomous driving. They can process camera feeds to
detect lanes, pedestrians, vehicles, and signs to develop a dynamic environmental stimulus, allowing vehicles to
process and instantly react.

4. Medical Imaging
CNNs are transforming medical imaging into healthcare by delivering faster, more accurate diagnostics. These models
can be trained to detect anomalies in MRIs, X-rays, mammograms, and CT scans, with more than 90% accuracy,
matching and even surpassing human radiologists' capabilities. Convolutional neural networks applications in
healthcare can help identify early signs of cancer, cardiovascular conditions, and neurological disorders.

5. Financial Services
CNN data science is redefining financial data analysis by detecting anomalies like fraud and financial risks with high
precision. Their ability to analyze multi-dimensional complex data sets and spot unusual patterns is a game-changer for
financial institutions handling large volumes of data.
6. Retail and E-commerce
Convolutional neural networks applications in retail and e-commerce enable smarter, more personalized shopping
experiences and drive customer satisfaction to new heights. With their ability to analyze product images and customer
behavior, they can recommend items that the shoppers are more likely to purchase and boost sales.

7. Industrial Automation
Convolutional neural networks applications can automate monotonous and complex tasks and transform how industries
work by making machines smarter. With features like visual inspection, defect detection, and predictive maintenance,
they can help examine production processes and anticipate equipment breakdowns before they cause disruptions.

8. Agricultural Optimisation and Analysing Crops

Farmers gain a lot from the use of CNNs. Aerial drones help by using photography to look at fields.
Images are analysed by CNNs to identify plant diseases, pests and any problems with soil.
Image classification: cat vs. dog, disease vs. healthy leaf, defect vs. OK part.

Object detection: find and label objects with boxes (people, cars, tumors on scans).

Instance & semantic segmentation: outline each object or label every pixel (road lanes, organs in CT, crop fields in satellite images).

Keypoints & pose: locate joints/landmarks (human/animal pose, facial landmarks).

Tracking in video: follow objects across frames (traffic analytics, sports).

Depth & optical flow: per-pixel depth/motion for AR/VR, robotics navigation.

Super-resolution / denoising / deblurring / inpainting: image enhancement and restoration (CCTV, microscopy, astronomy).

Style transfer & image-to-image: maps→photos, sketches→faces; photo colorization.

OCR & document AI: text detection/recognition, table/layout analysis, cheque/invoice parsing.

Biometrics: face recognition, fingerprint/iris matching.

Medical imaging: lesion detection, organ/tumor segmentation (MRI/CT/X-ray/fundus).

Remote sensing: land-use classification, building/road extraction from satellite.

Autonomous driving: perception stack—detection, segmentation, lane/obstacle parsing.

Robotics: scene understanding, grasp point detection, obstacle avoidance.

Industrial inspection: surface defect detection, PCB solder joint checks.

Agriculture: plant disease spotting, fruit counting, yield estimation.

Audio & speech (1D/2D convs): keyword spotting, music tagging, spectrogram tasks.

NLP (text CNNs): sentence classification, intent detection (fast, lightweight).

Time-series: ECG/seismology anomaly detection, sensor fault detection.

Communications: modulation classification, channel estimation/CSI compression, MIMO detection aids.

Security: malware classification (as images), steganalysis, anomaly detection in CCTV.

Pre-trained convolutional Architectures : AlexNet

The first CNN-based architecture.

Most important features of the AlexNet are:

● Overfitting Prevention: Dropout (0.5) was applied to the first two fully connected layers
and data augmentation dynamically expanded the dataset hence both helping in reducing
overfitting.
● Faster Training: ReLU activation was used instead of tanh or sigmoid, leading to a 6×
speedup in training by avoiding activation saturation.
AlexNet Architecture

Its architecture consists of 8 layers: 5 convolutional layers and 3 fully connected layers.

Input Layer
AlexNet takes images of the Input size of 227x227x3 RGB Pixels.
Convolutional Layers
● First Layer: The first layer uses 96 kernels of size 11×11 with a stride of 4, activates them with the ReLU activation
function, and then performs a Max Pooling operation.
● Second Layer: The second layer takes the output of the first layer as the input, with 256 kernels of size 5x5x48.
● Third Layer: 384 kernels of size 3x3x256. No pooling or normalization operations are performed on the third, fourth, and
fifth layers.
● Fourth Layer: 384 kernels of size 3x3x192.
● Fifth Layer: 256 kernels of size 3x3x192.

Fully Connected Layers

The fully connected layers have 4096 neurons each.
Output Layer
The output layer is a SoftMax layer that outputs probabilities of the 1000 class labels.
 AlexNet Features
● Deep Architecture: This model utilized deep architecture compared to any NN model released previously. It consisted
of five convolutional layers followed by three fully connected layers.
● ReLU Nonlinearity: CNNs at that time used functions such as Tanh or Sigmoid to process information between layers.
These functions slowed down the training. In contrast, ReLU (Rectified Linear Unit) made the entire process simpler
and many times faster. It outputs only if the input is given to it as positive, otherwise, it outputs a zero.
● Overlapping Pooling: Overlapping pooling is just like regular max pooling layers, but in overlapping pooling, as the
window moves across, it overlaps with the previous window. This improved the error percentage in AlexNet.
● Use of GPU: Before AlexNet, NNs were trained on the CPU, which made the process slow. However, the researcher of
AlexNet incorporated GPUs, which accelerated computation time significantly. This proved that Deep NNs can be
trained feasibly on GPUs.
● Local Response Normalization (LRN): This is a process of normalizing adjacent channels in the network, which
normalizes the activity of neurons within a local neighborhood.
ZFNet

ZFNet (Zeiler and Fergus Net) is a convolutional neural network

introduced by Matthew D. Zeiler and Rob Fergus, a refinement of
the AlexNet architecture.
It improved its performance by using smaller convolutional filters in
the early layers (7x7 instead of 11x11) and reducing the stride,
resulting in better retention of pixel information.
It was a significant development that also introduced
deconvolutional networks to visualize the internal workings of the
neural network, providing insights into how each layer extracts
features.
ZFNet
Architecture

The input image is of size 224x224x3.

First Layer
· In the first layer 96 filters of size 7x7 and stride of 2 are used to convolve
followed by relu activation.
The output feature map is then passed through Max Pooling Layer with pool
kernel of 3x3 and stride of 2 .Then the features are contrast normalized.

Second layer
· In the second layer 256 filters are applied of size 3x3 with stride of 2. Again
the obtained feature map is passed through MaxPooling layer with pooling
kernel of 3x3 with stride of [Link] that features are contrast normalized.
·
Third layer and Fourth Layer
· The third and fourth layers are identical with 384 kernels of size
3x3 and padding is kept as same and stride is set to 1.

Fifth Layer
· In the fifth layer 256 filters of size 3x3 are applied with stride 1. After then the
MaxPooling kernel of size 3x3 is applied with stride of 2 .Then the features are
contrast normalized.

Sixth Layer and Seventh Layer

· The sixth and seventh layers are fully connected dense layers with 4096
neurons each.

Eighth Layer
· The last layer is dense layer with 1000 neurons(number of classes).
Key Features of ZFNet
● Based on AlexNet: ZFNet is a modified version of the AlexNet architecture,
keeping a similar structure but making specific adjustments to the early
convolutional layers.
● Smaller Filters in Early Layers: The most significant change was the use of
smaller 7x7 filters in the first convolutional layers, compared to the 11x11 filters
used in AlexNet. This change helped retain more pixel information, leading to
improved accuracy.
● Deconvolutional Networks for Visualization: A key innovation was the use of
deconvolutional layers to visualize the features learned by each layer of the
network. This allowed researchers to understand what input stimuli excited specific
feature maps, a groundbreaking aspect for interpreting neural networks.
● Improved Accuracy: The architectural tweaks, particularly the
smaller filter sizes, led to a more accurate model compared to its
predecessor, AlexNet.
● Activation Function: Like many modern CNNs, ZFNet utilized the
Rectified Linear Unit (ReLU) as its activation function to introduce
non-linearity into the model.
● Deconvolutional Networks for Visualization
Differences Between AlexNet and ZFNet

Architecture
· AlexNet consists of eight layers, five convolutional layers followed by
three fully connected layers. ZFNet retained basic architecture of
AlexNet but made some architectural adjustments,particularly in the
first few layers.

Filters
· AlexNet used 11x11,5x5 and 3x3 filter sizes while ZFNet used 7x7 filter
size in the first layer only and 3x3 in the latter layers only.
Strides
· There is stride of 4 in the first layer of AlexNet while in
ZFNet there is stride of 2 used.
Normalization
· AlexNet used Local Response Normalization while
ZFNet used Local Contrast Normalization

Normalization done within spatial region - LCN

Normalization done across channels - LRN
VGGnet-19 (Visual Geometry Group (VGG) models)

VGG-19 is a deep convolutional neural network with 19 weight layers, comprising 16

convolutional layers and 3 fully connected layers. The architecture follows a straightforward
and repetitive pattern, making it easier to understand and implement.
The key components of the VGG-19 architecture are:
1. Convolutional Layers: 3x3 filters with a stride of 1 and padding of 1 to preserve spatial
resolution.
2. Activation Function: ReLU (Rectified Linear Unit) applied after each convolutional layer
to introduce non-linearity.
3. Pooling Layers: Max pooling with a 2x2 filter and a stride of 2 to reduce the spatial
dimensions.
4. Fully Connected Layers: Three fully connected layers at the end of the network for
classification.
5. Softmax Layer: Final layer for outputting class probabilities.
Architecture of VGG-Net 19

The VGG-19 model consists of five blocks of convolutional layers, followed by three fully connected layers.

Block 1

● Conv1_1: 64 filters, 3x3 kernel, ReLU activation

● Conv1_2: 64 filters, 3x3 kernel, ReLU activation
● Max Pooling: 2x2 filter, stride 2

Block 2

● Conv2_1: 128 filters, 3x3 kernel, ReLU activation

● Conv2_2: 128 filters, 3x3 kernel, ReLU activation
● Max Pooling: 2x2 filter, stride 2
Block 3

● Conv3_1: 256 filters, 3x3 kernel, ReLU activation

● Conv3_2: 256 filters, 3x3 kernel, ReLU activation
● Conv3_3: 256 filters, 3x3 kernel, ReLU activation
● Conv3_4: 256 filters, 3x3 kernel, ReLU activation
● Max Pooling: 2x2 filter, stride 2

Block 4

● Conv4_1: 512 filters, 3x3 kernel, ReLU activation

● Conv4_2: 512 filters, 3x3 kernel, ReLU activation
● Conv4_3: 512 filters, 3x3 kernel, ReLU activation
● Conv4_4: 512 filters, 3x3 kernel, ReLU activation
● Max Pooling: 2x2 filter, stride 2
Block 5

● Conv5_1: 512 filters, 3x3 kernel, ReLU activation

● Conv5_2: 512 filters, 3x3 kernel, ReLU activation
● Conv5_3: 512 filters, 3x3 kernel, ReLU activation
● Conv5_4: 512 filters, 3x3 kernel, ReLU activation
● Max Pooling: 2x2 filter, stride 2

Fully Connected Layers

● FC1: 4096 neurons, ReLU activation

● FC2: 4096 neurons, ReLU activation
● FC3: 1000 neurons, softmax activation (for 1000-class classification)
Features

1. Uniform Convolution Filters: Consistently using 3x3 convolution filters simplifies the
architecture and helps maintain uniformity.
2. Deep Architecture: Increasing the depth of the network enables learning more
complex features.
3. ReLU Activation: Introducing non-linearity helps in learning complex patterns.
4. Max Pooling: Reduces the spatial dimensions while preserving important features.
5. Fully Connected Layers: Combines the learned features for classification.
ResNet50 (Residual Networks)
Input
● Input frames/images: Typically 224 × 224 × 3 RGB images

Initial Convolution + Pooling

1. 7×7 convolution, 64 filters, stride 2

○ Kernel size: 7×7

○ Filters: 64

○ Stride: 2 (reduces spatial dimensions by half).

○ Output feature map: 112 × 112 × 64.

○ Purpose: Captures low-level features (edges, textures).

2. Max Pooling (3×3, stride 2)

○ Further reduces spatial size.

○ Output: 56 × 56 × 64.
🔹 Residual Stages (Bottleneck Blocks)

ResNet-50 is built from bottleneck residual blocks:

● Each block = 3 convolution layers (1×1, 3×3, 1×1).

● Skip connection adds input directly to output.

Stage 1 (Conv2_x) → Repeated 3 times

● Layers inside each block:

○ 1×1 conv (64 filters) → reduces dimensionality.

○ 3×3 conv (64 filters).

○ 1×1 conv (256 filters) → restores dimensionality.

● After 3 blocks → 56 × 56 × 256 feature maps.

● Contribution: 9 layers.
Stage 2 (Conv3_x) → Repeated 4 times
● Layers per block:

○ 1×1 conv (128 filters).

○ 3×3 conv (128 filters).

○ 1×1 conv (512 filters).

● After 4 blocks → 28 × 28 × 512 feature maps.

● Contribution: 12 layers.
Stage 3 (Conv4_x) → Repeated 6 times
● Layers per block:

○ 1×1 conv (256 filters).

○ 3×3 conv (256 filters).

○ 1×1 conv (1024 filters).

● After 6 blocks → 14 × 14 × 1024 feature maps.

● Contribution: 18 layers.
Stage 4 (Conv5_x) → Repeated 3 times
● Layers per block:

○ 1×1 conv (512 filters).

○ 3×3 conv (512 filters).

○ 1×1 conv (2048 filters).

● After 3 blocks → 7 × 7 × 2048 feature maps.

● Contribution: 9 layers.
🔹 Final Layers
1. Average Pooling (7×7)

○ Reduces each feature map to a single value.

○ Output: 1 × 1 × 2048 (feature vector).

2. Fully Connected (FC) Layer

○ Dense layer with 1000 units (for ImageNet classification).

3. Softmax

○ Converts output into probabilities across 1000 classes.

🔹 Layer Counting → 50 layers
● Initial Conv layer = 1

● Conv2_x = 3 × (3 layers) = 9

● Conv3_x = 4 × (3 layers) = 12

● Conv4_x = 6 × (3 layers) = 18

● Conv5_x = 3 × (3 layers) = 9

● Final FC layer = 1

● Total = 1 + 9 + 12 + 18 + 9 + 1 = 50 layers
Depth: 50 layers (hence the name).

Architecture: Built using residual blocks, where each block has a skip
connection that allows the input to bypass a few layers and be added
back to the output. This helps combat the vanishing gradient problem.

To overcome vanishing gradient problem, ResNet used

“SKIP CONNECTION”.
EFFICIENT CONVOLUTION ALGORITHMS

1. Naïve Convolution

Naïve convolution directly applies the kernel (filter) to the input image or
feature map by sliding the kernel across the input and computing a
weighted sum at each position.
2. im2col
“image to column”

● Instead of sliding the kernel over the image and recomputing sums
repeatedly, flatten all the overlapping image patches into columns of
a matrix.

That way, convolution becomes a matrix multiplication (GEMM)

problem.
 Algorithm
1. Rearrange input patches into a 2D matrix

● Each sliding window of the input becomes one column in a

big matrix.

2. Flatten filters into another matrix.

3. Multiply the two matrices (GEMM)

GEneral Matrix Multiply
Example
 Step 2: Extract all sliding patches.

Step 3 : Each patch gets flattened column-wise. Stack all these column
vectors side by side:

Step 4: Reshape kernel

Flatten kernel K into a row vector:
3. Spatially separable convolution
is a technique that reduces computation in 2D convolutions
by factoring a 2D kernel into two 1D kernels—a vertical
kernel and a horizontal kernel.
Instead of applying the full 2D kernel at once, the vertical
1D kernel is first applied along the columns of the input to
produce an intermediate output, and then the horizontal 1D
kernel is applied along the rows of this intermediate result.
This reduces the number of multiplications per output pixel,
making it computationally efficient, and it is effective when
the original kernel is separable (rank=1).
4. Depthwise Separable Convolution
is an efficient alternative to standard convolution that
splits the operation into two steps: a depthwise
convolution, where each input channel is filtered
separately using a small kernel (e.g., 3×3), and a
pointwise convolution (1×1), which combines these
outputs across channels to form the final feature maps.
This decomposition greatly reduces the number of
parameters and computations compared to full
convolutions.