Deep Learning

Unit 2
Convolutional Neural Networks (CNN)

Basics of Image Processing for Deep Learning

Image processing is the field of computer science and engineering concerned with analyzing,
manipulating, and transforming digital images to extract meaningful information or improve
image quality. In deep learning, image processing plays a crucial role because neural networks
require numerical and structured input data. Raw images must often be preprocessed and
transformed before being fed into models such as Convolutional Neural Networks (CNNs).

The goal of image processing in deep learning is not only visual enhancement but also feature
extraction, normalization, and representation learning, which improve model accuracy,
robustness, and generalization.

A Digital Image

A digital image is a two-dimensional function:

f(x,y)f(x, y)f(x,y)

Where:

• x,yx, yx,y represent spatial coordinates

• f(x,y)f(x, y)f(x,y) represents intensity or brightness at that point

In computer systems, images are stored as matrices of pixel values.

Types of Images:

• Grayscale images: Each pixel has one intensity value (0–255)

• Color images: Each pixel has three values (RGB channels)
• Binary images: Pixels have only two values (0 or 1)

Image Representation for Deep Learning

In deep learning:

• A grayscale image → 2D matrix

• A color image → 3D tensor (Height × Width × Channels)
• A batch of images → 4D tensor (Batch × Height × Width × Channels)

Pixel values are usually normalized to the range [0,1] or [-1,1] before training.
Importance of Image Processing in Deep Learning

Image processing is essential because:

• Raw images contain noise and irrelevant details

• Neural networks require fixed-size, normalized inputs
• Preprocessing improves learning speed and accuracy
• Feature extraction helps networks focus on meaningful patterns

Without proper image processing, deep learning models may converge slowly, overfit, or
perform poorly.

Convolutional Neural Network Working Architecture

Convolutional Operation

The convolutional operation is the core mathematical process used in Convolutional Neural
Networks (CNNs) to extract meaningful features from images such as edges, textures, shapes,
and patterns. Instead of connecting every neuron to every input pixel, convolution uses small
filters (kernels) that slide over the input image to detect local patterns efficiently.

This operation preserves spatial relationships and reduces the number of parameters compared
to fully connected networks.

Convolution is a mathematical operation in which a small matrix called a filter or kernel is

applied over an input image to produce a feature map by computing weighted sums of local
pixel regions.

Components of Convolution

a) Input Image
A 2D (grayscale) or 3D (color) array of pixel values.

b) Filter / Kernel

A small matrix (e.g., 3×3, 5×5) with trainable weights that detects specific patterns.

c) Feature Map

The output matrix formed after applying the filter across the image.

d) Stride

The number of pixels the filter moves at each step.

e) Padding

Adding extra pixels (usually zeros) around the image border to control output size.

Working of Convolution Operation

1. A filter is placed over a small region of the input image.

2. Element-wise multiplication is performed between the filter and image region.
3. The products are summed to produce a single output value.
4. The filter slides across the image using stride.
5. This process continues until the entire image is covered.
6. The result is a feature map.

Multiple filters produce multiple feature maps.

Mathematical Representation
Types of Convolution

a) 2D Convolution

Used in grayscale images.

b) 3D Convolution

Used in color images (RGB channels).

c) 1×1 Convolution

Used for channel reduction and feature mixing.

Padding Types

• Valid Padding: No padding, output size shrinks.

• Same Padding: Padding added so output size equals input size.

Stride Effect

• Stride = 1 → High resolution output

• Stride > 1 → Reduced spatial dimensions

Advantages of Convolution

• Parameter sharing reduces model size

• Captures local spatial features
• Translation invariance
• Efficient computation

Role in CNNs

Early layers detect edges and textures, deeper layers detect shapes and objects. Convolution
allows CNNs to automatically learn hierarchical features.

The convolutional operation is a fundamental building block of CNNs that applies small filters
across an image to extract important local features while preserving spatial structure. By using
shared weights, stride, and padding, convolution enables efficient and powerful learning for
image-based deep learning tasks.

Parameter Sharing

Parameter sharing is a key concept in Convolutional Neural Networks (CNNs) where the same
set of weights (filter/kernel) is used repeatedly across different spatial locations of the input
image. Instead of learning separate weights for every input position, a single filter is applied
across the entire image to detect the same feature everywhere.

This significantly reduces the number of parameters and improves learning efficiency.

Parameter sharing means using the same weights and bias values for multiple connections in a
neural network layer, especially across different regions of an input image.

Working of Parameter Sharing

• A small filter (e.g., 3×3) slides over the input image.

• At every position, the same filter weights are used.
• This produces a feature map highlighting where that pattern appears in the image.

Each filter learns to detect a specific feature such as an edge, corner, or texture.

Importance of Parameter Sharing

• Reduces model size: Fewer parameters than fully connected layers.

• Improves generalization: Prevents overfitting.
• Translation invariance: Detects features regardless of their position.
• Efficient computation: Faster training and inference.

Example

A 3×3 filter applied to a 100×100 image uses only 9 weights, but detects patterns across all
image regions, unlike a fully connected layer which would require thousands of weights.

Parameter sharing is a fundamental principle in CNNs that enables efficient learning by using
the same filter weights across spatial locations. It reduces computational cost, improves
generalization, and allows networks to recognize patterns regardless of their position in the
image.

Equivariant Representation

An equivariant representation in deep learning refers to a property where a transformation

applied to the input results in a corresponding transformation in the output. In other words,
when the input changes in a structured way (such as shifting or rotating an image), the output
representation changes in a predictable and consistent manner.

This concept is especially important in Convolutional Neural Networks (CNNs), where

convolution layers are naturally translation equivariant.

A representation is said to be equivariant to a transformation if applying that transformation to

the input causes a corresponding transformation to the output.

Mathematically:
Example in CNNs (Translation Equivariance)

If an image is shifted to the right, the feature map produced by a convolution layer also shifts
to the right. This means CNNs preserve spatial relationships and detect patterns wherever
they appear.

Difference Between Equivariance and Invariance

Property Meaning
Equivariance Output changes in the same way as input transformation
Invariance Output remains unchanged despite input transformation

Example:

• Convolution → Equivariant to translation

• Pooling → Helps achieve translation invariance

Importance of Equivariant Representation

• Preserves spatial structure

• Improves learning efficiency
• Reduces need for excessive training data
• Makes models robust to transformations

Equivariant representation allows neural networks to respond predictably to transformations in

input data. In CNNs, translation equivariance ensures that features are detected consistently
across different positions, forming the foundation for robust and efficient visual learning
systems.

Pooling Layers

Pooling layers are an important component of Convolutional Neural Networks (CNNs) used to
reduce the spatial dimensions (height and width) of feature maps while preserving the most
important information. This helps in lowering computational cost, reducing overfitting, and
making the network more robust to small translations and distortions in the input.

A pooling layer is a downsampling operation that summarizes local regions of a feature map
into a single representative value.

Purpose of Pooling
 • Reduces feature map size
• Decreases number of parameters
• Controls overfitting
• Improves computational efficiency
• Provides translation invariance

Types of Pooling

a) Max Pooling

• Selects the maximum value in each region

• Most commonly used
• Captures strongest features

b) Average Pooling

• Computes average value in each region

• Preserves overall feature distribution

c) Global Pooling

• Pools across entire feature map

• Often used before fully connected layers

Working of Pooling Operation

1. A window (e.g., 2×2) slides over the feature map.

2. A summary value (max or average) is computed.
3. The result forms a smaller feature map.
Stride usually equals pool size, resulting in downsampling.

Advantages of Pooling Layers

• Reduces computation and memory usage

• Improves generalization
• Makes features robust to small shifts
• Controls model complexity

Role in CNNs

Pooling layers are typically placed after convolution and activation layers to progressively
reduce spatial resolution while retaining meaningful features for classification or detection.

Pooling layers downsample feature maps by summarizing local regions, helping CNNs become
more efficient, robust, and less prone to overfitting. Max pooling is most commonly used,
though average and global pooling are also important in certain architectures.

Variants of Convolution (Dilated, Depthwise, Separable)

Standard convolution uses full connections between input and output channels, which increases
computational cost. To improve efficiency and performance, modern CNNs use specialized
convolution variants such as Dilated, Depthwise, and Separable convolutions. These
techniques reduce parameters, expand receptive fields, and speed up training while maintaining
accuracy.

1. Dilated Convolution (Atrous Convolution)

Dilated convolution inserts gaps between kernel elements, allowing the filter to cover a wider
area without increasing kernel size.
Purpose

• Increases receptive field

• Preserves spatial resolution
• Useful in segmentation tasks

Example

A 3×3 filter with dilation rate 2 behaves like a 5×5 filter but with fewer parameters.

2. Depthwise Convolution

Each input channel is convolved separately with its own filter, instead of combining all
channels together.

Purpose

• Reduces computation
• Extracts spatial features per channel

[Link] Convolution (Depthwise Separable Convolution)

Breaks standard convolution into two steps:

1. Depthwise convolution – spatial filtering per channel

2. Pointwise convolution (1×1) – combines channel information

Purpose

• Dramatically reduces parameters and computation

• Widely used in mobile and lightweight models (e.g., MobileNet)

Comparison

Feature Standard Conv Dilated Conv Depthwise Conv Separable Conv

Receptive field Normal Larger Normal Normal

Parameters High Same as standard Very low Very low

Channel mixing Yes Yes No Yes (in pointwise step)

Speed Slower Moderate Fast Very fast

Applications

• Dilated: Semantic segmentation, audio processing

• Depthwise & Separable: Mobile vision models, real-time applications
Dilated convolution expands receptive fields without increasing parameters, depthwise
convolution reduces computation by processing channels separately, and separable convolution
combines depthwise and pointwise operations for highly efficient CNN architectures. These
variants enable faster and lighter deep learning models.

Popular CNN Architectures: LeNet, AlexNet, VGG, ResNet, EfficientNet

Convolutional Neural Network (CNN) architectures have evolved to improve accuracy, depth,
efficiency, and scalability. Each architecture introduced important design ideas that shaped
modern deep learning models used in image classification, detection, and recognition tasks.

LeNet (1998)

LeNet, developed by Yann LeCun, was one of the earliest CNN architectures designed for
handwritten digit recognition (MNIST).

Architecture

• Input → Convolution → Pooling → Convolution → Pooling → Fully Connected →

Output
• Uses tanh/sigmoid activations
• Small number of parameters

Key Features

• Introduced convolution + pooling structure

• Demonstrated CNN effectiveness in vision tasks

Limitations

• Shallow network
• Not suitable for complex image datasets

AlexNet (2012)
AlexNet by Krizhevsky et al. won the ImageNet competition and triggered the deep learning
revolution.

Architecture

• 5 convolutional layers
• 3 fully connected layers
• Uses ReLU activation/Soft Max
• Includes dropout and data augmentation

Key Features

• First deep CNN trained on GPUs

• Reduced overfitting using dropout
• Used large filters (11×11, 5×5)

Impact

• Proved deep CNNs outperform traditional methods

VGGNet

Developed by Oxford’s Visual Geometry Group (VGG), VGG focused on simplicity and
depth.

Architecture
 • Uses only 3×3 convolution filters
• Very deep (VGG-16, VGG-19)
• Stacks convolution layers before pooling

Key Features

• Uniform architecture design

• Strong feature extraction capability

Limitations

• Very large number of parameters

• High memory and computation cost

ResNet (2015)

ResNet (Residual Network) by Microsoft Research introduced skip connections, allowing

very deep networks.

Architecture
 • Residual blocks with identity shortcuts
• Can have 34, 50, 101, or 152 layers

Key Features

• Solves vanishing gradient problem

• Enables training of extremely deep networks
• Improves accuracy significantly

Innovation

Residual learning:

H(x)=F(x)+x

EfficientNet (2019)

EfficientNet by Google focuses on balancing network depth, width, and resolution using
compound scaling.

Architecture

• Uses MBConv blocks (Mobile inverted bottleneck)

• Applies squeeze-and-excitation (SE) modules
• Scales models from B0 to B7 efficiently

Key Features

• High accuracy with fewer parameters

• Optimized for both speed and performance

Advantage

Better accuracy-per-parameter ratio compared to earlier models

Comparison of Architectures

Model Depth Key Innovation Strength

LeNet Shallow Conv + Pooling Foundation CNN

AlexNet Deep ReLU, Dropout, GPU Breakthrough in deep learning

VGG Very deep Uniform 3×3 filters Strong feature extraction

ResNet Very deep Skip connections Trains ultra-deep networks

EfficientNet Optimized Compound scaling High accuracy with efficiency

Applications

• Image classification
• Object detection
• Face recognition
• Medical imaging
• Autonomous driving
• Surveillance systems

Recent Trends: Vision Transformers (ViTs) & Self-Supervised Learning

Recent advances in deep learning have introduced new paradigms beyond traditional CNNs.
Two important trends are Vision Transformers (ViTs) and Self-Supervised Learning (SSL),
which focus on better feature learning, scalability, and reduced dependence on labeled data.
These approaches are transforming modern computer vision systems.

Vision Transformers (ViTs)

Vision Transformers adapt the Transformer architecture (originally designed for NLP) to image
processing tasks. Instead of using convolutions, ViTs divide images into fixed-size patches,
embed them as tokens, and process them using self-attention mechanisms.
Working

1. Image is split into patches (e.g., 16×16).

2. Each patch is flattened and embedded into vectors.
3. Positional embeddings are added.
4. Transformer encoder layers apply self-attention.
5. Final representation is used for classification.

Advantages

• Captures global dependencies across entire image.

• Scales well with large datasets.
• Strong performance on complex vision tasks.

Limitations

• Requires large training data.

• Computationally expensive for small datasets.

Self-Supervised Learning (SSL)

Self-supervised learning is a learning paradigm where models learn representations from

unlabeled data by solving automatically generated tasks (pretext tasks), without manual
annotation.

Common Techniques

• Contrastive learning (SimCLR, MoCo)

• Masked image modeling (MAE)
• Rotation prediction
• Image reconstruction

Working

The model learns meaningful visual representations by predicting missing or transformed parts
of images, which can later be fine-tuned for downstream tasks.
Advantages

• Reduces dependence on labeled datasets.

• Learns robust and transferable features.
• Improves performance when labeled data is limited.

Importance in Modern Computer Vision

• ViTs provide an alternative to CNNs for modeling global context.

• SSL enables training on massive unlabeled image collections.
• Together, they improve performance in medical imaging, autonomous driving, and
large-scale vision systems.

Vision Transformers replace convolution with self-attention to capture global image

relationships, while Self-Supervised Learning allows models to learn powerful representations
from unlabeled data. These trends represent a major shift in modern deep learning, leading to
more scalable, data-efficient, and general-purpose vision models.