Deep Learning

Convolutional
Networks
Evolution of Deep Architectures

PRESENTED BY
Dhruv Sharma
2203424
Introduction to
CNNs
Convolutional Neural Networks (CNNs) are crucial in deep learning,
enabling advanced image recognition, object detection, and
applications in diverse fields like medical imaging and autonomous
systems.
Basic Architecture of CNNs

INPUT LAYER CONVOLUTION LAYER ACTIVATION FUNCTIONS

The input layer receives raw The convolution layer applies Activation functions, like ReLU,
image data, transforming it into a filters or kernels to the input data, introduce non-linearity into the
format suitable for processing by extracting essential features such network, enabling it to learn
subsequent layers, effectively as edges and textures, allowing the complex patterns by allowing
representing pixel values as model to recognize patterns certain signals to pass while
numerical arrays. effectively. suppressing others during training.
Convolution KERNELS AND FILTERS

Operation Kernels, or filters, slide over input images to extract

important features. This process enables the model to
recognize patterns, outlines, and textures essential for
classification tasks.

FEATURE EXTRACTION

Feature extraction involves capturing essential information

from images, allowing CNNs to identify objects and
patterns effectively. This stage is crucial for improving the
model’s accuracy and performance.
Pooling Layers
MAX POOLING AVERAGE POOLING

Max pooling selects the highest value from each feature Average pooling computes the average of values in each
map region, reducing data. feature map region for reduction.

DIMENSIONALITY REDUCTION OVERFITTING PREVENTION

Pooling layers significantly reduce dimensionality, leading By reducing data representation, pooling helps prevent
to decreased computational complexity and memory overfitting during the training of models.
usage.

IMPROVED PERFORMANCE

Effective pooling methods enhance model performance

by maintaining important features while reducing noise.
LeNet HISTORICAL SIGNIFICANCE

Architecture LeNet-5, developed by Yann LeCun, is considered the first

successful CNN for handwritten digit recognition, paving
the way for modern deep learning applications in image
processing.

ARCHITECTURE COMPONENTS

The architecture consists of two convolutional layers

followed by subsampling layers, culminating in fully
connected layers, enabling effective feature extraction and
classification for digit recognition tasks.

IMPACT ON CNNS

LeNet's influence on subsequent architectures is profound,

demonstrating the effectiveness of convolutional networks
and inspiring the development of deeper and more
complex models in computer vision.
AlexNet BREAKTHROUGH ARCHITECTURE

Overview AlexNet, developed by Alex Krizhevsky in 2012, significantly

advanced deep learning, winning the ImageNet
competition and showcasing the power of deep
convolutional networks.

RELU ACTIVATION

The introduction of the ReLU activation function allowed

AlexNet to overcome the vanishing gradient problem,
leading to faster training times and improved model
performance compared to traditional functions.

USE OF DROPOUT

AlexNet implemented dropout layers to reduce overfitting,

enhancing generalization and enabling the model to
maintain accuracy while utilizing a larger number of
parameters.
ZF-Net FEATURE VISUALIZATION

Architecture ZF-Net enhances AlexNet by introducing a method for

visualizing intermediate feature maps, allowing
researchers to better understand how CNNs extract
features from images during processing.

HYPERPARAMETER TUNING

Improved hyperparameter tuning techniques in ZF-Net

lead to better performance compared to AlexNet, enabling
more effective training and optimization of the network for
various tasks and datasets.
VGGNet LAYER CONFIGURATION

Architecture VGGNet employs a deep architecture with 16 to 19 layers,

primarily utilizing small 3x3 convolution filters, allowing for
improved feature extraction without increasing the number
of parameters significantly.

FEATURE EXTRACTION

The use of small filters enables VGGNet to capture fine

details in images, enhancing its ability to extract high-level
features effectively, thus improving performance in various
image classification tasks.

IMPACT ON CNNS

VGGNet set new benchmarks in image classification,

influencing subsequent architectures by demonstrating
the efficacy of deeper networks and the importance of
consistent filter size in convolutional layers.
GoogLeNet Overview

EFFICIENT COMPUTATION
INCEPTION MODULES
By leveraging parallel processing and dimensionality
GoogLeNet introduced Inception modules, allowing multiple reduction, GoogLeNet achieves efficient computation,
convolutional filters to process input simultaneously, significantly reducing the number of parameters while
enhancing feature extraction while maintaining enhancing the model's accuracy and performance on
computational efficiency throughout the network. complex datasets.

ARCHITECTURAL DEPTH IMAGENET SUCCESS

With a deep architecture of 22 layers, GoogLeNet Winning the ImageNet 2014 competition, GoogLeNet's
demonstrates a robust design that effectively captures innovative design and efficient structure set new
features at multiple scales, improving image classification benchmarks in deep learning, showcasing its influence on
and object detection tasks. subsequent CNN architectures and research advancements.
ResNet Architecture

RESIDUAL LEARNING SKIP CONNECTIONS

Residual learning addresses the vanishing gradient Skip connections allow gradients to flow through layers
problem, enabling networks to learn residual mappings without diminishing, facilitating effective learning in very
instead of direct mappings, enhancing training for deeper deep architectures, thus improving performance and
networks. convergence rates.

DEEP NETWORKS PERFORMANCE GAINS

ResNet facilitates the construction of extremely deep With innovations in architecture, ResNet achieves state-
networks (over 100 layers) by utilizing shortcuts, of-the-art results on various benchmarks, demonstrating
significantly improving accuracy on challenging tasks like the effectiveness of residual learning in advancing deep
image classification. learning capabilities.
Comparison
of LeNet: Early CNN, 5 layers, 60K
parameters, 99.2% accuracy on MNIST.

Architectures AlexNet: 8 layers, 60M parameters,

84.7% accuracy on ImageNet.
ZF-Net: Improved hyperparameters, 8
layers, 14.8% error rate.
VGGNet: 16/19 layers, 138M parameters,
92.7% accuracy.
GoogLeNet: 22 layers, 5M parameters,
6.7% error rate.
ResNet: 152 layers, 25M parameters,
3.57% error rate.
Applications of CNNs

AUTONOMOUS VEHICLES MEDICAL IMAGING OBJECT DETECTION

Advanced Technology Diagnostic Analysis Real-Time Processing

The illustration highlights how CNNs enable This image depicts the application of CNNs The artwork illustrates the powerful use of
autonomous vehicles to recognize in medical imaging, allowing for accurate CNNs for real-time object detection in
surrounding objects and navigate safely. diagnosis and analysis of health conditions. dynamic environments, enhancing safety
and efficiency.
Advantages and Limitations of
CNNs
STRENGTHS OF CNNS COMPUTATIONAL CHALLENGES

Convolutional Neural Networks excel in automatic feature Despite their strengths, CNNs require significant
extraction, allowing them to learn rich representations computational resources for training large models. This
from raw data. This capability reduces the need for can lead to long training times and the necessity for
manual feature engineering, making CNNs highly effective specialized hardware, such as GPUs, to handle complex
in tasks like image recognition and classification. architectures efficiently.