BCSE332L-Deep Learning

Dr. Rohith G
Assistant Professor Senior, CAMIT
School of Electronics Engineering (SENSE)
Contents
• Module-1: Introduction to neural networks and deep neural
networks
• Module-2:Convolution neural networks
• Module-3:Improving deep neural networks
• Transfer Learning- Transfer
• Module-4:Recurrent networks Learning Models
• Generative Adversarial
• Module-5:Recursive neural networks Network and their variants
• Module-6:Advanced Neural networks • Region based CNN
• Fast RCNN
• Module-7:Deep reinforcement learning • You Only Look Once
• Single shot detector
 Module-6; Topic-1; Transfer Learning- Transfer
Transfer Learning Learning Models

Transfer learning is a technique where a model trained on one task is reused as a starting point for a model on a
different but related task, leveraging previously learned knowledge to improve performance and efficiency.

•Why it's useful:

•Reduced Training
Time: Instead of training
a model from scratch,
you can fine-tune a pre-
trained model, saving
significant time and
computational resources.

•Improved Performance: Pre-trained models have already learned useful features, which can lead to better
performance on the new task, especially when the target dataset is small.
•Handles Limited Data: When you have limited data for a new task, transfer learning can be particularly effective, as
the pre-trained model can provide a good starting point.
3
 Module-6; Topic-1; Transfer Learning- Transfer
Transfer Learning Learning Models

How Does Transfer Learning Work?

Transfer learning involves a structured process to leverage existing
knowledge from a pre-trained model for new tasks:
• Pre-trained Model: Start with a model already trained on a
large dataset for a specific task. This pre-trained model has
learned general features and patterns that are relevant across
related tasks.
• Base Model: This pre-trained model, known as the base model,
includes layers that have processed data to learn hierarchical
representations, capturing low-level to complex features.
• Transfer Layers: Identify layers within the base model that
hold generic information applicable to both the original and new Low-level features learned for task A should be
tasks. These layers, often near the top of the network, capture beneficial for learning of model for task B.
broad, reusable features.
• Fine-tuning: Fine-tune these selected layers with data from the
new task. This process helps retain the pre-trained knowledge
while adjusting parameters to meet the specific requirements of
the new task, improving accuracy and adaptability.

4
 Module-6; Topic-1; Transfer Learning- Transfer
Transfer Learning Learning Models

Frozen vs. Trainable Layers in Transfer Learning

In Transfer Learning, two main components help in adapting
models effectively:
• Frozen Layers: These layers from a pre-trained model
remain unchanged during fine-tuning. They retain general
features learned from the original task, extracting universal
patterns from input data.
• Trainable Layers: These layers are adjusted during fine-
tuning to learn task-specific features from the new dataset,
allowing the model to meet the new task’s unique
requirements.

5
 Module-6; Topic-1; Transfer Learning- Transfer
Transfer Learning Learning Models

How to Decide Which Layers to Freeze or Train?

• The extent to which you freeze or fine-tune layers
depends on the similarity and size of your target dataset:
• Small, Similar Dataset: For smaller datasets that
resemble the original dataset, you freeze most layers and
only fine-tune the last one or two layers to prevent
overfitting.
• Large, Similar Dataset: With large, similar datasets,
you can unfreeze more layers, allowing the model to
adapt while retaining learned features from the base
model.
• Small, Different Dataset: For smaller, dissimilar
datasets, fine-tuning layers closer to the input layer
helps the model learn task-specific features from
scratch.
• Large, Different Dataset: In this case, fine-tuning the
entire model helps the model adapt to the new task
while leveraging the broad knowledge from the pre-
trained model.
6
 Module-6; Topic-1; Transfer Learning- Transfer
Transfer Learning Learning Models
Advantages of Transfer Learning
•Speed up the training process: By
using a pre-trained model, the model
can learn more quickly and effectively
on the second task, as it already has a
good understanding of the features and
patterns in the data.
•Better performance: Transfer learning
can lead to better performance on the
second task, as the model can leverage
the knowledge it has gained from the
first task.
•Handling small datasets: When there
is limited data available for the second
task, transfer learning can help to
prevent overfitting, as the model will
have already learned general features
that are likely to be useful in the second
task.

7
 Module-6; Topic-1; Transfer Learning- Transfer
Transfer Learning Learning Models

Disadvantages of Transfer
Learning
•Domain mismatch: The pre-trained
model may not be well-suited to the
second task if the two tasks are vastly
different or the data distribution
between the two tasks is very different.
•Overfitting: Transfer learning can lead
to overfitting if the model is fine-tuned
too much on the second task, as it may
learn task-specific features that do not
generalize well to new data.
•Complexity: The pre-trained model
and the fine-tuning process can be
computationally expensive and may
require specialized hardware. Transfer learning enhances model performance by leveraging
knowledge from previously trained models. By starting with pre-
existing models and fine-tuning them for specific tasks, transfer learning
saves time, improves accuracy, and enables effective learning even with
limited data.
8
 Module-6; Topic-1; Transfer Learning- Transfer
Applications of Transfer Learning Learning Models

9
 Module-6; Topic-1; Transfer Learning- Transfer
Transfer Learning Learning Models
Types of transfer learning
- Inductive transfer
• Multitask learning, a form of
inductive transfer, involves learning
two tasks simultaneously on the
same dataset, such as image
classification and object detection,
despite differences in target and
source domains.
- Unsupervised learning.
• Unsupervised transfer learning,
similar to inductive transfer, is
unsupervised, without manually
labeled data. It's often used for fraud - Transductive transfer
detection by identifying common • Domain adaptation is a form of transductive learning where the
patterns across an unlabeled source and target tasks are the same but the datasets are
dataset, allowing models to learn different.
and identify deviating behaviors. • It applies knowledge from one data distribution to another, like
text classification.
10
 Module-6; Topic-1; Transfer Learning- Transfer
Transfer Learning Learning Models
Transfer learning versus fine-tuning

• Transfer learning and fine tuning are two methods of training machine learning models.
• Fine tuning involves training a model on a specific dataset to improve performance on the initial task, while
transfer learning involves adapting a model to a new, related problem.
• Both methods reuse preexisting models.
11
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants Learning Models

12
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants Learning Models

13
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants Learning Models

14
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants Learning Models

15
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants Learning Models

16
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants Learning Models

17
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants-GoogleNet Learning Models

18
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants-GoogleNet Learning Models

19
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants-ResNet-50 Learning Models

20
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants-Xception V3 Learning Models

21
 Module-6; Topic-1; Transfer Learning- Transfer
CNN Variants-Inception-ResNet Learning Models

22
 Module-6; Topic-2; GAN
GAN
A Generative Adversarial
Network (GAN) is a type of
neural network architecture used
for generative modeling. It
consists of two competing
networks:
[Link] (G) → Creates fake
samples from random noise.
[Link] (D) → Evaluates
whether the sample is real or fake.
🎯 The two networks play a min-
max game:
•The Generator tries to fool the
Discriminator by creating
realistic-looking fake data.
•The Discriminator tries to
differentiate real from fake
samples.

23
 Module-6; Topic-2; GAN
GAN
2. GAN Architecture
Input:
•The Generator takes a random noise vector (e.g.,
sampled from a Gaussian distribution) and generates a
synthetic sample.
Training Process:
•The Discriminator is trained on:
• Real samples (from the training set).
• Fake samples (from the Generator).
•The Generator is trained to fool the Discriminator.

24
 Module-6; Topic-2; GAN
GAN
GANs have evolved into several variants to address issues like mode collapse, training instability, and poor diversity.

(1) DCGAN – Deep Convolutional GAN

•Introduced by Radford et al.
•Uses convolutional layers instead of fully
connected layers in the Generator and
Discriminator.
•Includes batch normalization and Leaky
ReLU activations.
•More stable and generates higher-quality
images.
✅ Applications:
•Generating realistic images
•Image-to-image translation

25
 Module-6; Topic-2; GAN
GAN
GANs have evolved into several variants to address issues like mode collapse, training instability, and poor diversity.

(2) WGAN – Wasserstein GAN

•Introduced by Arjovsky et al.
•Uses the Wasserstein distance
instead of JS divergence for
better convergence.
•Replaces the sigmoid output in
the discriminator with a linear
activation.
•Uses weight clipping to enforce
the Lipschitz constraint.

26
 Module-6; Topic-2; GAN
GAN
GANs have evolved into several variants to address issues like mode collapse, training instability, and poor diversity.

(3) WGAN-GP – Wasserstein GAN with Gradient Penalty

27
 Module-6; Topic-2; GAN
GAN
(4) LSGAN – Least Squares GAN

28
 Module-6; Topic-2; GAN
GAN
(5) CGAN – Conditional GAN

29
 Module-6; Topic-2; GAN
GAN
(6) InfoGAN – Information Maximizing GAN

30
 Module-6; Topic-2; GAN
GAN
(7) CycleGAN

31
 Module-6; Topic-2; GAN
GAN
(8) StyleGAN

32
 Module-6; Topic-2; GAN
GAN
(9) BigGAN

BigGAN

A residual block in BigGAN 33

 Module-6; Topic-2; GAN
GAN (10) SRGAN – Super-Resolution GAN

34
 Module-6; Topic-2; GAN
GAN

35
 Module-6; Topic-3; Region based CNN
Region based CNN
R-CNN (Region-Based Convolutional Neural Network) is a deep learning architecture designed for object detection. It
combines:
•Selective Search for identifying regions of interest (ROIs)
•CNN for feature extraction
•SVM (Support Vector Machines) for classification
•Bounding box regression for precise object localization.
Purpose:
R-CNN was introduced to solve the object detection problem by first identifying proposed regions in an image and then
classifying them using CNN-based features.

36
 Module-6; Topic-3; Region based CNN
Region based CNN-Architecture
Input Image
Step 1: Region Proposal (Selective Search)
↓
•The image is divided into regions using a technique called
Selective Search
Selective Search, which generates around 2000 region
↓
proposals.
Region Proposals (ROIs)
•Selective Search is a segmentation algorithm that groups
↓
similar pixels together into candidate regions.
Resize ROIs to a fixed size
•These regions are called Region of Interests (ROIs).
↓
•Each region is treated as a separate image patch.
Pass through CNN for feature extraction
Step 2: Feature Extraction with CNN
↓
Each ROI is resized to a fixed size (e.g., 224×224) and passed
Feature vectors for each region
through a pre-trained CNN (such as AlexNet, VGG16, or
↓
ResNet).
SVM classifiers for object classification
The CNN extracts feature vectors from each region.
↓
These feature vectors are of fixed length (e.g., 4096 dimensions
Bounding Box Regression for localization
for VGG16).
↓
Final object detection with bounding boxes

37
 Module-6; Topic-3; Region based CNN
Region based CNN-Architecture

Step 3: Classification with SVM

• For each region, the extracted features are passed to multiple SVM classifiers, one for each object class.
• The SVMs predict the object class or background (no object).
• Multiple binary SVM classifiers are used, one for each class.
Step 4: Bounding Box Regression
• To improve the accuracy of object localization, R-CNN uses a bounding box regressor.
• This is a separate linear regression model that predicts the precise coordinates of the object bounding box.
• The goal is to refine the position of the detected object and reduce localization error.
38
 Module-6; Topic-3; Region based CNN
Region based CNN-Architecture

Key Challenges of R-CNN

Advantages and Disadvantages of R-CNN
•Slow Inference:
Advantages:
• It is computationally expensive and unsuitable for
✔️ Higher accuracy compared to traditional object detection
real-time applications.
algorithms.
•Redundant Computation:
✔️ Uses deep learning features that are more robust than
• Since overlapping regions share similar features,
handcrafted features.
R-CNN recomputes the same features multiple
✔️ Improved performance with bounding box regression.
times, leading to inefficiency.
Disadvantages:
•High Storage Requirement:
❌ Slow inference due to repeated CNN computations.
• R-CNN uses SVMs for classification, which
❌ Requires large storage space for feature extraction.
requires a large amount of disk space to store the
❌ Not suitable for real-time applications. 39
extracted features.
 Module-6; Topic-4; Fast RCNN
Fast RCNN
Fast R-CNN is an improvement over R-CNN for object
detection.
Why Fast R-CNN?
R-CNN has several inefficiencies, mainly because it:
Performs CNN computations separately for each
region of interest (ROI), leading to redundant
operations.
Uses SVMs for classification and separate
bounding box regressors, which adds complexity.
Is slow and memory-intensive, making it
impractical for real-time applications.
✅ Fast R-CNN solves these problems by introducing:
Single CNN pass over the entire image.
ROI Pooling layer to extract features from the shared
feature map.
Multi-task loss to jointly train classification and
bounding box regression.

40
 Module-6; Topic-4; Fast RCNN
Fast RCNN
Step 3: ROI Pooling Layer
The ROI Pooling layer is a key innovation in Fast R-CNN.
It converts ROIs of different sizes into a fixed-size feature
map (e.g., 7×7).
This is done by:
Dividing the ROI into a fixed grid (e.g., 7×7 cells).
Applying max pooling to each grid cell.
This ensures that all ROIs produce feature vectors of the
same size, regardless of their original dimensions.
Step 4: Fully Connected Layers
The output of the ROI pooling layer is flattened and passed
through fully connected (FC) layers.
These layers produce feature vectors that are used for:
Classification: Determining the class of the object.
Bounding Box Regression: Refining the coordinates of
the object’s bounding box.

41
 Module-6; Topic-4; Fast RCNN
Fast RCNN
Architecture of Fast R-CNN
Fast R-CNN has the following components: Input Image
🔹 Step 1: Input Image and Feature Extraction ↓
The entire image is passed through a pre-trained CNN (e.g., CNN Feature Extraction
VGG16, ResNet). ↓
The CNN extracts a feature map of the entire image. Region Proposals (Selective Search)
This is a major improvement over R-CNN, as the feature ↓
extraction step is performed only once instead of separately for Map ROIs to Feature Map
↓
each region.
ROI Pooling Layer
Step 2: Region of Interest (ROI) Proposals ↓
Instead of using Selective Search separately, Fast R-CNN uses it Fully Connected Layers
to generate around 2000 region proposals from the image. ↓
These region proposals (ROIs) are mapped onto the shared Classification (Softmax) &
feature map. Bounding Box Regression (Smooth L1)
Each ROI corresponds to a section of the feature map, not the ↓
original image. Final Object Detection with Bounding Boxes
This reduces redundant CNN computations significantly.

42
 Module-6; Topic-4; Fast RCNN
Fast RCNN
Faster R-CNN: The Successor
To overcome the limitations of
Fast R-CNN, Faster R-CNN was
introduced. It uses:
•Region Proposal Network
(RPN) to generate ROIs instead
of Selective Search.
•Fully trainable end-to-end
architecture, making it faster and
more accurate.
•Real-time inference capability.

Applications of Fast R-CNN

Fast R-CNN is used in various object detection applications, such as:
🔹 Autonomous Vehicles:
•Detecting cars, pedestrians, and obstacles.
🔹 Medical Imaging:
•Detecting tumors and abnormalities in medical scans.
🔹 Surveillance Systems:
•Detecting and tracking objects (e.g., people, vehicles) in real-time surveillance footage.
🔹 Facial Recognition:
•Identifying and verifying faces. 43
 Module-6; Topic-4; Fast RCNN
Fast RCNN
Advantages of Fast R-CNN
✔️ Speed and Efficiency: 6. Challenges of Fast R-CNN
Fast R-CNN is 10 times faster than R-CNN during training. ❌ Reliance on Selective Search:
It is 100 times faster during inference. •Although faster than R-CNN, Fast R-CNN still
✔️ End-to-End Training: uses Selective Search for region proposals,
The entire model is trained together using a single loss which is slow and not trainable.
function. •Selective Search limits the real-time
✔️ Efficient Region Proposals: performance of Fast R-CNN.
ROIs are applied directly to the shared feature map, making ❌ Inference Time:
the process efficient. •While faster than R-CNN, it is still not fast
✔️ Better Accuracy: enough for real-time applications.
Improved accuracy due to joint optimization of classification
and bounding box regression.
Faster R-CNN: The Successor
To overcome the limitations of Fast R-CNN, Faster R-CNN was introduced. It uses:
•Region Proposal Network (RPN) to generate ROIs instead of Selective Search.
•Fully trainable end-to-end architecture, making it faster and more accurate.
•Real-time inference capability.

44
 Module-6; Topic-5; YOLO
You Only Look Once (YOLO)
• You Only Look Once (YOLO) is a
popular real-time object detection
system that has been widely used in
computer vision tasks.
• The main goal of YOLO is to detect
and classify objects in images or video
streams with high speed and accuracy.
• It stands out from other object
detection algorithms because of its
unique approach: rather than
performing object localization and
classification separately (as many
traditional algorithms do), YOLO does
it in a single pass.

How YOLO Works:

•Single Pass: The image is processed once by the model, and it outputs predictions for all objects in the scene
(class and bounding box).
•Bounding Boxes: YOLO predicts multiple bounding boxes and assigns a class to each. It also calculates a
confidence score that indicates how certain it is that the box contains an object of a particular class.
45
 Module-6; Topic-5; YOLO
You Only Look Once (YOLO)

46
 Module-6; Topic-5; YOLO
You Only Look Once (YOLO)
Key Features:
[Link] Architecture: YOLO treats object
detection as a single regression problem, which
means it predicts bounding boxes and class
probabilities in one go, making the algorithm
very fast.
[Link]-based Prediction: The input image is
divided into an S×S grid, where each grid cell
is responsible for detecting objects whose
centers fall within that cell. Each grid cell
predicts:
1. The bounding box (center coordinates,
width, height).
2. The confidence score of the bounding
box.
3. The class probabilities (e.g., dog, cat,
car, etc.). 4. Accuracy: While YOLO is known for its speed, it also achieves a
[Link]: Because of its unified architecture, good balance between accuracy and performance.
YOLO can process images quickly, making it • It's not as accurate as slower algorithms like R-CNN or Fast R-
suitable for real-time applications like video CNN but is still highly effective for many real-time applications.
surveillance or autonomous vehicles.
47
 Module-6; Topic-5; YOLO
You Only Look Once (YOLO)

48
 Module-6; Topic-5; YOLO
You Only Look Once (YOLO)

49
 Module-6; Topic-5; YOLO
You Only Look Once (YOLO)
YOLO Versions Evolution:
Each version of YOLO
improved upon the original,
addressing limitations such as
small object detection, accuracy,
and efficiency.
1.YOLOv1: The original
YOLO, which introduced the
unified detection approach.
2.YOLOv2 (Darknet-19):
Improved accuracy and
performance with better training
techniques.
3.YOLOv3: Introduced multi-
scale predictions, more anchors,
and improvements in handling
small objects.
4.YOLOv4: Focused on
boosting both speed and
accuracy by incorporating 5. YOLOv5: While not officially part of the original YOLO series,
various modern deep learning YOLOv5 became a popular and optimized version for both speed and
accuracy. 50
tricks.
 Module-6; Topic-5; YOLO
You Only Look Once (YOLO)

51
 Module-6; Topic-5; YOLO
You Only Look Once (YOLO)
YOLO Applications:
Applications of YOLO:
•Autonomous Vehicles:
Detecting obstacles,
pedestrians, and other
vehicles in real-time.
•Surveillance: Detecting
suspicious activities,
people, or objects.
•Retail: Identifying
products on shelves or in
warehouses.
•Healthcare: Detecting
and classifying medical
images, such as tumors or
fractures.

52
You Only Look Once (YOLO) Module-6; Topic-5; YOLO
Loss Function

53