2024 First International Conference on Software, Systems and Information Technology (SSITCON)

Exploring Faster R-CNN Algorithms for Object

Detection
2024 First International Conference on Software, Systems and Information Technology (SSITCON) | 979-8-3503-5293-1/24/$31.00 ©2024 IEEE | DOI: 10.1109/SSITCON62437.2024.10796389

1st Abhishek Thoke 2nd Sakshi Rai

Department of Computer Science and Engineering Department of Computer Science and Engineering
LNCT University LNCT University
Indore, India Bhopal, India
[Link]@[Link] sakshirai.sr29@[Link]

Abstract—The task of object detection is widely recognised 1) Comparative Analysis of Multiple R-CNN Variants.
as a prominent challenge in computer vision applications,
particularly in the domains of robotics, autonomous vehicles, 2) Exploration of Few-Shot and Global Context
and augmented reality. Hence, this paper examines Faster R- Detection Models.
CNN, a well embraced object identification model that 3) Model Performance Evaluation on COCO Dataset.
effectively combines speed and accuracy. The authors evaluate
several novel models, such as Sparse R-CNN, Cascade R-CNN, The paper is structured as follows: Section three presents
Loss-Guided Attention RCNN, Global Context Aggregation an overview of various Faster R-CNN-based algorithms for
RCNN, and others, to demonstrate their unique features and object detection, including DEFRCN, HR-RCNN, iFS-
benefits in addressing the challenges of dense object candidates RCNN, Global Context Aware RCNN, LGA-RCNN, Cascade
and limited training data. This paper presents a performance R-CNN, and Sparse R-CNN. Section four offers a
evaluation of these models on the COCO dataset, elucidating comparative analysis of the performance of these algorithms.
their accuracy and usefulness for different object detecting Finally, section five concludes the study by summarizing the
applications. The authors conclude that the Sparse R-CNN key findings and suggesting future research directions to
model outperforms other models in terms of average precision. improve object detection performance.
They also offer recommendations for future optimisation of
object identification models in terms of architecture to better II. LITERATURE SURVEY
suit the demands of real-time processing.
Traditional object detection methods often face
Keywords—sparse R-CNN, cascade R-CNN, LGA-RCNN, limitations, such as slow processing speeds and difficulty in
GCA RCNN, iFS-RCNN, DeFRCN, and HR RCNN. handling complex scenarios. Recent innovations have led to
more efficient models, improving both accuracy and speed.
I. INTRODUCTION These advancements address various limitations, offering
The computer vision is a rapidly evolving field with enhanced adaptability. To address these challenges, various
significant advancements in recent years. Nowadays computer object detection methods have been presented by researchers
vision is used in every domain like medical, agriculture, to tackle these problems, each possessing distinct features and
transport etc. [1-3]. The latest developments in computer benefits. In this review paper, it discusses seven such models,
vision techniques have brought about faster and more accurate namely Sparse R-CNN [9,10], Cascade R-CNN [11], Loss-
object detection models [4]. With the growing need for Guided Attention RCNN (LGA-RCNN) [12], Global Context
efficient and reliable object detection, the latest Faster R- Aggregation RCNN (GCA RCNN) [13], IFS-RCNN [14],
CNN- based models have become an important area of DEFRCN [15], and HR RCNN [16]. Sparse R-CNN [9] was
research [5,6]. The origin of Faster R-CNN [7,8] can be traced developed to rethink the necessity of dense prior in object
back to the need for faster and more accurate object detection. detection, which suffers from limitations such as repetitious
Since its inception, Faster R-CNN [5,6] has become a popular results, heuristic assignment rules, and sensitivity Regarding
choice among researchers due to its ability to detect objects the dimensions and shape of anchor boxes. The authors hope
with high accuracy while also maintaining a fast detection that their work could inspire exploring the next generation of
speed. Its development has undoubtedly helped move the field object detectors. Cascade R-CNN [11] was developed to
of computer vision forward. This review paper discusses the investigate The difficulty of creating precise object algorithms
latest Faster R-CNN-based models and their applications. By detectors that generate minimal false positives in close
examining these models, it hopes to provide a comprehensive proximity. The model has several stages, each with a higher
wisdom of the existing shape of object detection in computer IOU threshold, to improve detection accuracy. The LGA-
vision and the potential impact it can have on diverse RCNN [17] model integrates a loss-guided attention
industries. mechanism to emphasize discriminative regions of objects
and improve detection performance. GCA RCNN [13] was
Object detection is a fundamental requirement for many developed to extract global features for the roi head, in which
computer vision applications that have found applications in Inputs are cropped from the overall feature map in a partial
various fields, including robotics, autonomous vehicles, and manner. The model fuses global context and local features to
augmented reality. Convolutional-neural-networks have boost and polish global context information DEFRCN [15] is
shown remarkable progress in object detection in recent times a model that has been proposed to overcome the shortcomings
[5]. However, traditional object detection methods have of Faster R-CNN, such as its failure to take into account few-
several limitations that pose a challenge for object detection in shot scenarios and conflicts between its components, in order
complex scenarios. In this work, we present a comprehensive to enhance the performance of few-shot object identification.
review of the different object detection model. The These tasks become challenging when there are few training
contribution of the work are as follow: examples available for new classes or When training on both

979-8-3503-5293-1/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:56:06 UTC from IEEE Xplore. Restrictions apply.
base and new classes is not feasible due to time constraints. simultaneously identifying and refining a group of anchors
To address these challenges, the IFS- RCNN [14] model was with varying scales from the feature map. Next, RCNN
developed to learn from a infrequent training instances of new conducts a box classifier and a regressor for enhancing the
classes while not forgetting the previously learned knowledge item classification probabilities and bounding box refinement
of the base classes. On the other hand, the Hierarchical based on the collective feature map and suggestions and RoI
Relational framework for object detection HRRCNN [16] was pooling, which combines individual region-of-interest into
developed to rectify the shortcomings of convolutional- such a set dimension of the feature map.
neural-networks(convnets) in explicitly modeling as well as
reasoning about contextual relationships in images. The B. DEFRCN: A Few-Shot Object Detection Method Based
HRRCNN model integrates three types of contextual on Decoupled-Faster-RCNN
relationships - pixel connections, scale dependencies, and Current detection frameworks, such as Faster R-CNN,
object associations in a unified model to improve object often need help in scenarios with limited data and when
detection performance. This review paper explores these dealing with multi-task learning [15] and shared backbone
models and their causes for development in greater detail. challenges. It proposes a new, efficient architecture called
DEFRCN to address these issues. In particular, It enhances
III. DIFFERENT METHODOLOGY FOR OBJECT DETECTION Faster R-CNN by incorporating a Gradient-Decoupled-
To address the identified challenges in object detection, Layer(GDL) for multistage separation and a Prototypical-
particularly the inefficiencies in speed and accuracy, this Calibration-Block(PCB) for multitask separation. The model
paper explores state-of-the-art Faster R-CNN models. By is illustrated in the accompanying Figure 1. The Gradient-
enhancing feature extraction through region proposal Decoupled-Layer is an innovative deep layer that alters the
networks and incorporating novel components like multi-task feature propagation and gradient computation procedures to
separation layers, this methodology aims to provide a robust separate it is preceding and succeeding layers. The
framework for precise and efficient object detection. The Prototypical-Calibration-Block is a classification model based
following sections elaborate on the design, implementation, on an offline prototypes model that utilizes proposals from the
and evaluation of these models, demonstrating how they detector as information and improves the calibrating
overcome the existing limitations. Table 1 presents the classification scores using pairwise comparisons
development years of various object detection models. C. HRRCNN: Novel Hierarchical-Relational-Reasoning
A. Revisiting Faster R-CNN Approach
Faster R-CNN functions as a two-stage detection Extraction of feature pyramids is accomplished through a
framework, consisting of three crucial elements for smooth backbone network and generate region proposals for an
end-to-end training: a combined convolutional backbone that image. Instead of processing features computed for each
extracts generalized features, a Region Proposal Network region of the image separately in a box head, in-between the
(RPN) that effectively produces proposals that are not feature pyramid and box head, the HR-RCNN introduces a
dependent on the class, and an RCNN head that is specifically hierarchical relational reasoning [16] (HR) component. This
configured to handle classification and localization tasks that component retains three relational reasoning elements: a pixel,
are specific to the class [9, 10]. More specifically, after the scaled, and region- of-interests graph. To make the model
backbone processes the input image and generates a high-level more efficient, It uses a novel graph-attention-module (GAM)
feature map, it is simultaneously sent to both the Region that can gather intake from different types of heterogeneous
Proposal Network (RPN) and the RCNN modules. These graphs by computing attention weight based on the
work in tandem to find probable object regions and refine the quantifying the semantic [18] and spatial proximities of nodes.
categorization and localization tasks. Second, the RPN
generates a limited set of high-quality region proposals by

Fig. 1. The architecture of DeFRCN

The subsequent Figure 2 depicts the given model. This

empowers the model to yield a more sophisticated rendering
of the image.

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:56:06 UTC from IEEE Xplore. Restrictions apply.
Fig. 2. The HR-RCNN structure

D. iFS RCNN: Segmentation of Few-Shot Instances with E. Global Context Aware RCNN for Object Detection
Incremental Learning The proposed system is contextually conscious and
In order to ensure optimum efficiency on all old and fresh facilitates the integration of global context information with
classes, this model presents a technique for incremental few- specific information from Regions of Interest (ROIs) in a two-
shot instance segmentation in which new object classes are stage object detection network. In traditional two-stage
presented when training instances of old classes are not there. methods, the initial stage (rpn head) separates the foreground
The procedure constructs the Mask-RCNN framework by from the background and forecasts the anchor box’s
providing an untried class classification established on the regression coefficient.
probit operation, which addresses the lack of training
TABLE I. FASTER R-CNN ALGORITHMS
examples for new classes through Bayesian learning [14]. The
paper proposes a novel bounding box predictor guided by [Link] Topic Year
uncertainty estimation. It estimates the uncertainty in 1 DeFRCN: (A Few-Shot Object Detection Method 2021
localization on new classes and uses it to refine bounding box based on Decoupled-Faster-RCNN)
2 HRRCNN: (Novel Hierarchical-Relational- 2021
predictions and height the loss. It feeds a new bounding-box Reasoning Approach)
of the calculated uncertainty together with the ROI-aligned 3 iFS RCNN (Segmentation of Few-Shot Instances 2022
pooled feature map. The segmentation head then receives the with Incremental Learning)
refined bounding box. A new loss is created to penalize errors 4 Global Context Aware RCNN for Object Detection 2020
on training examples with specific bounding-box predictions 5 LGA-RCNN: (Loss-Guided Attention for Object 2021
more severely by being designed to be less for extremely Detection)
6 Cascade R-CNN 2017
uncertain predictions described in Figure 3. 7 Sparse R-CNN 2021

Fig. 3. The iFS-RCNN structure

The second stage (roi head) forecasts the specific category

of RoI and computes to enhance the precision of the bounding
box. An offset value is employed. It fine-tunes the features
using global statistics for the rpn head to enhance the
network’s global feature recognition capabilities.
Nevertheless, implementing a more elaborate design for
extracting global features for the cropped features of the full
feature map is employed in the R0I head. Our method employs
the global context information extracted from diverse levels
of the fea- ture pyramid in the model by merging them through
thick connections in the R0I head, generating higher-
dimensional global descriptors using the module designed to
incorporate contextual information, as illustrated in Figure 4. Fig. 4. Visual depiction of GCA RCNN
Furthermore, like FPN [19], It uses shared, fully connected
several layers to extract features at different processing stages. F. LGA-RCNN: Loss-Guided Attention for Object
Moreover, ultimately merge prediction information from Detection
these stages to make the final decision.
LGA-RCNN introduces a novel approach by integrating a
Loss-Guided Attention (LGA) module. This module identifies
significant regions within an object, merging them with global
context to enhance both classification and localization

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:56:06 UTC from IEEE Xplore. Restrictions apply.
performance. The LGA module leverages a k Gaussian By combining highlighted features with the first RoI feature,
architecture to generate masks that highlight the most an improvement in both classification and regression is
informative regions in the R0I feature maps. It is supervised achieved, as shown in Figure 5. The fused RoI feature maps
by an additional classification loss to ensure optimal locations. combine local and global information to improve detection
LGA modules are employed to predict rep- presentative results, and the Gaussian masks [17] also focus on the
regions and highlight them to enhance classification accuracy. marginal regions of the object to enhance location accuracy.

Fig. 5. Illustration of LGA R-CNN

G. Cascade-RCNN: SOTA Object Detection TABLE II. COMPARISON OF THE METHODS

A novel detector architecture called Cascade R-CNN has Methods AP AP50 AP75 APs APm APl
been presented to deal with the problem of precise object DeFRCN 41.9 62.3 45.1 22.3 46.6 57.8
detection. It is an R-CNN extension with many stages, with DeFRCN 36.5 57.6 39.2 19.8 41.7 50.3
detector stages further down the cascade, more discriminating + GDL
HR-RCNN 47.7 68.2 51.7 30.8 50.4 59.4
against nearby false positives. Resampling progressively (R101-DCN-FPN)
improves the quality of hypotheses, ensuring a balanced HR-RCNN 44.9 65.1 48.4 26.7 47.6 56.5
positive training set for each detector and mitigating (R101-FPN)
overfitting. The subsequent R-CNN stage in the cascade GCA-RCNN 42.1 63.0 45.9 24.4 45.2 53.2
utilizes the output of the previous stage for training. Three Double-Head (GCA)
acute effects of this cascade learning on detector training are GCA-RCNN 40 61.6 43.5 22.8 43.2 50.3
Cascade-RCNN with 42.8 62.1 46.3 23.7 45.5 55.2
decreased likelihood of overfitting at high IoU thresholds R1O1
[11], the optimality of deeper stage detectors for higher IoU Sparse-RCNN with 46.9 66.3 51.2 28.6 49.2 58.7
thresholds, and an increase in the understanding efficiency of RX1O1
Regression of bounding box parameters are enhanced in the Sparse-RCNN with 48.9 68.3 53.4 29.9 50.9 62.4
later stages of the model. Refer to Figure 6 below for a RX1O1 and DCN
depiction of the model. Sparse-RCNN with 51.5 71.1 57.1 34.2 53.4 64.1
RX1O1 and DCN X
H. Sparse RCNN
Sparse R-CNN is a method for detecting entities in
illustrations that strictly utilizes sparsity.

Fig. 6. A graphical representation of Cascade R-CNN

In several works, the emphasis on dense object candidates Rather than generating dense object candidates, it employs a
is pronounced in object detection. Each grid of the image sparse set of learned object proposals with a fixed length of N
feature map with dimensions H x W contains a predetermined to feed into the object recognition head for object recognition
set of k anchor boxes. To perform classification and location, purposes. Figure 7 below illustrates the model. In Sparse R-

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:56:06 UTC from IEEE Xplore. Restrictions apply.
CNN, all efforts associated with the design and assignment of utilizing a fixed set of N (100) learnable propositions instead.
labels to object candidates are completely avoided by Final predictions are immediately output without unnecessary
restricting the number of handcrafted object candidates and post-procedure suppressions.

Fig. 7. Sparse RCNN in a nutshell

[6] Santoro, Adam, David Raposo, David G. Barrett, Mateusz Malinowski,

IV. COMPARISON OF THE METHODS Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. ”A simple
Table 2 compares several models established on Faster R- neural network module for relational reasoning.” Advances in neural
information processing systems 30 (2017).
CNN for object recognition tested on the COCO [20] dataset.
[7] Girshick, Ross, Jeff Donahue, Trevor Darrell, and Jitendra Malik.
The measurement metrics used are average precision [20, 21] ”Rich feature hierarchies for accurate object detection and semantic
(AP), AP50, AP75, APs, APm, APl, representing the precision segmentation.” In Proceedings of the IEEE conference on computer
at different intersections of union (IOU) thresholds and for vision and pattern recognition, pp. 580-587. 2014.
various object sizes. The DeFRCN model with GDL improves [8] Girshick, Ross. ”Fast R-CNN.” In Proceedings of the IEEE
the consistency of Faster R-CNN and achieves better international conference on computer vision, pp. 1440-1448. 2015.
productivity than baselines. The HR-RCNN with ResNet-101 [9] Sun, Peize, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei
backbone and a 2x training scheme achieves an overall AP of Zhan, Masayoshi Tomizuka et al. ”Sparse R-CNN: End-to-end object
44.9 while using deformable ResNet-101 as the backbone detection with learnable proposals.” In Proceedings of the IEEE/CVF
conference on computer vision and pattern recognition, pp. 14454-
increases the AP to 47.7. The GCA-RCNN with FPN 14463. 2021.
baselines achieves an overall AP of 40.0, while the Double- [10] T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, S. Belongie,
Head RCNN [4] strategy achieves an overall AP of 42.1. A Feature pyramid networks for object detection, in: Proceedings of the
total AP of 46.9 is achieved by sparse R-CNN with ResNeXt- IEEE conference on computer vision and pattern recognition, 2017, pp.
101(RX101) in its pure form, 48.9 with DCN [4], and 51.5 2117–2125
with additional test-time augmentations. Using COCO-style [11] Cai, Zhaowei, and Nuno Vasconcelos. ”Cascade R-CNN: Delving into
Average Precision, Cascade R-CNN with ResNet-101(R101) high quality object detection.” In Proceedings of the IEEE conference
on computer vision and pattern recognition, pp. 6154-6162. 2018.
[21] as its base achieves a total AP of 42.8. These models
[12] Rezatofighi, Hamid, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian,
show that to achieve better detection accuracy and inference Ian Reid, and Silvio Savarese. ”Generalized intersection over union: A
speed and satisfy the real-time requirements of video metric and a loss for bounding box regression.” In Proceedings of the
processing, deep learning-based object detection models IEEE/CVF conference on computer vision and pattern recognition, pp.
require an improved architecture. 658-666. 2019.
[13] Zhang, Wenchao, Chong Fu, Haoyu Xie, Mai Zhu, Ming Tie, and
V. CONCLUSION Junxin Chen. ”Global context aware RCNN for object detection.”
Neural Computing and Applications 33 (2021): 11627-11639.
This paper provides a comprehensive exploration of object
[14] Nguyen, Khoi, and Sinisa Todorovic. ”IFS-RCNN: An incremental
detection, offering an in-depth analysis of several models few- shot instance segmenter.” In Proceedings of the IEEE/CVF
based on the Faster R-CNN framework. The comparison of Conference on Computer Vision and Pattern Recognition, pp. 7010-
these models on the COCO dataset highlights Sparse R-CNN 7019. 2022.
as the top performer, achieving an impressive overall Average [15] Qiao, Limeng, Yuxuan Zhao, Zhiyuan Li, Xi Qiu, Jianan Wu, and Chi
Precision (AP) of 51.5. The study examines the architecture of Zhang. ”DeFRCN: Decoupled faster R-CNN for few-shot object
each model and the enhancements introduced to address detection.” In Proceedings of the IEEE/CVF International Conference
on Computer Vision, pp. 8681-8690. 2021.
challenges related to detection speed and accuracy. Given the
[16] Chen, Hao, and Abhinav Shrivastava. ”HR-RCNN: Hierarchical
complexity of object detection, it is crucial for deep learning Relational Reasoning for Object Detection.” arXiv preprint
models to continuously evolve in order to meet the demands arXiv:2110.13892 (2021).
of real-time applications. With ongoing advancements in [17] Yi, Xin, Jiahao Wu, Bo Ma, Yangtong Ou, and Longyao Liu. ”LGA-
technology and innovative methodologies, further RCNN: Loss-Guided Attention for Object Detection.” arXiv preprint
improvements in object detection are anticipated. arXiv:2104.13763 (2021).
[18] Xie, Saining, Ross Girshick, Piotr Dolla´r, Zhuowen Tu, and Kaiming
REFERENCES He. ”Aggregated residual transformations for deep neural networks.”
In Proceedings of the IEEE conference on computer vision and pattern
[1] Om Uparkar, et al., “Vision Transformer Outperforms Deep
recognition, pp. 1492-1500. 2017.
Convolutional Neural Network-based Model in Classifying X-ray
Images”, Procedia Computer Science, vol. 218. [19] Chen, Jer-Sen, Andres Huertas, and G. Medioni. ”Fast convolution
with Laplacian-of-Gaussian masks.” IEEE Transactions on Pattern
[2] Yatendra Sahu et al., “A CNN-SVM based computer aided diagnosis
Analysis and Machine Intelligence 4 (1987): 584-590.
of breast Cancer using histogram K-means segmentation technique”,
Multimedia Tools and Applications, 2022. [20] COCO Dataset: [Link]
[3] Sarth Kanani et al., “An AI-Enabled ensemble method for rainfall [21] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. ”Deep
forecasting using Long-Short term memory”, Mathematical residual learning for image recognition.” In Proceedings of the IEEE
Biosciences and Engineering, vol. 20, issue 5 2023 conference on computer vision and pattern recognition, pp. 770-778.
2016.
[4] Zhang, Yu, and Qiang Yang. ”A survey on multi-task learning.” IEEE
Transactions on Knowledge and Data Engineering 34, no. 12 (2021):
5586-5609.
[5] Ajit, Arohan, Koustav Acharya, and Abhishek Samanta. ”A review of
convolutional neural networks.” In 2020 international conference on
emerging trends in information technology and engineering (ic-
ETITE), pp. 1-5. IEEE, 2020.

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:56:06 UTC from IEEE Xplore. Restrictions apply.