2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

An Intelligent Approach to Object Detection Using

R-CNN
Raja K Satyam Pandey Shivangi Rani Verma
Department of CSE Department of CSE Department of CSE
SRM IST, Ramapuram SRM IST, Ramapuram SRM IST, Ramapuram
Rajak1@[Link] sp7224@[Link] sv3270@[Link]

Saket Sharma Senthilselvi A

Department of CSE Department of CSE
2024 Second International Conference on Advances in Information Technology (ICAIT) | 979-8-3503-8386-7/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICAIT61638.2024.10690590

SRM IST, Ramapuram SRM IST, Ramapuram

sr3156@[Link] senthila3@[Link]

Abstract—This research journal explores the highlighting their evolution. Additionally, the crucial role
implementation of region-based Convolutional Neural of transfer learning in enhancing object detection models
Networks (R-CNN) for precise object detection. Following a is examined. The journal aims to offer a holistic
systematic six-step approach, it begins by constructing an perspective on CNNs and object detection, providing
object detection dataset using Selective Search. This dataset insights into their symbiotic relationship and future
is then utilized to fine-tune a classification network initially
trained on ImageNet, enhancing object localization accuracy.
directions in computer vision.
Subsequently, an object detection inference script is The field of object detection within Machine Learning and
developed, leveraging Selective Search to propose regions Computer Vision confronts a pervasive challenge — the
potentially containing objects of interest. These regions necessity for more precise and accurate systems [7].
undergoclassification using the fine-tuned network, ensuring Despite widespread applications in real-time human
precise identification. Furthermore, to refine the results, identification across various sectors, the existing
non-maximum suppression is applied, effectively eliminating methodologies, including established algorithms like R-
weak and overlapping bounding boxes and thereby CNN, exhibit limitations in meeting evolving demands.
enhancing overall detection precision. The study culminates The overarching problem is to enhance object detection
in the presentation of final object detection outcomes. By
showcasing the versatility and efficacy of R-CNN
algorithms to achieve heightened precision and accuracy,
architecture, this project significantly contributes to the addressing the growing need for robust and reliable
fields of deep learning and computer vision. Its practical systems in diverse real-world scenarios. This paper
application using real-world datasets underscores its endeavors to contribute to the ongoing evolution of object
adaptability across diverse domains, providing valuable detection methodologies, aiming for advancements that
insights for scholars and researchers. transcend current limitations and align with the escalating
Keywords— Object detection, CNN, R-CNN, OpenCV, expectations for performance across industries.
TensorFlow, Keras. II. LITERATURE REVIEW
I. INTRODUCTION Shaoqing Ren et al. [1] introduced a groundbreaking two-
Object detection is crucial in computer vision, applied in stage architecture incorporating a Region Proposal
areas like autonomous vehicles and surveillance system [1]. Network (RPN) for efficient region proposal generation.
Convolutional Neural Networks (CNNs) [2,3] have proven Its innovation lies in achieving state-of-the-art accuracy.
highly effective, automatically learning visual features' However, the computational intensity of the model poses
hierarchical representations. This journal focuses on CNNs challenges for real-time applications, and the intricate
for object detection, covering foundational principles, architecture may hinder deployment on resource-
evolution of methodologies, and practical applications. constrained devices. Additionally, the reliance on anchor
Emphasis is on CNN architectures, exploring concepts like boxes can sometimes lead to suboptimal bounding box
region proposal networks and transfer learning for predictions.
improved generalization. The journal aims to provide a Joseph Redmon et al. [6] pioneered a unified detection
comprehensive overview, aiding researchers and model capable of handling a vast number of object classes
practitioners in understanding and applying CNN-based in real-time. It strikes a balance between speed and
object detection in diverse fields. accuracy. However, challenges arise in small object
CNNs are pivotal in image analysis[4,5] , excelling in detection, and the training process can be resource-
computer vision tasks like image classification, intensive, especially with a large number of classes.
segmentation, and object detection [6]. This journal YOLO9000 also faces difficulties in scenarios with highly
segment delves into CNNs' foundational principles, imbalanced class distributions, affecting its ability to
emphasizing their ability to capture intricate patterns and generalize. Wei Liu et al. [7] aimed to balance speed and
hierarchical representations. It explores CNN architectures accuracy with a focus on real-time applications. While
from LeNet to recent innovations like AlexNet and VGG, excelling in various scenarios, it may not perform as well

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:46:13 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

as Faster R-CNN on datasets dominated by large objects. Hanxiao Liu et al. [14] explained an anchor-free approach
Additionally, there is a tendency for more false positives, for a simplified architecture and improved accuracy. It
and the model's architecture might be complex for excels in scenarios with varying object scales and offers a
deployment on resource-constrained devices. Fine-tuning straightforward implementation. However, there might be
hyperparameters is crucial for optimal performance. a slight sacrifice in speed compared to anchor-based
Tsung-Yi Lin et al. [8] introduced focal loss to address methods, and careful hyperparameter tuning is still
class imbalance and improve accuracy, particularly in essential for optimal performance. Additionally, FCOS
dense object detection scenarios. The use of focal loss, may face challenges with extreme aspect ratios in object
while effective, introduces computational overhead. detection, affecting the accuracy of bounding box
Achieving optimal performance requires careful tuning of predictions [15].
hyperparameters, and the model may struggle with small III. PROPOSED MODEL
object detection. Additionally, the detection of objects Proposed object detection model builds on the versatile R-
with extreme aspect ratios can be challenging. CNN using TensorFlow and Keras. Integrating a region
[Link] et. al. [9] aimed to balance performance proposal network (RPN) and selective search, it efficiently
and resource efficiency, making it suitable for deployment identifies candidate object regions. Leveraging transfer
on edge devices. Its efficient architecture achieved state- learning for enhanced generalization, the model employs
of-the-art results while utilizing resources effectively. anchor-based mechanisms and feature pyramid networks
However, there may be a trade-off in accuracy compared for precise localization. Figure 1 shows the block diagram
to more complex models. Fine-tuning for specific of object detection. Proposed approach aims to surpass
applications is necessary, and it may not excel in scenarios current challenges, contributing to the advancement of
demanding the highest precision. Zhaowei Cai et al. [10] object detection within the TensorFlow and Keras
proposed a cascade structure for enhanced detection framework. Data Collection: The dataset utilized for
accuracy. It demonstrated top performance in challenging training the R-CNN object detector comprises 200 images
datasets. However, the increased model complexity and specifically selected for raccoon detection which was
computational cost compared to baseline models must be originally framed by Data scientist Dat Tran. This curated
considered. Additionally, sensitivity to hyperparameter collection, emphasizing precision in raccoon annotation, is
tuning may impact the robustness of the model in certain instrumental for training a robust and accurate model.
scenarios. Kaiming He et al. [11] extended the Faster R- Each image is meticulously annotated, with a total of 217
CNN architecture to include instance segmentation, raccoons across the dataset. Figure 2 shows the subset of
providing precise object masks. Its strengths include racoon images.
accurate instance segmentation and a relatively Data Preprocessing: The dataset of 200 raccoon images
straightforward implementation. However, the added underwent essential preprocessing, including resizing,
complexity makes it computationally demanding, pixel normalization, and addressing variations.
potentially compromising real-time performance. Fine- Additionally, a split into training and validation sets was
tuning for specific applications may also be necessary. performed for robust model training and evaluation.
Alexey Bochkovskiy et al. [12] aimed to improve speed Object Detection Models:
and accuracy by incorporating a one-level feature The proposed object detection model, using CNN
extraction approach. While achieving high performance techniques and some pre-trained models which are well
with fewer computational resources, attention is required known for their efficiency and accuracy in detection of
for fine-tuning hyperparameters. Challenges may arise in object using deep learning methods.
adapting to specific datasets or scenarios, and user- RPN (Regional Proposal Network): It is an integral part
friendly documentation is crucial for broad adoption. of object detection models, such as faster RCNN. It
Kaist Center for AI [13] introduced a novel approach efficiently generates candidate regions within an image by
using keypoint triplets to improve object localization predicting bounding boxes and objectness scores.
accuracy. It excels in precise localization, especially for Selective Search: An algorithmic approach for generating
small objects. On the downside, it may not perform as region proposals in object detection. It segments an image
well in scenarios with densely packed or overlapping based on color, texture, and intensity, merging segments
objects, and the computational cost is moderate. The to form potential object regions, provides a diverse set of
model may struggle with scale variations and complex region proposals.
scenes.

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:46:13 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

selection to balance knowledge retention and adaption.

Performance Evaluation: Model accuracy is evaluated
using Intersection over Union which quantifies the degree
of overlap between predicted and actual bounding boxes
in object detection, providing a nuanced assessment of
object localization precision. Higher IOU values indicate
better alignment, crucial for evaluating object detection
accuracy.
A. Ethical Considerations: Throughout the research,
priority was given to ethical considerations. Measures
were implemented to ensure the privacy and anonymity of
individuals contributing to the dataset. Diligent efforts
were undertaken to recognize and address potential biases
in both data sources and the sentiment analysis models
Figure 1. Block Diagram for Object Detection
utilized.

Figure 2. A subset of the racoon image dataset.

Feature Engineering:
In the realm of object detection, feature engineering
encompasses refining the representation and processing of
input data. Techniques like image resizing, normalization,
and augmentation are employed to amplify the model's
efficiency in recognizing and precisely localizing objects
within images. The features included are:
Data Augmentation: Augmenting the data enhances
model robustness by exposing it to diverse scenarios and B. Software and Tools: Throughout the project all the
it helps in mitigating overfitting and improving tasks were implemented using Python for scripting and
generalization. coding and libraries like TensorFlow (for deep learning
Transfer learning: It provides valuable feature model development), Keras (for Neural network
representations for limited annotated data. It helps in specifications) and OpenCV (for image processing and
enhancing model performance by leveraging generic computer vision tasks).
features before fine-tuning for the specific detection task. IV. METHODOLOGY
Anchor Box Configuration (For R-CNN): Proper The proposed model used R-CNN, RPN and Selective
anchor box setup is critical for accurate object localization. search model to analyze the task and give an optimized
It influences precision in bounding box predictions, result. R-CNN: abbreviated as Region-Based
directly contributing to overall detection accuracy. Figure Convolutional Neural Network is a foundational
3 shows the model training flowchart. architecture that has significantly contributed to the
advancement of computer vision tasks. It was introduced
by Ross Girshick and is particularly noteworthy for its
effectiveness in localizing and classifying objects within
images. The below fig is showing the architecture of R-
CNN how input images are put and then region proposals
are extracted using that image and after computing the
necessary features the regions are classified.
R-CNN is essential over CNN for object detection as it
excels in precisely identifying and localizing multiple
objects within images, thanks to its incorporation of
region proposals, offering superior accuracy and detail
compared to traditional CNNs designed for image
classification. Figure 4 shows R-CNN architecture.

Figure 3. Model Training Flowchart.

Fine-Tuning: During fine-tuning, the pre-trained RCNN Fig. 4: R-CNN architecture

model underwent a meticulous adjustment to adapt to
raccoon dataset. This process involved careful parameter Region Proposals: In object detection, region proposals

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:46:13 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

are algorithms that pinpoint potential object locations

within an image (figure 5), streamlining the detection
process by focusing on specific areas. Techniques like
Selective Search and Region Proposal Networks generate
candidate regions for subsequent analysis, enhancing
efficiency and accuracy in identifying objects within
complex scenes.

Figure. 5: Joining regions and predicting image with help of OpenCV

The two main methods for generating region proposals are

notably selective search and Region Proposal Networks
(RPN).
Selective Search: It operates by segmenting an image
based on color, texture, and intensity and then merging
Figure.6: The Basic R-CNN detector pipeline.
similar segments hierarchically. The resulting regions
form a diverse set of potential object candidates, covering
various scales and aspect ratios.
Region Proposal Networks (RPN): It is more modern
and learnable approach integrated into some object
detection architectures like Faster R-CNN. It involves a
small neural network that slides over the convolutional
feature map of an image and predicts potential bounding
box coordinates and objectness scores.

V. RESULTS AND DISCUSSIONS

Trained on a raccoon image dataset, the R-CNN model
excels in object detection. Its key stages, from region
proposal generation to final detection, resulting in Figure. 7 A no-racoon(left) vs racoon(right) images sample is used for
outstanding performance. The racoon dataset has been building and R-CNN object detection model.
provided by Dat Tran. Some of the following modules are:
[Link], [Link] and [Link] which are responsible for In Figure 7 (left), "No Raccoon" class includes image
performance metrics, configuration management and patches with minimal overlap from Selective Search. On
avoiding overlapping rectangles respectively. the right (Figure 7), intentional similarities in "Raccoon"
The pygamesearch module is essential for three Python class images align with expected behavior. Selective
scripts. Firstly, build_dataset.py processes Dat Tran's Search detecting multiple instances in similar regions is
raccoon dataset, creating a raccoon vs no_raccoon dataset attributed to its identification of potential object regions.
for fine-tuning MobileNet V2. Next, fine_tune_rcnn.py In Figure 8, Throughout the fine-tuning of the MobileNet
focuses on training the raccoon classifier through fine- V2 classifier on the raccoon dataset, monitored crucial
tuning. Lastly, detect_object_rcnn.py combines Selective accuracy and loss curves, essential for basic R-CNN
Search and classification for rudimentary R-CNN object object detection with Keras, TensorFlow, and Deep
detection. Figure 6 shows the basic R-CNN detector Learning. Now completed two major tasks. Firstly, build a
pipeline. model for object detection using selective search
algorithm (which still consists of overlapping rectangles)
followed by fine tuning network classifier.
The main motive of proposed method is to somehow
remove the overlapping boxes around the model which is
being created by selective search. To avoid this problem,
the proposed method will be applying NMS which can be
abbreviated as Non maxima Suppression.

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:46:13 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

Figure. 12 The final case (with NMS and without NMS)

VI. CONCLUSION
In conclusion, the project has successfully demonstrated
the effectiveness of an R-CNN object detection system,
leveraging the power of Keras, TensorFlow, and Deep
Learning techniques. The system’s performance was
rigorously tested and evaluated, yielding impressive
Fig.8: Fine-tuning of the MobileNet V2 classifier on the raccoon dataset.
empirical results that underscore the project’s success. The
R-CNN object detection system achieved outstanding

Figure 9 Evaluation Metrics after Fine-Tuning

precision (0.97) and perfect recall (1.00) for raccoon

detection, affirming its reliability and accuracy. The
meticulous fine-tuning process led to an impressive F1-
score of 0.99 for both raccoon and no-raccoon
classifications, based on 752 instances evaluated. These
results validate the model's effectiveness and highlight its
potential applicability in real-world scenarios such as
Figure.10 Object detection before NMS (getting overlapping rectangles) wildlife monitoring or urban surveillance.
The model achieved exceptional accuracy (0.99),
showcasing the adaptability and efficiency of R-CNN
models in object detection tasks. This project sets the stage
for future research exploring more complex architectures
and diverse object detection domains, utilizing larger
datasets to improve model robustness and performance.
The numerical results obtained from the project provide a
strong justification for the conclusion. The high precision,
recall, and F1-score values demonstrate the model’s ability
to accurately detect and classify raccoons within
images. The high overall accuracy further validates the
effectiveness of the R-CNN object detection system. This
project showcases significant advancements in object
detection methodologies, particularly with R-CNN models,
within computer vision. Its success in raccoon detection
underscores the model's broader applicability in diverse
real-world scenarios. The empirical results obtained affirm
Figure. 11 After applying the NMS the model's reliability and accuracy, paving the way for
future research in this domain.
Figure 10 and 11 shows output with and without NMS
References
respectively. Utilizing Non-Maximum Suppression allows
for the elimination of weaker bounding boxes, retaining [1] Shaoqing Ren et al., "Faster R-CNN: Towards Real-Time
only the correct one. Applying this technique to R-CNN Object Detection with Region Proposal Networks," in Advances in
Neural Information Processing Systems (NeurIPS), 2015.
object detection output results in the final object detection.
[2] S. P. S, K. T, J. M and M. A. Sheriff, "A Comparative Analysis on the
In some cases, both the NMS and NMS images will be Prediction of Heart Failure using Machine Learning Algorithms," 2024
identical like in the Figure 12 5th International Conference on Mobile Computing and Sustainable
Informatics (ICMCSI), Lalitpur, Nepal, 2024, pp. 206-211, doi:
10.1109/ICMCSI61536.2024.00037

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:46:13 UTC from IEEE Xplore. Restrictions apply.
 2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

[3]S. S. Pandi, M. S. Monesh and B. Lingesh, "A Novel Approach to Detect

Face Fraud Detection Using Artificial Intelligence," 2024 Second
International Conference on Emerging Trends in Information
Technology and Engineering (ICETITE), Vellore, India, 2024, pp. 1-6,
doi: 10.1109/ic-ETITE58242.2024.10493594.
[4] S. P. S, K. T, V. R. S R and V. R, "Predictive Modelling of Critical Vital
Signs in ICU Patients by Machine Learning: An Early Warning System
for Improved Patient Outcomes," 2024 3rd International Conference for
Innovation in Technology (INOCON), Bangalore, India, 2024, pp. 1-6,
doi: 10.1109/INOCON60754.2024.10512042.
[5] S.P. S, Senthilselvi A, Kumaragurubaran T & Dhanasekaran S (2024) Self-
attention-based generative adversarial network optimized with color
harmony algorithm for brain tumor classification, Electromagnetic
Biology and Medicine, DOI: 10.1080/15368378.2024.2312363
[6] Joseph Redmon et al., "YOLO9000: Better, Faster, Stronger," in
Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2016.
[7] Wei Liu et al., "SSD: Single Shot MultiBox Detector," in European
Conference on Computer Vision (ECCV), 2016.
[8] Tsung-Yi Lin et al., "RetinaNet: Focal Loss for Dense Object Detection,"
in Proceedings of the IEEE International Conference on Computer
Vision (ICCV), 2017.
[9] S. SenthilPandi, D. Paulraj, D. Mithun and N. K. A, "Object Detection
using Learning Algorithm and IoT," 2023 RMKMATE, Chennai, India,
2023, pp. 1-6, doi: 10.1109/RMKMATE59243.2023.10369005
[9] Zhaowei Cai et al., "Cascade R-CNN: Delving into High-Quality Object
Detection," in Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition (CVPR), 2018.
[10] Kaiming He et al., "Mask R-CNN," in Proceedings of the IEEE
International Conference on Computer Vision (ICCV), 2017.
[11] Alexey Bochkovskiy et al., "You Only Look One-level Feature
(YOLOv4)," in arXiv preprint arXiv:2004.10934, 2020.
[12] Kaist Center for AI, "CenterNet: Keypoint Triplets for Object Detection,"
in Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2019.
[12] Hanxiao Liu et al., "Anchor-Free at Last: A Simple and Effective
Approach to Object Detection (FCOS)," in Proceedings of the IEEE
International Conference on Computer Vision (ICCV), 201
[13] S. SenthilPandi, D. Paulraj, N. K. A and N. K. B, "A Novel Approach for
Image Background Elimination," 2023 RMKMATE, Chennai, India,
2023, pp. 1-6, doi: 10.1109/RMKMATE59243.2023.10369121

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Zhejiang University. Downloaded on March 11,2025 at 13:46:13 UTC from IEEE Xplore. Restrictions apply.