Traffic Sign Detection Under Adverse
Environmental Conditions Based on CNN
SEMINAR REPORT
submitted to the Savitribai Phule Pune University, Pune
In the partial fulfilment of the requirements
for the award of the degree
BACHELOR OF ENGINEERING
(IT Engineering)
BY
[Link] Ramesh Jadhav
Roll No: 243343
Under the guidance of
[Link]
Head Of Department
DEPARTMENT OF IT ENGINEERING
Vidya Pratishthan’s Kamalnayan Bajaj Institute of Engineering
and Technology,
Vidyanagari Bhigwan Road
Baramati- 413133
2024-25
CERTIFICATE
This is to certify that [Link] Ramesh Jadhav has successfully submitted his
project report to the Department of IT Engineering, VPKBIET, Baramati,on
”Traffic Sign Detection Under Adverse Environmental Conditions Based
on CNN”
during the academic year 2024-2025 in the partial fulfilment towards completion of
Fourth Year of Bachelor of Engineering in Information Technology
[Link] Mrs. [Link]
Assistant Professor, Assistant Professor,
Guide, HOD,
Dept of IT Engineering. Dept of IT Engineering.
Dr. [Link]
Principal
VPKBIET, Baramati.
Place:Vidya Pratishthan’s Kamalnayan Bajaj Institute of Engineering and Technology,
Baramati.
Date :
i
Acknowledgments
I am feel happy in forwarding this Seminar report as an image of sincere efforts.
The Successful seminar reflects my work, effort of my guide in giving me good informa-
tion.
I would like to express my heartiest thanks to my seminar guide [Link]
for his undying support which makes me possible to make this seminar knowledgeable.
He not only provided me the literature and guidance to study but also the platform
which required for me to prepare best for this seminar.
I am also equally indebted to our seminar coordinator I am also equally indebted
to our seminar coordinator Mr. Amol Rajapure who has been a constant source of
inspiration and guiding star in achieving my goal.
I thank Mrs. [Link], Head of Department of IT Engineering and re-
spected Principal [Link] for supporting and providing all facilities to com-
plete the work. I am thankful to and fortunate enough to get constant encouragement,
support and guidance from entire teaching staff of Department of IT engineering which
helped me in successfully completing my seminar work.
Rutuja Ramesh Jadhav
ii
Abstract
A strong and dependable Traffic Sign Detection and Recognition (TSDR) system
is super important for self-driving cars. Researchers have done a lot of work in this
area, but they mostly focus on perfect conditions. They often ignore tough weather
like rain, snow, or fog, which can blur the images and make it hard for the system to
work well. This paper looks at five types of bad conditions that mess with TSDR: rain,
snow, fog, dirt on the lens, and blurred lens. To tackle these issues, this paper pro-
poses a new method based on Convolutional Neural Networks (CNNs). The method has
three parts: a classifier to detect the bad weather (using the VGG19 model), an image
enhancer (called Enhance-Net) that improves the signs in the image, and a detection
system (YOLOv4) to find the traffic signs fast. The system was tested using a dataset of
traffic videos in bad weather and showed 95.03 per accuracy and a speed of 12.79 frames
per second. While the accuracy dropped slightly by 2.81 per, the speed improved by
12.03 fps, proving the method works well.
Keywords :CNN, adverse environments, image enhancement, TSDR
iii
Contents
Abstract ii
1 Introduction 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Literature Survey 2
2.1 A Deep Learning-Based Framework for Robust Traffic Sign Detection
Under Challenging Weather Conditions . . . . . . . . . . . . . . . . . . 2
2.2 CURE-TSD: Challenging Unreal and Real Environments for Traffic Sign
Detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.3 Simultaneous traffic sign detection and boundary estimation using con-
volutional neural network . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.4 Research on traffic sign detection algorithm based on convolutional neural
networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.5 Tabular Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3 Motivation 5
3.1 Limitations of Previous Traffic Sign Detection Tech- niques . . . . . . . 5
3.2 Features Of Traffic Sign Detection Technique . . . . . . . . . . . . . . . 5
4 System Architecture 7
5 Approaches for Solving the Problem 8
6 Comparison of the Approaches 9
7 Advantages and Disadvantages 10
7.1 Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7.2 Disadvantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
8 Conclusion 11
9 Future Scope 12
10 References 13
v
Chapter 1
Introduction
1.1 Introduction
Autonomous driving is becoming a game-changer in the transportation industry by mak-
ing travel safer, more efficient, and convenient. These vehicles rely on different onboard
sensors like radar, GPS, cameras, and ultrasonic sensors to ”see” and understand the
environment around them. By working together, these sensors help the vehicle make
decisions, plan routes, and navigate roads automatically and safely.
An autonomous driving system is made up of three key parts:
1. The positioning and navigation system (which uses GPS and magnetometers) to
figure out where the vehicle is and plan its path.
2. The environmental perception system (which uses radar, cameras, and other sen-
sors) to detect things like obstacles, traffic signs, and other cars.
3. The planning and control system uses all the data to make decisions in real-time,
adjusting speed, direction, and braking to safely get the vehicle to its destination.
Big companies like Google, Uber, Tesla, Volkswagen, and others are leading the
development of autonomous driving. They’re testing and deploying these vehicles
for things like taxi services and long-distance cargo in countries like the US, China,
and Germany. But one big challenge remains: making sure these vehicles can still
work well in difficult weather conditions like rain, snow, fog, or dirt, which can
mess up the sensors and make it harder to recognize traffic signs.
Research in this area is super important to improve how well these systems can
handle such tough conditions. It will not only make autonomous driving safer but
also allow these cars to be used in more places and different environments. These
advancements will help fully realize the potential of self-driving vehicles, making
them reliable in all kinds of real-world scenarios.
1
Chapter 2
Literature Survey
2.1 A Deep Learning-Based Framework for Robust Traf-
fic Sign Detection Under Challenging Weather Condi-
tions
This paper presents a method that uses artificial intelligence (AI) to detect traffic
signs, even when the weather is bad (like rain, fog, or snow). The AI system is designed
to be more reliable in tough conditions where regular systems might fail, making driv-
ing safer. It was published in 2022 in a respected journal about smart transportation
systems.
2.2 CURE-TSD: Challenging Unreal and Real Environ-
ments for Traffic Sign Detection.
This paper presents a new dataset and method for detecting traffic signs in both
simulated and real-world environments, focusing on challenging conditions. It helps im-
prove the accuracy of traffic sign detection systems, especially in tough situations like
bad weather. The dataset and findings aim to enhance the reliability of these systems,
which is important for self-driving cars and smart transportation. It was published in
2021 in a respected journal.
2
2.3 Simultaneous traffic sign detection and boundary es-
timation using convolutional neural network
This paper presents a method that uses a Convolutional Neural Network (CNN) to
detect traffic signs and estimate their boundaries at the same time. The AI system is
designed to improve the accuracy of traffic sign detection by providing precise bound-
ary information, which helps in understanding the location and size of the signs. This
method enhances the performance of traffic sign recognition systems, contributing to
safer driving.
2.4 Research on traffic sign detection algorithm based on
convolutional neural networks
This research provides an improved algorithm for traffic sign detection using CNNs,
contributing to better recognition systems and supporting advancements in autonomous
driving technology.
3
2.5 Tabular Literature Survey
Sr. Paper Technique Advantages Limitations
No. Year Used
1 2022 Deep Learn- improves the detection of Requires large amounts of
ing(DL), traffic signs in poor visibil- labeled data and may not
Convolutional ity situations like rain, fog, perform well in extreme or
Neural Net- or snow, where traditional uncommon weather condi-
work (CNN) methods might struggle. tions that were not seen
during training.
2 2021 CURE-TSD Provides a diverse set of Simulated environments
Dataset, Deep conditions to improve the may not fully capture all
Learning(DL), robustness of detection real-world complexities.
Convolutional systems.
Neural Net-
work(CNN)
3 2018 Object De- Using CNN for object de- The method may struggle
tection, tection and boundary esti- in complex environments
Boundary mation together improves with occluded or partially
Estimation, the accuracy and efficiency visible traffic signs, which
Convolutional of traffic sign recognition, can lead to reduced detec-
Neural Net- making it more reliable in tion accuracy and misinter-
work different conditions and en- pretation of the signs.
hancing safety for drivers.
4 2021 Image Pro- The use of CNNs in the The algorithm may face
cessing Tech- traffic sign detection algo- challenges in detecting
niques, Convo- rithm improves the accu- traffic signs in difficult
lutional Neural racy and speed of recogniz- conditions, such as poor
Network ing signs, making the sys- visibility or occlusion,
tem more effective in real- which can reduce its over-
world scenarios. all effectiveness.
Table 2.1: Summary of Papers
4
Chapter 3
Motivation
3.1 Limitations of Previous Traffic Sign Detection Tech-
niques
In the field of traffic sign detection, several traditional methods have been utilized
before the adoption of more advanced techniques. These earlier approaches present sev-
eral limitations
1. Struggles with Changes: They often have a hard time dealing with different light-
ing and weather, which affects accuracy.
2. Manual Features: Many rely on features that are manually chosen, making it tough
to recognize various signs.
3. Slow Processing: These techniques usually need a lot of processing power, which
can slow down real-time detection.
4. Fixed Sizes: They often use fixed block sizes, making it difficult to detect signs
that come in different sizes.
5. Motion Issues: They don’t handle moving vehicles and signs very well, which can
lead to missed detections.
6. No Learning Ability: Older methods don’t learn from new data, so they can’t
improve over time.
3.2 Features Of Traffic Sign Detection Technique
1. Works in Bad Weather: The system can detect traffic signs even in tough condi-
tions like rain, snow, fog, dirt on the lens, and blurry images.
5
2. Uses CNN Technology: It relies on Convolutional Neural Networks (CNN) to
accurately find and recognize traffic signs.
3. Environment Detection Module: A part of the system (using VGG19) figures out
which bad weather condition is present in the image, so it can process the image
accordingly.
4. Image Improvement Module (Enhance-Net): This module sharpens the traffic sign
areas in the image, making it easier to detect signs even when the rest of the image
is affected by bad weather.
5. Fast Detection with YOLOv4: It uses the YOLOv4 framework for detecting traffic
signs quickly, balancing both speed and accuracy.
6. High Accuracy: The system is very reliable, with an accuracy of 95.03 per, even
in challenging conditions.
6
Chapter 4
System Architecture
7
Chapter 5
Approaches for Solving the
Problem
1. Adverse Environment Classifier: This module detects traffic signs under chal-
lenging conditions (e.g., rain, fog) using a pre-trained VGG19 model fine-tuned on
the CURE-TSD dataset. It features 16 convolutional layers that extract hierar-
chical image features. Images are resized to 512 × 512 pixels to balance detail and
computational efficiency. The model uses categorical cross-entropy for training
and is optimized with the Adam optimizer.
2. Image Enhancement (Enhance-Net): Enhance-Net improves the visibility of
traffic signs in images affected by adverse conditions. Its architecture includes
an encoder, residual blocks, and a decoder. The encoder captures features, while
residual blocks help refine them, and the decoder reconstructs the enhanced image.
It uses Mean Absolute Error (MAE) and perceptual loss functions to focus on
preserving essential details of traffic signs.
3. Traffic Sign Localization and Recognition (YOLOv4):YOLOv4 is an ad-
vanced object detection model that efficiently identifies and localizes traffic signs.
It uses the CSPDarknet53 backbone for feature extraction, a neck for creating
multi-scale feature pyramids, and a head for predicting bounding boxes and class
probabilities. YOLOv4 includes various optimizations like data augmentation and
the Mish activation function to enhance performance.
8
Chapter 6
Comparison of the Approaches
Here are the comparisons between the different approaches to traffic sign detection in
reference to their strengths and limitations:
Approach Strengths Limitations
Adverse Environment Utilizes a pre-trained Performance is highly de-
Classifier VGG19 model, benefiting pendent on the quality
from transfer learning for of the fine-tuning on the
better feature extraction. CURE-TSD dataset.
Image Enhancement Focuses on enhancing the Performance can vary
(Enhance-Net) visibility of traffic signs based on the severity of
specifically, improving the adverse conditions and
their detection rate. the quality of the input
images.
Traffic Sign Localiza- Highly efficient and accu- Requires a large amount
tion and Recognition rate object detection with of training data for op-
(YOLOv4) the ability to identify and timal performance, which
localize multiple traffic can be a barrier if high-
signs in real time. quality data is limited.
9
Chapter 7
Advantages and Disadvantages
7.1 Advantages
1. Higher Accuracy: CNN models, especially those based on YOLOv4, offer better
detection accuracy compared to traditional methods.
2. Faster Detection: The use of deep learning algorithms speeds up the detection
process, making real-time applications possible.
3. Robust in Adverse Conditions: This method performs well even in challeng-
ing weather conditions such as rain, fog, and snow, where traditional methods
struggle
4. Selective Enhancement:The Enhance-Net module specifically focuses on en-
hancing traffic sign regions rather than the whole image, improving detection ef-
ficiency.
7.2 Disadvantages
1. High Computational Cost: CNN models are computationally intensive and
may require specialized hardware (like GPUs) for faster processing
2. Training Data Dependency:The accuracy of the system relies heavily on the
quality and quantity of labeled training data.
3. Complexity in Implementation: Designing and fine-tuning CNN architectures
can be complex, requiring expertise in deep learning.
10
Chapter 8
Conclusion
In this paper, we introduced a modular CNN-based approach for traffic sign detection
that effectively mitigates performance degradation under adverse environmental condi-
tions. Our method employs a VGG19-based adverse environment classifier followed by
an enhancement network, specifically designed to restore features crucial for traffic sign
detection. We utilize an innovative loss function that focuses on the traffic sign region,
leading to improved detection performance.
While our approach resulted in a slight decrease in detection accuracy (average of
2.81per lower than previous studies), it significantly increased detection speed from 2.76
to 12.79 frames per second. This demonstrates that our algorithm can enhance speed
with minimal accuracy loss.
Looking ahead, we plan to refine the modular architecture of our approach, opti-
mizing both the classifier and enhancement network for better adaptability to various
environmental challenges. We will also expand our training dataset to include a wider
range of conditions, aiming for an improved balance between detection accuracy and
speed for real-world applications.
11
Chapter 9
Future Scope
1. Enhanced Robustness and Accuracy: As autonomous driving technology
evolves, there will be a growing demand for more robust and accurate TSDR
systems to handle real-world environmental conditions.
2. Improved Datasets:Utilizing enhanced datasets, like CURE-TSD, will support
more extensive training and testing, allowing TSDR systems to handle diverse
road environments and unpredictable weather patterns more effectively.
3. Balancing Security, Scalability, and Speed:Achieving an optimal balance
between security, scalability, and system speed will be crucial for the safe and
efficient global deployment of autonomous driving technologies.
4. Real-time Data Processing and Transmission: Future TSDR systems may
incorporate secure data transmission and real-time processing solutions to prevent
tampering or distortion of detected traffic signs, ensuring accurate recognition for
autonomous vehicles.
5. Secure and Scalable Processing:The future of TSDR systems will focus on
integrating secure and scalable processing methods, drawing parallels to secure
video encryption techniques to maintain data integrity and user privacy.
12
Chapter 10
References
[1] S. Ahmed, U. Kamal, and Md. K. Hasan, “DFR-TSD: A deep learning based
framework for robust traffic sign detection under challenging weather conditions,”
IEEE Trans. Intell. Transp. Syst., vol. 23, no. 6, pp. 5150–5162, Jun. 2022.
[2] D. Temel, T. Alshawi, M. H. Chen, and G. AlRegib. (Sep. 2021). CURE-TSD:
Challenging Unreal and Real Environments for Traffic Sign Detection. IEEE Dat-
aport.
[3] H. S. Lee and K. Kim, “Simultaneous traffic sign detection and boundary estima-
tion using convolutional neural network,” IEEE Trans. Intell. Transp. Syst., vol.
19, no. 5, pp. 1652–1663, May 2018.
[4] Q. T. Yu, “Research on traffic sign detection algorithm based on convolutional neu-
ral networks,” M.S. thesis, School Inf. Eng., Nanchang Univ., Nanchang, China,
2021.
13