Enhancing the performance of Traffic Sign Recognition in
Autonomous Driving through Deep Learning
[Link] [1] and C.M Vidhyapathi [2]
1
Research Scholar, School of Electronics Engineering, Vellore Institute of Technology, Vellore, India
2
Assistant Professor, School of Electronics Engineering, Vellore Institute of Technology, Vellore, India
Abstract
In computer vision, traffic sign detection is an essential job with many applications in autonomous
driving, intelligent transportation systems, and traffic safety. Fast and efficient object detection
systems are essential to the development of autonomous vehicles. In this field, deep learning models
have demonstrated impressive skills that allow cars to successfully sense and navigate their
environment. To improve traffic sign detection, this study uses the German Traffic Sign Recognition
Benchmark (GTSRB) dataset in combination with MobileNet-v2, a small and effective convolutional
neural network. The model achieved a maximum accuracy of 99.3% on the German Traffic Sign
Recognition Benchmark Database (GTSRB). This approach demonstrates how deep learning
techniques like MobileNet-v2 can optimize the performance of object detection systems in
autonomous vehicles, contributing to safer and more intelligent transportation solutions.
Keywords: Intelligent transportation, MobileNet-v2, GTSRB Dataset, Traffic Sign Recognition
[Link]
Object detection is a fundamental and well-researched task in computer vision that aims to locate and
identify target objects in an image. Advanced driver assistance systems and autonomous driving have
drawn a lot of attention as a result of the development of intelligent driving technologies. Advanced
driver assistance systems and autonomous driving both depend on traffic sign detection and
recognition [1]. Object recognition presents numerous challenges, including variations in viewpoints,
changes in lighting conditions, occlusion, differences in scale, and the complexity of recognizing
individual objects. Overcoming these challenges necessitates the design of robust and efficient
algorithms capable of adapting to diverse object appearances, dynamic lighting, and environmental
changes. Ongoing research focuses on developing innovative methods and algorithms to enhance the
accuracy and robustness of object recognition systems [2]. Convolutional neural networks (CNNs)
are an excellent choice for traffic sign recognition tasks because of their many benefits, including
parameter sharing and local feature perception. Furthermore, CNNs have strong generalization
abilities through training on extensive datasets, which enables them to function reliably even in the
face of difficult circumstances like noise in images or variations in lighting [3].
The key contributions of this paper can be summarized as follows:
1.A performance improved traffic sign detection approach is proposed by extracting the features and
preprocessing of the traffic sign image.
2. The structure of MobileNet-v2 to classify the sub-classes of the three categories of traffic signs.
3. Performance analysis of the traffic sign recognition system [4].
The remainder of the paper is structured as follows. Section 2, discuss related works on traffic
sign recognition tasks. In Section 3, we introduce the proposed methodology and then describe data
collection and categorization methods for road scene images. Section 4, includes the experimental
results and evaluation metrics. Finally, Section 5 provides a conclusion and considers future research
directions.
[Link] and C.M Vidhyapathi
[Link] works
Currently, the recognition and classification of traffic signs play a crucial role, particularly in
autonomous driving systems. Significant research efforts have been dedicated to advancing this field
and improving the detection and interpretation of traffic and road signs.
The authors in [5] presented a method based upon the foundation provided by ResNet models V1
and V2. The study highlights the advantages of these methods achieves highest training and
validation accuracy in the field of autonomous driving technologies. However, the paper also
acknowledges some limitations it does not focus on different weather condition especially during
sandstorms which are frequent in Saudi Arabia, and signs that have suffered from degradation. The
review concludes that future focus on indoor signage as a method to assist individuals with
disabilities, such as individuals with visual impairments.
The paper [6] discussed various techniques and algorithms used in these systems, including
ResNet50, DenseNet121, and VGG16 for traffic sign recognition. The proposed approach achieves
higher accuracy for three different datasets such as GTSRB, BTSD & TSRD in detecting traffic signs
when compared to some existing algorithm. However, a limitation of this study is that attention-
based models, particularly vision transformers, face challenges in image recognition tasks. Despite
this limitation, the paper showcases the approach incorporates Ensemble learning method which is
more robust and less sensitive to noise in the data.
The authors in [7] proposed a CNN model to improve its ability to recognize little objects without
compromising the precision of detecting huge objects. When compared to the standard CNN, the
suggested method detects traffic signs with greater accuracy. However, the limitation of the This
system can only recognize forty-three classes of traffic signs, so the traffic sign recognition range is
limited. Nonetheless, the paper demonstrates an enhancement in detecting small objects while
maintaining high detection accuracy for large objects.
The paper in [8] introduced an approach for traffics sign detection and recognition are CNN. The
most popular datasets to measure traffic sign identification and small object detection performance
are the public GTSRB datasets. This method makes use of deep learning techniques to improve
traffic sign detection accuracy, which makes it appropriate for use in self-driving systems. However,
a notable limitation of the algorithm is difficulties to recognize traffic signs in real time. The review
of the existing traffic sign recognition systems with methodology, datasets used, evaluation metrics
are tabulated in Table 1.
Table 1. Summary of the existing literature on traffic sign recognition
Author Algorithm Dataset Accuracy (%)
Ghazanfar Latif et al. ResNet model Arabic Traffic 99.18%
Signs
Xin Roy Lim et al. ResNet50, GTSRB, 98.84%
DenseNet121, BTSD, 98.33%
VGG16 TSRD 94.55%
Pranay Shetty et al. CNN GTSRB 94.98%
Sridevi [Link] CNN GTSRB 91.30%
[Link] and C.M Vidhyapathi
[Link]
The proposed method is thoroughly explained in the following section. The suggested technique's
flowchart and its constituent parts, which are explained in the following subsections, are shown in
Figure 1[9].
Data Collection: Data collection refers to the process of gathering raw data from various sensors
such as Cameras, LIDAR or RADAR and external sources to enable the development and operation
of intelligent driving systems. This data serves as the foundation for training, validating, and
improving deep learning models and decision-making algorithms used in autonomous vehicles.
Data Preprocessing: Data preprocessing refers to the series of steps such as Data cleaning, Data
transformation, Data augmentation, Dimensionality Reduction, Data Segmentation are applied to raw
data collected from various sensors to prepare it for use in deep learning models or decision-making
algorithms. Here Rescaling preprocessing technique used for recognizing traffic sign.
Feature Extraction: Feature Extraction refers to the process of identifying and isolating meaningful
patterns or attributes from raw sensor data to facilitate accurate predictions and decision-making by
deep learning models such as CNN, Transformer etc...
Train/Test: Train/Test refers to the methodology of splitting datasets and evaluating deep learning
models to ensure their reliability and generalization in real-world scenarios. This process is crucial
for developing robust algorithms that enable autonomous vehicles to perform tasks such as object
detection, lane keeping, and path planning.
Models: Models refer to the computational frameworks such as MobileNet-v2 algorithms is designed
to process data and make classification out of 43 classes in GTSRB dataset and make decisions that
enable autonomous driving.
Figure 1. Block Diagram of supervised learning
Fig.2. Block Diagram of supervised learning
[Link] and C.M Vidhyapathi
3.1 MobileNetV2 Architecture
The research focused on the mobility of the MobileNet-v2 model and used it for image
categorization. MobileNet-v2's primary architecture is derived on that of its predecessor, MobileNet-
v1. In addition to employing Linear Bottlenecks to address the issue of information loss in non-linear
layers in convolution blocks, MobileNet-v2 also uses the Depthwise Separable Convolutions (DSC)
technique for reducing and introduces a new structure called Inverted the residuals to preserve the
information.
Depthwise Separable Convolution
MobileNet-v2 is additionally makes use of the Depthwise Separable Convolutions that were used
in MobileNet-v1 [10]. The total number of parameters and computational cost can be lowered to
roughly 18 of the standard convolutions by combining the Depthwise and Pointwise convolutions.
The depthwise convolution has been re-designed as shown in Figure 2 [11].
Fig.2. The Architecture of the Depthwise Separable Convolution Block Used in MobileNet-v2 Architecture
Inverted Residuals
The input is followed by several bottlenecks and expansion in each bottleneck block, which is
comparable to residual blocks. The two main reasons why shortcuts can be placed between various
bottlenecks to enhance the gradient propagation capabilities in multiplier layers are as follows: (1)
Nearly all of the information is contained in the bottlenecks. (2) With a non-linear transformation of
the tensor, the expansion layer can be thought of as an implementation detail. Additionally, compared
to the conventional structure, the inverted architecture (Inverted Residual) uses less memory.
Linear Bottlenecks
The concept of linear bottlenecks is inspired by two characteristics. On the one hand, it is evident
that the corresponding portion of input space (x) is restricted to a linear transformation if a layer's
result has a format of ReLu6 and the result is still non-zero. Stated otherwise, deep networks can only
perform a linear classifier on the output domain's non-zero volume portion. Conversely, ReLu6
invariably loses information in a channel when it collapses it. But if there are several channels and
the set of activation functions has an architecture that allows information to be retained in other
channels. ReLu6 is used as an activation function and the input space can be integrated into low-
dimensional space. Each depthwise and pointwise convolution layer is followed by a batch
normalization layer and a ReLU6 activation layer, applied sequentially as described in Eq.1
y=min(max (0 , x),6) (1)
[Link] and C.M Vidhyapathi
Model Architecture
The bottleneck of a depthwise separable convolution with residuals serves as the fundamental
building [Link] model filters the input using a lightweight depthwise convolution after first
expanding the low-dimensional compressed representation to high dimension. A linear convolution is
then used to project the features back to a low-dimensional form.
Squeeze-and-Excitation (SE) Blocks
Squeeze-and-excitation (SE) blocks are incorporated into MobileNet-v2 to improve its feature
representation. These blocks dynamically adjust channel-wise feature responses, enabling the model
to emphasize important features while diminishing less relevant ones.
[Link] Results
4.1Dataset
A publicly accessible dataset for the detection and recognition of traffic signs is the German Traffic
Sign Recognition Benchmark dataset was collected by the Institute of Neural Information Processing
(INI) at the University of Ulm in Germany [12]. There are 51839 photos in the GTSRB dataset that
depict 43 different kinds of traffic signals shown in Fig 3.[13]. Every image has a label that
corresponds to a category, such as speed limits, prohibited parking, warning signs, etc. There are
between 200 and 2000 pictures on each traffic sign. Ranges of image resolution between 15*15 and
250 * 250. Total of 39209 photos from the dataset are used to construct the model in order to improve
prediction and evaluation, training set and the verification set, with an 8:2 ratio between the two sets.
The training is constructed with 31368 photos and testing is constructed with 7841 photos.
Fig.3. German traffic sign recognition benchmark (GTSRB) dataset
4.2 Evaluation metrics
The model's performance can often be evaluated using confusion matrix-based performance
metrics. Four metrics True Positive (TP), False Positive (FP), True Negative (TN), and False
Negative (FN) are included in the confusion matrix, which is a tabular representation of the actual
and projected classification results [14].
Accuracy: Accuracy is the ratio of the number of correct predictions to the total number of
predictions made by the model, presented in Eq.1[15].
(TP+TN )
Accuracy= (2)
(TP+ FP+TN + FN )
[Link] and C.M Vidhyapathi
4.3 Implementation details
The proposed method was implemented using TensorFlow with a Python interface. The hardware
setup for training and testing included an Intel Core i5-13420H CPU, 16 GB of DDR4 RAM, and an
NVIDIA GeForce RTX4050 GPU.
4.4 Results and analysis
The initial CNN model was trained using a dataset split into a training set (80%) and a test set
(20%). Figure 4 shows that the accuracy and Figure 5 illustrates a sharp decline in loss were
evaluated by plotting training set performance (blue) against validation set performance (orange)
over multiple iterations. The accuracy of the model was 99.3%, which is relatively accurate. In
addition, the model can recognize traffic signs with complex background well as shown in Table 2.
Fig.4. Training and Validation Accuracy Fig.5. Training and Validation Loss
Table 2. Successful identification of complex background
Traffic sign image Traffic sign label Confidence
Speed limit 70km/h 100%
Road work 99.03%
Stop 100%
Go straight or right 99.9%
Wild animals crossing 98.95%
[Link] and C.M Vidhyapathi
[Link]
In this study, we present a lightweight model for traffic sign recognition, demonstrating its
effectiveness in this domain. The model leverages Depthwise Separable Convolution, Inverted
Residual structures, and Squeeze-and-Excitation Blocks to create a robust feature extraction
framework, based on the MobileNetV2 architecture. Depthwise Separable Convolution and Inverted
Residual structures enable efficient multi-level feature extraction and fusion, while SE Blocks
enhance the model's focus on critical features. Our approach is not only simple to implement but also
achieves competitive classification performance, surpassing several recent benchmarks. Additionally,
the model excels in recognizing low-resolution traffic signs and performs well in complex
backgrounds, thanks to the application of MobileNetV2 as the backbone for traffic sign detection.
Future work will focus on enhancing the generalization ability of the model to further improve its
performance across diverse conditions.
References
1. Yao, J., Huang, B., Yang, S., Xiang, X. and Lu, Z., 2023. Traffic sign detection and recognition under low
illumination. Machine Vision and Applications, 34(5), p.75.
2. Suma, M., Premananda, R. and Harakannanavar, S.S., 2023. Development of Object Recognition Model
Using Machine Learning Algorithms on MobileNet V2. Int. J. Advanced Networking and
Applications, 15(02), pp.5908-5914.
3. Wei, W., Zhang, L., Yang, K., Li, J., Cui, N., Han, Y., Zhang, N., Yang, X., Tan, H. and Wang, K., 2024. A
lightweight network for traffic sign recognition based on multi-scale feature and attention
mechanism. Heliyon, 10(4).
4. Amma, N.B. and Rajput, V., 2024. Towards improving the performance of traffic sign recognition using
support vector machine based deep learning model. Multimedia Tools and Applications, 83(3), pp.6579-
6600.
5. Latif, G., Alghmgham, D.A., Maheswar, R., Alghazo, J., Sibai, F. and Aly, M.H., 2023. Deep learning in
Transportation: Optimized driven deep residual networks for Arabic traffic sign recognition. Alexandria
Engineering Journal, 80, pp.134-143.
6. Lim, X.R., Lee, C.P., Lim, K.M. and Ong, T.S., 2023. Enhanced traffic sign recognition with ensemble
learning. Journal of Sensor and Actuator Networks, 12(2), p.33.
7. Shetty, P., Shrilatha Shripathi, P. and Sankhe, A., 2022. Traffic Sign Recognition Model. Traffic, 9(04).
8. Sridevi.P, Dharani.M, Kruthi.M, Maheswari.B, Pavani.V2024. Traffic Sign Recognition for Driver
Assistance in Autonomous Vehicles using Deep Neural Networks. Journal For Basic Sciences, ISSN NO:
1006-8341
9. Alghmgham, D.A., Latif, G., Alghazo, J. and Alzubaidi, L., 2019. Autonomous traffic sign (ATSR)
detection and recognition using deep CNN. Procedia Computer Science, 163, pp.266-274.
10. Dong, K., Zhou, C., Ruan, Y. and Li, Y., 2020, December. MobileNetV2 model for image classification
In 2020 2nd International Conference on Information Technology and Computer Application
(ITCA) (pp. 476-480). IEEE.
11. Nguyen, H., 2020. Fast object detection framework based on mobilenetv2 architecture and enhanced
feature pyramid. J. Theor. Appl. Inf. Technology, 98(05).
12. Khan, M.A., Park, H. and Chae, J., 2023. A lightweight convolutional neural network (CNN)
Architecture for traffic sign recognition in urban road networks. Electronics, 12(8), p.1802.
13. Youssouf, N., 2022. Traffic sign classification using CNN and detection using faster-RCNN and
Yolov4. Heliyon, 8(12).
14. Triki, N., Karray, M. and Ksantini, M., 2023. A real-time traffic sign recognition method using a new
attention-based deep convolutional neural network for smart vehicles. Applied Sciences, 13(8), p.4793.
15. Rani, A.R., Anusha, Y., Cherishama, S.K. and Laxmi, S.V., 2024. Traffic sign detection and recognition
using deep learning-based approach with haze removal for autonomous vehicle navigation. e-Prime-
Advances in Electrical Engineering, Electronics and Energy, 7, p.100442.