Automatic Helmet Detection System
Automatic Helmet Detection System
Abstract: In the developing countries like India, the motorcycle Among various kinds of accidents that occur, Motorcycles
riders are increasing day-by-day, wherein it also constitutes to accidents are abundant and cause serious injuries for both
the unprecedented increase in the number of motorcycle young and adult [7]. To reduce the risk, the motorcyclists
accidents across the country. To overcome this drawback, the should wear a helmet. The safety helmet is the motorcyclist’s
proposed research work explains and demonstrates a method to
fundamental p rotection fro m major accidents and injuries. But
enforce better safety protocols through the automatic detection
of motorcyclists with and without a safety helmet by using a
a large nu mber of the t wo-wheelers neglect the helmets during
real-time traffic surveillance footage. The real-time automatic their rides. The M inistry of the country has made it
detection of motorcyclists with and without a safety helmet is blameworthy to ride a bike without a helmet and
established through detecting a vehicle and track pipelining it implementing many plans of action to snatch the lawbreakers.
with OpenCV, sklearn, utilizing a descriptor known as the The traditional means [15] o f checking that bikers with
histogram of oriented gradients (HOG), and support vector helmets are done by traffic police standing at a junction or by
classification (S VC), which are the combination of tools the captured CCTV flicks and then impose fines to those who
pertaining to machine learning and image processing violate the laws, which takes the human efforts and
mechanisms. With OpenCV Library method, a bike rider is
time-consuming. Automat ion of this process is highly
identified in the surveillance video. Further by using a popular
machine learning algorithm model called LinearS VC, the
required for the government to impose penalties on the
classifier label identifies whether the rider is wearing a safety violators, reduce work-force and time. The smart method one
helmet. The data attained in correspondence to the count of bike can actually think of is having a sensor engine that only starts
riders with and without safety helmet is stored in MySQL when the helmet is properly worn by the person. But, the cost
database with respective timestamps and is also visualized of this setup could be higher than most of the individuals of
through tabular and graphical views in the developed desktop India could not effort it. To effectively address all these
interface application. With 87.6% model accuracy, our paper problems, we would like to bring an efficient approach for
proposes a solution to enhance the existing safety measures and automatic helmet detection using Machine learn ing and image
provide a time-efficient approach to handle traffic regulations.
processing techniques. This method is considered to be
efficient as there are CCTV's installed on roads and consume
Keywords: Machine learning, image processing, histogram of
oriented graphics (HOG), OpenCV, support vector
less time and reduce the need for large human force.
classification, MySQL Connector, LinearS VC, NumPy, Pandas, The suggested approach focusses on the absolute protection to
sklearn the motorcyclists. Though helmet wearing is mandatory,
many of the two-wheeler riders violate it, which leads to
I. INTRODUCTION increment in the deaths occur every year. Hence, automatic
helmet detection is the liab le solution for the government to
Across the world, Two-wheeler has become a popular lay hold of vio lators ensuring the public safety. In the idea to
passage. India with 1.3 b illion population has millions of make traffic and accident system acute, where it provides a
motor vehicles running on the roads. There is a high danger desktop application, wh ich can reduce the efforts of traffic
involved due to less protection. In the past 10 years, there was police and provide the functionality to them to identify the
a rise in the count of accidents occurring due to motorcyclists number of vio lators driving on the roads. The proposed
without a helmet. No wadays, accidents happening on the application will automatically ext ract the images of
roads are one of the significant reasons for hu man deaths. motorcyclists who refused to wear helmets and our system
Authorized licensed use limited to: Auckland University of Technology. Downloaded on October 28,2020 at 08:18:34 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Smart Electronics and Communication (ICOSEC 2020)
IEEE Xplore Part Number: CFP20V90-ART; ISBN: 978-1-7281-5461-9
will save the images for further p rocedure. Images will be mixtu re. After performing the required preprocessing steps ,
extracted fro m the surveillance video [13] on public roads. the classification is done to minimize the capita. Two-stage
Our application will also provide features like the classification is done to acquire the best output and reduce
visualizat ion of data related to the occurrence of vehicles on misclassification. Finally, the biker's head is obtained by
different attributes. The count of two-wheeler vehicles with calculating the ROI of the upper twenty percent of the picture
and without a safety helmet is stored in MySQL database with and converted them to grayscale pictures. Segmented pictures
timestamps to accurately verify the law violators. The are then sent to CNN for the classification of bikers with and
proposed application automatically captures the bikers without the helmet.
without helmet using real-time video mon itoring and helps the
police depart ment to impose a penalty on the motorcyclists , In [5], using HOG as a feature vector and SVM as a classifier,
who violate the rules of wearing a helmet while driving on the classification of b ikers having and not having a helmet is
roads. done in real-time. Background modeling utilizing background
subtraction is done at the beginning which contains an
II. LITERATURE SURVEY extraction of features and classification to differentiate
moving ob jects like cars, hu mans, and bikes fro m non-moving
like roads, trees, and builds which helps in minimizing the
In [1], using Wide-angle and Zoo m cameras, detection of the
efforts in next phases. In the second step classification of the
bikers with and without a helmet is depicted. The wide-angle biker is done. Different varieties of features and kernels are
lens captures the image intersection that detects the biker in a combined to inspect classification outputs. An average of
frame of width less than 80 and height less than 120 using the
11.58 milliseconds is taken to process a video of 30 frames per
HAAR a lgorith m, which is the first step and zoomed lens to seconds
detect the license plates. The next step is to verify all the
frames whether a biker is having helmet or not and creates the
boundaries to the head of a biker of height half of the area and
III. BASIC CONCEPTS
width one-third of the area of the frame, where the b iker is
detected. If the biker lacks the helmet, the zoo m lens spots the Classification is implemented to identify the set of classes that
number plate and stores the related data and pictures in the they are belongs to, by the data that is used for training or the
database, which acts as a proof for t raffic wing to impose instances of whose category or group is already known. There
fines. The obtained output is compared with manual outcomes are two major categories in the classification. They are
with 0.05 significance. Supervised classification is human-guided classification. The
outcome is based on the data instance or the image that are
In [2], by using segmentation, classification of moving representative of specific classes or groups. The training sets
objects, and helmet detection, and a biker is spotted with and are created based on the user knowledge.
without safety helmet. Seg mentation of moving objects Unsupervised classification is a software-based classification.
requires a background picture, which is reformed by using The outcome is based on the analysis of software without
adaptive mixture of Gaussians, diminishes the processing providing the classes by the user. The user will provide the
time and false-positives in classifiers. The obtained outcome number of output classes and the software that is used.
is transformed to feature vector, which helps in classifying the Support Vector Machine (SVM) [8] Classifier is a supervised
vehicles like bikes and non-bikes by using descriptors. classification algorith m. It is used for two-group classification
Finally, helmet identification is done with Region of Interest problems. Support Vector Machine is a fast and dependable
(ROI) determination to reduce computational capita and algorith m that provides accurate result when the input data is
helmet identification, image classificat ion using six different limited and sufficient. SVM uses hyperplanes to classify the
classifiers, and extract ion of features using three types of classes of data points.
descriptors. MLP using HOG obtained the best outcome with Hyperplanes are decision boundaries that helps to classify the
0.9137 accuracy classes of the datasets. Data that is falling on one phase or side
of the hyperplane is considered as one class and other side is
In [3], the paper focuses on deep learning strategies to classify classified as another class.
the bikers with and without a helmet. The evolution in CNN The aspect of the hyperplane will be decided by the count of
and SSD capable of seg mentation and classification of features. The hyperplane will be a straight line, if the input
images, once in runtime by creating bounding boxes. features are two. The hyperplane will be a 2-D plane, if the
Classification of images with and without a helmet is done by input features is three. If the features exceed three, then it will
CNN models like VGG19, VGG16, Mobile Nets, and Google be difficult to use the hyperplanes.
Nets (Inception_v3). Neural Net is trained with python The Object detection [9] is a technology that is used to detect
TensorFlow and accuracy is determined. SSD model can the things or objects from the input data that are given to the
collaborate with other nets structures such as Google Net and system. Object detection has mult iple applications like face
Mobile Nets into one net which constructs structure faster. detection, self-driving cars, obstacle detection, security
Among the four CNN models, Mobile Nets got the best systems etc.
outcome. Feature Extract ion [11] will reduce the number of resources
In [4], the fully enhanced real-t ime structure determines i.e. memo ry, co mputation power etc. that are required to
motorcyclists with and without helmets fro m vigilance using describe the data while applying classificat ion algorith m.
HOG as classifier and LBP as a feature vector. In itially, each Classification algorithm may overfit if the large number of
frame is studied fro m footage and extraction of vehicles variables are used in the classification.
moving is obtained by eliminating unnecessary areas, then The system requires large memo ry and co mputation power if
extracted pictures are enhanced by using an adaptive Gaussian we used the data directly without applying the pre-processing
techniques on the input data. The accuracy of the model will
Authorized licensed use limited to: Auckland University of Technology. Downloaded on October 28,2020 at 08:18:34 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Smart Electronics and Communication (ICOSEC 2020)
IEEE Xplore Part Number: CFP20V90-ART; ISBN: 978-1-7281-5461-9
depend on the methods that are used to decrease the number the features from each frame. The frames are captured using
variable as possible. the surveillance camera and these frames are used as a
Image resizing is the procedure to change the image fro m one background. The static objects in the environment i.e. trees,
size to another i.e. fro m m x n to a x b. The resizing can be parked vehicles, buildings etc. are captured as a background
done by scaling, cropping, padding. and used for further process. Fig 2.a indicates the image that
In scaling, we use some factors called as scale factors to contains the background.
change the size of the image. In cropping, we cut the pixels
based on the required size. In padding, we add the dummy
2. Moving Object Segmentation:
pixels based on the required size of the image.
Image blurring technique is used to highlight the required This process reduces the processing time because it eliminates
portion of the image. If we observe the blurred image the unnecessary part of the frames and provides the outcome
carefully, the image seems like s mooth and the edges are not for the classification. In this step we will detect the interested
observed. Generally, low pass filter is used for the blurring. area fro m the image and given as an input for further process.
The low frequency will be considered but the high frequency In this step we have to provide a cross line (CL), it is marked
will be prevented by this filter. Here frequency is the by us at a time we are starting the system. This cross line (CL)
difference in pixel values. must be cross the road. When a vehicle crosses this CL, the
A very important step required to recognize the region of moving object [14] segmentation process will start.
interest is to extract the pixels that belongs to the region of The frame is captured fro m the surveillance camera and the
interest. background elimination will be done by subtracting the
Image thresholding is an era of image segmentation that captured frame with the background image. Fig 2.b and 2.c
separate the image by converting the grayscale image into the shows the captured frame and background eliminated image
binary images [6]. It is effect ive for the images those have the respectively. The background eliminated image is converted
high level of contrast. into the grayscale image. The Gray scale image should be
resized to the required size using cropping or padding.
IV. PROPOSED M ETHODOLOGY
The proposed system can be divided into the two main parts.
The first part consists of segmentation and the second on
consists of classification of images. The first step is used to
capture the moving objects in the surveillance camera. The
vehicles are classified into two classes because it is necessary
to know that vehicle is motorcycle or not. And the second step
consists of helmet detection [12]. For vehicle classification
and helmet detection we used the Support Vector a b
Classification (SVC). Figure 1 describes each and every step
of the system to detect the motorcyclist’s with and without
helmet.
Proposed System:
c
Figure 2: a) Background Scenario b) Frame c) Background
Elimination
3. Vehicle Classification:
Before classifying the image, It is necessary to extract the
features of the images. To extract the features fro m the
frames, we use descriptors. The descriptors provides set of
values as an outcome. This is called as “feature vector”. This
feature vector is used by the classifier to classify the each
Figure 1: Proposed System frame i.e. outcome of the previous steps.
Histogram Oriented Gradients (HOG) descriptor is used to
1. Background Detection:
extract the features from the images or frames.
To detect the moving objects, we have to detect t he
The aim o f any classifier is to use the features of the object to
background of the scene. This background is used to extract identify the object that what class it belogs to. To classify the
Authorized licensed use limited to: Auckland University of Technology. Downloaded on October 28,2020 at 08:18:34 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Smart Electronics and Communication (ICOSEC 2020)
IEEE Xplore Part Number: CFP20V90-ART; ISBN: 978-1-7281-5461-9
vehicle we used the SVM Classifier[8]. This classification motorcycle and helmet in different frames, our dataset
consists two classes of motorcycle and non-motorcycle. If the collection is complete.
outcome of the classifier is 0, then the frame contains a
vehicle but not a motorcycle. If the outcome of the classifier is
1, then there is a motorcycle in the frame. So it goes to the
helmet detection phase. If the outcome is 2, then the frame
doesn’t contain a vehicle.
4. Helmet Detection:
Before detecting the helmet, we have to calcu late the Regoin
of Interest (RoI), Feature Ext raction, and Image classification.
The Region of Interest (RoI) is an important part in helmet
detection. The motorcyclist’s head must be in the part of RoI.
Because of RoI the co mputation cost and time are drastically
decreased. Actually top of the image is used as a region of
interest.
For extract ing the features from the frames, we used the
descriptor named HOG. The gray scale image was used for
Figure 3: Annotated Video Clips
the feature extraction process. HOG descriptor has bees set up
with a 16 histograms by 16 part ition window. This way, a
The dataset is categorized into two parts weighing 80% of the
vector of 256 features are generated. Here, we can use large
entire collect ion for train ing and the rest is utilized fo r testing.
variation of histogram and partiton window sizes to get the
best results.
Step 2:
For image classification process [10], we used Linear SVM
a. LinearSVC model is chosen as our image
classifier. If the outcome of the helmet detection classifier is
0, then the motorcyclist with a safety helmet. Otherwise the classification depiction. It determines a hyperplane which
categorizes whether the object in the frame is a
motorcyclist without a safety helmet..
motorcycle o r not. Likewise, a separate LinearSVC
model is developed using the helmet dataset to classify a
V. IMPLEMENTATION AND RESULTS helmet.
b. The inputted video data is collectively taken as
Step 1: frames. Each frame is a combination of several p ixels.
a. Machine Learning algorith ms use data as their The image is identified through its pixels. These pixilated
source of learning. To attain a well-detailed set of data forms of nu merical demonstrate a two-d imensional
points, we have based the CCTV surveillance footage as Nu mPy array. Through various feature extraction
our foundation to develop the algorith m, since the road parameters such as spatial _feat and hog channel, our
user summative and swift motorization are similar to the model is fed the data for classification predictions.
situation of the real-world. c. With LinearSVC’s mu lti-class strategy, the
b. The recorded samp les of our data are huge in formulated mu lti-class SVM determines the necessary
number, as shown in table 1. To train a model of such co-efficient and weights (sample_ weight parameter) of
high complexity is resource-wise impossible. To alter the the model to increase the accuracy rate of its predictions.
situation, we have extracted 1000 video c lip samp les to Predictions are represented in tables 2 and 3. The model
form the necessary dataset. Each video clip has an training parameters are shown in figure 4.
average duration of ten seconds portraying
non-overlapping vehicle passages . Vehicle Object Prediction
Classifier Value
Motorcycle 0
Non– 2
Motorcycle
Authorized licensed use limited to: Auckland University of Technology. Downloaded on October 28,2020 at 08:18:34 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Smart Electronics and Communication (ICOSEC 2020)
IEEE Xplore Part Number: CFP20V90-ART; ISBN: 978-1-7281-5461-9
Step 3:
a. The trained pickled model is loaded through
[Link]() into our desktop interface application.
b. HOG Descriptors are used for foreground
identification of the necessary object in particular frames.
Resizing through pixels_per_cell and using
transform_sqrt for power low co mpression, the object
identification is done through feature extraction fro m the
image data. Figure 8: Live Streaming Count
c. Keep track of the entire numerical data
corresponding to the number of t wo-wheelers in the e. Through Pandas library, a tabular and graphical
frame, motorcyclists with helmets, and motorcyclists representation of the data attained is demonstrated as
without a helmet. Update the count to MySQL Database shown in figures 9 and 10.
through [Link](), as shown in figure
6.
Authorized licensed use limited to: Auckland University of Technology. Downloaded on October 28,2020 at 08:18:34 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the International Conference on Smart Electronics and Communication (ICOSEC 2020)
IEEE Xplore Part Number: CFP20V90-ART; ISBN: 978-1-7281-5461-9
VI. CONCLUSION AND FUTURE WORK [6] K. Dahiya, D. Singh, C.K. Mohan, “Automatic detection
of bike-riders without helmet using surveillance videos in
real-t ime”. International Jo int Conference on Neural
Through this paper, a machine learning based approach is
Networks (IJCNN), IEEE (2016), pp. 3046-3051
positioned to identify the helmet usage among motorcyclists.
Video frames attained from surveillance footage, the object
[7] M .C. Fong, J.R. Measelle, J.L. Dwyer, Y.K. Taylor, A.
detection-based algorithm is trained to spot motorcycles and
Mobasser, T.M. Strong, S. Werner, Ouansavanh, A.
their helmet. Through various tools and methods
Mounmingkham, M. Kasuavang, et al. “Rates of
corresponding to OpenCV and support vector classification,
motorcycle helmet use and reasons for non-use among
the desktop interface application is made possible to visualize
adults and children” in Luang Prabang, Lao People's
the live streaming traffic surveillance footage. When trained,
Democratic Republic BM C Public Health, 15 (1) (2015),
the LinearSVC algorith m acts in correspondence to the
p. 970
already existing road traffic surveillance and produces the
corresponding data related to the usage of helmets owned by
[8] J. Li et al., "Safety helmet wearing detection based on
motorcyclists. The implemented algorithm has demonstrated
image p rocessing and machine learn ing," 2017 Ninth
high accuracy results and diverge slightly when the object is
International Conference on Advanced Computational
further away fro m the field. W ith 87.6% model accuracy rate,
Intelligence (ICA CI), Doha, 2017, pp. 201-205, doi:
this research work has proposed a solution to enhance the
10.1109/ICA CI.2017.7974509.
driving safety measurements, wh ich in turn deploys a
time-efficient approach to handle the t raffic regulat ions. An
[9] C. Ch iu, M. Ku and H. Chen, "Motorcycle Detection and
approach has to be proposed to identify the license number
Tracking System with Occlusion Seg mentation," Eighth
plate of each motorcyclist, who is disregarding the safety
International Workshop on Image Analysis for
protocols in the future. The identified number p late will be
Multimedia Interactive Services (WIAMIS '07),
updated and stored in the MySQL database to take further
Santorini, 2007, pp. 32-32, doi:
action on the riders. This method could eventually provide a
10.1109/WIAMIS.2007.60.
time-and-effort efficient approach to tackle road safety
abusers. Greater accuracy generating image classificat ion and
[10] J. Chiverton, "Helmet presence classification with
recognition algorith ms such as CNN will prove to be more
motorcycle detection and tracking," in IET Intelligent
beneficial. The only subsidiary due to the usage of CNN is the
Transport Systems, vol. 6, no. 3, pp. 259-269, September
requirement for larger data processing units and increases the
2012, doi: 10.1049/iet-its.2011.0138.
corresponding time taken for training the model.
[11] N. Dalal and B. Triggs, "Histograms of oriented
VII. REFERENCES gradients for human detection," 2005 IEEE Co mputer
[1] Wonghabut, J Ku mphong, T Satiennam, R Society Conference on Co mputer Vision and Pattern
Ung-arunyawee1, and W Leelapatra, “Automat ic Recognition (CVPR'05), San Diego, CA, USA, 2005, pp.
helmet-wearing detection for law enforcement using 886-893 vol. 1, doi: 10.1109/CVPR.2005.177.
CCTV cameras”. IOP Conf. Series: Earth and
Environmental Science 143 (2018) 012063 [12] C. Vishnu, D. Singh, C. K. Mohan and S. Babu,
"Detection of motorcyclists without helmet in videos
[2] e Silva, Ro muere Rodrigues Veloso, Kelson Rômu lo using convolutional neural network," 2017 International
Teixeira Aires, and Rodrigo de Melo Souza Veras. Joint Conference on Neural Networks (IJCNN),
"Helmet detection on motorcyclists using image Anchorage, AK, 2017, pp. 3036-3041, doi:
descriptors and classifiers." 2014 27th SIBGRAPI 10.1109/IJCNN.2017.7966233.
Conference on Graphics, Patterns and Images. IEEE,
2014 [13] W. Hu, T. Tan, L. Wang, and S. Maybank, “A survey on
visual surveillance of object motion and behaviors,”
[3] N. Boonsirisumpun, W. Puarungroj and P. IEEE Trans. Systems, Man, and Cybernetics, Part C:
Wairotchanaphuttha, "Automatic Detector for Bikers Applications and Reviews, vol. 34, no. 3,pp. 334– 352,
with no Helmet using Deep Learning," 2018 22nd 2004.
International Computer Science and Engineering
Conference (ICSEC), Chiang Mai, Thailand, 2018, pp. [14] Ramirez-Qu intana, Juan Alberto, and Mario Ignacio
1-4, doi: 10.1109/ICSEC.2018.8712778. Chacon-Murguia. "Self-adaptive SOM-CNN neural
system for dynamic object detection in normal and
[4] Shine, L., C. V., J. Automated detection of helmet on complex scenarios." Pattern Recognition 48.4 (2015):
motorcyclists from traffic surveillance videos: a 1137-1149.
comparative analysis using hand-crafted features and
CNN. Multimed Tools Appl 79, 14179–14199 (2020). [15] Y. Ku lkarni, S. Bodkhe, A. Kamthe and A. Patil,
"Automatic nu mber plate recognition for motorcyclists
[5] Kunal Dahiya, Dinesh Singh, C. Krishna Mohan , rid ing without helmet," 2018 International Conference on
“Automatic Detection of Bike-riders without Helmet Current Trends towards Converging Technologies
using Surveillance Videos in Real-time”. Conference (ICCTCT), Coimbatore, 2018, pp. 1-6, doi:
Paper · July 2016 ,DOI: 10.1109/IJCNN.2016.7727586 10.1109/ICCTCT.2018.8551001.
Authorized licensed use limited to: Auckland University of Technology. Downloaded on October 28,2020 at 08:18:34 UTC from IEEE Xplore. Restrictions apply.