International Journal of Progressive Research in Science and Engineering

Volume-1, Issue-5, August-2020

[Link]

Fire Detection Using Deep Learning

Suhas G1, Chetan Kumar1, Abhishek B S1, Digvijay Gowda K A1, Prajwal R1
1
Student, Department of Computer Science and Engineering, Maharaja Institute of Technology Mysore, Karnataka, India.
Corresponding Author: chetanbhat34@[Link]

Abstract: From sprawling urbans to dense jungles, fire accidents pose a major threat to the world. These could be prevented by
deploying fire detection systems, but the prohibitive cost, false alarms, need for dedicated infrastructure, and the overall lack of
robustness of the present hardware and software-based detection systems have served as roadblocks in this direction. In this work,
we endeavor to make a stride towards detection of fire in videos using Deep learning. Deep learning is an emerging concept based
on artificial neural networks and has achieved exceptional results in various fields including computer vision. We plan to overcome
the shortcomings of the present systems and provide an accurate and precise system to detect fires as early as possible and capable
of working in various environments thereby saving innumerable lives and resources.

Key Words: - Fire accidents, Fire detection, Surveillance video, Machine learning, Deep Learning, Transfer Learning.

I. INTRODUCTION manual work. This model can be used to detect fires in

surveillance videos. Unlike existing systems, this neither
Fire accidents pose a serious threat to industries, requires special infrastructure for setup like hardware-based
crowded events, social gatherings, and densely populated solutions, nor does it need domain knowledge and prohibitive
areas that are observed across India. These kinds of incidents computation for development.
may cause damage to property, environment, and pose a threat
to human and animal life. According to the recent National II. RELATED WORK
Risk Survey Report [1], Fire stood at the third position Among the different computer-based approaches to
overtaking corruption, terrorism, and insurgency thus posing detect fire, the prominent approaches we found were using
a significant risk to our country’s economy and citizens. The Artificial Neural network, Deep Learning, Transfer learning
recent forest-fires in Australia reminded the world, the and convolutional neural network. Artificial Neural Network
destructive capability of fire and the impending ecological based approaches seen in paper [2] uses Levenberg-
disaster, by claiming millions of lives resulting in billions of Maraquardt training algorithm for a fast solution. The
dollars in damage. accuracy of the algorithm altered between 61% to 92%. False
positives ranged from 8% to 51%. This approach yielded high
Early detection of fire-accidents can save accuracy and low false positive rate, yet it requires immense
innumerable lives along with saving properties from domain knowledge.
permanent infrastructure damage and the consequent financial
losses. In order to achieve high accuracy and robustness in In this paper [3], The author says that the present
dense urban areas, detection through local surveillance is hardware-based detection systems offer low accuracy along
necessary and also effective. Traditional opto-electronic fire with high occurrence of false alarms consequently making it
detection systems have major disadvantages: Requirement of more likely to misclassify actual fires. It is also not suitable
separate and often redundant systems, fault-prone hardware for detecting fires breaking out in large areas such as forests,
systems, regular maintenance, false alarms and so on. Usage warehouses, fields, buildings or oil reservoirs. The authors
of sensors in hot, dusty industrial conditions is also not used a simplified YOLO (You Only Look Once) model with
possible. Thus, detecting fires through surveillance video 12 layers. Image augmentation techniques such as rotation,
stream is one of the most feasible, cost-effective solution adjusting contrast, zooming in/out, saturation and aspect ratio
suitable for replacement of existing systems without the need were used to create multiple samples of each image, forming
for large infrastructure installation or investment. The existing 1720 samples in total. It aims to draw a bounding box around
video-based machine learning models rely heavily on domain the flame region. It outperformed existing models when the
knowledge and feature engineering to achieve detection color features of the flames varied from those in training set.
therefore, have to be updated to meet new threats.
Paper [4] provides two approaches First approach is
We aim to develop a classification model using Deep to perform training on the data set using Transfer Learning
learning and Transfer Learning to recognise fires in and later fine tune it. The next approach was to extract flame
images/video frames, thus ensuring early detection and save features, fuse them and classify it using a machine learning
1
 International Journal of Progressive Research in Science and Engineering
Volume-1, Issue-5, August-2020
[Link]

classifier. The transfer learning algorithms used were Source/input data which is in the form of videos is
Xception, Inception V3, ResNet-50, trained in ImageNet. In split into frames and preprocessed to convert it into a format
the first approach, accuracy up to 96% was achieved. The that is suitable to be fed as input to pre-built models for feature
second approach, stacking Xgboost and lightbgm achieved an extraction. The deep learning model returns a feature vector
AUC of 0.996. Transfer learning models greatly reduces the which is also known in transfer learning terminology as
training time required for our model. It requires bottleneck features.
comparatively smaller data set. Both approaches don't require
any sort of domain knowledge. In works [7] and [8], Deep
CNN approach was taken to detection and localization of
fires. The accuracy obtained was between 90 to 97% in both
of these papers. This approach is time consuming and training
was performed using Nvidia GTX Titan X with 12 GB of
onboard memory.

Traditional Machine learning using feature

extraction yielded high accuracy and low false positive rate,
yet it requires immense domain knowledge i.e., about color- Fig.1: System Architecture
model, color-space, patterns and motion vectors of flames.
In the next stage, the bottleneck features are passed
when the object changes, the models need to be rebuilt for the
through a classification model to obtain the result, which may
new objects. The traditional approach to feature engineering
be either fire of Non-fire. The classification model was built
[12] is manual in nature. It involves handcrafting features
through training using the training data set.
incrementally using domain knowledge, a tedious, time-
consuming, and error-prone process. The resultant model is
The result of the classification is displayed to the
problem dependent and might not perform well on new data.
user, and depending on the result, further actions are taken. If
Automated feature engineering ([3][5]) improves upon this
the result is a fire then, an email is sent to the concerned
inefficient workflow by automatically extracting useful and
stakeholders along with the video frame and date-time stamp
meaningful features from data with a framework that can be
to alert them. The email to which the mail is delivered can be
applied to any problem. It not only cuts down on the time
changed by the user. An entry will also be made in the cloud
spent, but creates features that can be interpreted and prevents
database for the purpose of analysis.
data leakage. With transfer learning, instead of developing a
model from scratch, we can start from a pre-trained model
with necessary fine-tunings. These models can be imported
directly from Keras. The use of pre-trained models saves a lot
of computational work, which otherwise, would require high
end GPU’s. Inception V3, Inception-ResNet-V2 were found
to be ideal algorithms for feature extraction as they showed
promising results with high accuracy ([3]). With transfer
learning, instead of developing a model from scratch, we can
start from a pre-trained model with necessary fine-tunings.
These models can be imported directly from Keras. The use
of pre-trained models saves a lot of computational work,
which otherwise, would require high end GPU’s. Inception
V3, Inception-ResNet-V2 were found to be ideal algorithms
for feature extraction as they showed promising results with
high accuracy.

III. SYSTEM ARCHITECTURE

The passive components of the system include data

preprocessing, feature engineering, model selection scripts
which were used to train and develop machine learning
Fig.2: Activity Diagram
model.
2
 International Journal of Progressive Research in Science and Engineering
Volume-1, Issue-5, August-2020
[Link]

IV. METHODOLOGY 97.46% and 97.66% respectively. Hence this would be used
in our application.
The model is divided into two parts
Table.1: Performance evaluation results of different model
1. Data Collection and Pre-processing. combinations

2. Building fire detection model by Transfer Network Algorithm Accuracy Precision Recall
Learning. Decision
94.78% 93.98% 94.44%
Tree
The first step is to gather video frames for the
Naïve
problem statement. The dataset has 2 classes - fire and non- Bayes
93.17% 91.92% 92.98%
Resnet 50
fire. Positive samples consist of images with real fire. False
Positives consists of images which have objects that look Logistic
97.73% 97.30% 97.66%
Regression
like fire but are not. False positives are easier to collect.
Thus, we need to collect diverse video frames which will SVM 97.80% 97.46% 97.66%
help better fire detection. The collected dataset is divided into Decision
90.51% 89.59% 89.25%
train and test video frames. The dataset currently has 1678 Tree
fire images/video frames and 1368 that of non-fire sourced Naïve
from google since there is no standard data set available. Inception- 94.32% 91.75% 95.97%
Bayes
Net V2
Logistic
The second step is to use various available pre- Regression
96.81% 96.23% 96.71%
trained models in Keras to extract the video frame features.
SVM 96.25% 94.71% 97.07%
The pre-trained models are trained on very large-scale video
frames classification problems. The convolutional layer’s Decision
92.58% 92.07% 91.37%
act as feature extractor and the fully connected layers’ act as Tree
Classifiers. Since these models are very large and have seen Naïve
Inception 94.71% 92.63% 96.19%
a huge number of images, they tend to learn very good, Bayes
V3
discriminative features. In order to do extract, the video Logistic
frames feature we remove the last layer i.e. fully connected 97.17% 96.53% 97.22%
Regression
layer. This provides us with a feature vector. The feature
SVM 96.51% 95.36% 97.00%
vector sizes differ from model to model. The central concept
of Transfer Learning is to use a more complex but successful
pre-trained DNN model to transfer its learning to our more
simplified problem. Instead of creating and training deep V. RESULTS AND DISCUSSION
neural nets from scratch (which takes significant time and The aim of our work was to develop to an application
computing resources), we use the pre-trained weights of capable of detecting fire in videos and images, which is robust
these deep neural net architectures (trained on ImageNet) and works in any environment. In this regard, we have
and use it for our own dataset. We have used ResNet-50, experimented with various deep learning models and
InceptionV3 and InceptionResNetV2. models to extract the classification models and have selected ResNet-50-SVM
features and various ML algorithms [SVM, Logistic combination for implementation as it offered the best
Regression, Naive Bayes and Decision Tree] on the performance metric values (Accuracy, Precision and Recall
extracted features to detect fire in video frames. values for this combination was 97.8%, 97.46% and 97.66%
respectively). An email alert feature has also been
All the possible feature-extractor and classifier combinations incorporated to our application to provide real time alerts to
were evaluated using stratified K-fold validation. Table 1 the concerned stakeholders along with a logging system,
shows the obtained performance metric values for different which is implemented using Firebase. The GUI offers a user-
combinations of deep learning networks and classifier friendly experience and allows user with non-technical
algorithms. The best performance was observed with ResNet- background to make use of the application. The application
50 as the deep learning feature extraction model and Support performed exceptionally well during testing. It was able to
Vector Machine as the ML classifier and the Accuracy, identify fires in all of the twelve test fire videos but
Precision and Recall values for this combination was 97.8%, misclassified some instances of non-fire videos.

3
 International Journal of Progressive Research in Science and Engineering
Volume-1, Issue-5, August-2020
[Link]

Compared to existing hardware solutions, our Piece of cloth set on fire

application is affordable, robust, reliable and provides high Fire Fire P
by a firefighter
performance without the need for setting up a dedicated
infrastructure. Due to the use of deep learning and transfer Trash can set on
Fire Fire P
learning techniques, our model is easier to build, alter, Fire
upgrade, requires fewer computing resources and offers better Close-up in view
performance than existing software solutions that make Fire Fire P
of fire
extensive use of feature engineering, domain knowledge.
Huge Campfire Fire Fire P
Table.2: Non-fire videos testing results. Forest Fire Fire Fire P

Test Bush fire Fire Fire P

Test Case Expected Actual
Status
Description Output Output Fire on mattress Fire Fire P
(P/F)
Video with visuals of
mountains, pools, Non-fire Non-fire P
indoors, driving
VI. CONCLUSION
Headlight Glare during
Non-fire Fire F
night
The present decade is marked by huge strides in
Kid playing with toys Non-fire Non-fire P areas of processing, computation and algorithms. This has
Man riding bicycle Non-fire Non-fire P enabled great progress in many fields including processing of
Close up view of light surveillance video streams for recognizing abnormal or
Non-fire Non-fire P
through of fabric unusual events and actions. Fire accidents have caused death
Daytime Traffic Non-fire Non-fire P and destruction all over the world, consuming countless lives
Camera focused on sun and causing billions in damages. This implies that developing
Non-fire Non-fire P an accurate, early, affordable fire-detection system is
in a garden setting
Man walking with an imperative Therefore, we have proposed a fire detection
orange bag in an office Non-fire Non-fire P model for videos/video frames using transfer learning for
environment deep learning. The models make use of ResNet-50,
Lake with yachts and InceptionV3 and Inception-ResNet-V2 models to extract the
Non-fire Non-fire P
windmill features and various ML algorithms such as SVM, Logistic
Yellow heavy-duty Regression, Naive Bayes and Decision Tree on the extracted
vehicles moving on a Non-fire Non-fire P features to detect fire in video frames. Looking at the results,
flyover ResNet-50 with SVM works best for our problem statement.
People waiting at train Coming to the application on the whole, it works in real-time
terminal with a train Non-fire Non-fire P and has the ability to send alert emails along with offering a
approaching
user-friendly graphical interface. It’s cost-effective, reliable,
People moving inside an robust, accurate compared to existing opto-electronic
Non-fire Non-fire P
airport, subway terminal
hardware and software-based systems in the market.
Table.3: Fire videos testing results
VII. FUTURE SCOPE
Test
Expected Actual The application can be enhanced by training the
Test Case Description Status
Output Output model with a larger dataset consisting of fires at various stages
P/F
and dimensions. With higher GPU memory, we could use two
Bus on Fire Fire Fire P deep learning models for feature extraction, whose output
Kitchen sink fire Fire Fire P
feature vectors are concatenated and classified to offer more
robustness. An R-CNN model can be used to implement fire
Twigs and leaves on fire Fire Fire P localization along with classification. We can also expect
better deep learning architectures to emerge in the future,
Fireplace Fire Fire P
offering better feature extraction. The application will also
Automobile on Fire Fire Fire P offer a considerably better performance when run on
4
 International Journal of Progressive Research in Science and Engineering
Volume-1, Issue-5, August-2020
[Link]

machines having better processing power compared to sequences,” The 26th Chinese Control and Decision
existing one of which it has been developed. Conference (2014 CCDC), Changsha, 2014, pp. 3005-
3009.
REFERENCES [15]. Toulouse, Tom & Rossi, Lucile & Celik, Turgay &
Akhloufi, Moulay. (2015). “Automatic fire pixel detection
[1]. National Risk Survey Report - Pinkerton, FICCI (2018). using image processing: A comparative analysis of Rule-
[2]. Janku P., Kominkova Oplatkova Z., Dulik T., Snopek P. based and Machine Learning-based methods. Signal,
and Liba J. 2018. “Fire Detection in Video Stream by Image and Video Processing”.
Using Simple Artificial Neural”. Network. MENDEL. 24, [16]. K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual
2 (Dec. 2018), 55–60. Learning for Image Recognition,” 2016 IEEE Conference
[3]. Shen, D., Chen, X., Nguyen, M., & Yan, W. Q. (2018). on Computer Vision and Pattern Recognition (CVPR), Las
“Flame detection using deep learning”. 2018 4th Vegas, NV, 2016, pp. 770-778.
International Conference on Control, Automation and [17]. S. Wilson, S. P. Varghese, G. A. Nikhil, I. Manolekshmi
Robotics (ICCAR). and P. G. Raji, “A Comprehensive Study on Fire
[4]. Li, C., & Bai, Y. (2018). “Fire Flame Image Detection Detection,” 2018 Conference on Emerging Devices and
Based on Transfer Learning”. 2018 5th IEEE International Smart Systems (ICEDSS), Tiruchengode, 2018, pp. 242-
Conference on Cloud Computing and Intelligence Systems 246.
(CCIS). [18]. X. Wu, X. Lu and H. Leung, “An adaptive threshold deep
[5]. K. Muhammad, J. Ahmad, I. Mehmood, S. Rho and S. W. learning method for fire and smoke detection,” 2017 IEEE
Baik, “Convolutional Neural Networks Based Fire International Conference on Systems, Man, and
Detection in Surveillance Videos," in IEEE Access, vol. 6, Cybernetics (SMC), Banff, AB, 2017, pp. 1954-1959.
pp. 18174-18183, 2018. [19]. Chenebert, Audrey & Breckon, Toby & Gaszczak, Anna.
[6]. S. J. Pan and Q. Yang, "A Survey on Transfer Learning," (2011). “A nontemporal texture driven approach to real-
in IEEE Transactions on Knowledge and Data time fire detection”.
Engineering, vol. 22, no. 10, pp. 1345-1359, Oct. 2010. [20]. Seebamrungsat, Jareerat et al. “Fire detection in the
[7]. [Zhang, Qingjie & Xu, Jiaolong & Xu, Liang & Guo, buildings using image processing.” 2014 Third ICT
Haifeng. (2016). “Deep Convolutional Neural Networks International Student Project Conference (ICTISPC)
for Forest Fire Detection”. (2014): 95-98.
[8]. K. Muhammad, J. Ahmad, Z. Lv, P. Bellavista, P. Yang
and S. W. Baik, “Efficient Deep CNN-Based Fire
Detection and Localization in Video Surveillance
Applications,” in IEEE Transactions on Systems, Man,
and Cybernetics: Systems, vol. 49, no. 7, pp. 1419-1434,
July 2019.
[9]. Z. Jiao et al., “A Deep Learning Based Forest Fire
Detection Approach Using UAV and YOLOv3,” 2019
1st International Conference on Industrial Artificial
Intelligence (IAI), Shenyang, China, 2019.
[10]. Faming Gong ,1 Chuantao Li,1 Wenjuan Gong ,1 Xin Li,1
Xiangbing Yuan,2 Yuhui Ma and Tao Song. “A Real-
Time Fire Detection Method from Video with Multi-
feature Fusion”. Hindawi, Computational Intelligence and
Neuroscience, Volume 2019.
[11]. L. Shao, F. Zhu and X. Li, “Transfer Learning for Visual
Categorization: A Survey,” in IEEE Transactions on
Neural Networks and Learning Systems, vol. 26, no. 5, pp.
1019-1034, May 2015.
[12]. S. Mohd Razmi, N. Saad and V. S. Asirvadam, “Vision-
based flame detection: Motion detection & fire analysis,”
2010 IEEE Student Conference on Research and
Development (SCOReD), Putrajaya, 2010, pp. 187-191.
[13]. T. Qiu, Y. Yan and G. Lu, “A new edge detection
algorithm for flame image processing,” 2011 IEEE
International Instrumentation and Measurement
Technology Conference, Binjiang, 2011, pp. 1-4.
[14]. M. Ligang, C. Yanjun and W. Aizhong, “Flame region
detection using color and motion features in video
5