Driver’s Drowsiness using

OpenCV & Keras

Submitted By

FATHIMA DILSHANA PK
PYTHON & DATA SCIENCE

IPCS GLOBAL,
THIRUVANANTHAPURAM

UNDER THE GUIDANCE OF:

MRS. NISHA ABHISHEK

Academic Head
IPCS GLOBAL ,
THIRUVANANTHAPURAM
 DECLARATION
I hereby declare that the project work entitled “DRIVER’S DROWSINESS USING
OPENCV & KERAS” submitted, is a record of an original work done by me under the
guidance of Mrs. Nisha Abishek, Academic head of IPCS Global Trivandrum. This
project work is submitted in the partial fulfillment of the requirements for the award of
the degree of Python & Data Science. The result embodied in this thesis have not been
submitted to any other University or Institute for the award of any degree or diploma.

Student Name: Fathima Dilshana PK

Place: Trivandrum
Date : 21-10-2023

2
 ABSTRACT
Nowadays, transport facility is a basic component of living. While driving, any one of us
can come across an accident, that may be due to lack of sleep, uneven physical health,
or prolonged travelling. The lack of sleep or any unfair condition deviates from a higher
chance of accidents. The weakness or sluggishness are among the major causes for
road accidents. Due to this, there is a higher number of accidents annually across the
globe. There have been several vehicle’s accidents across the globe in recent years.
Several vehicles are running over the road day and night. Due to continuous travelling,
it becomes obvious to get tired and the same can be measured in terms of drowsiness.
Due to lack of sleep, it becomes very dangerous to drive. Hence, it is an urgent need to
catch the fatigue nature of the motorist to protect his survival as well as of others. Driver
Drowsiness Detection System is the main target of this paper.

In this model, the camera being the initial point of the system by providing the live feed
of the driver to the framework that concentrates it straight towards the face of driver and
checks the driver's eyes with a particular objective to catch drowsiness of the driver. On
analyzing the live video an alert is issued to the driver in circumstances where
drowsiness is outcome of the analysis. The framework moves the control of the program
forward using information picked up from the picture to find the facial tourist spots,
which helps the system to identify where the eye’s location of an individual exist. If the
eyes of driver are closed for a specific amount of time, the proposed framework draws a
conclusion that the driver is feeling drowsy and an alarm for safety is sound. The
system works after initially face is recognized and eyes are spotted, it also works well in
dim lighting conditions too.

3
 CONTENT
Driver’s Drowsiness using OpenCV & Keras....................................................1
Submitted By................................................................................................1
FATHIMA DILSHANA PK.................................................................................1
DECLARATION..................................................................................................2
ABSTRACT........................................................................................................3
CONTENT.........................................................................................................4
TABLE OF FIGURE......................................................................................5
1. INTRODUCTION..........................................................................................6
1.1. PRIMARY GOAL...................................................................................6
1.2. COMPUTER’S VISION.........................................................................7
1.3. OPEN CV.............................................................................................7
1.4. WHY OPEN CV?..................................................................................7
1.5. MACHINE LEARNING........................................................................8
1.6. OPEN CV’S MACHINE LEARNING ALGORITHM............................8
1.7. TRANSFER LEARNING......................................................................8
2. LITERATURE REVIEW...............................................................................10
3. SCOPES & LIMITATIONS............................................................................11
3.1. SCOPES:............................................................................................11
3.2. LIMITATIONS:...................................................................................11
4. METHODOLOGY........................................................................................13
5. SYSTEM DESIGN......................................................................................15
5.1. SYSTEM ARCHITECTURE................................................................15
5.2. BLOCK DIAGRAM..............................................................................16
5.3. USER CASE DIAGRAM......................................................................17
6. REQUIREMENTS........................................................................................18
6.1. SOFTWARE REQUIREMENTS.........................................................18
6.2. HARDWARE REQUIREMENTS........................................................18
6.3. DATASET FOR TRAINING.................................................................18
7. MODULES DESCRIPTION...........................................................................19
7.1. WEBCAM CAPTURING.....................................................................19
7.2. FACE & EYE DETECTION.................................................................19

4
 7.3. DROWSINESS DETECTION.............................................................20
8. ALGORITHM..............................................................................................22
9. CODE IMPLEMENTATION..........................................................................25
9.1. IMPLEMENTATION OF TRAINING DATA........................................25
9.2. IMPLEMENTATION OF DROWSINESS DETECTION.......................29
10. OUTPUT.................................................................................................33
10.1. OPEN EYE DETECTION.................................................................33
10.2. CLOSED EYE DETECTION.............................................................33
11. CONCLUSION & FUTURE WORK.............................................................34
11.1. CONCLUSION................................................................................34
11.2. FUTURE WORK..............................................................................34
12. REFERENCES.........................................................................................35

TABLE OF FIGURE
Figure 1: Transfer learning.............................................................................9
Figure 2: Working diagram...........................................................................13
Figure 3: Real world image of the proposed system....................................14
Figure 4: real time test case.........................................................................14
Figure 5: folder content................................................................................16
Figure 6: Architecture design.......................................................................17
Figure 7: case diagram.................................................................................18
Figure 8: haar cascade..................................................................................21
Figure 9: simulation diagram........................................................................21
Figure 10: open eye detection......................................................................34
Figure 11: closed eye detection....................................................................34

5
 1. INTRODUCTION
With the increasing pace of life and work the human beings have started compromising
with the thing they need most, to efficiently function in a given situation i.e., adequate
rest and sleep to be active while doing a task. Driver’s drowsiness is a very cautious
thing and has already led to a lot of mishaps. Some researchers even provide a data
that nearly 1200 deaths and 76000 major injuries are caused due to the fatigue driver
was facing which eventually led to a crash. With the help of modern-day technology and
real time scanning systems using cameras we can prevent major mishaps on the road
by alerting car driver who is feeling drowsy through a drowsiness detection system. The
point of this undertaking is to build up a prototype drowsiness detection system. The
spotlight will be put on planning a framework that will precisely monitor the open or shut
condition of the driver's eyes continuously. By concentrating on the eyes, it has resulted
that the onset of driver fatigue can be detected to avoid a car accident. Detection of
drowsiness depends on the eye movements and time elapsed between blinks to help
generate a score on which a driver is assessed if he is drowsy or not. To prevent these
accidents, we have developed a system using python, open Cv and keras which works
on the live feed of the driver through a webcam which is then processed to generate a
score based on the time between eyes closing and opening and when eyes are closed
for a long time the score starts increasing and when it reaches a certain mark the alarm
starts beeping alerting the driver and waking him up.

1.1. PRIMARY GOAL

Driver drowsiness detection is a car safety technology which helps to save the life of the
driver by avoiding mishaps when the driver is feeling tired.
A) The primary goal is to initially plan a framework to distinguish driver's sluggishness
by persistently checking retina of the eye.
B) The framework works disregarding driver wearing displays and in different lighting
conditions.

6
C) To caution the driver on the identification of laziness by utilizing ringer or alert.
D) Speed of the vehicle can be reduced.
E) Traffic management can be maintained by reducing the accidents.

1.2. COMPUTER’S VISION

PC's vision is the change of information from a still, or camcorder into either a depiction
or another choice. Each and every such changes are performed to achieve a particular
target. A Computer gains a cross section of numbers from a camera or from the circle,
and it's just as simple as that. For the most part, there is no worked in example
acknowledgment or programmed control of center and gap, no cross-relationship with
long periods of experience. Generally, vision frameworks are still reasonably gullible.

1.3. OPEN CV

OpenCV is an open-source computer source library available in python encoding to

encrypt the visual skills of our smart pc. OpenCV was expected for computational
capability and having a high focus on ongoing picture location and distinguishing proof.
OpenCV is coded with streamlined C and can take work with multicore processors. If we
need progressively programmed improvement utilizing Intel models [Intel]. These
comprise of low-level schedules in different algorithmic regions which are streamlined.
OpenCV therefore uses the IPP library, at runtime if that library is introduced.

1.4. WHY OPEN CV?

A) Specific OpenCV was planned for picture handling. Each structure and structure of
the information is visualized in the Image Processing Plan. Then, Matlab, is very
conventional. You can get almost everything on the planet by methods for tool
compartments. It may be money related tool stash or then again concentrated DNA tool
compartments.

7
B) Speedy Matlab is just excessively moderate. Matlab itself depended on Java.
Similarly, Java depended on C. So, when we run the Matlab program, our PC gets
caught trying to translate and integrate all of this integrated Matlab code. It is then
converted to Java, eventually using code.
C) Efficient Matlab uses an excessive amount of system assets. With OpenCV, we can
extract as much as 10mb RAM for continuous application. Aside from the fact that with
current PCs, the RAM feature is by no means a significant concern. In any case, our
fatigue screen will be used inside the car in a non-slip and non-slip way; therefore, the
need for low management is important.

1.5. MACHINE LEARNING

The purpose of AI is to convert information received into usable data. After benefiting
from the issue of social information, we need a machine that can answer any questions
related to that information.

1.6. OPEN CV’S MACHINE LEARNING

ALGORITHM
ML statistics embedded in OpenCV are provided as follows. All statistics are available in
the ML library separated by Mahalanobis and K-implication, found in CVCORE, as well
as facial recognition calculations, found in CV.

1.7. TRANSFER LEARNING

In this project, we will use transfer learning to build the model. Transfer Learning is a
machine learning method where we use a pre-trained model for a new model with the
related problem statement.
For example, a model that is used for the recognition of cars can be used for the
recognition of trucks.

8
 Figure 1: Transfer learning

Here mainly it focuses on the knowledge that is gained while solving one problem and it
applies to a different but related problem.

9
 2. LITERATURE REVIEW

In the past, several significant and brilliant works have been done for drowsiness
monitoring and detection. Already several other programs have been developed for the
detection of driver’s sleep status in real time. They were done with the help of physical
cues of the driver. Yawning, jaw drop, eye closure is some of the features of a person’s
fatigue. To detect and analyse such features, usually a camera is mounted that can
detect the driver’s face clearly. Hence, a conclusion is drawn from all these works that
computer vision techniques are widely accepted to detect facial features of driver to
analyse fatigue.

Singh et al. developed a program which detects the duration of an eye blink with the
help of a developed technique called mean sift algorithm. Saito et al. use a technique
which utilised the driver’s vision path to analysed his/her state of mind i.e., whether the
driver is drowsy or not. Horng et al. uses a technology based on edge information which
localises the position of eyes in a frame containing full frontal face, then the program
uses dynamical template matching for detecting fatigue. Smith et al. constructed a
program which predicts the relevant features by tracking the head and facial features
with the help of optical flow and colour predicates. Vural et al. programmed an
Automatic classifier for several facial features taking them from Facing Action Coding
system database of sporadic expressions to separate fatigue and driver detection from
the stack of features. Vural et al. develops a system which captures the facial
movements during actual drowsiness and compares those features to the detected
features of the subject in real time. It not only uses facial features but also the motions
of the head and after collecting this data, it applies automatic measurement to compare
the data with the motions of the test subject and classifies whether he/she is drowsy or
not. Ji et al. proposes that detection of several physical cues simultaneously and
arranging them in an organised pattern and then comparing those patterns with the pre-
existing known features of drowsiness and fatigue might lead to more profound,

10
prominent and accurate results than using only a single physical cue system which is
used in already known Bayesian network.

3. SCOPES & LIMITATIONS

3.1. SCOPES:
1. Real-time Drowsiness Detection: The project's primary scope is to develop a
real-time driver's drowsiness detection system using computer vision techniques
with OpenCV and deep learning models with Keras. This system should monitor
the driver's state continuously and provide timely alerts when signs of drowsiness
are detected.
2. Data Collection: The project scope includes collecting data from various
sensors, such as cameras or infrared sensors, to capture facial features and eye
movements. This data serves as the input for the detection system.
3. Feature Extraction: Using OpenCV, the project can extract relevant facial
features and eye-related data, such as blink frequency and eyelid closure, to
determine drowsiness levels.
4. Deep Learning Model: The project will use Keras to build a deep learning model
for drowsiness detection. The model should be trained on a labeled dataset of
drowsy and alert states.
5. Real-time Alerting: When drowsiness is detected, the system should provide
real-time alerts to the driver, such as audible warnings or notifications on a
dashboard display.
6. Integration: The project may involve integrating the drowsiness detection
system into an existing vehicle's safety system or as a standalone device.

3.2. LIMITATIONS:

11
1. Hardware Requirements: The success of this project is dependent on the
availability of appropriate hardware, such as cameras, sensors, and computing
resources. Some vehicles may not have the required hardware for this system.
2. Lighting Conditions: The effectiveness of the drowsiness detection system can
be affected by varying lighting conditions. Poor lighting or extreme glare may
result in less accurate detection.
3. Individual Variability: People exhibit varying signs of drowsiness, and the
system's performance may vary among different individuals. Some drivers may
exhibit subtle signs of drowsiness that are challenging to detect.
4. False Positives/Negatives: Any drowsiness detection system can produce false
positives (incorrectly identifying a driver as drowsy) or false negatives (failing to
identify drowsiness when it is present). Achieving a balance between sensitivity
and specificity is challenging.
5. Privacy Concerns: Continuous monitoring of drivers can raise privacy concerns.
It is essential to address these concerns and ensure that data is used
responsibly and within legal boundaries.
6. Training Data: The performance of the deep learning model largely depends on
the quality and diversity of the training data. The model may not generalize well if
the dataset is limited in size or diversity.
7. Environmental Factors: External factors such as road conditions, weather, and
other drivers' behavior can influence a driver's alertness. The system does not
consider these factors.
8. Autonomous Vehicles: While this project is relevant for traditional vehicles,
autonomous vehicles may make drowsiness detection less critical since they can
take over driving when the driver is unable to do so.
9. Legal and Ethical Considerations: The project should consider the legal and
ethical aspects of drowsiness detection. For example, the responsibility of the
vehicle and driver in the event of a drowsiness alert needs to be defined.
10. Maintenance and Updates: The system should be regularly maintained and
updated to ensure its effectiveness. Software updates and model retraining may
be necessary to adapt to changing conditions and technologies.

12
 4. METHODOLOGY
The main purpose of this paper is to produce a simple and easy-to-use system that will
lead to a safe road trip.

Figure 2: working diagram

Fig 1 shows the method used to develop the system to an efficient way. The algorithm
used to process the image to feed the code to specify the face in the image, the image
is divided into sub-regions to determine whether the region is on the face or not. The
use of this algorithm means a time-saving method and only face-containing domains are
processed.

13
 Figure 3: Real world image of the proposed system

Figure 4: real time test case

Fig 4 Show the real time test case generating a score based on the time the eyes were
closed and sounding an alarm when reaching certain threshold.

14
 5. SYSTEM DESIGN
5.1. SYSTEM ARCHITECTURE
The model we used is built with Keras using Convolutional Neural Networks (CNN).
A convolutional neural network is a special type of deep neural network which performs
extremely well for image classification purposes. A CNN basically consists of an input
layer, an output layer and a hidden layer which can have multiple numbers of layers. A
convolution operation is performed on these layers using a filter that performs 2D matrix
multiplication on the layer and filter.
The CNN model architecture consists of the following layers:
● Convolutional layer; 32 nodes, kernel size 3
● Convolutional layer; 32 nodes, kernel size 3
● Convolutional layer; 64 nodes, kernel size 3
● Fully connected layer; 128 nodes

The final layer is also a fully connected layer with 2 nodes. In all the layers, a Relu
activation function is used except the output layer in which we used Softmax.

Python Project on Steps for Performing Driver Drowsiness Detection Download the
Python project source code from the zip and extract the files in your system:
Drowsiness detection\Drowsiness detection

The content of the folder are:

15
 Figure 5: folder content

1. The “haar cascade files” folder consists of the xml files that are needed to detect
objects from the image. In our case, we are detecting the face and eyes of the
person.
2. The models folder contains our model file “cnnCat2.h5” which was trained on
convolutional neural networks.
3. We have an audio clip “[Link]” which is played when the person is feeling
drowsy.
4. “[Link]” file contains the program through which we built our classification
model by training on our dataset. You could see the implementation of
convolutional neural network in this file.
5. “Drowsiness [Link]” is the main file of our project. To start the detection
procedure, we have to run this file.

5.2. BLOCK DIAGRAM

16
 Figure 6: Architecture design

The system will be continuously monitoring the movement of the driver’s eye by a live
camera and all the monitored signals are pre - processed. A Black Box with the
software installed is used to detect the driver drowsiness and alerts the driver with
buzzer, if driver is affected by drowsiness.

5.3. USER CASE DIAGRAM

17
 Figure 7: case diagram

Drowsiness detection is a safety technology that can prevent accidents that are caused
by drivers who fell asleep while driving. The project is to build a drowsiness detection
system that will detect that a person’s eyes are closed for a few seconds. This system
will alert the driver when drowsiness is detected. In this Python project, we will be using
OpenCV for gathering the images from webcam and feed them into Deep Learning a
model which will classify whether the person’s eyes are ‘Open’ or ‘Closed’.

6. REQUIREMENTS
6.1. SOFTWARE REQUIREMENTS

[Link]: Python is the basis of the program that we wrote. It utilizes many of the
python libraries.
Python 3 is used here.

18
[Link]:
 Numpy
 Tensorflow
 Open cv
 Keras
 Pygame.
[Link] Systems:
 Windows

6.2. HARDWARE REQUIREMENTS

 Laptop with basic hardware

 Webcam

6.3. DATASET FOR TRAINING

The dataset for training & testing is given in the folder given below:
Drowsiness detection\dataset_new

In the above folder, there are 2 sub-folders Train & Test . The folder Train & Test folder
again opens a sub-folder into 2 such as Open & Closed. The Train folder consist of
1,234 images which are sub divided into Open & Closed folders. The Test folder consist
of 218 images which is also divided as Open & Closed.

7. MODULES DESCRIPTION
 Webcam Capturing
 Face and Eye Detection
 Drowsiness Detection (with Alert Message and Alarm Sound)

19
 7.1. WEBCAM CAPTURING

With a webcam, we will take images as input. So to access the webcam, we made an
infinite loop that will capture each frame. We use the method provided by OpenCV,
[Link](0) to access the camera and set the capture object (cap). [Link]()
will read each frame and we store the image in a frame variable.

7.2. FACE & EYE DETECTION

 The “haar cascade files” folder consists of the xml files that are needed to detect
objects from the image (in machine learning). In our case, we are detecting the
face and eyes of the person.
 To detect the face in the image, we need to first convert the image into grayscale
as the Open CV algorithm for object detection takes gray images in the input. We don’t
need color information to detect the objects. We will be using haar cascade classifier to
detect faces. The line is used to set our classifier.
 The cascade themselves are just a bunch of XML files that contain Open CV
Data used to detect objects. You initialize your code with the cascade you want, and
then it does the work for you.
 It loads face cascade into memory and read the image and convert into
Grayscale. (Many operations are done in grayscale as we need not define Green , Red,
and Blue colours here)

Haar cascade approach:

20
 Figure 8: haar cascade

7.3. DROWSINESS DETECTION

Figure 9: simulation diagram

 The value we will use to determine how long the person has closed his eyes
is threshold value. So if both eyes are closed, we will keep on increasing score and
when eyes are open, we decrease the score. We are drawing the result on the screen
using [Link]() function which will display real time status of the person.

21
 A threshold is defined for example if score becomes greater than 15 that
means the person’s eyes are closed for a long period of time. This is when we beep
the alarm using [Link]().

8. ALGORITHM
22
The approach we will be using for this Python project is as follows:

 Step 1 – Take an image as input from a web-camera.

To access the webcam, we made an infinite loop that will capture each frame. We use
the method provided by OpenCV, [Link](0) to access the camera and set
the capture object (cap). [Link]() will read each frame and we store the image in a
frame variable.

 Step 2 – Detect the face in the image and create a Region of Interest (ROI).
To detect the face in the image, we need to first convert the image into grayscale as the
OpenCV algorithm for object detection takes gray images in the input. We don’t need
color information to detect the objects. We will be using a haar cascade classifier to
detect faces.
This line is used to set our classifier:
face = [Link](‘path to our haar cascade XML file’).
Then we perform the detection using faces = [Link](gray).
It returns an array of detections with x,y coordinates, and height, the width of the
boundary box of the object. Now we can iterate over the faces and draw boundary
boxes for each face.
Code Snippet:
1. for (x,y,w,h) in faces:
2. [Link](frame, (x,y), (x+w, y+h), (100,100,100), 1 )

 Step 3 – Detect the eyes from ROI and feed it to the classifier.
The same procedure to detect faces is used to detect eyes. First, we set the cascade
classifier for eyes in leye and reye respectively then detect the eyes using
left_eye = [Link](gray).
Now we need to extract only the eyes data from the full image. This can be achieved by
extracting the boundary box of the eye and then we can pull out the eye image from the
frame with this code.

23
Code Snippet:
l_eye = frame[ y : y+h, x : x+w ]
l_eye only contains the image data of the eye. This will be fed into our CNN classifier
which will predict if eyes are open or closed. Similarly, we will be extracting the right eye
into r_eye.

 Step 4 – The classifier will categorize whether eyes are open or closed.
We are using CNN classifiers for predicting eye status. To feed our image into the
model, we need to perform certain operations because the model needs the correct
dimensions to start with. First, we convert the color image into grayscale using
r_eye = [Link](r_eye, cv2.COLOR_BGR2GRAY).
Then, we resize the image to 24*24 pixels as our model was trained on 24* 24-pixel
images [Link](r_eye, (24,24)). We normalize our data for better convergence r_eye
= r_eye/255(All values will be between 0-1). Expand the dimensions to feed into our
classifier. We loaded our model using model = load_model(‘models/cnnCat2.h5’) .
Now we predict each eye with our model lpred = model.predict_classes(l_eye).
If the value of lpred[0] = 1, it states that eyes are open, if value of lpred[0] = 0 then, it
states that eyes are closed.

 Step 5 – Calculate score to check whether the person is drowsy.

The score is basically a value we will use to determine how long the person has closed
his eyes. So if both eyes are closed, we will keep on increasing the score and when
eyes are open, we decrease the score. We are drawing the result on the screen using
the [Link]() function which will display the real-time status of the person.
Code Snippet :
[Link](frame, “Open”, (10, height-20), font, 1, (255,255,255), 1, cv2.LINE_AA)
A threshold is defined for example if score becomes greater than 15 that means the
person’s eyes are closed for a long period of time. This is when we beep the alarm
using [Link]().

24
 9. CODE IMPLEMENTATION
9.1. IMPLEMENTATION OF TRAINING DATA
The first step of Driver’s drowsiness is to train a model to find whether the eyes are
open or closed by training and testing certain set of data’s. The folder link to the dataset
is given in the section 19. The code for training data is included in the driver drowsiness

25
folder titled as “[Link]”.The detailed code version is given below:

STEP1:

Import necessary libraries:

 os: for working with the file system.
 [Link]: for image data preprocessing.
 [Link]: for data visualization (not used in this code).
 numpy: for numerical operations.
 [Link].to_categorical: for one-hot encoding of labels.
 random and shutil: for randomization and file operations.
 [Link], [Link]: for defining and building the CNN model.
 [Link].load_model: for loading a saved model.

import os
from [Link] import image
import [Link] as plt
import numpy as np
from [Link] import to_categorical
import random,shutil
from [Link] import Sequential
from [Link] import Dropout,Conv2D,Flatten,Dense, MaxPooling2D,
BatchNormalization
from [Link] import load_model

STEP 2:
Define a function generator:
 This function generates a data generator using Keras' ImageDataGenerator for
image data augmentation and preprocessing.
 It takes several parameters like the directory of the dataset, batch size, and
target image size.
 It returns a data generator object that can be used to load batches of images for
training or validation.

def generator(dir, gen=[Link](rescale=1./255),

shuffle=True,batch_size=1,target_size=(24,24),class_mode='categorical' ):

26
 return
gen.flow_from_directory(dir,batch_size=batch_size,shuffle=shuffle,color_m
ode=
'grayscale',class_mode=class_mode,target_size=target_size)

STEP 3:
Set batch size and target image size for training and validation data.
BS= 32
TS=(24,24)

STEP 4:
Create data generators for the training and validation datasets:
 train_batch and valid_batch are created using the generator function,
specifying the directories of the training and testing datasets.

train_batch= generator(r"C:\Users\Lenovo\Downloads\Drowsiness detection\

dataset_new\train",shuffle=True, batch_size=BS,target_size=TS)
valid_batch= generator(r"C:\Users\Lenovo\Downloads\Drowsiness detection\
dataset_new\test",shuffle=True, batch_size=BS,target_size=TS)

STEP 5:
1. Calculate the number of steps per epoch (SPE) and validation steps (VS) based
on the batch sizes and the number of classes in the datasets.
2. Define the CNN model:
 The model is built using the Keras Sequential API, which allows you to
stack layers one after the other.
 It consists of multiple convolutional and pooling layers, dropout layers to
prevent overfitting, and fully connected layers.
 The last layer has two units with softmax activation, which is suitable for
multi-class classification (drowsy vs. not drowsy).
3. SPE= len(train_batch.classes)//BS
4. VS = len(valid_batch.classes)//BS
5. print(SPE,VS)

27
model = Sequential([
Conv2D(32, kernel_size=(3, 3), activation='relu',
input_shape=(24,24,1)),
MaxPooling2D(pool_size=(1,1)),
Conv2D(32,(3,3),activation='relu'),
MaxPooling2D(pool_size=(1,1)),
Conv2D(64, (3, 3), activation='relu'),
MaxPooling2D(pool_size=(1,1)),
Dropout(0.25),
Flatten(),
Dense(128, activation='relu'),
Dropout(0.5),
Dense(2, activation='softmax')
])

STEP 6:
Compile the model:
 The model is compiled with the Adam optimizer and categorical cross-entropy
loss function.
 The accuracy is used as the evaluation metric.

[Link](optimizer='adam',loss='categorical_crossentropy',metrics=['
accuracy'])

STEP 7:
Train the model using the model.fit_generator method:
 The training data (train_batch) and validation data (valid_batch) are used.
 The model is trained for 15 epochs, and the number of steps per epoch and
validation steps are specified.
 The training process is supervised using the categorical cross-entropy loss.
 model.fit_generator(train_batch,
validation_data=valid_batch,epochs=15,steps_per_epoch=SPE ,validatio
n_steps=VS)

STEP 8:

28
Save the trained model to a file: The model is saved to the specified file location using
‘[Link]’.

[Link](r"C:\Users\Lenovo\Downloads\Drowsiness detection\models\
cnnCat2.h5", overwrite=True)

This code trains a CNN for drowsiness detection using a dataset of images. After
training, the model can be used for making predictions on new images to detect whether
a person is drowsy or not.

9.2. IMPLEMENTATION OF DROWSINESS

DETECTION
The code for drowsiness detection is also included in the main folder mentioned in
15SYSTEM ARCHITECTURE

29
STEP 1 :

Firstly do make sure that you have the following python libraries installed on your
system.

import cv2
import os
from [Link] import load_model
import numpy as np
from pygame import mixer
import time

STEP 2:

Now we need to access the ‘haar cascade files’ through this program.

face = [Link](r"C:\Users\Lenovo\Downloads\Drowsiness
detection\haar cascade files\haarcascade_frontalface_alt.xml")
leye = [Link](r"C:\Users\Lenovo\Downloads\Drowsiness
detection\haar cascade files\haarcascade_lefteye_2splits.xml")
reye = [Link](r"C:\Users\Lenovo\Downloads\Drowsiness
detection\haar cascade files\haarcascade_righteye_2splits.xml")

STEP 3:

Next, we access the alarm file needed in this model.

[Link]()
sound = [Link](r"C:\Users\Lenovo\Desktop\project\Drowsiness
detection\[Link]")

STEP 4:

Now it is time for us to import the classification model that we have downloaded.

lbl=['Close','Open']

model = load_model(r"C:\Users\Lenovo\Downloads\Drowsiness detection\

models\cnnCat2.h5")
path = [Link]()
cap = [Link](0)
font = cv2.FONT_HERSHEY_COMPLEX_SMALL
count=0

30
score=0
thicc=2
rpred=[99]
lpred=[99]

STEP 5: Capture a pic from the camera

Our first step is to capture an image from the camera module so we need to access the
camera (webcam in this case) to do so.
Firstly we use [Link](0) function to access the camera module. The object
in the frame is then read by the [Link]() function and the image is then stored in a
variable.
The next step would be to create a rectangle around the detected facial region.

# Start of while

while(True):
ret, frame = [Link]()
height,width = [Link][:2]

gray = [Link](frame, cv2.COLOR_BGR2GRAY)

faces =
[Link](gray,minNeighbors=5,scaleFactor=1.1,minSize=(25,25)
)
left_eye = [Link](gray)
right_eye = [Link](gray)

[Link](frame, (0,height-50) , (200,height) , (0,0,0) ,

thickness=[Link] )

for (x,y,w,h) in faces:

[Link](frame, (x,y) , (x+w,y+h) , (100,100,100) , 1 )

STEP 6: Detecting eyes and classifying them as open or closed.

The next step is to detect the eyes in the captured image and hence narrow the
concentration region. We need to separately detect both the right eye and the left eye
for this.

31
# while continued

for (x,y,w,h) in right_eye:

r_eye=frame[y:y+h,x:x+w]
count=count+1
r_eye = [Link](r_eye,cv2.COLOR_BGR2GRAY)
r_eye = [Link](r_eye,(24,24))
r_eye= r_eye/255
r_eye= r_eye.reshape(24,24,-1)
r_eye = np.expand_dims(r_eye,axis=0)
rpred = [Link](r_eye)
rpred = [Link](rpred, axis=1)
if(rpred[0]==1):
lbl='Open'
if(rpred[0]==0):
lbl='Closed'
break

for (x,y,w,h) in left_eye:

l_eye=frame[y:y+h,x:x+w]
count=count+1
l_eye = [Link](l_eye,cv2.COLOR_BGR2GRAY)
l_eye = [Link](l_eye,(24,24))
l_eye= l_eye/255
l_eye=l_eye.reshape(24,24,-1)
l_eye = np.expand_dims(l_eye,axis=0)
lpred = [Link](l_eye)
lpred = [Link](lpred, axis=1)
if(lpred[0]==1):
lbl='Open'
if(lpred[0]==0):
lbl='Closed'
break

STEP 7: Drowsy or not

The next step is to classify whether the driver is drowsy or not. We might an issue here.
Closed eye need not mean that the driver is drowsy, he/she might just be blinking. So to
tackle this we would make a score and increase it every time the eyes are found closed
in continuation. And only if then score goes past 15 the alarm needs to alert the driver.

# while continued
if(score<0):

32
 score=0
[Link](frame,'Score:'+str(score),(100,height-20), font, 1,
(255,255,255),1,cv2.LINE_AA)
if(score>15):

[Link]([Link](path,'[Link]'),frame)
try:
[Link]()

except:
pass
if(thicc<16):
thicc= thicc+2
else:
thicc=thicc-2
if(thicc<2):
thicc=2
[Link](frame,(0,0),(width,height),(0,0,255),thicc)
[Link]('frame',frame)
if [Link](1) & 0xFF == ord('q'):
break

# End of while

STEP 8:

Now we are almost done. After classifying the driver as drowsy or not, the final step is to
close the windows.
[Link]()
[Link]()

After implementing both files , we are able to alert the drivers if they are drowsy.

10. OUTPUT

33
In this Python project, we have built a drowsy driver alert system that you can
implement in numerous ways. We used OpenCV to detect faces and eyes using a haar
cascade classifier and then we used a CNN model to predict the status.

10.1. OPEN EYE DETECTION

Figure 10: open eye detection

10.2. CLOSED EYE DETECTION

Figure 11: closed eye detection

34
 11. CONCLUSION & FUTURE
WORK
11.1. CONCLUSION
It completely meets the objectives and requirements of the system. The framework has
achieved an unfaltering state where all the bugs have been disposed of. The framework
cognizant clients who are familiar with the framework and comprehend it's focal points
and the fact that it takes care of the issue of stressing out for individuals having fatigue-
related issues to inform them about the drowsiness level while driving. The ultimate goal
of the system is to check the drowsiness condition of the driver. Based on the eye
movements of the driver, the drowsiness is detected and according to eye blink, the
alarm will be generated to alert the driver and to reduce the speed of the vehicle along
with the indication of parking light. By doing this, many accidents will be reduced and
provides safety to the driver and vehicle. A system that is driver safety and car security
is presented only in luxurious costly cars. Using eye detection, driver security and safety
can be implemented in normal car also.

11.2. FUTURE WORK

The model can be improved incrementally by using other parameters like blink rate,
yawning, state of the car, etc. If all these parameters are used it can improve the
accuracy by a lot. We plan to further work on the project by adding a sensor to track the
heart rate in order to prevent accidents caused due to sudden heart attacks to drivers.
Same model and techniques can be used for various other uses like Netflix and other
streaming services can detect when the user is asleep and stop the video accordingly. It
can also be used in application that prevents user from sleeping.

12. REFERENCES
35
 Vandna Saini, Rekha Saini “Driver Drowsiness Detection System and
Techniques”, IJCSIT, Vol. 5 (3), 2014.
 [Link], [Link] “Implementation of the Driver Drowsiness
Detection System”, IJSETR, Volume 2, Issue 9, September 2013.
 Arun Sahayadhas, Kenneth Sundaraj &amp; Murugappan “Detecting Driver
Drowsiness Based on Sensors”, Sensors, 7 December 2012.
 Mitharwal Surendra Singh L., Ajgar Bhavana G., Shinde Pooja S., Maske Ashish
M. “Eye Tracking Based Driver Drowsiness Monitoring &amp; Warning System”,
IJTRA, Volume 3, Issue 3, May-June 2015.
 Chisty, Jasmeen Gill, “Driver Drowsiness Detection System”, IJCST, Volume 3,
Issue 4,Jul-Aug 2015.
 Ralph Oyini Mbouna, Seong G. Kong, Senior Member, IEEE,(2013),”Visual
Analysis of Eye State and Head Pose for Driver Alertness Monitoring
 Weirwille, W.W. (1994). “Overview of Research on Driver Drowsiness Definition
and Driver Drowsiness Detection,” 14th International Technical Conference on
Enhanced Safety of Vehicles, pp23-26.

36