STRESS DETECTION USING MACHINE LEARNING

A PROJECT REPORT
(FINAL PROJECT)

Submitted by

JITHISH M 20EUEC068
KAMALESH A 20EUEC070
KARAN P 20EUEC071

In partial fulfillment for the award of the

degree Of
BACHELOR OF ENGINEERING

IN

ELECTRONICS AND COMMUNICATION ENGINEERING

SRI KRISHNA COLLEGE OF ENGINEERING AND TECHNOLOGY

(An Autonomous Institution Affiliated to Anna University Chennai- 600 025)

ACCREDITED BY NAAC WITH “ A ” GRADE

MARCH-2024
 BONAFIDE CERTIFICATE

Certified that this project report “STRESS DETECTION USING MACHINE

LEARNING” is the Bonafide work of “JITHISH M (20EUEC068) , KAMALESH A
(20EUEC070) , KARAN P (20EUEC071)” who carried out the project work under my

supervision.

SIGNATURE SIGNATURE

Dr. S. SASIPRIYA, M.E.,PH.D., HEAD OF Ms.D.V SOUNDARI,M.E.,

THE DEPARTMENT,
SUPERVISOR
ASSISTANT PROFESSOR

ELECTRONICS AND COMMUNICATION ELECTRONICS AND

ENGINEERING COMMUNICATION ENGINEERING

SRI KRISHNA COLLEGE OF SRI KRISHNA COLLEGE OF

ENGINEERING AND TECHNOLOGY ENGINEERING AND TECHNOLOGY

Submitted for the project viva-voce examination held on

INTERNAL EXAMINER EXTERNAL EXAMINER

 ABSTRACT

This project addresses the escalating prevalence of stress-related concerns by proposing a

multi-modal human stress detection system combining facial recognition and emotion
analysis. The method involves capturing facial images through a web camera using
OpenCV's Haar cascade algorithm for robust face detection. Isolated faces undergo
processing through a pre-trained Convolutional Neural Network (CNN) model, tailored for
stress estimation. Simultaneously, emotional features are extracted using a k-Nearest
Neighbors (KNN) algorithm. This multi-modal approach enhances stress level assessments
by capturing both physiological and emotional cues. The system offers real-time, accurate
stress monitoring, making it suitable for diverse applications, from workplaces to personal
well-being initiatives. The integration of OpenCV's face detection, CNN, and KNN provides
a versatile and potent stress detection solution, contributing to mental health monitoring and
intervention efforts. The project's innovative combination of computer vision, deep learning,
and machine learning techniques demonstrates a comprehensive and adaptable approach to
addressing the pressing challenges of stress detection in contemporary society.

3
 TABLE OF CONTENTS

CHAPTER NO TITLE PAGE NO

ABSTRACT 3

LIST OF CONTENTS 4

LIST OF TABLE 7

LIST OF FIGURE 8

LIST OF ABBREVIATION 9

10
1 INTRODUCTION

10
1.1 Background
10
1.2 Motivation
11
1.3 Proposed Solution
11
1.4 Necessity
12
1.5 Advantages

2 LITERATURE SURVEY 15

3 SYSTEM DESIGN 25

3.1 Existing System 25

3.2 Proposed System 25

26
3.3 Block Diagram
27
3.4 Methodology
`4 28
HARDWARE DESCRIPTION

4
 4.1 NODEMCU (ESP8266) 28
4.1.1 NODEMCU 28
4.2 Pin Diagram of ESP8266 29
4.2.1 Pin diagram of 30

ESP8266
30
4.3 Features of ESP8266
4.4 Port B (PB [7:0]) XTAL1/
31
XTAL2/TOSC1/TOSC2
4.5 Port C (PC [5:0])
32
4.6 PC6/RESET 32
4.7 Port D (PD [7:0]) 32
4.8 AVCC 33
4.9 NODEMCU UNO Board 33
Description
4.9.1 NODEMCU UNO
Board 33

4.10 Voltage Regulator

34
4.11 Crystal Oscillator
34
4.12 NODEMCU Reset
34
4.13 MAX30100
35
4.13.1 DHT11
Thermistor 36
4.13.2 Humidity
Sensor 37

4.13.3 NTC Thermistor

37
4.13.4 8 bit SOIC - 14
38
IC
4.14 DHT11 Module hardware
38
Overview
4.14.1 DHT11 Sensor
38
4.15 Supporting circuitry 39
4.15.1DHT11 Module 39

5
 4.15.2 DHT11 Module 40

Pinout
4.15.3 Arduino UNO 41

SOFTWARE DESCRIPTION
5 42

5.1 Arduino development

environment
42
5.2 Programming 44
5.3 Python 3.7 46
5.4 Theory IDE 53

WORKING 56
6
6.1 Facial Expression analysis
56
with CNN
6.2 Physiological Monitoring with
MAX30100 and KNN
56
6.3 Integration of facial and
Physiological data 56
6.4 Validation and Evaluation 57

58
RESULTS
7

62
8 CONCLUSION

63
9 REFERENCES

6
 LIST OF TABLE

TABLE NO TITLE PAGE NO

7.2.1 Classification report of CNN 60

7.2.2 60
Classification report of KNN

7
 LIST OF FIGURES

FIG NO FIGURE PAGE NO

4.1.1 NODEMCU (ESP8266) 29

4.2.1 Pin Diagram of ESP8266 31

4.9.1 NODEMCU UNO Board 35

4.13.1 DHT11 Thermistor 38

4.13.2 Humidity Sensor 38

4.13.3 NTC Thermistor 39

4.13.4 8 bit SOIC - 14 IC 39

4.14.1 DHT11 Sensor 40

4.15.1 DHT11 Module 41

4.15.2 DHT11 Module Pinout 42

4.15.3 Arduino UNO 43

7.1.1 Confusion matrix of CNN 58

7.1.2 Bar Graph of KNN 58

7.1.3 Accuracy vs Epoch 59

7.1.4 Loss vs Epoch 59

7.2.1 Model Output 61

8
 LIST OF ABBREVIATION

CNN - Convolutional Neural Network

KNN - K-Nearest Neighbor

IDE - Integrated Development Environment

GTK - GIMP Toolkit

GNOME - GNU Network Object Model

Environment

KDE - K Desktop Environment

9
 CHAPTER 1
INTRODUCTION

In modern society, the escalating prevalence of emotion-related concerns has

underscored the critical need for accurate and non-intrusive emotion detection
methods. Conventional approaches often fall short in terms of precision and
real-time capabilities. This project seeks to address these limitations by
proposing a human emotion detection system utilizing facial recognition. The
objective is to harness the capabilities of computer vision and deep learning to
precisely assess emotion levels through facial features, offering a versatile
solution for mental health monitoring and well-being initiatives.

1.1 BACKGROUND:
The increasing awareness of mental health and well-being has led to a
growing demand for innovative technologies capable of monitoring and
addressing emotion-related issues. Traditional emotion assessment methods,
such as self-reporting or physiological measurements, face challenges in
providing real-time and objective insights into an individual's emotion levels.
Leveraging advancements in computer vision and deep learning, this project
aims to pioneer a more accurate and efficient approach to emotion detection,
focusing on facial cues as indicators of emotion.

1.2 MOTIVATION:
The motivation behind this work stems from the limitations of existing
emotion detection methods and the potential impact on individual well-being.
Accurate and timely identification of emotion is crucial for early intervention
and personalized support. By combining the capabilities of OpenCV's face
detection with a pre-trained Convolutional Neural Network (CNN) model, we
aim to create a robust system that can adapt to diverse datasets and provide

10
nuanced emotion assessments. The motivation is rooted in the potential of this
technology to contribute significantly to mental health monitoring, offering
individuals and healthcare professionals valuable insights for timely
interventions.

1.3 PROPOSED SOLUTION:

The proposed method unfolds in two crucial steps. First, facial images are
captured through a web camera, and the Haar cascade algorithm from
OpenCV is employed for robust face detection. This initial step ensures the
efficient isolation of facial regions for subsequent analysis. Subsequently, the
isolated faces undergo processing through a meticulously trained pre-trained
Convolutional Neural Network (CNN) model tailored explicitly for emotion
estimation. This model, trained on a diverse dataset, is designed to identify
nuanced facial cues indicative of emotion, thereby enhancing the accuracy of
emotion level assessments. The amalgamation of OpenCV's face detection and
the deep learning model creates a potent and adaptable emotion detection
system, poised to revolutionize the field of mental health monitoring.

1.4 NECESSITY:
In contemporary society, the escalating prevalence of emotion-related
concerns has brought forth a pressing need for advanced and non-intrusive
emotion detection methods. Conventional approaches, often reliant on self-
reporting or physiological measurements, face limitations in terms of
precision and real-time capabilities. The challenges associated with these
traditional methods underscore the necessity for a more accurate and efficient
emotion detection system. Such a system would not only address the growing
concerns surrounding mental well-being but also contribute to early
intervention strategies, thereby mitigating the potential impact of prolonged
emotion on individuals.

11
1.5 ADVANTAGES:

Real-time Monitoring:
The emotion detection system enables real-time monitoring of emotion levels,
allowing for prompt identification and timely interventions.

Non-intrusive Assessment:
Leveraging facial recognition, the system offers a non-intrusive approach to
emotion detection, enhancing user comfort and acceptability.

Objective and Quantifiable Results:

Integration of computer vision and deep learning provides an objective and
quantifiable assessment, reducing subjectivity associated with traditional
methods.

Adaptability to Diverse Datasets:

The pre-trained Convolutional Neural Network (CNN) model ensures
adaptability to diverse datasets, improving accuracy across demographic
groups and cultural contexts.

Potential for Early Intervention:

The system identifies subtle facial cues indicative of emotion, presenting an
opportunity for early intervention and prevention of escalated emotion-related
issues.

Versatility for Mental Health Initiatives:

The system's versatility positions it as a valuable tool for broader mental
health initiatives, supporting remote monitoring and offering personalized

12
support for improved mental well-being. In modern society, the escalating
prevalence of stress-related concerns has highlighted the imperative need for
accurate and non-intrusive stress detection methods. Conventional approaches
often fall short in terms of precision and real-time capabilities.

This project addresses these limitations by proposing a multi-modal human

stress detection system that incorporates both facial recognition and emotion
analysis. The goal is to leverage the power of computer vision, deep learning,
and machine learning to precisely assess stress levels through facial features
and emotional cues, offering a versatile solution for mental health monitoring
and well-being initiatives.

The proposed method unfolds in three key steps. Initially, facial images are
captured through a web camera, utilizing the Haar cascade algorithm from
OpenCV for robust face detection. This ensures the efficient isolation of facial
regions for subsequent analysis. Following this, the isolated faces undergo
processing through a pre-trained Convolutional Neural Network (CNN) model,
specifically tailored for stress estimation. The CNN is meticulously trained on
a diverse dataset to identify nuanced facial cues indicative of stress, thereby
enhancing the accuracy of stress level assessments.

13
In parallel, an additional dimension is added to the stress detection system by
incorporating emotion analysis using a k-Nearest Neighbors (KNN) algorithm.
This step aims to extract emotional features from facial expressions,
contributing to a more comprehensive understanding of the individual's
mental state. The combination of OpenCV's face detection, the deep learning
model, and KNN for emotion analysis creates a potent and adaptable multi-
modal stress detection system.
The integration of facial recognition, deep learning, and machine learning
techniques provides a holistic approach to stress detection, capturing both
physiological and emotional aspects. The proposed system holds promise for
real-time stress monitoring in various settings, from workplaces to personal
well-being applications. The multi-modal nature of the system enhances its
versatility, making it a valuable tool for mental health initiatives and
interventions.

14
 CHAPTER 2
LITERATURE SURVEY

[Link]: Emotion Detection and Characterization using Facial Features

Authors: Charvi Jain,Kshitij Sawant,Mohammed Rehman,Rajesh Kumar,
Publication: 2018 3rd International Conference and Workshops on Recent
Advances and Innovations in Engineering (ICRAIE)
The human face has peculiar and specific characteristics, therefore it becomes
difficult in understanding and identifying the facial expressions. It is easy to
identify the facial expression of particular person in any image sequence. If
we look to automated recognition system, however, the systems available are
quite inadequate and incapable of accurately identify emotions. The area of
facial expression identification has many important applications. It is an
interactive tool between humans and computers. The user, without using the
hand can go-ahead with the facial expressions. Presently, the research on
facial expression are on the factors i.e. sad, happy, disgust, surprise, fear and
angry. This paper aims to detect faces from any given image, extract facial
features (eyes and lips) and classify them into 6 emotions (happy, fear, anger,
disgust, neutral, sadness). The training data is passed through a series of filters
and processes and is eventually characterized through a Support Vector
Machine(SVM), refined using Grid Search. The testing data then tests the data
and their labels and gives the accuracy of classification of the testing data in a
classification report. Various approaches, including passing the training
images through Gabor filter, or transforming images using Histogram of
Oriented Gradients(HOG) and Discrete Wavelet Transform(DWT) for better
classification of data are implemented. The best result achieved so far is by
passing the training images through Histogram of Oriented Gradients(HOG),
followed by characterization by SVM, which gives an average precision of
85%.",

15
[Link]: Facial Emotion Recognition using Deep Learning
Authors: Shubhanjay Pandey,Sonakshi Handoo,Yogesh,
Publication: 2022 International Mobile and Embedded Technology
Conference (MECON)
As it is known, sentiments influence information processing, attitude
formation, and decision making to a great extent in real-world scenarios.
Several recent efforts have been published about FER or facial expression
recognition, however, due to the diversity of human faces and fluctuations in
pictures, reliable and robust FER systems remain a challenge. Till date, every
study and work has proposed either a single network or an ensemble model.
The accuracy of ensemble models is higher, but they were associated with
many models and datasets and a few tweaked datasets to improve the accuracy,
increasing the computing complexity. While the majority of research in this
field focuses on improving accuracy, this study utilizes the proposed model to
a real-world scenario in which a personu2019s face contains a mix of
emotions, and a single-label sentiment can be highly noisy in such situations.
In view of this scenario, we developed and tested 15-20 models and methods.
In this paper, we propose a single standalone-based CNN model with its
implementation on a real-time Intelligent System for Sentiment Recognition,
which validates accuracy through transfer learning and performs tasks such as
face detection, sentiment classification, and providing a live list of
probabilistic labels in Realtime from a webcam feed in one blended step. The
proposed model achieves an accuracy of 76.62 % outperforming all
standalone-based models like VGG16, VGG19, EfficientNetB7, and other
proposed models on the FER2013 dataset solely, which is a challenging and
noisy crowd-sourced dataset, without using any other auxiliary dataset.",

[Link]: Facial Emotion Detection Using Deep Learning

Authors: Akriti Jaiswal,A. Krishnama Raju,Suman Deb,

16
Publication: 2020 International Conference for Emerging Technology
(INCET)
Human Emotion detection from image is one of the most powerful and
challenging research task in social communication. Deep learning (DL) based
emotion detection gives performance better than traditional methods with
image processing. This paper presents the design of an artificial intelligence
(AI) system capable of emotion detection through facial expressions. It
discusses about the procedure of emotion detection, which includes basically
three main steps: face detection, features extraction, and emotion
classification. This paper proposed a convolutional neural networks (CNN)
based deep learning architecture for emotion detection from images. The
performance of the proposed method is evaluated using two datasets Facial
emotion recognition challenge (FERC-2013) and Japaness female facial
emotion (JAFFE). The accuracies achieved with proposed model are 70.14
and 98.65 percentage for FERC-2013 and JAFFE datasets respectively.",

[Link]: Machine Learning Techniques for Real-Time Emotion Detection

from Facial Expressions
Authors: Akshita Sharma,Vriddhi Bajaj,Jatin Arora,
Publication: 2023 2nd Edition of IEEE Delhi Section Flagship Conference
(DELCON)
Facial expressions recognition by emotion is a crucial component in many
applications. This paper covers the recent trends in human emotion detection.
An overview of various facial emotion recognition and its applications are
presented. In the literature review, major machine-learning techniques used
for facial emotion identification have been explored. Machine learning
approaches are compared on the basis of their advantages, disadvantages, and
their accuracy. Theoretical analysis of existing approaches shows that the
algorithm providing the maximum accuracy should be used for facial emotion

17
recognition. The existing approaches are also suffered from some challenges
and those challenges should be addressed and considered for accurately
predicting the users' emotional state. The application of emotion detection is
also very vast and a few of the major applications are also discussed. Finally,
a brief analysis of existing Machine learning approaches and their conclusion
is given.",

[Link]: Facial Emotion Detection Using Convolutional Neural Networks

Authors: Vaishnavi Hosur,Ashwini Desai,
Publication: 2022 IEEE 2nd Mysore Sub Section International Conference
(MysuruCon)
The most important aspect to understand human behavior is face reading. The
expression speaks better than words. The expressions of face reflect human
perspective and its mental state. The aim of this paper is to sight faces from
image, extract facial expressions and classify them into different emotions like
sad, happy, anger, and neutral. This paper discusses a technique named facial
emotion recognition using convolutional neural networks (FERC). There are
two parts in convolution neural networks (CNN) first removal of background
from image and second facial feature extraction (EV). This application is used
in medical treatment, teaching field, police investigation, human robot
interface.",

[Link]: Human Emotion Recognition using Convolutional Neural Network in

Real Time
Authors: Rohit Pathar,Abhishek Adivarekar,Arti Mishra,Anushree
Deshmukh,
Publication: 2019 1st International Conference on Innovations in Information
and Communication Technology (ICIICT)
The human emotion recognition has attracted interest of many problem

18
solvers in the field of artificial intelligence. The emotions on a human face say
so much about our thought process and give a glimpse of what's going on
inside the mind. Real time emotion recognition is to acquaint the machine
with human like ability to recognize and analyse human emotions. This
project aims to categorize a facial image into one of the seven emotions which
we are considering in this study, by building a multi class classifier. In this
paper we are using convolutional neural networks (CNNs) for training over
gray scale images obtained from fer2013 dataset. We experimented with
different depths and max pooling layers to get the best accuracy and
ultimately achieving 89.98% accuracy. To combat overfitting, we have used
technique like dropout. We are also analyzing the performance of different
network architectures like shallow network and modern deep network in
recognizing human emotion. We also present the real-time implementation of
emotion recognition in web-camera which provides accurate results for
multiple faces simultaneously. The results obtained from the research are
quite interesting.",

[Link]: Emotion Detection through Facial Expression using DeepLearning

Authors: Manish Kumar,Swati Srivastava,
Publication: 2021 5th International Conference on Information Systems and
Computer Networks (ISCON)
This paper aims to detect the emotion of a person through facial expression
using artificial neural network. There is a wide range of applications where
emotion detection has been used which witnesses the significance of emotion
detection. There are 6 predefined facial expressions used in various researches.
It includes happiness, surprise, sadness, disgust, fear and anger. Based on the
above facial expressions, we determine the behavior and mood of the person.
This proposal employs deep learning algorithm for emotion detection.",

19
[Link]: Emotion Recognition System from Speech and Visual Information
based on Convolutional Neural Networks
Authors: Nicolae-Cu0103tu0103lin Ristea,Liviu Cristian
Duu0163u,Anamaria Radoi,
Publication: 2019 International Conference on Speech Technology and
Human-Computer Dialogue (SpeD)
Emotion recognition has become an important field of research in the human-
computer interactions domain. The latest advancements in the field show that
combining visual with audio information lead to better results if compared to
the case of using a single source of information separately. From a visual
point of view, a human emotion can be recognized by analyzing the facial
expression of the person. More precisely, the human emotion can be described
through a combination of several Facial Action Units. In this paper, we
propose a system that is able to recognize emotions with a high accuracy rate
and in real time, based on deep Convolutional Neural Networks. In order to
increase the accuracy of the recognition system, we analyze also the speech
data and fuse the information coming from both sources, i.e., visual and audio.
Experimental results show the effectiveness of the proposed scheme for
emotion recognition and the importance of combining visual with audio data.",

[Link]: Analysis of Facial Emotion Recognition

Authors: Balaji Balasubramanian,Pranshu Diwan,Rajeshwar Nadar,Anuradha
Bhatia,
Publication: 2019 3rd International Conference on Trends in Electronics and
Informatics (ICOEI)
Human beings rely a lot on non-verbal communication and facial emotion is a
large part of it. In this review paper we cover the datasets and algorithms that
are used for Facial Emotion Recognition (FER). The algorithms range from
simple Support Vector Machines (SVM) to complex Convolutional Neural

20
Network (CNN). We explain these algorithms through the fundamental
research papers and go through their application to the task of FER.",

[Link]: Speech Emotion Recognition using Machine Learning

Authors: Kotikalapudi Vamsi Krishna,Navuluri Sainath,A. Mary Posonia,
Publication: 2022 6th International Conference on Computing Methodologies
and Communication (ICCMC)
The aim of the paper is to detect the emotions which are elicited by the
speaker while speaking. Emotion Detection has become a essential task these
days. The speech which is in fear, anger, joy have higher and wider range in
pitch whereas have low range in pitch. Detection of speech is useful in
assisting human machine interactions. Here we are using different
classification algorithms to recognize the emotions , Support Vector Machine ,
Multi layer perception, and the audio feature MFCC, MEL, chroma, Tonnetz
were used. These models have been trained to recognize these emotions (Calm,
neutral, surprise, happy, sad, angry, fearful, disgust). We got an accuracy of
86.5% and testing it with the input audio we get the same.",

[Link]: A Study on Human Emotion Detection

Authors: Shambhavi Bhardwaj,Ashutosh Dixit,Kshitij Yadav,Gokul Rajan V,
Publication: 2021 International Conference on Advance Computing and
Innovative Technologies in Engineering (ICACITE)
This paper centers around an arrangement of perceiving a human's feeling
from an identified human's face. The dissected data is passed on by the locales
of the eye and the mouth into a combined new picture in different outward
appearances relating to six widespread essential facial feelings. The yield data
got could be taken care of as a contribution to a machine fit for collaborating
with social abilities, with regards to building socially wise frameworks. The
approach utilizes an order method of data into another melded picture which is

21
made out of two squares coordinated by the territory of the eyes and mouth,
delicate regions to changes human's demeanor and that are especially pertinent
for the disentangling of enthusiastic articulations. At long last we utilize the
blended picture as a contribution to a feed-forward neural organization
prepared by back-engendering. Such investigation of blended pictures causes
it conceivable, to acquire important data through the mix of appropriate
information in a similar picture and diminish the preparation set time while
saving grouping rate. It is appeared by test results that the proposed
calculation can distinguish feeling with great exactness.",

[Link]: Emotion Detection with Facial Feature Recognition Using CNN &
OpenCV
Authors: Sarwesh Giri,Gurchetan Singh,Babul Kumar,Mehakpreet
Singh,Deepanker Vashisht,Sonu Sharma,Prince Jain,
Publication: 2022 2nd International Conference on Advance Computing and
Innovative Technologies in Engineering (ICACITE)
Emotion Detection through Facial feature recognition is an active domain of
research in the field of human-computer interaction (HCI). Humans are able to
share multiple emotions and feelings through their facial gestures and body
language. In this project, in order to detect the live emotions from the human
facial gesture, we will be using an algorithm that allows the computer to
automatically detect the facial recognition of human emotions with the help of
Convolution Neural Network (CNN) and OpenCV. Ultimately, Emotion
Detection is an integration of obtained information from multiple patterns. If
computers will be able to understand more of human emotions, then it will
mutually reduce the gap between humans and computers. In this research
paper, we will demonstrate an effective way to detect emotions like neutral,
happy, sad, surprise, angry, fear, and disgust from the frontal facial expression
of the human in front of the live webcam.",

22
[Link]: Emotion Detection using Deep Facial Features
Authors: Hari Kishan Kondaveeti,Mogili Vishal Goud,
Publication: 2020 IEEE International Conference on Advent Trends in
Multidisciplinary Research and Innovation (ICATMRI)
Emotion Detection became one of the most efficient and challenging activities
in human interactions. In general, facial gestures are normal and clear means
of expressing their feelings and intentions to human beings. The main features
of non-verbal conversations are facial expressions. The study of the
techniques of Face Emotion Recognition involves three key phases, like pre-
processing, extracting of features and, classification techniques. This paper
proposes the comparison of deep learning architectures available in Keras for
emotion detection using the Deep Facial Features in images using Transfer
Learning from famous pre-trained models like VGG-16, ResNet152V2,
InceptionV3, and Xception and generating bottleneck features for our input
images. The performance of these models is evaluated based on the dataset
which is a combination of the Cohn-Kanade Dataset (CK+) and Japanese
female facial emotion (JAFFE). For the above-mentioned architectures, the
accuracies obtained are 83.16 %, 82.15 %, 77.1 %, 78.11 % respectively.",

[Link]: Facial Emotion Recognition Using Deep Convolutional Neural

Network
Authors: E. Pranav,Suraj Kamal,C. Satheesh Chandran,M.H. Supriya,
Publication: 2020 6th International Conference on Advanced Computing and
Communication Systems (ICACCS)
The rapid growth of artificial intelligence has contributed a lot to the
technology world. As the traditional algorithms failed to meet the human
needs in real time, Machine learning and deep learning algorithms have
gained great success in different applications such as classification systems,

23
recommendation systems, pattern recognition etc. Emotion plays a vital role in
determining the thoughts, behaviour and feeling of a human. An emotion
recognition system can be built by utilizing the benefits of deep learning and
different applications such as feedback analysis, face unlocking etc. can be
implemented with good accuracy. The main focus of this work is to create a
Deep Convolutional Neural Network (DCNN) model that classifies 5 different
human facial emotions. The model is trained, tested and validated using the
manually collected image dataset.",

[Link]: Facial Emotion Detection using Machine Learning and Deep

Learning Algorithms
Authors: Snehal Bhogan,Kedar Sawant,Nidhi Gondalekar,Rachel
Carvalho,Vassant Kalangutkar,Alleena Mathew,
Publication: 2023 2nd International Conference on Edge Computing and
Applications (ICECAA)
Emotion recognition plays a critical role in various domains, such as human-
computer interaction, psychology, and market research. With the growing
popularity of social media platforms and the increasing use of multimedia
content, the ability to automatically recognize emotions from images has
become a significant research area. In this research, through a pre-trained
CNN model, rich and high-level features that effectively capture the
emotional content present in the images are extracted. The dataset for the
same is FER2013 which is downloaded through Kaggle and the data to test
emotion will be taken live feed through a webcam. The extracted features are
then fed into three classifiers namely: CNN, KNN, and random forest.
Conducted experiments on a publicly available emotion recognition dataset
named FER-2013. It illustrates the efficacy of change in accuracy by
achieving a vast difference between the CNN, KNN, and Random Forest.

24
 CHAPTER 3
SYSTEM DESIGN

3.1 Existing System

Current stress detection methods predominantly rely on subjective self-
reporting or physiological measurements, such as heart rate and cortisol levels.
These approaches lack the precision and real-time capabilities required for
effective stress monitoring. Moreover, they often necessitate intrusive
measures, hindering their practicality in various settings. The absence of a
non-intrusive, accurate, and real-time stress detection system limits the scope
of proactive intervention and support for individuals experiencing stress. The
need for a more advanced and versatile solution becomes evident as the
prevalence of stress-related concerns continues to rise in contemporary society.

3.2 Proposed System

The proposed system represents a paradigm shift in stress detection by
introducing a multi-modal approach that integrates facial recognition and
emotion analysis. Leveraging computer vision, deep learning, and machine
learning techniques, the system captures facial images through a web camera
using OpenCV's Haar cascade algorithm for face detection. The isolated faces
undergo stress estimation via a pre-trained Convolutional Neural Network
(CNN), trained on a diverse dataset to recognize nuanced facial cues
indicative of stress. Additionally, emotional features are extracted using a k-
Nearest Neighbors (KNN) algorithm. This comprehensive approach not only
enhances the accuracy of stress level assessments but also provides a real-time,
non-intrusive, and adaptable solution suitable for various settings. The
integration of OpenCV's face detection, CNN, and KNN creates a robust
stress detection system, addressing the limitations of existing methods and
contributing to proactive mental health monitoring and intervention initiatives.

25
3.3 BLOCK DIAGRAM

26
3.4 METHODLOGY

The project's methodology begins with the collection of a diverse dataset for
training the Convolutional Neural Network (CNN) used for facial expression
analysis. This dataset encompasses a broad spectrum of facial expressions
associated with different stress levels. Concurrently, data from the
MAX30100 sensor, capturing physiological parameters such as temperature
and blood pressure, is collected. The K-Nearest Neighbors (KNN) algorithm
is then employed to process the sensor data. The integration of these two
datasets is achieved through advanced machine learning algorithms,
facilitating a comprehensive and nuanced synthesis of facial and physiological
metrics. This dual-modality approach enhances the depth and accuracy of
stress level assessment, overcoming the limitations of singular methodologies.

27
 CHAPTER 4
HARDWARE DESCRIPTION

4.1 NODEMCU (ESP8266)

Fig 4.1.1 NodeMCU (ESP8266)

The Atmel AVR® core combines a rich instruction set with 32 general
purpose working registers. All the 32 registers are directly connected to the
Arithmetic Logic Unit (ALU), allowing two independent registers to be
accessed in a single instruction executed in one clock cycle. The resulting
architecture is more code efficient while achieving throughputs up to ten times
faster than conventional CISC microcontrollers. The ESP8266/P provides the
following features: 32Kbytes of In-System Programmable Flash with Read-
While-Write capabilities, 1Kbytes EEPROM, 2Kbytes SRAM, 23 general
purpose I/O lines, 32 general purpose working registers, Real Time Counter
(RTC), three flexible Timer/Counters with compare modes and PWM, 1 serial
programmable USARTs , 1 byte-oriented 2-wire Serial Interface (I2C), a 6-
channel 10- bit ADC (8 channels in TQFP and QFN/MLF packages) , a
programmable Watchdog Timer with internal Oscillator, an SPI serial port,

28
and six software selectable power saving modes.
This allows very fast start-up combined with low power consumption. In
Extended Standby mode, both the main oscillator and the asynchronous timer
continue to run. Atmel offers the QTouch® library for embedding capacitive
touch buttons, sliders and wheels functionality into AVR microcontrollers.
The patented charge-transfer signal acquisition offers robust sensing and
includes fully debounced reporting of touch keys and includes Adjacent Key
Suppression® (AKS™) technology for unambiguous detection of key events.
The easy-to-use Q Touch Suite toolchain allows you to explore, develop and
debug your own touch applications. The device is manufactured using
Atmel’s high density non-volatile memory technology. The On-chip ISP Flash
allows the program memory to be reprogrammed In-System through an SPI
serial interface, by a conventional non-volatile memory programmer, or by an
On-chip Boot program running on the AVR core.

4.2 Pin diagram of ESP8266

The ESP8266/P is supported with a full suite of program and system

development tools including: C Compilers, Macro Assemblers, and Program
Debugger/Simulators, In-Circuit Emulators, and Evaluation kits. Below figure
4.1.2 pin diagram of ESP8266.

29
 Fig 4.2.1 Pin diagram of ESP8266

4.3 FEATURES OF ESP8266

28-pin AVR Microcontroller

Flash Program Memory: 32 kilo bytes
EEPROM Data Memory: 1 kilo bytes
SRAM Data Memory: 2 kilo bytes
I/O Pins: 23
Timers: Two 8-bit / One 16-bit
A/D Converter: 10-bit Six Channel
PWM: Six Channels

30
RTC: Yes with Separate Oscillator
MSSP: SPI and I²C Master and Slave Support
USART: Yes
External Oscillator: up to 20MHz
ADVANTAGES/ IMPROVEMENTS IN ESP8266
Still runs on 5 V, so legacy 5 V stuff interfaces cleaner
Even though it's 5 V capable, newer parts can run to 1.8 V. This wide range is
very rare.
Nice instruction set, very good instruction throughput compared to other
processors (HCS08, PIC12/16/18).
High quality GCC port (no proprietary crappy compilers!)
"PA" variants have good sleep mode capabilities, in micro-amperes.
Well rounded peripheral set
Q Touch capability
Pin Descriptions table
VCC
Digital supply voltage.
GND
Ground.

4.4 Port B (PB [7:0]) XTAL1/XTAL2/TOSC1/TOSC2

Port B is an 8-bit bi-directional I/O port with internal pull-up resistors
(selected for each bit). The Port B output buffers have symmetrical drive
characteristics with both high sink and source capability. As inputs, Port B
pins that are externally pulled low will source current if the pull-up resistors
are activated. The Port B pins are tri-stated when a reset condition becomes
active, even if the clock is not running.
Depending on the clock selection fuse settings, PB6 can be used as input to
the inverting Oscillator amplifier and input to the internal clock operating

31
circuit.
Depending on the clock selection fuse settings, PB7 can be used as output
from the inverting Oscillator amplifier.
If the Internal Calibrated RC Oscillator is used as chip clock source, PB [7:6]
is used as TOSC [2:1] input for the Asynchronous Timer/Counter2 if the AS2
bit in ASSR is set.

4.5 Port C (PC [5:0])

Port C is a 7-bit bi-directional I/O port with internal pull-up resistors (selected
for each bit). The PC [5:0] output buffers have symmetrical drive
characteristics with both high sink and source capability. As inputs, Port C
pins that are externally pulled low will source current if the pull-up resistors
are activated. The Port C pins are tri-stated when a reset condition becomes
active, even if the clock is not running.

4.6 PC6/RESET
If the RSTDISBL Fuse is programmed, PC6 is used as an I/O pin. Note that
the electrical characteristics of PC6 differ from those of the other pins of Port
C.
If the RSTDISBL Fuse is unprogrammed, PC6 is used as a Reset input. A low
level on this pin for longer than the minimum pulse length will generate a
Reset, even if the clock is not running. Shorter pulses are not guaranteed to
generate a [Link] various special features of Port ‘C’ are elaborated in the
Alternate Functions of Port C section.

4.7 Port D (PD [7:0])

Port D is an 8-bit bi-directional I/O port with internal pull-up resistors

32
(selected for each bit). The Port D output buffers have symmetrical drive
characteristics with both high sink and source capability. As inputs Port D
pins that are externally pulled low will source current if the pull-up resistors
are activated. The Port D pins are tri-stated when a reset condition becomes
active, even if the clock is not running.

4.8 AVCC
AVCC is the supply voltage pin for the A/D Converter, PC [3:0], and PE [3:2].
It should be externally connected to VCC, even if the ADC is not used. If the
ADC is used, it should be connected to VCC through a low-pass filter. Note
that PC [6:4] use digital supply voltage, VCC.

4.9 NODEMCU Uno Board Description

We will learn about the different components on the NODEMCU board. We
will study the NODEMCU UNO board because it is the most popular board in
the NODEMCU board family. In addition, it is the best board to get started
with electronics and coding. Some boards look a bit different from the one
given below figure 3.2.2, but most NODEMCUs have majority of these
components in common.

Fig 4.9.1 NODEMCU Uno Board

33
NODEMCU board can be powered by using the USB cable from computer.
All we need to do is connect the USB cable to the USB connection (1).
Power (Barrel Jack)
NODEMCU boards can be powered directly from the AC mains power supply
by connecting it to the Barrel Jack.

4.10 Voltage Regulator

The function of the voltage regulator is to control the voltage given to the
NODEMCU board and stabilize the DC voltages used by the processor and
other elements.

4.11 Crystal Oscillator

The crystal oscillator helps NODEMCU in dealing with time issues. How
does NODEMCU calculate time? The answer is, by using the crystal oscillator.
The number printed on top of the NODEMCU crystal is 16.000H9H. It tells
us that the frequency is 16,000,000 Hertz or 16 MHz.

4.12 NODEMCU Reset

We can reset wer NODEMCU board, i.e., start wer program from the
beginning. We can reset the UNO board in two ways. First, by using the reset
button (17) on the board. Second, we can connect an external reset button to
the NODEMCU pin labelled RESET (5).
Pins (3.3, 5, GND, Vin)
3.3V (6) − Supply 3.3 output volt
5V (7) − Supply 5 output volt
Most of the components used with NODEMCU board works fine with 3.3 volt
and 5 volt.
GND (8) (Ground) − There are several GND pins on the NODEMCU, any of

34
which can be used to ground wer circuit.
Vin (9) − This pin also can be used to power the NODEMCU board from an
external power source, like AC mains power supply.
Analog pins
The NODEMCU UNO board has five analog input pins A0 through A5.
These pins can read the signal from an analog sensor like the humidity sensor
or temperature sensor and convert it into a digital value that can be read by the
microprocessor.

4.13 MAX30100 :
The MAX30100 is an integrated pulse oximetry and heartrate monitor sensor
solution. It combines two LEDs, a photodetector, optimized optics, and low-
noise analog signal processing to detect pulse oximetry and heart-rate signals.
The MAX30100 operates from 1.8V and 3.3V power supplies and can be
powered down through software with negligible standby current, permitting
the power supply to remain connected at all times.

Applications
● Wearable Devices
● Fitness Assistant Devices
● Medical Monitoring Devices

Benefits and Features

● Complete Pulse Oximeter and Heart-Rate Sensor Solution Simplifies
Design
• Integrated LEDs, Photo Sensor, and High-Performance Analog Front -End
• Tiny 5.6mm x 2.8mm x 1.2mm 14-Pin Optically Enhanced System-in-
Package
● Ultra-Low-Power Operation Increases Battery Life for Wearable Devices

35
• Programmable Sample Rate and LED Current for Power Savings
• Ultra-Low Shutdown Current (0.7µA, typ)
● Advanced Functionality Improves Measurement Performance
• High SNR Provides Robust Motion Artifact Resilience
• Integrated Ambient Light Cancellation
• High Sample Rate Capability
• Fast Data Output Capability

Inside the DHT11, there is a humidity sensing component along with a

Thermistor.

Fig 4.13.1 DHT11 Thermistor.

Humidity sensing component has two electrodes with moisture holding
substrate sandwiched between them.
The ions are released by the substrate as water vapor is absorbed by it, which
in turn increases the conductivity between the electrodes.
The change in resistance between the two electrodes is proportional to the
relative humidity. Higher relative humidity decreases the resistance between
the electrodes, while lower relative humidity increases the resistance between
the electrodes.

36
 Fig 4.13.2 Humidity Sensor

DHt11 also contains a NTC/Thermistor to measure temperature. A thermistor

is a thermal resistor whose resistance changes drastically with temperature.
The term “NTC” means “Negative Temperature Coefficient”, which means
that the resistance decreases with increase of the temperature.

Fig 4.13.3 NTC Thermistor

On the other side, there is a small PCB with an 8-bit SOIC-14 packaged IC.
This IC measures and processes the analog signal with stored calibration
coefficients, does analog to digital conversion and spits out a digital signal
with the temperature and humidity.

37
 Fig 4.13.4 8-bit SOIC-14 IC

4.14 DHT11 Module Hardware Overview

At the heart of the module is the digital temperature & humidity sensor
manufactured by AOSONG – DHT11.
DHT11 Sensor
DHT11 can measure temperature from 0°C to 50°C with ±2.0°C accuracy,
and humidity from 20 to 80% with 5% accuracy.

Fig 4.14.1 DHT11 Sensor

Note that the sampling rate of the DHT11 is 1Hz, meaning you can get new
data from it once every second.

38
4.15 Supporting Circuitry
The module comes with all the essential supporting circuitry, so it should be
ready to run without any extra components.

Fig 4.15.1 DHT11 Module

The DHT11 sensors usually require external pull-up resistor of 10KΩ between
VCC and Out pin for proper communication between sensor and the Arduino.
However, the module has a built-in pull-up resistor, so you need not add it.
The module also has a decoupling capacitor for filtering noise on the power
supply.
DHT11 Module Pinout
The DHT11 module is fairly easy to connect. It has only three pins:

39
 Fig 4.15.2 DHT11 Module Pinout

+ (VCC) pin supplies power for the sensor. 5V supply is recommended,

although the supply voltage ranges from 3.3V to 5.5V. In case of 5V power
supply, you can keep the sensor as long as 20 meters. However, with 3.3V
supply voltage, cable length shall not be greater than 1 meter. Otherwise, the
line voltage drop will lead to errors in measurement.
Out pin is used to communication between the sensor and the Arduino.
– (GND) should be connected to the ground of Arduino.
Wiring DHT11 Module to Arduino
Let’s hook the DHT11 module up to the Arduino.
Connections are fairly simple. Start by connecting + (VCC) pin to the 5V
output on the Arduino and connect – (GND) to ground. Finally, connect the
Out pin to the digital pin #8.

The following diagram shows you how to wire everything.

40
Fig 4.15.3 Audrino UNO

41
 CHAPTER 5
SOFTWARE DESCRIPTION

5.1 Arduino Development Environment

The Arduino development environment contains a text editor for writing code,
a message area, a text console, a toolbar with buttons for common functions,
and a series of menus. It connects to the Arduino hardware to upload
programs and communicate with them.
Writing Sketches
Software written using Arduino are called sketches. These sketches are
written in the text editor. Sketches are saved with the file extension .ino. It has
features for cutting/pasting and for searching/replacing text. The message area
gives feedback while saving and exporting and also displays errors. The
console displays text output by the Arduino environment including complete
error messages and other information. The bottom righthand corner of the
window displays the current board and serial port. The toolbar buttons allow
you to verify and upload programs, create, open, and save sketches, and open
the serial monitor.
NB: Versions of the IDE prior to 1.0 saved sketches with the extension pde It
is possible to open these files with version 1.0, you will be prompted to save
the sketch with the .ino extension on save.
The Arduino environment uses the concept of a sketchbook: a standard place
to store your programs (or sketches). The sketches in your sketchbook can be
opened from the File Sketchbook menu or from the Open button on the
toolbar. The first time you run the Arduino software, it will automatically
create a directory for your sketchbook. You can view or change the location of
the sketchbook location from with the Preferences dialog.
'''Beginning with version 1.0, files are saved with a .ino file extension.
Previous versions use the .pde extension. You may still open .pde named files

42
in version 1.0 and later, the software will automatically rename the extension
to .ino.
Tabs, Multiple Files, and Compilation
Allows you to manage sketches with more than one file (each of which
appears in its own tab). These can be normal Arduino code files (no
extension), C files (.c extension), C++ files (.cpp), or header files (.h).
Uploading
Before uploading your sketch, you need to select the correct items from
the Tools Board and Tools Serial Portmenus. The boards are described below.
On the Mac, the serial port is probably something
like /dev/tty.usbmodem241(for an Uno or Mega2560 or Leonardo)
or /dev/[Link]-1B1 (for a Duemilanove or earlier USB board),
or/dev/tty.USA19QW1b1P1.1 (for a serial board connected with a Keyspan
USB-to-Serial adapter). On Windows, it's probably COM1 or COM2 (for a
serial board) or COM4, COM5, COM7, or higher (for a USB board) - to find
out, you look for USB serial device in the ports section of the Windows
Device Manager. On Linux, it should be /dev/ttyUSB0,/dev/ttyUSB1 or
similar.
Once you've selected the correct serial port and board, press the upload button
in the toolbar or select the Upload item from the File menu. Current Arduino
boards will reset automatically and begin the upload. With older boards (pre-
Diecimila) that lack auto-reset, you'll need to press the reset button on the
board just before starting the upload. On most boards, you'll see the RX and
TX LEDs blink as the sketch is uploaded. The Arduino environment will
display a message when the upload is complete, or show an error.
When you upload a sketch, you're using the Arduino bootloader, a small
program that has been loaded on to the microcontroller on your board. It
allows you to upload code without using any additional hardware. The
bootloader is active for a few seconds when the board resets; then it starts

43
whichever sketch was most recently uploaded to the microcontroller. The
bootloader will blink the on-board (pin 13) LED when it starts (i.e. when the
board resets).
Libraries
Libraries provide extra functionality for use in sketches, e.g. working with
hardware or manipulating data. To use a library in a sketch, select it from
the Sketch Import Library menu. This will insert one or more statements at
the top of the sketch and compile the library with your sketch. Because
libraries are uploaded to the board with your sketch, they increase the amount
of space it takes up. If a sketch no longer needs a library, simply delete it from
the top of your code.
There is a list of libraries in the reference. Some libraries are included with the
Arduino software. Others can be downloaded from a variety of sources.
Starting with version 1.0.5 of the IDE, you do can import a library from a zip
file and use it in an open sketch. See these instructions for installing a third-
party library.

5.2 PROGRAMMING
The Arduino Uno can be programmed with the Arduino software (download).
Select "Arduino Uno from the Tools Board menu (according to the
microcontroller on your board). For details, see the reference and tutorials.
The ATmega328 on the Arduino Uno comes preburned with a bootloader that
allows you to upload new code to it without the use of an external hardware
programmer. It communicates using the original STK500 protocol
(reference, C header files).
You can also bypass the bootloader and program the microcontroller through
the ICSP (In-Circuit Serial Programming) header; see these instructions for
details.
The ATmega16U2 (or 8U2 in the rev1 and rev2 boards) firmware source code

44
is available . The ATmega16U2/8U2 is loaded with a DFU bootloader, which
can be activated by:
• On Rev1 boards: connecting the solder jumper on the back of the board
(near the map of Italy) and then resetting the 8U2.
• On Rev2 or later boards: there is a resistor that pulling the 8U2/16U2
HWB line to ground, making it easier to put into DFU mode.
You can then use Atmel's FLIP software (Windows) or the DFU
programmer (Mac OS X and Linux) to load a new firmware. Or you can use
the ISP header with an external programmer (overwriting the DFU
bootloader). See this user-contributed tutorial for more information.
Automatic (Software) Reset
Rather than requiring a physical press of the reset button before an upload, the
Arduino Uno is designed in a way that allows it to be reset by software
running on a connected computer. One of the hardware flow control lines
(DTR) of theATmega8U2/16U2 is connected to the reset line of
the ATmega328 via a 100 nanofarad capacitor. When this line is asserted
(taken low), the reset line drops long enough to reset the chip. The Arduino
software uses this capability to allow you to upload code by simply pressing
the upload button in the Arduino environment. This means that the bootloader
can have a shorter timeout, as the lowering of DTR can be well-coordinated
with the start of the upload.
This setup has other implications. When the Uno is connected to either a
computer running Mac OS X or Linux, it resets each time a connection is
made to it from software (via USB). For the following half-second or so, the
bootloader is running on the Uno. While it is programmed to ignore
malformed data (i.e. anything besides an upload of new code), it will intercept
the first few bytes of data sent to the board after a connection is opened. If a
sketch running on the board receives one-time configuration or other data
when it first starts, make sure that the software with which it communicates

45
waits a second after opening the connection and before sending this data.
The Uno contains a trace that can be cut to disable the auto-reset. The pads on
either side of the trace can be soldered together to re-enable it. It's labeled
"RESET-EN". You may also be able to disable the auto-reset by connecting a
110 ohm resistor from 5V to the reset line; see this forum thread for details.

5.3 PYTHON 3.7:

Python is an interpreter, high-level, general-purpose programming language.
Created by Guido van Rossum and first released in 1991, Python's design
philosophy emphasizes code readability with its notable use of significant
whitespace.
Python is an easy to learn, powerful programming language. It has
efficient high-level data structures and a simple but effective approach to
object- oriented programming. Python’s elegant syntax and dynamic typing,
together with its interpreted nature, make it an ideal language for scripting and
rapid application development in manya reason most platforms and
may be freely distributed. The same site also contains distributions of and
pointers to many free third party Python modules, programs and tools, and
additional documentation. The Python interpreter is easily extended with new
functions and data types implemented in C or C++ (or other languages
callable from C). Python is also suitable as an extension language for
customizable applications. This tutorial introduces the reader informally to the
basic concepts and features of the Python language and system. It helps to
have a Python interpreter handy for hands-on experience, but all examples are
self-contained, so the tutorial can be read off- line as well. For a description of
standard objects and modules, see library-index. Reference-index gives a
more formal definition of the language. To write extensions in C or C++, read
extending-index and c-api-index. There are also several books covering
Python in depth. This tutorial does not attempt to be comprehensive and cover

46
every single feature, or even every commonly used feature. Instead, it
introduces many of Python’s most notes worthy features, and will give you a
good idea of the language’s flavor and style. After reading it, you will be able
to read and write Python modules and programs, and you will be ready to
learn more about the various Python library modules described in library-
index. If you do much work on computers, eventually you find that there’s
some task you’d like
to automate. For example, you may wish to perform a search-and-replace over
a large number of text files, or rename and rearrange a bunch of photo files in
a complicated way. Perhaps you’d like to write a small custom database, or a
specialized
GUI application or a simple game. If you’re a professional software developer,
you may have to work with several C/C++/Java libraries but find the usual
write/compile/test/re-compile cycle is too slow. Perhaps you’re writing a test
suite for such a library and find writing the testing code a tedious task. Or
maybe you’ve written a program that could use an extension language, and
you don’t want to design and implement a whole new language for your
application.
Typing an end-of-file character (Control-D on Unix, Control-Z on Windows)
at the primary prompt causes the interpreter to exit with a zero exit status. If
that doesn’t work, you can exit the interpreter by typing the following
command: quit(). The interpreter’s line-editing features include interactive
editing, history substitution and code completion on systems that support read
line. Perhaps the quickest check to see whether command line editing is
supported is typing Control-P to the first Python prompt you get. If it beeps,
you have command line editing; see Appendix Interactive Input Editing and
History Substitution for an introduction to the keys. Ifnothing appears to
happen, or if ^P is echoed, command line editing isn’t available; you’ll only
be able to use backspace to remove characters from the current line. The

47
interpreter operates somewhat like the Unix shell: when called with standard
input connected to a tty device, it reads and executes commands interactively;
when called with a file name argument or with a file as standard input, it reads
and executes a script from that file. A second way of starting the interpreter is
python -c command [arg] ..., which executes the statement(s) in command,
analogous to the shell’s -c option. Since Python statements often contain
spaces or other characters that are special to the shell, it is usually advised to
quote commands in its entirety with single [Link] Python modules are
also useful as scripts. These can be invoked using python-m module
[arg]...,which executes the source file for the module as if you had spelled out
its full name on the command line. When a script file is used, it is sometimes
useful to be able to run the script and enter interactive mode afterwards. This
can be done by passing -i before the script.
There are tools which use doc strings to automatically produce online or
printed documentation or to let the user interactively browse through code; it’s
good practice to include doc strings in code that you write, so make a habit of
it. The execution of a function introduces a new symbol table used for the
local variables of the function. More precisely, all variable assignments in a
functions to read the value in the local symbol table; whereas variable
references first look in the local symbol table, then in the local symbol tables
of enclosing functions, then in the global symbol table, and finally in the table
of built-in names. Thus, global variables cannot be directly assigned a value
within a function (unless named in a global statement), although they may be
referenced. The actual parameters (arguments) to a function call are
introduced in the local symbol table of the called function when it is called;
thus, arguments are passed using call by value (where the value is always an
object reference, not the value of the object).1 When a function calls another
function, a new local symbol table is created for that call. A function
definition introduces the function name in the current symbol table. The value

48
of the function name has a type that is recognized by the interpreter as a user-
defined function. This value can be assigned to another name which can then
also be used as a function.
Annotations are stored in the annotations attribute of the function as a
dictionary and haven o effect on any other part of the function. Parameter
annotations are defined by a colon after the parameter name, followed by an
expression evaluating to the value of the annotation. Return annotationsare
defined by a literal ->, followed by an expression, between the parameter list
and the colon denoting the end of the def statement.
The comparison operators in and not in check whether a value occurs (does
not occur) in a sequence. The operator is and does not compare whether two
objects are really the same object; this only matters for mutable objects like
lists. All comparison operators have the same priority, which is lower than
that of all numerical operators. Comparisons can be chained. For example, a <
b == c tests whether a is less than b and moreover b equals c. Comparisons
may be combined using the Boolean operators and the outcome of a
comparison (or of any other Boolean expression) may be negated with not.
These have lower priorities than comparison operators; between them, not has
the highest priority and or the lowest, so that A and not B or C is equivalent to
(A and (not B)) or C. As always, parentheses can be used to express the
desired composition. The Boolean operators and are so-called short-circuit
operators: their arguments are evaluated from left to right, and evaluation
stops as soon as the outcome is determined. For example, if A and C are true
but Bis false, A and B and C does not evaluate the expression C. When used
as a general value and not as a Boolean, the return value of a short-circuit
operator is the last evaluated argument.
Classes provide a means of bundling data and functionality together. Creating
a new class creates a new type of object, allowing new instances of that type
to be made. Each class instance can have attributes attached to it for

49
maintaining its state. Class instances can also have methods (defined by its
class) for modifying its state. Compared with other programming languages,
Python’s class mechanism adds classes with a minimum of new syntax and
semantics. It is a mixture of the class mechanisms found in C++ and Modula-
3. Python classes provide all the standard features of Object Oriented
Programming: the class inheritance mechanism allows multiple base classes, a
derived class can override any methods of its base class or classes, and a
method can call the method of a base class with the same name. Objects can
contain arbitrary amounts and kinds of data. As is true for modules, classes
partake of the dynamic nature of Python: they are created at runtime, and can
be modified further after creation. In C++ terminology, normally class
members (including the data members) are public (except see below Private
Variables), and all member functions are virtual. A sin Modula-3, there are no
short hands for referencing the object’s members from its methods: the
method function is declared with an explicit first argument representing the
object, which is provided implicitly by the call. A sin Small talk, classes
themselves are objects. This providesSemantics for importing and renaming.
Unlike C++ and Modula-3, built-in types can be used as base classes for
extension by the user. Also, like in C++, most built-in operators with special
syntax (arithmetic operators, sub scripting etc.) can be redefined for class
instances.(Lacking universally accepted terminology to talk about classes, I
will make occasional use of Smalltalk and C++ terms. I would use Modula-3
terms, since its object- oriented semantics are closer to those of Python than
C++, but I expect that few readers have heard of it.)
Objects have individuality, and multiple names (in multiple scopes) can be
bound to the same object. This is known as aliasing in other languages. This is
usually not appreciated on a first glance at Python, and can be safely ignored
when dealing with immutable basic types (numbers, strings, tuples).However,
aliasing has a possibly surprising effect on these mantic of Python code

50
involving mutable objects such as lists, dictionaries, and most other types.
This is usually used to the benefit of the program, since aliases behave like
pointers in some respects. For example, passing an object is cheap since only
a pointer is passed by the implementation; and if a function modifies an object
passed as an argument, the caller will see the change — this eliminates the
need for two different argument passing mechanisms as in Pascal.
A namespace is a mapping from names to objects. Most name spaces are
currently implemented as Python dictionaries, but that’s normally not
noticeable in any way (except for performance), and it may change in the
future. Examples of name spaces are: these to f built-in names (containing
functions such as abs(), and built-in exception names); the global names in a
module; and the local names in a function invocation. In a sense the set of
attributes of an object also form a namespace. The important thing to know
about namespaces is that there is absolutely no relation between names in
different namespaces; for instance, two different modules may both define a
function maximize without confusion — users of the modules must prefix it
with the module name. By the way, I use the word attribute for any name
following a dot — for example, in the expression z. real, real is an attribute of
the object z. Strictly speaking, references to names in modules are attribute
references: in the expression [Link], modname is a module
object and funcname is an attribute of it. In this case there happens to be a
straight forward mapping between the module’s attributes and the global
names defined in the module: they share the same namespace!1 Attributes
may be read-only or writable. In the latter case, assignment to attributes is
possible. Module attributes are writable: you can write
modname.the_answer = 42. Writable attributes may also be deleted with the
del statement. For example, del mod name .the_ answer will remove the
attribute the_answer from the object named by mod name. Namespaces are
created at different moments and have different lifetimes. The namespace

51
containing the built-in names is created when the Python interpreter starts up,
and is never deleted. The global namespace for a module is created when the
module definition is read in; normally, module namespaces also last until the
interpreter [Link] statements executed by the top-level invocation of the
interpreter, either read from a script file or interactively, are considered part of
a module called main, so they have their own global namespace.(The built-in
names actually also live in a module; this is called built ins.) The local
namespace for a function is created when the function is called, and deleted
when the function returns or raises an exception that is not handled within the
function. (Actually, forgetting would be a better way to describe what actually
happens.) Of course, recursive invocations each have their own local
namespace.
To speed uploading modules, Python caches the compiled version of each
module in the pycache directory under the name [Link], where
the version encodes the format of the compiled file; it generally contains the
Python version number. For example, in CPython release 3.3 the compiled
version of [Link] would be cached as pycache/[Link]. This
naming convention allows compiled modules from different releases and
different versions of Python to coexist. Python checks the modification date of
the source against the compiled version to see if it’s out of date and needs to
be recompiled. This is a completely automatic process. Also, the compiled
modules are platform-independent, so the same library can be shared among
systems with different architectures. Python does not check the cache in two
circumstances. First, it always recompiles and does not store the result for the
module that’s loaded directly from the command line. Second, it does not
check the cache if there is no source module. To support anon-source
(compiled only) distribution, the compiled module must be in the source
directory, and there must not be a source module. Some tips for experts:
You can use the -O or -OO switches on the Python command to reduce the

52
size of a compiled module. The -O switch removes assert statements, the -OO
switch removes both assert statements and doc strings. Since some programs
may rely on having these available, you should only use this option if you
know what you’re doing. “Optimized” modules have an opt- tag and are
usually smaller. Future releases may change the effects of optimization. A
program doesn’t run any faster when it is read from a .pyc file than when it is
read from a .py file; the only thing that’s faster about .pyc files is the speed
with which they are loaded. The module compile all can create .pyc files for
all modules in a directory. There is more detail on this process, including a
flow chart of the decisions

5.4 THONNY IDE:

Thonny is as mall and light weight Integrated Development Environment. It

was developed to provide a small and fast IDE, which has only a few
dependencies from other packages. Another goal was to be as independent as
possible from a special Desktop Environment like KDE or GNOME, so
Thonny only requires the GTK2 toolkit and therefore you only need the
GTK2 runtime libraries installd to run it.
For compiling Thonny yourself, you will need the GTK (>= 2.6.0) libraries
and header files. You will also need the Pango, Gliband ATK libraries and
header files. All these files are available at [Link] Furthermore
you need, of course, a C compiler and the Make tool; a C++ compiler is also
required for the included Scintilla library. The GNU versions of these tools
are recommended.
Compiling Thonny is quite easy. The following should do it:

53
% ./configure

% make

% make install

The configure script supports several common options, for a detailed list, type
% ./configure --help
There are also some compile time options which can be found in
src/Thonny .h. Please see Appendix C for more information. In the case that
your system lacks dynamic linking loader support, you probably want to pass
the option --disable-vte to the configure script. This prevents
compiling Thonny with dynamic linking loader support to automatically load
[Link].4 if available. Thonny has been successfully compiled and tested
under Debian 3.1 Sarge, Debian 4.0 Etch, Fedora Core 3/4/5, Linux From
Scratch and FreeBSD 6.0. It also compiles under Microsoft Windows.

At startup, Thonny loads all files from the last time Thonny was launched.
You can disable this feature in the preferences dialog (see Figure 3-4). If you
specify some files on the command line, only these files will be opened, but
you can find the files from the last session in the file menu under the "Recent
files" item. By default this contains the last 10 recently opened files. You can
change the amount of recently opened files in the preferences dialog. You can
start several instances of Thonny , but only the first will load files from the
last session. To run a second instance of Thonny , do not specify any file
names on the command-line, or disable opening files in a running instance
using the appropriate command line option.
Thonny detects an already running instance of itself and opens files from the

54
command-line in the already running instance. So, Thonny can be used to
view and edit files by opening them from other programs such as a file
manager. If you do not like this for some reason, you can disable using the
first instance by using the appropriate command line option If you have
installed [Link] in your system, it is loaded automatically by Thonny , and
you will have a terminal widget in the notebook at the bottom. If Thonny
cannot find [Link] at startup, the terminal widget will not be loaded. So
there is no need to install the package containing this file in order to run
Thonny . Additionally, you can disable the use of the terminal widget by
command line option, for more information see [Link] can use this
terminal (from now on called VTE) nearly as an usual terminal program like
xterm. There is basic clipboard support. You can paste the contents of the
clipboard by pressing the right mouse button to open the popup menu and
choosing Paste. To copy text from the VTE, just select the desired text and
then press the right mouse button and choose Copy from the pop up menu. On
systems running the X Window System you can paste the last selected text by
pressing the middle mouse button in the VTE (on 2-button mice, the middle
button can often be simulated by pressing both mouse buttons together). As
long as a project is open, the Make and Run commands will use the project’s
settings, instead of the defaults. These will be used whichever document is
currently displayed. The current project’s settings are saved when it is closed,
or when Thonny is shut down. When restarting Thonny , the previously
opened project file that was in use at the end of the last session will be
[Link] will run the corresponding executable file, shell script or
interpreted script in a terminal window. Note that the Terminal tool path must
be correctly set in the Tools tab of the Preferences dialog - you can use any
terminal program that runs a Bourne compatible shell and accept the "-e"
command line argument to start a command. After your program or script has
finished executing, you will be prompted to press the return key. This allows

55
you to review any text output from the program before the terminal window is
closed.
By default the Compile and Build commands invoke the compiler and linker
with only the basic arguments needed by all programs. Using Set Includes and
Arguments you can add any include paths and compile flags for the compiler,
any library names and paths for the linker, and any arguments you want to use
when running Execute.
Thonny has basic printing support. This means you can print a file by
passing the filename of the current file to a command which actually prints the
file.
However, the printed document contains no syntax highlighting.

56
 CHAPTER 6
WORKING

6.1 Facial Expression Analysis with CNN:

Preprocess the collected facial expression dataset, including image
normalization and augmentation techniques to enhance model robustness.
Train a Convolutional Neural Network (CNN) using frameworks like
TensorFlow or PyTorch, leveraging architectures suitable for facial expression
recognition.
Implement real-time facial expression analysis using a webcam to capture and
process live video frames.
The CNN predicts stress levels based on the recognized facial expressions,
providing a dynamic assessment in real-time.

6.2 Physiological Monitoring with MAX30100 and KNN:

Set up the MAX30100 sensor to capture physiological parameters, ensuring
proper calibration and data acquisition.
Design a K-Nearest Neighbors (KNN) algorithm to process the sensor data,
considering factors such as temperature and blood pressure patterns associated
with stress.
Implement real-time data streaming from the MAX30100 sensor, feeding the
processed information into the KNN algorithm for continuous stress level
prediction.

6.3 Integration of Facial and Physiological Data:

Develop a system for merging outputs from the CNN and KNN, creating a
unified stress assessment model.
Fine-tune the integration process using machine learning techniques to ensure
seamless blending of facial and physiological metrics.

57
Implement a user-friendly interface for real-time stress level visualization,
providing insights into both emotional and physiological aspects.

6.4 Validation and Evaluation:

Validate the system's performance through a comprehensive evaluation
process, utilizing a separate test dataset for both facial expressions and
physiological parameters.
Assess the accuracy, precision, recall, and F1-score of the CNN for facial
expression analysis and the KNN for physiological monitoring.
Conduct real-world testing scenarios to validate the system's robustness and
generalizability across different stress-inducing situations.

58
 CHAPTER 7
RESULTS
7.1 Output of CNN and KNN :

Fig 7.1.1 Confusion Matrix of CNN

Fig 7.1.2 Bar Graph of KNN

59
Fig 7.1.3 Accuracy vs Epoch

Fig 7.1.4 Loss vs Epoch

60
7.2 Classification Reports :
7.2.1 CNN :

Precision recall f1-score support

0 0.87 0.86 0.87 834
1 0.92 0.95 0.94 1432
2 0.88 0.86 0.87 980
3 0.85 0.86 0.86 968
4 0.95 0.92 0.93 643
accuracy 0.90 4857
macro avg 0.90 0.89 0.89 4857
weighted 0.90 0.90 0.90 4857
avg

Accuracy is: 0.9841143764892772

Confusion Matrix :

[[509 4]

[ 16 730]]

7.2.2 KNN :

Precision recall f1-score support

0 0.97 0.99 0.98 513
1 0.99 0.98 0.99 746
accuracy 0.98 1259
macro avg 0.98 0.99 0.98 1259
weighted 0.98 0.98 0.98 1259
avg

61
Accuracy is: 0.9841143764889992

Fig 7.2.1 Model Output

62
 CHAPTER 8
CONCLUSION

In conclusion, this project introduces an advanced multimodal stress detection

system that combines Convolutional Neural Networks (CNN) for real-time
facial expression analysis and k-Nearest Neighbors (KNN) for physiological
monitoring using the MAX30100 sensor.

The integration of these dual modalities provides a comprehensive

understanding of an individual's stress levels, considering both emotional and
physiological aspects. By overcoming the limitations of singular
methodologies, the system offers a more nuanced and accurate approach to
stress assessment.

The real-time nature of the analysis, coupled with the seamless fusion of
facial and physiological data, positions the system as a valuable tool for
mental health monitoring and personalized stress management.

As technology continues to evolve, this project lays the groundwork for

innovative solutions that contribute to a holistic understanding of stress and
ultimately enhance overall well-being.

63
 CHAPTER 9
REFERENCES

[Link] Jain,Kshitij Sawant,Mohammed Rehman,Rajesh Kumar,"Emotion

Detection and Characterization using Facial Features",2018 3rd International
Conference and Workshops on Recent Advances and Innovations in
Engineering (ICRAIE)
[Link] Pandey,Sonakshi Handoo,Yogesh,"Facial Emotion Recognition
using Deep Learning",2022 International Mobile and Embedded Technology
Conference (MECON)
[Link] Jaiswal,A. Krishnama Raju,Suman Deb,"Facial Emotion Detection
Using Deep Learning",2020 International Conference for Emerging
Technology (INCET)
[Link] Sharma,Vriddhi Bajaj,Jatin Arora,"Machine Learning Techniques
for Real-Time Emotion Detection from Facial Expressions",2023 2nd Edition
of IEEE Delhi Section Flagship Conference (DELCON)
[Link] Hosur,Ashwini Desai,"Facial Emotion Detection Using
Convolutional Neural Networks",2022 IEEE 2nd Mysore Sub Section
International Conference (MysuruCon)
[Link] Pathar,Abhishek Adivarekar,Arti Mishra,Anushree
Deshmukh,"Human Emotion Recognition using Convolutional Neural
Network in Real Time",2019 1st International Conference on Innovations in
Information and Communication Technology (ICIICT)
[Link] Kumar,Swati Srivastava,"Emotion Detection through Facial
Expression using DeepLearning",2021 5th International Conference on
Information Systems and Computer Networks (ISCON)
[Link]-Cu0103tu0103lin Ristea,Liviu Cristian Duu0163u,Anamaria

64
Radoi,"Emotion Recognition System from Speech and Visual Information
based on Convolutional Neural Networks",2019 International Conference on
Speech Technology and Human-Computer Dialogue (SpeD)
[Link] Balasubramanian,Pranshu Diwan,Rajeshwar Nadar,Anuradha
Bhatia,"Analysis of Facial Emotion Recognition",2019 3rd International
Conference on Trends in Electronics and Informatics (ICOEI)
[Link] Vamsi Krishna,Navuluri Sainath,A. Mary Posonia,"Speech
Emotion Recognition using Machine Learning",2022 6th International
Conference on Computing Methodologies and Communication (ICCMC)
[Link] Bhardwaj,Ashutosh Dixit,Kshitij Yadav,Gokul Rajan V,"A
Study on Human Emotion Detection",2021 International Conference on
Advance Computing and Innovative Technologies in Engineering (ICACITE)
[Link] Giri,Gurchetan Singh,Babul Kumar,Mehakpreet Singh,Deepanker
Vashisht,Sonu Sharma,Prince Jain,"Emotion Detection with Facial Feature
Recognition Using CNN & OpenCV",2022 2nd International Conference on
Advance Computing and Innovative Technologies in Engineering (ICACITE)
[Link] Kishan Kondaveeti,Mogili Vishal Goud,"Emotion Detection using
Deep Facial Features",2020 IEEE International Conference on Advent Trends
in Multidisciplinary Research and Innovation (ICATMRI)
14.E. Pranav,Suraj Kamal,C. Satheesh Chandran,M.H. Supriya,"Facial
Emotion Recognition Using Deep Convolutional Neural Network",2020 6th
International Conference on Advanced Computing and Communication
Systems (ICACCS)
[Link] Bhogan,Kedar Sawant,Nidhi Gondalekar,Rachel Carvalho,Vassant
Kalangutkar,Alleena Mathew,"Facial Emotion Detection using Machine
Learning and Deep Learning Algorithms",2023 2nd International Conference
on Edge Computing and Applications (ICECAA)

65