IoT Model for Early Depression Prediction
IoT Model for Early Depression Prediction
APPLICANT:
Dr. S. Kavi Priya
ADDRESS
3 Associate Professor/CSE, Mepco Schlenk Engineering College (Autonomous),
Mepco Schlenk Engineering College (PO), Sivakasi, Tamilnadu 626005, India
COMPLETE SPECIFICATION
TITLE
"IOT BASED REGRESSION MODEL FOR EARLY PREDICATION OF
HUMAN DEPRESSION LEVELS "
The following specification particularly describes the invention and the manner in
which it is to be performed
TITLE OF THE INVENTION
5 FIELD OF INVENTION
The present invention relates to the human depression level prediction at early
stage using IoT based machine learning model. More particularly, the present
invention incorporates the parameters computed from the following multiple
feature sets(in terms of format, objective function etc..): physical activity, heart
10 rate variability, steps taken, sleep quality, mobility, call log, application usage log,
web browser log. Theses feature sets have the ability to capture the depression
symptoms outlined in Diagnostic and Statistical Manual of Mental Disorders
(DSM-5). Further, the system predicts the depression level of an individual as
Normal, Mild, Moderate or Severe. Moreover, it provides the self-diagnosis
15 service to the individual using Chatbot when mild depression is detected or send
SMS immediately to the individual’s parent or guardian when moderate or
severe depression is detected.
Depression has become one of the most common and serious mental health
20 disorders worldwide. Globally, more than 264 million people are getting affected
by depression. Due to the economic crisis caused by Covid-19 pandemic, the
number is still increasing. Such pandemics may cause situations like university
closures, loss of jobs, income loss which leads to poor mental health in individual
human. The conventional methods for the diagnosis of depression includes the
25 self-report questionnaires and oral interviews conducted in clinical centres.
Mostly the depressed people are hesitant to approach the depression diagnosis
centre and attend the depression counselling by the clinicians. So, there is an
emerging need to develop an automatic system for depression detection without
2
causing any additional stress to the user. In this context, there are several
systems and methods providing solution to detect the depression in an
automated or computerized manner. Some of them are;
Prior Art
3
brainwaves in a patient for generating sleep data and an analyzer block to
predict whether the patient is experiencing depression using multivariate
analysis on the biomarkers of sleep data. To monitor the sleep patterns of
the patient, the EEG electrodes are mounted on the patient’s scalp. The
5 biological markers include at least one chronobiological marker,
microarchitectural marker, one continuity of sleep marker, REM density
and coherency analysis. The sleep data is then analyzed using Digital
Period Analysis and processed by a diagnostic device. To carry out the
classification analysis on the sleep data, a classifier block is adapted. The
10 prior sleep reports are sent to the analyzer using sleep report parser.
3. EP2660745A2 and US20130297536A1 discloses a system and method to
monitor the mental health behaviour of each patient using one of the
following electronic devices: a mobile phone, a personal computer or
laptop and a personal digital assistant. The data collected from the
15 above-mentioned devices can be: web activities, calls made, message
sending and user’s location tracking using GPS. Then, the unique
behavioral patterns of each patient is learned by the system and is used
as a baseline for depicting the chronic phase of the patient. The
processing unit of the system contains an artificial intelligence algorithm
20 to detect the deviation in patient’s regular behavioral pattern. If the
deterioration exceeds the fixed threshold, then an alert is sent to a
mental health professional.
4. US10299717B2 discloses a system and method for detecting stress level
of the user based on thermal measurements of the face. The system
25 comprises of an inward-facing head-mounted thermal cameras (CAM1
and CAM2) and a computer. The thermal measurements are taken on the
region in periorbital area of right eye ([Link].ROI1) and left eye
([Link].ROI2) of the user using the configured CAM1 and CAM2
respectively. Then, the feature values are generated by the computer
30 based on [Link].ROI1 and [Link].ROI2. The training samples of
4
[Link].ROI1and [Link].ROI2 were collected while the user was under
elevated stress and not under elevated stress. The stress level prediction
model was trained based on previously collected thermal measurement
samples ([Link].ROI1 and [Link].ROI2). The additional measurements
5 can also be generated by the computer to detect the stress level using
one of the following features: user’s movement, heart rate, heart rate
variability, activity of brainwave, user’s muscle activity and galvanic skin
response.
5. US11045134B2 discloses a brain computer interface (BCI) system to
10 assess the depression index of a patient. The system comprises of
electroencephalogram (EEG) headset and a computing device coupled
together by a communication medium. The components present in the
computing device are: a memory, a processor and a display. The system is
programmed to compute current depression index of the patient based
15 on current intrinsic network activity and current extrinsic activity of the
patient using EEG headset. Then, the association between current
depression index and a baseline depression index is displayed on the
device.
Limitations of the Prior Systems
20 1. The disclosed systems are mainly used for depression detection of
patients under medical observation or for elderly people.
2. Most of the disclosed systems are suitable for individuals who must
volunteer themselves to approach mental health experts for
medication.
25 3. Wearable devices designed are not user friendly.
4. Most of the devices are available only as the hospital equipment
which need clinical expertise to wear and record observations.
5
1. The main objective of the present invention is to develop an end-to-
end framework for detecting human depression level at early stage
based on the symptoms outlined in DSM-5.
2. It is another objective of the present invention to design a smart
5 wearable device which is very comfortable to wear and user friendly
and can be used by anyone who needs self-monitoring of their
depression level.
3. It is also an objective of the present invention to develop a mobile
application which serves as a bridge between the wearable device
10 and the cloud platform.
4. It is also an objective of the present invention to monitor the user
without disturbing their daily routine and without causing additional
stress to the user.
5. It is also an objective of the present invention to create self-
15 awareness to the people who are more hesitant to reach out to clinics
for medication.
6
depression level of the user. The aggregated data is stored temporarily in the
mobile internal storage (9). When the internet becomes available, it is
transferred to the permanent secured cloud storage (44) where the history of
records for every user is maintained. The cloud framework allows the user to
5 register with the system using wearable device ID by means of device
administration(13) and visualize the data (40) collected using the web portal (39).
The depression detection application hosted on the cloud server (15) creates
activity profile for every user based on their regular routine. The standard
activity profile is created using the history of records available in the cloud
10 storage. Then, a machine learning model is trained to assign a score by learning
the deviation between the current user activity record with the standard activity
profile. Based on the range of deviation score, the depression level is categorized
as Normal, Mild, Moderate and Severe. Then, the predicted depression level is
sent to the mobile device and wearable device which in turn will be displayed in
15 the respective user interfaces. When the depression level is mild, then a chatbot
application (34) embedded in the mobile application is used to create self-
awareness and improves the mood of the user. When the depression level is
moderate or severe, an alert message is sent to the parent/care taker of the
user. To reduce power consumption and increase the battery lifetime of the
20 wearable device, the sleep mode is enabled when it is not in usage.
7
9. Wireless Fidelity is referred as WiFi
1. Wearable device
10 2. Sensing unit
3. Background service of application in wearable device
4. User interface of wearable device
5. Signal Processing unit
6. Mobile device
15 7. Mobile application
8. Background Service of mobile application
9. Mobile internal storage
10. Cloud platform
11. Bluetooth connectivity between wearable device and mobile device
20 12. WiFi connectivity between mobile device and cloud
13. Device administration functionality in cloud
14. Cloud gateway
15. Cloud server
16. Cloud user interface
25 17. WiFi connectivity between cloud and wearable device
18. Accelerometer Sensor
19. Gyroscope Sensor
20. Step Counter Sensor
21. Heart Rate Monitor
8
22. Orientation Sensor
23. Light Sensor
24. GPS Sensor
25. Microphone
5 26. Analog to Digital Converter
27. Flash Memory
28. Battery Charger with USB Type-C Connector
29. Battery
30. Display unit of wearable device
10 31. BLE module
32. WiFi module
33. User interface of mobile application
34. SMS service
35. Chatbot service
15 36. Log Data Collection
37. Data integration and aggregation functionality
38. Mobile APIs
39. Web portal
40. Data Visualization
20 41. Device Manager
42. Application Manager
43. Model Construction
44. Cloud Storage
25 The present invention provides the end-to-end IoT framework for the real time
automatic prediction of human depression level at early stage. Figure 1 depicts
the overall architecture of the present invention. The three major components in
the present invention are: wearable device (1), mobile application (7) and cloud
platform (10). Figure 2 illustrates the internal architecture of the wearable
30 device. The wearable device consists of sensing unit embedded with eight
9
different sensors (18)-(25), signal processing unit (26), flash memory (27), display
unit (4), external power source (29) and two wireless data transfer modules:
Bluetooth (31) and WiFi (32). The mobile application architecture is shown in
Figure 3. The top layer is the user interface (33), background service (7) in the
5 next layer and mobile internal storage (local storage) (9) in the bottom layer. The
mobile application sends and receives data to other device or cloud using the
APIs (38). Figure 4 presents the overview of the cloud platform used in the
present invention. The components present in the cloud platform are: a cloud
gateway (14), user interface (33), device administration (13) and a cloud server
10 (15). The detailed description of each and every component is discussed below.
Initial Setup
The users are identified with the unique ID associated with their wearable device
(1). They can register to the system using the ID, personal details, height and
weight of the user and configure the wearable device. Then, the mobile
15 application (7) can be downloaded from the AppStore/Play Store available in
mobile device and synchronize it with the wearable device and cloud.
Wearable Device
The sensing unit (2) of the wearable device consists of eight different sensors
(18)-(25) to generate signals based on the user’s current physical state. The list of
20 sensors and the signals/data generated by the sensors are tabulated in Table 1.
The signal processing unit (5) process the raw sensor signals and convert it into
digital values for further processing of the data. The application installed on the
wearable device collects the sensor data and transfer (11) it to mobile
application at particular intervals using BLE module (31). The online data transfer
25 mode is not necessary for sensor data collection. If the wearable device is not
able to connect to the mobile application, the sensor data can be stored locally in
the flash memory (27) for more than a month. When the connectivity becomes
available, the stored data is transferred to the mobile device in batches and once
it is received completely, the data can be deleted from the wearable device.
10
The user interface (4) of the wearable device consists of a display unit
(30) with an OLED display to show the user’s depression level, current physical
activity, heart rate and sleep quality. To reduce the power consumption and
enhance the battery lifetime, the wearable device is either switched off or goes
5 to sleep mode during non-wear times. The sensors can detect the non-wear
times efficiently i.e., the times when the device is charging or the times when it is
not worn on the wrist during bathing/showering.
Table 1
Sensor Data
Accelerometer 3 axes (x, y, z)
Gyroscope 6 axes (3 acceleration axes, 3 angular axes)
Step counter Number of steps taken
Heart Rate Monitor Heart rate
9 values (3 acceleration axes, 3 angular axes,
Orientation sensor
magnetic field of 3 axes)
Resistance signal (low value – bright light, high
Light sensor
value - dark)
GPS sensor Latitude, Longitude
10 Mobile Application
The mobile application receives the sensor data collected by the wearable device
using Bluetooth API (38) in the background service (7) running in it. The
background service also records the call log, application usage log and web
browser log from the mobile device. Retaining the raw sensor data for extended
15 duration in the mobile storage increases the memory usage and transferring
such large volume to the cloud server over long duration may consumes more
mobile data. Therefore, the mobile application performs the data cleaning in the
mobile. Then, data aggregation is performed to reduce the size of the data and
to compute the parameters required for the feature sets in the mobile
11
application itself. The various features extracted from the sensor data is
displayed in Table 2.
Table 2
Symptom
Sensors/Data Used Features Extracted
Identification Sets
Accelerometer and Mean and Standard deviation
Gyroscope sensor of the respective axes
Physical Activity
HRM sensor Heart Rate
Step counter sensor Number of steps
Sleep posture (supine, left
lateral, right lateral, prone),
Micro body movements
Accelerometer,
(hand movement, body
Gyroscope and
trembling, arm raising), Hand
Orientation sensor
positions (chest, head,
Sleep Quality
abdomen), Body rollover
count.
Acoustic events(snore,
Microphone
cough)
Illumination condition
Light sensor
(strong, weak)
Heart Rate HRM, Accelerometer
Heart Rate variability
Variability sensor
Distance variance, Entropy,
Mobility
GPS sensor Normalized entropy,
Pattern/Cluster
Quotidian
Number of incoming and
Call Log outgoing calls, call duration
Mood and frequent contacts.
Frequently used applications,
Application usage log
and duration of usage
12
Symptom
Sensors/Data Used Features Extracted
Identification Sets
Frequently visited websites
Web browser log
and search queries.
Once the features are extracted, the extracted feature vectors are sent to
the cloud server (15) for depression level prediction. Before transferring the
extracted data to permanent cloud storage, it is stored temporarily in the mobile
5 internal storage (9). The internal storage is also utilized to store the application
meta data. The user interface (16) designed in mobile application displays the
user’s current activity, sleep quality, heart rate, steps counted, and the
depression level predicted by the present invention. The front end also includes
the chatbot application (35) and SMS (34). The chatbot application is provided in
10 three different languages(Hindi, Tamil and English). When mild depression is
detected, the self-diagnosis is provided using the chatbot application. It also
creates self-awareness and promote the mental health well-being by means of
links to some motivational speeches and lectures from psychiatrists/medical
expertise. When moderate/severe depression is detected, an alert notification
15 will be sent to the parent/care taker/guardian of the user.
Cloud Platform
The vital components present in the cloud platform is given in Figure 4. Cloud
Storage gateway(14) acts as bridge between the local applications and remote
cloud storage. It allows data transfer between the developed mobile/wearable
20 application and cloud server. The three major functionalities of the cloud
platform are: device administration (13) which allows the user to register with
the wearable device in the framework and configure the application for
synchronized data collection; cloud server (15) is responsible for the efficient
storage and retrieval of data from the cloud storage (44) moreover, user profiling
25 and model learning and prediction is also performed in the cloud server; user
13
interface (16) provides the web portal to login and data visualization (40) to
display the graphs and summaries of the data collected from the device. The
detailed description of each module is given in the following:
The cloud server (15) serves as the prime component in depression level
prediction. The cloud storage (44) maintains the history of records for every user.
An application program is hosted on the cloud server which has two sub
modules: User pattern recognition/Profiling and Depression level prediction.
15 Profiling is about creating a regular routine pattern of the user based on their
physical activities, sleep pattern, mobility pattern and mobile usage pattern. It
helps to learn about the user. For early depression detection at initial stage, first
two weeks of record history is used for baseline profile creation. Two different
kinds of profile are created for a user: weekdays profile and weekend profile
20 since the activities done in regular days differs from the weekends. The two
weeks duration is dedicated only for collecting the data monitored from the
wearable device. Once the baseline profile is created, the neural network is
trained using the previously created profile and the current records. The neurons
represent the symptoms/feature set related to depression. The weights in the
25 neural network are assigned with the scores based on the impact of symptoms in
the depression detection. For example, physical activity and sleep quality are
important factors to identify the healthy and depressed state of an individual, so
they are assigned with high scores initially to train the model. Progressively, the
weights are updated based on the prediction accuracy. The depression level
14
index is obtained by computing the deviation between the known activity profile
and the current profile. Based on the deviation measure, the depression level is
categorized as Normal, Mild, Moderate and Severe. The predicted outcome is
sent back to the user’s mobile device and wearable device to take further
5 actions. Regular usage of the wearable device produces more history of records
which leads to strong profile creation for better prediction accuracy.
10
15
20
15
I CLAIM:
5 a wearable device (1) having a unique ID, wherein the unique ID is used
for registering the wearable device to a cloud system using the unique ID and for
configuring the wearable device, wherein the personal details and the physical
parameters of the user, such as height and weight of the user are registered into
the cloud system;
a signal processing unit (5) located within the wearable device (1),
15 wherein the signal processing unit (5) is configured for processing the raw sensor
signals into digital values for further processing of the data, wherein the raw
sensor signals are generated from the sensors (18,19,20,21,22) located within the
wearable device;
a wearable device application installed within the wearable device (1) for
20 collecting the sensor data generated from the sensors (18,19,20,21,22) located
within the wearable device (1) and transmitting the sensor data to the mobile
device (6) for further processing wherein the data is transmitted via a Bluetooth
low energy (BLE) module (31) located within the wearable device (1);.
25 a display unit (30) crafted with a user interface of wearable device (4)
arranged within the wearable device (1) displays the physical state and depression
level of the user;
a mobile background service (8) operating within the mobile application
(7) located within the mobile device (6), for recording call log, application usage
30 log and web browser log from the mobile device (6);
16
a mobile application (7) located within the mobile device (6) performs the
data cleaning and subsequently undertakes data aggregation to reduce the size of
the data and to compute the parameters required for the feature sets in the mobile
application (7) itself;
a cloud storage (44) located within the cloud platform (10) for storing and
retrieving the data efficiently;
10 a cloud user interface (16) located within the mobile application (7)
installed within the mobile device (6) provides a web portal to login and data
visualization (40) for displaying graphs and summaries of the data collected from
the wearable device (1);
a Cloud Storage gateway (l4) located within the cloud platform (10) for
15 transmitting the data from the cloud platform (10) to the mobile device (6) and
vice versa;
20 a SMS service application (34) located within the mobile application (7)
installed within the mobile device (6) on detection of a particular level of
depression in the user relays an alert notification to the caretaker of the user,
wherein the particular level of depression includes moderate to higher level of
depression, wherein level characterized as normal, mild, moderate, and severe;
25 and
a chat box application (35) located within the user interface (16) of the
mobile application (7) installed within the mobile device (6) on detection of a
mild level of depression in the user undertakes self-diagnosis and by providing
30 links for obtaining medical advice to the user promotes the mental health well-
17
being of the user, wherein the chat box (34) provides services in at least three
languages, wherein languages include Hindi, Tamil, and English.
5. The mobile device (6) as claimed in claim 1, consists of an internal storage (9)
for storing of extracted data prior to the transmission of the data to the cloud
platform (10).
15
6. The application program hosted on the cloud server (15) located in the cloud
platform (10) comprises of two sub modules:
a user pattern profiling is based on the physical state and mobile usage pattern of
20 the user for learning about the user, wherein the user pattern profiling is one of
the two sub modules of the application program hosted on the cloud server (15)
located in the cloud platform (10);
a baseline profile of the user being created for detecting depression at an initial
25 stage;
a neural network is trained based on the baseline profile and current records of the
user, wherein weights in the neural network are assigned with the scores based on
the impact of symptoms in the depression detection, wherein neurons within the
30 neural network represent the symptoms related to depression in the user;
18
a depression level index is obtained by computing the deviation between the
known activity profile and the current profile, wherein predicted depression level
in the user is categorized as Normal, Mild, Moderate and Severe, wherein the
predicted depression level is the second sub module of the application program
5 hosted on the cloud server (15) located in the cloud platform (10) and
transmission of the predicted outcome to the wearable device (1) and mobile
device (6) via Bluetooth for perusal of the user.
7. The sensors (18, 19, 20, 21, 22) located within the wearable device (1) as
claimed in claim I, includes an accelerometer sensor (18), step counter sensor (20)
10 and gyroscope sensor (19).
8. The wearable device (1) mentioned in claim 1, wherein the user's sleep quality
is monitored using accelerometer sensor (18), gyroscope sensor (19), orientation
sensor (22) and light sensor (23) to detect sleep postures and movements, micro-
15 body movements, acoustic events and track illumination conditions of the
sleeping environment.
9. The wearable device (1) mentioned in claim 1, wherein the mobility pattern of
the user is monitored using the embedded GPS sensor (24) and the heart rate
20 variability of the user is monitored using the embedded Heart Rate Monitor (21)
and accelerometer sensor (18).
30
19
ABSTRACT
20