EMOTION DETECTION FROM FACIAL EXPRESSIONS USING

MACHINE LEARNING

ACKNOWLEDGEMENT

I would like to express my sincere gratitude to my project guide for

their continuous guidance, valuable suggestions, and encouragement
throughout the development of this project. Their support helped me
understand both theoretical concepts and practical implementation of
machine learning techniques.

I am also thankful to my institution for providing the necessary

facilities, software tools, and academic environment to successfully
complete this project. The support from faculty members played a
significant role in the completion of this work.

I would like to thank my friends and classmates for their help and
constructive feedback during the project. Their discussions helped me
improve the overall quality of the system.

Finally, I am deeply grateful to my family for their constant motivation

and support.
ABSTRACT

Emotion detection is an important field in artificial intelligence that

enables machines to understand human emotions through facial
expressions. This project aims to develop a real-time emotion detection
system using machine learning and computer vision techniques.

The system uses OpenCV for detecting human faces and a

Convolutional Neural Network (CNN) for classifying emotions. The
model is trained on the FER-2013 dataset, which contains grayscale
images categorized into seven emotions: angry, disgust, fear, happy, sad,
surprise, and neutral.

The system captures live video through a webcam, detects faces,

processes the image data, and predicts emotions in real time. The output
is displayed on the screen along with a bounding box around the
detected face.

This project demonstrates how machine learning can be applied to

create intelligent systems capable of understanding human behavior. It
has applications in healthcare, education, security, and customer
service.
INDEX / CONTENTS

1. Acknowledgement

2. Abstract

3. Chapter 1: Introduction

4. Chapter 2: Literature Survey

5. Chapter 3: Problem Statement

6. Chapter 4: Implementation Methodology

7. Chapter 5: Result and Discussion

8. Chapter 6: Conclusion and Future Scope

9. References

[Link]
 Chapter 1: Introduction

1.1 Overview of Emotion Detection

Emotion detection, also known as facial emotion recognition, is a field of artificial

intelligence that focuses on identifying human emotions through analysis of facial
expressions. Human beings naturally express emotions such as happiness, sadness,
anger, fear, surprise, and disgust through movements of facial muscles. These
expressions can be captured using cameras and analyzed using computational
techniques.

The ability to detect emotions is an essential part of human communication. In

human-to-human interaction, emotions provide context and meaning beyond
spoken words. However, traditional computer systems lack this capability, as they
only respond to explicit inputs such as text or commands. Emotion detection aims to
bridge this gap by enabling machines to understand human feelings.

1.2 Importance of Emotion Detection

Emotion detection plays a significant role in improving human-computer

interaction. Modern systems are moving towards being more intelligent and
adaptive. By understanding the emotional state of users, systems can respond in a
more personalized manner.

For example:

 In online education platforms, detecting confusion or frustration can help

provide additional explanations.

 In customer service systems, detecting dissatisfaction can improve response

quality.

 In healthcare, monitoring emotional states can assist in diagnosing mental

health conditions.
This makes emotion detection an important tool in developing smarter and more
responsive systems.

1.3 Objectives of the Project

The main objective of this project is to design and implement a real-time emotion
detection system using machine learning techniques.

The specific objectives include:

 Capturing live video using a webcam

 Detecting human faces in real time

 Extracting relevant facial features

 Classifying emotions using a deep learning model

 Displaying results on the screen

The project aims to create a system that is both accurate and efficient.

1.4 Technologies Used

This project uses two main technologies:

1.4.1 OpenCV

OpenCV (Open Source Computer Vision Library) is used for image processing and
face detection. It provides pre-trained models and efficient algorithms that allow
real-time performance.

Key features:

 Face detection using Haar Cascade

  Image processing functions

 Real-time video handling

1.4.2 Convolutional Neural Network (CNN)

CNN is a deep learning model designed specifically for image analysis. It

automatically extracts features from images and classifies them into categories.

Advantages:

 High accuracy

 Automatic feature extraction

 Ability to learn complex patterns

1.5 Applications of Emotion Detection

Emotion detection has a wide range of applications across different fields:

Education

Used to monitor student engagement and improve learning outcomes.

Healthcare

Helps in detecting mental health issues such as depression and anxiety.

Security

Can identify suspicious behavior or stress levels in individuals.

Entertainment

Used in gaming and virtual reality to create immersive experiences.

Customer Experience

Improves user satisfaction by adapting responses based on emotions.

1.6 Challenges in Emotion Detection

Despite its advantages, emotion detection faces several challenges:

 Lighting Conditions: Poor lighting affects image quality

 Facial Variations: Different individuals express emotions differently

 Occlusion: Glasses, masks, or hair can block facial features

 Pose Variation: Face may not always be directly visible

Addressing these challenges is important for improving system performance.

1.7 Scope of the Project

This project focuses on real-time facial emotion detection using a webcam. It is

designed for basic emotion classification and does not include advanced features
such as speech analysis.

Future improvements can expand the system to include multimodal emotion

detection.
 Chapter 2: Literature Survey

2.1 Introduction

Emotion detection has been widely studied in artificial

intelligence. Researchers have developed various methods
to recognize emotions from facial expressions using
different techniques.

2.2 Traditional Machine Learning Approaches

Earlier systems used traditional algorithms such as:

Support Vector Machine (SVM)

SVM is used for classification by finding the optimal

boundary between different classes.

K-Nearest Neighbors (KNN)

KNN classifies data based on similarity with neighboring

data points.

Decision Trees
Decision trees use a hierarchical structure to make
classification decisions.

Limitations:

 Require manual feature extraction

 Lower accuracy compared to deep learning

 Time-consuming process

2.3 Feature Extraction Techniques

In traditional systems, features were manually extracted

using methods such as:

 Edge detection

 Texture analysis

 Shape detection

These features were then used for classification.

2.4 Deep Learning Approaches

With the advancement of technology, deep learning
methods became popular.

Convolutional Neural Networks (CNN)

CNN automatically extracts features from images and

provides better accuracy.

Advantages:

 No manual feature extraction

 High performance

 Ability to handle large datasets

2.5 Dataset Used in Emotion Detection

The FER-2013 dataset is commonly used for training

emotion detection models.

Features:

 48×48 grayscale images

 Labeled with emotions

 Large dataset for training

Emotion categories include:

 Angry

 Disgust

 Fear

 Happy

 Sad

 Surprise

 Neutral

2.6 Recent Research Trends

Recent advancements include:

 Deep CNN architectures

 Transfer learning

 Real-time detection systems

 Multimodal emotion recognition

These techniques improve both accuracy and performance.

2.7 Comparison of Methods

Method Accuracy Complexity

SVM Medium Low

KNN Medium Low

CNN High High

CNN is preferred due to better performance.

2.8 Limitations of Existing Systems

 Sensitive to lighting

 Dataset bias

 Difficulty in detecting subtle emotions

2.9 Summary
Deep learning approaches, especially CNN, provide better
results compared to traditional methods. This project uses
CNN for emotion classification.
 Chapter 3: Problem Statement

The main problem addressed in this project is to develop a system that can detect human
emotions in real time using facial expressions.

The system should be able to:

 Capture video using a webcam

 Detect faces in the video

 Analyze facial expressions

 Predict the correct emotion

However, there are several challenges involved in solving this problem.

One challenge is variation in lighting conditions. Poor lighting can make it difficult to
detect faces accurately. Another challenge is different facial angles. The system should
work even if the face is slightly tilted or not directly facing the camera.

Facial expressions can also vary between individuals. Different people express emotions
in different ways, which makes classification more difficult.

Real-time performance is another important factor. The system must process images
quickly to avoid delays.

The goal is to create a system that is accurate, fast, and easy to use.
 Chapter 4: Implementation Methodology

4.1 Introduction to Methodology

The implementation methodology describes how the entire system is designed,

developed, and executed. It explains the step-by-step process used to build the emotion
detection system, from capturing input data to producing the final output.

In this project, the methodology is based on combining computer vision techniques with
deep learning models. The system processes real-time video input, detects faces,
extracts features, and classifies emotions.

The methodology is divided into multiple stages to ensure clarity and efficiency in
system design.

4.2 System Overview

The emotion detection system follows a structured pipeline. Each stage performs a
specific task and passes the output to the next stage.

The main stages are:

1. Image Acquisition (Webcam Input)

2. Face Detection

3. Image Preprocessing

4. Feature Extraction

5. Emotion Classification (CNN Model)

6. Output Display

This pipeline ensures that raw video input is converted into meaningful emotional
information.
4.3 Image Acquisition

The first step in the system is capturing live video data using a webcam.

The webcam continuously records frames (images) in real time. Each frame acts as an
input for the system. Since video is a sequence of frames, the system processes each
frame independently.

Key Points:

 Frames are captured using OpenCV

 Frame rate affects system performance

 Higher resolution improves detail but increases computation

The captured frames are then passed to the face detection module.

4.4 Face Detection

Face detection is a critical step because the system must identify the region of interest
(ROI) where the face is present.

In this project, Haar Cascade Classifier is used for detecting faces.

How Haar Cascade Works:

 It uses a trained model to detect facial patterns

 It scans the image at different scales

 It identifies areas that match facial features

Steps:

1. Convert the image to grayscale

 2. Apply the Haar Cascade classifier

3. Detect coordinates of the face

4. Extract the face region

Why Grayscale?

 Reduces computational complexity

 Removes unnecessary color information

Challenges in Face Detection:

 Poor lighting conditions

 Multiple faces in frame

 Face partially covered

Once the face is detected, only that region is processed further.

4.5 Image Preprocessing

Before feeding the image to the neural network, preprocessing is required to standardize
the input.

Steps involved:

1. Resizing

The detected face is resized to 48 × 48 pixels because the model is trained on this size.

2. Normalization

Pixel values are scaled between 0 and 1.

Formula:
Pixel Value / 255
This helps in faster and more stable training.

3. Reshaping

The image is reshaped into the required format:

(48, 48, 1)

This indicates a grayscale image with one channel.

Importance of Preprocessing:

 Improves model accuracy

 Ensures consistency

 Reduces noise

4.6 Feature Extraction

Feature extraction is the process of identifying important patterns in the image.

In traditional methods, features were manually defined. However, in this project, CNN
automatically extracts features.

Types of Features:

 Edges

 Corners

 Shapes

 Facial structures

These features help the model distinguish between different emotions.

4.7 Convolutional Neural Network (CNN)

CNN is the core component of the system responsible for emotion classification.

Structure of CNN:

1. Convolution Layer

 Applies filters to the image

 Extracts features such as edges and textures

2. Activation Function (ReLU)

 Introduces non-linearity

 Helps model learn complex patterns

3. Pooling Layer

 Reduces image size

 Keeps important features

 Improves efficiency

4. Flatten Layer

 Converts 2D data into 1D

5. Fully Connected Layer

 Performs final classification

6. Output Layer

 Uses Softmax activation

 Outputs probability for each emotion

4.8 Model Training

The CNN model is trained using the FER-2013 dataset.

Training Process:

 Input: Preprocessed images

 Output: Emotion labels

Parameters:

 Optimizer: Adam

 Loss Function: Categorical Crossentropy

 Epochs: 10

 Batch Size: 64

During training, the model adjusts its weights to minimize error.

4.9 Real-Time Emotion Detection

After training, the model is used for real-time prediction.

Steps:

1. Capture frame

2. Detect face

3. Preprocess image

4. Predict emotion

5. Display result

The system continuously repeats this process.

4.10 Output Display

The final output is shown on the screen.

Display includes:

 Bounding box around face

 Emotion label (e.g., Happy, Sad)

 Confidence score

This makes the system interactive and user-friendly.

4.11 Advantages of the Methodology

 Real-time processing

 High accuracy with CNN

 Simple implementation

 Scalable system

4.12 Limitations

 Performance depends on lighting

 Requires good dataset

 Limited accuracy in extreme conditions

 Chapter 5: Result and Discussion

5.1 Introduction

This chapter presents the results obtained from the implementation of the emotion
detection system. It also includes a detailed discussion of the system’s performance
under different conditions.

The system was tested in real-time using a webcam. Various facial expressions were
captured and analyzed to evaluate the accuracy and reliability of the model.

5.2 Experimental Setup

The system was implemented using Python along with libraries such as OpenCV
and TensorFlow/Keras. A standard webcam was used to capture live video input.

Setup Details:

 Programming Language: Python

 Libraries: OpenCV, NumPy, TensorFlow/Keras

 Input Device: Webcam

 Dataset: FER-2013

 Model Type: CNN

The system processes each video frame and performs face detection followed by
emotion classification.

5.3 Evaluation Criteria

The performance of the system is evaluated based on the following factors:

Accuracy
Accuracy refers to how correctly the system predicts the emotion. It depends on the
quality of training data and model design.

Speed

Since the system works in real time, processing speed is very important. The system
should provide results without noticeable delay.

Robustness

The ability of the system to perform under different conditions such as lighting, face
angle, and background noise.

5.4 Observations

During testing, several observations were made:

5.4.1 Performance in Good Lighting

The system performs very well in good lighting conditions. The face is clearly
detected, and emotions are classified accurately.

5.4.2 Performance in Low Lighting

In low-light conditions, the accuracy decreases. Face detection may fail or give
incorrect results.

5.4.3 Effect of Face Orientation

When the face is directly facing the camera, results are accurate. However, tilted or
partially visible faces reduce accuracy.

5.4.4 Multiple Faces Detection

The system can detect multiple faces in a single frame. Each face is processed
separately, and emotions are displayed individually.
5.5 Emotion-wise Analysis

Happy

This emotion is detected with high accuracy because smiling creates distinct facial
features.

Sad

Moderate accuracy. Sometimes confused with neutral expression.

Angry

Detected correctly in most cases due to strong facial features such as frowning.

Surprise

High accuracy because of wide eyes and open mouth.

Neutral

Sometimes confused with other emotions due to lack of strong features.

5.6 Error Analysis

Some errors were observed during testing:

 Misclassification between similar emotions (e.g., sad and neutral)

 Failure in low lighting

 Difficulty in detecting partially visible faces

These errors are mainly due to limitations in dataset and model complexity.
5.7 Screenshot Explanation (IMPORTANT FOR REPORT)

Angry emotion detection

Happy emotion detection

Sad emotion detection

Example Explanation:

Figure 1: Angry Emotion Detection

The system successfully detected the face and classified the emotion as "Angry".
The bounding box highlights the detected face region, and the label shows the
predicted emotion.

Figure 2: Happy Emotion Detection

The system detected a smiling face and classified it as "Happy" with high
confidence.

Figure 3: Sad Face Detection

The system detected sad faces. and classified it as "sad" with low confidence

5.8 Advantages of the System

 Real-time detection

 Easy to use

 Accurate in controlled conditions

  Can detect multiple faces

5.9 Limitations of the System

 Sensitive to lighting

 Limited dataset

 Not suitable for extreme conditions

5.10 Discussion

Overall, the system performs well for real-time emotion detection. It demonstrates
the effectiveness of CNN in image classification tasks.

However, improvements are needed to handle complex real-world conditions.

Increasing dataset size and using advanced models can enhance performance.
 Chapter 6: Conclusion and Future Scope

6.1 Conclusion

This project successfully demonstrates the implementation of an emotion detection

system using machine learning and computer vision techniques.

The system is capable of detecting human faces and classifying emotions in real time. It
uses OpenCV for face detection and a Convolutional Neural Network (CNN) for emotion
classification.

The results show that the system performs well under normal conditions. It can accurately
detect emotions such as happy, angry, and surprise. The real-time performance makes it
suitable for practical applications.

The project also highlights the importance of deep learning in image processing tasks.
CNN plays a crucial role in extracting features and improving classification accuracy.

6.2 Learning Outcomes

Through this project, the following concepts were learned:

 Basics of machine learning and deep learning

 Working of Convolutional Neural Networks

 Image processing using OpenCV

 Real-time system development

 Handling datasets and model training

6.3 Future Scope

There are several ways to improve and extend this project:

1. Use Advanced Models

More complex models such as ResNet or VGG can improve accuracy.

2. Larger Dataset

Using a larger and more diverse dataset will improve generalization.

3. Mobile Application

The system can be converted into a mobile app for wider usage.

4. Web Integration

It can be integrated into web applications for online services.

5. Multimodal Emotion Detection

Combining facial expressions with voice analysis can improve results.

6. Real-world Deployment

The system can be used in industries such as healthcare and education.

6.4 Final Remarks

Emotion detection is an important step toward creating intelligent systems that

understand human behavior. This project provides a foundation for further research and
development in this field.

📘 REFERENCES (EXPANDED)

1. FER-2013 Dataset – Kaggle

2. OpenCV Official Documentation

3. TensorFlow/Keras Documentation
4. Goodfellow, Ian – Deep Learning Book

5. Research papers on CNN-based emotion detection

6. Online tutorials and academic resources

 References
1. FER-2013 Dataset (Kaggle)

2. OpenCV Documentation

3. TensorFlow Documentation
 Appendix (Code)

from fastapi import FastAPI, Request

from [Link] import CORSMiddleware

from [Link] import StaticFiles

from [Link] import Jinja2Templates

from [Link] import HTMLResponse

import uvicorn

import os

# Create directories relative to this file

BASE_DIR = [Link]([Link](__file__))

static_dir = [Link](BASE_DIR, "static")

templates_dir = [Link](BASE_DIR, "templates")

[Link](static_dir, exist_ok=True)

[Link](templates_dir, exist_ok=True)

app = FastAPI(title="Facial Emotion Intelligence API")

# Mount static files

# Important: Ensure .json and binary shards are served correctly

[Link]("/static", StaticFiles(directory=static_dir), name="static")

# Setup templates

templates = Jinja2Templates(directory=templates_dir)

# Enable CORS

app.add_middleware(

CORSMiddleware,

allow_origins=["*"],

allow_credentials=True,

allow_methods=["*"],

allow_headers=["*"],

@[Link]("/", response_class=HTMLResponse)

async def read_item(request: Request):

return [Link](request=request, name="[Link]")

if __name__ == "__main__":

# Use reload=True for easier development if needed, but not required

here

[Link](app, host="[Link]", port=8001)