Email Classification Using RNN and Bidirectional

LSTM layers
A MINOR PROJECT REPORT

Submitted by

Samvardhan Singh - RA2111043020005

Pramod Gururajan - RA2111043020006
Anumita R. Ajit - RA2111043020008
Nikhil Shivanath - RA2111043020037

Under the guidance of

Dr. Vani R.
(Professor, Department of Electronics and Communication Engineering)

in partial fulfilment for the award of the

degree of

BACHELOR OF TECHNOLOGY
in

ELECTRONICS AND COMPUTER ENGINEERING

of

FACULTY OF ENGINEERING AND TECHNOLOGY

SRM INSTITUTE OF SCIENCE AND TECHNOLOGY

RAMAPURAM, CHENNAI -600089

NOV 2024
 SRM INSTITUTE OF SCIENCE AND TECHNOLOGY
(Deemed to be University U/S 3 of UGC Act, 1956)

BONAFIDE CERTIFICATE

Certified that this minor project report titled “Email Classification Using RNN and
Bidirectional LSTM layersis the bona-fide work of Samvardhan Singh[Reg No:
RA2111043020005], Pramod Gururajan[Reg No: RA2111043020006], Anumita
R. Ajit[Reg No: RA2111043020008], Nikhil Shivanath[Reg No:
RA2111043020037] who carried out the minor project work under my supervision.
Certified further, that to the best of my knowledge, the work reported herein does not
form any other project report or dissertation on the basis of which a degree or award
was conferred on an occasion on this or any other candidate.

SIGNATURE SIGNATURE

Dr. Vani R Dr. N. V. S. Sree Rathna Lakshmi

Professor Professor and Head
Supervisor
Electronics & Computer Engineering, Electronics & Computer Engineering,
SRM Institute of Science and Technology, SRM Institute of Science and Technology,
Ramapuram, Chennai. Ramapuram, Chennai.

Submitted for the minor project viva-voce held on

26.10.2024 at SRM Institute of Science and Technology,
Ramapuram, Chennai -600089.

INTERNAL EXAMINER 1 INTERNAL EXAMINER 2

 SRM INSTITUTE OF SCIENCE AND TECHNOLOGY
RAMAPURAM, CHENNAI - 89

DECLARATION

We hereby declare that the entire work contained in this minor project report titled
“Email Classification Using RNN and Bidirectional LSTM layers ” has been carried
out by Samvardhan Singh [Reg No: RA2111043020005], Pramod Gururajan [Reg
No: RA2111043020006], Anumita R. Ajit [Reg No: RA2111043020008], Nikhil
Shivanath [Reg No: RA2111043020037] at SRM Institute of Science and Technology,
Ramapuram Campus, Chennai- 600089, under the guidance of Dr. Vani R, Professor,
Department of Computer Science and Engineering.

Place: Chennai
Date: 26 Oct 2024 Samvardhan Singh

Pramod Gururajan

Anumita R. Ajit

Nikhil Shivanath
 ABSTRACT

We created an AI-driven approach for classifying emails, using a Recurrent Neural

Network(RNN) architecture with Bidirectional Long Short-Term Memory(BiLSTM) layers,
combines with GloVe pre-trained word embedding, sorting them into 18 distinct categories.
The categories used are promotional, educational, confirmational, personal, transactional,
newsletters, feedback/surveys, alerts/notifications, invitations, apologies/corrections, job
applications/career-related, collaboration/partnership, legal/compliance, internal
communications, holiday greetings, reminders, unsubscribe, and welcome emails. By using
contextual information from both forward and backward directions, the model achieves a
classification accuracy of 97% which outperforms existing methods. This makes integration
into email management systems efficient and offers a more streamlined and automated
approach to email organization, improving productivity by reducing manual sorting .

iv
 ACKNOWLEDGEMENTS

We are expressing our deep sense of gratitude to our beloved chancellor

Dr. T. R. PAARIVENDHAR, for providing us with the required infrastructure
throughout the course.
We take this opportunity to extend our hearty thanks to our Chairman
Dr. R. SHIVAKUMAR, SRM Ramapuram & Trichy Campus for his constant support.
We take this opportunity to extend our hearty thanks to our
C o - Chairman Mr. S. NIRANJAN, SRM Ramapuram & Trichy Campus for his
constant support.

We take this opportunity to extend our hearty thanks to our Dean

Dr. M. SAKTHI GANESH, Ph.D., for his constant support.

We convey our sincere thanks to our Head of the Department

Dr. N.V.S. SREE RATHNA LAKSHMI, Ph.D., for his suggestions, interest,
encouragement and support throughout the project.

We convey our sincere thanks to our Project coordinator Dr. DURGA DEVI,
Ph.D., for her suggestions, interest, encouragement, and support throughout the project.

We express our heartfelt thanks to our guide Dr. VANI R., Professor, for her
sustained encouragement, and constant guidance throughout the project work.

We express our deepest gratitude to, our parents, Teaching and Non- Teaching
faculties for their sustained encouragement, and constant support throughout our studies.

v
 TABLE OF CONTENTS
[Link]

ABSTRACT………………………………………………………………………….iv

ACKNOWLEDGMENTS………………………………………………………….. v

LIST OF FIGURES…………………………………………………………………vi

LIST OF TABLES………………………………………………………………….vii

LIST OF ACRONYMS AND ABBREVIATIONS viii

1 INTRODUCTION……………………………………………………………… 2
1.1 Introduction………………………………………………………………..2
1.2 Problem Statement………………………………………………………...2
1.3 Objective of the Project…………………………………………………...3
1.4 Project Domain3
1.5 Scope of the Project 3
1.6 Methodology 4

2 LITERATURE REVIEW……………………………………………………….5

3 PROJECT DESCRIPTION 7
3.1 Existing System…………………………………………………………...7
3.2 Proposed System………………………………………………………….7
3.2.1 Advantages……………………………………………………..8
3.3 Feasibility Study………………………………………………………….8
3.3.1 Economic Feasibility…………………………………………...9
3.3.2 Technical Feasibility………………………………………….. 9
3.3.3 Social Feasibility……………………………………………….9

4 PROPOSED WORK 11
4.1 Introduction..............................................................................................11
 4.2 General Architecture...............................................................................13
4.3 Design Phase............................................................................................15
4.3.1 Data Flow Diagram..................................................................16
4.3.2 UML Diagram..........................................................................13
4.3.3 Use Case Diagram...................................................................18
4.3.4 Sequence Diagram...................................................................19
4.4 Module Description.................................................................................19
4.4.1 DATA COLLECTION AND TRAINING DATA...................20
4.4.2 Step:1 Data collecting..............................................................20
4.4.3 Step:2 Processing of data.........................................................20
4.4.4 Step:3 Split the Data................................................................21
4.4.5 DATASETS SAMPLE.............................................................21
4.4.6 Step:4 Building the Model.......................................................22

5 IMPLEMENTATION AND TESTING 23

5.1 Testing …………………………………………………………… ...23

…

6 RESULTS AND DISCUSSIONS 27

6.1 Efficiency of the Proposed System...........................................................27

7 CONCLUSION AND FUTURE ENHANCEMENTS 29

7.1 Conclusion................................................................................................29
7.2 Future Enhancements……………………………………………………29

8 SOURCE CODE & POSTER PRESENTATION 31

8.1 Sample Code…………………………………………………………….31

References 34
 LIST OF FIGURES

4.1 Distribution of Email Categories...................................................................12

4.2 Architecture Diagram............................................................................13
4.3 Dataflow Diagram...........................................................................................15
4.4 UML Diagram........................................................................................16
4.5 Use Case Diagram...........................................................................................18
4.6 Sequence Diagram .................................................................................19
4.7 Preprocessing of Data.....................................................................................21
4.8 Dataset of 1800 emails of 18 classes......................................................21
4.9 Confusion Matrix............................................................................................24

5.1 Model Training.......................................................................................25

5.2 Model Accuracy Curve ..................................................................................25
5.3 Model Loss..............................................................................................27
 LIST OF ACRONYMS AND ABBREVIATIONS

BiLSTM BIDIRECTIONAL LONG SHORT

TERM MEMORY

RNN RECURRENT NUERAL NETWORK

GloVe GLOBAL VECTORS FOR WORD

REPRESENTATION
NLP NATURAL LANGUAGE PROCESSOR

SVM SUPPORT VECTOR MACHINES

1
 Chapter 1

INTRODUCTION

1.1 Introduction
Email communication is an important asset in both personal and professional settings, but managing the
sheer volume of emails has become very challenging. As there are several types of emails, automated
solutions to efficiently classify messages has been sought out more than ever. Traditional systems often fail
to account for the complexity and variability of modern email content, resulting in misclassifications. In
contrast, deep learning techniques, particularly Recurrent Neural Networks (RNNs), provide a more dynamic
and robust solution by capturing the semantic and contextual nuances of email content, adapting to evolving
patterns and offering a higher degree of classification accuracy. These advanced methods can significantly
improve workflow efficiency, especially in environments where timely and accurate email sorting is critical.

1.2 Problem Statement

Managing the sheer number and variations of emails has become increasingly challenging, with traditional
systems often misclassifying emails due to their inability to capture complex nuances of the email content.
This leads to suboptimal situations, especially in environments where timely and accurate classification is
critical.

1.3 Objective of the Project

The objective is to develop an AI-driven email classification system that uses Recurrent Neural Networks
with Bidirectional Long Short-Term Memory layers and GloVe word embeddings to accurately classify
emails into 18 distinct categories to improve the efficiency and accuracy of email management.

1.4 Project Domain

This project falls within the domains of Natural Language Processing (NLP) and Machine Learning,
focusing specifically on email classification. It aims to enhance automated systems for organizing and
prioritizing emails in both personal and professional communication settings.

1.5 Scope of the Project

The project covers the development, training, and evaluation of a deep learning model capable of
classifying emails into 18 categories. It integrates into email management systems, aiming to streamline
email organization and improve productivity in environments handling large volumes of diverse email
content.

2
1.6 Methodology

The project employs a Recurrent Neural Network (RNN) architecture, specifically focusing on Bidirectional
Long Short-Term Memory (BiLSTM) layers, to efficiently classify emails into distinct categories. RNNs are
well-suited for sequential data processing, and BiLSTM enhances this by processing information in both
forward and backward directions, which allows the model to understand the contextual relationships
between words in an email, no matter their order in the sentence. This is especially useful for complex email
types where the context of a word depends on both preceding and following words.
The model uses pre-trained GloVe (Global Vectors for Word Representation) embeddings to convert the
words in the emails into numerical vectors, which serve as input for the BiLSTM layers. GloVe embeddings
are powerful because they capture semantic relationships between words based on their co-occurrence in
large corpora, allowing the model to generalize better even for unseen word combinations. Each word in the
email is transformed into a high-dimensional vector, which then gets passed through two stacked BiLSTM
layers.
For training, the dataset is split into 80% for training and 20% for testing, ensuring that the model has
sufficient data to learn and generalize effectively. The Adam optimizer is used to fine-tune the learning
process, adjusting the model's weights based on the first and second moments of the gradients, which helps
speed up convergence and improves accuracy. The model's performance is monitored using metrics like
accuracy, precision, recall, and F1-score to ensure robust classification across all 18 categories.

3
 Chapter 2

LITERATURE REVIEW

1. Amudha Poobalan et al. [1] developed a secured email classification system using a
deep neural network with BiLSTM and hybrid encryption (AES-Rabbit) for enhanced
cloud security, outperforming other classifiers.
2. Alejandro De La Noval et al. [2] showed that large language models like BERT
improved spam classification accuracy, demonstrating their effectiveness in handling
complex email data.
3. Bandar Alshawi et al. [3] applied NLP techniques and BERT, achieving high accuracy
in spam detection, proving the capability of advanced NLP models.
4. Congying Dai [4] compared deep learning algorithms, introducing new methods that
enhanced accuracy and spam detection.
5. Dr. K. Parthiban et al. [5,6] introduced a BiLSTM-based hybrid deep neural network
for email classification, including emotion detection and phishing email handling with
hybrid encryption for data security.
6. Harshini B.V, Vadivazhagi S. [7] proposed a machine learning system for automated
email sorting, focusing on workflow efficiency.
7. Jiacheng Xu [8] evaluated machine learning models for spam classification, suggesting
future improvements in accuracy.
8. Marouane Zaizoune et al. [9] used supervised learning to classify emails on user-
defined categories, enhancing email management.
9. N. Saraswathi et al. [10] compared seven machine learning approaches for spam
classification, providing insights into classifier performance.
10. Pallavi N, P. Jayarekha [11] achieved high accuracy in spam classification using
machine learning algorithms with feature engineering.
11. Qianhe Ouyang et al. [12] compared KNN and Naive Bayes, finding KNN superior in
spam detection.

4
12. Shao Bo Cheng [13] highlighted Naive Bayes’ effectiveness in spam clustering,
comparing it with classifiers used by Google and Yahoo.
13. Srinivasa Rao Dangeti et al. [14] enhanced spam classification accuracy using
machine learning and feed-forward neural networks.
14. Wolfgang Grundmann [15] studied SMS and email spam classification, achieving
high accuracy with NLP and machine learning models.
15. Wahab A. Iddrisu [16] used Random Forest for content-based academic email spam
classification, achieving 94.2% accuracy.
16. Xiaoke Wang [17] found LSTMs superior to traditional methods for spam
classification.
17. Yi-Chu Huang [18] compared models like CNN, RNN, LSTM, and Naive Bayes,
concluding that deep learning models outperform traditional ones.
18. Zhenghao Miao [19] compared six models for spam classification, achieving 98.7%
accuracy.
19. Zili Luo, Farhana Zulkernine [20] proposed a CNN-BiLSTM model for multiclass
email categorization, outperforming other classifiers.

5
 Chapter 3

PROJECT DESCRIPTION

3.1 Existing System

Traditional email classification methods primarily use rule-based systems or machine
learning models such as Naive Bayes and Support Vector Machines (SVM). While these
approaches work for simple tasks like spam detection, they struggle with complex tasks
involving semantic and contextual understanding of email content. Rule-based systems rely on
static keywords, leading to inaccuracies, and machine learning models do not capture the
sequential nature of text, often failing to account for word dependencies and long-term patterns
in emails.

3.2 Proposed System

The proposed system utilizes a Recurrent Neural Network (RNN) with Bidirectional Long
Short-Term Memory (BiLSTM) layers to significantly enhance email classification compared to
traditional rule-based or machine learning systems. By leveraging GloVe word embeddings, the
model transforms words into dense vector representations, capturing their semantic relationships
based on co-occurrence in a large corpus. The BiLSTM layers process email content in both
forward and backward directions, ensuring that the model captures contextual meaning from
surrounding words, which is crucial for accurately classifying complex and context-sensitive
emails. This architecture enables the model to classify emails into 18 distinct categories,
including promotional, legal, transactional, and more, with a high level of accuracy.
Additionally, the integration of techniques like dropout, batch normalization, and early stopping
prevents overfitting and ensures robust performance, achieving an impressive classification
accuracy of 97%. This makes the system both efficient and reliable for real-world email
classification tasks.

Advantages
• Improves classification accuracy by capturing email sequence and context.

• Handles complex emails using BiLSTM layers and GloVe embeddings.

• Achieves accurate classification across various email types.

• Efficient, scalable, and adaptable to changing email formats.

6
 3.3 Feasibility Study

A Feasibility study is carried out to check the viability of the project and to analyze the strengths
and weaknesses of the proposed system. The feasibility study is carried out in three forms

• Economic Feasibility

• Technical Feasibility

• Social Feasibility

3.3.1 Economic Feasibility

The system is economically feasible due to its use of pre-trained GloVe embeddings, which
reduces the need for extensive computational resources during the training phase. While the
initial setup may require hardware with moderate processing capabilities, the cost is justified by
the increased productivity and reduced manual effort in email management. The system’s
adaptability ensures that it will remain effective with minimal retraining, making it cost-efficient
in the long term.

3.3.2 Technical Feasibility

The project is technically feasible as it builds upon well-established deep learning techniques
such as BiLSTM and RNN, which have been widely applied in natural language processing
tasks. The use of GloVe embeddings enhances the model's ability to understand word semantics,
while dropout and batch normalization techniques ensure robustness. The system can be
integrated into existing email platforms with minimal technical challenges, given the availability
of libraries like TensorFlow or PyTorch for implementation.

3.3.3 Social Feasibility

The automated email classification system addresses a widespread issue of managing high
email volumes in both personal and professional settings. By reducing manual effort and
improving productivity, the system has a positive social impact. It can be applied in various
industries where timely and accurate email sorting is crucial, including legal, education, and
customer service sectors. Moreover, by improving workflow efficiency, the system can reduce
employee stress related to email overload.

7
3.4 System Specification

3.4.1 Hardware Specification

• The hardware requirements for the project include a system with a multi-core processor, at
least 16GB of RAM, and a GPU for efficient training of the BiLSTM model. A storage
capacity of 500GB is recommended to handle large email datasets and pre-trained
embeddings. The system should also support a compatible deep learning framework, such as
TensorFlow or PyTorch, for model training and evaluation.

3.4.2 Software Specification

• The software requirements include Python as the primary programming language, along with
libraries such as TensorFlow or PyTorch for deep learning, NumPy and Pandas for data
manipulation, and GloVe for pre-trained word embeddings. Jupyter Notebook or any IDE can
be used for code development and testing. The system should also have access to an email
dataset, and libraries such as Scikit-learn can be used for data preprocessing and evaluation.

8
 Chapter 4

PROPOSED WORK

4.1 Introduction
The study "Email Classification Using RNN and Bidirectional LSTM layers" describes a novel AI-based
technique for email classification that employs a Recurrent Neural Network (RNN) architecture with
Bidirectional Long Short-Term Memory (BiLSTM) layers, supplemented with GloVe pre-trained word
embedding. The suggested algorithm classifies emails into 18 different categories, ranging from promotional
and transactional emails to legal notices and internal communications, with an accuracy rate of 97%. By
analyzing text sequences in both forward and backward directions, the BiLSTM model improves its
knowledge of contextual and semantic subtleties in the text, which is required for appropriately classifying
various email kinds.

Traditional email categorization systems have relied on rule-based approaches and simpler machine learning
techniques, such as Naive Bayes and Support Vector Machines (SVM), but these models frequently fail to
generalize effectively across complex and developing email formats. Deep learning models such as RNNs,
particularly BiLSTMs, solve these issues by successfully capturing long-term relationships and sequence
patterns in text, which is critical for comprehending context inside emails. GloVe embeddings improve the
model by giving rich, pre-trained vector representations of words, allowing it to grasp word semantics in a
larger context, including previously undiscovered word combinations.

To prevent overfitting, the model architecture comprises embedding layers, BiLSTM processing, and thick
layers, as well as regularization techniques like as dropout and batch normalization. It also uses an adaptive
optimizer (Adam) and categorical cross-entropy for multi-class classification. This technique represents a
substantial development in automated email categorization, decreasing the human work required for email
sorting and increasing productivity by properly managing complicated email types in both personal and
commercial settings. Figure 4.1 represents the distribution of email categories within the dataset.

9
 Figure 4.1: Distribution of Email Categories

4.2 General Architecture

Figure 4.2: Architecture Diagram

10
Figure 4.2 represents the detailed architecture of the RNN model with BiLSTM layers for email
classification. The flowchart defines the following:

1. User Input → Data Preprocessing:

a. Converting text to lowercase
b. Removing HTML tags
c. Removing URLs
d. Removing special characters
e. Removing stopwords
2. Preprocessing includes:
a. Padding: Standardizing input length
b. Feature Extraction
c. Tokenization: Converting preprocessed text into sequence of integers
3. Embedding Layer:
a. Uses GloVe pre-trained embeddings (100-dimensional vectors)
b. Converts tokenized text into word embeddings
4. BiLSTM Layers:
a. BiLSTM Layer 1: 128 units
b. BiLSTM Layer 2: 64 units
c. Processes text in both forward and backward directions
5. Model Components:
a. Dropout layer: For preventing overfitting (30% dropout rate)
b. Batch Normalization: For stabilizing training
c. Regularization: Additional measure against overfitting
6. Dense Layers:
a. Dense Layer: 64 units with ReLU activation
b. Output Layer: 18 units (corresponding to email categories) with softmax activation
7. Final Steps:
a. Load Trained Model
b. Predict Email Category
c. Display Results

11
 4.3 Design Phase
4.3.1 Data Flow Diagram

Figure 4.3: Data Flow Diagram

Figure 4.3 shows a flow diagram of the email classification system's process flow. It illustrates:
1. The process starts with "User Input" at the top
2. Branches into two parallel paths:
a. Left path: Goes through Raw E-mail text → Preprocessing →
Make Plan Adjustments → Monitor Project Progress
b. Right path: Goes through Feature Extraction → TF-IDF Calculation → Make Plan Adjustments →
Monitor Project Progress
3. Both paths converge into the "Classification model"
4. Finally leads to "Predicated E-mail category" (note: there appears to be a typo, should be "Predicted")

12
 4.3.2 UML Diagram

Figure 4.4: UML Diagram

Figure 4.4 shows the class diagram of the email classification system architecture with two main
components:

1. E-MAIL CLASSIFIER class:

a. Attributes: Preprocessor, Embedding Layer, bilstm_model, dense_layer
b. Methods: preprocess_text(), tokenize(), pad_sequence(), embed_text(), classify_email()
2. BI-LSTM MODEL class:
a. Attributes:
i. lstm_units1: int
ii. lstm_units2: int
iii. dropout_rate: float
b. Methods: forward(), train(), predict()

13
 4.3.3 Use Case Diagram

Figure 4.5: Use Case Diagram

Figure 4.5 illustrates the overall flow of the email classification system, where the user inputs an email
that undergoes preprocessing to clean and prepare the text. The system then classifies the email using the
RNN/BiLSTM model, which can also be updated through training with new data to improve performance.
The user can view the classified category results and interact with the model as needed.

14
4.3.4 Sequence Diagram

Figure 4.6: Sequence Diagram

Figure 4.6 represents a step-by-step sequence, starting with the user's email input. The email is cleaned,
tokenized, padded for uniform length, and then transformed using word embeddings. It is processed through
the Bi-LSTM model, and the output is passed through a dense layer and a softmax layer for classification.
The predicted email category is finally returned to the user.

4.4 Module Description

Our entire project is divided into two modules.

4.4.1 Data Collection and Training Data

Data Collection and training using Machine Learning Algorithms

15
4.4.2 Step:1 Data collecting
 Identify Data Sources: Collect emails from reliable sources, like publicly available datasets (e.g., Enron
email dataset, spam classification datasets) or organizational email systems with permission.
 Category Selection: Define clear email categories (e.g., promotional, transactional, personal) to guide
data labeling.
 Data Labeling: Use manual or automated labeling methods to assign emails to categories, ensuring
high-quality labeled data.
 Data Diversity: Ensure the dataset represents various email types, topics, and formats to improve the
model’s robustness and generalization.

4.4.3 Step:2 Processing of data

Figure 4.7 represents a step-by-step process of preprocessing the dataset before model training.
 Data Cleaning: Remove unwanted characters, HTML tags, hyperlinks, and non-essential metadata to
standardize text.
 Normalization: Convert text to lowercase, and apply lemmatization or stemming to unify word forms.
 Tokenization: Split the text into individual tokens (words or phrases) for easier processing by the
model.
 Remove Stop Words: Remove common stop words (like "the," "and") to reduce noise in the data.
 Word Embedding/Vectorization: Convert tokens to numerical form using pre-trained embeddings
(e.g., GloVe, Word2Vec) or encoding methods to represent semantic relationships.
 Train-Validation-Test Split: Divide the processed data into training, validation, and test sets to prevent
overfitting and assess model performance.

Figure 4.7: Preprocessing of Data

16
4.4.4 Step:3 Split the Data

After the pre-processing part, the dataset was split into 80% for training and 20% for testing. This split
helps to train the model on the majority of the data while reserving a portion for evaluating its performance
on unseen samples, ensuring better generalization. Figure 4.8 shows a sample of the split dataset.

4.4.5 Datasets Sample

Figure 4.8: Dataset of 1800 emails of 18 classes

4.4.6 Step:4 Building the Model

 Data Preprocessing: Clean the email text data (remove special characters, convert to lowercase, etc.),
then tokenize and vectorize it using pre-trained embeddings like GloVe.
 Model Architecture: Choose a recurrent neural network (RNN) architecture with BiLSTM layers to
handle the text's sequential nature. Add an embedding layer, followed by stacked BiLSTM layers with
dropout and batch normalization to prevent overfitting.
 Output Layer: Set a dense layer with a softmax activation to classify into multiple categories.

17
 Chapter 5

TESTING AND IMPLEMENTATION

5.1 Testing

Testing is the process of evaluating a system or its component(s) with the intent to find whether it
satisfies the specified requirements or not.
1. Performance Metrics: To comprehensively evaluate the model’s classification accuracy, a combination
of performance metrics is used, such as:
 Accuracy: Measures the proportion of total correct predictions, providing an overall sense of model
accuracy.
 Precision: Evaluates the ratio of true positive predictions to the total predicted positives, ensuring the
model’s relevance when identifying specific classes.
 Recall: Determines the proportion of true positives identified from the actual positives, assessing the
model's effectiveness in capturing all relevant instances.
 F1-Score: Balances precision and recall, providing a harmonic mean that highlights the model’s ability
to maintain accuracy across diverse categories. A high F1-score indicates robust, well-rounded
classification performance, particularly in scenarios with imbalanced datasets.
2. Confusion Matrix Analysis: Check the confusion matrix to understand category-wise classification
accuracy.
3. Fine-Tuning: Adjust and retrain the model if certain categories show low performance to improve
accuracy and generalization.

Figure 5.1: Confusion Matrix

18
 For training, the dataset is divided with 80% used for training and 20% for testing. The Adam optimizer, a
commonly used variant of stochastic gradient descent, is employed as it adapts the learning rate based on the
first and second moments of the gradients. The loss function us ed is categorical cross-entropy, ideal for
multi-class classification tasks like this, where more than two categories exist. Early stopping is applied
during training to halt the process if the validation loss does not improve for three consecutive epochs,
preventing overfitting and ensuring the model does not learn noise from the data. The model is trained for up
to 15 epochs with a batch size of 32, where each epoch represents a full pass through the training data, and
the batch size refers to the number of samples processed before the model updates its weights. Figure 5.1
represents the confusion matrix to understand the class wise accuracy of the model.

Figure 5.2: Model Training

Figures 5.2 and 5.3 display the training and validation curves for the model's accuracy and loss. The model
stops training after three consecutive epochs with no substantial improvement in validation loss, which is
done using early stopping. The F1-score is a balanced measure of performance by taking the harmonic mean
of accuracy and recall. Recall indicates how many positive examples were properly predicted by the model.

Category Precision Recall F1-Score Support

[Link] 1.00 1.00 1.00 24
[Link] 0.95 1.00 0.98 20
[Link] 0.92 0.92 0.92 12
[Link] 0.96 1.00 0.98 22
[Link] 1.00 1.00 1.00 22
[Link] 1.00 1.00 1.00 17
[Link] 1.00 1.00 1.00 21
[Link] 1.00 1.00 1.00 19
Figure [Link] 0.96 0.96 0.96 25 5.3: Model
[Link] 0.95 1.00 0.98 21 Accuracy Curve
[Link] 1.00 1.00 1.00 27
[Link] 1.00 1.00 1.00 15
[Link] 1.00 1.00 1.00 22
Table 5.1. Precision, Recall, F1
[Link] 0.95 0.95 0.95 19
[Link] 0.90 0.9519 0.95 19
[Link] 0.90 0.95 0.92 15
[Link] 0.94 0.83 0.88 18
[Link] 1.00 1.00 1.00 22
 score and support of each category

In summary, this proposed model offers a robust solution for classifying emails into multiple categories
using an RNN architecture with BiLSTM layers and GloVe embeddings. By leveraging advanced
techniques like dropout, batch normalisation, and early stopping, the model generalizes effectively to fresh
data and achieves excellent classification accuracy across a variety of email formats.

Implementing the model.

 Training Configuration: Split the dataset into training and testing sets (typically 80-20). Use
categorical cross-entropy as the loss function and an optimizer like Adam. Train for a set number of
epochs with early stopping to avoid overfitting.
 Model Training: Run the training process, monitor training and validation loss/accuracy, and adjust
hyperparameters if necessary.

20
 Chapter 6
RESULTS AND DISCUSSION

6.1 Efficiency of the Proposed System

The suggested model was trained using a dataset of 1800 emails divided into 18 separate categories. After
preprocessing the data and dividing it into training and testing sets (80% training and 20% testing), the
model was trained for a maximum of 15 epochs with a batch size of 32, using the Adam optimizer with
categorical cross-entropy as the loss function. Early stopping was used to prevent overfitting by interrupting
the training process after three consecutive epochs with no significant improvement in validation loss.

Figure 6.1: Model Loss

The model's performance was tested using measures like asaccuracy, precision, recall, and F1-score. The
model obtained an astounding 97% accuracy on the test set, proving its ability to categorize emails across all
18 categories. The high accuracy suggests that the model successfully trained to distinguish between distinct
sorts of emails, such as promotional, transactional, personal, and legal/compliance emails. Figure 6.1
represents the loss curves for the model during training and validation.

21
 Table 6.1. Comparison of Existing and Proposed System

Model Algorithm Accuracy

(%)
Naive Bayes Probability 87.5
Classifier
SVM Support Vector 90.1
Machine
CNN Convolutional 92.7
Neural Net
LSTM Long Short-Term 95.0
BiLSTM Bidirectional 97.0
(proposed) LSTM + GloVe

Above is a comparison table of various algorithms and models used in the previously referenced
research papers. In conclusion, the results demonstrate the effectiveness of combining Bidirectional
LSTM layers with pre- trained GloVe embeddings for email classification. The model not only
outperformed traditional methods like Naive Bayes and SVM but also showed substantial
improvements in handling complex and context-dependent email categories. While there are areas
for improvement, particularly in categories with overlapping content, the overall performance of the
model highlights its potential for real-world applications in automating email categorization in
business and personal contexts.

22
 Chapter 7
CONCLUSION AND FUTURE
ENHANCEMENTS
7.1 Conclusion
The proposed model utilizes a Recurrent Neural Network (RNN), applying
Bidirectional Long Short-Term Memory (BiLSTM) layers and GloVe embeddings, demonstrated excellent
accuracy in categorizing emails into 18 distinct categories while simultaneously handling simple and
complex types of emails. The utilization of the BiLSTM architecture allows the model to gather contextual
information from both previous and following terms in the text, increasing its ability to distinguish meaning
and context, this is especially notable in delicate categories such as legal/compliance and collaborative
communications. With a classification accuracy of 97% with the use of this architecture, the model
outperforms standard approaches like Naive Bayes and SVM, indicating its potential for real-world use in
automated email sorting and management. The resilience of the model makes it ideal for contexts where
accurate email classification is critical for workflow management.

7.2 Future Enhancements

The models performance has proven to be exceptional, However, we have also been able to determine areas
for future enhancement to increase its accuracy and versatility:
1) Expansion of existing Email categories: By including more detailed subcategories inside current classes,
such as dividing promotional emails into multiple sorts (e.g., seasonal promotions, tailored offers),
customers may receive more specialized email classification.
2) Mixed-Content Email handling: By improving the model's ability to recognize and categorize emails
including multiple types of content (for example, a promotional email with legal disclaimers) we can
improve the models efficiency and accuracy with respect to handling complicated email structures.
3) Sentiment Analysis: By incorporating sentiment analysis layers, the model might give emotional context,
such as urgency or purpose, which would be useful in ranking emails, and adding a level of depth in
understanding the true nature of the email.
4) Implementing transfer learning to improve generalization: Applying transfer learning techniques or
exploring pre-trained big language models may push the model to adapt to new or unknown email
categories with minimum retraining.
5) Real-Time Processing Optimization: Smoother, real-time email categorization can be enabled with
23
Additional optimizations, such as decreasing computing complexity or utilizing cloud-based processing.
This is especially useful in business environments with large influxes and volumes of emails.

24
 Chapter 8
SOURCE CODE
8.1 Sample Code
import tensorflow as tf
print(tf.__version__)

!pip install keras

import pandas as pd
import numpy as np
import re
import nltk
from [Link] import stopwords
from sklearn.model_selection import train_test_split
from [Link] import Tokenizer
from [Link] import pad_sequences
from [Link] import Sequential
from [Link] import Embedding, LSTM, Dense, Dropout, Bidirectional
from [Link] import Adam
from [Link] import EarlyStopping, ModelCheckpoint

[Link]('stopwords')

# Load the dataset

df = pd.read_csv("[Link]", header=None)

# Renaming columns
[Link] = ['email_type', 'content']

# Check for invalid label values

invalid_labels = df[~df['email_type'].isin(range(1, 19))]
if not invalid_labels.empty:
print("Invalid label values found:")
print(invalid_labels)

# Remove rows with invalid label values

df = df[df['email_type'].isin(range(1, 19))]

# Data preprocessing
def preprocess_text(text):
text = [Link]()
text = [Link](r'http\S+', '', text) # Remove URLs
text = [Link](r'<.*?>', '', text) # Remove HTML tags
text = [Link](r'[^a-zA-Z\s]', '', text) # Remove special characters (retain numbers)
text = ' '.join([word for word in [Link]() if word not in [Link]('english')]) #
Remove stopwords
return text

df['content'] = df['content'].apply(preprocess_text)
# Tokenization and Padding
tokenizer = Tokenizer()
tokenizer.fit_on_texts(df['content'])
X_seq = tokenizer.texts_to_sequences(df['content'])
max_length = 500
X_pad = pad_sequences(X_seq, maxlen=max_length, padding='post')

# Update labels
y = df['email_type'].values - 1 # Convert labels to 0-based indices

# Split dataset into train and test sets

X_train, X_test, y_train, y_test = train_test_split(X_pad, y, test_size=0.2, random_state=42)

# Define the model

embedding_dim = 300
vocab_size = len(tokenizer.word_index) + 1

model = Sequential()
[Link](Embedding(input_dim=vocab_size, output_dim=embedding_dim))
[Link](Bidirectional(LSTM(128)))
[Link](Dropout(0.5))
[Link](Dense(18, activation='softmax'))

# Compile the model

optimizer = Adam(learning_rate=0.001)
[Link](optimizer=optimizer,loss='sparse_categorical_crossentropy',
metrics=['accuracy'])

# Train the model

early_stop = EarlyStopping(monitor='val_loss', patience=3)
checkpoint = ModelCheckpoint('best_model.keras', monitor='val_loss', save_best_only=True)
history = [Link](X_train, y_train, epochs=45, batch_size=32, validation_data=(X_test,
y_test),
callbacks=[early_stop, checkpoint])

# Evaluate the model

loss, accuracy = [Link](X_test, y_test)
print(f"Test Loss: {loss}, Test Accuracy: {accuracy}")

# Save tokenizer and model

[Link]("email_classification_model.h5")
tokenizer_json = tokenizer.to_json()
with open("[Link]", "w") as json_file:
json_file.write(tokenizer_json)

References
Material Type Works Cited

Book in print [1] J. Clerk Maxwell, A Treatise on Electricity and Magnetism,

3rd ed., vol. 2. Oxford: Clarendon, 1892, pp. 68–73.
Journal article [1] Poobalan, A., et al. "A novel and secured email classification
using deep neural network with bidirectional long short-term
memory." Computer Speech & Language, vol. 89, 2025, p.
101667.
[5] Krishnamoorthy, Parthiban, et al. "A novel and secured
email classification and emotion detection using hybrid deep
neural network." International Journal of Cognitive Computing in
Engineering, vol. 5, 2024, pp. 44-57.
[7] Harshini, B. V. "Enhancing Workflow Efficiency through
Machine Learning-Based Email Sorting." Journal of Soft
Computing and Computational Intelligence, vol. 1, no. 2, 2024,
pp. 13-24.
[3] Alshawi, Bandar, et al. "Classification of SPAM mail utilizing
machine learning and deep learning techniques." International
Journal on Information Technologies and Security, vol. 16, no.
2, 2024, pp. 71-82.
[19] Douzi, Samira, et al. "Hybrid email spam detection model
using artificial intelligence." International Journal of Machine
Learning and Computing, vol. 10, no. 2, 2020.

eJournal (from [20] Miao, Zhenghao. "Efficient Spam Classification Using

internet) Machine Learning Methods." Highlights in Science, Engineering
and Technology, vol. 34, 2023, pp. 60-64.

Conference [2] De La Noval, Alejandro, et al. "Methodologies for Email

paper Spam Classification Using Large Language Models." In 2023
International Conference on Computational Science and
Computational Intelligence (CSCI), pp. 179-185. IEEE, 2023.
[6] Savitha, G., et al. "Advanced Email Spam Detection: A
Machine Learning Solution." In 2023 International Conference
on Evolutionary Algorithms and Soft Computing Techniques
(EASCT), pp. 1-5. IEEE, 2023.
[8] Xu, Jiacheng. "Automatic Classification and Analysis of
Spam Based on Machine Learning." In 2023 International
Conference on Industrial IoT, Big Data and Supply Chain
(IIoTBDSC), pp. 28-32. IEEE Computer Society, 2023.
[9] Zaizoune, Marouane, et al. "Automatic emails classification."
In 2023 10th International Conference on Wireless Networks
and Mobile Communications (WINCOM), pp. 1-4. IEEE, 2023.
 [10] Saraswathi, N., et al. "Email Spam Classification and
Detection using Various Machine Learning Classifiers." In 2024
International Conference on Advances in Computing,
Communication and Applied Informatics (ACCAI), pp. 1-7.
IEEE, 2024.
[11] Pallavi, N., and P. Jayarekha. "Efficient Spam Email
Classification Using Machine Learning Algorithms." In 2023 7th
International Conference on Computation System and
Information Technology for Sustainable Solutions (CSITSS), pp.
1-6. IEEE, 2023.
[12] Ouyang, Qianhe, et al. "E-mail Spam Classification using
KNN and Naive Bayes." Highlights in Science, Engineering and
Technology, vol. 38, 2023, pp. 57-63.
[13] Cheng, Shaopeng. "Classification of Spam E-mail based on
Naïve Bayes Classification Model." Highlights in Science,
Engineering and Technology, vol. 39, 2023, pp. 749-753.
[14] Dangeti, Srinivasa Rao, et al. "Classification Analysis for e-
mail Spam using Machine Learning and Feed Forward Neural
Network Approaches." In International Conference on
Computational Innovations and Emerging Trends (ICCIET-
2024), pp. 66-75. Atlantis Press, 2024.
[16] Bari, Prince, et al. "SMS and E-mail Spam Classification
Using Natural Language Processing and Machine Learning." In
International Conference on Communication, Electronics and
Digital Technology, pp. 103-115. Singapore: Springer Nature
Singapore, 2023.
[17] Iddrisu, Wahab Abdul, et al. "Content-Based Spam
Classification of Academic E-mails: A Machine Learning
Approach." In Advances in Information Communication
Technology and Computing: Proceedings of AICTC 2022, pp.
83-92. Singapore: Springer Nature Singapore, 2023.

[1] Poobalan, A., K. Ganapriya, K. Kalaivani, and K. Parthiban. "A novel and secured email
classification using deep neural network with bidirectional long short-term memory." Computer
Speech & Language 89 (2025): 101667.J. Clerk Maxwell, A Treatise on Electricity and
 Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68–73.
[2] De La Noval, Alejandro, Diana Gutierrez, Jayesh Soni, Himanshu Upadhyay, Alexander Perez-
Pons, and Leonel Lagos. "Methodologies for Email Spam Classification Using Large Language
Models." In 2023 International Conference on Computational Science and Computational
Intelligence (CSCI), pp. 179-185. IEEE, 2023.
[3] Alshawi, Bandar, Amr Munsh, Majid Alotaibi, Ryan Alturki, and Nasser Allheeib.
"Classification of SPAM mail utilizing machine learning and deep learning techniques."
International Journal on Information Technologies and Security 16, no. 2 (2024): 71-82R.
[4] Dai, Na, Brian D. Davison, and Xiaoguang Qi. "Looking into the past to better classify web
spam." In Proceedings of the 5th international workshop on adversarial information retrieval
on the web, pp. 1-8. 2009.
[5] Krishnamoorthy, Parthiban, Mithileysh Sathiyanarayanan, and Hugo Pedro Proença. "A novel
and secured email classification and emotion detection using hybrid deep neural network."
International Journal of Cognitive Computing in Engineering 5 (2024): 44-57.
[6] Savitha, G., K. Hithyshi, J. Harshitha, Dhanya Shree JN, and Pratiksha Soori. "Advanced Email
Spam Detection: A Machine Learning Solution." In 2023 International Conference on
Evolutionary Algorithms and Soft Computing Techniques (EASCT), pp. 1-5. IEEE, 2023.
[7] Harshini, B. V. "Enhancing Workflow Efficiency through Machine Learning-Based Email
Sorting."Journal of Soft Computing and Computational Intelligence (e-ISSN: 3048-6610) 1, no.
2 (2024): 13- 24.
[8] Xu, Jiacheng. "Automatic Classification and Analysis of Spam Based on Machine Learning." In
2023 International Conference on Industrial IoT, Big Data and Supply Chain (IIoTBDSC), pp.
28-32. IEEE Computer Society, 2023.
[9] Zaizoune, Marouane, Youssef Fakhri, and Siham Boulaknadel. "Automatic emails
classification." In 2023 10th International Conference on Wireless Networks and Mobile
Communications (WINCOM), pp. 1-4. IEEE, 2023.
[10] Saraswathi,N.,[Link],[Link],[Link],[Link] Kambattan. "Email Spam
Classification and Detection using Various Machine Learning Classifiers." In 2024
International Conference on Advances in Computing, Communication and Applied Informatics
(ACCAI), pp. 1-7. IEEE, 2024.
[11] Pallavi, N., and P. Jayarekha. "Efficient Spam Email Classification Using Machine Learning
Algorithms." In 2023 7th International Conference on Computation System and Information
Technology for Sustainable Solutions (CSITSS), pp. 1-6. IEEE, 2023
[12] Ouyang, Qianhe, Jiahe Tian, and Jiale Wei. "E-mail Spam Classification using KNN and
Naive Bayes." Highlights in Science, Engineering and Technology 38 (2023): 57-63.
[13] Cheng, Shaopeng. "Classification of Spam E-mail based on Naïve Bayes Classification
Model." Highlights in Science, Engineering and Technology 39 (2023): 749-753.
[14] Dangeti,SrinivasaRao,DileepKumarKadali,YesujyothiYerramsetti, Ch Raja Rajeswari, D.
Venkata Naga Raju, and Srinath Ravuri. "Classification Analysis for e-mail Spam using
Machine Learning and
[15] Feed Forward Neural Network Approaches." In International Conference on Computational
Innovations and Emerging Trends (ICCIET-2024), pp. 66-75. Atlantis Press, 2024.
[16] Bari,Prince,VimalaMathew,SuchiPrabhuTandel,PadvariyaAniket, Kishor S. Chaudhari, and
Swapnali Naik. "SMS and E-mail Spam Classification Using Natural Language Processing and
Machine Learning." In International Conference on Communication, Electronics and Digital
Technology, pp. 103-115. Singapore: Springer Nature Singapore, 2023.
[17] Iddrisu, Wahab Abdul, Sylvester Kwasi Adjei-Gyabaa, and Isaac Akoto. "Content-Based
Spam Classification of Academic E-mails: A Machine Learning Approach." In Advances in
Information
[18] Communication Technology and Computing: Proceedings of AICTC 2022, pp. 83-92.
Singapore: Springer Nature Singapore, 2023.
[19] Douzi,Samira,[Link],MouadLemoudden,andBouabid El Ouahidi. "Hybrid email
spam detection model using artificial intelligence." International Journal of Machine Learning
and Computing 10, no. 2 (2020).
[20] Miao, Zhenghao. "Efficient Spam Classification Using Machine Learning Methods."
Highlights in Science, Engineering and Technology34 (2023): [Link] Luo, Farhana
Zulkernine, "An Intelligent Email Classification System Using CNN-BiLSTM."
[21] Miao, Zhenghao. "Efficient Spam Classification Using Machine Learning Methods."
Highlights in Science, Engineering and Technology 34 (2023): 60-64
[22] Luo,Zili,andFarhanaZulkernine."AnIntelligentEmailClassification System." In 2023 IEEE
Symposium Series on Computational Intelligence (SSCI), pp. 1126-1131. IEEE, 2023.