CHAPTER 1

INTRODUCTION

In today’s competitive digital environment, retaining existing customers has

become significantly more challenging than acquiring new ones. With the abundance of
similar products and services available across platforms, users can effortlessly switch to
competitors when their expectations are not met. Even small factors such as slow app
performance, lack of personalization, or unsatisfactory customer support can lead to user
churn. For businesses, this translates into revenue loss, reduced engagement, and lower
brand loyalty. Most existing analytical systems only focus on descriptive insights -
showing what happened in the past - rather than offering predictive and preventive
intelligence. These systems fail to explain why users are leaving and what actions can be
taken to retain them. To overcome these limitations, there is a growing need for
intelligent, AI-driven churn prediction models that not only forecast potential user drop-
offs but also provide actionable insights into the underlying causes. Retention-AI is
designed to bridge this gap by developing a comprehensive, data-driven platform capable
of predicting and preventing user churn with high accuracy and transparency. The system
integrates three core technologies - Machine Learning (ML), Natural Language
Processing (NLP), and Explainable Artificial Intelligence (XAI) - to analyze both user
behaviour and customer sentiment.

1.1 Overview

In the rapidly expanding digital ecosystem, user engagement and customer

retention have become decisive factors for the success of online platforms. While
acquiring new customers is important, retaining existing users is far more cost-effective
and directly contributes to long-term profitability. Many applications—whether in e-
commerce, streaming, fintech, or social media - face the challenge of user churn, where
customers gradually reduce interaction or uninstall the application. Understanding why
users disengage is therefore critical for sustaining growth and improving user experience.
Traditionally, churn analysis relied on static statistical models and simple
business intelligence dashboards that could describe historical trends but failed to
provide predictive or explanatory insights. Recent research has introduced machine

Department of AI&ML, BIT 2024-25 1

Introduction Chapter 1

learning (ML)–based methods capable of learning behavioural patterns from large user
datasets to predict churn risk. Parallel advances in Natural Language Processing (NLP)
have enabled sentiment extraction from user feedback, reviews, and social-media
discussions, allowing organizations to capture the emotional drivers behind customer
decisions. However, most ML systems remain black-box models - accurate but opaque -
leaving decision-makers uncertain about why particular users are predicted to churn.
To address this limitation, Explainable Artificial Intelligence (XAI) frameworks
such as SHAP and LIME have emerged. These techniques apply principles from
cooperative game theory to highlight the relative contribution of each feature to a
prediction, thereby restoring transparency and trust in AI systems. Recent academic
work, including hybrid churn-prediction architectures combining behavioural analytics
with sentiment analysis, has shown improved accuracy and interpretability.
Building on these developments, the Retention-AI project integrates ML, NLP,
and XAI into a unified platform that predicts churn, interprets its causes, and
recommends personalized retention strategies. By leveraging both structured behavioural
data and unstructured textual feedback, the system not only forecasts potential user drop-
offs but also provides clear, actionable insights for businesses to enhance engagement
and loyalty.

1.2 Objectives

1. To implement Machine Learning (ML) techniques for analyzing user behavior

and detecting churn patterns from structured datasets such as session frequency,
usage duration, and purchase history.
2. To design and develop a Natural Language Processing (NLP) model for
extracting and interpreting user sentiment from unstructured feedback, app
reviews, and customer comments.
3. To build an Explainable Artificial Intelligence (XAI) framework that ensures
transparency in predictions and provides clear, interpretable insights into the
factors influencing churn.
4. To recommend personalized retention strategies that enable businesses to
proactively engage at-risk users and enhance long-term user retention and
satisfaction.

Department of AI&ML, BIT 2024-25 2

Introduction Chapter 1

1.3 Purpose, Scope, and Applicability

1.3.1 Purpose

The primary purpose of this project is to help businesses understand and reduce
user churn by leveraging the capabilities of Artificial Intelligence. Many organizations
collect large volumes of user data but struggle to interpret it effectively to identify users
at risk of leaving. Retention-AI aims to bridge this gap by providing an intelligent,
interpretable, and proactive churn prediction system.

By integrating Machine Learning (ML) for behavioural analytics, Natural

Language Processing (NLP) for sentiment extraction, and Explainable AI (XAI) for
transparent decision-making, the project enhances both the theoretical and practical
understanding of AI-driven retention systems. This framework not only predicts churn
risk but also explains the underlying causes, thereby improving business decision-
making, customer satisfaction, and the overall efficiency of digital platforms.

1.3.2 Scope

The scope of Retention-AI extends across the entire churn prediction and
prevention pipeline. It involves collecting and preprocessing both structured data (user
behavior logs, demographics, usage metrics) and unstructured data (user feedback,
reviews).
Two models are used - Model A (XGBoost) for structured data and Model B (Random
Forest with VADER) for unstructured data. After preprocessing and balancing with
SMOTE, churn probabilities from both models are combined to produce a final churn
score.

The system incorporates Explainable AI (SHAP) to interpret the most influential

factors affecting user churn and provides personalized retention recommendations such as
targeted offers or notifications. The project also includes real-time retraining of models as
new data arrives and a dynamic dashboard for continuous performance monitoring.

While the project effectively predicts and explains churn, its scope is limited by
data availability, the quality of unstructured feedback, and computational resources.

Department of AI&ML, BIT 2024-25 3

Introduction Chapter 1

1.3.3 Applicability

Retention-AI has wide applicability across industries where customer engagement

is critical. It can be deployed in SaaS platforms, mobile applications, e-commerce, OTT
services, telecom, and fintech to monitor user engagement and reduce churn. Indirectly, it
contributes to the field of Artificial Intelligence in Business Intelligence, showcasing how
ML and XAI can be integrated for transparent predictive analytics. The project also
serves as a learning framework for future researchers and developers aiming to build
ethical, explainable, and data-driven AI systems. Through its predictive insights and
interpretability, Retention-AI empowers organizations to take proactive action, reduce
revenue loss, and build stronger, long-lasting customer relationships.

1.4 Organization of Report

The report is structured into nine chapters. Chapter 1 introduces the project along
with its background, objectives, and scope. Chapter 2 reviews related research work and
existing churn prediction systems. Chapter 3 outlines the functional and non-functional
requirements, while Chapter 4 discusses project planning and system architecture.
Chapter 5 presents the module decomposition, interface design, data structures, and
algorithm design. Chapter 6 focuses on the implementation process, and Chapter 7
explains the testing approaches used for system validation. Chapter 8 presents the results,
performance analysis, and user documentation, and Chapter 9 concludes the report with
applications, limitations, and future scope of the project.

Department of AI&ML, BIT 2024-25 4

CHAPTER 2
LITERATURE SURVEY
Understanding existing research is essential for identifying the gaps, challenges,
and opportunities in developing effective user churn prediction and retention models.
This chapter reviews previous studies, techniques, and methodologies that have been
applied in the field of customer retention and churn analysis. It explores the evolution
from traditional statistical approaches—such as logistic regression and decision trees—to
advanced machine learning, deep learning, and hybrid AI models that combine
behavioural analytics with sentiment analysis. The objective of this review is to evaluate
the strengths and limitations of existing systems, assess how recent innovations have
enhanced prediction accuracy and interpretability, and identify research gaps that
Retention-AI aims to address. In particular, this study emphasizes the integration of
Natural Language Processing (NLP) for understanding user sentiment, and Explainable
AI (XAI) for making model predictions transparent. By synthesizing prior research, this
chapter establishes the foundation for designing an AI-driven, explainable churn
prediction framework that delivers both predictive performance and business
interpretability.

2.1 Introduction
In any research-oriented project, reviewing existing literature is crucial for
understanding the current state of the field and identifying areas that require
improvement. For this project, the focus lies on user churn prediction and customer
retention, where businesses strive to anticipate and prevent user drop-offs using Artificial
Intelligence (AI) techniques. This chapter presents a detailed review of previous studies,
methodologies, and technologies applied in churn prediction, sentiment analysis, and
explainable AI (XAI). The goal of this literature survey is to examine the strengths and
limitations of existing systems, understand how recent advancements have improved
prediction accuracy and interpretability, and identify research gaps that led to the
development of the Retention-AI framework. By integrating findings from past studies,
this chapter provides the theoretical foundation for designing a system that combines
ML, NLP, and XAI to deliver transparent, data-driven churn insights and personalized
retention strategies.

Department of AI&ML, BIT 2024-25 5

Literature Survey Chapter 2

2.2 Summary of Papers

[1] Usman Gani Joy, Kazi Ekramul Hoque, Mohammed Nazim Uddin, Linkon
Chowdhury, and Seung-Bo Park (2024), “A Big Data-Driven Hybrid Model for
Enhancing Streaming Service Customer Retention Through Churn Prediction
Integrated With Explainable AI,” IEEE Access, 2024.

Objective: To develop a hybrid AI model that enhances customer retention by

predicting churn in streaming platforms using large - scale behavioural data
integrated with explainable AI.

Methodology: The study combines traditional machine learning models with

Explainable Boosting Machine (EBM) and SHAP (SHapley Additive exPlanations)
techniques to analyze user behaviour patterns. The integration of SHAP enables
better feature attribution, improving the interpretability of churn predictions.

Key Findings: The model effectively identifies at-risk users and highlights
behavioural factors contributing to churn, allowing targeted retention strategies. The
use of explainable AI improves transparency and trust in model outputs.

Limitations: While interpretability is significantly enhanced, real-time churn

prediction remains challenging due to the computational overhead associated with
large-scale data processing.

[2] Upma Singh, Sandeep Singh, Tripti Rathee, and Manav Vaish (2024), “Customer
Retention Modelling over the OTT Platform using Machine Learning,” Indian
Journal of Science and Technology, 2024.

Objective: To apply machine learning techniques for predicting customer retention

in OTT (Over-The-Top) platforms based on user demographics, subscription details,
and engagement patterns.

Methodology: The study employs multiple ML models, including classification

algorithms, to analyze customer data such as viewing frequency, subscription

Department of AI&ML, BIT 2024-25 6

Literature Survey Chapter 2

duration, and content preferences. These models are trained to identify factors
influencing user retention and predict potential churners.

Key Findings: The models effectively predict customer churn and retention rates,
enabling OTT platforms to design targeted promotional strategies and improve user
satisfaction.

Limitations: The approach demands high computational resources and may be

difficult for smaller OTT platforms to adopt due to infrastructure constraints.

[3] Shuzlina Abdul-Rahman, Muhamad Faidi Akif Md Ali, Azuraliza Abu Bakar,
and Sofianita Mutalib (2024), “Enhancing Churn Forecasting with Sentiment
Analysis of Steam Reviews,” Social Network Analysis and Mining, Springer,
2024.

Objective: To enhance churn forecasting accuracy by integrating sentiment scores

derived from user-generated reviews on the Steam gaming platform.

Methodology: The study incorporates Vector Autoregression (VAR) and Support

Vector Machine (SVM) models, combining structured user activity data with
unstructured textual feedback. Sentiment polarity (positive or negative) extracted
from Steam reviews is used as an additional predictive feature for churn analysis.

Key Findings: The integration of sentiment data improves the precision of churn
forecasts and provides deeper behavioural insights into player engagement and
dissatisfaction.

Limitations: The model focuses only on sentiment polarity, lacking emotion, intent,
and contextual understanding. Additionally, Steam reviews may not accurately
represent the overall player experience, limiting the generalizability of results.

[4] Lucio Ciabattoni, Marco Maiolatesi, Martina Mancinelli, Maria Di Tillo,

Riccardo Fiandra, et al. (2024), “Explainable AI Based Approach for Broadband
Customers’ Churn Prediction,” IEEE, 2024.

Department of AI&ML, BIT 2024-25 7

Literature Survey Chapter 2

Objective: To develop an explainable AI-based framework for predicting customer

churn in broadband services using operational metrics.

Methodology: The study employs a fuzzy logic approach integrated with explainable
AI principles to analyze broadband usage patterns, network performance indicators,
and customer service interactions. The model generates churn predictions while
maintaining transparency through rule-based explanations.

Key Findings: The proposed system effectively identifies churn-prone customers

and provides actionable insights that can guide service providers in decision-making
and retention strategy design. The use of fuzzy logic enhances interpretability for
non-technical stakeholders.

Limitations: Although interpretability is improved, the fuzzy logic-based model

exhibits lower predictive accuracy compared to modern deep learning approaches,
especially on large and complex datasets.

[5] Kapil Arora, M. Lalitha, Poonam, Hemalatha Yadav J., and Biswo Ranjan
Mishra (2024), “Machine Learning for Customer Retention in E-Commerce
Healthcare Startups,” South Eastern European Journal of Public Health
(SEEJPH), TensorGate, 2024.

Objective: To apply machine learning techniques for predicting customer churn and
enhancing user retention in healthcare-focused e-commerce startups.

Methodology: The study utilizes CatBoost algorithms along with customer

segmentation and recommendation models to analyze purchase frequency, browsing
behaviour, and user engagement data.

Key Findings: The model successfully improves retention rates by identifying high-
risk customers and providing data-driven recommendations for personalized
engagement strategies.

Limitations: The research is restricted to the healthcare e-commerce domain and

faces challenges related to data privacy and scalability due to high computational

Department of AI&ML, BIT 2024-25 8

Literature Survey Chapter 2

costs and limited generalizability to other industries.

[6] Tamer Ahmed Ibrahim Abou El-Fotouh and Musliudeen Toheeb Akanbi (2024),
“Impact of Predictive Analytics and Machine Learning on Customer Retention and
Loyalty in Service-Oriented Businesses,” International Journal of Business
Intelligence and Big Data Analytics, 2024.

Objective: To examine the role of predictive analytics and machine learning in

enhancing customer retention and loyalty across various service-oriented industries
such as telecommunications and banking.

Methodology: The study applies decision tree classifiers and ensemble learning
methods to analyze customer transaction data and behavioural patterns. These
techniques are used to identify churn indicators and predict customer loyalty levels
across different service sectors.

Key Findings: Predictive analytics, when combined with ML models, significantly

improves customer retention by enabling proactive interventions and personalized
engagement strategies. The study highlights the versatility of ML across multi-sector
service domains.

Limitations: The research lacks real-world case studies validating model

performance in diverse service contexts. Additionally, it faces challenges related to
computational cost, data bias, and the absence of cross-industry implementation
testing.

[7] Ziru Liu, Shuchang Liu, Bin Yang, Zhenghai Xue, Qingpeng Cai, Xiangyu Zhao,
Zijian Zhang, Lantao Hu, Han Li, and Peng Jiang (2024), “Modelling User
Retention through Generative Flow Networks,” Proceedings of the ACM
SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2024.

Objective: To develop a generative modelling approach for improving user retention

prediction and engagement optimization using Generative Flow Networks (GFNs).

Department of AI&ML, BIT 2024-25 9

Literature Survey Chapter 2

Methodology: The study introduces GFN4Retention, a generative flow network that

models user retention signals and backpropagates them through the system to
optimize engagement strategies. The framework integrates reinforcement learning
principles to generate retention-oriented user actions and evaluate long-term
behavioural outcomes.

Key Findings: The proposed GFN4Retention model significantly enhances user

engagement and retention forecasting by effectively learning complex behavioral
dependencies in sequential user data.

Limitations: Despite strong predictive performance, the model suffers from limited
interpretability, making it difficult for recommender systems to understand why users
churn. This lack of transparency restricts its applicability for actionable business
decision-making.

[8] Kapil Arora, M. Lalitha, Poonam, Hemalatha Yadav J., and Dr. Biswo Ranjan
Mishra (2024), “Machine Learning for Customer Retention in E-Commerce
Healthcare Startups,” South Eastern European Journal of Public Health
(SEEJPH), 2024.

Objective: To design machine learning-based models for personalized churn

prediction in healthcare-focused e-commerce startups by leveraging recommendation
systems and customer segmentation techniques.

Methodology: The study integrates ML algorithms with customer segmentation and

recommendation systems to identify at-risk users and generate targeted offers. The
models analyze behavioural and transactional data such as purchase frequency,
browsing habits, and customer feedback.

Key Findings: The approach enhances retention by predicting customer churn

patterns and enabling personalized engagement strategies that strengthen customer
loyalty in healthcare e-commerce. Overall, it emphasizes the practical potential of
ML-based churn prediction in improving business profitability and customer
satisfaction.

Department of AI&ML, BIT 2024-25 10

Literature Survey Chapter 2

Limitations: The study lacks real-world implementation, dataset transparency, and

detailed performance evaluation. It also does not adequately address scalability,
privacy, or explainability issues, which are crucial for practical deployment in
healthcare e-commerce environments.

[9] Suchita Sharma and Nishith Desai (2023), “Identifying Customer Churn Patterns
Using Machine Learning Predictive Analysis,” IEEE SMART GENCON, 2023.

Objective: To identify customer churn patterns by applying machine learning

techniques on transactional and behavioural data to achieve high prediction accuracy.

Methodology: The study employs supervised learning models trained on numerical

and categorical data representing user demographics, purchase frequency, and
account activity. Various ML algorithms are evaluated to determine the most
effective approach for churn classification.

Key Findings: The research demonstrates that machine learning can accurately
predict customer churn, providing businesses with early insights to design data-
driven retention strategies.

Limitations: Despite achieving high predictive accuracy, it excludes unstructured

data sources such as customer feedback, reviews, chat logs, and social media
interactions, which often contain rich contextual and emotional insights into user
behavior. As a result, the system fails to capture sentiment-driven churn factors, such
as dissatisfaction expressed in text form or changing customer opinions over time.

[10] Sabreen Abulhaija, Shyma Hattab, Ahmad Abdeen, and Wael Etaiwi (2022),
“Predicting Mobile Apps Performance using Machine Learning,” Journal of
System and Management Sciences, 2022.

Objective: To evaluate and predict the performance of mobile applications using

machine learning models trained on large-scale app store data.

Methodology: The study analyzes data from the Google Play Store, applying
algorithms such as Random Forest to predict app performance based on factors

Department of AI&ML, BIT 2024-25 11

Literature Survey Chapter 2

including user ratings, downloads, and update frequency. The research focuses on
improving prediction generalizability through extensive feature extraction and dataset
expansion.

Key Findings: The model achieves strong performance in predicting app success,
demonstrating that integrating multiple app-related features enhances prediction
accuracy and robustness.

Limitations: Although the study uses a significantly large dataset of 2.24 million
records, it remains limited by feature diversity and a lack of qualitative insights such
as user sentiment or contextual engagement data.

[11] Manzura Jorayeva, Akhan Akbulut, Cagatay Catal, and Alok Mishra (2022),
“Machine Learning-Based Software Defect Prediction for Mobile Applications,”
Sensors, 2022.

Objective: To provide a comprehensive review of machine learning-based

approaches used for predicting software defects in mobile applications and to
evaluate their performance based on object-oriented metrics.

Methodology: The study systematically reviews research papers focused on

software defect prediction (SDP) using ML models such as Decision Trees, Random
Forest, and Neural Networks. It analyzes how object-oriented design metrics (e.g.,
coupling, inheritance, complexity) influence defect prediction accuracy.

Key Findings: The research offers a consolidated perspective on the effectiveness of

ML-based defect prediction models for mobile applications. It identifies critical
metrics that significantly impact defect rates, providing valuable insights for
improving software quality and development efficiency.

Limitations: The study highlights a lack of unified frameworks for evaluating fault
prediction models across mobile platforms. It also reports issues such as data
imbalance, high computational cost, and the absence of real-world validation,
limiting the practical applicability of these models in large-scale mobile app
development.

Department of AI&ML, BIT 2024-25 12

Literature Survey Chapter 2

[12] Carson K. Leung, Adam G. M. Pazdor, and Joglas Souza (2021), “Explainable
Artificial Intelligence for Data Science on Customer Churn,” IEEE International
Conference on Data Science and Advanced Analytics (DSAA), 2021.

Objective: To develop an Explainable Artificial Intelligence (XAI) framework that

interprets and explains customer churn predictions generated by machine learning
models, particularly Random Forest classifiers.

Methodology: The study applies Random Forest algorithms to real-world churn

datasets and integrates XAI visualization tools to interpret model outcomes. The
framework focuses on highlighting key factors influencing customer churn, thereby
improving transparency and understanding of ML decision-making processes.

Key Findings: The proposed framework successfully enhances the interpretability of

churn prediction results, allowing business analysts to understand which variables
most strongly contribute to customer churn. It demonstrates the potential of XAI in
bridging the gap between model performance and human interpretability.

Limitations: Although the model provides strong interpretive insights, it lacks

targeted retention action mechanisms that could translate explanations into business
strategies. Additionally, its reliance on a single dataset reduces the generalizability of
the approach to different industries or large-scale multi-source environments.

[13] Zeynep Hilal Kilimci, Hasan Yörük, and Selim Akyokus (2020), “Sentiment
Analysis Based Churn Prediction in Mobile Games using Word Embedding Models
and Deep Learning Algorithms,” IEEE International Conference on Machine
Learning,2022.
Objective: To predict mobile game churn by applying deep learning models using
sentiment extracted from user reviews represented through advanced word
embedding models.

Methodology: The study employs CNN, LSTM, and RNN architectures to process
user reviews, utilizing Word2Vec, GloVe, and FastText embeddings to represent
semantic relationships between words. Sentiment scores are derived from these

Department of AI&ML, BIT 2024-25 13

Literature Survey Chapter 2

embeddings and used as predictive features for churn classification.

Key Findings: The deep learning models outperform traditional churn prediction
approaches based solely on behavioral or demographic data. Integrating sentiment
representations significantly improves accuracy, emphasizing the emotional tone.
Limitations: The model focuses only on textual data, excluding behavioral and
demographic factors that could improve prediction reliability. Moreover, its
application is restricted to the gaming domain, limiting scalability to other industries.

[14] Phongsatorn Amornvetchayakul and Naragain Phumchusri (2020), “Customer

Churn Prediction for a Software - as - a - Service Inventory Management Software
Company,” IEEE International Conference on Industrial Engineering and
Engineering Management (IEEM), 2020.
Objective: To predict customer churn in a SaaS-based inventory management
company using traditional machine learning algorithms and to identify key business
factors influencing churn.

Methodology: The study implements multiple supervised learning models including

Logistic Regression, Support Vector Machine (SVM), Decision Tree, and Random
Forest. It analyzes customer usage patterns, subscription data, and business attributes
to determine the most significant predictors of churn.

Key Findings: Random Forest and Decision Tree models demonstrate high
predictive accuracy, revealing that usage frequency, contract length, and service
satisfaction are key churn determinants. The study highlights the effectiveness of
traditional ML models for churn prediction in SaaS contexts.

Limitations: The research lacks domain-specific interpretability mechanisms,

limiting the ability to translate predictions into actionable strategies. Additionally, it
does not incorporate advanced AI techniques such as XAI or NLP for deeper insight
generation.

[15] Carl Yang, Xiaolin Shi, Jie Luo, and Jiawei Han (2019), “I Know You’ll Be
Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social

Department of AI&ML, BIT 2024-25 14

Literature Survey Chapter 2

Application,”2019.
Objective: To develop an interpretable churn prediction framework that identifies
churn-prone users in mobile social applications by analyzing behavioral patterns of
new users.
Methodology: The study proposes ClusChurn, a two-step framework combining
LSTM networks and attention mechanisms with k-means clustering to group new
users based on engagement behavior. It is applied on large-scale Snapchat user data
to predict churn and interpret user retention patterns.
Key Findings: The model effectively captures sequential user behavior and improves
churn prediction accuracy for new users. The clustering component enhances
interpretability by revealing behavioral segments associated with different churn
risks.
Limitations: Despite offering improved interpretability compared to traditional
models, ClusChurn still lacks comprehensive explainability and actionable insights
for targeted retention strategies. Its reliance on a single platform dataset also limits
generalizability to other applications.

2.3 Drawbacks of Existing System

Despite significant advancements in the fields of customer churn prediction and retention
analytics, existing systems still face several limitations that restrict their effectiveness,
scalability, and practical adoption.

1. Limited Integration of Structured and Unstructured Data: Most existing

churn prediction models rely heavily on structured data such as user
demographics, subscription details, and transaction history. However, they often
ignore unstructured data sources like user reviews, feedback, and social media
interactions that contain valuable contextual and emotional insights about
customer dissatisfaction and intent to churn.

2. Lack of Explainability in Black-Box Models: Many studies utilize complex

machine learning and deep learning algorithms that provide high accuracy but
function as black boxes, offering little to no interpretability. Businesses often find
it difficult to understand why a particular user is predicted to churn, making it

Department of AI&ML, BIT 2024-25 15

Literature Survey Chapter 2

harder to design effective and targeted retention strategies.

3. Absence of Personalized Retention Actions: Several models stop at predicting

churn probability without suggesting personalized interventions or actionable
insights. As a result, the output remains purely analytical and fails to translate into
practical business actions that could improve customer engagement or reduce
churn.

4. Data Imbalance and Limited Real-Time Learning: Many systems suffer from
class imbalance, where churn instances are significantly fewer than non-churn
instances. This leads to biased predictions and lower sensitivity toward churn-
prone users. Additionally, most models are static and do not retrain dynamically
with new incoming data, making them unsuitable for real-time applications.

5. Computational Complexity and Scalability Issues: Deep learning and hybrid

models, while powerful, often come with high computational costs. This limits
their deployment on platforms with limited hardware resources, especially in
small-scale or startup environments that cannot afford high-end computing
infrastructure.

6. Lack of Domain Generalization: Several existing approaches are designed for

specific sectors such as telecom, OTT, or e-commerce. These models exhibit poor
cross-domain adaptability, as their features and parameters are tailored to one
industry, reducing their generalizability to other domains.

7. Limited Focus on Customer Sentiment and Behavior Dynamics: Although

some studies have begun incorporating sentiment analysis, most still rely on
simple sentiment polarity (positive/negative) rather than a deeper understanding of
emotion, intent, and context. This reduces the system’s ability to interpret nuanced
human feedback or identify early warning signals of churn.

8. Lack of Comprehensive Evaluation Metrics and Case Studies: Several models

are validated on small or single datasets, with limited real-world case studies or
performance benchmarks. As a result, their robustness and effectiveness in diverse
operational conditions remain uncertain.

Department of AI&ML, BIT 2024-25 16

Literature Survey Chapter 2

2.4 Problem Statement

―To develop an AI-driven and explainable churn prediction system that provides
proactive measures to reduce churn and improve customer engagement across various
platforms by implementing machine learning-based behavioral analytics, NLP-powered
sentiment analysis, and explainable AI (XAI) for transparent decision-making.‖

2.5 Proposed Solution

The proposed system, Retention-AI, aims to bridge the gaps observed in existing
churn prediction frameworks by integrating Machine Learning (ML), Natural Language
Processing (NLP), and Explainable AI (XAI) into a unified and interpretable customer
retention platform. The system focuses on providing both predictive accuracy and
actionable insights, enabling organizations to not only identify potential churners but also
understand the reasons behind churn and take proactive measures. FIN-NET employs
Artificial Neural Network (Multi layered Perceptron) for capturing complex patterns and
interdependencies.

1. Dual-Model Architecture: The system utilizes two complementary models to

process different types of data:

 Model A: Employs XGBoost to analyze structured data such as user

demographics, behavioral metrics, and engagement statistics.
 Model B: Utilizes Random Forest integrated with VADER (Valence
Aware Dictionary and sEntiment Reasoner) to process unstructured data
like customer feedback and reviews.

2. Both models generate independent churn predictions, which are later combined to
produce a final aggregated churn score through averaging.
3. Balanced and Preprocessed Data Pipeline: The system uses data preprocessing
techniques to clean and normalize inputs. To address the class imbalance problem,
it applies SMOTE (Synthetic Minority Oversampling Technique) to balance churn
and non-churn samples, ensuring fair model training and better prediction
reliability.

Department of AI&ML, BIT 2024-25 17

Literature Survey Chapter 2

4. Explainable AI (XAI) Integration: To overcome the ―black-box‖ nature of ML

models, SHAP (SHapley Additive exPlanations) is integrated into the system.
SHAP values highlight the most influential features contributing to each
individual’s churn score, offering transparency and trust in model predictions.
This helps decision-makers understand why a user is likely to churn.
5. Real-Time Retraining and Adaptability: The models are designed for dynamic
retraining -

 Model A retrains after every 20 new structured records, and Model B

retrains after every 10 new unstructured inputs. This ensures that the
system continuously learns from new user behavior and feedback,
maintaining high relevance in real-world applications.

6. Interactive Dashboard and Joblib Integration: The use of the Joblib library
ensures that while new computations are being processed, the dashboard remains
responsive and displays cached results to avoid user downtime.
7. Personalized Retention Strategies: Based on the churn score and SHAP
explanations, the system recommends personalized retention actions such as
targeted discounts, promotional offers, or re-engagement messages. These insights
are sent to the business owners, allowing them to make data-driven decisions to
retain high-risk customers.

Department of AI&ML, BIT 2024-25 18

CHAPTER 3

REQUIREMENT ENGINEERING
Requirement Engineering is a crucial phase in the software development life cycle
that defines what the system should accomplish and how it should perform under
specified conditions. This chapter focuses on identifying and analyzing user needs,
system expectations, and transforming them into clearly defined functional and non-
functional requirements for the proposed system, Retention-AI. The objective of this
phase is to ensure that the churn prediction and retention system is technically feasible,
user-centric, and aligned with real-world business needs related to customer engagement
and retention. It also details the software and hardware tools used in the development
process, the conceptual and analytical modeling, and various UML diagrams such as use
case, sequence, activity, and state chart diagrams. This chapter forms the foundation for
the subsequent system design and implementation phases, ensuring that every functional
requirement is systematically aligned.

3.1 Software and Hardware Tools used

3.1.1 Hardware Requirements

The development and execution of Retention-AI require a computing environment

capable of efficiently handling machine learning model training, text processing, and real-
time data visualization. The following hardware specifications are recommended to
ensure smooth operation and optimal system performance:

1. Processor
A mid-range processor is required to ensure smooth execution of machine
learning and natural language processing tasks. It supports efficient data
preprocessing, model training, and SHAP-based interpretability computations.
The processor also maintains responsiveness while handling simultaneous
operations such as structured and unstructured data analysis.
2. RAM
Adequate memory is essential for training ML models, processing large datasets,

Department of AI&ML, BIT 2024-25 19

Requirement Engineering Chapter 3

and executing multiple retraining cycles. Higher RAM ensures smooth

functioning of development environments such as Jupyter Notebook or VS Code,
as well as the Streamlit dashboard for real-time churn visualization.
3. Storage
Ample storage is required for maintaining datasets, preprocessed files, trained
model weights, SHAP explanation outputs, and dashboard assets. SSDs provide
faster read/write speeds compared to HDDs, significantly reducing model load
times and improving data access efficiency.
4. GPU
While not mandatory, a dedicated GPU can accelerate model training and
inference - particularly during XGBoost or Random Forest computations on large
datasets and NLP operations like VADER-based sentiment analysis. GPU support
is beneficial for achieving faster retraining and improved system responsiveness.

3.1.2 Software Tools Used

1. Programming Language: Python serves as the core development language due

to its extensive support for data analysis, machine learning, and natural language
processing. It also offers excellent integration with libraries used for explainable
AI and dashboard development.
2. Frameworks and Libraries: Essential Python libraries include Pandas, NumPy,
and Scikit-learn for data preprocessing and model building; XGBoost and
RandomForestClassifier for churn prediction; NLTK and VADER for sentiment
analysis; and SHAP for model explainability. Visualization libraries such as
Matplotlib and Seaborn are used for performance evaluation and plotting feature
importance. Joblib is used to cache trained models and intermediate computations.
3. Visualization and Dashboard Tools: Streamlit is used to build the interactive
user dashboard that displays churn predictions, SHAP interpretability plots, and
personalized retention recommendations. It enables real-time user interaction with
the predictive system.
4. Data Management: The system utilizes structured and unstructured datasets
stored in CSV or JSON formats. These formats ensure easy compatibility with

Department of AI&ML, BIT 2024-25 20

Requirement Engineering Chapter 3

Python libraries and enable seamless data preprocessing and model input
handling.
5. Version Control System: is used for local version tracking, while GitHub serves
as the remote repository for collaborative development, version management, and
code maintenance.

3.2 Conceptual/ Analysis Modeling

To effectively design and implement a software solution, it is crucial to first

understand the problem domain and create a conceptual model of the system. This model
outlines the key operations the system will perform and defines the logical flow and
sequence of these operations. Conceptual modeling helps visualize system behavior, data
interactions, and user processes, forming the foundation for further development.

3.2.1 Use case diagram

Department of AI&ML, BIT 2024-25 21

Requirement Engineering Chapter 3

Fig

This diagram illustrates the working of the Retention-AI System, which is designed to
predict and prevent customer churn using artificial intelligence and data analytics. The
use case represents how different actors interact with the system to achieve the goal of
improving user retention.

Key Actors

 User: Regular app users whose engagement, behavior, and feedback are
continuously monitored to assess churn risk.
 Admin : Business stakeholders who access the system’s dashboard, interpret
churn insights, and implement retention strategies based on AI recommendations.
 AI System:The intelligent computational engine that processes user data, predicts
churn risk, and provides explainable insights to the admin.

System Workflow

The Retention-AI workflow consists of multiple interconnected processes:

1. User Interaction Layer: Users interact with the app by performing regular
actions (such as browsing, purchasing, or engaging with content) and submitting
feedback or reviews. This generates both structured (user activity logs) and
unstructured (text feedback) data that feed into the AI system.
2. AI Analysis Engine: The AI system performs three major analyses:

 Analyze User Behavior: Tracks activity frequency, engagement level, and

usage duration to detect patterns of disengagement.
 Analyze Sentiment: Uses NLP-based VADER analysis to interpret user
feedback and determine satisfaction or dissatisfaction.
 Predict Churn Risk: Combines behavioral and sentiment-based insights
using XGBoost and Random Forest models to estimate churn probability.

Department of AI&ML, BIT 2024-25 22

Requirement Engineering Chapter 3

3. Strategic Output: Based on the aggregated churn score, the system recommends
personalized retention strategies such as targeted offers, in-app notifications, or
content suggestions for at-risk users.
4. Business Intelligence: Admins can:

View Churn Dashboard: Monitor churn statistics, predictions, and segment-level

trends.

Generate Explainable Insights: Use SHAP-based explainability to understand the

top features contributing to churn and take corrective actions.

The purpose of this system is to create a proactive retention ecosystem where AI

continuously monitors user engagement, identifies early churn indicators, and
recommends real-time, data-driven retention actions.

3.2.2 Sequence diagram

Department of AI&ML, BIT 2024-25 23

Requirement Engineering Chapter 3

The Sequence Diagram provides a detailed technical overview of how the

Retention-AI System operates across its interconnected components to predict customer
churn and recommend personalized retention strategies. It captures the dynamic
interaction between users, system modules, and administrative interfaces in a time-
sequenced manner.

System Components

1. User - The end user of the application who interacts with the platform and
provides behavioral and feedback data.
2. Retention-AI System - The central orchestration unit that manages
communication among AI modules.
3. Behavior Analyzer (ML) - The machine learning component responsible for
analyzing user engagement patterns and activity data.
4. Sentiment Analyzer (NLP) -The natural language processing module that
extracts sentiment from user feedback using models such as VADER.
5. Churn Predictor - The machine learning model that aggregates structured and
unstructured data to predict churn probability.
6. XAI Engine - The Explainable AI layer utilizing SHAP values to interpret model
outputs and highlight key churn factors.
7. Admin / Data Analyst - The business user who views predictions, explanations,
and personalized retention recommendations on the dashboard.

Interaction Flow

Phase 1: Data Collection

In the initial phase, the User interacts with the application by performing regular
activities such as browsing, purchasing, or engaging with app features. These interactions
generate structured behavioral data in the form of activity logs, frequency patterns, and
engagement metrics. Simultaneously, the user may provide textual feedback or reviews.

Department of AI&ML, BIT 2024-25 24

Requirement Engineering Chapter 3

The Retention-AI System captures this data in real time and forwards it to the Behavior
Analyzer (ML).

Phase 2: Sentiment Analysis

In this phase, the Retention-AI System sends the collected textual feedback to the
Sentiment Analyzer (NLP). The NLP module, implemented using the VADER sentiment
analysis model, evaluates the polarity of the feedback-categorizing it as positive,
negative, or neutral. The system interprets this sentiment data as a measure of user
satisfaction, allowing it to complement the behavioral data collected earlier.

Phase 3: Churn Prediction

Once both behavioral and sentiment data have been analyzed, the Retention-AI
System consolidates the results and forwards them to the Churn Predictor. This model
combines insights from both data types to calculate a churn probability score for each
user. The predictor determines the likelihood that a user will discontinue app usage based
on past engagement behavior, satisfaction levels, and historical patterns.

Phase 4: Explainability

To ensure transparency, the Churn Predictor interacts with the XAI Engine, which
provides an interpretable explanation of the model’s decision-making process. Using
SHAP (SHapley Additive exPlanations), the XAI Engine identifies key features such as
session frequency, review sentiment, or inactivity duration that contribute most to each
user’s churn risk. These explainable outputs enable business users to understand the
reasoning behind churn predictions, converting complex model outputs into actionable
business intelligence.

Phase 5: Results Distribution

After obtaining churn scores and their corresponding explanations, the Retention-
AI System compiles the results and displays them through an interactive dashboard. The
Admin or Data Analyst accesses this dashboard to view churn rates, customer segments,

Department of AI&ML, BIT 2024-25 25

Requirement Engineering Chapter 3

and SHAP-based feature attributions. These visualizations help decision-makers track

user engagement health, monitor churn trends, and focus on high-risk segments.

Phase 6: Retention Action

In the final phase, the Admin can request personalized retention strategies for
users identified as high-risk. The Retention-AI System then generates recommendations
such as promotional offers, loyalty rewards, or personalized engagement messages. These
strategies are designed to re-engage users and enhance their experience, ultimately
reducing churn. The system can also deliver these recommendations directly to users
through notifications or app-based messages, closing the loop between prediction and
proactive retention action.

3.2.3 Activity Diagram

The activity diagram illustrates the end-to-end operational flow of the

Retention-AI System, depicting how data moves through various stages—from user
interactions to churn prediction and strategy recommendation. It captures the sequence of

Department of AI&ML, BIT 2024-25 26

Requirement Engineering Chapter 3

processes executed by the system’s components and highlights decision-making points

where human intervention shapes the final action. This diagram provides a clear view of
the dynamic behavior and logical progression of activities in the system.

Description of Activities

1. User Interaction: The workflow begins when a user engages with the application
by performing actions such as browsing, ordering, or using specific app features.
These interactions generate structured behavioral data like session counts, feature
usage, and time spent in the app. Additionally, users may provide written
feedback or reviews, contributing unstructured textual data that reflects their
experience and satisfaction.
2. Data Collection : Once user interaction occurs, the Retention-AI System initiates
its automated data collection pipeline. It gathers structured behavioral metrics and
unstructured sentiment data simultaneously. These datasets form the foundation
for the system’s analytical process, capturing both quantitative and qualitative
aspects of user engagement.
3. Behavioral Analysis (ML Model): The collected behavioral data is sent to the
Machine Learning module for processing. This model analyzes engagement
frequency, activity duration, and interaction diversity to identify key patterns and
anomalies that may signal a decline in user activity. The analysis output includes
summarized engagement trends and behavioral risk indicators.
4. Sentiment Analysis (NLP Model): Parallelly, the unstructured textual feedback
is processed by the Natural Language Processing module using the VADER
sentiment analysis model. The system interprets the tone and polarity of the
feedback—categorizing it as positive, negative, or neutral—and extracts
emotional cues and dissatisfaction points that influence user behavior. This
sentiment information complements behavioral insights to provide a holistic user
understanding.
5. Churn Prediction: Once both behavioral and sentiment insights are derived, the
system aggregates them and forwards the combined dataset to the Churn
Predictor. This model, based on algorithms like XGBoost and Random Forest,
computes a churn probability score for each user. The churn score quantifies the

Department of AI&ML, BIT 2024-25 27

Requirement Engineering Chapter 3

likelihood that a user may stop using the app, helping businesses identify at-risk
users proactively.
6. Explainability (XAI Engine): Following churn prediction, the system engages
the Explainable AI (XAI) Engine, which uses SHAP (SHapley Additive
exPlanations) to interpret the model’s output. The engine identifies key
contributing factors—such as low session frequency, negative sentiment, or
feature inactivity—and visualizes their impact on the churn probability. This
ensures that the system remains transparent and interpretable for business
stakeholders.
7. Results Processing and Visualization: After obtaining churn scores and SHAP
explanations, the Retention-AI System consolidates the results. These outputs are
displayed through an interactive dashboard where the Admin or Data Analyst
can monitor churn rates, visualize user risk segments, and review feature-level
insights. This stage enables data-driven decision-making by providing actionable
intelligence in a user-friendly format.
8. Generate Personalized Retention Strategy: When retention strategies are
requested, the AI system generates tailored responses based on each user’s churn
drivers. These strategies may include personalized discounts, reward offers, re-
engagement messages, or feature-based recommendations. The system aligns the
recommendations with the factors identified by SHAP to ensure that interventions
are targeted and effective.
9. Deliver Retention Action to User: The final step involves delivering these
personalized strategies to the respective users through notifications or in-app
messages. This closes the feedback loop, where AI-driven insights translate into
proactive business actions aimed at improving user satisfaction and retention.

Summary:
The Retention-AI activity diagram represents a closed-loop system that continuously
learns from user behavior. It integrates automated data analysis, explainable predictions,
and human decision-making to enhance customer retention. By combining Machine
Learning, Natural Language Processing, and Explainable AI, the system ensures a
transparent, adaptive, and personalized approach to mitigating user churn.

Department of AI&ML, BIT 2024-25 28

Requirement Engineering Chapter 3

3.2.4 Data Flow Diagram

Fig

The Data Flow Diagram (DFD) illustrates the complete flow of data within the Retention-
AI System, showing how user interactions and feedback are transformed into intelligent

Department of AI&ML, BIT 2024-25 29

Requirement Engineering Chapter 3

retention strategies. It highlights each stage of data processing—from collection to

prediction and explainability—demonstrating how AI and human oversight combine to
reduce customer churn.

Stage 1.0: Collect Data

The process begins when users interact with the application, generating behavioral and
feedback data. The system captures app usage patterns such as session time, frequency,
and feature interactions, along with textual feedback like reviews and complaints. This
data is categorized and stored in two distinct repositories: the User Behavior Database for
structured logs and the Feedback Database for unstructured text data. This bifurcation
ensures efficient processing and easy retrieval during later stages of analysis.

Stage 2.0: Analyze Behavior (ML)

In this stage, behavioral data from the User Behavior Database is analyzed using Machine
Learning algorithms. Metrics such as user activity frequency, feature adoption, and
engagement duration are studied to identify declining trends or inactivity patterns. The
output of this stage is a detailed behavior analysis report that helps the system understand
user engagement levels and detect early signs of churn risk.

Stage 3.0: Analyze Sentiment (NLP)

Simultaneously, the system processes unstructured feedback through Natural Language

Processing (NLP) techniques. Using the VADER sentiment analysis model, the system
evaluates the emotional tone of user reviews, identifies pain points, and classifies
sentiments as positive, negative, or neutral. These sentiment insights provide qualitative
depth to the behavioral metrics, revealing user emotions and satisfaction levels that
contribute to churn prediction.

Stage 4.0: Predict Churn

The Churn Prediction stage serves as the integration point where results from the ML and
NLP modules converge. The combined behavioral insights and sentiment scores are input
into predictive models like XGBoost and Random Forest to calculate a churn probability

Department of AI&ML, BIT 2024-25 30

Requirement Engineering Chapter 3

for each user. The output is a churn risk score, indicating how likely a user is to
discontinue app usage. This predictive intelligence allows proactive identification of
high-risk customers.

Stage 5.0: Explain Churn (XAI)

To ensure transparency, the Explainable AI (XAI) Engine interprets the results of the
churn prediction model. Using SHAP (SHapley Additive exPlanations), it highlights key
contributing features such as reduced activity, low engagement, or negative sentiment
toward specific features. These interpretable results are then displayed to the admin,
providing clear reasoning behind each prediction. This interpretability builds trust and
supports data-driven decision-making.

Stage 6.0: Generate Retention Strategy

If a retention request is approved, the system proceeds to generate personalized strategies

tailored to individual users. These strategies are based on churn causes identified by the
XAI engine—such as offering discounts for inactive users, sending engagement
reminders, or promoting underused app features. The generated retention actions are then
delivered to users through the application interface, closing the feedback loop.

This structured data flow allows the Retention-AI System to operate as a transparent,
adaptive, and intelligent customer retention solution that blends machine learning
precision with human insight to reduce churn and enhance user loyalty.

3.3 Software Requirement Specification

This section outlines the functional and non-functional requirements of the system
developed for the project titled Retention-AI. The primary objective of this system is to
help businesses identify potential customer churn and implement proactive strategies to
retain users. It processes both structured data (user behavior metrics) and unstructured
data (customer feedback) using advanced Artificial Intelligence techniques such as
Machine Learning (ML), Natural Language Processing (NLP), and Explainable AI
(XAI).The system predicts the churn risk of users based on their engagement behavior

Department of AI&ML, BIT 2024-25 31

Requirement Engineering Chapter 3

and sentiment analysis, provides interpretability through SHAP-based explanations, and

generates personalized retention recommendations to improve customer loyalty. This
document defines the system’s functional components, user interactions, performance
expectations, and technical constraints to ensure clarity, scalability, and maintainability
throughout the development lifecycle.

i. User Requirements

 The Admin or Data Analyst shall be able to access a home dashboard

displaying key metrics such as churn rate, user segments, and overall retention
trends.

 The user shall be able to upload new user data (CSV format) or connect to a
live database for churn prediction.

 The system shall provide a churn prediction output for each user, including
the churn probability score and status (e.g., At Risk or Retained).

 The user shall be able to view SHAP-based explainable insights that highlight
the top contributing factors influencing churn predictions.

 The user shall have the option to generate personalized retention strategies
for high-risk users directly from the dashboard.

 The interface shall be user-friendly, visually intuitive, and responsive across

desktop and mobile devices, ensuring smooth navigation and data visualization.

ii. System Requirements

System requirements define the essential functionalities and performance

characteristics that the Retention-AI System must fulfill. They include both functional
and non-functional requirements, ensuring that the system operates reliably, efficiently,
and meets user expectations in real-world customer retention and churn prediction
scenarios. These requirements guarantee that the system not only provides accurate

Department of AI&ML, BIT 2024-25 32

Requirement Engineering Chapter 3

predictions but also maintains transparency, scalability, and usability across diverse
business environments.

1. Functional Requirements

 The system shall enable uploading, storage, and preprocessing of customer data,
including both structured (user activity logs, engagement metrics) and
unstructured (user feedback, reviews) datasets. Data preprocessing shall include
handling missing values, normalization, and encoding to ensure model readiness.
 The system shall support training and evaluation of multiple machine learning
models, such as Random Forest, XGBoost, and Ensemble Classifiers, to identify
the model that provides the highest churn prediction accuracy.
 The system shall be capable of accepting new user data inputs in CSV format or
through a live database connection for churn prediction in real time.
 The system shall generate and return churn prediction results for each user,
categorizing them as ―At Risk‖ or ―Retained,‖ along with the churn probability
score.
 The system shall incorporate sentiment analysis using NLP techniques like
VADER to extract and quantify emotions from user feedback, contributing to the
overall churn prediction process.
 The system shall integrate an Explainable AI (XAI) module utilizing SHAP to
interpret model outputs, highlighting the top features that influenced each churn
prediction.
 The system shall provide an interactive dashboard for the Admin or Data Analyst
to visualize churn metrics, review insights, and generate personalized retention
strategies.
 The system shall support real-time retraining of models based on defined
thresholds (e.g., retraining after a set number of new users), ensuring adaptive and
updated churn predictions.

2. Non-Functional Requirements

Department of AI&ML, BIT 2024-25 33

Requirement Engineering Chapter 3

 Usability: The system shall provide an intuitive, visually appealing, and easy-
to-navigate user interface through the dashboard. Users should be able to
access model results, churn insights, and personalized strategy
recommendations without technical complexity.
 Performance: The system shall process uploaded datasets, perform churn
predictions, and return interpretable results within 3 - 5 seconds under normal
operating conditions, ensuring a smooth real-time experience for end-users.
 Scalability: The system shall be capable of handling increasing data volumes
and concurrent user requests as more customers and feedback data are
integrated into the platform. It should support future expansion to enterprise-
level datasets without performance degradation.
 Reliability: The system shall maintain operational accuracy and stability at
least 99% of the time, ensuring that model predictions and dashboards remain
consistently available and accurate.
 Security: The system shall validate all user inputs and ensure that sensitive
customer data is securely stored and protected from unauthorized access
through appropriate encryption and authentication mechanisms.
 Maintainability: The system’s codebase shall be modular, well-documented,
and structured to allow easy debugging, model updates, and integration of new
AI components or additional features in the future.

Department of AI&ML, BIT 2024-25 34

CHAPTER 4

PROJECT PLANNING
This chapter presents the structured plan and timeline followed for developing the
Retention-AI System. Proper project planning ensures smooth execution, efficient
resource utilization, and timely completion of all deliverables. The development process
was divided into systematic phases — beginning with problem identification and
literature survey, followed by data collection, preprocessing, model development,
integration of Explainable AI (XAI), dashboard implementation, and testing. Each phase
was strategically scheduled to ensure steady progress toward the project’s objectives and
timely milestone achievement.

4.1 Project Planning and Scheduling

The Gantt chart illustrates the overall timeline, showing interdependencies
between stages such as data preparation, model training, performance evaluation, and
system deployment. This structured planning approach ensured that all activities were
well-coordinated and aligned toward the successful implementation of Retention-AI,

Department of AI&ML, BIT 2024-25 35

Project Planning Chapter 4

minimizing delays and maintaining consistent workflow throughout the project lifecycle.

4.1: Project Planning Gantt Chart

Project Timeline Breakdown

Phase 1: Foundation (Feb – Apr)

During this stage, the project scope, objectives, KPIs, and success metrics were defined.
The development team was assembled, roles were assigned, and resource allocation was
finalized. Data governance, ethical AI use, and privacy policies were also established to
ensure compliance and transparency.

A comprehensive literature review was conducted to understand existing churn prediction

methodologies, NLP applications in sentiment analysis, and XAI interpretability
frameworks. Studies on Random Forest, XGBoost, and SHAP-based models were
analyzed to identify optimal algorithms. Competitive benchmarking helped identify
industry gaps that Retention-AI would address.

The system architecture was designed based on modular integration between structured
and unstructured data pipelines. Data schemas for the User Behavior Database and
Feedback Database were defined. Appropriate ML and NLP algorithms were selected,
and API integration plans were outlined for dashboard connectivity and model
deployment.

Phase 2: Data Infrastructure (Apr – May)

Data pipelines were developed to capture both behavioral and feedback data. The system
tracked user engagement metrics and feature usage patterns while also recording textual
reviews through feedback forms. ETL (Extract, Transform, Load) mechanisms were built
to ensure seamless and structured data ingestion.

The collected data underwent rigorous preprocessing, including handling missing values,
detecting outliers, and normalizing user activity logs. Textual data from feedback was

Department of AI&ML, BIT 2024-25 36

Project Planning Chapter 4

cleaned, tokenized, and transformed into numerical embeddings for NLP analysis.
Training datasets were prepared for ML and sentiment models.

Phase 3: AI Development (May – Aug)

Relevant behavioral and sentiment-based features were engineered to improve model

learning. This included metrics such as session frequency, engagement duration, and
sentiment polarity. Feature selection methods were applied to eliminate redundancy and
enhance model accuracy.

Machine learning models were trained for behavioral analysis, while NLP models
(VADER) were used for sentiment scoring. Both outputs were fused in a churn prediction
ensemble model using algorithms such as Random Forest and XGBoost. Performance
validation ensured consistency and minimized overfitting.

Phase 4: Intelligence & Deployment (Aug – Oct)

Models were optimized using hyperparameter tuning and cross-validation techniques. The
Explainable AI (XAI) layer, powered by SHAP, was integrated to provide interpretable
insights. The models were evaluated across precision, recall, and F1-score metrics to
benchmark performance.

Comprehensive testing was performed, including User Acceptance Testing (UAT) and
A/B testing of retention strategies. Scalability and stress tests were conducted to evaluate
system performance under large data loads. Results validated that the Retention-AI
system operated efficiently with reliable churn prediction accuracy.

Phase 5: Launch (Oct – Nov)

In the final phase, system documentation, algorithm explanations, and dashboard user
manuals were compiled. The system was deployed for production, followed by post-
launch monitoring to ensure operational stability and continuous performance tracking.

Department of AI&ML, BIT 2024-25 37

CHAPTER 5
SYSTEM DESIGN
System design serves as the foundation for transforming theoretical models and
conceptual understanding into a practical, implementable system. This chapter discusses
the architectural, modular, and algorithmic design of Retention-AI, a unified AI-driven
churn prediction and retention recommendation system. The design incorporates Machine
Learning (ML), Natural Language Processing (NLP), and Explainable AI (XAI) into a
cohesive pipeline that processes user behaviour logs and textual feedback to generate
churn insights and personalized retention strategies. The chapter also highlights the
interfaces involved, data structures used, and detailed component designs that ensure
scalability, modularity, and seamless interaction between the system layers.

5.1 System Architecture

The architecture of Retention-AI is organized into multiple layers that handle data
ingestion, preprocessing, model execution, explainability, and user interaction. It is
designed to support structured behavioural logs and unstructured textual feedback,
enabling comprehensive churn analysis.

Figure 5.1: System Architecture Diagram of Rentention AI

Department of AI&ML, BIT 2024-25 38

System Design Chapter 5

The diagram illustrates the complete end-to-end workflow of Retention-AI,

beginning with data ingestion and ending with actionable churn insights. Structured data
such as user engagement logs and unstructured data including reviews or social media
comments are fed into the Data Preprocessing Layer, where cleaning, transformation, and
feature extraction occur.

After preprocessing, the ML Model (XGBoost) handles churn prediction from

behavioural datasets, while the NLP Engine (VADER) performs sentiment analysis on
textual data. Both results are then passed to the Explainable AI (XAI) Layer, where
SHAP-based interpretability reveals the major factors contributing to churn.
Finally, processed churn insights and retention recommendations are sent to the Frontend
Dashboard, enabling users to view predictions, explanations, and personalized analysis.

System Layers

1. Data Input & Preprocessing Layer: Handles ingestion of user logs and feedback,
followed by cleaning, tokenization, feature extraction, and model-specific dataset
splitting.

2. ML & NLP Processing Layer: Uses XGBoost for structured data prediction and
VADER-based sentiment scoring for textual inputs. Outputs are combined for ensemble
prediction.

3. Explainable AI Layer (XAI): Generates SHAP value–based explanations and

highlights key behavioural and sentiment factors influencing churn.

4. Insights & Recommendation Layer: Produces aggregated churn reports, user risk
segmentation, and personalized retention strategy recommendations.

5. Frontend UI Layer: Displays dashboards, sentiment analytics, churn prediction,

explanations, and retention strategies through a streamlined web interface.

Department of AI&ML, BIT 2024-25 39

System Design Chapter 5

5.2 Module Decomposition

Figure 5.2: Module Decomposition Diagram

Module decomposition describes the breakdown of Retention-AI into smaller,

logically independent subsystems, each responsible for carrying out a specific task. This
decomposition improves maintainability, enables smooth integration, and supports clear
mapping between functional requirements and system behaviour. The platform is divided
into six key modules: Data Input Module, Preprocessing Module, Sentiment Analysis
Module, Churn Prediction Module, XAI Utility Module, Dashboard Output Module, and
the Automation & Pipeline Orchestrator.

1. Data Input Model : The Data Input Module serves as the entry point for all raw data.
It enables admins to upload structured logs such as session data, activity frequency, and
retention metrics, along with unstructured textual feedback from user reviews. The
module validates file formats, performs schema checks, and ensures the uploaded datasets

Department of AI&ML, BIT 2024-25 40

System Design Chapter 5

conform to expected structures. It routes structured data to the behavioural preprocessing

pipeline and unstructured text to the NLP pipeline. By enforcing strict data hygiene at the
input stage, the module prevents malformed or corrupted data from entering the system.

2. Preprocessing Module: The Preprocessing Module is responsible for cleaning,

preparing, and transforming the raw data into a consistent format. For structured data, the
module handles missing values, caps outliers, normalizes numerical variables, and
encodes categorical fields. For unstructured text, it performs tokenization, stop-word
removal, punctuation filtering, and sentiment preprocessing. This module ensures that
dataset integrity is preserved while preparing separate versions for Model A and Model B.
It also splits the processed dataset into training, validation, and testing subsets,
guaranteeing standardized model input across multiple runs.

3. Sentiment Analysis Module: This module performs all NLP-based operations on

textual feedback. It uses the VADER sentiment analyzer to classify user reviews into
positive, neutral, or negative categories and compute a sentiment score between -1 and
+1. These sentiment scores form additional features for the ML prediction pipeline. The
module extracts emotional tone, dissatisfaction indicators, and behaviour-related context
embedded in text. By converting qualitative user opinions into quantitative signals, it
enhances the overall prediction accuracy when combined with behavioural data.

4. Churn Prediction Module: The Churn Prediction Module is the core intelligence
engine of Retention-AI. It deploys XGBoost for behavioural churn detection and Random
Forest for sentiment-augmented churn prediction. Each submodel generates a churn
probability based on its respective input data. The module then combines the two scores
using a weighted ensemble approach to produce a final, more reliable churn probability. It
also categorizes users into different churn risk segments such as Low Risk, Medium Risk,
and High Risk, enabling businesses to prioritize intervention.

5. XAI Utility Layer (Explainability Module): Once the churn probability is computed,
the XAI module provides full interpretability through SHAP values. It explains how each
feature contributed to the prediction outcome, ensuring transparency and trust in the
model. The module generates local explanations for individual users and global
explanations to highlight dominant churn factors across the user base. These explanations

Department of AI&ML, BIT 2024-25 41

System Design Chapter 5

guide admins in understanding user pain points, behavioural declines, or sentiment-driven

dissatisfaction.

6. Dashboard Output Layer: This module handles the graphical output and visualization
layer through Streamlit. It provides interactive charts, churn risk summaries, SHAP
visualizations, sentiment distributions, and data upload interfaces. The dashboard ensures
that all model results, explanations, and retention recommendations are displayed in a
clear and business-friendly manner. Admins can explore individual user profiles, filter
risk segments, and download analytical reports.

7. Automation & Pipeline Module: The Automation Orchestrator monitors data

directories for new uploads, triggers preprocessing, checks retraining thresholds, and
updates churn predictions and XAI explanations. It ensures continuous system adaption
by retraining Model A after every 20 new structured inputs and Model B after every 10
new unstructured inputs. Through automated scheduling, logging, and threshold-based
triggers, the pipeline module maintains operational efficiency without requiring manual
intervention.

5.3 Component Design / Detailed Class Diagram

The component design defines the internal structure, class relationships, attributes,
and operational methods that form the core logic of Retention-AI. Each major module is
implemented as a dedicated class, encapsulating its functionality and ensuring modular,
object-oriented development. The following components define the backbone of the
system:

1. Data Processing Module

Class: DataProcessor
The DataProcessor class forms the foundation of the system’s data pipeline. It handles
raw data ingestion, cleaning, transformation, and preparation for machine learning and
NLP tasks. The class ensures dataset consistency across all operational branches,
preserving data quality and improving downstream model performance.

Department of AI&ML, BIT 2024-25 42

System Design Chapter 5

Attributes

 inputData (DataFrame): Stores the loaded structured or unstructured dataset.

 splitCriteria (dict): Defines conditions for creating model-specific datasets such
as behavioural-only or review-linked input.

Methods

 cleanMissing(data): Imputes missing values and standardizes datatypes.

 capOutliers(data, columns, cap): Caps extreme numeric values to mitigate
skewness.
 preprocessText(text): Cleans and normalizes textual reviews for NLP analysis.
 saveOutputs(dataMap, paths): Saves cleaned and split datasets to appropriate
storage directories.

2. Model Training Module

Class: ModelTrainer
This class orchestrates the training and evaluation of ML models used in the prediction
pipeline. It supports multiple model architectures and handles balancing, splitting, and
validation.

Attributes

 modelType (enum): Specifies the selected model, such as XGBOOST or

RANDOM_FOREST.
 trainTestSplit (float): Defines the ratio for dataset division.
 classBalance (bool): Indicates whether SMOTE-based balancing is applied.

Methods

 trainModel(X_train, y_train): Fits the chosen algorithm using specified

hyperparameters.
 evaluateModel(model, X_test, y_test): Computes metrics such as accuracy,
precision, and confusion matrix.

Department of AI&ML, BIT 2024-25 43

System Design Chapter 5

 saveModel(model, filePath): Persists trained model objects for reuse during

inference.

3. Prediction & Analysis Module

Class: PredictionEngine
Responsible for generating churn predictions and explainable insights. It bridges ML
outputs with XAI reasoning.

Attributes

 explainer (object): SHAP explainer instance for feature attributions.

 outputPaths (dict): Storage locations for predictions and explanation files.

Methods

 predictUser(userId): Returns churn probability, risk label, and classification

results.
 combineProbabilities(pA, pB): Applies ensemble rules to produce the final
churn probability.
 explainUser(userId): Generates SHAP-based explanations for each user.

4. Frontend Module

Class: Frontend Website

Implements the complete user interface for interacting with Retention-AI.

Attributes

 pages (enum): Dashboard, Data Upload, Sentiment Analysis, Churn Prediction,

Explainable AI, Retention Strategies.
 navigation (object): Routing configuration for switching between views.
 apiBaseURL (string): Base endpoint for backend interactions.

Methods

Department of AI&ML, BIT 2024-25 44

System Design Chapter 5

 displayDashboard(): Renders KPIs, charts, and churn insights.

 uploadData(file): Uploads datasets to backend storage.
 viewExplanation(userId): Displays SHAP interpretation for selected users.

5. Automation & Pipeline Module

Class: AutomationOrchestrator
Coordinates scheduling, automated triggers, and continuous model updates.

Attributes

 dataPaths (dict): File directories for structured and unstructured data.

 thresholds (dict): Retraining rules for Models A and B.
 schedules (dict): Periodic job intervals.

Methods

 monitorUploads(): Detects new files and triggers preprocessing.

 checkRetrainConditions(): Determines if retraining is needed.
 refreshOutputs(): Recomputes predictions and explanations for the dashboard.

5.4 Interface Design

The interface design of Retention-AI focuses on providing a seamless user

experience for administrators who interact with the system to upload behavioural logs,
analyze sentiment, generate churn predictions, and explore explainable AI insights. The
system’s user interface follows a simple and intuitive layout centered around a central
dashboard from which all major functionalities can be accessed. The login page acts as
the entry point where authenticated users enter the system. Upon successful login, the
dashboard acts as the main hub, offering panels for behaviour log uploads, feedback text
uploads, churn prediction, explainable AI visualization, retention recommendation
generation, and analytics/report downloads. Each panel connects directly to an internal
processing component such as the preprocessing module, NLP-based sentiment analyzer,

Department of AI&ML, BIT 2024-25 45

System Design Chapter 5

ML churn scorer, XAI engine, and strategy engine. This modular panel-based structure
ensures that each operation is clearly segregated and easily accessible.

The dashboard is designed to present a clean navigation structure with minimal

user friction. Users can move quickly between different sections without reloading the
page or navigating through multiple nested menus, ensuring fast execution of tasks. This
layout is particularly important for an admin-facing analytics system where efficiency,
clarity, and rapid access to analytical outputs are critical. Each panel contains interactive
elements such as file upload components, status messages, loading indicators, and result
visualizations.

1. User Interaction with the System (GUI)

When an administrator wants to upload behavioural log data, they navigate to the
―Upload Behavior Logs‖ section from the dashboard. The UI Manager displays a file-
upload widget allowing the admin to choose a CSV file containing user activity metrics
such as screen time, usage frequency, last seen date, and session duration. Once the file is
selected, the admin clicks the ―Upload‖ button, which triggers the frontend’s
handleLogUpload() function. This function packages the selected file into a multipart form-
data request and sends it securely to the backend via the API Client.

Upon receiving the upload, the backend validates the file structure, saves the
dataset to the designated storage path, and registers the upload event. It then returns a
response summarizing the number of records successfully received and any detected
issues. Once the frontend receives the backend response, the UI Manager displays a
success message confirming that the behavioural logs were uploaded and queued for
preprocessing. This feedback loop ensures that admins have real-time awareness of
upload outcomes and can immediately proceed to other analytical operations.

For uploading user reviews or free-text feedback, the admin navigates to the
―Upload Feedback Text‖ section. The UI presents a file-upload component similar to
behavioural logs but expects text-based entries. When a text file or CSV containing user
reviews is selected and uploaded, the UI’s handleFeedbackUpload() function is triggered.
This function sends the file to the backend pipeline, where the NLP Preprocessing

Department of AI&ML, BIT 2024-25 46

System Design Chapter 5

Module processes it through tokenization, text cleaning, and sentiment extraction. After
processing, the backend responds with a summary including the count of reviews
processed and the associated sentiment scores. The frontend displays a confirmation
message and updates the dashboard to reflect that sentiment data is now available for
downstream churn prediction.

2. Component-to-Component Interaction

Frontend Module to Automation & Pipeline Module: Once an admin uploads a

behaviour log or feedback file, the frontend sends the file to the backend where the
Automation & Pipeline Module receives the upload event and registers new data. This
module acts as the central dispatcher for pipeline operations. It determines whether full
preprocessing, partial preprocessing, or additional model refreshing is required based on
the data type and upload size.

Automation & Pipeline Module to Data Processing Module: After registering the new
data, the Automation & Pipeline Module triggers the Data Processing Module. The
preprocessing component performs cleaning, missing-value imputation, outlier capping,
text preprocessing, and dataset segmentation. This ensures that the data is in the correct
format for both the ML churn model and the NLP sentiment engine.

Data Processing Module to Prediction & Analysis Module: Once data preparation is
complete, the cleaned dataset is transferred to the Prediction & Analysis Module. This
module loads the currently active churn models and computes churn probabilities, risk
labels, and combined ensemble scores for users. It generates user-level prediction objects
and compiles the results into structures that are ready for backend delivery.

Prediction & Analysis Module to Frontend Module: The Prediction & Analysis
Module sends the churn predictions, risk categories, and explainability outputs back to the
frontend via API responses. The UI immediately updates the corresponding dashboard
panels, showing key metrics such as number of high-risk users, average sentiment trends,
and the top SHAP features contributing to churn predictions. This enables admins to
interpret user behaviour trends quickly and take informed action.

Department of AI&ML, BIT 2024-25 47

System Design Chapter 5

Automation & Pipeline Module to Model Training Module: If the Automation &
Pipeline Module identifies that the number of newly uploaded records exceeds predefined
retraining thresholds (e.g., Model A retrains after 20 new logs, Model B after 10 new
feedback entries), it initiates a retraining cycle. The Model Training Module re-trains the
ML and NLP models on the updated dataset and stores the new model versions. These
updated models are then used for all subsequent predictions and explanations, ensuring
that the system continuously adapts to new user behaviour.

5.5 Data Structure Design

The data structure design defines how information is represented, stored,

accessed, and exchanged between modules in Retention-AI. Efficient and consistent data
structures ensure that preprocessing, prediction, and sentiment analysis operations
execute correctly and in a modular manner. The system uses a combination of Pandas
DataFrames, Python classes, dictionaries, and timestamped structures to represent users,
feedback, and prediction outputs. Below is the complete set of data structures following
the same style as your friend’s report.

Data Structure: Feedback

review_id: string
user_id: string
text: string
sentiment_score: float
timestamp: datetime

This structure represents each individual feedback entry. The review_id uniquely
identifies each review. The user_id links the feedback to the corresponding user profile.
The text field contains the full review content. The sentiment_score stores the VADER-
derived sentiment value, while timestamp records when the feedback was created.

Data Structure: UserProfile

user_id: string
screen_time: float

Department of AI&ML, BIT 2024-25 48

System Design Chapter 5

spent_on_app: float
left_review: bool
rating: int
sentiment_score: float
churn_risk: enum("High", "Medium", "Low")
explainability_factors: dict<string, float>

The UserProfile structure is used to store combined behavioural and sentiment attributes
of each user. The profile includes both structured metrics such as screen_time and
spent_on_app and unstructured metrics such as sentiment_score. The churn_risk attribute
holds the model-generated risk level, while explainability_factors stores SHAP-derived
influence scores for each relevant feature.

Primary Data Structure: DataFrame

The Pandas DataFrame is the primary data structure used within Retention-AI for storing
behaviour logs, user profiles, feature vectors, prediction outputs, and SHAP explanations.
Its tabular nature supports rapid merging, filtering, grouping, and statistical computation.
It also enables smooth conversion to NumPy arrays, which are required for machine
learning training and inference.

Class/Object Representation: User

Within the internal logic of Retention-AI, each user can be represented as a User object.
This class allows the system to bundle all relevant user information into a single unit for
easy manipulation across modules.

class User:
def __init__(self, user_id, screen_time, spent_on_app, feedback_text):
self.user_id = user_id
self.screen_time = screen_time
self.spent_on_app = spent_on_app
self.feedback_text = feedback_text
self.sentiment_score = None
self.churn_probability = None

Department of AI&ML, BIT 2024-25 49

System Design Chapter 5

self.risk_level = None

When new data is uploaded and processed, the system can create a list of User objects.
The Sentiment Analysis Module updates the sentiment_score attribute, and the Churn
Prediction Module fills the churn_probability and risk_level attributes. This sequential
update mechanism makes it easy to track how each user progresses through the pipeline
and how their churn risk is computed.

5.6 Algorithm Design

The algorithm design of Retention-AI outlines the logical flow and computational
processes that enable the system to perform churn prediction, sentiment analysis,
explainable reasoning, and automated retraining. Each algorithm is structured to ensure
consistent behavior, reproducibility, and modular execution across all system
components. The system integrates multiple algorithmic pipelines, including behavioural
preprocessing, text preprocessing, churn prediction through ensemble learning, SHAP-
based explainability, and automated retraining. These algorithms collectively ensure that
Retention-AI can efficiently handle large datasets, extract relevant patterns, and produce
actionable insights for the admin. The following subsections describe the key algorithms
that drive the core functionalities of the platform.

1. Algorithm: Behavioral Data Preprocessing

This algorithm processes raw behavioural logs uploaded by the admin and prepares them
for input into the churn prediction model. It handles missing values, caps extreme
numeric values, normalizes key features, and splits the dataset into training and testing
subsets. This ensures that the data is clean, consistent, and suitable for ML operations.

 Load uploaded behavioural dataset.

 Identify and impute missing values (mean/median for numerical fields).
 Detect extreme numeric values and apply capping based on the specified
threshold.

Department of AI&ML, BIT 2024-25 50

System Design Chapter 5

 Normalize or standardize numerical columns such as screen_time and

spent_on_app.
 Encode categorical or boolean fields (e.g., left_review) if required.
 Split the processed dataset into Model A inputs (behaviour-only features).
 Save the cleaned dataset for prediction and model training.

Through this algorithm, Retention-AI ensures that the behavioural dataset remains free
from noise and anomalies, improving the stability of downstream model inference.

2. Algorithm: Text Preprocessing & Sentiment Extraction

This algorithm transforms unstructured textual feedback into usable sentiment features. It
applies cleaning, tokenization, lexicon-based scoring, and sentiment aggregation.

 Retrieve the uploaded text feedback file.

 Convert all text to lowercase and remove punctuation, special characters, and
HTML tags.
 Tokenize the cleaned text into individual words.
 Remove stopwords and irrelevant tokens.
 Pass the processed text through the VADER sentiment analyzer.
 Generate a sentiment_score for each review.
 Merge sentiment scores with corresponding user IDs.
 Output final sentiment-enhanced dataset for Model B.

This algorithm ensures that qualitative feedback is quantified into meaningful numerical
signals that contribute to churn prediction.

3. Algorithm: Churn Prediction (Ensemble-Based)

The churn prediction algorithm combines the outputs of two predictive models—Model A
(behaviour-based XGBoost) and Model B (sentiment-augmented Random Forest). The
ensemble mechanism produces a final churn probability that is more stable and accurate
than individual model predictions.

Department of AI&ML, BIT 2024-25 51

System Design Chapter 5

 Load processed behavioural dataset (Model A input).

 Load sentiment-enhanced dataset (Model B input).
 Pass Model A input into the XGBoost model to obtain probability P_A.
 Pass Model B input into the Random Forest model to obtain probability P_B.
 Apply an ensemble rule:
 Final Probability = (0.6 * P_A) + (0.4 * P_B)
 Categorize users based on thresholds:
 P ≥ 0.75 → High Risk
 0.40 ≤ P < 0.75 → Medium Risk
 P < 0.40 → Low Risk
 Generate churn labels and store prediction outputs.
 Send final results to the Prediction Engine for UI rendering.

By combining behavioural and sentiment cues, this algorithm increases prediction

reliability and better reflects real-world churn behaviour.

4. Algorithm: Explainable AI (SHAP-Based)

Retention-AI uses SHAP (SHapley Additive exPlanations) to compute the feature

contributions behind each user's churn probability. This algorithm generates both local
explanations (per user) and global explanations (overall dataset).

 Load the trained XGBoost model.

 Initialize SHAP TreeExplainer with the model.
 For each user record, compute SHAP values across all input features.
 Identify the top contributing features (positive and negative).
 Generate local explanations for individual predictions.
 Aggregate SHAP values across the dataset to generate global explanations.
 Export SHAP plots, bar charts, and explanation summaries.
 Send explanations to the frontend visualization module.

Department of AI&ML, BIT 2024-25 52

System Design Chapter 5

5. Algorithm: Retention Strategy Recommendation

This algorithm uses churn risk levels and sentiment signals to assign personalized
retention strategies to users. It maps high-impact factors to specific business actions.

 Take churn probability and sentiment_score as inputs.

 If churn_risk = ―High‖:
 Prioritize negative sentiment factors.
 Suggest discount offers, personalized re-engagement messages, or issue-based
outreach.
 If churn_risk = ―Medium‖:
 Suggest activity-boosting nudges or soft notifications.
 If churn_risk = ―Low‖:
 Maintain engagement through regular content updates or loyalty rewards.
 Combine SHAP global factors to refine strategy recommendations.
 Produce a final recommendation object.
 Display results in the Retention Strategy Panel.

6. Algorithm: Automated Retraining Trigger

Retention-AI includes a continuous-learning component that retrains its models whenever

sufficient new data becomes available. This ensures that the models remain up-to-date
with evolving user behaviour and feedback trends.

 Automation Module monitors upload directories for new datasets.

 Count the number of new behavioural logs since the last training cycle.
 Count the number of new feedback entries since the last NLP training cycle.
 If behavioural logs ≥ 20 → trigger retraining of Model A.
 If feedback entries ≥ 10 → trigger retraining of Model B.
 Call the Model Training Module to perform full preprocessing, training,
evaluation, and model saving.
 Register the new model versions.

Department of AI&ML, BIT 2024-25 53

System Design Chapter 5

 Notify the Prediction Engine to use updated models for future predictions.

Department of AI&ML, BIT 2024-25 54

Chapter 6

IMPLEMENTATION

The implementation of Retention-AI involves the translation of the designed

architecture, algorithms, and data structures into functional software modules. Each
component is implemented using Python-based backend services, integrated ML/NLP
libraries, and a streamlined API layer that communicates with the frontend interface. The
system is implemented in a modular fashion to allow independent development, testing,
and modification of each pipeline stage. The core technologies used include Pandas for
data manipulation, Scikit-learn and XGBoost for model training, VADER for sentiment
analysis, SHAP for explainability, and FastAPI for serving backend endpoints. Each
module interacts with others through clean data contracts and well-defined API calls,
ensuring smooth orchestration from data ingestion to final insights.

During implementation, emphasis was placed on ensuring robustness, error

handling, and consistency across the pipeline. User-facing operations such as uploading
behavioural logs or feedback files are implemented through dedicated API routes that
handle input validation, file storage, and pipeline triggering. Backend modules handle all
preprocessing operations, model inference computations, and explainability generation.
The automation layer monitors dataset updates and retrains models based on predefined
thresholds, ensuring that the system remains adaptable and up-to-date with user
behaviour. The final outputs, including churn predictions, sentiment scores, SHAP
explanations, and retention strategies, are delivered to the frontend dashboard for
visualization.

The following sections describe the implementation pseudocode for major

components of Retention-AI, following the same structure as the reference project.

6.1 Implementation Approaches

The implementation of Retention-AI was carried out using a structured, modular

approach to ensure accuracy, efficiency, and scalability. The primary objective of this
phase was to convert the system design into a fully functioning AI-based user churn
prediction system. Development followed a disciplined workflow where each

Department of AI&ML, BIT 2024-25 55

Implementation Chapter 6

component—preprocessing, sentiment extraction, machine learning models,

explainability engine, automation, and API services—was implemented and validated
before integration. FastAPI served as the backend interface, supporting smooth data flow
between the UI and the machine learning modules. The system adheres to the principles
of the Model-View-Controller (MVC) architecture, ensuring a clear separation between
the data pipeline, business logic, and dashboard presentation layers.

Implementation Plan

1. Environment Setup: The implementation began with configuring the FastAPI

backend, setting up the project directory, and installing required Python libraries such as
Pandas, NumPy, Scikit-learn, XGBoost, NLTK, VADER-Sentiment, and SHAP. Folder
structures were created for uploads, processed datasets, model files, SHAP charts,
predictions, and log files.

2. Data Integration and Preprocessing: Behavioural logs and text feedback were
uploaded through the frontend dashboard. These datasets were cleaned, normalized,
tokenized (for text), scored using VADER sentiment analysis, and stored in their
respective processed folders. Outlier capping, missing-value imputation, type
standardization, and text preprocessing were implemented to ensure high-quality model
inputs.

3. Model Development: Two prediction pipelines were implemented:

 Model A: XGBoost classifier for structured behavioural features

 Model B: Random Forest classifier augmented with sentiment features

Both models were trained, grid-searched, evaluated, and saved into the models directory.
Prediction files were generated for each user and stored for ensemble processing.

4. System Integration: Backend modules, preprocessing pipelines, ML models,

sentiment analyzers, and SHAP engines were integrated through FastAPI. The frontend
dashboard was connected to backend API endpoints to display predictions, explanations,
risk labels, and retention strategies.

Department of AI&ML, BIT 2024-25 56

Implementation Chapter 6

5. File-Based Storage & Automation: Retention-AI uses file-based storage to hold

datasets, processed outputs, predictions, and trained models. Automation systems track
how many new users have been uploaded and trigger retraining when thresholds are
reached (Model A: 20 new logs, Model B: 10 new reviews).

6. Testing & Debugging: Each module was tested independently to ensure correctness—
upload operations, preprocessing steps, training accuracy, SHAP value correctness, API
responses, and dashboard functionality. Integration testing was performed to validate the
full pipeline.

6.2 Coding Details and Code Efficiency

The implementation of Retention-AI focuses on efficient integration of

preprocessing, ML, NLP, and XAI modules. Each module is designed to handle a specific
stage in the pipeline, ensuring readability, maintainability, and scalability. The system
uses vectorized data operations, cached explanations, and trained model persistence to
optimize performance. Below are the exact pseudocode implementations used in the
system (as shown in your screenshots), representing the core backend logic used for
building the Retention-AI engine:

[Link] Ingestion & Preprocess

FUNCTION upload(file):
IF file missing OR not CSV -> ERROR "Invalid Input"
SAVE file -> datasets/[Link]
df <- READ CSV
df <- CLEAN (missing/outliers) + NORMALIZE + ENCODE
SAVE -> datasets/[Link]
INCREMENT retrain_counters
RETURN message, records_processed

[Link] Model A

Department of AI&ML, BIT 2024-25 57

Implementation Chapter 6

FUNCTION train_model_a():
df <- READ datasets/model_a_train.csv
DROP leakage cols [userid, churn_risk, ...]
(X_tr, X_te, y_tr, y_te) <- SPLIT 80/20 (stratify)
(X_trb, y_trb) <- SMOTE(X_tr, y_tr)
model_a <- XGBClassifier()
FIT model_a ON (X_trb, y_trb)
SAVE model_a -> models/model_a_latest.pkl
SAVE feature_names -> models/model_a_features.json
PREDICT probs (all users) ->datasets/model_a_predictions.csv

[Link] Model B

FUNCTION train_model_b():
df <- READ datasets/model_b_train.csv
text <- PREPROCESS_TEXT(df.review_text)
feats <- SENTIMENT_SCORES(text)
model_b <- XGBClassifier()
FIT model_b ON (feats, y)
SAVE -> models/model_b_latest.pkl
PREDICT probs -> datasets/model_b_predictions.csv

4. Ensemble & Risk Label

FUNCTION build_churn_predictions():
A <- READ model_a_predictions.csv (userid, a_prob)
B <- READ model_b_predictions.csv (userid, b_prob)
FOR each userid:
final_prob <- (a_prob + b_prob)/2 IF b_prob exists ELSE a_prob
risk <- High if final_prob>=80
Medium if final_prob>=50
Low otherwise

Department of AI&ML, BIT 2024-25 58

Implementation Chapter 6

top_factors <- per-user factors if available ELSE [ ]

WRITE {userid, final_prob, risk, top_factors}
SAVE -> datasets/churn_prediction.csv

5. Explainable AI

FUNCTION explain_user(user_id):
IF cache outputs/explanations/user_{id}.json exists -> RETURN cache
model_a <- LOAD models/model_a_latest.pklfeats <- LOAD user features
shap_vals <- SHAP(model_a, feats)
top_features <- TOP_K(shap_vals, 10)
chart_path <- outputs/charts/user_{id}.png (Agg)
SAVE JSON {user_id, prob, risk, top_features, recommendations, chart_path}
RETURN JSON

Code Efficiency

The Retention-AI system is designed with a strong emphasis on code efficiency to

ensure fast execution, low resource usage, and smooth real-time performance. The entire
architecture follows a modular approach, allowing each component - preprocessing,
model inference, explainability, and strategy generation - to operate independently
without unnecessary coupling. Vectorized operations in Pandas and NumPy are used
extensively, reducing computational overhead by avoiding slow iterative loops.

Shared preprocessing functions are reused across models to prevent redundant

processing, and all trained models are stored as persistent files, eliminating the need for
retraining unless new data crosses a predefined threshold. Cached SHAP explanations
further optimize performance by avoiding repeated computation for the same user.
FastAPI asynchronous endpoints significantly improve response times for prediction and
explanation requests. Consistent directory structures and efficient file-based storage
reduce memory usage and speed up dataset/model loading operations.

Department of AI&ML, BIT 2024-25 59

Implementation Chapter 6

The entire codebase follows PEP8 standards, improving readability,

maintainability, and scalability. Overall, the system is structured to deliver high accuracy
while maintaining optimal speed, resource efficiency, and ease of future enhancements.

Department of AI&ML, BIT 2024-25 60

CHAPTER 7

TESTING

7.1 Testing Approach

The testing of the Retention-AI system was carried out using both unit testing and
integration testing to ensure that every component, from churn prediction to explainable
AI and retention strategy generation, performs correctly and reliably. Unit testing focused
on validating individual modules such as data upload, preprocessing, prediction logic, and
explanation generation. Integration testing evaluated how these modules work together
across the full flow — from user actions in the UI to backend API responses and database
updates.

This combined approach ensured that both the independent modules and the entire end-to-
end pipeline behaved as expected under different scenarios, user inputs, and data
conditions. Testing confirmed that Retention-AI consistently produces accurate churn
predictions, handles invalid inputs, and displays results through a smooth and intuitive
user interface.

7.1.1 Unit Testing

Unit testing was carried out to verify that each functional component of Retention-
AI behaves correctly in isolation. The main focus was on validating:

 the CSV upload module,

 the data preprocessing pipeline,
 the churn prediction output,
 the Explainable-AI module,
 and general input-handling reliability.

Each test case assessed how the system responded to different inputs such as valid files,
incorrect formats, missing values, invalid symbols, and corrupted explanation caches. The
tests confirmed that Retention-AI successfully handles proper inputs by generating
accurate churn probabilities, user-level insights, and correct SHAP-based explanations.

Department of AI&ML, BIT 2024-25 61

Testing Chapter 7

Some limitations were also identified. For instance, the system initially did not validate
unknown columns in uploaded CSV files and crashed when the explanation API was
called without a valid user ID. These issues highlight areas for stronger input validation
and error-handling mechanisms. Nevertheless, the overall unit testing phase demonstrated
that the core functionalities - including prediction accuracy, explanation generation, and
data processing - are functioning correctly while revealing important opportunities for
refinement.

Table 1: File Upload Module

Test Test Scenario / Expected Output Actual Output Status

Input
1 Upload valid ―File uploaded records_processed = 100 Pass
CSV file (100 successfully‖;
rows) records_processed = 100
2 Upload .txt file ―Invalid Input‖ ―Invalid Input‖ Pass

3 Upload empty ―Invalid Input‖ Validation error Pass

CSV
4 CSV with ―Invalid columns‖ File accepted but fails Fail
unknown during processing
columns

The File Upload Module correctly handles valid files, invalid file formats, and empty
datasets. However, it fails when unknown columns are present, indicating the need for
stricter schema validation before processing.

Table 2: Explanation Generation Module

Test Test Scenario / Input Expected Output Actual Output Status

1 Explanation request with Valid JSON with all JSON contains all Pass
cache available fields fields
2 User not found (User ID: ―User not found in Message returned Pass
999999) churn data‖
3 Corrupted cache JSON Regenerate Explanation Pass
explanation regenerated
4 Missing user_id parameter ―Bad Request‖ API crashes Fail

Department of AI&ML, BIT 2024-25 62

Testing Chapter 7

The Explanation Module performs well in generating valid results, handling missing
users, and recovering from corrupted cache files. The failure occurs when the user_id
parameter is missing, showing that API input validation needs improvement.

Table 3: System Validation & Error Handling

Test Test Scenario / Input Expected Output Actual Output Status

1 Malformed JSON request ―Invalid JSON Error thrown Pass
format‖
2 Upload large CSV (>50MB) ―File too large‖ File processed Fail
slowly
3 API under high load Stable response Delayed response Fail

4 Valid churn prediction request Probability + label Correct output Pass

output returned
5 Feedback with emojis/special Clean sentiment Proper sentiment Pass
characters output score

System-level tests reveal strong handling of malformed requests, prediction accuracy, and
NLP processing. However, performance limitations exist for large file uploads and high-
load API usage, highlighting areas requiring optimization to ensure scalability.

7.1.2 Integrated Testing

Integration testing was conducted after successful unit testing to verify the
seamless interaction between all system components. The key modules tested together
include the React frontend, Flask API layer, XGBoost-based churn model, sentiment
analysis module, explainable AI engine, and the backend dataset storage.

The goal of integration testing was to ensure that data flows correctly across the
system -from file uploads and prediction requests to dashboard refreshes, user explanation
retrieval, and retention strategy generation. During testing, the system successfully
processed uploaded CSV files, refreshed churn predictions, generated updated
dashboards, and visualized user-specific insights through SHAP values. The
communication between the frontend and backend remained stable, confirming that the
APIs, data processing logic, and visualization components are well integrated.

Department of AI&ML, BIT 2024-25 63

Testing Chapter 7

Table 1: Integrated Testing – File Upload & Model Pipeline

Test Test Scenario / Expected Result Actual Result Status

Input
1 Upload valid CSV Time and file stored; Success toast; Pass
via UI (100 rows) predictions refreshed predictions regenerated

2 Upload CSV missing Show ―Invalid Input‖; Error shown; no file Pass
required columns no DB update saved; predictions
unchanged
3 Upload valid CSV Trigger retrain; new Retrain logs present; Pass
exceeding retrain model saved; model updated
threshold predictions updated
4 Upload CSV during Upload request queued Upload blocked but UI Fail
active retraining or blocked gracefully freezes

This table validates the integration between the upload module, retraining logic, and
prediction pipeline. The system successfully handles valid/invalid uploads and triggers
retraining correctly when thresholds are met. However, when a retraining process is
active, the UI freezes instead of gracefully queuing or blocking the request, indicating a
concurrency-handling issue.

Table 2: Integrated Testing – Explanation Engine & Dashboard Rendering

Test Test Scenario / Input Expected Result Actual Result Status

1 Click user 1002 (exists, Load explanation Explanation JSON Pass
cached) instantly with chart loads; chart renders
2 Click user with no cache Explanation Explanation Pass
file regenerated without regenerated
errors
3 Select user explanation Display ―Chart not API returns file-not- Fail
when charts folder available‖ found exception
missing

This table verifies the integration between the explanation engine, cache layer, and
dashboard visualization. The failure occurs when the system attempts to load a chart that
does not exist, resulting in an API exception instead of a controlled error message.

Department of AI&ML, BIT 2024-25 64

Testing Chapter 7

Table 3: Integrated Testing – System Behavior, Error Handling & API

Coordination

Test Test Scenario / Input Expected Result Actual Result Status

1 User explanation request with Return ―Bad API throws error Fail
missing user_id Request‖

2 Server under moderate load Stable response; All responses Pass

(10 users requesting no timeouts returned
explanation) successfully

3 Server under high load (50 System should Significant delays; Fail
concurrent uploads + degrade partial hangs
explanations) gracefully

This table evaluates system-level interactions under stress and error conditions. The
system performs well under moderate load but struggles under heavy load, indicating
performance bottlenecks. The missing user_id case also fails, showing insufficient API-
level validation. These tests highlight areas requiring optimization for robustness and
scalability.

Department of AI&ML, BIT 2024-25 65

Chapter 8

RESULTS, DISCUSSION AND PERFORMANCE

ANALYSIS

The Results, Discussion, and Performance Analysis chapter presents a complete

evaluation of the RETENTION-AI system, focusing on how effectively it predicts churn,
analyzes user sentiment, and generates explainable insights in real-world conditions. This
section highlights the system’s ability to handle diverse operational scenarios, process
user-uploaded behavioral data, and deliver reliable churn predictions through machine
learning models. The chapter also examines the platform’s usability, robustness, and its
response to different test cases, ensuring that RETENTION-AI meets all functional and
user-defined requirements. The combination of explainable AI, sentiment scoring, and
automated retention strategy generation makes RETENTION-AI a comprehensive churn-
management solution, and this chapter captures its performance across various
dimensions.

8.1 Test Reports

The test reports summarize how RETENTION-AI performed under various
operational and input-related conditions. The testing covered key features such as data
upload, preprocessing, prediction generation, sentiment extraction, explainable AI
outputs, and retention strategy recommendations. The following observations outline the
system’s behavior across different tests:

1. Valid Input Testing:

 When valid CSV files containing complete user behavior logs and feedback were
uploaded, the system successfully processed and normalized the data.
 Churn predictions were generated accurately, reflecting the correct probabilities
and risk levels.
 The explainability module displayed correct SHAP insights, validating the
correctness of model inference.

Department of AI&ML, BIT 2024-25 66

Results Chapter 8

2. Invalid Input Handling:

 On uploading unsupported file formats (TXT, JSON) or empty CSV files, the
system produced appropriate error messages such as ―Invalid Input‖ or ―Missing
required columns‖.
 This confirms strong validation checks in the upload and ingestion pipeline.

3. System Stability and Performance:

 RETENTION-AI remained stable across multiple uploads, model runs, and

explanation requests..
 Even under heavy data load (e.g., datasets with 1000+ users), predictions and
analytics displayed without lags.

4. Sentiment Analysis Verification:

 The NLP-based sentiment analyzer correctly classified user feedback into

Positive, Neutral, and Negative labels.
 Sentiment scores were consistent for repeated inputs and aligned with expected
linguistic patterns.

5. User Interface Accuracy:

 The system provided clear visualizations for risk distribution, churn probability
distribution, and explanation charts.
 Searching users, filtering by risk levels, and generating insights worked smoothly
across all test cycles

6. Explainable AI and SHAP Insights:

 For every user ID requested, the system returned impact scores and top risk
factors.
 Cached explanations were retrieved correctly, reducing computation time for
repeated users.

Department of AI&ML, BIT 2024-25 67

Results Chapter 8

7. Retention Strategy Generation:

 The retention module recommended correct message templates and allowed

admins to customize and send emails.
 For high-risk users, appropriate interventions such as support outreach, exclusive
offers, and engagement-boosting messages were displayed.

8. Overall System Performance:

 RETENTION-AI demonstrated high responsiveness, precise churn scoring, and

consistent predictive outcomes.
 The system combines user behavior data, sentiment features, and ML-driven
predictions effectively, confirming its suitability for real-world churn management
workflows.

These findings collectively prove that RETENTION-AI is robust, accurate, and capable
of handling diverse churn-prediction scenarios with stability and clarity.

8.2 User Documentation

RETENTION-AI is a fully integrated churn-prediction and retention-
recommendation system built with FastAPI, Python, XGBoost, and NLP-based sentiment
analysis. Its streamlined user interface ensures that even non-technical users can analyze
churn risk, view explanations, and take action through personalized retention strategies.

Below is the user documentation written exactly like FIN-NET’s style:

1. Dashboard Overview

The Dashboard provides a complete snapshot of user churn and retention metrics. Users
can view the total number of customers, churn rate, retention rate, and visual breakdowns
such as risk distribution and churn probability charts.
(Ref: Screenshot – Dashboard Overview)

Department of AI&ML, BIT 2024-25 68

Results Chapter 8

2. Data Upload

In the Data Upload interface, admins can upload user behavior logs or feedback data via
CSV. The platform validates the file, preprocesses features, and updates the dataset. The
page also provides formatting instructions to ensure correct upload structure.
(Ref: Data Upload screenshot)

3. Churn Prediction Module

This page displays churn predictions for all users after processing the dataset. Each user is
assigned a churn probability and a corresponding risk category (Low, Medium, High).
The table supports searching, filtering, and exporting results.
(Ref: Churn Prediction screenshot)

4. Sentiment Analysis Module

RETENTION-AI analyzes customer feedback using NLP models and assigns sentiment
labels along with numerical sentiment scores. This helps businesses understand customer
emotions contributing to churn.
(Ref: Sentiment Analysis screenshot)

5. Explainable AI (XAI)

The Explainable AI module displays SHAP-based explanations for individual users. It

shows top influencing behavioral factors, impact scores, user summaries, and visual
charts illustrating how each feature affected the churn probability.
(Ref: Explainable AI screenshot)

6. Retention Strategies

This module provides actionable recommendations tailored to each user’s risk level.
Admins can select a user, choose from generated suggestions, customize the message, and
send retention emails directly through the interface.
(Ref: Retention Strategies screenshot)

Department of AI&ML, BIT 2024-25 69

Results Chapter 8

SNAPSHOTS

Fig 8.1 Dashboard Overview

This figure represents the main dashboard of RETENTION-AI, which serves as

the central interface for monitoring user engagement and churn behavior. The dashboard
provides a clear and intuitive overview of key performance indicators such as total users,
active users, churn rate, and overall retention rate, enabling stakeholders to quickly
assess the health of the user [Link] addition to numerical KPIs, the dashboard features
visual analytics including pie charts, bar graphs, and distribution plots that break down
churn patterns across different user segments. These visual insights help administrators
identify trends such as the proportion of high-risk users, variations in churn across time
periods, and sentiment-driven churn factors.

The interface is designed to be interactive and user-friendly, allowing admins or

analysts to explore detailed churn insights, filter data by segments, and understand the
underlying behavioral patterns that drive churn. By combining real-time metrics with
visual interpretability, the dashboard acts as an essential decision-support tool, helping
businesses take timely actions to improve user retention.

Department of AI&ML, BIT 2024-25 70

Results Chapter 8

Fig 8.2 Data Upload Interface

This screenshot highlights the data upload portal, where administrators can drag and drop
CSV files containing customer logs. It also provides required fields and format
guidelines.

Department of AI&ML, BIT 2024-25 71

Results Chapter 8

Fig 8.3 Churn Prediction Table

This page lists users alongside their churn risk and predicted probability. Users can filter,
sort, and search through 1029 predictions efficiently.

Fig 8.4 Sentiment Analysis Page

The sentiment analysis module displays customer reviews and their sentiment labels. It
helps identify key emotional signals contributing to churn.

Department of AI&ML, BIT 2024-25 72

Results Chapter 8

Fig 8.5 Retention Strategy Panel

This interface presents recommended user-specific retention actions and allows sending
personalized emails to targeted users.

Fig 8.6 Explainable AI Insights

This figure shows SHAP explanations including key risk factors, bar plots, and a user
summary with churn risk justification.

Department of AI&ML, BIT 2024-25 73

Chapter 9

CONCLUSION, APPLICATIONS AND FUTURE

WORK
9.1 Conclusion
RETENTION-AI is a comprehensive AI-powered platform designed to detect,
interpret, and reduce user churn. By bringing together machine learning, sentiment
analysis, explainable AI, and automated retention strategies, the system provides an end-
to-end solution for churn management. It enables businesses to upload user data, analyze
churn probabilities, understand the behavioural and emotional triggers behind churn, and
take targeted action through personalized [Link] platform's modular design,
intuitive interface, and robust backend architecture ensure scalability, transparency, and
consistent performance. Through SHAP explainability, organizations gain clarity on why
specific users are at risk, making retention strategies more meaningful and effective.
RETENTION-AI ultimately empowers businesses to improve customer retention,
enhance engagement, and make informed decisions backed by data-driven insights.

9.2 Applications

RETENTION-AI can be applied across various industries where customer churn directly
impacts revenue and growth:

1. SaaS Platforms : Retention-AI helps subscription-based software platforms track user

engagement, feature usage, and decline patterns to proactively prevent churn. By
identifying high-risk users early, SaaS companies can intervene with personalized offers,
onboarding assistance, or premium feature trials. This enables improved customer
lifetime value and reduces monthly subscription losses.

2. Mobile Applications: Mobile apps benefit from Retention-AI through continuous

monitoring of user activity, interaction frequency, and session behaviour.

Department of AI&ML, BIT 2024-25 74

Conclusion, Application and Future Work Chapter 9

3. E-Commerce & Online Services: Retention-AI analyzes buying patterns, browsing

behavior, cart abandonment, and review sentiment to detect early signs of customer
dissatisfaction. E-commerce companies can use these insights to personalize
recommendations, improve user experience, and reduce drop-offs. This strengthens
customer loyalty and increases repeat-purchase rates.

4. Telecommunications & Subscription Services: Telecom and subscription providers

can utilize Retention-AI to predict churn based on usage frequency, data consumption,
service complaints, and customer feedback. The system highlights at-risk users and the
factors influencing their dissatisfaction, enabling targeted retention campaigns. This
reduces service cancellations and boosts customer satisfaction.

5. Digital Marketing & Customer Support: Marketing teams can leverage Retention-AI
for targeted outreach campaigns by identifying customers most at risk of leaving.
Customer support teams can use churn insights and sentiment data to address issues
before they escalate. Together, these insights help create personalized, data-driven
communication strategies that improve retention.

9.3 Limitations of the System

1. Static Dataset Usage: The current system relies on uploaded CSV files rather than
real-time data streams, which limits its ability to reflect sudden changes in user behavior.
Because predictions are based on periodic updates, continuous monitoring is not yet
possible. This reduces responsiveness in rapidly changing environments.

2. Limited Domain Adaptability: Although Retention-AI works well for general user
churn prediction, different industries may require customized features or behavior
metrics. Highly specialized domains like healthcare or finance might not fit the default
feature set. This necessitates domain-specific tuning and retraining of models.

3. Sentiment Model Constraints: The sentiment analysis component may struggle with
sarcasm, slang, or complex multilingual expressions. This can lead to misclassification of

user reviews or feedback. As a result, certain emotional cues may not be captured
accurately, affecting overall prediction quality.

Department of AI&ML, BIT 2024-25 75

4. Computational Cost: Generating SHAP-based explanations for large datasets or
thousands of users requires significant processing time. Without GPU optimization,
explanation generation may slow down the system during peak operations. This can
impact real-time user experience for explainable insights.

5. No Automated Multi-Channel Outreach: Currently, Retention-AI supports email-

based retention strategies only, limiting communication flexibility. Channels like SMS,
push notifications, and WhatsApp are not yet integrated. This restricts the effectiveness of
personalized outreach campaigns for diverse user groups.

9.4 Future Scope of the Project

1. Real-Time Data Streaming: Future versions can integrate real-time event tracking
pipelines using tools like Kafka or AWS Kinesis. This will allow instant churn updates
whenever user behaviour changes. Such real-time predictions significantly improve
responsiveness and retention accuracy.

2. Graph-Based User Network Analysis: Introducing network analysis will help

identify peer influence, community behaviour, and clusters of users likely to churn
together. This provides deeper behavioral understanding beyond individual predictions. It
also helps design group-specific retention strategies.

3. Advanced Multilingual Sentiment Models: Future upgrades can include transformer-

based NLP models capable of understanding multiple languages and cultural expressions.
This improves accuracy for global applications across regions. It also strengthens
sentiment insights for international user bases.

4. Cloud Deployment for Scalability: Deploying the system on AWS, Azure, or GCP
will support large-scale churn monitoring across thousands of users in parallel. Cloud
services also offer automated scaling, faster processing, and high reliability.

5. Multi-Channel Retention Actions: Integrating SMS, push notifications, WhatsApp,

and in-app messaging will enable more personalized and immediate communication.
Businesses can reach users through their preferred channels, increasing the effectiveness
of retention strategies. Automated triggers can further streamline outreach.

Department of AI&ML, BIT 2024-25 76

6. Automated A/B Testing: The platform can later include built-in A/B testing to
compare the effectiveness of different retention strategies. This helps businesses
understand which messages or interventions work best. Continuous experimentation leads
to optimized campaigns and improved user engagement.

Department of AI&ML, BIT 2024-25 77

 REFERENCES
[1] Usman Gani Joy, Kazi Ekramul Hoque, Mohammed Nazim Uddin, Linkon
Chowdhury, Seung-Bo Park. ―A Big Data-Driven Hybrid Model for Enhancing
Streaming Service Customer Retention Through Churn Prediction Integrated with
Explainable AI‖, IEEE Access, Vol. 12, pp. 69130–69149, 2024.
[2] Upma Singh, Sandeep Singh, Tripti Rathee, Manav Vaish. ―Customer Retention
Modeling over the OTT Platform using Machine Learning‖, Indian Journal of
Science and Technology, Vol. 17, No. 42, pp. 1–9, 2024.
[3] Shuzlina Abdul-Rahman, Muhamad Faidi Akif Md Ali, Azuraliza Abu Bakar,
Sofianita Mutalib. ―Enhancing Churn Forecasting with Sentiment Analysis of
Steam Reviews‖, Social Network Analysis and Mining, Vol. 14, No. 178, pp. 1–
14, 2024.
[4] Lucio Ciabattoni, Marco Maiolatesi, Martina Mancinelli, Maria Di Tillo, Riccardo
Fiandra, Nicolò Gerosa, Lorenzo Trimeloni, Matteo Borghi, Massimo Bertolotti.
―Explainable AI Based Approach for Broadband Customers’ Churn Prediction‖,
IEEE Conference Proceedings, pp. 1–8, 2024.
[5] Carson K. Leung, Adam G.M. Pazdor, Joglas Souza. ―Explainable Artificial
Intelligence for Data Science on Customer Churn‖, IEEE International
Conference on Data Science and Advanced Analytics (DSAA), pp. 1–7, 2021.

[6] Suchita Sharma, Nishith Desai. ―Identifying Customer Churn Patterns Using
Machine Learning Predictive Analysis‖, Proceedings of the 3rd International
Conference on Smart Generation Computing, Communication and Networking
(SMART GENCON), IEEE, pp. 1–6, 2023.
[7] Kapil Arora, M. Lalitha, Poonam, Hemalatha Yadav J, Biswa Ranjan Mishra.
―Machine Learning for Customer Retention in E-Commerce Healthcare Startups‖,
South Eastern European Journal of Public Health (SEEJPH), Vol. XXV, Suppl.
2, pp. 4303–4308, 2024.
[8] Sabreen Abulhaija, Shyma Hattab, Ahmad Abdeen, Wael Etaiwi. ―Predicting
Mobile Apps Performance using Machine Learning‖, Journal of System and
Management Sciences, Vol. 12, No. 6, pp. 300–314, 2022.
[9] Manzura Jorayeva, Akhan Akbulut, Cagatay Catal, Alok Mishra. ―Machine
Learning-Based Software Defect Prediction for Mobile Applications: A
Systematic Literature Review‖, Sensors, Vol. 22, No. 7, pp. 1–17, 2022
[10] Zeynep Hilal Kilimci, Hasan Yörük, Selim Akyokus. ―Sentiment Analysis Based
Churn Prediction in Mobile Games using Word Embedding Models and Deep
Learning Algorithms‖, IEEE International Conference on Machine Learning and
Applications (ICMLA), pp. 1–8, 2020.
[11] Tamer Ahmed Ibrahim Abou El-Fotouh, Musliuende Toheeb Akanbi. ―Impact of
Predictive Analytics and Machine Learning on Customer Retention and Loyalty in
Service-Oriented Businesses‖, International Journal of Business Intelligence and
Big Data Analytics, Vol. 07, No. 03, pp. 1–11, 2024.
[12] Naragain Phumchusri, Phongsatorn Amornvetchayakul. ―Machine Learning
Models for Predicting Customer Churn: A Case Study in a Software-as-a-Service
Inventory Management Company‖, International Journal of Business Intelligence
and Data Mining, Vol. 24, No. 1, pp. 1–18, 2024.
[13] Carl Yang, Xiaolin Shi, Jie Luo, Jiawei Han. ―I Know You’ll Be Back:
Interpretable New User Clustering and Churn Prediction on a Mobile Social
Application‖, KDD, pp. 1–9, 2019.
[14] Ziru Liu, Shuchang Liu, Bin Yang, Zhenghai Xue, Qingpeng Cai, Xiangyu Zhao,
Zijian Zhang, Lantao Hu, Han Li, Peng Jiang. ―Modeling User Retention through
Generative Flow Networks‖, KDD, pp. 1–12, 2024.
[15] Mudassir Rafi, Md. Faiz Ahmad, Varshitha K, Siri Varsha T, Lahari K, Md.
Asadul Haque, Pavan Kumar Pagadala, Sushama Rani Dutta. ―Customer Churn
Prediction employing Ensemble Learning‖, IEEE 6th International Conference on
Cybernetics, Cognition and Machine Learning Applications (ICCMLA), pp. 207–
211, 2024.