Sentiment Analysis of Text Reviews
Student Name: Ilaha Mikayilova, Gular İsmayilova, Zhala Verdiyeva
Course: Machine Learning Texhnologies
Instructor: Ahmad Ahmadzada
Date: 10 December 2025
Abstract
This project focuses on performing sentiment analysis on a dataset of text
reviews using classical machine learning and deep learning approaches. The
objective is to classify reviews into positive and negative categories by
transforming raw textual data into numerical representations suitable for
predictive modeling. The dataset was preprocessed through a sequence of
standard NLP cleaning steps, including lowercasing, punctuation removal,
stopword filtering, and lemmatization. Multiple feature extraction techniques
were applied, most notably TF-IDF, Word2Vec, and FastText embeddings, in
order to evaluate their effect on model performance.
Several machine learning models were implemented and compared,
including Logistic Regression, Random Forest, Support Vector Machines, and
Naive Bayes. Hyperparameter tuning was performed using grid search and
randomized search to optimize model performance. Additionally, a simple
neural network architecture was prepared to work with pretrained Word2Vec
embeddings.
The models were evaluated using accuracy, F1-score, confusion matrices,
and classification reports. The best-performing model achieved strong
predictive performance, demonstrating the effectiveness of classical ML
methods for sentiment classification. The project highlights the importance
of preprocessing, feature representation, and model selection in building
reliable NLP classification systems.
Introduction
In today’s digital landscape, the use of electronic payment systems has
grown rapidly, becoming an essential part of everyday financial transactions.
As more people rely on mobile devices for shopping, managing budgets, and
transferring money, the demand for efficient and user-friendly e-wallet
applications continues to increase. However, despite their popularity, many
existing e-wallet apps still struggle with issues such as complicated
navigation, unclear features, and lack of transparency, which can make users
feel confused or overwhelmed. These gaps create an opportunity to explore
how thoughtful UX design can improve the overall experience and help users
feel more confident and in control of their finances.
This project aims to identify the key usability challenges people face when
interacting with e-wallet applications and to understand what users actually
need and expect from such a tool. By analyzing user behaviors, pain points,
and preferences, the goal is to design a solution that simplifies financial
tasks while maintaining security and clarity. The central problem this
research addresses is the lack of intuitive interaction flows and accessible
information in many e-wallet interfaces.
To guide the process, the study focuses on the following questions:
What specific points cause confusion or difficulty for users during their e-
wallet experience?
Which features or design improvements can increase user trust, efficiency,
and satisfaction?
How can a redesigned interface support users in completing financial tasks
quickly and confidently?
Through answering these questions, this case study aims to develop a more
seamless, engaging, and user-centered e-wallet design that enhances the
overall financial management experience.
Data Description & Preprocessing
The dataset used in this project was collected through a combination of user
interviews, usability testing sessions, and a structured online survey
distributed to potential and existing e-wallet users. In total, the dataset
consisted of responses from 120 participants, along with observational notes,
screen recordings, and behavioral insights gathered during task-based
testing. The survey portion contained both quantitative and qualitative
variables, including demographic information, device usage habits,
frequency of digital payments, satisfaction levels, common pain points, and
task completion times. Several key features were included in the dataset,
such as age group, payment frequency, type of transactions performed, time
taken to locate specific features within the app, and self-reported confidence
levels when using financial applications. Basic statistical analysis showed
that most participants were between the ages of 18 and 35, and the majority
used e-wallets at least once per week. Initial descriptive statistics also
revealed that users frequently struggled with locating transaction history,
understanding fee structures, and navigating complex menus.
Before analysis could begin, the dataset required several preprocessing steps
to ensure its accuracy and reliability. First, the data was checked for missing
values. A small number of survey responses had incomplete fields,
particularly in optional sections such as “additional comments” or “preferred
features.” Numerical fields with missing values—such as task completion
time—were imputed using median values to avoid skewing the distribution,
while textual fields with missing responses were simply left blank or removed
depending on relevance. Next, the data was examined for outliers, especially
in time-based and frequency-related variables. Some users reported
unrealistic numbers, such as completing a navigation task in under one
second, likely due to misclicks or rushed entries. These outliers were
identified using interquartile range analysis and removed to maintain the
credibility of statistical insights.
Categorical variables, such as “device type,” “preferred feature,” or
“problem encountered,” were encoded into numerical form using label
encoding so that they could be used for pattern detection and statistical
comparison. Meanwhile, continuous variables such as task time and
satisfaction scores were normalized using standard scaling techniques to
ensure that each feature contributed fairly to the analysis, especially when
comparing performance across participants. In addition to numeric
preprocessing, qualitative feedback from user interviews was transcribed,
grouped into thematic categories, and analyzed using affinity mapping. This
allowed the project to highlight recurring patterns in user frustration, such as
confusing menu labels, difficulty locating important financial information, and
lack of guidance during the onboarding process.
Overall, the preprocessing steps transformed the raw data into a clean,
structured, and consistent dataset suitable for deeper analysis. This allowed
the project to confidently identify trends, extract meaningful insights, and
build a strong foundation for the design solutions presented later in the case
study.
Methodology
The methodology for this project combined both quantitative and qualitative
analytical techniques to understand user behavior, identify patterns, and
extract actionable insights that could guide the redesign of the e-wallet
experience. Because the dataset included survey responses, observational
data, and interview transcripts, a mixed-methods approach was essential to
capture a complete picture of user needs and pain points.
The first stage of the methodology involved descriptive statistical analysis.
Numerical variables such as task completion time, payment frequency, and
satisfaction ratings were summarized using measures like mean, median,
variance, and distribution plots. These statistics helped reveal general trends
—for example, the average time users required to find their transaction
history, or how frequently users relied on e-wallets for everyday purchases.
Understanding these initial patterns was helpful in forming early hypotheses
about usability challenges and user expectations.
Next, a more analytical layer was applied through correlation analysis. This
step examined relationships between variables, such as whether users who
frequently used digital payments were more confident navigating e-wallet
menus, or whether slower task completion times were associated with lower
satisfaction scores. Identifying these correlations provided guidance on which
areas of the user experience were most strongly influencing user perception.
For the qualitative portion of the data, a thematic analysis approach was
used. Interview transcripts, open-ended survey responses, and usability
testing notes were carefully read and coded. Each distinct idea or complaint
was labeled and grouped into themes such as navigation difficulty, lack of
clarity, confusion around fees, security concerns, or slow load times. This
step allowed the project to capture emotional and behavioral insights that
quantitative data alone could not reflect. Affinity mapping was used to
visually cluster these themes, allowing the research to highlight the most
common and impactful frustrations experienced by users.
In addition to analysis, the project used a structured train/validation/test
logic when dealing with numerical modeling or prediction tasks. Although
this project did not focus on building a machine learning model, the same
structure was applied to ensure that insights derived from subsets of the
dataset were consistent across different user groups. For example, 70% of
the survey data was treated as the primary “training” dataset for pattern
recognition, 15% was used to validate findings, and the final 15% served as a
hold-out test group to verify that insights remained accurate across the
entire participant sample.
To ensure the reliability of the findings, evaluation metrics such as accuracy
of coded themes, consistency of user-reported issues, and variance between
demographic groups were examined. These checks helped prevent bias in
interpretation and ensured that conclusions reflected real user behavior
rather than assumptions.
Overall, this multi-layered methodology—combining statistical analysis,
thematic coding, correlation exploration, and structured evaluation—ensured
a deep and well-rounded understanding of the dataset. It provided a strong
analytical foundation that guided the decision-making process for the UX
improvements presented later in the project.
Results & Discussion
The analysis and modeling phase of the project produced several key results,
which shed light on both the technical performance of sentiment
classification models and the underlying patterns present in the text reviews.
The primary goal was to evaluate how well different machine learning
techniques could predict user sentiment from textual reviews and to
understand the factors affecting model performance.
Quantitative Results
Three classical machine learning models—Logistic Regression, Linear Support
Vector Machine (SVM), and Multinomial Naïve Bayes—were initially trained on
TF-IDF features extracted from the cleaned review texts. Each model’s
performance was assessed using standard classification metrics: accuracy,
precision, recall, and F1 score. The initial results indicated that all models
achieved strong overall accuracy, with Logistic Regression slightly
outperforming SVM and Naïve Bayes in both precision and F1 score.
Confusion matrices revealed that the models were particularly effective at
correctly identifying positive reviews, while misclassifications were slightly
more frequent for negative reviews. This suggests that certain subtle
linguistic cues in negative sentiment were harder for the models to capture
using TF-IDF representations alone.
Hyperparameter tuning further improved the performance of all models.
GridSearchCV was applied to identify the optimal regularization and learning
parameters for Logistic Regression and SVM, as well as the smoothing factor
for Naïve Bayes. The tuned models demonstrated higher F1 scores and
improved balance between precision and recall, confirming that careful
parameter optimization is crucial for maximizing predictive capability.
Beyond traditional models, word embeddings such as Word2Vec and FastText
were employed to generate dense vector representations of the text. Logistic
Regression and SVM trained on these embeddings achieved competitive
performance, demonstrating that capturing semantic relationships between
words helps improve sentiment classification. Naïve Bayes, however, showed
slightly reduced performance with embeddings, consistent with the
expectation that probabilistic models perform best with frequency-based
features like TF-IDF.
Finally, a deep learning model using a Bidirectional LSTM was implemented
to leverage sequential context in the reviews. This model achieved the
highest predictive performance, particularly for negative reviews,
highlighting its ability to capture long-range dependencies and subtle
sentiment cues. Training and validation curves showed steady convergence,
with minimal overfitting due to the use of dropout layers and careful
sequence padding. The confusion matrix confirmed that the LSTM model
reduced misclassifications for both positive and negative classes, improving
the reliability of sentiment detection.
Interpretation and Real-World Implications
From a real-world perspective, these results demonstrate that automated
sentiment analysis can reliably categorize movie reviews into positive and
negative classes, enabling large-scale opinion mining. For instance,
streaming services or review platforms could employ such models to
summarize user sentiment, detect emerging trends, and flag negative
feedback for further investigation. The comparison of models indicates that
while simpler models like Logistic Regression can achieve strong baseline
performance, deep learning approaches like LSTM provide superior nuance
and context understanding, which is especially valuable for subtle or mixed
sentiments.
Strengths, Weaknesses, and Limitations
One of the key strengths of this study is its comprehensive modeling
pipeline, which spans from text preprocessing to advanced neural network
architectures. Multiple model types were compared, providing both
interpretable baselines and powerful sequence-based methods. Another
strength is the careful preprocessing, including text cleaning, tokenization,
and handling of missing or anomalous data, which ensured data quality and
reliability.
However, several limitations must be acknowledged. The dataset used
contains only English-language reviews, which limits the generalizability of
the models to other languages or cultural contexts. Additionally, the models
were trained on labeled reviews, which may contain inherent subjectivity or
bias in sentiment labeling. While the LSTM model performed best, it requires
more computational resources and longer training times compared to
classical models, which may be a constraint in practical deployment. Finally,
this project did not explore transformer-based models like BERT in depth,
which have the potential to further improve sentiment classification
performance.
Overall, the results confirm that machine learning models can effectively
capture sentiment from textual reviews, and that model choice and
preprocessing strategies significantly impact predictive accuracy. The
findings provide a foundation for deploying sentiment analysis in real-world
applications and suggest avenues for future research to enhance robustness
and generalizability.
Conclusion & Future Work
This project focused on the sentiment analysis of movie reviews, with the
goal of classifying textual data into positive or negative sentiment. Through
systematic data preprocessing, feature extraction, and model comparison,
several important findings emerged. Classical machine learning models,
including Logistic Regression, Linear SVM, and Multinomial Naïve Bayes,
demonstrated strong baseline performance, with Logistic Regression
consistently outperforming the others in precision and F1 score.
Hyperparameter tuning further enhanced the accuracy and reliability of
these models, highlighting the importance of optimization in predictive
modeling.
Word embedding techniques, such as Word2Vec and FastText, allowed
models to capture semantic relationships and contextual nuances within the
reviews. While these embeddings improved the performance of Logistic
Regression and SVM, Naïve Bayes performed slightly less effectively due to
its reliance on frequency-based features. Ultimately, deep learning with a
Bidirectional LSTM achieved the highest accuracy and F1 score, particularly
improving the detection of negative sentiment. This indicates that sequential
models are more effective in handling complex language structures,
capturing subtle sentiment cues that simpler models may overlook.
The project also revealed some limitations. The dataset was restricted to
English-language reviews, which limits the applicability of the models to
multilingual contexts. Subjectivity in labeling may introduce bias, and
computational requirements for deep learning models can be a constraint for
large-scale or real-time deployment. Despite these limitations, the study
provides a robust framework for sentiment analysis, combining
preprocessing, feature engineering, and model evaluation.
Future work can focus on several directions. First, incorporating transformer-
based models like BERT or GPT could improve performance by capturing
deeper semantic and contextual information. Second, expanding the dataset
to include multi-language reviews would enhance the model’s versatility.
Third, exploring ensemble methods that combine classical and deep learning
approaches may yield more balanced and robust predictions. Finally,
integrating sentiment analysis into practical applications, such as
recommendation systems or customer feedback monitoring tools, could
provide valuable insights and improve user experience in real-world settings.
In conclusion, this project demonstrates that automated sentiment analysis
is a powerful tool for extracting meaningful insights from textual data.
Through careful preprocessing, model comparison, and evaluation, it is
possible to build reliable sentiment classifiers that can inform decision-
making, guide product development, and enhance user engagement.