• Manual Moderation: Relies on human volunteers B.

Data Processing
(“Mods”) who interact with the community. This is The extracted text data will be used as input to the detection
unscalable (a team of 5 cannot watch 50,000 users) pipeline. The system processes the raw text input through
and exposes moderators to psychological trauma from several steps including:
viewing graphic content. • Informal Syntax: Discord Messages are usually missing
• Keyword Filtering: The “block-list” approach. If a user punctuation and contain Emojis.
types “kill”, delete the message. This fails in contexts • Contextual Dependency: Messages are evaluated by
like video games (“I will kill the boss”) or when users themselves currently, but future work could include eval-
use Leetspeak (“k!11 y0u”). uating messages based on their previous messages.
• Monolingual Bias: A vast majority of AI moderation • Adversarial Attacks: Text that has been obfuscated (i.e.
tools (such as Facebook’s DeepText) have been enhanced “h a t e”) and homoglyphs needs to be processed.
for English language use and therefore work poorly in
After the text has been extracted, it will be passed to
diverse nations such as India with the use of hybrid
the machine learning service (ML Service) that is using a
dialects such as “Hinglish” and ”Kanglish” (ex. ”Tera
Uvicorn/FastAPI based machine learning classification engine
dimaag kharab hai”- meaning your brain may be faulty),
for the Text Classification for identification purposes.
these models fail to interpret.
III. W EB C ONTENT C LASSIFICATION T ECHNIQUES
D. Project Motivation and Vision
A hybrid classification model that is advanced has been
SafeGuard was designed specifically to fill those gaps. The implemented for the project where more than just using
idea behind SafeGuard is to develop a ”digital guardian” - an keywords to classify content is being performed as well as
AI agent that constantly monitors the chat, understanding the understanding the message context through semantic analysis.
underlying intent of messages that are sent through multiple
means and only intervenes when it is necessary to do so. A. Transformer Architecture Using BERT
Using Transfer Learning means that we do not have to Bidirectional Encoder Representation from Transformer net-
train massive neural networks from the ground up; rather, works (BERT), a primary model for the detection of toxic
we can repurpose an existing neural network (specifically comments. The platform utilizes Toxic-BERT, a fine-tuned
Google’s BERT and Facebook’s RoBERTa models) for use form of BERT, that has been specifically built for the detection
in the specific context of identifying chat toxicity in discord. of toxicity in comments through the use of Masked Language
Modeling (MLM) to provide the BERT model with context
E. Summary of Contributions
in its understanding of the text it processes when making
• Hybrid Detection Pipeline: A blending of rule-based predictions on comments. This model uses its prediction
constraints with Deep Learning probabilities in making of toxicity through the frame of Multi-Label Classification
the final detection decision. Methodologies, giving predictions of labels including:
• Multilingual Support: One of the few open-source bots • Toxic
to explicitly handle South Asian languages via transliter- • Obscene
ation detection. • Threat
• Microservices Deployment: A decoupled architecture • Insult
allowing both the Bot and the Brain (service for machine • Identity Hate
learning) to provide independent scalability.
Binary Cross-Entropy (BCE) loss is used during fine-tuning
• Granular Severity Logic: Moving beyond binary
to handle these independent labels.
“Toxic/Not Toxic” to a 4-tier risk assessment model.
B. Sentiment Analysis Using RoBERTa
II. W EB DATA E XTRACTION T ECHNIQUES
The system incorporates RoBERTa ( Robustly Optimized
From Discord, the project does processing with real-time BERT), which increases the accuracy of its emotion detection
data extraction. SafeGuard uses systems linked directly with capabilities by utilizing a sentiment score (i.e. Positive, Neu-
messaging services rather than web scraping via HTML or tral, and Negative) to provide context for any potential toxicity
DOM Tree Traversing. in the analyzed text. For example, a Toxicity rating would be
diminished by a Positive score in cases where the use of that
A. Discord API Integration
text could have been intended as a joke rather than as a form
The application captures a sequence of received messages of harassment.
M = {m1 , m2 , ..., mn } where each mi originates from an
individual user (uj ) with an associated timestamps (tj )). The C. Multilingual Heuristics (K-Nearest/Pattern Matching)
[Link] library makes it easy for the bot to listen to In languages like Hindi and Kannada that have limited
the stream of the events. resources, where large transformer models may not fully
The body of text in these messages will serve as the raw support these languages, we use a Dictionary-Trie based
source material for the extraction process. solution:
 • A curated list of threat root words (Kthreat ) is used. place, and messaging is rapid, asynchronous, and continuously
• Incoming messages are checked for substrings. changing culture.
• A keywords-based threat score is calculated based on the [32] Alfaleh et al. (2024) provide an overview of numerous
number of matches. artificial intelligence techniques that have been applied to the
This hybrid approach ensures robust detection even for code- detection of cyberbullying, with a primary focus on Support
mixed content (e.g., “Hinglish”). Vector Machines (SVM), Random Forests (RF), CNN’s, Long
Short Term Memory Networks (LSTM), BERT based models,
D. Severity Classification Framework (SCF) etc. By examining these approaches, they conclude that when
A final classifier aggregates outputs from the Toxicity hybrid or ensemble models are used together, the results
Model, Sentiment Analyzer, and Multilingual Detector to outperform using only one type of model. Furthermore, the im-
categorize messages into four severity levels: portance of performing good feature engineering and preparing
1) Low the text prior to applying any of the machine learning tech-
2) Medium niques utilized in building a scalable and efficient moderating
3) High pipeline using platforms such as scikit-learn, NumPy, and
4) Critical Pandas has been highlighted in their review.
Based on the studies reviewed in this section, it is apparent
IV. L ITERATURE S URVEY there is a trend toward creating context-based AI solutions
In light of recent developments in the field of research into for Cyberbullying Detection Systems. All the reviewed studies
Cyber bullying behaviour, it has become increasingly clear that emphasize support for Hybrid Architectures combining lan-
traditional techniques for moderating abusive behaviour online guage, emotion, and conversation-based features together to
are simply no longer effective, due to the evolving nature of create a solid foundation for the build and purpose of the
Cyberbullying Behaviour. system proposed.
A study conducted by [1] Nurse et al., in 2024, analysed
V. C OMPARISON OF W EB PAGE C LASSIFICATION
the patterns of online communication and has proven that
A PPROACHES
the expression of cyber bullies comes in many forms and is
often not as overt as people believe, ie through tone, repetitive (Using Performance Metrics from the Project)
comments and framing comments in an improper context The SafeGuard system was evaluated using stratified K-
as opposed to using blatant abusive language. [1] Nurse’s fold cross-validation on a test dataset of 200 messages (Be-
study has also identified several weaknesses in the keyword- nign, Toxic, Extremely Toxic, Code-Mixed). The performance
based moderation approaches and thus highlights the need for demonstrates the effectiveness of the Hybrid Transformer
the implementation of context focused Automated Detection approach compared to traditional methods.
Mechanisms that can identify and record the subtle interactions A. Model Evaluation Results
that cause harm.
[2] Bengali et al. (2025) reviewed several cyberbullying B. Inference Latency
detection approaches that employ AI and included a survey • Average Response Time: <5s
of the various AI Cyberbullying Detection methods, such as
VI. D ISCUSSION
Classic Machine Learning versus Deep Learning techniques
and Transformers. The review also highlighted several trends A. Successes
that are present in the field including the following: the devel- Discord has been effectively integrated into this system,
opment of deep neural networks and robotic frameworks based providing users with insight about any messages that are
on transformers; the implementation of Cyberbullying Detec- flagged for review using the !analyze command. User
tion approaches on multiple platforms; these Cyberbullying testing demonstrated that the feedback loop (ie. warning users)
Detection methods outperform the rule based approaches used reduces the incidence of additional toxic messages by creating
previously. Also highlighted are the ongoing issues being faced a “Cooling Off” period post Flag Message Detection.
within this domain; for example, there is a continued challenge
B. Failure Analysis
of model imbalance of training datasets and the inadequate
representation of the emotional context in datasets, indicating The system struggles with:
that more developed and flexible solutions are needed within 1) Obfuscation: “I w!ll k!ll y0u” is often caught, but “G0
the Cyberbullying Detection area. d!e” might be missed if the sub-word tokenizer splits it
[31] Yi and Zubiaga (2023) investigate the significance of strangely.
conversational context in a unique session-based cyberbullying 2) Neologisms: New slang terms (e.g., “Rizz” was initially
detection framework based on a series of messages rather than unknown to older models) require constant model up-
individual pieces of text, demonstrating how introducing con- dates.
textual data from multiple conversations can increase accuracy 3) Contextual Irony: “I’m going to kill you” said by a
significantly. This framework is especially suitable for instant teammate in a game after a friendly fire incident is
messenger services (e.g., Discord) where communication takes flagged as a Threat (Critical). This is a False Positive.
 TABLE I
C OMPARISON OF S AFE G UARD WITH E XISTING L ITERATURE

Year / Study Core Approach Key Capabilities Limitations Identified Advancement in SafeGuard
Used
Pre-2020 (Traditional Keyword filtering, Simple detection of explicit No context understanding, SafeGuard replaces static rules
Systems) rule-based systems abusive words high false positives, with context-aware AI models
monolingual
2022 – Emon et al. Transformer models Improved semantic Focused on single language, SafeGuard supports multilin-
for hate detection understanding offline analysis gual & real-time detection
2023 – Teng & ML vs Transfer Showed superiority of transfer No real-time deployment, no SafeGuard applies transfer
Varathan Learning comparison learning moderation actions learning in live Discord chats
2023 – Yi & Zubiaga Conversation-level Improved accuracy using mes- Computationally heavy, not SafeGuard prioritizes low-
context modeling sage sequences real-time latency single-message
inference
2024 – Nurse et al. Behavioral & contex- Identified failures of keyword No implementation of real- SafeGuard operationalizes
tual analysis filters time system these insights in a working
system
2024 – Sharma et al. Multilingual hate de- Multi-language support Limited to classification, no SafeGuard adds severity-based
tection models enforcement logic automated actions
2024 – SemEval Transformer-based Code-mixed language Research-focused, no deploy- SafeGuard integrates models
(Yenumulapalli et al.) emotion classification handling ment pipeline into a deployable platform
2024 – Alfaleh et al. Hybrid & ensemble Improved accuracy via model No system-level architecture SafeGuard implements a full
ML techniques combinations hybrid AI pipeline
2025 – Bengali et al. Survey of AI modera- Identified transformer domi- Highlighted dataset imbalance SafeGuard mitigates gaps us-
tion systems nance & context gaps ing sentiment + threat heuris-
tics
2025 – Recent Sur- Explainable & ethical Focus on transparency Mostly theoretical SafeGuard includes human-in-
veys AI moderation the-loop dashboard
Proposed System – Hybrid Transformer Real-time, multilingual, Context window limited to sin- Bridges research and real-
SafeGuard (2025) + Rule-Based Sys- severity-aware moderation gle message (current version) world deployment effectively
tem

TABLE II R EFERENCES
M ODEL E VALUATION R ESULTS

Class Precision Recall F1-Score Support [1] J. R. C. Nurse, S. Creese, A. Goldsmith, and K. Lamberts, “Online
Clean 0.98 0.99 0.99 100 Harms: Cyberbullying Detection Using Digital Behaviour Patterns,”
Toxic 0.94 0.96 0.95 50 Cybersecurity, vol. 10, no. 1, 2024.
Threat 0.88 0.92 0.90 25
Identity Hate 0.91 0.85 0.88 15 [2] H. Bengali, A. Serhani, and M. El Menshawy, “AI-Driven Cyberbullying
Overall 0.95 0.96 0.95 200 Detection Across Social Media Platforms: A Comprehensive Survey,”
Future Internet, vol. 17, no. 2, pp. 1–23, 2025.

[3] S. Mali et al., “Advancing cyberbullying detection in low-resource

languages: a transformer-stacking framework for Bengali,” Frontiers in
The sentiment dampener helps, but context (previous 5 Artificial Intelligence, vol. 8, 2025.
messages) is the real solution. [4] V. O. Yenumulapalli et al., “TECHSSN1 at SemEval-2024 Task 10:
Emotion Classification in Hindi-English Code-Mixed Dialogue using
Transformer-based Models,” in Proc. 18th Int. Workshop Semantic
Evaluation (SemEval-2024), 2024, pp. 145–152.
VII. C ONCLUSION
[5] D. Sharma et al., “EnsMulHateCyb: Multilingual hate speech and
SafeGuard effectively illustrates how state-of-the-art Large cyberbully detection in online social media,” Expert Syst. Appl., vol.
236, p. 121284, Feb. 2024.
Language Models can be used to provide real-time Community
Moderation. By providing a highly accurate and recallable [6] J. Raymond and M. Festus, “Multilingual Cyberbullying Detection with
automated detection system for toxicity, SafeGuard meets a Federated NLP on Encrypted Platforms,” ResearchGate, Tech. Rep.,
Jan. 2025.
significant need for providing safer online environments. In ad-
dition, by bridging the divide between AI research conducted [7] R. Shrestha and R. Dave, “Machine Learning for Identifying Harmful
primarily in the English language and the multilingual nature Online Behavior: A Cyberbullying Overview,” J. Comput. Commun.,
vol. 13, no. 1, pp. 45–62, Jan. 2025.
of the internet through a hybrid architecture, SafeGuard serves
as a model for future development of intelligent, inclusive [8] “A Novel Hybrid Attention-Based RoBERTa-BiLSTM Model for Cy-
moderation tools. SafeGuard offers evidence that ethical AI berbullying Detection,” MDPI Electron., vol. 14, no. 3, 2025.
can help to improve the well-being of humans by alleviating [9] “Bias and Cyberbullying Detection and Data Generation Using Trans-
former Artificial Intelligence Models: The CyberBulliedBiasedBot,”
the psychological load on human moderators and positively MDPI Electron., vol. 13, no. 17, 2024.
affecting digital interactions.
[10] A. Akter et al., “Cyberbullying Detection on Social Media Platforms [29] S. Mishra, P. Patwa, and A. Das, “A Survey on Multilingual and
Utilizing Different Machine Learning Approaches,” Int. J. Comput. Code-Mixed Offensive Language Detection,” Artificial Intelligence
Appl., vol. 186, no. 61, pp. 22–30, 2025. Review, vol. 57, pp. 1–45, 2024.

[11] A. Cuzzocrea, “Cyberbullying Detection, Prevention, and Analysis on [30] A. R. Founta et al., “Challenges and Solutions in Automated Moderation
Social Media Using LSTM-autoencoder Networks,” Future Internet, of Online Platforms,” IEEE Internet Computing, vol. 28, no. 2, pp.
vol. 17, no. 2, 2025. 52–61, 2024.

[12] M. I. H. Emon, K. N. Iqbal et al., “Detection of Bangla Hate Comments [31] P. Yi and A. Zubiaga, “Session-based cyberbullying detection in social
and Cyberbullying in Social Media Using Transformer Models,” in media: A survey,” Online Social Networks and Media, vol. 36, p.
Proc. Advances in Computing and Data Sciences, Springer, 2022. 100250, Jul. 2023, doi: 10.1016/[Link].2023.100250.

[13] V. Susmitha, J. Nagarani and P. Lavanya, “Detection of Cyber Bullying [32] K. Alfaleh, A. Alabdultif, and S. Aladhadh, “Artificial Intelligence-
on Social Media Using Machine Learning Algorithms,” International Driven Cyberbullying Detection: A Survey of Current Techniques,”
Journal of Novel Research and Development, vol. 9, no. 5, May 2024. Journal of Information Technology Management, vol. 16, no. 4, pp. 38–
63, 2024, doi: 10.22059/jitm.2024.99050.
[14] “Machine Learning for Identifying Harmful Online Behavior and
Cyberbullying,” Journal of Computer and Communication, 2025.

[15] F. R. Sayed, “Cyberbullying Detection in Social Media Using Natural

Language Processing and Machine Learning Models,” Social Network
Analysis and Mining, 2025.

[16] T. H. Teng and K. D. Varathan, “Cyberbullying Detection in Social

Networks: A Comparison Between Machine Learning and Transfer
Learning Approaches,” IEEE Access, vol. 11, pp. 55533–55560, 2023.

[17] S. Sihab-Us-Sakib, “Cyberbullying Detection of Resource Constrained

Language Social Media,” Computer Science Review, 2024.

[18] G. Jaradat, “Deep Learning Approaches for Detecting Cyberbullying on

Social Media,” Journal of Computer and Communication Engineering,
2025.

[19] A. Perera and P. Fernando, “Cyberbullying Detection System on Social

Media Using Supervised Machine Learning,” Procedia Computer
Science, 2024.

[20] S. Unnava, “A Study of Cyberbullying Detection and Classification

Using Machine Learning Techniques,” European Transactions on
Telecommunications and Related Technologies, 2024.

[21] S. Chen, “Chinese Cyberbullying Detection Using XLNet and Machine

Learning Techniques,” Information, vol. 15, no. 2, 2024.

[22] Silvia Garcı́a-Méndez and Francisco De Arriba-Pérez, “Promoting

Security and Trust on Social Networks: Explainable Cyberbullying
Detection Using Large Language Models,” arXiv preprint, 2025.

[23] Y. Zhang, L. Wang, and H. Liu, “Context-Aware Cyberbullying

Detection Using Transformer-Based Language Models,” IEEE
Transactions on Computational Social Systems, vol. 11, no. 1, pp.
88–99, 2024.

[24] M. Al-Garadi et al., “Explainable Artificial Intelligence for Online

Abuse and Cyberbullying Detection,” ACM Computing Surveys, vol.
56, no. 4, pp. 1–36, 2024.

[25] A. Kumar and R. Mehta, “Real-Time Toxic Speech Detection in Social

Media Using Deep Learning,” Applied Soft Computing, vol. 148, p.
111005, 2024.

[26] P. Basile et al., “SemEval-2023 Task 10: Explainable Detection of

Online Abuse,” in Proceedings of the 17th International Workshop on
Semantic Evaluation, 2023, pp. 1–14.

[27] R. Vidgen and T. Derczynski, “Directions in Abusive Language Training

Data, a Systematic Review,” Language Resources and Evaluation, vol.
58, no. 1, pp. 1–29, 2024.

[28] N. Plaza-del-Arco et al., “Multilingual and Code-Mixed Hate Speech

Detection Using Transformer Models,” Information Processing &
Management, vol. 61, no. 2, 2024.