Sentiment Analysis of Cyberbullying Data in Social Media

Pradeep Pujari∗ Arvapalli Sai Susmitha∗

EleutherAI / USA IIT Kanpur / India
pkpujari@[Link] arvapallisaisusmitha@[Link]

Abstract concerning in educational settings (Xu et al., 2012).

The tragic case of Megan Meier, a young victim of
Social media has become an integral part
harassment on MySpace, highlights the devastating
of modern life, but it has also brought with
it the pervasive issue of cyberbullying a se-
impact of cyberbullying (Vines, 2015). Given the
arXiv:2411.05958v1 [[Link]] 8 Nov 2024

rious menace in today’s digital age. Cyber- growing concern in society and workplaces, and
bullying, a form of harassment that occurs the fact that studies show many students face cy-
on social networks, has escalated alongside berbullying (Li, 2006; Cross, 2008), detecting and
the growth of these platforms. Sentiment preventing cyberbullying promptly is essential.
analysis holds significant potential not only
In this work, we use Sentiment Analysis (SA) to
for detecting bullying phrases but also for
identifying victims who are at high risk of classify texts as positive or negative with respect
harm, whether to themselves or others. Our to cyberbullying. While traditional embeddings
work focuses on leveraging deep learning used in Recurrent Neural Networks (RNNs) are ef-
and natural language understanding tech- fective in capturing sequential patterns, they strug-
niques to detect traces of bullying in social gle with contextual dependencies in longer sen-
media posts. We developed a Recurrent Neu- tences and have limitations in understanding com-
ral Network with Long Short-Term Mem- plex language structures (Elman, 1990; Pascanu,
ory (LSTM) cells, using different embed-
2013).In contrast, we incorporate state-of-the-art
dings. One approach utilizes BERT embed-
dings, while the other replaces the embed- Large Language Model (LLM) embeddings, specif-
dings layer with the recently released em- ically BERT embeddings (Kenton and Toutanova,
beddings API from OpenAI. We conducted 2019), and use the OpenAI API to obtain embed-
a performance comparison between these dings that leverage self-attention mechanisms for
two approaches to evaluate their effective- sentiment analysis. These models excel in captur-
ness in sentiment analysis of Formspring Cy- ing intricate language features, making them highly
berbullying data. Our Code is Available at :
effective for a range of downstream tasks (Radford
[Link]
et al., 2019; Raffel et al., 2020). These embeddings
1 Introduction offer a deeper contextual understanding compared
to earlier models, allowing for a more nuanced
In recent years, social media has become an integral detection of harmful content and a more precise
part of daily life, facilitating communication and evaluation of sentiment polarity. The task involves
connection on an unprecedented scale. However, identifying sentences containing bullying tokens,
it has also contributed to a rise in cyberbullying assessing their polarity, and accurately classifying
harassment conducted online (Feinberg and Robey, them into the relevant sentiment categories.
2009). Unlike traditional bullying, cyberbullying
The development of such Natural Language
can happen anytime, with online anonymity encour-
Processing (NLP) algorithms, however, requires
aging perpetrators to act without facing immediate
high-quality annotated data to measure perfor-
consequences (Hasan et al., 2023). This anonymity
mance accurately. Many popular Machine Learn-
often leads to higher rates of cyberbullying com-
ing (ML) techniques, especially Deep Neural Net-
pared to in-person bullying. Bullying is recognized
works (DNNs), need large, annotated corpora to
as a major health issue by institutions like the Amer-
achieve high-quality classification results. Un-
ican Academy of Pediatrics, has become especially
fortunately, the availability of publicly accessible
0∗
Equal contribution datasets for cyberbullying detection is limited. In
this work, we use Kaggle’s Formspring data (Pujari, machine learning systems (Joachims, 1999; Mc-
2022; Reynolds et al., 2011) for Cyberbullying De- Callum and Nigam, 1998). These methods im-
tection to train and evaluate our models. We apply proved flexibility but still faced challenges with
the advanced LLM-based techniques and assess the complex sentiment detection and domain general-
model’s performance in identifying cyberbullying. ization. Feature selection and careful tuning were
In this work, we made the following contribu- often necessary to optimize performance.
tions:
Around the same time, distributed representa-
• Introduce two distinct hybrid methods for sen- tion methods like Word2Vec(Mikolov et al., 2013)
timent analysis in cyberbullying detection, and GloVe(Pennington et al., 2014) were intro-
leveraging advanced embeddings instead of duced, representing words in continuous vector
traditional techniques. spaces to capture their semantic relationships. Un-
like TF-IDF’s sparse features, word embeddings
• Utilizes BERT embeddings with an RNN encode words based on context, enabling more
framework in one method and OpenAI em- effective sentiment classification. Word2Vec of-
beddings with an RNN framework in another, fered two model architectures, Continuous Bag-
specifically for sentiment analysis of cyberbul- of-Words (CBOW) and Skip-Gram, both of which
lying data, aiding in cyberbullying detection. generated word embeddings by considering context
words. GloVe took a different approach by using a
• Compare the performance of the proposed global word co-occurrence matrix to create word
hybrid architectures using BERT and Ope- embeddings, capable of capturing linear relation-
nAI embeddings for sentiment analysis on the ships within the vector space. (Yin et al., 2009)
Formspring dataset. utilized a traditional bag-of-words model with sen-
timent and contextual features to boost detection
2 Literature review performance, while (Nahar et al., 2014) developed
a semi-supervised fuzzy SVM approach incorpo-
In the early stages of sentiment analysis, tradi-
rating sentiment and user metadata.
tional methods such as rule-based and lexicon-
based systems were widely used. Rule-based sys- While these word embeddings revolutionized the
tems applied predefined linguistic rules to clas- way sentiment data was processed, they were still
sify text, while lexicon-based methods relied on largely task-independent. In sentiment analysis,
sentiment lexicons like AFINN and SentiWord- this posed challenges since word meanings could
Net(Baccianella et al., 2010), which assigned sen- vary drastically depending on the context or do-
timent scores to individual words. These scores main. To address this, more advanced deep learn-
were then aggregated to determine the overall sen- ing models, such as Recurrent Neural Networks
timent of a sentence or document. While these (RNNs) and Long Short-Term Memory (LSTM)
approaches were straightforward, they struggled to networks, were introduced to capture sequential
capture context-dependent meanings and often re- dependencies in text (Hochreiter and Schmidhu-
quired extensive, manually curated lexicons. Simi- ber, 1997). This enabled better handling of long-
larly, pattern-based approaches like those described range context, a critical factor in sentiment anal-
by (Pang and Lee, 2008) utilized syntactic and se- [Link] paper (Sahoo et al., 2023) discusses the
mantic rules to detect sentiment-bearing structures use of RNNs, LSTMs, and transformer models in
within text, but they too required significant manual sentiment analysis, highlighting data preprocess-
effort to maintain. ing, feature extraction, and model architectures.
With the rise of machine learning, traditional fea- Convolutional Neural Networks (CNNs) were also
ture engineering techniques became popular. Meth- adapted to text processing, particularly excelling
ods like TF-IDF (Term Frequency-Inverse Docu- at capturing sentiment-bearing phrases and struc-
ment Frequency) (Shi and Li, 2011) transformed tures(Kim, 2014). A study by (Dang et al., 2020),
text data into meaningful features for sentiment compares different deep learning models such as
classification, marking a shift from rule-based ap- DNN, RNN, and CNN for sentiment analysis tasks,
proaches. Classification algorithms such as Naive including sentiment polarity and aspect-based sen-
Bayes, Support Vector Machines (SVM), and Lo- timent analysis. The paper (Sharma et al., 2024)
gistic Regression gained popularity in these early explores the significance, challenges, and evolving
methodologies in sentiment analysis, emphasizing BERT embeddings(base model) with Recurrent
the dynamic nature of the field. These studies col- Neural Networks (RNNs), while the other com-
lectively provide a robust understanding of how bines OpenAI API embeddings(text-embedding-3-
deep learning techniques can be leveraged for ef- small) with RNNs. Each hybrid model harnesses
fective sentiment analysis. the rich contextual insights of its respective em-
Several studies have explored various ap- bedding method BERT or OpenAI while taking
proaches for cyberbullying detection, leveraging advantage of RNN’s ability to model the sequential
different models and techniques. (Raj et al., 2021) nature of abusive language. This approach pro-
demonstrated the use of Bi-GRU and Bi-LSTM for vides a comprehensive and robust framework for
high classification accuracy, while (Dewani et al., detecting cyberbullying across multiple posts.
2021) analyzed LSTM, BiLSTM, RNN, and GRU
3 Formspring Dataset
for detecting antisocial behavior. A hybrid neural
network combining RNN for text and CNN for im- The Formspring dataset (Reynolds et al., 2011),
ages was proposed by (Agbaje and Afolabi, 2024) curated by Kelly Reynolds and made available on
to detect cyberbullying from tweets. (Iwendi et al., Kaggle(Pujari, 2022), is one of the primary datasets
2020) applied four deep learning models Bidirec- used for sentiment analysis-based cyberbullying
tional LSTM (BLSTM), GRU, LSTM, and RNN detection. [Link], a now-defunct question-
for cyberbullying detection. The Hybrid Recurrent and-answer-based social network, allowed users to
Residual Convolutional Neural Network (HRecR- post questions and comments anonymously, mak-
CNN) integrating sentiment analysis features was ing it a significant source for studying online bully-
introduced by (T.T and Jeetha, 2021). Sentiment ing behaviors. The dataset was collected through a
analysis was also leveraged by (Atoum, 2020) us- crawl of the site in Summer 2010.
ing SVM and Naïve Bayes for harmful content The dataset contains 12,772 samples of user
classification, and later enhanced by (Atoum, 2021) posts and answers, where each post was labeled as
with CNNs for improved feature extraction. either containing cyberbullying or not. These labels
The introduction of transformer-based models were generated through Amazon Mechanical Turk,
like BERT and GPT revolutionized sentiment anal- with each post annotated by three workers. A post
ysis and cyberbullying detection. BERT, intro- was classified as cyberbullying if at least two of the
duced by (Devlin et al., 2019), utilized bidirectional three annotators indicated the presence of bullying.
attention mechanisms to capture nuanced relation- Out of the 12,772 samples, 802 samples (6.3%)
ships between words in both forward and back- were labeled as cyberbullying, which mirrors the
ward contexts, making it particularly effective for real-world imbalance between non-bullying and
sentiment classification tasks. GPT, developed by bullying content (with over 84% of the content
(Radford et al., 2018), employed a unidirectional labeled as non-bullying).
architecture and excelled in tasks like text genera- Key fields in the dataset include:
tion and understanding. These models significantly • Userid: ID of the person responding to a post.
outperformed earlier techniques like Word2Vec and
GloVe, thanks to their ability to capture deep con- • Asker: ID of the person asking the question.
textual information. However, they also introduced
challenges in terms of computational complexity • Post: The complete question and answer
and resource requirements (Vaswani et al., 2017). string, separated by markers such as "Q:" and
"A:". As part of preprocessing, these mark-
The adoption of transformer-based models like ers, along with any HTML encodings, were
BERT has not only advanced sentiment analysis removed.
but also cyberbullying detection, as these models
provide superior text classification by understand- • Ans#, severity#, bully#: The responses pro-
ing the subtle nuances of human language espe- vided by Mechanical Turk workers. Ans#
cially crucial for identifying abusive language on refers to the answer given by the annotators,
social media platforms. Our approach builds upon severity# indicates a score from 0 to 10 based
these advancements by leveraging two distinct hy- on the perceived intensity of the bullying, and
brid architectures for sentiment analysis-based cy- bully# identifies the specific words or phrases
berbullying detection. One architecture integrates flagged as bullying.
 The dataset contains approximately 300,000 to-
kens. Harmful posts are slightly shorter on average,
with 23 words per post, compared to non-harmful
posts, which have about 25 words. Although the
dataset is not particularly large by current machine
learning standards, it remains a valuable source for
studying cyberbullying due to its focus on anony-
mous interactions, which often encourage harmful
behaviors.
The annotation process using Mechanical Turk Figure 1: OpenAI API Embedding genration
introduced some inconsistency in the labeling.
Analysis suggests that 2.5% of posts classified as
non-cyberbullying likely contain harmful content, 4 Models
and 15-20% of posts labeled as cyberbullying
could be considered benign. Ideally, the dataset 4.1 BERT Embeddings
should be re-annotated for higher accuracy, but
due to time constraints, the original labels were BERT (Bidirectional Encoder Representations
used in this work. from Transformers) embeddings are employed to
extract contextual word representations. Unlike
Preprocessing steps included: static word embeddings like Word2Vec, where
each word has a fixed vector, BERT (Kenton and
• Handling Missing Values: Replace ’None’ Toutanova, 2019) generates dynamic representa-
values with NaN and map missing values in tions for each word based on its context. This
answer-related columns to 0 (indicating no allows for improved performance in NLP tasks, in-
bullying found). cluding cyberbullying detection. We used BERT
base which has 12 encoder layers, 768 hidden units,
• Handling Corrupted User IDs: Drop rows 12 attention heads.
where Userid is corrupted and occurs fewer These transformer-based models outperform tradi-
than 3 times. tional methods by leveraging self-attention mech-
anisms and contextual [Link] attention
• Removal of HTML Tags: Remove all HTML
mechanism isT defined as Attention(Q, K, V ) =
tags and encoded characters such as <br>.
QK
softmax √d V , where Q, K, and V are query,
k
• Normalization of Repeated Characters: key, and value matrices, and dk is the dimension of
Normalize words with exaggerated repetitions the key.
(e.g., goooood becomes good).

These are a few preprocessing steps that ensured 4.2 OpenAI Embeddings
cleaner data for model training. Furthermore, the
The OpenAI embedding API introduces an end-
dataset’s target class was not annotated.
point that maps text and code to vector represen-
tations. We use text-embedding-3-small (OpenAI,
Logic to annotate the target as follows:
2024), which are effective for tasks like clustering,
• If two answers were identical and the severity topic modeling, and classification, outperforming
score was greater than zero, we classified the many traditional models (OpenAI, 2023). These
data as ’Negative’, indicating cyberbullying. embeddings achieved a 20% relative improvement
If these conditions were not met, we classified in code search tasks (Neelakantan et al., 2022).
the data as ’Positive’, indicating non-bullying. The embeddings encode semantic relationships be-
tween words, phrases, or code snippets and can
• The dataset was then converted into numerical be obtained via a simple API request, as shown in
classes: No Bullying (Positive) = 1, Bullying Fig1. This encoding allows semantically similar
(Negative) = -1. inputs to cluster in vector space, making them use-
ful for similarity search, grouping, or automated
labeling.
5 Experimentation Implementation Details:
The data was split into an 80:20 ratio for training
Algorithm: Sentiment Analysis using BERT or
and testing. We used a learning rate of 0.001 and
OpenAI Embeddings in RNN
applied gradient clipping to prevent exploding gra-
1. Input: dients, and dropout to mitigate overfitting. The
Input text T = {t1 , t2 , . . . , tn }, where ti rep- Adam optimizer was employed for the training pro-
resents each token in the input sentence. cess, which was run for 10 epochs.

2. Step 1: Tokenization
Tokenize the input text T using:
• BERT: Use BERT’s tokenizer
to obtain subword tokens
Tsub = {s1 , s2 , . . . , sm }.
• OpenAI: Input the raw text T directly to
OpenAI’s embedding API.

3. Step 2: Embedding Extraction Figure 2: RNN network with BERT/OpenAI API em-
Extract contextual embeddings using: beddings
• BERT: Pass tokenized text Tsub through
a pre-trained BERT model to obtain em- 6 Results
beddings E = {e1 , e2 , . . . , em }.
• OpenAI: Use OpenAI’s API to get high- We observed that the embedding representations,
dimensional vector embeddings E = both BERT and OpenAI, were rich and dense
[Link](input = T ). with information, making them well-suited for the
task of sentiment analysis in cyberbullying detec-
4. Step 3: Classification Model tion. However, OpenAI embeddings exhibited
Input the embeddings E into a RNN model to marginally better performance than BERT embed-
perform the classification task. dings.
The primary metrics used to evaluate model per-
5. Step 4: Output
formance in this context were accuracy and macro
The final output is the predicted class ypred
F1 score. Accuracy is calculated as the ratio of
with the highest probability.
correctly predicted instances to the total number of
instances in the dataset, defined by the following
The model is trained using Binary Cross formula:
Entropy Loss (BCELoss), which is suitable for
binary classification tasks. Number of Correct Predictions
Accuracy = ×100
Total Number of Predictions
The BCELoss is defined as
The macro F1 score provides an average F1 score
L = (y log(p) + (1 − y) log(1 − p)) across all classes, giving equal weight to each class.
Since we have two classes, N = 2. The macro F1
where y is the true label and p is the predicted score is calculated as follows:
probability for the positive class.
N
Incorporating BERT-base or OpenAI embed- 1 X 2 × Precisioni × Recalli
Macro F1 =
dings, i.e., our hybrid architectures, enhances the N Precisioni + Recalli
i=1
performance of the classifier compared to using
only the RNN. This improvement is achieved by where Precisioni and Recalli are the precision and
providing high-quality feature vectors extracted recall for each class i. The results are summarized
from the text data. Fig 2 shows the RNN network below:
with BERT or OpenAI embeddings. These results indicate that employing more ad-
vanced contextual embeddings, such as those from
 Model Accuracy Macro F1 References
Basic RNN 51% 38%
BERT-base Hybrid (ours) 56% 51% Michael Agbaje and Oreoluwa Afolabi. 2024. Neural
network-based cyber-bullying and cyber-aggression
OpenAI Hybrid (ours) 80% 79%
detection using twitter(x) text. Revue d’Intelligence
Artificielle.
Table 1: Results of different embeddings
Jalal Omer Atoum. 2020. Cyberbullying detection
through sentiment analysis. 2020 International Con-
OpenAI, can lead to significant improvements in ference on Computational Science and Computa-
performance. The enhanced accuracy of OpenAI tional Intelligence (CSCI), pages 292–297.
embeddings suggests that they are better at captur-
Jalal Omer Atoum. 2021. Cyberbullying detection neu-
ing the complexities and nuances of language in ral networks using sentiment analysis. 2021 Inter-
cyberbullying contexts, making them a valuable as- national Conference on Computational Science and
set for future research and development in this area. Computational Intelligence (CSCI), pages 158–164.
Overall, this analysis underscores the importance
Stefano Baccianella, Andrea Esuli, Fabrizio Sebastiani,
of embedding choice in sentiment analysis tasks, et al. 2010. Sentiwordnet 3.0: an enhanced lexical
especially in sensitive domains like cyberbullying resource for sentiment analysis and opinion mining.
detection. In Lrec, volume 10, pages 2200–2204. Valletta.

7 Conclusion D Cross. 2008. Cyberbullying: International compar-

isons, implications, and recommendations. In 20th
An in-depth analysis of the Formspring dataset biennial meeting of the International Society for the
highlights significant opportunities for improving Study of Behavioural Development, Wurzburg, Ger-
many.
cyberbullying detection through advanced method-
ologies. Our study introduced a novel hybrid ap- Nhan Cach Dang, María N. Moreno García, and Fer-
proach for sentiment analysis, integrating BERT nando de la Prieta. 2020. Sentiment analysis based
and OpenAI embeddings within an RNN frame- on deep learning: A comparative study. ArXiv,
abs/2006.03541.
work to address the complexities of cyberbullying
data. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and
Our findings revealed that both OpenAI and Kristina Toutanova. 2019. Bert: Pre-training of deep
BERT embeddings significantly outperformed a bidirectional transformers for language understand-
ing. In North American Chapter of the Association
basic RNN model in capturing the subtleties of cy- for Computational Linguistics.
berbullying language. Among them, OpenAI em-
beddings demonstrated the highest effectiveness, Amirita Dewani, Mohsin Ali Memon, and Sania Bhatti.
indicating their superior ability to understand con- 2021. Cyberbullying detection: advanced prepro-
cessing techniques & deep learning architecture for
text and sentiment in user-generated content. Look- roman urdu data. Journal of Big Data, 8.
ing ahead, the small size of the dataset poses chal-
lenges for model performance. To address this, we Jeffrey L Elman. 1990. Finding structure in time. Cog-
plan to explore zero-shot or few-shot learning tech- nitive science, 14(2):179–211.
niques as future work, which could enhance the Ted Feinberg and Nicole Robey. 2009. Cyberbullying.
model’s generalization and accuracy in predicting The education digest, 74(7):26.
cyberbullying instances with limited labeled data.
Md. Tarek Hasan, Md Al Emran Hossain, Md. Sad-
This direction promises to improve the robustness dam Hossain Mukta, Arifa Akter, Mohiuddin Ahmed,
and applicability of detection systems in real-world and Salekul Islam. 2023. A review on deep-learning-
scenarios. based cyberbullying detection. Future Internet,
15:179.
8 Acknowledgments
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long
We thank Prof. Christopher Potts, Petra Parikova, short-term memory. Neural Computation, 9:1735–
and the XCS224U course for inspiring this work, 1780.
which started as part of the course. Celestine Iwendi, Gautam Srivastava, Suleman Khan,
and Praveen Kumar Reddy Maddikunta. 2020. Cy-
berbullying detection solutions based on deep learn-
ing architectures. Multimedia Systems, 29:1839–
1852.
Thorsten Joachims. 1999. Text categorization with sup- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine
port vector machines: Learning with many relevant Lee, Sharan Narang, Michael Matena, Yanqi Zhou,
features. In European Conference on Machine Learn- Wei Li, and Peter J Liu. 2020. Exploring the lim-
ing. its of transfer learning with a unified text-to-text
transformer. Journal of machine learning research,
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina 21(140):1–67.
Toutanova. 2019. Bert: Pre-training of deep bidirec-
tional transformers for language understanding. In Chahat Raj, Ayush Agarwal, Gnana Bharathy, Bhuva
Proceedings of naacL-HLT, volume 1, page 2. Narayan, and Mukesh Prasad. 2021. Cyberbullying
detection: Hybrid models based on machine learning
Yoon Kim. 2014. Convolutional neural networks for and natural language processing techniques. Elec-
sentence classification. In Conference on Empirical tronics.
Methods in Natural Language Processing.
Kelly Reynolds, April Kontostathis, and Lynne Edwards.
Q Li. 2006. Bullying, cyberbullying, and victimization
2011. Using machine learning to detect cyberbully-
in canada. In annual conference of the Canadian
ing. 2011 10th International Conference on Machine
Society for the Study of Education, Toronto, Ontario,
Learning and Applications and Workshops, 2:241–
Canada.
244.
Andrew McCallum and Kamal Nigam. 1998. A compar-
ison of event models for naive bayes text classifica- Chinmayee Sahoo, Mayuri Wankhade, and Binod Ku-
tion. In AAAI Conference on Artificial Intelligence. mar Singh. 2023. Sentiment analysis using deep
learning techniques: a comprehensive review. In-
Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. ternational Journal of Multimedia Information Re-
Corrado, and Jeffrey Dean. 2013. Distributed repre- trieval, 12:1–23.
sentations of words and phrases and their composi-
tionality. In Neural Information Processing Systems. Neeraj Anand Sharma, A. B. M. Shawkat Ali, and
Muhammad Ashad Kabir. 2024. A review of senti-
Vinita Nahar, Sanad Al-Maskari, Xue Li, and Chaoyi ment analysis: tasks, applications, and deep learning
Pang. 2014. Semi-supervised learning for cyberbul- techniques. International Journal of Data Science
lying detection in social networks. In Australasian and Analytics.
Database Conference.
Han-Xiao Shi and Xiao-Jun Li. 2011. A sentiment anal-
Arvind Neelakantan, Tao Xu, Raul Puri, Alec Rad- ysis model for hotel reviews based on supervised
ford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, learning. In 2011 International Conference on Ma-
Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. chine Learning and Cybernetics, volume 3, pages
2022. Text and code embeddings by contrastive pre- 950–954. IEEE.
training. arXiv preprint arXiv:2201.10005.
Sherly T.T and B. Rosiline Jeetha. 2021. Sentiment
OpenAI. 2023. Text comparison examples. Accessed: analysis and deep learning based cyber bullying de-
2024-11-07. tection in twitter dataset. International Journal of
Recent Technology and Engineering (IJRTE).
OpenAI. 2024. Openai embeddings guide.
[Link] Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob
guides/embeddings. Accessed: 2024-03-07. Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz
Kaiser, and Illia Polosukhin. 2017. Attention is all
Bo Pang and Lillian Lee. 2008. Opinion mining and
you need. In Neural Information Processing Systems.
sentiment analysis. Found. Trends Inf. Retr., 2:1–135.
R Pascanu. 2013. On the difficulty of training recurrent James E Vines. 2015. An embedded case study of the
neural networks. arXiv preprint arXiv:1211.5063. proposed Megan Meier Cyberbullying Prevention
statute & the proposed Tyler Clementi Higher Educa-
Jeffrey Pennington, Richard Socher, and Christopher D. tion Anti-Harassment statute. Ph.D. thesis, Clemson
Manning. 2014. Glove: Global vectors for word University.
representation. In Conference on Empirical Methods
in Natural Language Processing. Jun-Ming Xu, Kwang-Sung Jun, Xiaojin Zhu, and Amy
Bellmore. 2012. Learning from bullying traces in
Pratik Pujari. 2022. Formspring dataset for cyberbully- social media. In Proceedings of the 2012 conference
ing detection. Accessed: 2022-11-06. of the North American chapter of the association
for computational linguistics: Human language tech-
Alec Radford, Karthik Narasimhan, Tim Salimans, and nologies, pages 656–666.
Ilya Sutskever. 2018. Improving language under-
standing by generative pre-training. Dawei Yin, Zhenzhen Xue, Liangjie Hong, Brian D
Davison, April Kontostathis, Lynne Edwards, et al.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, 2009. Detection of harassment on web 2.0. Proceed-
Dario Amodei, Ilya Sutskever, et al. 2019. Language ings of the Content Analysis in the WEB, 2(0):1–7.
models are unsupervised multitask learners. OpenAI
blog, 1(8):9.