Detecting AI Generated Text Based on NLP and

Machine Learning Approaches

Nuzhat Noor Islam Prova
Department of Seidenberg School of CSIS
Pace University
NewYork, USA
nuzhatnsu@[Link]

Abstract— Recent advances in natural language processing investigation, delving into the motivations that underscore the
(NLP) may enable artificial intelligence (AI) models to generate significance of our findings. The prevalence of artificial
writing that is identical to human written form in the future. This intelligence (AI)-generated content in several industries gives
might have profound ethical, legal, and social repercussions. This rise to concerns over its accuracy, dependability, and potential
study aims to address this problem by offering an accurate AI for manipulation. As we embark on this journey, several
detector model that can differentiate between electronically
important questions come up: What distinguishes information
produced text and human-written text. Our approach includes
machine learning methods such as XGB Classifier, SVM, BERT generated by algorithms from that generated by human
architecture deep learning models. Furthermore, our results show cognitive processes? What cultural effects results from this
that the BERT performs better than previous models in inability to discriminate? This introduction explains the logic
identifying information generated by AI from information underlying our AI sensor model, setting the stage for our
provided by humans. Provide a comprehensive analysis of the research of these problems. In addition to providing a
current state of AI-generated text identification in our assessment technological solution, our research aims to initiate a wider
of pertinent studies. Our testing yielded positive findings, showing conversation on the ethical implications of AI-generated
that our strategy is successful, with the BERT emerging as the writing, opening the door for the responsible and proper use of
most probable answer. We analyze the research's societal
implications, highlighting the possible advantages for various
advanced computational language models in the era of digital
industries while addressing sustainability issues pertaining to communication. Our study is motivated by the urgent need to
morality and the environment. The XGB classifier and SVM give provide a concrete and useful solution to the moral, societal,
0.84 and 0.81 accuracy in this article, respectively. The greatest and legal challenges posed by the increasing complexity of AI-
accuracy in this research is provided by the BERT model, which generated text. The threat of misinformation, content
provides 0.93% accuracy. manipulation, and eroding trust intensifies when AI models
have text generation abilities comparable to those of humans.
Keywords— AI generated, NLP, Machine learning, XGB, SVM, The lack of a thorough process for differentiating material
BERT created by AI from that produced by humans impedes
I. INTRODUCTION responsibility and openness. By developing an AI-detecting
framework that can differentiate between artificially generated
In the rapidly evolving fields of artificial intelligence (AI) and and human-crafted literature. The detection technique not only
natural language processing (NLP), computers are already able resolves existing moral and legal issues, but it also lays the
to produce writing that is completely unlike anything that is foundation for promoting reasonable and approachable
written by a human. Although there are many potentials uses behavior in the expanding field of AI-generated interactions.
for this scientific competence, it has also led to a number of Recently, AI language models have shown remarkable ability
constitutional, legal, and societal issues. An artificial to generate text that mimics human writing. These models use
intelligence detector model that is specifically designed to massive data and advanced algorithms to generate coherent and
differentiate between human-written text and text that is created contextually appropriate writing across many themes and
programmatically. A new era of human-like writing pattern styles. This has led to many applications in content generation,
emulation by robots has begun with the development of huge virtual assistants, and automated customer support, but it has
language models. This has led to serious moral conundrums and also raised concerns about the misuse of AI-generated text for
necessitated a fundamental change in the way we interpret and malicious purposes like misinformation, public opinion
interact with text. Addressing the critical need for a method to manipulation, and fraud. Thus, there is a rising need to identify
distinguish between content created by AI and content created and limit AI-generated material, especially on online platforms
by humans. Motivated by the need to navigate the ethical where it might be difficult to discern between human and AI-
ambiguities present in writing produced by artificial generated writing. Language's intricacy and small differences
intelligence. The introduction delineates the trajectory of our between human and machine-generated text make spotting AI-
generated material difficult. AI language models excel in stance-level sarcasm detection a novel job. Finding the author's
mimicking human language's syntactic and semantic structures, hidden position and using that information to ascertain the text's
but they frequently show evidence of their non-human origin. ironic polarity are the goals of this effort. The term "stance-
AI-generated writing may be incoherent, inconsistent, or level sarcasm detection" is abbreviated as "SLSD." Afterward,
implausible, deviating from human language conventions. AI- we provide a complete framework consisting of an innovative
generated writing may also expose training data biases or stance-centered graph attention network and Bidirectional
preferences, making it harder to discern between human and Encoder Representations from Transformers (BERT). It is
AI-authored material. NLP and machine learning methods for evident from the experiment results that the SCGAT framework
AI-generated text detection have been developed to overcome outperforms the state-of-the-art baselines by a wide margin. In
these issues. These methods analyze text's linguistic, statistical, the [5] By using Natural Language Processing (NLP) and Deep
or behavioral aspects to find abnormalities or departures from Learning, it is possible to identify, explore, and develop a
human-generated material. Stylometric analysis, anomaly global comprehension of the emotions that were expressed in
detection, and adversarial testing are common methods. the first months of the COVID-19 pandemic. Transfer Learning
Stylometric analysis examines vocabulary usage, sentence and Robustly Optimized BERT Pretraining Approach, an
structure, and writing style. Anomaly detection identifies advanced deep learning approach, were used to collect and
textual anomalies compared to a baseline of human-authored analyze approximately two million tweets. The collection of
content. these tweets took place between February and June of [Link]
standard Emotion Dataset from CrowdFlower, which was
sourced from Reddit, was used to facilitate transfer learning and
II. LITERATURE REVIEW compiling the Twitter dataset, a multi-class emotion
The machine-learning-based algorithm developed in [1] against classification system was constructed. Compared to the
the popular text generation technique known as Generative Pre- previously used AI-based emotion classification techniques,
trained Transformer (GPT) to see how well it could distinguish they were able to achieve an accuracy of 80.33% in tweet
between texts produced by AI and writings created by humans. categorization and an average MCC score of 0.78. The novel
Using a Neural Network with three hidden layers and Small use of the Roberta model during the epidemic is described in
BERT, we get a high accuracy performance. The degree of this article. This program offered insights on the evolving
precision attained varies and relies on the loss function used to mental health of several global residents throughout time.
the classification detection. In order to counteract In the [6] This article presents an approach based on Arabic text
disinformation and explore possible future research subjects, recognition for detecting misogynistic words in Arabic tweets.
the goal of this effort is to assist future studies in text bot Tests using the Arabic Levantine Twitter Dataset for
identification. Evaluation of a statistical language model for Misogynistic revealed that the proposed technique obtained
text steganography based on artificial intelligence is the main 90.0% and 89.0% recognition accuracies for binary and multi-
objective of this project, as stated in [2]. The advantageous class tasks, respectively. The method that was recommended
properties of a Markov chain model based on natural language seems to be useful in providing sensible and useful answers for
processing are suggested in this study for an autonomously identifying sexism in Arabic on social media. They describe the
produced cover text. This paper presents a comparative analysis basic process of chatbots in [7] and compare them according to
of the steganographic embedding rate, volume, and other the technology approach used as well as a few other significant
properties of two RNN steganographic schemes: RNN- criteria. To offer a relevant response, a chatbot must precisely
generated Lyrics and RNN-Stega. The acronym for recurrent analyze and understand user input. This will make it possible to
neural networks is RNN. Included in the essay is a case study have more productive conversations using natural language.
about information security and artificial intelligence. This case These days, they are used in many important domains,
study investigates the background, applications, challenges, including research, education, and healthcare. The relationship
and ways in which artificial intelligence (AI) might help between people and technology may be more natural with
mitigate security threats and weaknesses. As stated in [3], chatbots. The evolution of synthetic text generation has raised
identifying machine-generated text is an essential preventative a number of crucial issues that might have an impact on society
precaution against the misuse of natural language generation and the internet. Programs powered by LLM are expected to
models yet, there are several unresolved issues and significant replace a significant portion of the human labor [8]. A plethora
technical challenges in this area. An extensive analysis of threat of tasks in the domains of research, education, law, advertising,
models presented by contemporary NLG systems and the most and creative writing for leisure are now being performed by
thorough investigation of methods available for machine- computers. It is more difficult to spot cases of phishing,
generated text detection to date. To underline how crucial it is spamming, academic fraud, fake news, and reviews [9]. As a
that detection systems themselves demonstrate their reliability result, it will be very difficult in the future to identify a phrase
by being impartial, strong, and responsible. In [4], it is said that that has been falsely constructed. In order to identify unique
the advancement of natural language processing methods and traits and patterns in text material that has been intentionally
sarcasm detection technologies would enable intelligent and generated, to look into a number of approaches in this research
economical collaboration with machine devices as well as project. The problem of text recognition posed by artificial
advanced human-machine interactions. In this work, introduce intelligence, or more broadly, machine-generated textual
material, has drawn a lot of interest. It started with the Turing commonly deleted from text data to save computational
test and was then used to the evaluation of chatbots [10]. In this overhead and concentrate on more significant terms to increase
paper, computerized procedures were the major focus instead algorithm efficiency and accuracy.
of hybrid or human-cantered detection techniques. According
to Crothers, Japkowicz, and Viktor [11], autonomous AI- Remove unwanted characters or links:
generated text recognition systems may be broadly classified Preprocessing text data by removing unwanted letters and links
into two categories: feature-based and neural language models. is necessary for natural language processing, web scraping, and
Nonetheless, specific domains were the focus of other research, data analysis. Remove undesired special characters,
such as [12] scientific settings and fake news false reviews punctuation, HTML components, and URLs. Regular
misinformation. In addition to, two methods for automatically expressions target and eliminate URLs and punctuation. For
identifying text generated by AI were proposed in this study filtering, tokenization may split text into words or tokens.
text resemblance-based approaches and feature-based Filters eliminate unwanted text and leave just what's needed.
approaches using machine learning models. Change all content to lowercase or expand contractions to
standardize format. Quality assurance tests during cleaning
save important information and eliminate useless items.
III. PROPOSED METHODOLOGY Analysts and developers may ensure downstream processes and
analytics are correct by cleaning and sanitizing text data.

Stemming:
Words are reduced to their stem, or basis, by the application of
the stemming approach in natural language processing. By
considering word variants as the same, stemming aims to
normalize words with similar meanings to a common form,
which may enhance text analysis and retrieval tasks. For
instance, the basic form "run" would be stemmed to produce the
words "running," "runs," and "ran."

Lemmatization:
Lemmatization is a term used in natural language processing to
describe the process of learning words based on their basic
lexical components. It is used in computer programming and
artificial intelligence, as well as natural language processing
and interpretation. In more complex cases, lemmatization
allows the computer to group words that share an inflected
meaning but do not share a stem, such as "good" with phrases
like "better" and "best."
C. Data Visualization

Fig. 1. The proposed methodology of the work

A. Dataset
This dataset was essentially built by myself. which I separated
into 0 and 1 levels. in which AI-generated data is ranked as 1,
and human-generated data as 0. I have produced 3000 total data
points, 1500 of which are produced by humans and 1500 by
artificial intelligence. I used both handwritten and online
platform. The datasets include text written by both AI and
humans in order to provide a thorough understanding of
distinguishing factors.
Fig. 2: Word cloud of text dataset
B. Data Preprocessing
Stopwords removal: In figure 2, are showing word cloud of text in the dataset. Word
Stop words are typical natural language terms that are filtered clouds provide text data in different sizes depending on
before or after text processing. In search engines, text mining, frequency or relevance. It helps you understand a text's main
and machine learning, these phrases are usually redundant or ideas quickly and easily. A word cloud, unlike a list or
non-informative. Stop words in English include "the", "is", "at", paragraph, enlarges words proportionately to their frequency,
"which", "on", "in", "to", "of", "and", "or". These words are making the most important keywords stand out. This
visualization method simplifies enormous volumes of textual accuracy is 83%, meaning the classifier found 83% of true class
data into an attractive manner, making it easy to see patterns. 0 instances. Class 1 has 78% recall and 82% accuracy. Recall
Data analysis, market research, and text summarization employ and accuracy are balanced in both groups' F1-scores. The
word clouds to identify key themes and patterns in a corpus of disorientation matrix exhibits 249 genuine negatives, 235 real
text. positives, 51 false positives, and 65 incorrect negatives to
illustrate the classifier's prediction accuracy. The SVM
D. Feature Engineering
classifier categorizes human-authored and AI-generated textual
In natural language processing (NLP), CountVectorizer is used material with a median accuracy of 80.67%.
to extract token counts from text texts. By converting raw text
input to numbers, machine learning algorithms can process it. IV. RESULTS AND DISCUSSION
Tokenize text documents, break them into words or terms, and
count their occurrences using CountVectorizer. Thus, a sparse TABLE I. PERFORMANCES OF D IFFERENT CLASSIFIERS
matrix with rows representing documents and columns
representing unique terms in the corpus of documents is Algorithm Class Precision Recall F1 Accuracy
produced. Values in matrix cells represent phrase frequency in Name Score
the document. The numerical representation of text data allows XGB 0 0.86 0.82 0.84 0.84
NLP tasks including text categorization, grouping, and Classifier
1 0.83 0.87 0.85
information retrieval. Text processing pipelines employ
CountVectorizer and other methods like term frequency- SVM 0 0.79 0.83 0.81 0.81
inverse document frequency (TF-IDF) to enhance NLP models. 1 0.82 0.78 0.80
E. Model Generate
BERT:
Bidirectional Encoder Representations from Transformers, or The evaluation of three distinct machine learning algorithms'
BERT for short, is a cutting-edge language model that effectiveness in identifying text created by artificial intelligence
combines self-attention mechanisms with the Transformer (AI): BERT, XGBoost (XGB), and Support Vector Machines
architecture for comprehensive text processing. This algorithm (SVM). With an accuracy of 93%, BERT was the most accurate
operates with an exceptional 93% accuracy in our development. of these algorithms; XGBoost and SVM came in at 84% and
BERT is a cutting-edge model in natural language processing 81%, respectively Google created BERT, or Bidirectional
that uses pre-training tasks like Next Sentence Prediction and Encoder Representations from Transformers, a cutting-edge
Masked Language Modelling, bidirectional training, and task- natural language processing (NLP) approach. It performs better
specific fine-tuning to obtain outstanding results. It excels at in a variety of NLP tasks, such as text classification and
handling distant dependencies, collecting context-aware sentiment analysis, by comprehending the context of words in
embeddings, and adjusting to different NLP applications. a phrase by considering both the words that come before and
after. To evaluate how well these algorithms, differentiate
XGB: between material created by AI and content created by humans,
XGBClassifier, a powerful machine learning model, performs it is essential to compare how well they recognize text written
well in your assignment with 84.33% accuracy. The classifier's by AI. BERT's much greater accuracy indicates that it is better
accuracy and recall statistics reveal its ability to distinguish at identifying the subtleties and subtle patterns in text data that
human-written and AI-written material. Class 0 accuracy is are suggestive of artificial intelligence (AI) development. On
86%, indicating 86% of predicted instances are negative. The the other hand, while XGBoost and SVM exhibit reasonable
classification system detected 82% of class 0 instances, its accuracy, they may not be able to fully grasp the subtleties and
recall. Class 1 also has 83% recall and 87% accuracy. The F1 complexity of language created by artificial intelligence to the
ratings for the two categories reveal a decent accuracy-recall same degree as BERT.
trade-off, making the machine learning approach robust. The
confusion matrix reveals 246 real negatives, 260 genuine
V. CONCLUSION
positives, 54 erroneous positives, and 40 erroneous negatives,
illustrating the classifier's anticipated accuracy. Due to its high Recent breakthroughs in natural language processing (NLP)
accuracy and precision-recall efficiency, the XGBClassifier is may allow AI models to write like humans. This might have
ideal for this project's classification task. major ethical, legal, and societal consequences. An accurate AI
detector model that can distinguish electronically generated text
from human-written text is proposed in this paper. XGB
SVM:
Classifier, SVM, and BERT architecture deep learning models
This project's Support Vector Machine (SVM) classifier
are used in our methodology. Our findings also reveal that the
distinguishes AI-generated text from human-written material BERT identifies AI-generated information from human-
with 80.67% success. Accuracy and recall measures reveal the provided information better than earlier models. Assess
classifier's discriminating capacity. Class 0 accuracy is 79%,
relevant research and analyze AI-generated text identification's
indicating 79% of anticipated instances are negative. Class 0
present stage. Our testing showed that our technique works,
with the BERT being the most likely response. We examine the [5] S. Göring, R. R. Ramachandra Rao, R. Merten, and A. Raake, “Analysis of
appeal for realistic AI-generated photos,” IEEE Access, vol. 11, pp.
research's social impacts, emphasizing industry benefits while 38999–39012, 2023. doi:10.1109/access.2023.3267968
addressing ethical and environmental sustainability challenges. [6] C. Li et al., “Agiqa-3K: An open database for AI-Generated Image Quality
This article's XGB classifier and SVM have 0.84 and 0.81 Assessment,” IEEE Transactions on Circuits and Systems for Video
accuracy. The BERT model has the highest accuracy in this Technology, pp. 1–1, 2024. doi:10.1109/tcsvt.2023.3319020
[7] H. Du et al., “Exploring collaborative distributed diffusion-based AI-
investigation at 0.93%.The examination of BERT, XGBoost
generated content (AIGC) in wireless networks,” IEEE Network, pp. 1–
(XGB), and Support Vector Machines' ability to detect AI- 8, 2024. doi:10.1109/mnet.006.2300223
generated text. BERT was the most accurate at 93%, followed [8] N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using
by XGBoost at 84% and SVM at 81%. Google developed Siamese Bert-Networks,” Proceedings of the 2019 Conference on
Empirical Methods in Natural Language Processing and the 9th
cutting-edge NLP method BERT, or Bidirectional Encoder
International Joint Conference on Natural Language Processing
Representations from Transformers. It improves NLP tasks like (EMNLP-IJCNLP), 2019. doi:10.18653/v1/d19-1410
text classification and sentiment analysis by understanding the [9] S. Gehrmann, H. Strobelt, and A. Rush, “GLTR: Statistical detection and
context of words in a phrase by considering both preceding and visualization of generated text,” Proceedings of the 57th Annual Meeting
of the Association for Computational Linguistics: System
following words. It is crucial to assess how effectively these
Demonstrations, 2019. doi:10.18653/v1/p19-3019
algorithms detect AI-written language to determine how well [10] Y. Wang, Y. Pan, M. Yan, Z. Su, and T. H. Luan, “A survey on
they distinguish between AI and human-created information. CHATGPT: Ai–generated contents, challenges, and solutions,” IEEE
BERT's higher accuracy suggests it can better recognize text Open Journal of the Computer Society, vol. 4, pp. 280–302, 2023.
doi:10.1109/ojcs.2023.3300321
data peculiarities that signify AI progress. XGBoost and SVM
are accurate, however they may not understand the intricacies [11] Y. Lin et al., “Blockchain-aided secure semantic communication for AI-
and complexity of artificial intelligence-generated language generated content in Metaverse,” IEEE Open Journal of the Computer
like BERT. Society, vol. 4, pp. 72–83, 2023. doi:10.1109/ojcs.2023.3260732
[12] H. Alamleh, A. A. AlQahtani, and A. ElSaid, “Distinguishing human-
written and CHATGPT-generated text using machine learning,” 2023
Systems and Information Engineering Design Symposium (SIEDS), Apr.
REFERENCES 2023. doi:10.1109/sieds58326.2023.10137767
[13] J. P. Wahle, T. Ruas, S. M. Mohammad, N. Meuschke, and B. Gipp, “AI
usage cards: Responsibly reporting AI-generated content,” 2023
[1] R. Merine and S. Purkayastha, “Risks and benefits of AI-generated text ACM/IEEE Joint Conference on Digital Libraries (JCDL), Jun. 2023.
summarization for expert level content in Graduate Health Informatics,” doi:10.1109/jcdl57899.2023.00060
2022 IEEE 10th International Conference on Healthcare Informatics [14] T. T. Nguyen, A. Hatua, and A. H. Sung, “How to detect AI-generated
(ICHI), Jun. 2022. doi:10.1109/ichi54592.2022.00113 texts?,” 2023 IEEE 14th Annual Ubiquitous Computing, Electronics
[2] A. M. Elkhatat, K. Elsaid, and S. Almeer, “Evaluating the efficacy of AI & Mobile Communication Conference (UEMCON), Oct. 2023.
content detection tools in differentiating between human and ai-generated doi:10.1109/uemcon59035.2023.10316132
text,” International Journal for Educational Integrity, vol. 19, no. 1, Sep. [15] I. Katib, F. Y. Assiri, H. A. Abdushkour, D. Hamed, and M. Ragab,
2023. doi:10.1007/s40979-023-00140-5 “Differentiating chat generative pretrained transformer from humans:
[3] F. R. Elali and L. N. Rachid, “AI-generated research paper fabrication and Detecting chatgpt-generated text and human text using machine
plagiarism in the scientific community,” Patterns, vol. 4, no. 3, p. 100706, learning,” Mathematics, vol. 11, no. 15, p. 3400, Aug. 2023.
Mar. 2023. doi:10.1016/[Link].2023.100706 doi:10.3390/math11153400
[4] G. Gritsay, A. Grabovoy, and Y. Chekhovich, “Automatic detection of
machine generated texts: Need more tokens,” 2022 Ivannikov Memorial
Workshop(IVMEM),Sep.2022. doi:10.1109/ivmem57067.2022.9983964