"Detecting AI-Generated Text: Building an Intelligent Classification System"
Abstract
With the rise of advanced AI models capable of generating human-like text (e.g., GPT, Grok),
distinguishing between human-written and AI-generated content has become a critical challenge.
This project aims to design and implement an AI-powered detection system that identifies
whether a given piece of text was authored by a human or generated by an AI model. The system
will leverage machine learning techniques, natural language processing (NLP), and statistical
analysis to achieve high accuracy, addressing applications in education, journalism, and online
content moderation.
Introduction
• Background: AI text generation tools are increasingly accessible, producing content that
mimics human writing styles. However, this raises concerns about misinformation,
plagiarism, and authenticity.
• Problem Statement: There’s a growing need to detect AI-generated text to ensure
transparency and trust in digital content. Existing tools are limited in accuracy and
adaptability.
• Objective: To develop a robust, scalable system that classifies text as "human-written" or
"AI-generated" with explainable results.
• Scope: Focus on English-language text, targeting outputs from popular AI models (e.g.,
GPT-3, Grok) and human-written samples from diverse sources.
Literature Review
• Overview of existing AI text detectors (e.g., Turnitin’s AI detection, OpenAI’s classifier).
• Techniques used: Linguistic pattern analysis, perplexity scoring, and deep learning
models (e.g., BERT, RNNs).
• Limitations: Over-reliance on specific model signatures, lack of generalization across AI
tools.
• Gap: Need for a detector that adapts to evolving AI models and provides user-friendly
insights.
Methodology
1. Data Collection:
o Gather a dataset of human-written texts (e.g., essays, articles) and AI-generated
texts (e.g., outputs from GPT, Grok, LLaMA).
o Use web scraping, X posts, and synthetic text generation for diversity.
2. Preprocessing:
o Clean text (remove noise, normalize formatting).
o Extract features like sentence length, word frequency, syntactic complexity, and
coherence.
3. Model Development:
o Train a machine learning model (e.g., Random Forest, SVM) for baseline
performance.
o Implement a deep learning approach (e.g., fine-tuned BERT or LSTM) for higher
accuracy.
o Explore hybrid methods combining statistical metrics (e.g., perplexity) with
neural networks.
4. Evaluation:
o Metrics: Accuracy, precision, recall, F1-score.
o Test on unseen data, including new AI models not used in training.
5. Interface:
o Develop a simple web or command-line tool for users to input text and receive
detection results.
Expected Outcomes
• A working prototype capable of distinguishing AI-generated text with at least 85%
accuracy.
• Insights into linguistic differences between human and AI writing.
• A report detailing the system’s performance and potential improvements.
Tools and Technologies
• Programming: Python (TensorFlow, PyTorch, Scikit-learn).
• NLP Libraries: NLTK, SpaCy, Hugging Face Transformers.
• Dataset Sources: Open datasets, X posts, web articles.
• Hardware: Laptop/PC with optional GPU support for training.
Significance
• Academic: Helps educators detect AI-assisted plagiarism.
• Commercial: Assists content platforms in verifying authenticity.
• Ethical: Promotes transparency in AI usage.
Challenges
• Adapting to new AI models with minimal retraining.
• Avoiding bias toward specific writing styles or AI tools.
• Balancing accuracy with computational efficiency.
Future Work
• Extend to multilingual text detection.
• Integrate real-time analysis for online platforms.
• Explore explainability features (e.g., highlighting AI-like patterns).