Text Classification & Sentiment Analysis — Teaching Reference

Text Classification &

Sentiment Analysis
Table of Contents
Section 1: Introduction to Text Classification
Section 2: Real-Time Applications of Text Classification
Section 3: Feature Engineering for Text Data
Section 4: Evaluation Metrics for Text Classification
Section 5: Sentiment Analysis — Concepts & Definition
Section 6: Types of Sentiment Analysis
Section 7: Approaches for Sentiment Analysis
Section 8: Evaluation of Sentiment Analysis Models
Section 9: Example Case Study

Page 1
 Text Classification & Sentiment Analysis — Teaching Reference

Section 1: Introduction to Text Classification

1.1 Definition of Text Classification
Text Classification is the task of assigning one or more predefined categories (labels) to a given text
document based on its content. It is one of the most fundamental and widely-used tasks in Natural
Language Processing (NLP).
In simple terms: given a piece of text (email, review, news article, tweet), the system automatically
determines which category it belongs to. The categories are decided in advance — the system must
choose from this fixed set.
Input: "Congratulations! You won a free iPhone!" → Output: SPAM
A text classification system maps raw text to a predefined category label

1.2 Importance in Natural Language Processing

Text classification is important because the volume of text data generated daily is enormous. Humans
cannot manually read and categorize millions of documents, emails, and social media posts. Automated
text classification enables organizations to process, organize, and extract value from text data at scale.
• Over 300 billion emails are sent daily — spam filters classify each one automatically
• Social media platforms process millions of posts per minute for content moderation
• Customer support systems route thousands of tickets to the right department
• News aggregators categorize articles into Sports, Politics, Technology, etc.
• Healthcare systems classify medical records, lab reports, and clinical notes

1.3 Types of Text Classification Tasks

Type Description Example
Two possible classes (Yes/No, Spam/Not
Binary Classification Email → Spam or Not Spam
Spam)
Multi-Class Three or more classes (one per News → Sports / Politics / Tech /
Classification document) Business
Multi-Label Multiple labels per document
Movie → [Action, Comedy, Romance]
Classification simultaneously
Hierarchical Product → Electronics → Phones →
Labels organized in a tree/hierarchy
Classification Smartphones

1.4 Workflow of a Text Classification System

Building a text classification system follows a systematic pipeline with five key stages. Each stage is
essential — poor preprocessing or bad features will hurt even the best model.

Stage 1: Data Collection

Page 2
 Text Classification & Sentiment Analysis — Teaching Reference

Gather labeled text data — documents paired with their correct category labels. This is the foundation
of supervised learning. Sources include manually labeled datasets, crowdsourcing platforms, existing
databases, or web scraping followed by annotation.
Quality > Quantity: 1,000 carefully labeled examples often outperform 10,000 noisy labels. Always
verify label quality.

Stage 2: Text Preprocessing

Raw text contains noise that must be removed before the model can learn effectively. Preprocessing
standardizes and cleans the text:
• Lowercasing — convert "MOVIE" to "movie" so they are treated as the same word
• Punctuation removal — strip characters like !, ?, . that don't carry classification signal
• Tokenization — split text into individual words (tokens): "I love NLP" → ["I", "love", "NLP"]
• Stopword removal — remove common words ("the", "is", "and") that appear everywhere
• Stemming — reduce words to root form using rules: "running" → "run", "movies" → "movi"
• Lemmatization — reduce to dictionary form intelligently: "better" → "good", "ran" → "run"

Stage 3: Feature Extraction

Machine learning models work with numbers, not text. Feature extraction converts preprocessed text
into numerical vectors that models can process. Common methods include Bag of Words, TF-IDF, and
word embeddings (covered in detail in Section 3).

Stage 4: Model Training

A supervised learning algorithm learns the mapping from feature vectors to class labels using the
training data. The model identifies patterns in the features that distinguish one class from another.
Popular algorithms for text classification include Naive Bayes, Decision Trees, SVM, Logistic Regression,
and neural networks.

Stage 5: Evaluation
The trained model is evaluated on a held-out test set (data it has never seen) using metrics like accuracy,
precision, recall, and F1-score. The test set must be separate from the training data to measure how
well the model generalizes to new, unseen text.
Pipeline: Raw Text → Preprocess → Extract Features → Train Model →
Evaluate → Deploy
The complete text classification workflow from raw data to production system

1.5 Real-World Example: Spam Detection

Let us trace the entire pipeline using email spam detection as an example:
Training Data:
Email Text Label
"Congratulations! You won a free iPhone! Click here now!" SPAM
"Meeting scheduled for 3 PM tomorrow in conference room B." NOT SPAM

Page 3
 Text Classification & Sentiment Analysis — Teaching Reference

"URGENT: Your account will be suspended! Verify immediately!" SPAM

"Hi Ajay, please review the attached research paper draft." NOT SPAM
"Get rich quick! Make $5000 per day from home!" SPAM

Preprocessing: Lowercase, remove punctuation, tokenize, remove stopwords.

Feature Extraction: TF-IDF converts each email into a numerical vector.
Model Training: Naive Bayes learns that words like "free", "congratulations", "urgent", "click" are
associated with spam.
Prediction: New email "You won a free vacation! Click to claim!" → classified as SPAM because it
contains high-probability spam words.

Page 4
 Text Classification & Sentiment Analysis — Teaching Reference

Section 2: Real-Time Applications of Text Classification

Text classification is deployed across virtually every industry. Here are the most important real-world
applications with detailed explanations:

1. Spam Email Detection

Gmail, Outlook, and Yahoo Mail use text classification to filter spam. The model analyzes email subject
lines and body text, flagging messages containing patterns like phishing links, urgency language ("act
now!"), and known spam phrases. Gmail's spam filter blocks over 100 million spam emails every day.
The model continuously retrains on user feedback (marking emails as spam/not spam).

2. Sentiment Analysis of Social Media

Brands monitor Twitter, Instagram, and Facebook to understand public opinion about their products.
Text classifiers analyze millions of posts in real-time, categorizing them as positive, negative, or neutral.
For example, after launching a new phone, Samsung might track sentiment across 500,000 tweets to
identify which features customers love or hate.

3. News Categorization
Google News, Apple News, and news aggregators automatically classify articles into categories like
Sports, Politics, Technology, Entertainment, and Business. This enables personalized news feeds — if you
read mostly Technology articles, the system surfaces more tech content. Reuters processes over 12,000
news stories daily using automated classification.

4. Customer Support Ticket Classification

Companies like Amazon, Flipkart, and Swiggy receive thousands of support tickets daily. Text
classification routes each ticket to the right department — billing issues go to finance, delivery
complaints go to logistics, product defects go to quality. This reduces response time from hours to
minutes by eliminating manual triage.

5. Document Organization
Law firms, hospitals, and government agencies deal with massive document archives. Text classification
automatically organizes documents by type (contracts, invoices, medical records, legal filings), enabling
fast retrieval. A hospital might classify 10,000 patient records into diagnoses, treatment plans, and lab
results.

6. Medical Text Classification

Healthcare systems classify clinical notes, radiology reports, and discharge summaries. This aids in
diagnosis coding (ICD codes), identifying patients at risk, and automating insurance claims processing.
IBM Watson for Oncology analyzed millions of medical papers to help classify cancer treatment
recommendations.

7. Fake News Detection

Social media platforms use text classification to identify and flag potentially false or misleading content.
The models analyze writing style, source credibility, emotional language, and factual consistency.
Facebook's fake news detection system processes millions of posts daily, especially during elections and
health crises.

8. Product Review Analysis

Page 5
 Text Classification & Sentiment Analysis — Teaching Reference

E-commerce platforms analyze product reviews to extract insights. Text classification identifies whether
a review is positive, negative, or neutral, and what specific aspects (price, quality, delivery) the reviewer
is discussing. Amazon uses this to generate star ratings, product summaries, and "most helpful review"
rankings.
Industry Impact: According to McKinsey, NLP and text classification can automate 15-25% of
knowledge worker tasks, saving companies millions of hours annually. Text classification is not a
niche academic topic — it is a critical business technology.

Page 6
 Text Classification & Sentiment Analysis — Teaching Reference

Section 3: Feature Engineering for Text Data

Machine learning models require numerical input — they cannot process raw text directly. Feature
engineering is the process of converting text into numerical representations (feature vectors) that
capture the meaning and patterns in the text. The quality of features directly determines model
performance.

3.1 Basic Techniques

3.1.1 Bag of Words (BoW)
The Bag of Words model represents a document as a vector of word counts. Each unique word in the
vocabulary becomes a dimension (column), and the value is how many times that word appears in the
document. Word order is completely ignored — hence the name "bag" (like throwing words into a bag).
Example with three documents:
Doc 1: "I love NLP"
Doc 2: "I love ML"
Doc 3: "NLP and ML are great"
Document I love NLP ML and are great
Doc 1 1 1 1 0 0 0 0
Doc 2 1 1 0 1 0 0 0
Doc 3 0 0 1 1 1 1 1

Pros: Simple, intuitive, easy to implement. Cons: Ignores word order, treats common words same as
rare words, high-dimensional sparse vectors.

3.1.2 N-grams
N-grams extend BoW by considering sequences of N consecutive words instead of individual words. This
partially captures word order and context.
Example sentence: "I love natural language processing"
N-gram Type N Extracted N-grams
Unigram 1 ["I", "love", "natural", "language", "processing"]
Bigram 2 ["I love", "love natural", "natural language", "language processing"]
Trigram 3 ["I love natural", "love natural language", "natural language processing"]

Bigrams capture phrases like "not good" (which unigrams would split into "not" + "good", losing the
negation). However, higher N increases vocabulary size exponentially.

3.1.3 Term Frequency (TF)

Term Frequency measures how often a word appears in a specific document, normalized by document
length:

Page 7
 Text Classification & Sentiment Analysis — Teaching Reference

TF(t, d) = Count of term t in document d / Total number of terms in

document d
Higher TF means the word appears more frequently in that specific document

Example: In a document with 100 words, if "machine" appears 5 times, TF("machine") = 5/100 = 0.05

3.1.4 TF-IDF (Term Frequency × Inverse Document Frequency)

TF-IDF is the most widely-used feature extraction method for text classification. It addresses a key
limitation of BoW: common words like "the", "is", "and" get high counts but carry no classification
signal. TF-IDF downweights common words and upweights rare, informative words.
TF-IDF(t, d) = TF(t, d) × IDF(t)
Product of term frequency and inverse document frequency

IDF(t) = log(Total documents / Number of documents containing term t)

Rare words get high IDF, common words get low IDF

Example: Word "machine" appears in 10 out of 1000 documents.

IDF("machine") = log(1000/10) = log(100) = 2.0 (rare word → high IDF)
Word "the" appears in 990 out of 1000 documents.
IDF("the") = log(1000/990) = log(1.01) = 0.004 (common word → near-zero
IDF)
Key Insight: TF-IDF gives HIGH scores to words that are frequent in a specific document but rare
across all documents — these are exactly the words that distinguish one document class from
another. This is why TF-IDF is the default choice for text classification.

3.2 Advanced Feature Engineering Techniques

3.2.1 Word Embeddings (Word2Vec, GloVe)
Word embeddings represent each word as a dense vector of real numbers (typically 100-300
dimensions) in a continuous vector space. Unlike BoW/TF-IDF where each word is independent,
embeddings capture semantic relationships — similar words have similar vectors.
Key property: Words with similar meanings are close together in vector space:
king - man + woman ≈ queen
Paris - France + India ≈ Delhi
Method Training Approach Key Idea
Predict surrounding words from "You shall know a word by the company
Word2Vec (Skip-gram)
center word it keeps"
Predict center word from Context window determines word
Word2Vec (CBOW)
surrounding words meaning
Global word co-occurrence Combines count-based and prediction
GloVe
statistics methods

Page 8
 Text Classification & Sentiment Analysis — Teaching Reference

3.2.2 Sentence Embeddings

While word embeddings represent individual words, sentence embeddings represent entire sentences
as single vectors. Methods include averaging word vectors (simple but loses word order), Doc2Vec
(extension of Word2Vec for documents), and specialized models like Sentence-BERT that produce high-
quality sentence representations.

3.2.3 Contextual Embeddings (BERT)

The most significant advance in NLP feature engineering. Unlike Word2Vec where each word has a
single fixed vector, BERT (Bidirectional Encoder Representations from Transformers) generates different
vectors for the same word depending on its context.
Example: The word "bank" gets different embeddings in:
"I went to the bank to deposit money" → bank = financial institution
"I sat on the river bank" → bank = edge of a river
BERT is pre-trained on massive text corpora (Wikipedia + BookCorpus) and can be fine-tuned for specific
classification tasks. It currently achieves state-of-the-art results on most text classification benchmarks.
Captures Context-
Feature Method Dimensionality Best For
Semantics? Aware?
Bag of Words Very High (sparse) No No Simple baselines
Traditional ML
TF-IDF Very High (sparse) Partially No
classifiers
Similarity, analogy
Word2Vec/GloVe 100-300 (dense) Yes No
tasks
State-of-the-art
BERT 768 (dense) Yes Yes
classification

Page 9
 Text Classification & Sentiment Analysis — Teaching Reference

Section 4: Evaluation Metrics for Text Classification

After training a text classification model, we need to measure how well it performs. Different metrics
capture different aspects of performance. Relying on a single metric can be misleading — a complete
evaluation uses multiple metrics together.

4.1 Confusion Matrix — The Foundation

A confusion matrix is a 2×2 table that summarizes all prediction outcomes for a binary classifier. Every
other metric is derived from this matrix.
Predicted: Positive Predicted: Negative
Actual: Positive True Positive (TP) False Negative (FN)
Actual: Negative False Positive (FP) True Negative (TN)

• True Positive (TP): Model said Positive, and it IS Positive (correct!)

• True Negative (TN): Model said Negative, and it IS Negative (correct!)
• False Positive (FP): Model said Positive, but it IS Negative (false alarm!)
• False Negative (FN): Model said Negative, but it IS Positive (missed!)
Example: Spam detection on 100 emails (40 actual spam, 60 actual not-spam):
Predicted: Spam Predicted: Not Spam
Actual: Spam TP = 35 FN = 5
Actual: Not Spam FP = 3 TN = 57

4.2 Accuracy
Accuracy = (TP + TN) / (TP + TN + FP + FN)
Proportion of ALL predictions that are correct

Using our spam example: Accuracy = (35 + 57) / (35 + 57 + 3 + 5) = 92/100 =

92%
When to use: When classes are balanced (roughly equal numbers of positive and negative examples).
Problem: With imbalanced data, accuracy is misleading. If 95% of emails are not spam, a model that
ALWAYS predicts "not spam" gets 95% accuracy but catches zero spam!

4.3 Precision
Precision = TP / (TP + FP)
Of all items PREDICTED as positive, how many are actually positive?

Spam example: Precision = 35 / (35 + 3) = 35/38 = 92.1%

Interpretation: When our model says "spam", it's correct 92.1% of the time. Only 3 legitimate emails
were incorrectly flagged.

Page 10
 Text Classification & Sentiment Analysis — Teaching Reference

When precision matters most: When the COST of false positives is high. In spam filtering, a false
positive means a legitimate email goes to spam — you might miss an important message. High precision
minimizes this risk.

4.4 Recall (Sensitivity)

Recall = TP / (TP + FN)
Of all items that ARE actually positive, how many did we find?

Spam example: Recall = 35 / (35 + 5) = 35/40 = 87.5%

Interpretation: Our model caught 87.5% of all actual spam. But 5 spam emails slipped through to the
inbox.
When recall matters most: When the COST of false negatives is high. In disease detection, a false
negative means a sick patient is told they are healthy — potentially life-threatening. High recall
minimizes missed cases.

4.5 F1-Score
F1-Score = 2 × (Precision × Recall) / (Precision + Recall)
Harmonic mean of Precision and Recall — balances both metrics

Spam example: F1 = 2 × (0.921 × 0.875) / (0.921 + 0.875) = 2 × 0.806 / 1.796

= 89.7%
When to use: When you need a single metric that balances precision and recall. Especially important
when classes are imbalanced. F1 is the standard metric for text classification benchmarks.
Summary: Use Accuracy for balanced datasets, Precision when false alarms are costly, Recall when
missing positives is costly, and F1-Score as a balanced single metric. Always report the confusion
matrix for the complete picture.

Page 11
 Text Classification & Sentiment Analysis — Teaching Reference

Section 5: Sentiment Analysis — Concepts & Definition

5.1 What Is Sentiment Analysis?
Sentiment Analysis (also called Opinion Mining) is the computational task of identifying and extracting
subjective information from text — specifically, determining whether the writer's attitude toward a
topic is positive, negative, or neutral.
More formally: Sentiment analysis automatically detects the emotional tone, opinion, or attitude
expressed in a piece of text. It answers the question: "How does the author feel about this subject?"
"This restaurant has amazing food but terrible service" → Food:
POSITIVE, Service: NEGATIVE
Sentiment analysis can identify different sentiments toward different aspects of the same entity

5.2 Why Sentiment Analysis Matters

• Businesses track customer satisfaction across millions of reviews automatically
• Political campaigns monitor public opinion on social media in real-time
• Stock market analysts gauge investor sentiment from financial news and tweets
• Product teams prioritize feature improvements based on user feedback sentiment
• Healthcare researchers analyze patient experiences from survey responses

5.3 Levels of Sentiment Analysis

Level Granularity Example
One sentiment for the entire
Document Level Movie review → Positive/Negative
document
"The food was great. The wait was long." →
Sentence Level One sentiment per sentence
Sentence 1: Positive, Sentence 2: Negative
"Camera is excellent, battery is poor" → Camera: +,
Aspect Level Sentiment per feature/aspect
Battery: −
Word/Phrase Sentiment of individual
"amazing" → Positive, "terrible" → Negative
Level expressions

Page 12
 Text Classification & Sentiment Analysis — Teaching Reference

Section 6: Types of Sentiment Analysis

6.1 Binary Sentiment Analysis
The simplest form: classify text into exactly two categories — Positive or Negative. There is no neutral
option. This is the most common type used in practice because many applications only need to know if
feedback is favorable or unfavorable.
Input Text Binary Sentiment
"This movie was absolutely fantastic!" POSITIVE ✅
"Terrible experience, never going back." NEGATIVE ❌
"I loved the acting and the storyline." POSITIVE ✅
"The worst product I have ever purchased." NEGATIVE ❌

6.2 Multi-Class Sentiment Analysis

Extends binary analysis to three or more categories. The most common setup includes Positive,
Negative, and Neutral. Some systems use finer granularity like Very Positive, Positive, Neutral, Negative,
and Very Negative (corresponding to 5-star ratings).
Input Text Multi-Class Sentiment
"Best phone I've ever used! Absolutely love it!" VERY POSITIVE (5★)
"Good product, meets expectations." POSITIVE (4★)
"It's okay, nothing special." NEUTRAL (3★)
"Disappointed with the quality." NEGATIVE (2★)
"Completely broken on arrival. Waste of money." VERY NEGATIVE (1★)

6.3 Aspect-Based Sentiment Analysis (ABSA)

The most sophisticated type. Instead of assigning one overall sentiment to the text, ABSA identifies
specific aspects (features, topics, entities) mentioned in the text and determines the sentiment toward
EACH aspect independently. This is crucial because a single review can contain mixed sentiments about
different aspects.
Example: "The phone has an excellent camera and beautiful display, but the battery life is terrible and
it overheats during gaming."
Aspect Sentiment Evidence Words
Camera POSITIVE ✅ "excellent"
Display POSITIVE ✅ "beautiful"
Battery Life NEGATIVE ❌ "terrible"
Heat Management NEGATIVE ❌ "overheats"

Page 13
 Text Classification & Sentiment Analysis — Teaching Reference

ABSA is the most valuable type for businesses. Knowing that "reviews are 60% positive" is useful,
but knowing that "camera sentiment is 90% positive while battery sentiment is 30% negative" is
actionable — the company knows exactly what to fix.

Page 14
 Text Classification & Sentiment Analysis — Teaching Reference

Section 7: Approaches for Sentiment Analysis

There are two fundamentally different approaches to sentiment analysis: lexicon-based (rule-based) and
machine learning-based. Each has distinct strengths and is suitable for different scenarios.

7.1 Lexicon-Based Approach

The lexicon-based approach uses a pre-built dictionary (lexicon) of words with associated sentiment
scores. No training data is needed — the system simply looks up each word in the lexicon and combines
the scores.

How It Works — Step by Step:

1. Start with a sentiment lexicon — a dictionary where each word has a polarity score
2. Tokenize the input text into individual words
3. Look up each word in the lexicon and retrieve its sentiment score
4. Aggregate the scores (sum, average, or weighted combination)
5. Apply a threshold: if aggregate score > 0 → Positive, < 0 → Negative

Example Sentiment Lexicon:

Word Polarity Score Sentiment
excellent +3 Strongly Positive
good +1 Positive
okay 0 Neutral
bad −1 Negative
terrible −3 Strongly Negative

Worked Example:
Input: "The camera quality is excellent but the battery is terrible"
Scores: excellent (+3) + terrible (−3) = 0 → Neutral overall
But with ABSA: Camera → Positive (+3), Battery → Negative (−3)

Popular Sentiment Lexicons:

Lexicon Description Best For
Valence Aware Dictionary, handles social
VADER Tweets, reviews, informal text
media, emojis, caps, slang
Returns polarity (−1 to +1) and subjectivity (0
TextBlob General-purpose, quick analysis
to 1)
WordNet-based, scores for positivity, Academic research, detailed
SentiWordNet
negativity, objectivity analysis
AFINN 2,477 words rated −5 to +5 by Finn Nielsen Simple applications, teaching

7.2 Machine Learning Approach

Page 15
 Text Classification & Sentiment Analysis — Teaching Reference

The ML approach treats sentiment analysis as a supervised classification problem. A model is trained on
labeled examples (text + correct sentiment) and learns to predict sentiment for new, unseen text. This
approach can adapt to specific domains and vocabularies.

Workflow:
1. Collect labeled training data (text with known sentiment labels)
2. Preprocess: lowercase, remove punctuation, tokenize, remove stopwords, lemmatize
3. Extract features: convert text to numerical vectors (TF-IDF, word embeddings)
4. Train a classifier: Naive Bayes, SVM, Logistic Regression, or neural network
5. Evaluate on held-out test data using precision, recall, F1-score
6. Predict sentiment on new, unseen text

Comparison: Lexicon vs Machine Learning

Criterion Lexicon-Based Machine Learning
Training data needed? No Yes (labeled)
High (learns domain-specific
Domain adaptability Limited (general-purpose)
language)
Handles slang/context Moderate (VADER handles some) Good (learns from data patterns)
Setup effort Very low (plug and play) Higher (data collection + training)
Accuracy Moderate (70-80%) Higher (85-95% with good data)
Handles negation Difficult ("not good") Learns negation patterns from data
Production systems, high accuracy
Best for Quick analysis, no labeled data
needs

Page 16
 Text Classification & Sentiment Analysis — Teaching Reference

Section 8: Evaluation of Sentiment Analysis Models

Evaluating sentiment analysis models requires the same metrics as general text classification (accuracy,
precision, recall, F1, confusion matrix) plus additional considerations specific to sentiment tasks.

8.1 Standard Metrics Recap

The metrics from Section 4 apply directly to sentiment analysis:
• Accuracy: Overall correctness across all sentiment classes
• Precision: Of reviews predicted as Positive, how many are actually Positive?
• Recall: Of all actual Positive reviews, how many did we detect?
• F1-Score: Harmonic mean of Precision and Recall — the go-to single metric
• Confusion Matrix: Full breakdown of TP/TN/FP/FN for each class
For multi-class sentiment (Positive/Neutral/Negative), we compute precision, recall, and F1 for EACH
class separately, then average them (macro-average for equal class weight, weighted-average
accounting for class size).

8.2 Cross-Validation
Instead of a single train/test split, k-fold cross-validation divides data into k equal parts (folds). The
model trains on k−1 folds and tests on the remaining fold. This process repeats k times, each time using
a different fold for testing. The results are averaged across all k runs.
k-Fold CV: Split data into k folds → Train on k−1 folds → Test on 1 fold
→ Repeat k times → Average results
Typical choice: k = 5 or k = 10. Provides more reliable estimate than a single split.

Advantage: Every data point gets used for both training and testing. This gives a more reliable
performance estimate, especially with small datasets.

8.3 Dataset Imbalance Issues

Sentiment datasets are often imbalanced — for example, product reviews on Amazon tend to be
overwhelmingly positive (4-5 stars) with fewer negative reviews. This creates problems:
• The model becomes biased toward the majority class (predicts Positive for everything)
• Accuracy is misleading: 90% accuracy when 90% of data is Positive means the model might not
have learned anything useful
• Minority class (Negative) gets poor precision and recall

Solutions for Imbalance:

Technique How It Works
Oversampling (SMOTE) Create synthetic examples of the minority class to balance the dataset
Undersampling Randomly remove examples from the majority class
Class Weights Tell the model to penalize errors on minority class more heavily
Use F1 instead of Accuracy F1-Score is not fooled by class imbalance like accuracy is

Page 17
 Text Classification & Sentiment Analysis — Teaching Reference

Stratified Splitting Ensure train/test splits maintain the same class ratio as the full dataset

8.4 Error Analysis

After evaluation, examine the errors the model makes to understand its weaknesses:
• Sarcasm: "Oh great, another software update that breaks everything" — model may see "great"
and predict Positive
• Negation: "This product is not good" — model may focus on "good" and miss "not"
• Domain mismatch: A model trained on movie reviews may struggle with restaurant reviews
• Mixed sentiment: "The food was delicious but the price was outrageous" — hard to classify as
purely positive or negative
• Implicit sentiment: "The phone lasted 2 hours on a full charge" — negative sentiment implied
but no explicit negative words
Best Practice: Always perform error analysis after evaluating metrics. Look at the specific examples
the model gets wrong, categorize the error types, and use these insights to improve preprocessing,
features, or training data.

Page 18
 Text Classification & Sentiment Analysis — Teaching Reference

Section 9: Example Case Study

9.1 Problem Statement
Input sentence:
"The phone design is excellent but the battery drains quickly."

Task: Classify the sentiment of this sentence using both lexicon-based and machine learning approaches.

9.2 Lexicon-Based Analysis (VADER)

VADER assigns compound scores to each word/phrase:
Word/Phrase VADER Score Contribution
excellent +0.75 Strong positive signal
but 0.00 Conjunction, signals contrast
drains quickly −0.45 Negative connotation (problems with battery)

VADER Compound Score: +0.75 + 0 + (−0.45) = +0.30

VADER applies the "but" rule (text after "but" is weighted more heavily in some implementations),
which may shift the compound score. With VADER's standard rules:
Final compound score ≈ +0.25 → POSITIVE (but weakly positive)
Limitation: VADER gives an overall Positive verdict but misses the nuance that battery sentiment is
clearly negative.

9.3 Aspect-Based Analysis

An aspect-based approach identifies two distinct aspects and assigns sentiment to each:
Aspect Identified Evidence Words Aspect Sentiment
Phone Design "excellent" POSITIVE ✅
Battery Life "drains quickly" NEGATIVE ❌

This is far more informative: the manufacturer knows the design is praised but the battery needs
improvement. A single overall sentiment ("Positive") would hide the battery problem.

9.4 Machine Learning Approach

A supervised ML model trained on labeled phone reviews would process the sentence through the full
pipeline:
1. Preprocess: "phone design excellent battery drains quickly"
2. Feature extraction (TF-IDF): Convert to numerical vector using the trained vocabulary
3. Prediction: The trained SVM or Naive Bayes model evaluates the feature vector against learned
patterns
4. If trained with binary labels → likely POSITIVE (positive words outweigh negative)

Page 19
 Text Classification & Sentiment Analysis — Teaching Reference

5. If trained with multi-class labels → could output MIXED or NEUTRAL

9.5 Comparison of All Approaches on This Sentence

Approach Overall Prediction Aspect-Level? Key Insight
Combines word scores but misses
VADER (Lexicon) Weakly Positive No
aspect nuance
Positive words dominate the feature
Binary ML Positive No
vector
Most actionable: identifies exactly
Aspect-Based Design: +, Battery: − Yes ✅
what is good/bad

Teaching Point: This case study demonstrates why aspect-based sentiment analysis is the most
valuable approach for real-world applications. Overall sentiment can hide critical insights that
businesses need to act on. Students should understand that choosing the right granularity of
analysis depends on the business question being asked.

Document Summary
This document covered the complete landscape of Text Classification and Sentiment Analysis — from
foundational concepts through feature engineering, supervised ML algorithms, lexicon-based
methods, evaluation metrics, and a worked case study. Students should be able to: (1) explain the
text classification pipeline, (2) convert text to features using BoW and TF-IDF, (3) evaluate models
using appropriate metrics, (4) distinguish between binary, multi-class, and aspect-based sentiment
analysis, and (5) compare lexicon-based and ML approaches with their trade-offs.

Page 20