Thynk Unlimited

SUPERVISED
LEARNING:
CLASSIFICATION
Thynk Unlimited

OBJECTIVE OF THE
CHAPTER
1. To provide an
understanding of
popular classifiers
and their work
principls.
Thynk Unlimited

WHAT IS SUPERVISED
LEARNING?
Supervised learning uses labeled data with
known answers.
The model is trained using input–output
pairs, where the correct output is already
provided..
 WHAT IS
Thynk Unlimited

CLASSIFICATION?
Classification is the process of assigning data into
predefined categories.

Types of Classification:
Binary Classification – Only 2 classes
Example: Email is either Spam or Not Spam
Multi-class Classification – More than 2 classes
Example: Fruit can be Apple, Banana, or Orange
Thynk Unlimited

PRINCIPLES OF SUPERVISED
CLASSIFICATION
Supervised classification is based on
the idea that a model can learn
patterns from labeled data and then
use these patterns to classify new
data. The following principles guide
how supervised classification works:
Performance Tracking
 Example: Predicting if an email is
spam or not. LABELED
SPAM
TRAINING DATA
Sender: “Nigerian Prince”
Subject: “URGENt: Millions!” The model requires a dataset with
Content: “Dear friend, am known labels.
contacting..... Each example consists of features
(inputs) and a class label (output)
NOT SPAM
Sender: Principle: “You cannot teach a model
“newsletter@[Link]” without showing it correct answers
Subject: Weekly Tech News
Content: “Read about the latest
first.”
gadgets...
Example: In predicting loan approval,
FEATURE
income and credit score are more
important than favorite color.
SELECTION
The model learns from features that
describe the data.
Choosing the most relevant features
improves classification accuracy.

Principle:
Features: age, income,“Better features → better
spending habits
predictions.”
Example: Customers with income >
50k are likely to buy a product. LEARNING
PATTERNS
The model identifies relationships
between features and labels.
It “learns” rules or boundaries that
separate different classes.

Principle: “Classification is based on

learning patterns from examples, not
BUYING guessing.”
NOT BUYING
 MODEL
TRAINING
The model uses algorithms (e.g., Decision
Tree, SVM, KNN) to learn patterns.
During training, the algorithm adjusts itself
to minimize errors on the training data.

Features: age, income,

Principle: “Training is the process of learning the
spending habits
best way to separate classes.”
 TESTING AND
VALIDATION
After training, the model is tested on new,
unseen data to measure performance.
Techniques like cross-validation ensure the
model generalizes well.

Principle: “A good classifier performs well not just

Features: age, income,
on training data but also spending
on newhabits
data.”
 DECISION
BOUNDARIES
Classification often works by creating
boundaries between classes in feature space.
The model predicts a new example’s class based
on which side of the boundary it falls.

Principle: “Classes are separated by learned

Features: age, income,
boundaries based on patterns
spendinginhabits
the features.”
 EVALUATION
METRICS
To assess accuracy, metrics like accuracy,
precision, recall, and F1-score are used.
Evaluation ensures that the model makes
reliable predictions.

Principle: “Classification is only meaningful

Features: age, income,
if
performance is measured objectively.”
spending habits
 Thynk Unlimited

COMMON
CLASSIFICATION
METHODS
Classification methods are algorithms used in supervised
learning to assign data into predefined categories or
classes. Each method follows a different approach in
learning patterns from labeled data.

Decision Trees Naive Bayes (KNN) (SVM)

Decision Trees

- A non-parametric supervised learning method that predicts the value

of a target variable by learning simple decision rules inferred from the
data features. It partitions the feature space into a set of rectangles and
fits a simple model (like a majority vote) in each one.

Theory & Proof: The core of a decision tree is the Splitting Criterion,
typically Gini Impurity or Entropy.

- Entropy (H): Measures the disorder in a set S.

- Proof of Concept: The algorithm uses Information Gain, which is the
reduction in entropy after a split. By maximizing Information Gain at each
node, the tree mathematically proves it is creating the most
Strategic Alignment
homogeneous (pure) child nodes possible.
Naive Bayes

- A probabilistic classifier based on applying Bayes' Theorem with

the "naive" assumption of conditional independence between every
pair of features given the value of the class variable.

Theory & Proof: It is rooted in the Maximum A Posteriori (MAP)

decision rule.

The "Naive" Proof: By assuming P(x_i | y, x_1, \dots, x_{i-1}) = P(x_i |

y), we simplify the complex joint probability into a product of
individual probabilities: Strategic Alignment
K-Nearest Neighbors (KNN)

A type of instance-based learning where the function is only

approximated locally and all computation is deferred until
classification. An object is classified by a plurality vote of its
neighbors.

Theory & Proof: KNN is supported by the Cover-Hart Theorem,

which provides an upper bound on the error rate.

The Proof: As the number of training samples n \to \infty, the error
rate of the 1-Nearest Neighbor classifier is guaranteed to be no
worse than twice the Bayes Error Rate (the minimum possible error
rate for any classifier). Strategic Alignment
Support Vector Machines (SVM)

A classifier that finds the optimal hyperplane which maximizes

the margin between two classes in a high-dimensional space.

Theory & Proof: SVM is based on Structural Risk Minimization

(SRM) rather than Empirical Risk Minimization.

- The Math: It solves a constrained optimization problem:

- The Proof: By minimizing \|w\|, we are maximizing the margin

(\frac{2}{\|w\|}). According to Vapnik-Chervonenkis (VC) Theory,
maximizing this margin minimizes the "VC dimension," which
mathematically guarantees a lower bound on the generalization
Strategic Alignment
error (reducing overfitting).
 Thynk Unlimited

REAL-WORLD
APPLICATION

Spam Detection Fraud Detection Image

Classification
Thynk Unlimited
REAL-WORLD APPLICATION: SPAM DETECTION
Spam detection refers to the process of identifying and
categorizing electronic messages—primarily emails—as either
spam (unwanted or malicious) or ham (legitimate).

Email Classification Common Algorithms Industry Standard

Major email services like
Incoming emails are Naive Bayes Classifier
Gmail, Outlook, and Yahoo
analyzed based on
Mail heavily rely on
content, sender, and Support Vector
sophisticated classification
structure to determine if Machines (SVM)
systems to protect users
they are legitimate or from phishing attempts,
spam. malware, and unsolicited
advertisements.
 Thynk Unlimited
REAL-WORLD APPLICATION: FRAUD DETECTION
refers to the use of machine learning and data analysis techniques
to categorize transactions or activities as either fraudulent or non-
fraudulent based on historical patterns and anomalies.

Key Approaches in Classification-Based Fraud Detection:

The model is trained on a labeled dataset where each transaction is
marked as "fraud" or "non-fraud.

Common algorithms include:

Logistic Regression Decision Trees & Support Vector Machines

Random Forests (SVM) and Neural Networks
 REAL-WORLD APPLICATION: IMAGE CLASSIFICATION
Image classification is a cornerstone of computer vision, enabling
machines to "see" and interpret the visual world by assigning
predefined categories to images.

Single-label classification: Multi-label classification: Binary classification:

 REAL-WORLD APPLICATION: IMAGE CLASSIFICATION
Image classification is a cornerstone of computer vision, enabling
machines to "see" and interpret the visual world by assigning
predefined categories to images.

Face Recognition Medical Image Diagnosis Everyday Integration

Platforms like Google

Used in security AI assists doctors in
Photos and Facebook
systems, smartphone detecting
leverage image
unlocking, and social abnormalities in X-rays,
classification to
media tagging, these MRIs, and CT scans,
automatically organize
models identify classifying images as
photos, tag friends,
individuals by indicative of diseases
and enable searching
classifying unique like cancer or
for specific objects or
facial features. pneumonia, aiding
scenes within vast
early diagnosis.
image libraries.
 CONCLUSION:

Classification is undeniably a foundational and indispensable task within the realm of

supervised machine learning. Its ability to categorize data into discrete classes
underpins countless technological advancements, making it a critical component of
modern AI systems across various industries.

The diversity of classification algorithms, from the straightforward Decision Tree to

the probabilistic Naïve Bayes, highlights the need to select the right tool for the right
problem. Each algorithm comes with its own strengths and weaknesses, making
algorithm selection a key step in model development.

From detecting spam in our inboxes to identifying fraudulent financial activities and
recognizing objects in complex images, classification's real-world applications are
vast and continue to expand. As AI evolves, so too will the sophistication and reach of
classification techniques, further integrating them into the fabric of our digital lives.