Machine Learning Paradigms:
Supervised, Unsupervised, and
Semi-Supervised Learning
Comprehensive Study Notes
Abstract
Machine learning (ML) algorithms process large quantities of historical data to
identify data patterns through inference. This document outlines the fundamental
differences between supervised, unsupervised, and semi-supervised machine learn-
ing, detailing various techniques, use cases, and expanded real-world examples for
each paradigm.
1 Introduction: What’s the Difference?
Supervised and unsupervised machine learning are the two primary categories of ML
algorithms. They differ primarily in the type of data they ingest and their end objectives.
Supervised Learning: Trains on sample data that specifies both the algorithm’s
input and its corresponding desired output (labeled data). Given sufficient labeled
data, the system learns to map inputs to specific outputs (e.g., recognizing hand-
written digits).
Unsupervised Learning: Trains on unlabeled data. The algorithm scans through
new data to establish meaningful connections, structures, or groupings without
predetermined outputs (e.g., categorizing news articles by latent topics).
2 Supervised Learning
In supervised learning, models are trained to make predictions or inferences using a paired
set of input data and labeled output data. The objective is to learn a mapping function
from the input to the output.
2.1 Key Techniques
Logistic Regression: Predicts a categorical output based on one or more inputs. It
is used for binary classification (e.g., Yes/No, Pass/Fail) or multi-class classification
(e.g., categorizing images into Cat, Dog, or Rabbit).
Example: Predicting whether a student will pass a course based on their login fre-
quency.
1
Linear Regression: Predicts a continuous numerical value based on one or more
inputs.
Example: Predicting the price of a house based on location, age, and number of rooms
using historical sales data.
Decision Trees & Random Forests: Uses an if-else structure applied to input
features to predict an outcome. Random Forests combine multiple decision trees to
improve accuracy.
Example: Predicting customer churn. If a customer rarely logs in and session time is
low, the model predicts churn.
Neural Networks: Employs layers of interconnected nodes (neurons) to perform
complex mathematical transformations through adjusted weightings.
Example: Recognizing complex patterns like diagnosing diseases from medical X-ray
imaging, or transcribing speech to text.
Support Vector Machines (SVM) (Added Example): Finds the optimal hyper-
plane that separates data points into different classes in high-dimensional space.
Example: Spam detection in emails, classifying text as spam or not spam based on
word frequencies.
3 Unsupervised Learning
Unsupervised machine learning provides the algorithm with input data without any cor-
responding labeled outputs. The algorithm must identify patterns, structures, and rela-
tionships on its own.
3.1 Key Techniques
Clustering: Groups similar data points together so they can be categorized as a dis-
tinct cohort.
Example: Identifying different types of network traffic to predict potential security in-
cidents, or customer segmentation for targeted marketing campaigns based on browsing
behavior.
Association Rule Learning: Uncovers rule-based relationships between variables in
large databases.
Example: Market Basket Analysis (e.g., the Apriori algorithm) revealing that cus-
tomers who buy diapers often buy beer.
Probability Density Estimation: Predicts the likelihood of an output value falling
within a range considered ”normal,” often used for anomaly detection.
Example: Monitoring server room temperatures. A sudden, highly improbable drop
could trigger an equipment malfunction alert.
Dimensionality Reduction: Reduces the number of features (variables) in a dataset
while retaining the most important information. Techniques include Principal Compo-
nent Analysis (PCA).
Example: Compressing high-resolution images by blurring out irrelevant background
features to reduce computational overhead before feeding them into an image recogni-
tion system.
2
4 Semi-Supervised Learning
Semi-supervised learning combines a small amount of labeled data with a large amount of
unlabeled data during training. It bridges the gap between supervised and unsupervised
learning, offering higher accuracy than unsupervised methods and lower data preparation
costs than purely supervised methods.
4.1 When to Use
This approach is highly effective when data acquisition is cheap, but data labeling is
prohibitively expensive, slow, or requires specialized domain experts (e.g., medical experts
annotating thousands of CT scans).
4.2 Expanded Examples
Fraud Identification: Out of millions of daily transactions, only a few are confirmed
(labeled) by experts as fraudulent. The model first trains on the massive unlabeled
dataset to understand typical transactional structures, then uses the small labeled
dataset to pinpoint fraudulent boundaries.
Sentiment Analysis: Labeling the sentiment (positive/negative/neutral) of every
customer review across the internet is impossible. A model trains on millions of un-
labeled reviews to learn language structure and context, followed by fine-tuning on a
small manually labeled subset to accurately gauge sentiment.
Document Classification: Categorizing vast archives of legal documents or research
papers. The model clusters similar documents unsupervised, and human operators only
need to label a few representative documents from each cluster to classify the entire
database.
Web Page Classification (Added Example): Search engines must categorize bil-
lions of web pages. By labeling only a fraction of sites (e.g., news, sports, education),
algorithms can propagate these labels to other structurally and contextually similar
pages across the web.
3
5 Comparison Table: Supervised vs. Unsupervised vs. Semi-
Supervised
Feature Supervised Unsupervised Semi-Supervised
Learning Learning Learning
Input Data Fully labeled data Entirely unlabeled Small amount of
(inputs and known data (only inputs). labeled data + large
outputs). amount of unlabeled
data.
Primary Predict outcomes Discover hidden Improve prediction
Objective for new data based patterns, accuracy when fully
on past labels. groupings, or labeled data is scarce.
structures.
Human High (requires Low (algorithm Moderate (requires
Intervention intensive manual works labeling only a small,
data labeling). autonomously to representative sample).
find patterns).
Common Classification, Clustering, Classification,
Tasks Regression. Dimensionality Clustering,
Reduction, Image/Speech
Association. Recognition.
Complexity High cost for data Low data prep Cost-effective data
& Cost preparation; cost; complex to prep; highly complex
simpler evaluation. evaluate accuracy. model architecture.
Typical Linear/Logistic K-Means Self-training,
Algorithms Regression, Clustering, Apriori, Generative models,
Decision Trees, PCA, Graph-based methods.
SVM. Autoencoders.
Table 1: Key Differences Across Machine Learning Paradigms