UNSUPERVISED LEARNING

In unsupervised learning, the model learns from unlabelled data — it doesn’t know what
the output should be.
Instead of predicting something, it tries to:
 Find hidden patterns
 Group similar data points
 Reduce dimensions
 Detect anomalies

🚀 Popular Types of Unsupervised Learning:

1. 🔹 Clustering
Group similar data points together.
 🔸 K-Means – clusters data into k groups
 🔸 Hierarchical Clustering
 🔸 DBSCAN – density-based, good for uneven clusters
✅ Example: Segmenting customers into buyer personas.

2. 🔹 Dimensionality Reduction
Reduce features while preserving information.
 🔸 PCA (Principal Component Analysis)
 🔸 t-SNE / UMAP (for visualization)
✅ Example: Visualizing high-dimensional data in 2D plots.

3. 🔹 Anomaly Detection
Find outliers in the data.
 🔸 Isolation Forest
 🔸 One-Class SVM
✅ Example: Credit card fraud detection
4. 🔹 Association Rule Mining
Discover rules in transactions.
 🔸 Apriori
 🔸 Eclat
✅ Example: Market Basket Analysis – "People who buy bread also buy butter"
Transaction Items Bought

T1 Milk, Bread

T2 Milk, Diaper, Beer, Eggs

T3 Milk, Diaper, Beer, Coke

T4 Bread, Milk, Diaper, Beer

T5 Bread, Milk, Diaper, Coke

Now let’s say we want to analyze the rule:

"If Diaper, then Beer"

🔹 1. Support
How often does this combo occur in all transactions?
Formula:
\text{Support}(A → B) = \frac{\text{Transactions with both A and B}}{\text{Total
transactions}} ]
In our case:
  Diaper & Beer together appear in T2, T3, T4 → 3 times
 Total transactions = 5
 🔥 Support = 3/5 = 0.6 (60%)
➡️This means 60% of customers bought both Diaper and Beer.

🔹 2. Confidence
How often B is bought when A is bought?
Formula:
\text{Confidence}(A → B) = \frac{\text{Transactions with both A and B}}{\text{Transactions
with A}} ]
 Diaper appears in T2, T3, T4, T5 → 4 times
 Diaper + Beer = 3 times (T2, T3, T4)
 🔥 Confidence = 3/4 = 0.75 (75%)
➡️So, when someone buys Diapers, there’s a 75% chance they also buy Beer.

🔹 3. Lift
How much more likely someone buys B given A, compared to just buying B in general?
Formula:
\text{Lift}(A → B) = \frac{\text{Confidence}(A → B)}{\text{Support}(B)} ]
 Beer appears in T2, T3, T4 → 3 times
 Support(Beer) = 3/5 = 0.6
 Confidence(Diaper → Beer) = 0.75
 🔥 Lift = 0.75 / 0.6 = 1.25
➡️This means buying Diaper increases the likelihood of buying Beer by 25% compared to
random chance.

🔍 What is the Hopkins Statistic?

The Hopkins Statistic is a numerical test used to determine the cluster tendency of a
dataset — in simple terms:
“Is your data naturally grouped into clusters, or is it randomly spread out?”
Before you use algorithms like K-Means, DBSCAN, etc., Hopkins tells you whether it's even
worth trying.
Steps to Calculate Hopkins Statistic (No Formulas):
1. Choose a sample size (e.g., 10% of your dataset).
2. Randomly select points from your dataset — these are your real sample points.
3. Generate synthetic random points in the same space as your dataset.
4. For each synthetic point, measure distance to the nearest real point.
5. For each real sample point, measure distance to its nearest neighbor (excluding
itself).
6. Sum all distances from synthetic points and real points.
7. Compare the two sums to get the Hopkins value.

K-MEANS
K-Means is an unsupervised learning algorithm used to group data into K clusters based on
similarity.
It tries to minimize the distance between data points and the center of their assigned
cluster.
Here’s how K-Means works, like a smart organizer:
1. Pick how many groups you want — say 3.
2. Drop 3 random dots (these are your starting cluster centers).
3. Look at each data point and say:
“Which of these 3 centers am I closest to?” — assign it to that group.
4. Once all points are grouped, move each center to the middle of its group.
5. Now repeat: regroup based on the new centers → move centers → regroup...
 6. Stop when everything settles down and centers hardly move anymore.

🧠 What is Silhouette Analysis?

It’s a method to evaluate the quality of clustering — telling you how well each data point
fits into its cluster vs. other clusters.
It gives a score between -1 and 1 for each data point.

Score Interpretation

~1.0 Point is well matched to its own cluster ✅

~0.0 Point is on the border between clusters ⚠️

< 0.0 Point is likely in the wrong cluster ❌

🧪 What is make_blobs?
It’s a function from [Link] used to generate synthetic data points that naturally
form clusters.
Basically:
🔧 You tell it how many clusters, how many features, how much spread, and it gives you
beautiful grouped data to test your algorithms.

Within-Cluster Sum of Squares (WCSS) measures the compactness or cohesion of clusters. It

quantifies how close data points within a cluster are to its centroid. A lower WCSS indicates
that data points are more tightly grouped around their respective cluster centroids
cluster_std: Standard deviation (spread) of clusters
 This controls how tightly packed or spread out the clusters are.
 Lower values → points in each cluster are close to each other (tight, compact
clusters)
 Higher values → clusters are more spread out and may even overlap

✅ Returns:
 X → the feature data (coordinates of points) as

 y → the true cluster labels (useful for checking accuracy)

wcss = [] Create an empty list to store WCSS values

for i in range(1, 11): Try cluster counts from 1 to 10

n_clusters=i Set the number of clusters for this run

init='k-means++' Smart initialization to reduce chance of bad clusters

random_state=0 Fix randomness for reproducibility

[Link](X) Run KMeans on the data

kmeans.inertia_ This is WCSS: sum of squared distances of points to their cluster center

[Link](...) Store the WCSS for plotting

Then draws a graph called the Elbow Curve, used to find the best number of clusters.
K- MEANS CLUSTERING WITH ELBOW METHOD:
For each number of clusters, we find the sum of squared distances

Find the elbow, here 4

🌳 What is Hierarchical Clustering?
Hierarchical Clustering is an unsupervised learning algorithm that builds a hierarchy of
clusters.
It’s like creating a family tree of data points — where similar points are "merged" together
step by step.

📈 Two Types:
1. Agglomerative (Bottom-Up – most common):
o Start with each point as its own cluster
o Merge the closest two clusters step-by-step
o Stop when everything is one big cluster (or as many as you want)
2. Divisive (Top-Down):
o Start with one big cluster
o Recursively split it into smaller clusters

📊 Dendrogram (Tree Diagram)

A dendrogram is used to visualize the merging process.
 The vertical axis = distance (or dissimilarity)
 The horizontal axis = individual data points
 Cutting the tree at a certain height = desired number of clusters
 MEAN – SHIFT CLUSTERING
🌀 What is Mean Shift Clustering?
Mean Shift is a centroid-based unsupervised clustering algorithm — like K-Means, but
smarter in some ways.
Instead of fixing the number of clusters (k), Mean Shift automatically finds the number of
clusters by locating dense areas in the data.

🧠 How It Works:
1. Start with a random point in the dataset.
2. Define a window (bandwidth) (think of a circle around the point).
3. Find all data points within that window.
4. Compute the mean (center of mass) of those points.
5. Shift the window to the new mean.
6. Repeat steps 3–5 until convergence (the window stops moving).
7. Repeat for all points → nearby windows merge into clusters.

🔍 What is OPTICS?
OPTICS stands for:
Ordering Points To Identify the Clustering Structure
It’s a density-based clustering algorithm (like DBSCAN), but instead of assigning hard cluster
labels right away, OPTICS builds an ordered list of points that reveals the clustering structure
at different density levels.
 It’s ideal for datasets with clusters of varying density.
 DBSCAN struggles when different clusters have different densities — OPTICS handles
it smoothly.
OPTICS steps:
1. For each unvisited point:
o Mark it as visited.
o Retrieve its neighbors.
o Compute core/reachability distances.
o Update neighbors' distances if necessary.
o Add them to a priority queue.
2. This creates an ordered list of points, based on reachability.
3. You can extract clusters by setting a reachability threshold — or visualize the
reachability plot.

📉 Reachability Plot
This is how clusters are found!
 The valleys in the plot = dense clusters
 The peaks = sparse areas or noise
Unlike DBSCAN, which gives a single clustering result, OPTICS lets you visually explore
clusters at multiple density levels.

Gaussian Mixture Model (GMM)

🎯 What is a Mixture Model?
A Mixture Model assumes that the data is generated from a mixture of several underlying
probability distributions, where each distribution represents a cluster.
The most common one? 👉 Gaussian Mixture Model (GMM)

🧪 What is a Gaussian Mixture Model (GMM)?

A GMM is a probabilistic model that assumes each cluster is a Gaussian (normal)
distribution.
Instead of assigning each point to a cluster hard like K-Means, GMM gives each point a
probability (soft assignment) of belonging to a cluster.
Example: Point X has
 80% chance of being in Cluster 1
 20% in Cluster 2

🧠 How It Works (in steps):

1. Assume k clusters, each with its own Gaussian.
2. Initialize:
o Mean (μ), variance (σ²), and mixing weight (π) for each cluster.
3. Use Expectation-Maximization (EM) algorithm:
E-Step:
o For each point, calculate the probability it belongs to each cluster.
M-Step:
o Update the parameters (means, variances, and weights) based on those
probabilities.
4. Repeat until convergence.