Clustering
By Jazib Ali
Clustering in Data Science
Clustering is a fundamental technique in data science and
machine learning used to group data points into clusters or
subsets such that data points within the same cluster are more
similar to each other than to those in other clusters. It is an
unsupervised learning method, meaning that the algorithm tries
to identify patterns in the data without using pre-labeled
outcomes.
1. Introduction to Clustering
What is Clustering?
Clustering is the task of dividing a set of data points into
groups or clusters, where:
• Intra-cluster similarity – Data points within a cluster are as
similar as possible.
• Inter-cluster dissimilarity – Data points in different clusters
are as different as possible.
Why Clustering?
Clustering is widely used in various fields, including:
✅ Market segmentation – Identifying customer groups based
on behavior.
✅ Document clustering – Organizing large sets of documents
based on content similarity.
✅ Image segmentation – Identifying different objects or
regions in an image.
✅ Anomaly detection – Identifying unusual patterns or
outliers in data.
✅ Recommendation systems – Grouping users with similar
preferences.
2. Types of Clustering
Clustering techniques can be broadly classified into four types:
(a) Centroid-based Clustering
• Based on finding the center of a cluster (called a centroid).
• The most common algorithm is K-Means.
• Tends to work well with spherical clusters of similar size.
• Example: Grouping customers based on spending patterns.
2. Types of Clustering (Cont.)
(b) Hierarchical Clustering
• Builds a tree-like hierarchy of clusters (dendrogram).
• Can be agglomerative (bottom-up) or divisive (top-down).
• No need to specify the number of clusters in advance.
• Example: Organizing documents based on content similarity.
2. Types of Clustering (Cont.)
(c) Density-based Clustering
• Based on the concept of high-density areas separated by low-
density areas.
• The most common algorithm is DBSCAN (Density-Based
Spatial Clustering of Applications with Noise).
• Effective for identifying clusters of arbitrary shape and
handling noise.
• Example: Identifying geographical clusters based on GPS data.
2. Types of Clustering (Cont.)
(d) Distribution-based Clustering
• Assumes that data is generated from a mixture of probability
distributions.
• The most common algorithm is Gaussian Mixture Models
(GMM).
• Suitable for identifying clusters with soft boundaries.
• Example: Modeling customer behavior using a mixture of
normal distributions.
3. Clustering Algorithms
(a) K-Means Clustering
Objective: Partition data into K clusters by minimizing the variance within
clusters.
Example:
👉 Customer Segmentation: A retail company wants to group customers
based on their spending habits.
• Data includes customer age, income, and spending scores.
• K-Means is used to form clusters of customers with similar spending
patterns to create targeted marketing campaigns.
Other Examples:
• Grouping products based on customer reviews.
• Clustering website visitors based on behavior (e.g., pages visited,
time spent).
3. Clustering Algorithms
Challenges:
• Sensitive to the initial choice of centroids.
• Requires the number of clusters (K) to be defined in advance.
• Poor performance with non-spherical clusters.
3. Clustering Algorithms
(b) Hierarchical Clustering
1. Objective: Create a tree-like structure of nested clusters.
2. Types:
1. Agglomerative – Start with each point as its own cluster, then
merge.
2. Divisive – Start with one large cluster, then split iteratively.
3. Linkage Criteria:
1. Single Linkage – Minimum distance between points in two clusters.
2. Complete Linkage – Maximum distance between points in two
clusters.
3. Average Linkage – Average distance between points in two
clusters.
3. Clustering Algorithms
Example:
👉 Document Clustering: A company wants to organize a large set
of documents based on content similarity.
• Text data is processed using word embeddings.
• Agglomerative clustering groups documents with similar
content into a tree structure.
Other Examples:
• Organizing customer complaints into categories.
• Grouping genetic sequences based on similarity.
3. Clustering Algorithms
Challenges:
• Computationally expensive for large datasets.
• Sensitive to noise and outliers.
Example:
👉 Grouping articles based on shared content.
3. Clustering Algorithms
(c) DBSCAN (Density-Based Spatial Clustering of Applications with
Noise)
Objective: Group points that are closely packed together based on
density.
Example:
👉 Social Media Behavior: A social media company wants to identify
clusters of users who interact similarly.
• Data includes likes, comments, and shares.
• DBSCAN groups users into clusters based on similar engagement
patterns.
• Outliers (like bots or fake accounts) are treated as noise.
Other Examples:
• Detecting fraud in financial transactions.
• Clustering GPS locations to identify popular meeting points.
3. Clustering Algorithms
Challenges:
• Sensitive to parameter tuning (ε and MinPts).
• Struggles with varying density clusters.
Example:
👉 Identifying clusters of similar behavior in social media
interactions.
3. Clustering Algorithms
(d) Gaussian Mixture Models (GMM)
Objective: Model the data as a mixture of several Gaussian
distributions.
Example:
👉 Customer Demographics: A business wants to segment
customers based on demographic and purchasing behavior.
• Data includes age, income, and purchase frequency.
• GMM models the data as a mixture of normal distributions
to find soft clusters (allowing overlap).
Other Examples:
• Image compression (segmenting image pixels).
• Identifying speaker clusters in voice recognition.
3. Clustering Algorithms
Challenges:
• Requires knowledge of the number of clusters.
• Can struggle with high-dimensional data.
Example:
👉 Segmenting customer data based on demographic and
purchasing behavior.
4. Evaluation of Clustering Performance
Since clustering is unsupervised, evaluating the quality of
clusters is challenging. Common evaluation methods include:
✅ Silhouette Score
• Measures how similar a point is to its own cluster compared
to other clusters.
• Ranges from -1 to +1:
• +1 – Points are well clustered.
• 0 – Overlapping clusters.
• -1 – Poorly clustered.
4. Evaluation of Clustering Performance
✅ Inertia (Within-Cluster Sum of Squares – WCSS)
• Measures the compactness of clusters.
• Lower values indicate better clustering.
✅ Davies–Bouldin Index
• Measures the average similarity ratio between clusters.
• Lower values indicate better separation between clusters.
✅ Rand Index
• Measures how well clustering matches ground truth (if
available).
5. Applications of Clustering
Application Area Description Example
Marketing Customer segmentation Grouping customers by
purchase behavior
Healthcare Disease clustering Grouping patients based
on symptoms
Finance Identifying suspicious
Fraud detection transactions
Biology Gene expression analysis Identifying similar gene
patterns
Social Media Community detection Finding similar user
behavior
6. Conclusion
Clustering is a powerful unsupervised learning technique that
helps uncover hidden patterns in data. The choice of clustering
algorithm depends on the data structure, size, and complexity.
Understanding the strengths and weaknesses of each method is
crucial for achieving meaningful insights from clustering tasks.