Clustering in Machine Learning
Clustering is an unsupervised machine learning technique that automatically groups unlabeled data
points into clusters based on shared similarities, such as proximity or data density.
Clustering in machine learning refers to grouping similar data points based on specific characteristics
or patterns. The algorithm identifies underlying relationships in data and organizes it into clusters
that represent these similarities.
Unlike supervised learning, where models learn from labeled data, clustering techniques work on
unlabeled datasets, meaning there’s no predefined output or category. Instead, the algorithm discovers
structure automatically, making it particularly useful for exploratory data analysis.
For example, a retailer can use clustering to identify different customer groups based on purchasing
behavior. Similarly, healthcare organizations can cluster patient data to predict disease risk
categories.
How Clustering Works in Machine Learning
Clustering in machine learning follows a structured process that helps group similar data points
together logically and efficiently. It transforms raw, unorganized data into meaningful patterns that
can be analyzed and interpreted. Here’s how it works:
1. Data Preprocessing:
The first step is to clean and prepare the data. Missing values, duplicate entries, and irrelevant features
are removed. The data is then normalized and scaled so that features with larger numerical values do
not overshadow smaller ones.
2. Feature Extraction:
Next, key features that best describe the dataset are selected or transformed. This step ensures that
the algorithm focuses on the most important characteristics of the data.
3. Distance Measurement:
Clustering algorithms rely on distance or similarity metrics to measure how close or far data points
are from one another. Common measures include:
Euclidean distance
Manhattan distance
Cosine similarity
4. Cluster Formation:
After computing the distances, the algorithm groups data points into clusters. Each cluster contains
points that are more similar to each other than to those in other clusters.
5. Evaluation and Refinement:
Finally, the clusters are reviewed and refined. The algorithm iteratively adjusts boundaries to make
each cluster more coherent and meaningful.
Types of Clustering
1. Hard Clustering
Hard clustering assigns each data point to exactly one cluster. A data point cannot belong to multiple
clusters, making the grouping clear and easy to interpret.
Each data point belongs to only one cluster
No overlap between clusters
Simple and easy to interpret
If customers are divided into two clusters, each customer belongs completely to either Cluster 1 or
Cluster 2. A customer cannot belong to both clusters at the same time.
Common Uses
Market segmentation: Businesses group customers with similar buying behaviour to design
targeted marketing strategies.
Customer grouping: Companies organize customers into clear categories for better service
and analysis.
Document clustering: Documents with similar topics or keywords are grouped together for
easier organization.
2. Soft Clustering
Soft clustering allows a data point to belong to multiple clusters with different probabilities. Instead of
assigning a strict cluster, it gives a degree of membership to each cluster.
Example
A data point may belong 70% to Cluster 1 and 30% to Cluster 2, indicating that it shares
characteristics with both groups.
Use Cases
Overlapping class boundaries: Useful when data points cannot be clearly separated into
distinct groups.
Customer personas: Helps represent customers who share traits with multiple behavioral
groups.
Medical diagnosis: Patients may show symptoms related to multiple condition
Cluster Formation Methods
It is not necessary that clusters will be formed in spherical form. Followings are some other
cluster formation methods
Partitioning
In these methods, the clusters are formed by portioning the objects into k clusters. Number of
clusters will be equal to the number of partitions. Ex. K-means, Clustering Large Applications
based upon randomized Search (CLARANS).
Centroid based Clustering
Centroid based clustering groups data points around central points called centroids. Each cluster is
represented by the average of its points and data points are assigned to the nearest centroid.
Algorithms:
K-means: Iteratively assigns points to nearest centroid and recalculates centroids to
minimize intra cluster variance.
K-medoids: Similar to K-means but uses actual data points (medoids) as centers, robust to
outliers.
Advantages:
Fast and scalable for large datasets.
Simple to implement and interpret.
[Link] Based Clustering
In these methods, the clusters are formed as the dense region. The advantage of these methods is that
they have good accuracy as well as good ability to merge two clusters. Ex. Density-Based Spatial
Clustering of Applications with Noise (DBSCAN), Ordering Points to identify Clustering structure
(OPTICS) etc.
3. Hierarchical-based
In these methods, the clusters are formed as a tree type structure based on the hierarchy. They have
two categories namely, Agglomerative (Bottom up approach) and Divisive (Top down approach). Ex.
Clustering using Representatives (CURE), Balanced iterative Reducing Clustering using Hierarchies
(BIRCH) etc.
Applications
Clustering is widely used in data analysis and machine learning to identify patterns in unlabelled data.
Customer Segmentation: Group customers based on behaviour or demographics.
Anomaly Detection: Detect unusual activities in finance, security or sensor data.
Image Segmentation: Divide images into meaningful regions for computer vision tasks.
Recommendation Systems: Group similar users or items for personalized suggestions.
Market Basket Analysis: Identify products frequently purchased together.