Chapter Four
cluster analysis
What is cluster analysis?
Types of data in cluster analysis
Categorization of major clustering methods
Partitioning methods
Hierarchical methods
Density based methods & Outlier analysis
1
What is cluster analysis?
Clustering or cluster analysis is a machine learning technique, which groups the
unlabelled dataset.
A way of grouping the data points into different clusters, consisting of similar data
points. The objects with the possible similarities remain in a group that has less or
no similarities with another group.
It does it by finding some similar patterns in the unlabelled dataset such as shape,
size, color, behavior, etc., and divides them as per the presence and absence of
those similar patterns.
It is an unsupervised learning method, hence no supervision is provided to the
algorithm, and it deals with the unlabelled dataset.
2
---
Cluster analysis is the process of finding similar
groups of objects/data/ in the form of cluster.
Properties of cluster
-cluster scalability
-algorithm usability with different types of data.
- Dealing with unstructured data
- Interoperability
3
What is a Good cluster
• The quality of a clustering result depends on:
– The similarity measure used by the method
and its implementation.
– Its ability to discover some or all of the hidden patterns.
• A good clustering method will produce high quality clusters in
which:
– The intra-class similarity is low.
– The inter-class similarity is high.
Basic Steps in Clustering
Feature Selection– minimal information redundancy Choose relevant
features or transform the data into a suitable representation for clustering
Proximity measure
Similarity of feature vectors
Clustering criterion
Expressed via a cost function or some rules
Clustering algorithms – choice
Validation of the result
Interpretation of the result – integration with application
Types of data in cluster
analysis
Cluster analysis can be applied to various types of data, including:
Numerical data: Data represented by numbers
Numerical data consists of values that are continuous or discrete numbers.
These values can be measured and represented on a scale.
Example: Height, weight, temperature, income, and stock prices.
Categorical data: Categorical data consists of categories or labels that represent
qualitative attributes.
Example: Gender (male, female), marital status (single, married, divorced), and
educational level (high school, college, graduate).
6
Types of data in cluster
analysis
Mixed data: Data containing a mixture of numerical and categorical variables.
Mixed data includes a combination of numerical, categorical, and possibly other types
of data within the same dataset.
Example: Customer data with numerical features like age and income, categorical
features like occupation and marital status, and binary features like subscription status.
Binary data: Data with only two possible values (e.g., yes/no, true/false).
Binary data consists of variables with only two possible values, often represented as 0
and 1.
Example: Yes/no responses, true/false indicators, and presence/absence of a feature.
Text Data:
Text data consists of unstructured text documents, such as reviews, tweets, or articles.
Example: Customer reviews, product descriptions, and social media posts.
Image Data:
Image data consists of visual information in the form of pixels arranged in a grid.
Example: Photographs, medical images, and satellite imagery.
7
clustering methods
Clustering methods can be categorized into several major types:
1. Partitioning methods: Divide the data into non-
overlapping clusters.
2. Hierarchical methods: Create a tree of clusters, showing
nested relationships.
3. Density-based methods: Form clusters based on dense
regions in the data space.
8
Partitioning methods/K-means
clustering
Process of dividing n data items/objects in to k partitions/groups/parts.
Size of k in cluster must be (K<=N) where N is total number of dataset
Partitioning methods refer to techniques used in various fields to divide a set of
items or data into distinct groups or subsets based on certain criteria or
characteristics.
Partitioning methods aim to partition the data into a predetermined number of
clusters.
The most common partitioning method is K-means clustering, where K represents
the number of clusters.
9
Rule of partitioning
1. Each partition must have at least
one object.
2. Each object should belong only one
partition.
10
Example
Apply K- Means algorithm over the data (185, 72), (170, 56),
(168, 60), (179,68), (182,72), (188,77) up to two iterations and
show the clusters. Initially choose first two objects as initial
centroids.(use k=2)
11
Solution
Given, number of clusters to be created (K) = 2 say c1 and c2,
number of iterations = 2 and
The given data points can be represented in tabular form as:
12
..
first select two objects as initial centroids randomly :
Centroid for first cluster c1 = (185, 72)
Centroid for second cluster c2 = (170, 56)
13
-
14
Representing above information in tabular form:
15
Iteration 2: Now calculating centroid for
each cluster:
Now, again calculating similarity:
16
.
17
Representing above information in tabular form
18
..
As we have already completed two iteration as asked by our
question, the numerical ends here.
Since, the clustering doesn’t change after second iteration, so
terminate the iteration even if question doesn’t say so.
19
Homework
Hierarchical Clustering
Density-based clustering
20