Chapter 20: Cluster Analysis

Advance Marketing Research

 Chapter Outline
1) Overview
2) Basic Concept
3) Statistics Associated with Cluster Analysis
4) Conducting Cluster Analysis
i. Formulating the Problem
ii. Selecting a Distance or Similarity Measure
iii. Selecting a Clustering Procedure
iv. Deciding on the Number of Clusters
v. Interpreting and Profiling the Clusters
vi. Assessing Reliability and Validity
5) Applications of Nonhierarchical Clustering
6) Clustering Variables
7) Summary
 Cluster Analysis
• Cluster analysis is a class of techniques used to classify objects
or cases into relatively homogeneous groups called clusters.
• Objects in each cluster tend to be similar to each other and
dissimilar to objects in the other clusters. Cluster analysis is
also called classification analysis, or numerical taxonomy.
• Both cluster analysis and discriminant analysis are concerned
with classification. However, discriminant analysis requires
prior knowledge of the cluster or group membership for each
object or case included, to develop the classification rule.
• In contrast, in cluster analysis there is no a priori information
about the group or cluster membership for any of the objects.
• Groups or clusters are suggested by the data, not defined a
priori.
An Ideal Clustering Situation
Fig. 20.1

Variable 1

Variable 2
A Practical Clustering Situation

Fig. 20.2

Variable 1

X
Variable 2
 Statistics Associated with Cluster
Analysis
• Agglomeration schedule: An agglomeration schedule
gives information on the objects or cases being combined
at each stage of a hierarchical clustering process.
• Cluster centroid: The cluster centroid is the mean
values of the variables for all the cases or objects in a
particular cluster.
• Cluster centers: The cluster centers are the initial
starting points in nonhierarchical clustering. Clusters are
built around these centers, or seeds.
• Cluster membership: Cluster membership indicates the
cluster to which each object or case belongs.
• Dendrogram: A dendrogram, or tree graph, is a
graphical device for displaying clustering results.
Vertical lines represent clusters that are joined
together.
• The position of the line on the scale indicates the
distances at which clusters were joined. The
dendrogram is read from left to right. Figure 20.8
is a dendrogram.
• Distances between cluster centers: These
distances indicate how separated the individual
pairs of clusters are. Clusters that are widely
separated are distinct, and therefore desirable.
Fig. 20.8
• Icicle plot: An icicle plot is a graphical display of
clustering results, so called because it resembles a
row of icicles hanging from the eaves of a house.
• The columns correspond to the objects being
clustered, and the rows correspond to the number
of clusters. An icicle plot is read from bottom to
top. Figure 20.7 is an icicle plot.
• Similarity/distance coefficient matrix: A
similarity/distance coefficient matrix is a lower-
triangle matrix containing pairwise distances
between objects or cases.
Fig. 20.7
 Conducting Cluster Analysis
Fig. 20.3

Formulate the Problem

Select a Distance Measure

Select a Clustering Procedure

Decide on the Number of Clusters

Interpret and Profile Clusters

Assess the Validity of Clustering

 Formulate the Problem
• Perhaps the most important part of formulating the
clustering problem is selecting the variables on which the
clustering is based.
• Inclusion of even one or two irrelevant variables may
distort an otherwise useful clustering solution.
• Basically, the set of variables selected should describe the
similarity between objects in terms that are relevant to the
marketing research problem.
• The variables should be selected based on past research,
theory, or a consideration of the hypotheses being tested.
In exploratory research, the researcher should exercise
judgment and intuition.
 Attitudinal Data For Clustering
Table 20.1

Case No. V1 V2 V3 V4 V5 V6
1 6 4 7 3 2 3
2 2 3 1 4 5 4
3 7 2 6 4 1 3
4 4 6 4 5 3 6
5 1 3 2 2 6 4
6 6 4 6 3 3 4
7 5 3 6 3 3 4
8 7 3 7 4 1 4
9 2 4 3 3 6 3
10 3 5 3 6 4 6
11 1 3 2 3 5 3
12 5 4 5 4 2 4
13 2 2 1 5 4 4
14 4 6 4 6 4 7
15 6 5 4 2 1 4
16 3 5 4 6 4 7
17 4 4 7 2 2 5
18 3 7 2 6 4 3
19 4 6 3 7 2 7
20 2 3 2 4 7 2
Select a Distance or Similarity Measure
• The most commonly used measure of similarity is the
Euclidean distance or its square.
• The Euclidean distance is the square root of the sum
of the squared differences in values for each variable.
Other distance measures are also available.
• The city-block or Manhattan distance between two
objects is the sum of the absolute differences in
values for each variable.
• The Chebychev distance between two objects is the
maximum absolute difference in values for any
variable.
• If the variables are measured in vastly different
units, the clustering solution will be influenced by
the units of measurement.
• In these cases, before clustering respondents, we
must standardize the data by rescaling each
variable to have a mean of zero and a standard
deviation of unity. It is also desirable to eliminate
outliers (cases with atypical values).
• Use of different distance measures may lead to
different clustering results. Hence, it is advisable
to use different measures and compare the results.
 A Classification of Clustering Procedures
Fig. 20.4 Clustering Procedures

Hierarchical Nonhierarchical Other

Agglomerative Divisive Two-Step

Linkage Variance Centroid Sequential Parallel Optimizing

Methods Methods Methods Threshold Threshold Partitioning

Ward’s
Method

Single Complete Average

Linkage Linkage Linkage
Select a Clustering Procedure–Hierarchical
• Hierarchical clustering is characterized by the development
of a hierarchy or tree-like structure. Hierarchical methods can
be agglomerative or divisive.
• Agglomerative clustering starts with each object in a
separate cluster. Clusters are formed by grouping objects into
bigger and bigger clusters. This process is continued until all
objects are members of a single cluster.
• Divisive clustering starts with all the objects grouped in a
single cluster. Clusters are divided or split until each object is
in a separate cluster.
• Agglomerative methods are commonly used in marketing
research. They consist of linkage methods, error sums of
squares or variance methods, and centroid methods.
Select a Clustering Procedure – Linkage Method
• The single linkage method is based on minimum distance, or
the nearest neighbor rule. At every stage, the distance
between two clusters is the distance between their two
closest points (see Figure 20.5).
• The complete linkage method is similar to single linkage,
except that it is based on the maximum distance or the
furthest neighbor approach.
• In complete linkage, the distance between two clusters is
calculated as the distance between their two furthest points.
• The average linkage method works similarly. However, in
this method, the distance between two clusters is defined as
the average of the distances between all pairs of objects,
where one member of the pair is from each of the clusters
(Figure 20.5).
 Linkage Methods of Clustering
Fig. 20.5 Single Linkage
Minimum Distance

Cluster 1 Cluster 2
Complete Linkage
Maximum
Distance

Cluster 1 Cluster 2
Average Linkage

Average Distance
Cluster 1 Cluster 2
 Select a Clustering Procedure – Variance
Method
• The variance methods attempt to generate clusters to minimize
the within-cluster variance.
• A commonly used variance method is the Ward's procedure. For
each cluster, the means for all the variables are computed. Then,
for each object, the squared Euclidean distance to the cluster
means is calculated (Figure 20.6). These distances are summed
for all the objects. At each stage, the two clusters with the
smallest increase in the overall sum of squares within cluster
distances are combined.
• In the centroid methods, the distance between two clusters is the
distance between their centroids (means for all the variables), as
shown in Figure 20.6. Every time objects are grouped, a new
centroid is computed.
• Of the hierarchical methods, average linkage and Ward's methods
have been shown to perform better than the other procedures.
Other Agglomerative Clustering Methods
Fig. 20.6
Ward’s Procedure

Centroid Method
 Select a Clustering Procedure –
Nonhierarchical
• The nonhierarchical clustering methods are
frequently referred to as k-means clustering. These
methods include sequential threshold, parallel
threshold, and optimizing partitioning.
• In the sequential threshold method, a cluster center
is selected and all objects within a pre-specified
threshold value from the center are grouped together.
• Then a new cluster center or seed is selected, and the
process is repeated for the un-clustered points.
• Once an object is clustered with a seed, it is no longer
considered for clustering with subsequent seeds.
• The parallel threshold method operates
similarly, except that several cluster centers are
selected simultaneously and objects within the
threshold level are grouped with the nearest
center.
• The optimizing partitioning method differs from
the two threshold procedures in that objects can
later be reassigned to clusters to optimize an
overall criterion, such as average within cluster
distance for a given number of clusters.
• It has been suggested that the hierarchical and nonhierarchical
methods be used in tandem.
• First, an initial clustering solution is obtained using a hierarchical
procedure, such as average linkage or Ward's. The number of
clusters and cluster centroids so obtained are used as inputs to
the optimizing partitioning method.
• Choice of a clustering method and choice of a distance measure
are interrelated.
• For example, squared Euclidean distances should be used with
the Ward's and centroid methods. Several nonhierarchical
procedures also use squared Euclidean distances.
Results of Hierarchical Clustering
Table 20.2
Agglomeration Schedule Using Ward’s Procedure
Stage cluster
Clusters combined first appears
Stage Cluster 1 Cluster 2 Coefficient Cluster 1 Cluster 2 Next stage
1 14 16 1.000000 0 0 6
2 6 7 2.000000 0 0 7
3 2 13 3.500000 0 0 15
4 5 11 5.000000 0 0 11
5 3 8 6.500000 0 0 16
6 10 14 8.160000 0 1 9
7 6 12 10.166667 2 0 10
8 9 20 13.000000 0 0 11
9 4 10 15.583000 0 6 12
10 1 6 18.500000 6 7 13
11 5 9 23.000000 4 8 15
12 4 19 27.750000 9 0 17
13 1 17 33.100000 10 0 14
14 1 15 41.333000 13 0 16
15 2 5 51.833000 3 11 18
16 1 3 64.500000 14 5 19
17 4 18 79.667000 12 0 18
18 2 4 172.662000 15 17 19
19 1 2 328.600000 16 18 0
Table 20.2, cont.

Cluster Membership of Cases Using Ward’s Procedure

Number of Clusters
Label case 4 3 2

1 1 1 1
2 2 2 2
3 1 1 1
4 3 3 2
5 2 2 2
6 1 1 1
7 1 1 1
8 1 1 1
9 2 2 2
10 3 3 2
11 2 2 2
12 1 1 1
13 2 2 2
14 3 3 2
15 1 1 1
16 3 3 2
17 1 1 1
18 4 3 2
19 3 3 2
20 2 2 2
 Vertical Icicle Plot Using Ward’s Method
Fig. 20.7
Dendrogram Using Ward’s Method
Fig. 20.8
 Decide on the Number of Clusters
• Theoretical, conceptual, or practical considerations may
suggest a certain number of clusters.
• In hierarchical clustering, the distances at which
clusters are combined can be used as criteria. This
information can be obtained from the agglomeration
schedule or from the dendrogram.
• In nonhierarchical clustering, the ratio of total within-
group variance to between-group variance can be
plotted against the number of clusters. The point at
which an elbow or a sharp bend occurs indicates an
appropriate number of clusters.
• The relative sizes of the clusters should be meaningful.
 Interpreting and Profiling the Clusters
• Interpreting and profiling clusters involves
examining the cluster centroids. The centroids
enable us to describe each cluster by assigning it a
name or label.

• It is often helpful to profile the clusters in terms of

variables that were not used for clustering.

• These may include demographic, psychographic,

product usage, media usage, or other variables.
 Cluster Centroids
Table 20.3

Means of Variables

Cluster V1 V2 V3 V4 V5 V6

1 5.750 3.625 6.000 3.125 1.750 3.875

2 1.667 3.000 1.833 3.500 5.500 3.333

3 3.500 5.833 3.333 6.000 3.500 6.000

 Assess Reliability and Validity
1. Perform cluster analysis on the same data using different
distance measures. Compare the results across measures to
determine the stability of the solutions.
2. Use different methods of clustering and compare the results.
3. Split the data randomly into halves. Perform clustering
separately on each half. Compare cluster centroids across the
two subsamples.
4. Delete variables randomly. Perform clustering based on the
reduced set of variables. Compare the results with those
obtained by clustering based on the entire set of variables.
5. In nonhierarchical clustering, the solution may depend on the
order of cases in the data set. Make multiple runs using
different order of cases until the solution stabilizes.
 Results of Nonhierarchical Clustering
Table 20.4
Initial Cluster Centers
Cluster
1 2 3
V1 4 2 7
V2 6 3 2
V3 3 2 6
V4 7 4 4
V5 2 7 1
V6 7 2 3
a
Iteration History
Change in Cluster Centers
Iteration 1 2 3
1 2.154 2.102 2.550
2 0.000 0.000 0.000
a. Convergence achieved due to no or small distance
change. The maximum distance by which any center
has changed is 0.000. The current iteration is 2. The
minimum distance between initial centers is 7.746.
Table 20.4 cont.
Cluster Membership
Case Number Cluster Distance
1 3 1.414
2 2 1.323
3 3 2.550
4 1 1.404
5 2 1.848
6 3 1.225
7 3 1.500
8 3 2.121
9 2 1.756
10 1 1.143
11 2 1.041
12 3 1.581
13 2 2.598
14 1 1.404
15 3 2.828
16 1 1.624
17 3 2.598
18 1 3.555
19 1 2.154
20 2 2.102
Table 20.4, cont.

Final Cluster Centers

Cluster
1 2 3
V1 4 2 6
V2 6 3 4
V3 3 2 6
V4 6 4 3
V5 4 6 2
V6 6 3 4

Distances between Final Cluster Centers

Cluster 1 2 3
1 5.568 5.698
2 5.568 6.928
3 5.698 6.928
Table 20.4, cont.

ANOVA
Cluster Error
Mean Square df Mean Square df F Sig.
V1 29.108 2 0.608 17 47.888 0.000
V2 13.546 2 0.630 17 21.505 0.000
V3 31.392 2 0.833 17 37.670 0.000
V4 15.713 2 0.728 17 21.585 0.000
V5 22.537 2 0.816 17 27.614 0.000
V6 12.171 2 1.071 17 11.363 0.001
The F tests should be used only for descriptive purposes because the clusters have been
chosen to maximize the differences among cases in different clusters. The observed
significance levels are not corrected for this, and thus cannot be interpreted as tests of the
hypothesis that the cluster means are equal.

Number of Cases in each Cluster

Cluster 1 6.000
2 6.000
3 8.000
Valid 20.000
Missing 0.000
 Results of Two-Step Clustering
Table 20.5 Auto-Clustering

Akaike's Ratio of
Information AIC Ratio of AIC Distance
Number of Clusters Criterion (AIC) Change(a) Changes(b) Measures(c)
1 104.140
2 101.171 -2.969 1.000 .847
3 97.594 -3.577 1.205 1.583
4 116.896 19.302 -6.502 2.115
5 138.230 21.335 -7.187 1.222
6 158.586 20.355 -6.857 1.021
7 179.340 20.755 -6.991 1.224
8 201.628 22.288 -7.508 1.006
9 224.055 22.426 -7.555 1.111
10 246.522 22.467 -7.568 1.588
11 269.570 23.048 -7.764 1.001
12 292.718 23.148 -7.798 1.055
13 316.120 23.402 -7.883 1.002
14 339.223 23.103 -7.782 1.044
15 362.650 23.427 -7.892 1.004
a The changes are from the previous number of clusters in the table.
b The ratios of changes are relative to the change for the two cluster solution.
c The ratios of distance measures are based on the current number of clusters
against the previous number of clusters.
 Cluster Distribution
Table 20.5, cont.

% of
N Combined % of Total
Cluster 1 6 30.0% 30.0%
2 6 30.0% 30.0%
3 8 40.0% 40.0%
Combined 20 100.0% 100.0%
Total 20 100.0%
 Cluster Profiles
Table 20.5, cont.

Fun Bad for Budget Eating Out

Mean Std. Deviation Mean Std. Deviation Mean Std. Deviation
Cluster 1 1.67 .516 3.00 .632 1.83 .753
2 3.50 .548 5.83 .753 3.33 .816
3 5.75 1.035 3.63 .916 6.00 1.069
Combined 3.85 1.899 4.10 1.410 3.95 2.012

Best Buys Don't Care Compare Prices

Mean Std. Deviation Mean Std. Deviation Mean Std. Deviation
3.50 1.049 5.50 1.049 3.33 .816
6.00 .632 3.50 .837 6.00 1.549
3.13 .835 1.88 .835 3.88 .641
4.10 1.518 3.45 1.761 4.35 1.496
 Clustering Variables
• In this instance, the units used for analysis are the variables, and
the distance measures are computed for all pairs of variables.
• Hierarchical clustering of variables can aid in the identification
of unique variables, or variables that make a unique contribution
to the data.
• Clustering can also be used to reduce the number of variables.
Associated with each cluster is a linear combination of the
variables in the cluster, called the cluster component.
• A large set of variables can often be replaced by the set of cluster
components with little loss of information.
• However, a given number of cluster components does not
generally explain as much variance as the same number of
principal components.