4

Multivariate Analysis
■ Multivariate Analysis Overview, 4-2
■ Principal Components, 4-3
■ Factor Analysis, 4-6
■ Discriminant Analysis, 4-16
■ Clustering of Observations, 4-22
■ Clustering of Variables, 4-30
■ K-Means Clustering of Observations, 4-33
■ Balanced MANOVA, 4-38
■ General MANOVA, 4-44
See also,
■ Correspondence Analysis. See Simple Correspondence Analysis on page 6-20
or Multiple Correspondence Analysis on page 6-30 to perform a
decomposition of a cross-classification.

To Table of Contents To Index

MINITAB User’s Guide 2 4-1

Chapter 4 Multivariate Analysis Overview

Multivariate Analysis Overview

Use MINITAB’s multivariate analysis procedures to analyze your data when you have
made multiple measurements on items or subjects. You can choose a method
depending on whether you want to
■ analyze the data covariance structure for the sake of understanding it or to reduce
the data dimension
■ assign observations to groups
■ test the equality of mean vectors using MANOVA (multivariate analysis of variance)
Analyzing the data covariance structure and assigning observations to groups are
characterized by their non-inferential nature, that is, tests of significance are not
computed. There may be no single answer but what may work best for your data may
require knowledge of the situation. Inference tests can be computed, however, with
MANOVA’s tests of the equality of mean vectors.

Analysis of the data structure

MINITAB offers two procedures that you can use to analyze the data covariance
structure:
■ Principal Components Analysis is used to help you to understand the covariance
structure in the original variables and/or to create a smaller number of variables
using this structure.
■ Factor Analysis, like principal components, is used to summarize the data
covariance structure in a smaller number of dimensions. The emphasis in factor
analysis, however, is the identification of underlying “factors” that might explain the
dimensions associated with large data variability.

Grouping observations
MINITAB offers discriminant analysis and three-cluster analysis methods for grouping
observations:
■ Discriminant Analysis is used for classifying observations into two or more groups if
you have a sample with known groups. Discriminant analysis can also used to
investigate how the predictors contribute to the groupings.
■ Cluster Observations is used to group or cluster observations that are “close” to
each other, when the groups are initially unknown. This method is a good choice
when there is no outside information about grouping. The choice of final grouping
is usually made according to what makes sense for your data after viewing clustering
statistics.

4-2 MINITAB User’s Guide 2

Principal Components Multivariate Analysis

■ Cluster Variables is used to group or cluster variables that are “close” to each other,
when the groups are initially unknown. The procedure is similar to clustering of
observations. One reason to cluster variables may be to reduce their number.
■ K-means clustering, like clustering of observations, is used to group observations
that are “close” to each other. K-means clustering works best when sufficient
information is available to make good starting cluster designations.

Testing the equality of means from multiple response

Balanced MANOVA and general MANOVA are procedures for testing the equality of
vectors of means from multiple responses. Your choice between these two procedures
depends upon the experimental design and the available options. Both these
procedures can fit MANOVA models to balanced data with up to 9 factors.
■ Balanced MANOVA is used to perform multivariate analysis of variance with
balanced designs. See Balanced designs on page 3-18. You can also specify factors to
be random and obtain expected means squares. Use general MANOVA with
unbalanced designs.
■ General MANOVA is used to perform multivariate analysis of variance with either
balanced or unbalanced designs that can also include covariates. You cannot specify
factors to be random as you can for balanced MANOVA, though you can work
around this restriction by specifying the error term for testing different model terms.
Here is a summary of the differences between Balanced and General MANOVA:

Balanced General
MANOVA MANOVA
Can fit unbalanced data no yes

Can specify factors as random and

yes no
obtain expected means squares

Fits up to 50 covariates no yes

Principal Components
Use principal component analysis to help you to understand the underlying data
structure and/or form a smaller number of uncorrelated variables (for example, to avoid
multicollinearity in regression).

MINITAB User’s Guide 2 4-3

Chapter 4 Principal Components

Data
Set up your worksheet so that a row contains measurements on a single item or subject.
You must have two or more numeric columns, with each column representing a
different measurement (response). MINITAB automatically omits rows with missing data
from the analysis.

h To perform principal component analysis

1 Choose Stat ➤ Multivariate ➤ Principal Components.

2 In Variables, enter the columns containing the measurement data.

3 If you like, use one or more of the options listed below, then click OK.

Options

Principal Components dialog box

■ specify the number of principal components to calculate (the default number is the
number of variables).
■ use the covariance matrix rather than the correlation matrix to calculate the
principal components. Use the correlation matrix if it makes sense to standardize
variables (the usual choice when variables are measured by different scales); use the
covariance matrix if you do not wish to standardize.

Graphs subdialog box

■ display a scree plot, also called an eigenvalue profile plot. Scree plots display the
eigenvalues versus their order. Use this plot to judge the relative magnitude of
eigenvalues.

4-4 MINITAB User’s Guide 2

Principal Components Multivariate Analysis

■ display a plot of the second principal component scores (y-axis) versus the first
principal component scores (x-axis). You can also create plots for other components,
by storing the scores and using Graph ➤ Plot.

Storage subdialog box

■ store the coefficients and scores of the principal components. Coefficients are
eigenvector coefficients and scores are the linear combinations of your data using
the coefficients.

Nonuniqueness of coefficients
The coefficients are unique (except for a change in sign) if the eigenvalues are distinct
and not zero. If an eigenvalue is repeated, then the “space spanned” by all the principal
component vectors corresponding to the same eigenvalue is unique, but the individual
vectors are not. Therefore, the coefficients that MINITAB prints and those in a book or
another program may not agree, though the eigenvalues (variances) will always be the
same.
If the covariance matrix has rank r < p, where p is the number of variables, then there
will be p − r eigenvalues equal to zero. Eigenvectors corresponding to these eigenvalues
may not be unique. This can happen if the number of observations is less than p or if
there is multicollinearity.

e Example of principal components analysis

You record the following characteristics for 14 census tracts: total population (Pop),
median years of schooling (School), total employment (Employ), employment in
health services (Health), and median home value (Home). The data were obtained
from [5], Table 8.2.
You wish to understand the underlying data structure and so you perform principal
components analysis. You use the correlation matrix to standardize the measurements
because they are not measured with the same scale.
1 Open the file EXH_MVAR.MTW.

2 Choose Stat ➤ Multivariate ➤ Principal Components.

3 Type Pop-Home in Variables. Choose Correlation under Type of Matrix. Click

OK.

MINITAB User’s Guide 2 4-5

Chapter 4 Factor Analysis

Session Principal Component Analysis

window
output Eigenanalysis of the Correlation Matrix
Eigenvalue 3.0289 1.2911 0.5725 0.0954 0.0121
Proportion 0.606 0.258 0.114 0.019 0.002
Cumulative 0.606 0.864 0.978 0.998 1.000
Variable PC1 PC2 PC3 PC4 PC5
Pop -0.558 -0.131 0.008 0.551 -0.606
School -0.313 -0.629 -0.549 -0.453 0.007
Employ -0.568 -0.004 0.117 0.268 0.769
Health -0.487 0.310 0.455 -0.648 -0.201
Home 0.174 -0.701 0.691 0.015 0.014

Interpreting the results

The first principal component has variance (eigenvalue) 3.0289 and accounts for
60.6% of the total variance. The coefficients listed under PC1 show how it was
calculated:
PC1 = −.558 Pop − .313 School − .568 Employ − .487 Health + .174 Home
One could think of the first principal component as representing an overall population
size, level of schooling, employment level, and employment in health services effect,
because the coefficients of these terms have the same sign and are not close to zero.
The second principal component has variance 1.2911 and accounts for 25.8% of the
data variability. It is calculated from the original data using the coefficients listed under
PC2. This component could be thought of as contrasting level of schooling and home
value with health employment to some extent.
Together, the first two and the first three principal components represent 86.4% and
97.8%, respectfully, of the total variability. Thus, most of the data structure can be
captured in two or three underlying dimensions. The remaining principal components
account for a very small proportion of the variability and are probably unimportant.

Factor Analysis
Use factor analysis, like principal components analysis, to summarize the data
covariance structure in a few dimensions of the data. However, the emphasis in factor
analysis is the identification of underlying “factors” that might explain the dimensions
associated with large data variability.

4-6 MINITAB User’s Guide 2

Factor Analysis Multivariate Analysis

Data
You can have three types of input data:
■ columns of raw data
■ a matrix of correlations or covariances
■ columns containing factor loadings
The typical case is to use raw data. Set up your worksheet so that a row contains
measurements on a single item or subject. You must have two or more numeric
columns, with each column representing a different measurement (response).
MINITAB automatically omits rows with missing data from the analysis. If you want to
store coefficients, factor scores, or the residual matrix, or view an eigenvalue or scores
plot, you must enter raw data.
Usually the factor analysis procedure calculates the correlation or covariance matrix
from which the loadings are calculated. However, you can enter a matrix as input data.
You can also enter both raw data and a matrix of correlations or covariances. If you do,
MINITAB uses the matrix to calculate the loadings. MINITAB then uses these loadings
and the raw data to calculate storage values and generate graphs. See Using a matrix as
input data on page 4-11.
If you store factor initial loadings, you can later input these initial loadings to examine
the effect of different rotations. You can also use stored loadings to predict factor scores
of new data. See Using stored loadings as input data on page 4-11.

h To perform factor analysis with raw data

1 Choose Stat ➤ Multivariate ➤ Factor Analysis.

2 In Variables, enter the columns containing the measurement data.

3 If you like, use one or more of the options listed below, then click OK.

MINITAB User’s Guide 2 4-7

Chapter 4 Factor Analysis

Options

Factor Analysis dialog box

■ specify the number of factors to extract (required if you use maximum likelihood as
your method of extraction). With principal components extraction, the default
number is the number of variables.
■ use maximum likelihood rather than principal components for the initial solution.
See The maximum likelihood method on page 4-10.
■ perform an equimax, varimax, quartimax, or orthomax rotation of the initial factor
loadings. See Rotating the factor loadings on page 4-10.

Options subdialog box

■ use a covariance matrix rather than the default correlation matrix. Use the
correlation matrix if it makes sense to standardize variables (the usual choice when
variables are measured by different scales); use the covariance matrix if you do not
wish to standardize.
■ enter a covariance or correlation matrix as input data. See Using a matrix as input
data on page 4-11.
■ use stored loadings for the initial solution. See Using stored loadings as input data
on page 4-11.
■ when using maximum likelihood extraction, you can specify:
– initial values for the communalities
– the maximum number of iterations allowed for a solution (by default, 25)
– the criterion for convergence (by default, 0.005)
See The maximum likelihood method on page 4-10.

Graphs subdialog box

■ display a scree plot, also called an eigenvalue profile plot. Scree plots display the
eigenvalues versus their order. Use this plot to judge the relative magnitude of
eigenvalues.
■ display a plot of the factor two scores (y-axis) versus the factor one scores (x-axis). You
can create plots for other factors, by storing the scores and using Graph ➤ Plot.
■ display a plot of the factor two loadings (y-axis) versus the factor one loadings (x-axis).
You can create loadings plots for other factors, by storing the loadings and using
Graph ➤ Plot.

4-8 MINITAB User’s Guide 2

Factor Analysis Multivariate Analysis

Storage subdialog box

■ store the loadings, factor score coefficients, factor or standard scores, the rotation
matrix, residual matrix, eigenvalues, and eigenvectors. See Factor analysis storage on
page 4-12.

Results subdialog box

■ control the amount of Session window output. You can display
– no output
– loadings (and sorted loadings) for the final solution
– the default output, which includes loadings (and sorted loadings) for the final
solution, and factor score coefficients
– the default output, plus information on each iteration when you use maximum
likelihood extraction
■ sort the loadings in the Session window display (within a factor if the maximum
absolute loading occurs there). You can also display all loadings less than a given
value as zero.

Factor analysis in practice

The goal of factor analysis is to find a small number of factors, or unobservable
variables, that explains most of the data variability and yet makes contextual sense.
Some important decisions to be made are deciding how many factors to use, and
finding loadings that make the most sense for your data.
The choice of the number of factors is often based upon the proportion of variance
explained by the factors, subject matter knowledge, and reasonableness of the solution
[5]. Initially, try using the principal components extraction method without specifying
the number of components. Examine the proportion of variability explained by
different factors and narrow down your choice of how many factors to use. An
eigenvalue plot may be useful here in visually assessing the importance of factors.
Once you have narrowed this choice, examine the fits of the different factor analyses.
Communality values, the proportion of variability of each variable explained by the
factors, may be especially useful in comparing fits. You may decide to add a factor if it
contributes to the fit of certain variables. Try the maximum likelihood method of
extraction as well.
Once you have selected the number of factors, you will probably want to try different
rotations. Johnson and Wichern [5] suggest the varimax rotation. A similar result from
different methods can lend credence to the solution you have selected. At this point
you may wish to interpret the factors using your knowledge of the data.

MINITAB User’s Guide 2 4-9

Chapter 4 Factor Analysis

The maximum likelihood method

The maximum likelihood method estimates the factor loadings, assuming the data
follows a multivariate normal distribution. As its name implies, this method finds a
solution by maximizing the likelihood function. Equivalently, this is done by
minimizing an expression involving the variances of the residuals. The algorithm
iterates until a minimum is found or until the maximum specified number of iterations
(the default is 25) is reached. MINITAB uses an algorithm based on [6], with some
adjustments to improve convergence. See Help, [5], or [6] for details.
When minimizing the variance expression, it is possible to find residual variances that
are 0 or negative. To prevent this, MINITAB’s algorithm bounds these values away from
0. Specifically, if a unique variance is less than the value specified for convergence, it is
set equal to this convergence value. Once the algorithm converges, a final check is
done on the unique variances. If any unique values are less than the convergence value
(default is 0.005), they are set equal to 0. The corresponding communality is then equal
to 1. This condition is called a Heywood case and a message is printed to this effect.
Optimization algorithms, such as the one used for maximum likelihood factor analysis,
can give different answers with minor changes in the input. For example, if you change
a few data values, change the starting communality estimates, or change the
convergence value, you may see differences in estimated loadings, especially if the
solution lies in a relatively flat place on the maximum likelihood surface.

Rotating the factor loadings

There are four methods to orthogonally rotate the initial factor loadings found by either
principal components or maximum likelihood extraction. An orthogonal rotation
simply rotates the axes to give you a different perspective. The methods are equimax,
varimax, quartimax, and orthomax. MINITAB rotates the loadings in order to minimize
a simplicity criterion [4]. A parameter, γ, within this criterion is determined by the
rotation method. If you use a method with a low value of γ, the rotation will tend to
simplify the rows of the loadings; if you use a method with a high value of γ, the
rotation will tend to simplify the columns of the loadings. The table below summarizes
the rotation methods.

Rotation
method Goal is . . . γ
equimax to rotate the loadings so that a variable number of factors / 2
loads high on one factor but low on others

varimax to maximize the variance of the squared 1

loadings

quartimax simple loadings 0

orthomax user determined, based on the given value of γ 0-1

4-10 MINITAB User’s Guide 2

Factor Analysis Multivariate Analysis

Using a matrix as input data

You can calculate the factor loadings from a correlation or covariance matrix. If it
makes sense to standardize variables (usual choice when variables are measured by
different scales), enter a correlation matrix; if you do not wish to standardize, enter a
covariance matrix.
You can use both raw data and a matrix of correlations or covariances as input data. If
you do, MINITAB uses the matrix to calculate the factor loadings. MINITAB then uses
these loadings and the raw data to calculate storage values and generate graphs.

h To perform factor analysis with a correlation or covariance matrix

1 Choose Stat ➤ Multivariate ➤ Factor Analysis.

2 Optionally, enter the columns containing raw data in Variables.

3 Choose Stat ➤ Multivariate ➤ Factor Analysis ➤ Options.

4 Choose Use matrix and enter the matrix.

5 Next, you need to specify what type of matrix you entered in step 4. Under Matrix to
factor, choose Correlation or Covariance.
6 Click OK.

Using stored loadings as input data

If you store initial factor loadings from an earlier analysis, you can input these initial
loadings to examine the effect of different rotations. You can also use stored loadings to
predict factor scores of new data.

h To perform factor analysis with stored loadings

1 Choose Stat ➤ Multivariate ➤ Factor Analysis and choose Options.

2 Under Loadings for Initial Solutions, choose Use loadings. Enter the columns
containing the loadings.
3 If you wish to:
■ examine the effect of a different rotation method, click OK to go back to the
main dialog and choose an option under Type of Rotation. See Rotating the
factor loadings on page 4-10 for a discussion of the various rotations.
■ predict factor scores with new data, click OK to go back to the main dialog and
enter the columns containing the new data in Variables.
4 Click OK.

MINITAB User’s Guide 2 4-11

Chapter 4 Factor Analysis

Factor analysis storage

To store loadings, factor score coefficients, or factor scores, enter a column name or
column number for each factor that has been extracted. The number of storage
columns specified must be equal in number to the number of factors calculated. If a
rotation was specified, MINITAB stores the values for the rotated solution. MINITAB
calculates factor scores by multiplying factor score coefficients and your data they have
been centered by subtracting means.
You can also store the rotation matrix and residual matrix. Enter a matrix name or
matrix number. The rotation matrix is the matrix used to rotate the initial loadings. If L
is the matrix of initial loadings and M is the rotation matrix that you store, LM is the
matrix of rotated loadings. The residual matrix is (A-LL′), where A is the correlation or
covariance matrix and L is a matrix of loadings. The residual matrix is the same for
initial and rotated solutions.
You can also store the eigenvalues and eigenvectors of the correlation or covariance
matrix (depending on which is factored) if you chose the initial factor extraction via
principal components. Enter a single column name or number for storing eigenvalues,
which are stored from largest to smallest. Enter a matrix name or number to store the
eigenvectors in an order corresponding to the sorted eigenvalues.

e Example of factor analysis using the principal components method

You record the following characteristics of 14 census tracts (see also Example of
principal components analysis on page 4-5): total population (Pop), median years of
schooling (School), total employment (Employ), employment in health services
(Health), and median home value (Home) (data from [5], Table 8.2). You would like to
investigate what “factors” might explain most of the variability. As the first step in your
factor analysis, you use the principal components extraction method and examine an
eigenvalues plot in order to help you to decide upon the number of factors.
1 Open the file EXH_MVAR.MTW.

2 Choose Stat ➤ Multivariate ➤ Factor Analysis.

3 In Variables, enter Pop-Home.

4 Click Graphs. Check Eigenvalue (Scree) plot. Click OK twice.

4-12 MINITAB User’s Guide 2

Factor Analysis Multivariate Analysis

Session Factor Analysis

window
output Principal Component Factor Analysis of the Correlation Matrix

Unrotated Factor Loadings and Communalities

Variable Factor1 Factor2 Factor3 Factor4 Factor5 Communality

Pop -0.972 -0.149 0.006 0.170 -0.067 1.000

School -0.545 -0.715 -0.415 -0.140 0.001 1.000
Employ -0.989 -0.005 0.089 0.083 0.085 1.000
Health -0.847 0.352 0.344 -0.200 -0.022 1.000
Home 0.303 -0.797 0.523 0.005 0.002 1.000

Variance 3.0289 1.2911 0.5725 0.0954 0.0121 5.0000

% Var 0.606 0.258 0.114 0.019 0.002 1.000

Factor Score Coefficients

Variable Factor1 Factor2 Factor3 Factor4 Factor5

Pop -0.321 -0.116 0.011 1.782 -5.511
School -0.180 -0.553 -0.726 -1.466 0.060
Employ -0.327 -0.004 0.155 0.868 6.988
Health -0.280 0.272 0.601 -2.098 -1.829
Home 0.100 -0.617 0.914 0.049 0.129

Graph
window
output

Interpreting the results

Five factors describe these data perfectly, but the goal is to reduce the number of
factors needed to capture the variability in the data. Examine the Session window
output line of % Var or the eigenvalues plot. The proportion of variability explained by
the last two factors is minimal and they can be eliminated as being important. The first

MINITAB User’s Guide 2 4-13

Chapter 4 Factor Analysis

two factors together represent 86% of the variability while three factors explain 98% of
the variability. The question is whether to use two or three factors. The next step might
be to perform separate factor analyses with two and three factors and examine the
communalities to see how individual variables are represented. If there were one or
more variables not well represented by the more parsimonious model, you might select
the model with more factors. See the next example for a rotation of loadings extracted
by the maximum likelihood method with a selection of two factors.

e Example of factor analysis using maximum likelihood and a rotation

You decide to examine the factor analysis fit with two factors in the above census tract
example. Here, you perform a maximum likelihood extraction with varimax rotation.
1 Open the file EXH_MVAR.MTW.
2 Choose Stat ➤ Multivariate ➤ Factor Analysis.

3 In Variables, enter Pop-Home.

4 Number of factors to extract, enter 2.

5 Under Method of Extraction, choose Maximum likelihood.

6 Under Type of Rotation, choose Varimax.

7 Click Graphs. Check Loading plot for first 2 factors. If it is already checked,
uncheck Eigenvalue (Scree) plot. Click OK.
8 Click Results. Check Sort loadings. Click OK twice.

Session Factor Analysis

window
output Maximum Likelihood Factor Analysis of the Correlation Matrix

* NOTE * Heywood case

Unrotated Factor Loadings and Communalities

Variable Factor1 Factor2 Communality

Pop 0.971 0.160 0.968
School 0.494 0.833 0.938
Employ 1.000 -0.000 1.000
Health 0.848 -0.395 0.875
Home -0.249 0.375 0.202
Variance 2.9678 1.0159 3.9837
% Var 0.594 0.203 0.797

4-14 MINITAB User’s Guide 2

Factor Analysis Multivariate Analysis

Rotated Factor Loadings and Communalities

Varimax Rotation

Variable Factor1 Factor2 Communality

Pop 0.718 0.673 0.968
School -0.052 0.967 0.938
Employ 0.831 0.556 1.000
Health 0.924 0.143 0.875
Home -0.415 0.173 0.202
Variance 2.2354 1.7483 3.9837
% Var 0.447 0.350 0.797

Sorted Rotated Factor Loadings and Communalities

Variable Factor1 Factor2 Communality

Health 0.924 0.143 0.875
Employ 0.831 0.556 1.000
Pop 0.718 0.673 0.968
Home -0.415 0.173 0.202
School -0.052 0.967 0.938
Variance 2.2354 1.7483 3.9837
% Var 0.447 0.350 0.797

Factor Score Coefficients

Variable Factor1 Factor2
Pop -0.165 0.246
School -0.528 0.789
Employ 1.150 0.080
Health 0.116 -0.173
Home -0.018 0.027
Graph
window
output

MINITAB User’s Guide 2 4-15

Chapter 4 Discriminant Analysis

Interpreting the results

The output indicates that this is a Heywood case. For a description of this condition,
see The maximum likelihood method on page 4-10. There are three tables of loadings
and communalities: unrotated, rotated, and sorted and rotated. The unrotated factors
explain 79.7% of the data variability (see last line under Communality) and the
communality values indicate that all variables but Home are well represented by these
two factors (communalities are 0.202 for Home, 0.875-1.0 for other variables). The
percent of total variability represented by the factors does not change with rotation, but
after rotating, these factors are more evenly balanced in the percent of variability that
they represent, being 44.7% and 35.0%, respectfully.
Let’s examine the sorted and rotated results. Sorting is done by the maximum absolute
loading for any factor. Variables that have their highest absolute loading on factor 1 are
printed first, in sorted order. Variables with their highest absolute loadings on factor 2
are printed next, in sorted order, and so on. Factor 1 has large positive loadings on
Health (.924), Employ (.831), and Pop (.718), and a -.415 loading on Home while the
loading on School is small. Factor 2 has a large positive loading on School of .967 and
loadings of .556 and .673, respectively, on Employ and Pop, and small loadings on
Health and Home.
You can view the loadings graphically in the loadings plot. What stands out for factor 1
are the high loadings on the variables Pop, Employ, and Health and the negative
loading on Home. School has a high positive loading for factor 2 and somewhat lower
values for Pop and Employ.
Let’s give a possible interpretation to the factors. The first factor positively loads on
population size and on two variables, Employ and Health, that generally increase with
population size. It negatively loads on home value, but this may be largely influenced
by one point. We might consider factor 1 to be a “health care - population size” factor.
The second factor might be considered to be a “education - population size” factor.
Both Health and School are correlated with Pop and Employ, but not much with each
other.
In addition, MINITAB displays a table of factor score coefficients. These show you how
the factors are calculated. MINITAB calculates factor scores by multiplying factor score
coefficients and your data after they have been centered by subtracting means.
You might repeat this factor analysis with three factors to see if it made better sense for
your data.

Discriminant Analysis
Use discriminant analysis to classify observations into two or more groups if you have a
sample with known groups. Discriminant analysis can also used to investigate how
variables contribute to group separation.

4-16 MINITAB User’s Guide 2

Discriminant Analysis Multivariate Analysis

MINITAB offers both linear and quadratic discriminant analysis. With linear
discriminant analysis, all groups are assumed to have the same covariance matrix.
Quadratic discrimination does not make this assumption but its properties are not as
well understood.
In the case of classifying new observations into one of two categories, logistic regression
may be superior to discriminant analysis [3], [9].

Data
Set up your worksheet so that a row of data contains information about a single item or
subject. You must have one or more numeric columns containing measurement data,
or predictors, and a single grouping column containing up to 20 groups. The column
of group codes may be numeric, text, or date/time. If you wish to change the order in
which text groups are processed from their default alphabetized order, you can define
your own order. See Ordering Text Categories in the Manipulating Data chapter in
MINITAB User’s Guide 1. MINITAB automatically omits observations with missing
measurements or group codes from the calculations.
If a high degree of multicollinearity exists (i.e., if one or more predictors is highly
correlated with another) or one or more of the predictors is essential constant,
discriminant analysis calculations cannot be done and MINITAB displays a message to
that effect.

h To perform linear discriminant analysis

1 Choose Stat ➤ Multivariate ➤ Discriminant Analysis.

2 In Groups, enter the column containing the group codes.

3 In Predictors, enter the column(s) containing the measurement data.

4 If you like, use one or more of the options listed below, then click OK.

MINITAB User’s Guide 2 4-17

Chapter 4 Discriminant Analysis

Options

Discriminant Analysis dialog box

■ perform quadratic discrimination. The default is linear. See Quadratic discriminant
analysis on page 4-19.
■ perform cross-validation. See Cross-Validation on page 4-20. You can store the fitted
values from cross-validation.
■ store the coefficients from the linear discriminant function.
■ store the fitted values. The fitted value for an observation is the group into which it is
classified.

Options subdialog box

■ specify prior probabilities. See Prior probabilities on page 4-19.
■ predict group membership for new observations. See Predicting group membership
for new observations on page 4-19.
■ control the amount of Session window output. You can display
– no output.
– the classification matrix.
– the default output, which includes the classification matrix, distance between
all pairs of group centers (i.e., group means), the linear discriminant function,
and a summary of misclassified observations.
– the default output, plus the means, standard deviations, and covariance
matrices (for each group and pooled).
– the output described above, plus display a summary of how all observations
were classified. MINITAB marks misclassified observations with two asterisks.

Linear discriminant analysis

An observation is classified into a group if the squared distance (also called the
Mahalanobis distance) of observation to the group center (mean) is the minimum. An
assumption is made that covariance matrices are equal for all groups. There is a unique
part of the squared distance formula for each group and that is called the linear
discriminant function for that group. For any observation, the group with the smallest
squared distance has the largest linear discriminant function and the observation is
then classified into this group.
Linear discriminant analysis has the property of symmetric squared distance: the linear
discriminant function of group i evaluated with the mean of group j is equal to the
linear discriminant function of group j evaluated with the mean of group i.

4-18 MINITAB User’s Guide 2

Discriminant Analysis Multivariate Analysis

We have described the simplest case, no priors and equal covariance matrices. If you
consider Mahalanobis distance a reasonable way to measure the distance of an
observation to a group, then you do not need to make any assumptions about the
underlying distribution of your data. See Help for more information.

Quadratic discriminant analysis

There is no assumption with quadratic discriminant analysis that the groups have equal
covariance matrices. As with linear discriminant analysis, an observation is classified
into the group that has the smallest squared distance. However, the squared distance
does not simplify into a linear function, hence the name quadratic discriminant
analysis.
Unlike linear distance, quadratic distance is not symmetric. In other words, the
quadratic discriminant function of group i evaluated with the mean of group j is not
equal to the quadratic discriminant function of group j evaluated with the mean of
group i. On the output, quadratic distance is called the generalized squared distance. If
the determinant of the sample group covariance matrix is less than one, the
generalized squared distance can be negative.

Prior probabilities
Sometimes items or subjects from different groups are encountered according to
different probabilities. If you know or can estimate these probabilities a priori,
discriminant analysis can utilize these so-called prior probabilities in calculating the
posterior probabilities, or probabilities of assigning observations to groups given the
data. With the assumption that the data have a normal distribution, the linear
discriminant function is increased by ln(pi), where pi is the prior probability of group i.
Because observations are assigned to groups according to the smallest generalized
distance, or equivalently the largest linear discriminant function, the effect is to
increase somewhat the posterior probabilities for a group with a high prior probability.

Predicting group membership for new observations

The usual case for discriminant analysis is to calculate the discriminant functions from
observations with known groups. When new observations are made, you can use the
discriminant function to predict which group that they belong to. You can do this by
either calculating (using Calc ➤ Calculator) the values of the discriminant function
for the observation(s) and then assigning it to the group with the highest function value
or by using MINITAB’s discriminant procedure:

MINITAB User’s Guide 2 4-19

Chapter 4 Discriminant Analysis

h To predict group membership for new observations

1 Choose Stat ➤ Multivariate ➤ Factor Analysis and choose Options.

2 In Groups, enter the column containing the group codes from the original sample.

3 In Predictors, enter the column(s) containing the measurement data of the original
sample.
4 Click Options. In Predict group membership for, enter constants or columns
representing one or more observations. The number of constants or columns must
be equivalent to the number of predictors.

Cross-Validation
Cross-validation is one technique that is used to compensate for an optimistic apparent
error rate. The apparent error rate is the percent of misclassified observations. This
number tends to be optimistic because the data being classified are the same data used
to build the classification function.
The cross-validation routine works by omitting each observation one at a time,
recalculating the classification function using the remaining data, and then classifying
the omitted observation. The computation time takes approximately four times longer
with this procedure. When cross-validation is performed, MINITAB prints an additional
summary table.
Another technique that you can use to calculate a more realistic error rate is to split
your data into two parts. Use one part to create the discriminant function, and the other
part as a validation set. Predict group membership for the validation set and calculate
the error rate as the percent of these data that are misclassified.

e Example of discriminant analysis

In order to regulate catches of salmon stocks, it is desirable to identify fish as being of
Alaskan or Canadian origin. Fifty fish from each place of origin were caught and
growth ring diameters of scales were measured for the time when they lived in
freshwater and for the subsequent time when they lived in saltwater. The goal is to be
able to identify newly-caught fish as being from Alaskan or Canadian stocks. The
example and data are from [5], pages 519-520.
1 Open the file EXH_MVAR.MTW.

2 Choose Stat ➤ Multivariate ➤ Discriminant Analysis.

3 In Groups, enter SalmonOrigin. In Predictors, enter Freshwater Marine. Click OK.

4-20 MINITAB User’s Guide 2

Discriminant Analysis Multivariate Analysis

Session Discriminant Analysis

window
output Linear Method for Response: SalmOrig

Predictors: Freshw Marine

Group Alaska Canada

Count 50 50

Summary of Classification

Put into ....True Group....

Group Alaska Canada
Alaska 44 1
Canada 6 49
Total N 50 50
N Correct 44 49
Proportion 0.880 0.980

N = 100 N Correct = 93 Proportion Correct = 0.930

Squared Distance Between Groups

Alaska Canada
Alaska 0.00000 8.29187
Canada 8.29187 0.00000

Linear Discriminant Function for Group

Alaska Canada
Constant -100.68 -95.14
Freshw 0.37 0.50
Marine 0.38 0.33

Summary of Misclassified Observations

Observation True Pred Group Squared Probability

Group Group Distance
1 ** Alaska Canada Alaska 3.544 0.428
Canada 2.960 0.572
2 ** Alaska Canada Alaska 8.1131 0.019
Canada 0.2729 0.981
12 ** Alaska Canada Alaska 4.7470 0.118
Canada 0.7270 0.882
13 ** Alaska Canada Alaska 4.7470 0.118
Canada 0.7270 0.882
30 ** Alaska Canada Alaska 3.230 0.289
Canada 1.429 0.711
32 ** Alaska Canada Alaska 2.271 0.464
Canada 1.985 0.536
71 ** Canada Alaska Alaska 2.045 0.948
Canada 7.849 0.052

MINITAB User’s Guide 2 4-21

Chapter 4 Clustering of Observations

Interpreting the results

The discriminant analysis output gives a classification summary, the squared distance
between group centroids, the linear discriminant functions, and a summary of
misclassified observations. The Summary of Misclassified Observations table shows the
squared distances from each misclassified point to group centroids and the posterior
probabilities.
Discriminant analysis correctly identified 93 of 100 fish in this example, though the
probability of misclassifying an Alaskan fish was higher (6/50) than was the probability
of misclassifying a Canadian fish (1/50). To identify newly-caught fish, you could
compute the linear discriminant functions associated with Alaskan and Canadian fish
and identify the new fish as being of a particular origin depending upon which
discriminant function value is higher. You can either do this by using Calc ➤
Calculator, or by performing discriminant analysis again and predicting group
membership for new observations.
The last table of output gives a summary of misclassified observations. The squared
distance value is the squared distance from the observation to the group centroid, or
mean vector. The probability value is the posterior probability, or the probability of a
group given the data. Observations are assigned to the group with the highest posterior
probability.

Clustering of Observations
Use clustering of observations to classify observations into groups when the groups are
initially not known.
This procedure uses an agglomerative hierarchical method that begins with all
observations being separate, each forming its own cluster. In the first step, the two
observations closest together are joined. In the next step, either a third observation joins
the first two, or two other observations join together into a different cluster. This process
will continue until all clusters are joined into one, but you must decide how many
groups are logical for your data. See Determining the final cluster grouping on page
4-26.

Data
You can have two types of input data: columns of raw data or a matrix of distances.
Typically, you would use raw data. Each row contains measurements on a single item
or subject. You must have multiple numeric columns, with each column representing a
different measurement. You must delete rows with missing data from the worksheet
before using this procedure.

4-22 MINITAB User’s Guide 2

Clustering of Observations Multivariate Analysis

If you store an n × n distance matrix, where n is the number of observations, you can
use this matrix as input data. The (i, j) entry in this matrix is the distance between
observations i and j. If you use the distance matrix as input, statistics on the final
partition are not available.

h To perform clustering of observations

1 Choose Stat ➤ Multivariate ➤ Cluster Observations.

2 In Variables or distance matrix, enter either columns containing the raw

(measurement) data or a matrix of distances.
3 If you like, use one or more of the options listed below, then click OK.

Options

Cluster Observations dialog box

■ specify the method to measure distance between observations if you enter raw data.
Available methods are Euclidean (default), Squared Euclidean, Pearson, Squared
Pearson, or Manhattan. See Distance measures for observations on page 4-24.
■ choose the linkage method—single (default), average, complete, McQuitty,
centroid, median, or Ward’s—that will determine how the distance between two
clusters is defined. See Linkage methods on page 4-25.
■ standardize all variables by subtracting the means and dividing by the standard
deviation before the distance matrix is calculated—a good idea if variables are in
different units and you wish to minimize the effect of scale differences. If you
standardize, cluster centroids and distance measures are in standardized variable
space.
■ determine the final partition by the specified number of clusters (default is 1) or by
the similarity level. See Determining the final cluster grouping on page 4-26.
■ display the dendrogram or tree diagram, showing the amalgamation steps.

MINITAB User’s Guide 2 4-23

Chapter 4 Clustering of Observations

Customize subdialog box

■ customize the default display of the dendrogram:
– add a title
– display similarities (the default) or distances on the y-axis
– show the dendrogram in one window (default) or in separate windows for each
cluster
– specify the line type, line color, and line size used to represent each cluster. See
Specifying dendrogram attributes on page 4-26.

Storage subdialog box

■ store cluster membership
■ store distances between observations and cluster centroids for each cluster group
■ store the n × n distance matrix, where n is the number of observations

Distance measures for observations

If you do not supply a distance matrix, MINITAB’s first step is to calculate an n × n
distance matrix, D, where n is the number of observations. The matrix entries, d(i, j), in
row i and column j, is the distance between observations i and j.
There is more than one way to measure distance, but there are no hard rules about
which distance measure to use. You might choose one according to properties of your
data.
■ The Euclidean method is a standard mathematical measure of distance (square root
of the sum of squared differences).
■ The Pearson method is a square root of the sum of square distances divided by
variances. This method is for standardizing.
■ Manhattan distance is the sum of absolute distances, so that outliers receive less
weight than they would if the Euclidean method were used.
■ The squared Euclidean and squared Pearson methods use the square of the
Euclidean and Pearson methods, respectfully, which means that distances that are
large under the Euclidean and Pearson methods will be even larger under the
squared Euclidean and squared Pearson methods.

Tip If you choose average, centroid, median, or Ward as the linkage method, it is generally
recommended [7] that you use one of the squared distance measures.

4-24 MINITAB User’s Guide 2

Clustering of Observations Multivariate Analysis

Linkage methods
The linkage method that you choose determines how the distance between two clusters
is defined. At each amalgamation stage, the two closest clusters are joined. At the
beginning, when each observation constitutes a cluster, the distance between clusters is
simply the inter-observation distance. Subsequently, after observations are joined
together, a linkage rule is necessary for calculating inter-cluster distances when there
are multiple observations in a cluster.
You may wish to try several linkage methods and compare results. Depending on the
characteristics of your data, some methods might be better choices than others.
■ With single linkage, or “nearest neighbor,” the distance between two clusters is the
minimum distance between an observation in one cluster and an observation in the
other cluster. Single linkage is a good choice when clusters are clearly separated.
When observations lie close together, single linkage tends to identify long chain-like
clusters that can have a relatively large distance separating observations at either end
of the chain [5].
■ With complete linkage, or “furthest neighbor,” the distance between two clusters is
the maximum distance between an observation in one cluster and an observation in
the other cluster. This method ensures that all observations in a cluster are within a
maximum distance and tends to produce clusters with similar diameters. The results
can be sensitive to outliers [8].
■ With average linkage, the distance between two clusters is the mean distance
between an observation in one cluster and an observation in the other cluster.
Whereas the single or complete linkage methods group clusters based upon single
pair distances, average linkage uses a more central measure of location.
■ With centroid linkage, the distance between two clusters is the distance between the
cluster centroids or means. Like average linkage, this method is another averaging
technique.
■ With median linkage, the distance between two clusters is the median distance
between an observation in one cluster and an observation in the other cluster. This
is another averaging technique, but uses the median rather than the mean, thus
downweighting the influence of outliers.
■ With McQuitty’s linkage, when two clusters are be joined, the distance of the new
cluster to any other cluster is calculated as the average of the distances of the soon to
be joined clusters to that other cluster. For example, if clusters 1 and 3 are to be
joined into a new cluster, say 1∗, then the distance from 1∗ to cluster 4 is the
average of the distances from 1 to 4 and 3 to 4. Here, distance has a combinatorial
meaning rather than being interpretable from individual observations in the
clusters.
■ With Ward’s linkage, the distance between two clusters is the sum of squared
deviations from points to centroids. The objective of Ward’s linkage is to minimize
the within-cluster sum of squares. It tends to produce clusters with similar numbers

MINITAB User’s Guide 2 4-25

Chapter 4 Clustering of Observations

of observations, but it is sensitive to outliers [8]. In Ward’s linkage, it is possible for

the distance between two clusters to be larger than dmax, the maximum value in the
original distance matrix. If this happens, the similarity will be negative.

Determining the final cluster grouping

The final grouping of clusters, also called the final partition, is the grouping of clusters
which will, hopefully, identify groups whose members share common characteristics.
The decision about final grouping is also called cutting the dendrogram. The complete
dendrogram is a graphical depiction of the amalgamation of observations into one
eventual cluster. Cutting the dendrogram is akin to drawing a line across the
dendrogram to specify the final grouping.
How do you know where to cut the dendrogram? You might first execute cluster
analysis without specifying a final partition. Examine the similarity and distance levels
in the Session window output and in the dendrogram. The similarity level at any step is
the percent of the minimum distance at that step relative to the maximum
inter-observation distance in the data. The pattern of how similarity or distance values
change from step to step can help you to choose the final grouping. The step where the
values change abruptly may identify a good point for cutting the dendrogram, if this
makes sense for your data.
After choosing where you wish to make your partition, rerun the clustering procedure,
using either Number of Clusters or Similarity Level to give you either a set number of
groups or a similarity level for cutting the dendrogram. Examine the resulting clusters
in the final partition to see if the grouping seems logical. Looking at dendrograms for
different final groupings can also help you to decide which one makes the most sense
for your data.

Note For some data sets, average, centroid, median and Ward's methods may not produce a
hierarchical dendrogram. That is, the amalgamation distances do not always increase
with each step. In the dendrogram, such a step will produce a join that goes downward
rather than upward.

Specifying dendrogram attributes

You can specify the line type, line color, and line size used to draw the portion of the
dendrogram corresponding to each cluster in the final partition. If there are k clusters,
you can give up to k values for each of these attributes. If you give less than k values, the
ones that you enter will cycle until one is assigned to each cluster. For line type and
line color, enter numbers that correspond to the types and colors below.

4-26 MINITAB User’s Guide 2

Clustering of Observations Multivariate Analysis

Line types Line colors

0 null (invisible) 0 white 8 dark red
(default) 1 solid (default) 1 black 9 dark green
2 dashes 2 red 10 dark blue
3 dots 3 green 11 dark cyan
4 dash 1-dot 4 blue 12 dark magenta
5 dash 2-dots 5 cyan 13 dark yellow
6 dash 3-dots 6 magenta 14 dark gray
7 long dashes 7 yellow 15 light gray

You can specify any positive real number for the line sizes. Larger values yield wider
lines. The default size is 1.

e Example of cluster observations

You make measurements on five nutritional characteristics (protein, carbohydrate, and
fat content, calories, and percent of the daily allowance of Vitamin A) of 12 breakfast
cereal brands. The example and data are from p. 623 of [5]. The goal is to group cereal
brands with similar characteristics. You use clustering of observations with the
complete linkage method, squared Euclidean distance, and you choose
standardization because the variables have different units. You also request a
dendrogram and assign different line types and colors to each cluster.
1 Open the file [Link].

2 Choose Stat ➤ Multivariate ➤ Cluster Observations.

3 In Variables or distance matrix, enter Protein-VitaminA.

4 For Linkage Method, choose Complete. For Distance Measure choose Squared
Euclidean.
5 Check Standardize variables.

6 Under Specify Final Partition by, choose Number of clusters and enter 4 in the
box.
7 Check Show dendrogram.

8 Click Customize. In Title, enter Dendrogram for Cereal Data. In Type, enter 1 2 1.
In Color, enter 2 3 4. Click OK twice.

MINITAB User’s Guide 2 4-27

Chapter 4 Clustering of Observations

Session Hierarchical Cluster Analysis of Observations

window
output Standardized Variables, Squared Euclidean Distance, Complete Linkage

Amalgamation Steps

Step Number of Similarity Distance Clusters New Number of Obs

clusters level level joined cluster in new cluster
1 11 100.00 0.000 5 12 5 2
2 10 99.82 0.064 3 5 3 3
3 9 98.79 0.435 3 11 3 4
4 8 94.68 1.913 6 8 6 2
5 7 93.41 2.373 2 3 2 5
6 6 87.33 4.560 7 9 7 2
7 5 86.19 4.970 1 4 1 2
8 4 80.60 6.981 2 6 2 7
9 3 68.08 11.487 2 7 2 9
10 2 41.41 21.085 1 2 1 11
11 1 0.00 35.987 1 10 1 12

Final Partition

Number of clusters: 4

Number of Within cluster Average distance Maximum distance

observations sum of squares from centroid from centroid
Cluster1 2 2.485 1.115 1.115
Cluster2 7 8.999 1.043 1.769
Cluster3 2 2.280 1.068 1.068
Cluster4 1 0.000 0.000 0.000

Cluster Centroids

Variable Cluster1 Cluster2 Cluster3 Cluster4 Grand centrd

Protein 1.9283 -0.3335 -0.2030 -1.1164 0.0000
Carbo -0.7587 0.5419 0.1264 -2.5289 0.0000
Fat 0.3385 -0.0967 0.3385 -0.6770 -0.0000
Calories 0.2803 0.2803 0.2803 -3.0834 -0.0000
VitaminA -0.6397 -0.2559 2.0471 -1.0235 0.0000

Distances Between Cluster Centroids

Cluster1 Cluster2 Cluster3 Cluster4

Cluster1 0.0000 2.6727 3.5418 4.9896
Cluster2 2.6727 0.0000 2.3838 4.7205
Cluster3 3.5418 2.3838 0.0000 5.4460
Cluster4 4.9896 4.7205 5.4460 0.0000

4-28 MINITAB User’s Guide 2

Clustering of Observations Multivariate Analysis

Graph
window
output

Interpreting the results

MINITAB displays the amalgamation steps in the Session window. At each step, two
clusters are joined. The table shows which clusters were joined, the distance between
them, the corresponding similarity level, the identification number of the new cluster
(this number is always the smaller of the two numbers of the clusters joined), the
number of observations in the new cluster, and the number of clusters. Amalgamation
continues until there is just one cluster.
The amalgamation steps show that four clusters appear to be good candidates for the
final partition. Notice that the similarity level decreases by increments of about 6 or less
until it decreases by about 13 at the step from four clusters to three. If this grouping
makes intuitive sense for the data, then it may be a good choice.
When you specify the final partition, MINITAB displays three additional tables. The first
table summarizes each cluster by the number of observations, the within cluster sum of
squares, the average distance from observation to the cluster centroid, and the
maximum distance of observation to the cluster centroid. In general, a cluster with a
small sum of squares is more compact than one with a large sum of squares. The
centroid is the vector of variable means for the observations in that cluster and is used
as a cluster midpoint. The second table displays the centroids for the individual clusters
while the third table gives distances between cluster centroids.
The dendrogram displays the information printed in the amalgamation table in the
form of a tree diagram. Because this book is in black and white, you cannot see the
assigned cluster colors. Using color can make it easier to discriminate between the
clusters. In our example, cereals 1 and 4 make up the first cluster; cereals 2, 3, 5, 12,
11, 6, and 8 make up the second; cereals 7 and 9 make up the third; cereal 10 makes up
the fourth.

MINITAB User’s Guide 2 4-29

Chapter 4 Clustering of Variables

Clustering of Variables
Use clustering of variables to classify variables into groups when the groups are initially
not known. One reason to cluster variables may be to reduce their number. This
technique may give new variables that are more intuitively understood than those
found using principal components.
This procedure is an agglomerative hierarchical method that begins with all variables
separate, each forming its own cluster. In the first step, the two variables closest
together are joined. In the next step, either a third variable joins the first two, or two
other variables join together into a different cluster. This process will continue until all
clusters are joined into one, but you must decide how many groups are logical for your
data. See Determining the final cluster grouping on page 4-26.

Data
You can have two types of input data to cluster observations: columns of raw data or a
matrix of distances.
Typically, you would use raw data. Each row contains measurements on a single item
or subject. You must have multiple numeric columns, with each column representing a
different measurement. You must delete rows with missing data from the worksheet
before using this procedure.
If you store a p × p distance matrix, where p is the number of variables, you can use this
matrix as input data. The (i, j) entry in this matrix is the distance between observations
i and j. If you use the distance matrix as input, statistics on the final partition are not
available.

h To perform clustering of variables

1 Choose Stat ➤ Multivariate ➤ Cluster Variables.

4-30 MINITAB User’s Guide 2

Clustering of Variables Multivariate Analysis

2 In Variables or distance matrix, enter either columns containing the raw

(measurement) data or a matrix of distances.
3 If you like, use one or more of the options listed below, then click OK.

Options

Cluster Variables dialog box

■ choose the linkage method—single (default), average, complete, McQuitty,
centroid, median, or Ward’s—that will determine how the distance between two
clusters is defined. See Linkage methods on page 4-25.
■ choose correlation or absolute correlation as a distance measure if you use raw data.
See Distance measures for variables on page 4-31.
■ determine the final partition by the specified number of clusters or the specified
level of similarity. See Determining the final cluster grouping on page 4-26.
■ display the dendrogram or tree diagram, showing the amalgamation steps.

Customize subdialog box

■ customize the default display of the dendrogram:
– add a title
– display similarities (the default) or distances on the y-axis
– show dendrogram in one window (default) or separate windows for each cluster
– specify the line type, line color, and line size used to represent each cluster in the
final partition. See Specifying dendrogram attributes on page 4-26.

Storage subdialog box

■ store the p × p distance matrix, where p is the number of variables.

Distance measures for variables

You can use correlations or absolute correlations for distance measures. With the
correlation method, the (i,j) entry of the distance matrix is dij = 1 − ρij and for the
absolute correlation method, dij = 1 − |ρij|, where ρij is the (Pearson product moment)
correlation between variables i and j. Thus, the correlation method will give distances
between 0 and 1 for positive correlations, and between 1 and 2 for negative correlations.
The absolute correlation method will always give distances between 0 and 1.
Use the correlation method if it makes sense for your data to consider negatively
correlated data to be farther apart than positively correlated data. Use the absolute
correlation method if you think that it is the strength of the relationship and not the
sign that is important in considering distance.

MINITAB User’s Guide 2 4-31

Chapter 4 Clustering of Variables

Clustering variables in practice

You must make similar decisions to cluster variables as you would to cluster
observations. Follow the guidelines in Determining the final cluster grouping on page
4-26 to help you determine groupings. However, if the purpose behind clustering of
variables is data reduction, you may decide to use your knowledge of the data to a
greater degree in determining the final clusters of variables. See the following example.

e Example of clustering variables

You conduct a study to determine the long-term effect of a change in environment on
blood pressure. The subjects are 39 Peruvian males over 21 years of age who had
migrated from the Andes mountains to larger towns at lower elevations. You recorded
their age (Age), years since migration (Years), weight in kg (Weight), height in mm
(Height), skin fold of the chin, forearm, and calf in mm (Chin, Forearm, Calf), pulse
rate in beats per minute (Pulse), and systolic and diastolic blood pressure (Systol,
Diastol).
Your goal is to reduce the number of variables by combining variables with similar
characteristics. You use clustering of variables with the default correlation distance
measure, average linkage and a dendrogram.
1 Open the file [Link].

2 Choose Stat ➤ Multivariate ➤ Cluster Variables.

3 In Variables or distance matrix, enter Age-Diastol.

4 For Linkage Method, choose Average.

5 Check Show dendrogram. Click OK.

Session Hierarchical Cluster Analysis of Variables

window
output Correlation Coefficient Distance, Average Linkage

Amalgamation Steps

Step Number of Similarity Distance Clusters New Number of Obs

clusters level level joined cluster in new cluster
1 9 86.78 0.264 6 7 6 2
2 8 79.41 0.412 1 2 1 2
3 7 78.85 0.423 5 6 5 3
4 6 76.07 0.479 3 9 3 2
5 5 71.74 0.565 3 10 3 3
6 4 65.55 0.689 3 5 3 6
7 3 61.34 0.773 3 8 3 7
8 2 56.60 0.868 1 3 1 9
9 1 55.44 0.891 1 4 1 10

4-32 MINITAB User’s Guide 2

K-Means Clustering of Observations Multivariate Analysis

Graph
window
output

Interpreting the results

MINITAB displays shows the amalgamation steps in the Session window. At each step,
two clusters are joined. The table shows which clusters were joined, the distance
between them, the corresponding similarity level, the identification number of the new
cluster (this is always the smaller of the two numbers of the clusters joined), the
number of variables in the new cluster and the number of clusters. Amalgamation
continues until there is just one cluster.
If you had requested a final partition you would also receive a list of which variables are
in each cluster.
The dendrogram displays the information printed in the amalgamation table in the
form of a tree diagram. This dendrogram suggests variables which might be combined,
for example, by averaging or totaling. The chin, forearm, and calf skin fold
measurements are similar and you decide to combine those. The age and year since
migration variables are similar, but you will investigate this relationship. If subjects
tend to migrate at a certain age, then these variables could contain similar information
and be combined. Weight and the two blood pressure measurements are similar. You
decide to keep weight as a separate variable but you will combine the blood pressure
measurements into one.

K-Means Clustering of Observations

Use K-means clustering of observations, like clustering of observations on page 4-22, to
classify observations into groups when the groups are initially unknown. This
procedure uses non-hierarchical clustering of observations according to MacQueen’s
algorithm [5]. K-means clustering works best when sufficient information is available to
make good starting cluster designations. See Initializing the K-means clustering process
on page 4-35.

MINITAB User’s Guide 2 4-33

Chapter 4 K-Means Clustering of Observations

Data
You must use raw data as input to K-means clustering of observations. Each row
contains measurements on a single item or subject. You must have multiple numeric
columns, with each column representing a different measurement. You must delete
rows with missing data from the worksheet before using this procedure.
To initialize the clustering process using a data column, you must have a column that
contains a cluster membership value for each observation. The initialization column
must contain positive, consecutive integers or zeros (it should not contain all zeros).
Initially, each observation is assigned to the cluster identified by the corresponding
value in this column. An initialization of zero means that an observation is initially
unassigned to a group. The number of distinct positive integers in the initial partition
column equals the number of clusters in the final partition.

h To perform K-means clustering of observations

1 Choose Stat ➤ Multivariate ➤ Cluster K-Means.

2 In Variables, enter the columns containing the measurement data.

3 If you like, use one or more of the options listed below, then click OK.

Options

Cluster K-Means dialog box

■ specify the number of clusters to form or specify a column containing cluster
membership to begin the partition process. See Initializing the K-means clustering
process on page 4-35.
■ standardize all variables by subtracting the means and dividing by the standard
deviation before the distance matrix is calculated, which is a good idea if variables
are in different units and you wish to minimize the effect of scale differences. If you

4-34 MINITAB User’s Guide 2

K-Means Clustering of Observations Multivariate Analysis

standardize, cluster centroids and distance measures are in standardized variable

space.

Storage subdialog box

■ store the final cluster membership for each observation
■ store the distance between each observation and each cluster centroid

Initializing the K-means clustering process

K-means clustering begins with a grouping of observations into a predefined number of
clusters.
1 MINITAB evaluates each observation, moving it into the cluster whose centroid it is
closest to, using Euclidean distance.
2 When a cluster changes, by losing or gaining an observation, MINITAB recalculates
the cluster centroid, the vector of variable means.
3 This process is repeated until no more observations can be moved into a different
cluster, that is, until all observations are in the cluster whose centroid they are
closest to.
Unlike hierarchical clustering of observations, it is possible for two observations to be
split into separate clusters after they are joined together.
K-means procedures work best when you provide good starting points for clusters [8].
There are two ways to initialize the clustering process: specifying a number of clusters
or supplying an initial partition column that contains group codes.

h How to initialize the process by specifying the number of clusters

1 Choose Stat ➤ Multivariate ➤ Cluster K-Means.

2 In Variables, enter the columns containing the measurement data.

3 Under Specify Partition by, choose Number of clusters and enter a number, k, in
the box. MINITAB will use the first k observations as initial cluster seeds, or starting
locations. Click OK.
For guidance in setting up your worksheet, see below.

h How to initialize the process using a data column

1 Choose Stat ➤ Multivariate ➤ Cluster K-Means.

2 In Variables, enter the columns containing the measurement data.

3 Under Specify Partition by, choose Initial partition column. Enter the column
containing the initial cluster membership for each observation. Click OK.

MINITAB User’s Guide 2 4-35

Chapter 4 K-Means Clustering of Observations

You may be able to initialize the process when you do not have complete information
to initially partition the data. Suppose you know that the final partition should consist
of three groups, and that observations 2, 5, and 9 belong in each of those groups,
respectively. Proceeding from here depends upon whether you specify the number of
clusters or supply an initial partition column.
■ If you specify the number of clusters, you must rearrange your data in the Data
window to move observations 2, 5 and 9 to the top of the worksheet, and then specify
3 for Number of clusters.
■ If you enter an initial partition column, you do not need to rearrange your data in
the Data window. In the initial partition worksheet column, enter group numbers 1,
2, and 3, for observations 2, 5, and 9, respectively, and enter 0 for the other
observations. See the following example.
The final partition will depend to some extent on the initial partition that MINITAB
uses. You might try different initial partitions.

e Example of K-means clustering

You live-trap, anesthetize, and measure one hundred forty-three black bears. The
measurements are total length and head length (Length, Head.L), total weight and
head weight (Weight, Weight.H), and neck girth and chest girth (Neck.G, Chest.G).
You wish to classify these 143 bears as small, medium-sized, or large bears. You know
that the second, seventy-eighth, and fifteenth bears in the sample are typical of the
three respective categories. First, you create an initial partition column with the three
seed bears designated as 1 = small, 2 = medium-sized, 3 = large, and with the
remaining bears as 0 (unknown) to indicate initial cluster membership. Then you
perform K-means clustering and store the cluster membership in a column named
BearSize.
1 Open the file [Link].

2 To create the initial partition column, choose Calc ➤ Make Patterned Data ➤
Simple Set of Numbers.
3 In Store patterned data in, enter Initial for the storage column name. In both From
first value and To last value, enter 0. In List each value, type 143. Click OK.
4 Go to the Data window and type 1, 2, and 3 in the second, seventy-eighth, and
fifteenth rows, respectively, of the column named Initial.
5 Choose Stat ➤ Multivariate ➤ Cluster K-Means.

6 In Variables, enter Head.L –Weight.

7 Under Specify Partition by, choose Initial partition column and enter Initial in
the box.
8 Check Standardize variables.

9 Click Storage. In Cluster membership column, enter BearSize. Click OK twice.

4-36 MINITAB User’s Guide 2

K-Means Clustering of Observations Multivariate Analysis

Session K-means Cluster Analysis

window
output Standardized Variables

Final Partition
Number of clusters: 3

Number of Within cluster Average distance Maximum distance

observations sum of squares from centroid from centroid
Cluster1 41 63.075 1.125 2.488
Cluster2 67 78.947 0.997 2.048
Cluster3 35 65.149 1.311 2.449

Cluster Centroids

Variable Cluster1 Cluster2 Cluster3 Grand centrd

Head.L -1.0673 0.0126 1.2261 -0.0000
Head.W -0.9943 -0.0155 1.1943 0.0000
Neck.G -1.0244 -0.1293 1.4476 -0.0000
Length -1.1399 0.0614 1.2177 0.0000
Chest.G -1.0570 -0.0810 1.3932 -0.0000
Weight -0.9460 -0.2033 1.4974 -0.0000

Distances Between Cluster Centroids

Cluster1 Cluster2 Cluster3

Cluster1 0.0000 2.4233 5.8045
Cluster2 2.4233 0.0000 3.4388
Cluster3 5.8045 3.4388 0.0000

Interpreting the results

K-means clustering classified the 143 bears as 41 small bears, 67 medium-size bears,
and 35 large bears. MINITAB displays, in the first table, the number of observations in
each cluster, the within cluster sum of squares, the average distance from observation
to the cluster centroid, and the maximum distance of observation to the cluster
centroid. In general, a cluster with a small sum of squares is more compact than one
with a large sum of squares. The centroid is the vector of variable means for the
observations in that cluster and is used as a cluster midpoint.
The centroids for the individual clusters are printed in the second table while the third
table gives distances between cluster centroids.
The column BearSize contains the cluster designations.

MINITAB User’s Guide 2 4-37

Chapter 4 Balanced MANOVA

Balanced MANOVA
Use balanced MANOVA to perform multivariate analysis of variance (MANOVA) for
balanced designs. You can take advantage of the data covariance structure to
simultaneously test the equality of means from different responses.
Your design must be balanced, with the exception of one-way designs. Balanced means
that all treatment combinations (cells) must have the same number of observations.
Use General MANOVA (page 4-44) to analyze either balanced and unbalanced
MANOVA designs or if you have covariates. You cannot designate factors to be random
with general MANOVA, unlike for balanced ANOVA, though you can work around
this restriction by supplying error terms to test the model terms.
Factors may be crossed or nested, fixed or random. See Crossed vs. nested factors on
page 3-18 and Fixed vs. random factors on page 3-18.

Data
You need one column for each response variable and one column for each factor, with
each row representing an observation. Regardless of whether factors are crossed or
nested, use the same form for the data. Factor columns may be numeric, text, or date/
time. If you wish to change the order in which text categories are processed from their
default alphabetical order, you can define your own order. See Ordering Text
Categories in the Manipulating Data chapter in MINITAB User’s Guide 1. You may
include up to 50 response variables and up to 9 factors at one time.
Balanced data are required except for one-way designs. The requirement for balanced
data extends to nested factors as well. Suppose A has 3 levels, and B is nested within A.
If B has 4 levels within the first level of A, B must have 4 levels within the second and
third levels of A. MINITAB will tell you if you have unbalanced nesting. In addition, the
subscripts used to indicate the 4 levels of B within each level of A must be the same.
Thus, the four levels of B cannot be (1 2 3 4) in level 1 of A, (5 6 7 8) in level 2 of A,
and (9 10 11 12) in level 3 of A. You can use general MANOVA if you have different
levels of B within the levels of A.
If any response or factor column specified contains missing data, that entire
observation (row) is excluded from all computations. The requirement that data be
balanced must be preserved after missing data are omitted. If an observation is missing
for one response variable, that row is eliminated for all responses.

4-38 MINITAB User’s Guide 2

Balanced MANOVA Multivariate Analysis

h To perform a balanced MANOVA

1 Choose Stat ➤ Multivariate ➤ Balanced MANOVA.

2 In Responses, enter up to 50 numeric columns containing the response variables.

3 In Model, type the model terms that you want to fit. See Overview of Balanced
ANOVA and GLM on page 3-17.
4 If you like, use one or more of the options described below, then click OK.

Options

Balanced Analysis of Variance dialog box

■ specify which factors are random factors—see Fixed vs. random factors on page 3-18

Options subdialog box

■ use the restricted form of the mixed models (both fixed and random effects). The
restricted model forces mixed interaction effects to sum to zero over the fixed
effects. By default, MINITAB fits the unrestricted model. See Restricted and
unrestricted form of mixed models on page 3-26.

Graphs subdialog box

■ draw five different residual plots. You can display the following plots:
– histogram
– normal probability plot
ˆ)
– plot of residuals versus the fitted values ( Y
– plot of residuals versus data order. The row number for each data point is shown
on the x-axis—for example, 1 2 3 4… n
– separate plot for the residuals versus each specified column
For a discussion of the residual plots, see Residual plots on page 2-6.

MINITAB User’s Guide 2 4-39

Chapter 4 Balanced MANOVA

Results subdialog box

■ display different MANOVA output. You can request the display of the hypothesis
matrix H, the error matrix E, and a matrix of partial correlations (see MANOVA tests
on page 4-40), the eigenvalues and eigenvalues for the matrix E-1 H, univariate
analysis of variance for each response, and when you have requested univariate
analyses of variance, the expected means squares.
■ display a table of means corresponding to specified terms from the model. For
example, if you specify A B D A∗B∗D, four table of means will be printed, one for
each main effect, A, B, D, and one for the three-way interaction, A∗B∗D.
■ perform 4 multivariate tests for model terms that you specify. See Specifying terms to
test on page 4-40. Default tests are performed for all model terms.

Storage subdialog box

■ store the fits and residuals separately for each response. If you fit a full model, fits are
cell means. If you fit a reduced model, fits are least squares estimates. See Specifying
reduced models on page 3-21.

Specifying terms to test

In the Results subdialog box, you can specify model terms in Custom multivariate test
for the following terms and designate an error term in Error and MINITAB will
perform four multivariate tests (see below) for those terms. This option is probably less
useful for balanced MANOVA than it is for general MANOVA; because you can
specify factors to be random with balanced MANOVA, MINITAB will use the correct
error terms. This option exists for special purpose tests.
If you specify an error term, it must be a single term that is in the model. This error
term is used for all requested tests. If you do not specify an error term, MINITAB
determines an appropriate error term.

MANOVA tests
MINITAB automatically performs four multivariate tests—Wilk’s test, Lawley-Hotelling
test, Pillai’s test, and Roy’s largest root test—for each term in the model and for
specially requested terms (see above). All four tests are based on two SSCP (sums of
squares and cross products) matrices: H, the hypothesis matrix and E, the error matrix.
There is one H associated with each term. E is the matrix associated with the error for
the test. These matrices are printed when you request the hypothesis matrices and are
labeled by SSCP Matrix.
The test statistics can be expressed in terms of either H and/or E or the eigenvalues of
E-1 H. You can request to have these eigenvalues printed. (If the eigenvalues are
repeated, corresponding eigenvectors are not unique and in this case, the eigenvectors

4-40 MINITAB User’s Guide 2

Balanced MANOVA Multivariate Analysis

MINITAB prints and those in books or other software may not agree. The MANOVA
tests, however, are always unique.) See Help for computational details on the tests.
You can also print the matrix of partial correlations, which are the correlations among
the residuals, or alternatively, the correlations among the responses conditioned on the
model. The formula for this matrix is W-.5 E W-.5, where E is the error matrix and W
has the diagonal of E as its diagonal and 0’s off the diagonal.

Hotelling’s T2 Test
Hotelling’s T2 test to compare the mean vectors of two groups is a special case of
MANOVA, using one factor that has two levels. MINITAB’s MANOVA option can be
used to do this test. The usual T2 test statistic can be calculated from MINITAB’s output
using the relationship T2=(N-2)U, where N is the total number of observations and U
is the Lawley-Hotelling trace. S, the pooled covariance matrix, is E/(N-2), where E is
the error matrix.

e Example of balanced MANOVA

You perform a study in order to determine optimum conditions for extruding plastic
film. You measure three responses—tear resistance, gloss, and opacity—five times at
each combination of two factors—rate of extrusion and amount of an additive—each
set at low and high levels. The data and example are from Johnson and Wichern [5],
page 266. You use Balanced MANOVA to test the equality of means because the design
is balanced.
1 Open the file EXH_MVAR.MTW.

2 Choose Stat ➤ Multivariate ➤ Balanced MANOVA.

3 In Responses, enter Tear Gloss Opacity.

4 In Model, enter Extrusion | Additive.

5 Click Results. Under Display of Results, check Matrices (hypothesis, error, partial
correlations) and Eigen analysis. Click OK twice.

Session Analysis of Variance (Balanced Designs)

window
output MANOVA for Extrusn s = 1 m = 0.5 n = 6.0

Criterion Test Statistic F DF P

Wilk's 0.38186 7.554 ( 3, 14) 0.003
Lawley-Hotelling 1.61877 7.554 ( 3, 14) 0.003
Pillai's 0.61814 7.554 ( 3, 14) 0.003
Roy's 1.61877

MINITAB User’s Guide 2 4-41

Chapter 4 Balanced MANOVA

SSCP Matrix for Extrusn

Tear Gloss Opacity

Tear 1.740 -1.504 0.8555
Gloss -1.504 1.300 -0.7395
Opacity 0.855 -0.739 0.4205

SSCP Matrix for Error

Tear Gloss Opacity

Tear 1.764 0.0200 -3.070
Gloss 0.020 2.6280 -0.552
Opacity -3.070 -0.5520 64.924

Partial Correlations for the Error SSCP Matrix

Tear Gloss Opacity

Tear 1.00000 0.00929 -0.28687
Gloss 0.00929 1.00000 -0.04226
Opacity -0.28687 -0.04226 1.00000

EIGEN Analysis for Extrusn

Eigenvalue 1.619 0.00000 0.00000

Proportion 1.000 0.00000 0.00000
Cumulative 1.000 1.00000 1.00000

Eigenvector 1 2 3
Tear 0.6541 0.4243 0.0992
Gloss -0.3385 0.5141 0.0478
Opacity 0.0359 0.0410 -0.1176

---multivariate output for Additive and Extrusion∗Additive would follow---

Interpreting the results

By default, MINITAB displays a table of the four multivariate tests (Wilk’s,
Lawley-Hotelling, Pillai’s, and Roy’s) for each term in the model. The values s, m, and
n are used in the calculations of the F-statistics for Wilk’s, Lawley-Hotelling, and
Pillai’s tests. The F-statistic is exact if s = 1 or 2, otherwise it is approximate [5]. Because
you requested the display of additional matrices (hypothesis, error, and partial
correlations) and an eigen analysis, this information is also displayed. The output is
shown only for one model term, Extrusion, and not for the terms Additive or
Extrusion∗Additive.
Examine the p-values for the Wilk’s, Lawley-Hotelling, and Pillai’s test statistic to judge
whether there is significant evidence for model effects. These values are 0.003 for the
model term Extrusion, indicating that there is significant evidence for Extrusion main
effects at α levels greater than 0.003. The corresponding p-values for Additive and for

4-42 MINITAB User’s Guide 2

Balanced MANOVA Multivariate Analysis

Additive∗Extrusion are 0.025 and 0.302, respectively (not shown), indicating that there
is no significant evidence for interaction, but there is significant evidence for Extrusion
and Additive main effects at α levels of 0.05 or 0.10.
You can use the SSCP matrices to assess the partitioning of variability in a similar way
as you would look at univariate sums of squares. The matrix labeled as SSCP Matrix for
Extrusn is the hypothesis sums of squares and cross-products matrix, or H, for the three
response with model term Extrusion. The diagonal elements of this matrix, 1.740,
1.300, and 0.4205, are the univariate ANOVA sums of squares for the model term
Extrusion when the response variables are Tear, Gloss, and Opacity, respectfully. The
off-diagonal elements of this matrix are the cross products.
The matrix labeled as SSCP Matrix for Error is the error sums of squares and
cross-products matrix, or E. The diagonal elements of this matrix, 1.764, 2.6280, and
64.924, are the univariate ANOVA error sums of squares when the response variables
are Tear, Gloss, and Opacity, respectfully. The off-diagonal elements of this matrix are
the cross products. This matrix is printed once, after the SSCP matrix for the first
model term.
You can use the matrix of partial correlations, labeled as Partial Correlations for the
Error SSCP Matrix, to assess how related the response variables are. These are the
correlations among the residuals or, equivalently, the correlations among the responses
conditioned on the model. Examine the off-diagonal elements. The partial correlations
between Tear and Gloss of 0.00929 and between Gloss and Opacity of -0.04226 are
small. The partial correlation of -0.28687 between Tear and Opacity is not large.
Because the correlation structure is weak, you might be satisfied with performing
univariate ANOVA for these three responses. This matrix is printed once, after the
SSCP matrix for error.
You can use the eigen analysis to assess how the response means differ among the levels
of the different model terms. The eigen analysis is of E-1 H, where E is the error SCCP
matrix and H is the response variable SCCP matrix. These are the eigenvalues that are
used to calculate the four MANOVA tests.
Place the highest importance on the eigenvectors that correspond to high eigenvalues.
In the example, the second and third eigenvalues are zero and therefore the
corresponding eigenvectors are meaningless. For both factors, Extrusion and Additive,
the first eigenvectors contain similar information The first eigen vector for Extrusion is
0.6541, -0.3385, 0.0359 and for Additive it is -0.6630, -0.3214, -0.0684 (not shown).
The highest absolute value within these eigenvectors is for the response Tear, the
second highest is for Gloss, and the value for Opacity is small. This implies that the
Tear means have the largest differences between the two factor levels of either
Extrusion or Additive, the Gloss means have the next largest differences, and the
Opacity means have small differences.

MINITAB User’s Guide 2 4-43

Chapter 4 General MANOVA

General MANOVA
Use general MANOVA to perform multivariate analysis of variance (MANOVA) with
balanced and unbalanced designs or if you have covariates. This procedure takes
advantage of the data covariance structure to simultaneously test the equality of means
from different responses.
Calculations are done using a regression approach. A “full rank” design matrix is
formed from the factors and covariates and each response variable is regressed on the
columns of the design matrix.
Factors may be crossed or nested, but they cannot be declared as random; it is possible
to work around this restriction by specifying the error term to test model terms (see
Specifying terms to test on page 4-46). Covariates may be crossed with each other or
with factors, or nested within factors. You can analyze up to 50 response variables with
up to 9 factors and 50 covariates at one time.

Data
Set up your worksheet in the same manner as with balanced MANOVA: one column
for each response variable, one column for each factor, and one column for each
covariate, so that there is one row of the worksheet for each observation. The factor
columns may be numeric, text, or date/time. If you wish to change the order in which
text categories are processed from their default alphabetical order, you can define your
own order. See Ordering Text Categories in the Manipulating Data chapter in MINITAB
User’s Guide 1. You may include up to 50 response variables and up to 9 factors at one
time.
Although models can be unbalanced in general MANOVA, they must be “full rank.”
That is, there must be enough data to estimate all the terms in your model. For
example, suppose you have a two-factor crossed model with one empty cell. Then you
can fit the model with terms A B, but not A B A∗B. MINITAB will tell you if your model
is not full rank. In most cases, eliminating some of the high order interactions in your
model (assuming, of course, they are not important) can solve non-full rank problems.
Nesting does not need to be balanced. If factor B is nested within factor A, there can be
unequal levels of B within each level of A. In addition, the subscripts used to identify
the B levels can differ within each level of A.
If any response, factor, or covariate column contains missing data, that entire
observation (row) is excluded from all computations. If an observation is missing for
one response variable, that row is eliminated for all responses.

4-44 MINITAB User’s Guide 2

General MANOVA Multivariate Analysis

h To perform an analysis using general MANOVA

1 Choose Stat ➤ Multivariate ➤ General MANOVA.

2 In Responses, enter up to 50 numeric columns containing the response variables.

3 In Model, type the model terms you want to fit. See Overview of Balanced ANOVA
and GLM on page 3-17.
4 If you like, use one or more of the options described below, then click OK.

Options

Covariates subdialog box

■ include up to 50 covariates in the model

Options subdialog box

■ enter a column containing weights to perform weighted regression—see Weighted
regression on page 2-7

Graphs subdialog box

■ draw five different residual plots for regular, standardized, or deleted residuals—see
Choosing a residual type on page 2-6. Available residual plots include a
– histogram
– normal probability plot
– plot of residuals versus the fitted values ( Yˆ)
– plot of residuals versus data order. The row number for each data point is shown
on the x-axis—for example, 1 2 3 4… n
– separate plot for the residuals versus each specified column
For a discussion of the residual plots, see Residual plots on page 2-6.

MINITAB User’s Guide 2 4-45

Chapter 4 General MANOVA

Results subdialog box

■ display different MANOVA output. You can request the display of the hypothesis
matrix H, the error matrix E, and a matrix of partial correlations (see MANOVA tests
on page 4-46), the eigenvalues and eigenvalues for the matrix E-1 H, and univariate
analysis of variance for each response.
■ display a table of means corresponding to specified terms from the model. For
example, if you specify A B D A∗B∗D, four table of means will be printed, one for
each main effect, A, B, D, and one for the three-way interaction, A∗B∗D.
■ perform 4 multivariate tests for model terms that you specify. See Specifying terms to
test on page 4-46. Default tests are performed for all model terms.

Storage subdialog box

■ store model coefficients and fits, in separate columns for each response.
■ regular, standardized, and deleted residuals separately for each response—see
Choosing a residual type on page 2-6.
■ store leverages, Cook’s distances, and DFITS, for identifying outliers—see
Identifying outliers on page 2-9.
■ store the design matrix. The design matrix multiplied by the coefficients will yield
the fitted values. See Design matrix used by General Linear Model on page 3-40.

Specifying terms to test

In the Results subdialog box, you can specify model terms in Custom multivariate test
for the following terms and designate the error term in Error. MINITAB will perform
four multivariate tests (see below) for those terms. This option is most useful when you
have factors that you consider as random factors. Model terms that are random or that
are interactions with random terms may need a different error term than general
MANOVA supplies. You can determine the appropriate error term by entering one
response variable with General Linear Model (page 3-34), choose to display the
expected mean squares, and determine which error term was used for each model
terms (see Expected mean squares on page 3-27).
If you specify an error term, it must be a single term that is in the model. This error
term is used for all requested tests. If you have different error terms for certain model
terms, enter each separately and exercise the general MANOVA dialog for each one. If
you do not specify an error term, MINITAB uses MSE.

MANOVA tests
The MANOVA tests with general MANOVA are similar to those performed for
balanced MANOVA. See MANOVA tests on page 4-40 for details.

4-46 MINITAB User’s Guide 2

References Multivariate Analysis

However, with general MANOVA, there are two SSCP matrices associated with each
term in the model, the sequential SSCP matrix and the adjusted SSCP matrix. These
matrices are analogous to the sequential SS and adjusted SS in univariate General
Linear Model (see page 3-34). In fact, the univariate SS’s are along the diagonal of the
corresponding SSCP matrix. If you do not specify an error term in Error when you
enter terms in Custom multivariate tests for the following terms, then the adjusted
SSCP matrix is used for H and the SSCP matrix associated with MSE is used for E. If
you do specify an error term, the sequential SSCP matrices associated with H and E are
used. Using sequential SSCP matrices guarantees that H and E are statistically
independent. See Help for details on these tests.
You can also perform Hotelling’s T2 test to compare the mean vectors of two groups
(see Hotelling’s T2 Test on page 4-41). Refer to Example of balanced MANOVA on page
4-41 for an example of MANOVA. The dialog operation of general MANOVA is
similar to that of balanced MANOVA.

References
[1] T.W. Anderson (1984). An Introduction to Multivariate Statistical Analysis, Second
Edition, John Wiley & Sons.

[2] W. Dillon and M. Goldstein (1984). Multivariate Analysis, Methods and

Applications, John Wiley & Sons.

[3] S.E. Fienberg (1987). The Analysis of Cross-Classified Categorical Data. The MIT
Press.

[4] H. Harmon (1976). Modern Factor Analysis, Third Edition, University of Chicago
Press.

[5] R. Johnson and D. Wichern (1992). Applied Multivariate Statistical Methods,

Third Edition, Prentice Hall.

[6] K. Joreskog (1977). “Factor Analysis by Least Squares and Maximum Likelihood
Methods,” Statistical Methods for Digital Computers, ed. K. Enslein, A. Ralston and
H. Wilf, John Wiley & Sons.

[7] G.N. Lance and W.T. Williams (1967), “A General Theory of Classificatory
Sorting Strategies, I. Hierarchical systems,” Computer Journal, 9, 373–380

[8] G. W. Milligan (1980). “An Examination of the Effect of Six Types of Error
Pertubation on Fifteen Clustering Algorithms,” Psychometrika, 45, 325-342.

[9] S.J. Press and S. Wilson (1978). “Choosing Between Logistic Regression and
Discriminant Analysis,” Journal of the American Statistical Association 73, 699-705.

MINITAB User’s Guide 2 4-47

Chapter 4 References

4-48 MINITAB User’s Guide 2