Topic 6: Discriminant Function

Analysis (DFA)
 Reading
• Manly, Bryan F.J. Multivariate Statistical Methods: A Primer,
Fourth Edition, CRC Press.
– Chapter Eight: Discriminant Function Analysis
– Section 8.2 only briefly and skip all of section 8.10.

Data
 Datasets:
– [Link]
– [Link]

 R code:
– Topic 6 lecture code.R
 R package to install: lattice, MASS, caret
 Objectives
 Understand the purpose of DFA
 Understand the assumptions of DFA and the difference
between DFA and PCA
 Perform a linear DFA from given data using R

Introduction
 DFA is a predominantly classification method that
incorporates dimension reduction.
 It is usually used to predict membership in 2 or more
naturally occurring groups.
 It answers the question: can a combination of variables be
used to predict group membership?
 Usually, several variables are included in a study to see
which ones contribute to the discrimination between group
membership.
 Introduction
 The data requires random samples for 2 or more groups.
 Sparrow data – is it possible to classify sparrows into
survivor or non-survivor groups based on their 5 different
body measurements?
 Egyptian skulls data – is it possible to successfully classify
skulls into the correct time period (of 5 possible periods)
based on the 4 skull dimension measurements.?
 In each example there is one dependent variable with at
least 2 groups and several independent variables.
 There should be measurements for multiple individuals
within each group for all variables - remember missing data
can be a big problem for most forms of multivariate analysis
and an individual is removed from analysis completely if
they are missing data in just one variable (list-wise deletion)
 Variables do not need to be centred or scaled – this does not
influence results.
 Goals of DFA
 To determine which variables discriminate between two or
more naturally occurring groups.
 To model a function that can be used to predict membership
in groups based on measured variables.

Similarities and differences between

DFA and MANOVA
 DFA is MANOVA reversed.
 In MANOVA, the independent variable is the group variable
and the dependent variables are the multiple continuous
measurement variables.
 In DFA, the independent variables are the multiple
continuous measurement variables and the dependent
variable is the group variable.
 Similarities and differences between
DFA and PCA
 Both create linear combinations of the original variables.
 In DFA ordination of variables is made to maximize
discrimination of groups.
 In PCA ordination of variables is made to reduce the number
of variables by developing composite variables (from co-
linear variables). This is done without respect to Groups.
 In PCA the linear combinations of variables (components)
are orthogonal (uncorrelated).
 In DFA the linear combinations of variables (DFs) are
orthogonal within each group.
 The number of possible Discriminant Functions is determined
by the smallest of either: p (number of original variables) or
m-1 (m=number of groups).
 In PCA the number of components possible is determined by
p (number of original variables).
 Similarities and differences between
DFA and Logistic regression
 Section 8.10 of Manly briefly introduces Logistic regression
and works through examples of its application. The topic of
logistic regression is covered extensively in the course
STA3301 Statistical Models and will not be covered here
other than:

 Logistic regression answers the same questions as DFA.

 Logistic regression is often preferred to DFA as it is more
flexible in its assumptions and types of data that can be
analysed.
 Logistic regression can handle both categorical and
continuous independent variables, and the predictors do not
have to be normally distributed, linearly related, or of equal
variance within each group.
 DFA using Mahalanobis distances
 We have yet to consider Mahalanobis distances in detail.
 At this stage it is still sufficient to proceed with the
conceptual understanding stated in Topic 2:
 Mahalanobis distance measures the distance of single
observations from the centre of the multivariate
population (defined by a centroid of the means of all
variables).

 For DFA analysis the distance from the centre of each group
is calculated for each individual and the shortest distance
identifies the group to which the individual is allocated.
 The % of correct allocations indicates how well groups can
be discriminated.
 DFA using canonical discriminant
functions
 Canonical is another term for analysing latent, unobserved
variables.
 We are familiar with the idea of determining linear functions
of variables that describe another variable:
Z= a1 X 1 + a2 X 2 +  + a p X p

 a1, a2 etc. are constants particular to each X variable. For

different values of X1, X2 etc. there is a different outcome
value for Z.
 The greater the differences in the mean of Z for each group
(m) the more likely we will be able to correctly allocate
individuals to groups.
 We want to find estimates of a that maximise the F-ratio
from ANOVA: variation between groups
F=
variation within groups
 DFA using canonical discriminant
functions Source: Manly, Table 8.2

variation between groups

F=
variation within groups

 Maximising F requires that the numerator be bigger than the

denominator.
 The number of possible Z equations for any analysis is equal
to the smallest of either:
 the number of original variables (p)
 the number of groups -1 (m-1)
 The first Z equation explains most of the variation between
and within groups (maximises F)
 Each additional Z equation is modelled on the successive
remaining unexplained variation – each trying to maximise F
while remaining uncorrelated with preceding Z equations
within groups.
 DFA using canonical discriminant
functions
 So, the ith canonical discriminant function:
Z=i ai1 X 1 + ai 2 X 2 +  + aip X p

Is the linear combination for which the F-ratio is maximised,

subject to Zi being uncorrelated with all other Zi within the
groups of interest.
 Finding a for each X in each Z is once again an eigenvalue
problem.
 In Topic 3 we looked at the calculation of T, W and B sums of
squares matrices for defining F

Source: Manly, Table 4.3

 DFA using canonical discriminant
functions
 Next the eigenvalues and vectors for the matrix W-1B can be
found, i.e.
variation between groups BSS B
= =
variation within groups WSS W

 The largest eigenvalue λ1 represents the largest ratio of

between group SS to within group SS and so is the Z1
discriminant function. The elements of the eigenvector
associated with λ1 are the coefficients (a) for each X variable
in Z1.
 Each successively smaller eigenvalue is associated with the
next Z discriminant function and captures as much of the
group differences not already explained by the larger
eigenvalues.
 It is hoped that the first few functions account for almost all
group differences. Two makes graphing easier!
 Tests of significance
 Once we have the Z functions defined we can go back and
use MANOVA to test if there are any significant differences in
the mean of the discriminant functions between groups.
 When applying the MANOVA, the DFs are the multiple
dependent variables and the groups are the independent
variable.
define DFs (Zs): groups = X 1 + X 2 + X 3 + X 4 + X 5
test differences between groups MANOVA : Z 1 + Z 2 + Z 3 = groups
apply DFA to classify individuals into groups: groups = Z 1 + Z 2 + Z 3

 Any one of the 4 MANOVA statistics covered in Topic 3 can

be used to test for differences between the means of >2
groups: Wilk’s Lambda; Roy’s largest root; Pillai’s trace;
Lawes-Hotelling trace.
 Hotellings’ T2 test is used to test for differences in the mean
values between any pair of groups.
 Tests of significance
 Manly also discussed another significance test based on
incorporating the eigenvalues and using the chi-square
distribution. We will not look at this method any further.
However, it does raise a good point for consideration.
 The first DF defined from the sample may not represent the
first DF in the population due to sampling error. For the
population, the estimated second DF may actually have the
largest eigenvalue.
 Representative sampling is a concern for all statistical analyses
and is not unique to this method.
 However, simulations have shown that this can upset the chi-
square tests (described by Manly) quite a bit.
 The key implication of this is that we should not rely on
significance tests to help decide how many of the obtained
discriminant functions represent real group differences.
 We should not run MANOVA with different numbers of DFs to
see how many we need to find significant differences between
groups.
 Assumptions of DFA
 Multivariate normality within groups.
 Equal within-group covariances, i.e. the covariance matrix C
should be similar for all groups.
 Some MV analyses that assume MVN can fail badly if this
assumption is not correct.
 Univariate methods such as regression and ANOVA are quite
robust to deviations from normality and equal covariance.
 A failure of one or both assumptions does not necessarily
mean that DFA or any MV method is not useful.
 Good discrimination may be possible between groups that
deviate from MVN. The procedure may be very useful for
classification purposes without being able to show
statistically significant differences between groups.
 Alternatively, group discriminating methods such as logistic
regression do not require MVN distribution of the
independent variables.
 DFA by calculation R
 Manly, Example 8.1 using the data in [Link].
> es <-[Link]("[Link]", header=TRUE)
> head(es)
MB BH BL NH Year Period
1 131 138 89 49 -4000 Early.P
2 125 131 92 48 -4000 Early.P
3 131 132 99 50 -4000 Early.P
4 119 132 96 44 -4000 Early.P
5 136 143 100 54 -4000 Early.P
6 138 137 89 56 -4000 Early.P
> str(es)
'[Link]': 150 obs. of 6 variables:
$ MB : int 131 125 131 119 136 138 139 125 131 134 ...
$ BH : int 138 131 132 132 143 137 130 136 134 134 ...
$ BL : int 89 92 99 96 100 89 108 93 102 99 ...
$ NH : int 49 48 50 44 54 56 48 48 51 51 ...
$ Year : int -4000 -4000 -4000 -4000 -4000 -4000 -4000 -4000 -4000 -4000 ...
$ Period: Factor w/ 5 levels "Dyn.12.13","Early.P",..: 2 2 2 2 2 2 2 2 2 2 ...
> table(es$Period)

Dyn.12.13 Early.P Late.P Ptolemaic Roman

30 30 30 30 30

 The data is made up of 5 time periods (groups) with 30

skulls measured in each. Four variables describe the shape
of the skulls.
 In Topic 3 we found a significant difference between groups
using MANOVA on the original variables.
 The solution for the W and B matrices can be found using the
code in the ‘Topic 3 lecture code.R’ file
 DFA by calculation in R
> ##the matrices for W and B were calculated in ‘Topic 4 lecture code.R’

> (F<-(solve(W))%*%B)
MB BH BL NH
MB 0.15626222 -0.06961845 -0.19384987 0.04036370
BH -0.03959910 0.05803624 0.05143415 -0.01342543
BL -0.17502509 0.07371863 0.22450109 -0.05081136
NH 0.09171218 -0.05559491 -0.12388564 0.04301953
> (eig<- eigen(F))
$values
[1] 0.425095375 0.038998898 0.015704460 0.002020346

$vectors
[,1] [,2] [,3] [,4]
[1,] -0.5948890 0.1633252 -0.29908168 0.708118906
[2,] 0.1739077 0.8858071 0.07920781 -0.001998589
[3,] 0.6815272 -0.2871818 -0.04754893 0.630384196
[4,] -0.3890808 -0.3258794 0.94974490 0.318087073

 The eigenvalues match those in Manly Exercise 8.1. The

eigenvector elements are different, probably due to a default
normalisation implemented by the software used by Manly. R
does not normalise.

 DF1 explains most of differences in skulls between groups.

 MB (maximum breadth) is strongly -ve = very narrow skulls
 BH (basal height) weakly +ve= higher than average skulls
 BL (basal length) strongly +ve = very long jaws
 NH (nasal height) moderately –ve = short nasal heights
 DFA by calculation in R
> ##the matrices for W and B were calculated in eskulls.R week 4

> (F<-(solve(W))%*%B)
MB BH BL NH
MB 0.15626222 -0.06961845 -0.19384987 0.04036370
BH -0.03959910 0.05803624 0.05143415 -0.01342543
BL -0.17502509 0.07371863 0.22450109 -0.05081136
NH 0.09171218 -0.05559491 -0.12388564 0.04301953
> (eig<- eigen(F))
$values
[1] 0.425095375 0.038998898 0.015704460 0.002020346

$vectors
[,1] [,2] [,3] [,4]
[1,] -0.5948890 0.1633252 -0.29908168 0.708118906
[2,] 0.1739077 0.8858071 0.07920781 -0.001998589
[3,] 0.6815272 -0.2871818 -0.04754893 0.630384196
[4,] -0.3890808 -0.3258794 0.94974490 0.318087073

 There were (5-1) groups and 4 variables so the number of

possible DFs =4

 We can calculate the Z value for each DF for each individual

−0.595 X 1 + 0.174 X 2 + 0.682 X 3 − 0.389 X 4

Z1 =
 DFA in R
 We can use the linear discriminant function (lda) in the Mass
package in R.
 First, using the lattice package we can look at the splom
plot. It doesn’t look like there is much discrimination
between periods (colours) for any of the original variables,
but we will continue with the analysis:

> library(lattice)
> head(es)
MB BH BL NH Year Period
1 131 138 89 49 -4000 Early.P
2 125 131 92 48 -4000 Early.P
3 131 132 99 50 -4000 Early.P
4 119 132 96 44 -4000 Early.P
5 136 143 100 54 -4000 Early.P
6 138 137 89 56 -4000 Early.P
> splom(es[,1:4], groups=es$Period)
 DFA in R
> #Linear DA (LDA) = DFA
> library(MASS)
> ([Link]<-lda(Period~MB+BH+BL+NH, data=es)) #defines DF's
Call:
lda(Period ~ MB + BH + BL + NH, data = es)

Prior probabilities of groups:

Dyn.12.13
0.2
Early.P
0.2
Late.P Ptolemaic
0.2 0.2
Roman
0.2
 The prior probabilities are
Group means: the observed proportions
of observations in each of
MB BH BL NH
Dyn.12.13 134.4667 133.8000 96.03333 50.56667
Early.P 131.3667 133.6000 99.16667 50.53333
Late.P
Ptolemaic
Roman
132.3667
135.5000
136.1667
132.7000
132.3000
130.3333
99.06667
94.53333
93.50000
50.23333
51.96667
51.36667
the groups.
Coefficients of linear discriminants:  They are the probabilities
of group membership
LD1 LD2 LD3 LD4
MB 0.12667629 0.03873784 0.09276835 0.1488398644
BH -0.03703209 0.21009773 -0.02456846 -0.0004200843
BL -0.14512512 -0.06811443 0.01474860 0.1325007670
NH 0.08285128 -0.07729281 -0.29458931 0.0668588797 without any information
Proportion of trace:
LD1 LD2 LD3 LD4 on the X variables
0.8823 0.0809 0.0326 0.0042

 Standardized beta coefficients are given for each variable on

each DF. The larger the standardized coefficient (+ or -), the
greater the contribution of the respective variable to the
discrimination between groups.
 The trace are eigenvalues as proportion of total variance.
The first 3 DFs account for 99.58% of the between group
variance.
 DFA in R
 Caution: The % variance tells us only that the best DFs have
been found. If there is not much variance (difference)
between groups to start with then DFA may not be the best
method for analysis.
 We can next check if MANOVA shows any difference between
groups using the 4 DFs as dependent variables.
> es.manova1<-manova(cbind(LD1, LD2, LD3, LD4) ~ class, data=[Link])
> summary(es.manova1, test="Pillai")
Df Pillai approx F num Df den Df Pr(>F)
class 4 0.35331 3.512 16 580 4.675e-06 ***
Residuals 145
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> summary(es.manova1, test="Wilks")
Df Wilks approx F num Df den Df Pr(>F)
class 4 0.66359 3.9009 16 434.45 7.01e-07 ***
Residuals 145
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> summary(es.manova1, test="Roy")
Df Roy approx F num Df den Df Pr(>F)
class 4 0.4251 15.41 4 145 1.588e-10 ***
Residuals 145
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> summary(es.manova1, test="Hotelling-Lawley")
Df Hotelling-Lawley approx F num Df den Df Pr(>F)
class 4 0.48182 4.231 16 562 8.278e-08 ***
Residuals 145
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

 There are significant differences between average DF values

for at least 2 Periods as all 4 tests show p<0.05
 DFA in R
 Next we can predict group membership for each of the 150
skulls based on the DFs (all 4 DFs used here). Membership is
based on the posterior probabilities. The largest posterior
probability for skull 1 was for Dyn.12.13 (12th and 13th
Dynasties) and for skull 7 it was for Late Predynastic.
> ([Link]<-predict([Link], es)) #predicts period
$class
[1] Dyn.12.13 Early.P Late.P Early.P Dyn.12.13 Ptolemaic Late.P Early.P Early.P
[10] Late.P Early.P Roman Early.P Early.P Dyn.12.13 Early.P Late.P Ptolemaic
.
. Sample of output only

$posterior
Dyn.12.13 Early.P Late.P Ptolemaic Roman
1 0.38574650 0.126661578 0.101186457 0.23328032 0.153125144
2 0.20220845 0.321980768 0.255595802 0.11876318 0.101451799
3 0.18031220 0.308741058 0.311401139 0.11350692 0.086038685
4 0.08982797 0.533701270 0.347482903 0.01717823 0.011809618
5 0.33800349 0.218032265 0.169904097 0.20550568 0.068554464
6 0.18452577 0.025981337 0.024793787 0.45730883 0.307390282
7 0.14141679 0.263419397 0.473156261 0.06052421 0.061483345
.
. Sample of output only

 Assuming MVN distribution and equal covariance matrices

the posterior probabilities ( fi ) are calculated by:
1
fi =µi C −1 xkT − µi C −1µiT + ln( P(i ))
2
 The calculation is beyond the scope of this course other than
to recognise that it is the relationship of the distance of
individuals from group means and the prior probabilities of
group membership.
 DFA in R
 A summary table of the prior and posterior group
membership, i.e. observed vs predicted shows how accurate
the model is. > table(es$Period, [Link]$class) #compares true period to
predicted

Dyn.12.13 Early.P Late.P Ptolemaic Roman

Dyn.12.13 15 4 4 2 5
Early.P 4 12 8 4 2
Late.P 5 10 8 4 3
Ptolemaic 7 3 3 5 12
Roman 4 2 4 9 11

 Add up the diagonal and only 51 of the 150 individuals

(34%) were correctly classified. Not great.
 This matches Table 8.4 in Manly, although the order of the
rows and columns is different.
 Although there are differences in average skull dimensions
between some periods (MANOVA), DFA has not been able to
successfully classify them.
 As part of the output from the predict function the co-
ordinates for individuals on the DFs are given and we will use
for plotting. $x

1
LD1 LD2 LD3 LD4
0.34385509 1.688118889 0.049703532 -1.562557613
2 -0.67520466 -0.142042770 0.003907799 -2.122012788
3 -0.80235231 -0.330904641 0.050011004 -0.168170557
4 -2.38420012 -0.127658572 0.660080886 -2.752904509
5 -0.39004387 1.796573961 -0.920009008 0.971344123
6 1.84758020 1.208136380 -1.338474744 -0.052246320
 DFA in R
 We can plot the relationship between all skulls on the first
two DFs and group first by their true Period and then by the
predicted Period >
>
# DFs with individuals grouped by original Period classifications
[Link]<-[Link]([Link]$x, class=es$Period)
> xyplot(LD2~LD1, data=[Link], groups=class,
+ [Link]=list(title="Period", space = "top", cex=1.0))
> # DFs with individuals grouped by predicted Period classifications
> lda.temp1<-[Link]([Link]$x, class=[Link]$class)
> xyplot(LD2~LD1, data=lda.temp1, groups=class,
+ [Link]=list(title="Period", space = "top", cex=1.0))

 Original Periods on the left and predicted on the right.

 DFA in R
 It would be useful if we could calculate the DFs using one
data set, a training set, and then see how the model works
to classify skulls in a completely independent data set, a test
set.
 This gives an estimate of how well the model behaves on
completely unseen data that was not used to help build the
model.
 We can select random samples from each Period easily using
the caret package createDataPartition function.
> library(caret)
> [Link](107)
> inTrain <- createDataPartition(y = es$Period, ## the outcome data are needed
+ p = .75, ## The percentage of data in the training set
+ list = FALSE)
>
> estrain <- es[ inTrain,]
> estest <- es[-inTrain,]
> table(estrain$Period)

Dyn.12.13 Early.P Late.P Ptolemaic Roman

23 23 23 23 23
> table(estest$Period)

Dyn.12.13 Early.P Late.P Ptolemaic Roman

7 7 7 7 7

 A check of the data will confirm that rows included in estrain

are not also in estest.
 DFA in R
 Dividing the data into 75% training and 25% testing is a
good rule of thumb.
 Setting the seed value before sub-setting will ensure that if
the code is run again exactly the same sample will be
selected for each set = reproducibility.
 With 5 categories the eskulls sample of 150 with 30 skulls in
each group may be a little small. The DF analysis on the full
data wasn’t very helpful discriminating between groups and
proceeding with a training and test set is not likely to help,
but we will proceed as an exercise.
> [Link]<-lda(Period~MB+BH+BL+NH, data=estrain) #defines DF's based on training data
> [Link]<-predict([Link], estest) #predicts period for estest set
> table(estest$Period, [Link]$class) #compares true period to predicted for test group

Dyn.12.13 Early.P Late.P Ptolemaic Roman

Dyn.12.13 1 1 3 1 1
Early.P 2 2 1 1 1
Late.P 1 4 0 0 2
Ptolemaic 1 1 0 2 3
Roman 0 1 1 3 2

 The DFA model is built using the training data which is then
used to predict the group membership for the test set using
their X values in the model.
 Only 7/35 = 20% of skulls were correctly classified for skulls
whose data did not help estimate DF loadings.
 DFA in R
 Manly, Example 8.2 using the data in [Link]. R
code is available in ‘Topic 6 lecture code.R’
 We will cover this example only briefly, in particular to show
how Manly, Table 8.5 is produced. The sample size (n=30) is
too small to implement training and test sets
> eeg <- [Link]("[Link]", header=TRUE)
> library(MASS)
> ([Link]<-lda(Group~AGR+MIN+MAN+PS+CON+SER+FIN+SPS, data=eeg)) #defines DF's
Call:
lda(Group ~ AGR + MIN + MAN + PS + CON + SER + FIN + SPS, data = eeg)

Prior probabilities of groups:

Eastern EFTA EU Other
0.2666667 0.2000000 0.4000000 0.1333333

Group means:
AGR MIN MAN PS CON SER FIN SPS
Eastern 21.462500 11.7875000 20.56250 0.6375000 6.912500 9.38750 3.000000 19.45000
EFTA 6.833333 0.3166667 20.53333 0.8666667 7.900000 16.75000 8.500000 31.21667
EU 7.666667 0.4500000 20.99167 0.7916667 7.283333 18.55833 8.391667 29.63333
Other 15.225000 0.4500000 17.25000 1.0500000 8.950000 17.70000 5.950000 27.82500

Coefficients of linear discriminants:

LD1 LD2 LD3
AGR 0.4269282 0.6741257 -0.7315911
MIN 0.2951339 0.5787195 -0.8889400
MAN 0.3589331 0.5497880 -0.8728458
PS 0.3386116 1.5764251 -0.4101754
CON 0.2222024 0.6820566 -0.5239559
SER 0.6878378 0.6581655 -0.8953256
FIN 0.4638659 0.3485413 -0.7140873
SPS 0.5138482 0.6819368 -0.7635014

Proportion of trace:
LD1 LD2 LD3
0.8739 0.0931 0.0330
> [Link]<-predict([Link], eeg)

 There are 3 DFs because 4 groups-1=3 which is less than

the number of variables =8
 DFA in R
 The first DF alone describes 87% of the multivariate
differences in employment patterns between the groups
(from proportion of trace).
 24/30 of the countries were successfully allocated to their
correct group based on the employment patterns. 1 eastern,
2 EFTA, 1 EU and 2 Other were misclassified.
> table(eeg$Group, [Link]$class)
> (original<-eeg[c(2:10)])
Eastern EFTA EU Other AGR MIN MAN PS CON SER FIN SPS TC
Eastern 7 0 0 1 1 2.6 0.2 20.8 0.8 6.3 16.9 8.7 36.9 6.8
EFTA 0 4 2 0 2 5.6 0.1 20.4 0.7 6.4 14.5 9.1 36.3 7.0
EU 0 1 11 0 3 5.1 0.3 20.2 0.9 7.1 16.7 10.2 33.1 6.4
Other 0 0 2 2 4 3.2 0.7 24.8 1.0 9.4 17.2 9.6 28.4 5.6
.

 Manly Table 8.5 also looks at how > (dfa<-[Link]$x)

LD1 LD2 LD3

the original data for each variable 1

2
3
1.77033991 -0.596519910 -0.06271140
1.09282366 -0.790709610 0.49041695
1.46904531 -0.680099756 0.09218174
correlates with the predicted data. 4
.
0.62246705 -0.559198140 -0.56533154

This is useful because the coefficients > cor(original,dfa)

LD1 LD2 LD3

of all variables on each DF share

AGR -0.49745605 0.371967370 -0.09055516
MIN -0.62235739 0.025444044 -0.19564975
MAN -0.01563794 -0.199104296 -0.11705855

the same sign making interpretation

PS 0.16582232 0.182138815 0.22504699
CON 0.13607158 0.255003049 0.33871557
SER 0.82097975 -0.005855368 -0.07977667

difficult. FIN 0.60676980 -0.361737323

SPS 0.56329282 -0.189112974
TC -0.21967516 -0.466591936
0.08936932
0.27857035
0.40630115

 Using the correlations we can see that:

 On DF1 it is AGR and MIN in opposition to SER and FIN that
most help distinguish groups.
 DFA in R
 The lda function in the MASS package does allow for a vector
of prior probabilities of group membership to be defined.
They should be provided in the order of the groups in the
dataset. For example:
> ([Link]<-lda(Group~AGR+MIN+MAN+PS+CON+SER+FIN+SPS, prior=c(0.15,0.05, 0.3, 0.5), data=eeg))
Call:
lda(Group ~ AGR + MIN + MAN + PS + CON + SER + FIN + SPS, data = eeg,
prior = c(0.15, 0.05, 0.3, 0.5))

Prior probabilities of groups:

Eastern EFTA EU Other
0.15 0.05 0.30 0.50

 If this information is not provided, the prior probabilities are

the observed proportions of observations in each of the
groups.

 Generally the sample proportions should reflect any naturally

occurring bias in the population and are satisfactory as priors
– unless the sample size is small or the sample proportions
are an aspect of experimental design (purposefully collecting
data from equal numbers across groups).
 DFA in R
 Stepwise analysis – the problems associated with trying
different numbers of DFs and using significance tests to
determine the best number of DFs was address earlier.
 Attempting different numbers and combinations of variables
to produce the DFs should also be approached with caution.
 In addition to the randomisation method suggested by
Manly, corrections for multiple testing (e.g. bonferroni) and
using training and test sets to allow cross-validation should
be considered.
 Jackknife – I mentioned earlier that when calculating
posterior probabilities it is a measure of distance between
individuals and the mean of the group that is being
calculated. An individual is more likely to be closer to the
centre of a group if it has contributed to defining the group
centre. In jackknife classification the mean of that group is
calculated without that individual before the distance
between the individual and the group is calculated. Jackknife
is not implemented by the predict function in R.