Chapter 7

Measure of Association and

Independence problem

1
 Question 7.1

 x1   xn 
    
 y1   yn 

What is the typical method to measure

association?
Explain to a non-specialist the meaning of
positive association.

2
 Question 7.2

 x1   xn 
    
 y1   yn 

To construct a non-parametric measure,

how should we rank the data?

3
1. Sperarman’s Rank Correlation Coefficient

 x1   xn 
Data:     
 y1   yn 
Find a robust measure of association between X & Y.

Classical Product Moment Correlation Coefficient:

p   ( x  x )( y  y )
i i
1   p  1
 (x  x)  ( y  y)
i
2
i
2

 p is appropriate for bivariate normal data, not clear that  p is

appropriate for data that is not normally distributed.
4
Spearman’s Rank Correlation Coefficient:

rs 
 ( R  R )(S  S )
i i
(*)
 ( R  R )  (S  S )
i
2
i
2

Ri = Rank(xi); Si = Rank(yi)

Assuming no ties: R S 
1
(n  1)
2
n3  n
 ( Ri  R )   R  2 R  Ri  nR  12
2
i
2 2

n3  n
 (Si  S )  12
2

1
 i i 2
R S  n ( ( n  1)) 2
6 ( Ri  Si ) 2
 rs   1 (**)
n n
3
n3  n
12

 rs ,  Ri Si and  ( Ri  Si ) 2 are all equivalent.

5
Test for no correlation: (usually used as a test of
independence) :
Test statistic : rs
Case A: (Two –Tailed Test)
H0: The Xi and Yi are mutually independent
H1: Either
(a) there is a tendency for the larger values of X to be paired
with the larger values of Y, or
(b) there is a tendency for the smaller values of X to be
paired with the larger values of Y.
Reject H0 if rs is too large or too small (exceeds the upper  / 2
or the lower / 2 probability point of the null distribution of rs).
6
 Case B: (One-Tailed Test for Positive Correlation)
H0: The Xi and Yi are mutually independent.
H1: There is a tendency for the larger values of X and Y to be
paired together.
Reject H0 if rs is too large (exceeds the upper probability
point of the null distribution of rs).

Case C: (One-Tailed Test for Negative Correlation)

H0: The Xi and Yi are mutually independent.

H1: There is a tendency for the smaller values of X to be paired
with the larger values of Y, and vice verse.
Reject H0 if rs is too small (exceeds the lower  probability
point of the null distribution of rs).
7
Exact Distribution:
Calculate rs for each possible permutation of ( S1 , S 2 , S n )
for fixed permutation ( R1 , R2 , Rn )  (1,2, n).
A total of n! permutations, each occurs equally likely.

An example for n=4. File: rs_exact.exl

1
In large sample : Under H 0 : E (rs )  0 Var (rs ) 
n 1
rs
Z ~ N (0,1)
1
n 1

Ties: Use average ranks.

Use (*) but not (**) to compute rs because
n3  n
 ( Ri  R ) or  (Si  S ) not necessary  12 .
2 2

8
Critical values for rs has been tabled:
(Derived from the exact distribution of rs)
Table A.10 from Conover.

Example:
n=9,  =0.01
Two-sided test (Case A).
Reject H0 if rs is greater than (but not equal to) 0.8167 or
small than (but not equal to) - 0.8167.
(0.8167 is the entry in Table A.10 with n=9 and p=0.995
=1- /2= 1- 0.005.)

9
Example
1 2 3 4 5
X 6 9 2 8 5
Y 8 10 4 7 3

Ri 3 5 1 4 2
Si 4 5 2 3 1

R S
i i  53
2
1  1
 i i 2
R S  n  ( n  1) 
 
53  5(  6) 2
2
rs   0.8
n n
3
5 5
3

12 12

For Case B: One-Tailed Test for Positive Correlation, Table A.10 tells us that
for n=5 and  =0.05 (p=0.95), one can reject H0 if rs is greater than (but not
equal to) 0.8. As a result, the null hypothesis cannot be rejected at  =0.05. 10
 Question 7.3

Plot five data points for which X and Y

may have a positive correlation
coefficient, say between 0.8 to 0.9.

Use a straight line to connect two data

points.

11
 Question 7.4

Plot five data points for which X and Y

have a negative correlation coefficient,
say between 0.8 to 0.9.

Use a straight line to connect two data

points.

12
 Question 7.5

Construct a test statistics for testing no

association.

13
2. Kendall’s Tau
A pair of points is concordant if ( xi  x j )( yi  y j )  0
A pair of points is discordant if ( xi  x j )( yi  y j )  0
Y
x , y 
j j
Y
xi , yi 

xi , yi  x , y 
j j

X X
concordant discordant

ˆ  Proportion of concordant pairs - proportion of discordant pairs

 sign(( x  x )( y
1i  j  n
i j i  y j ))
where
 1 a0
 n sign(a ) 
  1 a  0
 2 14
Notes:
 n  n(n  1)
1.     total no. of pairs.
 2 2

2.  1  ˆ  1

3. Let S = # of concordant pairs

D = # of discordant pairs

If no ties,  n
S  D   
 2

ˆ  S  D  S  D  2 S  1  1  2 D
SD  n  n  n
     
 2  2  2
15
 1 1
4.
 n
  sign (( xi  x j )( yi  y j )) 
 n
  sign (( Ri  R j )( S i  S j ))
   
 2  2

Knowing the ranks can determine whether the pair is concordant or

discordant.

5. ̂ estimates the population parameter 

 = Prob(concordant) – Prob(discordant)
= 2 Prob(concordant) – 1
 ̂ is easy to interpret (when compared to rS ).

6. X and Y independen t    0
  0 does not imply X and Y independent.
16
7. Exact distribution of ̂ under X & Y independent: Calculate ̂
for each of n! permutations of ( S1 , S 2  S n ) for ( R1 , R2  Rn )  (1,2 n)
fixed, then you can get the exact distribution of ̂.

The exact distribution of Kendall’s K is in Table A.21, where

ˆ  n ˆ  K  n 
K   *   or  2
 2  

Note: K can assume only every other integer (always

even or always odd, depending on the value of n).
Table A.21: First half for n that leads to even integers.
Second half for n that leads to odd integers.

ˆ
H0
2 ( 2 n  5)
8. In large sample,  ~ N ( 0, ).
9 n ( n  1) 17
9. Test for independence, use

ˆ
Z
(var(ˆ ))1/ 2
for large sample, and
use Kendall’s K and its exact distribution for small sample.

10. The numerical of ̂ and rs are not the same, and they are
not directly comparable, empirically
2
ˆ  rs
3

Both coefficients can be used to detect the existence of

association in the population.
(Comparison of ̂ & rs , see p.251 Siegel) 18
11. Ties (three types: in X’s, in Y’s, in both X’s and Y’s)

Y Y Y

X X X
in X’s in Y’s in both

SD
ˆ 
 n
 
 2

 n
In the presence of ties S  D   , ˆ underestimates .
 2
Adjust for Var (ˆ ) : tedious See Hollander & Wolfe, p.187
19
Some estimates that adjust for ties:

a. Tau b

SD
ˆ b 
 n  n
   Tx    Ty
 2  2

1
Tx 
2
 t i (t i  1) = total no. of pairs tied on X.

ti = multiplicity of ties for X’s

Ref: A. Agresti (1984). Analysis of Ordinal
Ty: defined similarly Categorical Data. Chapter 9: Measures of
Association for Oridnal Variables.

Mainly used for estimation, test becomes tedious.

20
b. Gamma SD
 
* Mainly used for estimation,
SD test becomes tedious.

Justification of using * :
 n
Let P = # of pairs with ties, S  D  P   
 2
Assign P to be either concordant or discordant, then estimate
gamma by
S  D  (# of pairs of P  S ) - (# of pairs of P  D)
n
 
 2
As   Prob(Concordant) - Prob(Discordant)
 Prob(Concordant) - (1 - Prob(Discordant) )
 2 Prob(Concordant) - 1
 1
 Prob(Concordant) 
2 21
Assign the fraction   1 of the P pairs to be concordant and
2
the rest to be discordant.

*  1 *  1
S  D  P( )  P (1  )
 *  2 2
n
 
 2
S  D  P *

n
 
 2

 n * SD SD
    S  D  P*  *

 2  n SD
   P
 2
22
Example
1 2 3 4 5
X 6 9 2 8 5
Y 8 10 4 7 3 See p.8
rs  0.8
Ri 3 5 1 4 2
Si 4 5 2 3 1

sign(( xi  x j )( yi  y j ))
i \ j 2 3 4 5
1 1 1 -1 1 82
 ˆ   0.6
2 1 1 1 10
3 1 -1
4 1 23
ˆ  8  2  0.6
10

Test H 0 :   0

ˆ 5 ˆ 5 
Pr(   0 .6 )  Pr        0 .6   Pr( K  6 )  0 .117
 2  2 
from Table A.21 (n=5)

Do not reject H0 at  = 0.05

24
3. Online Information

Click “Nonparametric Statistics” then “Nonparametric

Correlations” in the following site to review various
kinds of correlations
[Link]

“Chapter 3b” in the following site has a nice description of rs:

[Link]

25