RELIABILITY
The RELIABILITY procedure employs one of two different computing methods,
depending upon the MODEL specification and options and statistics requested.
Method 1 does not involve computing a covariance matrix. It is faster than
method 2 and, for large problems, requires much less workspace. However, it can
compute coefficients only for ALPHA and SPLIT models, and it does not allow
computation of a number of optional statistics, nor does it allow matrix input or
output. Method 1 is used only when alpha or split models are requested and only
FRIEDMAN, COCHRAN, DESCRIPTIVES, SCALE, and/or ANOVA are
specified on the STATISTICS subcommand and/or TOTAL is specified on the
SUMMARY subcommand.
Method 2 requires computing a covariance matrix of the variables. It is slower
than method 1 and requires more space. However, it can process all models,
statistics, and options.
The two methods differ in one other important respect. Method 1 will continue
processing a scale containing variables with zero variance and leave them in the
scale. Method 2 will delete variables with zero variance and continue processing if
at least two variables remain in the scale. If item deletion is required, method 2 can
be selected by requesting the covariance method.
Notation
There are N persons taking a test that consists of k items. A score X ji is given to
the jth person on the ith item.
i items
1
1
X 11
X 12
X 1k
X ji
Pj
M
N
P1
X N1
X N2
T1
T2
Ti
X Nk
PN
Tk
2 RELIABILITY
If the model is SPLIT, k1 items are in part 1 and k 2 = k k1 are in part 2. If the
number of items in each part is not specified and k is even, the program sets
k1 = k 2 = k 2 . If k is odd, k1 = k + 1 2 . It is assumed that the first k1 items are
in part 1.
1 6
wj
W=
Sum of the weights, where w j is the weight for case j
j =1
Pj =
ji
The total score of the jth person
i =1
Pj = Pj k
Mean of the observations for the jth person
Ti =
X ji w j
The total score for the ith item
j =1
k
G=
X ji w j
Grand sum of the scores
i =1 j =1
G =
G
Wk
Grand mean of the observations
Scale and Item StatisticsMethod 1
Item Means and Standard Deviations
Mean for the ith Item
Ti = Ti W
RELIABILITY 3
Standard Deviation for the ith Item
N
w j X 2ji WTi 2
j =1
Si =
W 1
Scale Mean and Scale Variance
Scale Mean
M =G W
For the split model:
Mean Part 1
k1
M1 =
i =1
Mean Part 2
k
M2 =
Ti
i = k1 +1
Scale Variance
S 2p
6 !
1
=
W 1
j =1
W T
k
Pj2 w j
i =1
"#
##
$
4 RELIABILITY
For the split model:
Variance Part 1
S 2p1
1
=
W 1
w
X
!
k1
j =1
W T
k1
ji
i =1
i =1
"#
##
$
Variance Part 2
1
S 2p2 =
W 1
N k 2 k 2 "#
w j X ji W Ti ##
! j =1 i = k +1 i = k +1 $
1
Item-Total Statistics
Scale Mean if the ith Item is Deleted
~
M i = M Ti
Scale Variance if the ith Item is Deleted
~
Si2 = S 2p + Si2 2 cov X i , P
where the covariance between item i and the case score is
cov X i , P =
1
W 1
P X
N
j =1
k
ji w j
Tl Ti
l =1
RELIABILITY 5
Alpha if the ith Item Deleted
S
k 1
Ai =
1
k 2
2
l
~
Si2
l =1
l i
Correlation between the ith Item and Sum of Others
Ri =
cov X i , P Si2
~
Si Si
The ANOVA Table (Winer, 1971)
Source of
variation
Sum of Squares
df
Between
people
Pj2 w j
W 1
j =1
k
Within
people
k G 2 Wk
w j X 2ji
i =1 j =1
Pj2 w j
1 6
W k 1
j =1
Between
measures
i =1 j =1
Total
W G 2 Wk
k 1
i =1
k
Residual
w j X 2ji
Pj2 w j
j =1
Ti2 W G 2 Wk 1W 161k 16
i =1
w j X 2ji G 2 Wk
i =1 j =1
Wk 1
6 RELIABILITY
Each of the mean squares is obtained by dividing the sum of squares by the
corresponding degrees of freedom. The F ratio for between measures is
F=
MS between measures
,
MS residual
61 67
df = k 1, W 1 k 1
Friedman Test or Cochran Test
2 =
SS between measures
,
MS within people
df = k 1
Note: Data must be ranks for the Friedman test and a dichotomy for the Cochran test.
Kendalls Coefficient of Concordance
W=
SS between measures
SS total
(Will not be printed if Cochran is also specified.)
Tukeys Test for Nonadditivity
The residual sums of squares are further subdivided to
SS nonadd = M 2 D ,
df = 1
RELIABILITY 7
where
D = SS bet. meas SS bet. people
8 1Wk 6
"
= 3T G 8 "# w 3 P G 8 ##
#$ !
#$
!
k
i =1
j =1
T P X w G P w
= w 3 P G 8 X 3T G 8"##
!
$
M=
i =1
ji
2
j
j =1
k G SS bet. meas
j =1
ji
j =1
i =1
SS bal = SS res SS nonadd ,
61 6
df = W 1 k 1 1
The test for nonadditivity is
F=
MS nonadd
MS balance
2 1
61 6 7
df = 1, W 1 k 1 1
The regression coefficient for the nonadditivity term is
B$ = M D ,
and the power to transform to additivity is
$
p$ = 1 BG
8 RELIABILITY
Scale Statistics
Reliability coefficient alpha (Cronbach 1951)
S
k
A=
1 S
k 1
k
2
i
i =1
2
p
If the model is split, separate alphas are computed:
A1 =
SS
k1
1
k1 1
k1
2
i
2
p1
i =1
k
A2 =
2
Si2
k2
1
k2 1
S2
i = k +1 p 2
1
For Split Model Only
Correlation Between the Two Parts of the Test
1 2
S p S 2p1 S 2p 2
2
R=
S p1S p 2
Equal Length Spearman-Brown Coefficient
Y=
2R
1+ R
RELIABILITY 9
Guttman Split Half
G=
2 S 2p S 2p1 S 2p2
S 2p
Unequal Length Spearman-Brown
ULY =
R 2 + R 4 + 4 R 2 1 R 2 k1k 2 k 2
2 1 R 2 k1k 2 k 2
Basic ComputationsMethod 2
Items with zero variance are deleted from the scale and from k , k1 , and k 2 . The
inverses of matrices, when needed, are computed using the sweep operator
described by Dempster (1969). If V < 10 30 , a warning is printed and statistics
that require V 1 are skipped.
Covariance Matrix V and Correlation Matrix R
%K 1
W 1 X X w
K
=&
KKr S S
K'
li
vij
lj
WTi Tj , i , j = 1, K , k
l =1
vij
Si Si
, where Si2 =
if raw data input
if correlation matrix and SD input
ij i j
rij =
vij
W 1
10 RELIABILITY
Scale Variance
k
S 2p =
Si2 + 2
i =1
vij
i< j
If the model is split,
S p21 =
k1
Si2 + 2
i =1
k1
ij
i< j
S p22 =
k1
Si2 + 2
i = k1 +1
ij
i = k1 +1 j > i
where the first k1 items are in part 1.
Scale StatisticsMethod 2
Alpha Model
Estimated Reliability
S
k
1
k 1
S
k
i =1
2
i
2
p
Standardized Item Alpha
k Corr
1 6
1 + k 1 Corr
RELIABILITY 11
where
k
rij
1 6 i <
j
2
Corr =
k k 1
Split Model
Correlation between Forms
k1
ij
i =1 j = k1 +1
S p1S p2
Guttman Split-Half
k1
v
G=
ij
i =1 j = k1 +1
S 2p
Alpha and Spearman-Brown equal and unequal length are computed as in method 1.
12 RELIABILITY
Guttman Model (Guttman 1945)
k
2
i
L1 = 1
i =1
S p2
2k
k 1
L2 = L1 +
L3 =
2
ij
i< j
2
Sp
k
L1
k 1
k
4
L4 =
ij
i< j
S p2
2 max
i
L5 = L1 +
2
ij
j i
S p2
k
L6 = 1
2
i
3 8
S p2 ; where i2 = V 1
i =1
Parallel Model (Kristof 1963)
Common Variance
CV = var =
1
k
S
i =1
2
i
1
ii
RELIABILITY 13
True Variance
TV = cov =
vij
1 6 i<
j
2
k k 1
Error Variance
EV = var cov
Common Inter-Item Correlation
R$ = cov var
Reliability of the Scale
S
k
A=
1 S
k 1
k
2
i
i =1
2
p
Unbiased Estimate of the Reliability
1 6
1 6
2+ W 3 A
A$ =
W 1
where A is defined above.
14 RELIABILITY
Test for Goodness of Fit
k 1 k + 16 12 k 36
= 1W 16 1
121k 16 k 1k + 16 2 1W 16 log L
2
2
where
L=
3var cov8 3var + 0k + 15cov8
k 0k + 15
df =
2
k 1
Log of the Determinant of the Unconstrained Matrix
log UC = log V
Log of the Determinant of the Constrained Matrix
4
log C = log var cov
9k 14var + 1k 16cov9
Strict Parallel (Kristof 1963)
Common Variance
CV = var +
1
k
3T G 8
i
i =1
RELIABILITY 15
Error Variance
EV = MS within people
All mean squares are calculated as described later in the analysis of variance table.
True Variance
TV = var +
1
k
3T G 8
i
EV
i =1
Common Inter-Item Correlation
R$ =
1 6 3T G 8
1
var + 3T G 8
k
1
cov
k 1 k
i =1
i =1
Reliability of the Scale
Re l =
kR$
1 + k 1 R$
1 6
Unbiased Estimate of the Reliability
Re l =
3 + W 3 Re l
W
16 RELIABILITY
Test for Goodness of Fit
6
2 = W 1 1
1 6 12k 36 log L
121 k 162 k 1 k + 36 2 371W 16
k k +1
where
L=
k
k 1
2
1
4var + 1k 16cov9 var cov + k 3Ti G 8
i =1
df = k 1 k + 36 2 3
Log of the Determinant of the Unconstrained Matrix
log UC = log V
Log of the Determinant of the Constrained Matrix
1
log C = log4 var + 1 k 16cov9 var cov +
T
G
3
8
k 1
k
i =1
k 1
RELIABILITY 17
Additional StatisticsMethod 2
Descriptive and scale statistics and Tukeys test are calculated as in method 1.
Multiple R 2 if an item is deleted is calculated as
2
~
Ri2 = 1 i2
Si
i2 =
4V 9ii
1
Analysis of Variance Table
Source of
variation
Sum of Squares
df
1
1W 16 k S
!
k
Between
people
Within
people
i =1
k 1
"# 2
v # + 1k 16 v
#$
ij
i< j
W T
T
1W 161k 16 S 2
k 1v
k
!
i
i =1
i =1
Between SS + Within SS
2
T 2 = WY B 1Y
k 1
2
i
Hotellings T (Winer, 1971)
1 6
W k 1
i =1
Total
W 1
i< j
Residual
ij
"
1W 161k 16 k Si2 2
vij # + 1W 16SS bet. people
k
! i =1 k 1 i < j ##$
k
Between
measures
2
i
ij
i< j
"#
##$
1W 161k 16
Wk 1
18 RELIABILITY
where
T T "#
T T #
Y=
##
M
!T T #$
B = CVC
1
k 1
where C is an identity matrix of rank k 1 augmented with a column of 1 on the right.
bij = vij vik v jk + S k2
The test will not be done if W < k or B < 10 30 .
The significance level of T 2 is based on
F=
W k +1
1W 161k 16 T
Item Mean Summaries
k
Mean =
i =1
T T
Variance =
1k 16
k
i =1
i =1
Maximum = max Ti
i
Minimum = min Ti
i
Range = Maximum Minimum
Ratio =
Maximum
Minimum
, with df = k 1, W k + 1
RELIABILITY 19
Item Variance Summaries
Same as for item means excepts that Si2 is substituted for Ti in all calculations.
Inter-Item Covariance Summaries
v
Mean =
k 1 k 16
ij
i< j
1 6
!
1
Variance =
k k 1 1
vij2
i< j
1 6 v
k k 1
ij
i< j
"#
##
$
Maximum = max vij
i, j
Minimum = min vij
i, j
Range = Maximum Minimum
Ratio =
Maximum
Minimum
Inter-Item Correlations
Same as for inter-item covariances, with vij being replaced by rij .
If the model is split, statistics are also calculated separately for each scale.
Intraclass Correlation Coefficients
Intraclass correlation coefficients are always discussed in a random/mixed effects
model setting. McGraw and Wong (1996) is the key reference for this document.
See also Shrout and Fleiss (1979).
In this document, two measures of correlation are given for each type under each
model: single measure and average measure. Single measure applies to single
measurements, for example, the ratings of judges, individual item scores, or the
body weights of individuals, whereas average measure applies to average
20 RELIABILITY
measurements, for example, the average rating for k judges, or the average score
for a k-item test.
One-Way Random Effects Model: People Effect Random
Let Xji be the response to the i-th measure given by the j-th person, i = 1, , k, j =
1, , W. Suppose that Xji can be expressed as Xji = + pj + wji, where pj is the
between-people effect which is normal distributed with zero mean and a variance
of 2p , and wji is the within-people effect which is also normal distributed with zero
mean and a variance of w2 .
Let MS BP and MSWP be the respective between-people Mean Squares and withinpeople Mean Squares. Formulas for these two quantities can be found on page 479
of SPSS 7.5 Statistical Algorithms by dividing the corresponding Sum of Squares
with its degrees of freedom.
Single Measure Intraclass Correlation
The single measure intraclass correlation is defined as
(1) =
2p
2p + 2w
Estimate
The single measure intraclass correlation coefficient is estimated by
ICC (1) =
MS BP - MSWP
.
MS BP + ( k - 1) MSWP
In general,
-1
< ICC(1) 1.
k -1
Confidence Interval
RELIABILITY 21
For 0 < < 1, a (1- )100% confidence interval for (1) is given by
Fp / w - F / 2 ,W -1,W ( k -1)
Fp / w + ( k - 1) F / 2 ,W -1,W ( k -1)
< (1) <
Fp / w - F1- / 2 ,W -1,W ( k -1)
Fp / w + ( k - 1) F1- / 2 ,W -1,W ( k -1)
where
MS BP
MSWP
Fp / w =
and F ,v1 ,v2 is the upper point of a F-distribution with degrees of freedom v1
and v2.
Hypothesis Testing
The test statistic F (1) for H0 : (1) = 0 , where 1 > 0 0 is the hypothesized
value, is
F (1) = Fp / w
1- 0
.
1 + ( k - 1) 0
Under the null hypothesis, the test statistic has an F-distribution with
W - 1, W ( k - 1) degrees of freedom.
Average Measure Intraclass Correlation
The average measure intraclass correlation is defined as
(k ) =
2p
2p + 2w / k
Estimate
The average measure intraclass correlation coefficient is estimated by
22 RELIABILITY
ICC ( k ) =
MS BP - MSWP
.
MS BP
Confidence Interval
A (1- )100% confidence interval for ( k ) is given by
Fp / w - F / 2 ,W -1,W ( k -1)
Fp / w
< (k) <
Fp / w - F1- / 2 ,W -1,W ( k -1)
Fp / w
Hypothesis Testing
The test statistic F ( k ) for H0 : ( k ) = 0 , where 1 > 0 0 is the hypothesized
value, is
F ( k ) = Fp / w 1 - 0 .
Under the null hypothesis, the test statistic has an F-distribution with
W - 1, W ( k - 1) degrees of freedom.
Two-Way Random Effects Model: People and Measures Effects Random
Let Xji be the response to the i-th measure given by the j-th person, i = 1, , k, j =
1, , W. Suppose that Xji can be expressed as Xji = + pj + mi + pmji + eji, where pj
is the people effect which is normal distributed with zero mean and a variance of
2p , mi is the measures effect which is normal distributed with zero mean and a
variance of m2 , pmji is the interaction effect which is normal distributed with zero
mean and a variance of 2pm , and eji is the error effect which is again normal
distributed with zero mean and a variance of e2 .
Let MS BP , MS BM and MS Res be the respective between-people Mean Squares,
between-measures Mean Squares and Residual Mean Squares. Formulas for these
quantities can be found on page 479 of SPSS 7.5 Statistical Algorithms by dividing
the corresponding Sum of Squares with its degrees of freedom.
RELIABILITY 23
Type A Single Measure Intraclass Correlation
The type A single measure intraclass correlation is defined as
( A,1,r )
%K
K
=&
KK
K'
2p
2
p
if interaction effect pm ji is present
+ 2m + 2pm + 2e
2p
2
p
if interaction effect pm ji is absent
+ 2m + 2e
Estimate
The type A single measure intraclass correlation coefficient is estimated by
ICC ( A,1, r ) =
MS BP
MS BP - MS Res
.
+ ( k - 1) MS Res + k ( MS BM - MS Res ) / W
Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval
A (1- )100% confidence interval is given by
W ( MS BP - F / 2,W -1,v MS Res )
< ( A,1,r )
F / 2 ,W -1,v [ k MS BM + ( kW - k - W ) MS Res ] + W MS BP
<
W ( MS BP - F1- / 2 ,W -1,v MS Res )
F1- / 2 ,W -1,v [ k MS BM + ( kW - k - W ) MS Res ] + W MS BP
where
(aMS BM + bMS Res ) 2
(aMS )
! k -1
BM
(bMS Res ) 2
(W - 1)( k - 1)
"#
#$
24 RELIABILITY
a=
k ICC ( A,1, r )
W (1 - ICC ( A,1, r ))
and
b = 1+
k ICC ( A,1, r ) (W - 1)
.
W (1 - ICC ( A,1, r ))
Hypothesis Testing
The test statistic F ( A,1,r ) for H0 : ( A,1,r ) = 0 , where 1 > 0 0 is the
hypothesized value, is
F ( A,1,r ) =
MS BP
a0 MS BM + b0 MS Res
where
a0 =
k 0
W (1 - 0 )
and
b0 = 1 +
k 0 (W - 1)
.
W (1 - 0 )
Under the null hypothesis, the test statistic has an F-distribution with W - 1, (01)
degrees of freedom.
(01) =
(a0 MS BM + b0 MS Res ) 2
(a MS
! k -1
0
BM )
(b MS Res ) 2
+ 0
(W - 1)( k - 1)
"# .
#$
RELIABILITY 25
Type A Average Measure Intraclass Correlation
The type A average measure intraclass correlation is defined as
( A,k ,r )
%K
K
=&
KK
K'
2p
2
p
2
+ ( m
+ 2pm + 2e ) / k
2p
2
p
2
+ ( m
+ 2e ) / k
if interaction effect pm ji is present
.
if interaction effect pm ji is absent
Estimate
The type A average measure intraclass correlation coefficient is estimated by
ICC ( A, k , r ) =
MS BP
MS BP - MS Res
.
+ ( MS BM - MS Res ) / W
Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval
A (1- )100% confidence interval is given by
W ( MS BP - F / 2 ,W -1,v MS Res )
< ( A,k ,r )
F / 2 ,W -1,v ( MS BM - MS Res ) + W MS BP
<
W ( MS BP - F1- / 2,W -1,v MS Res )
F1- / 2,W -1,v ( MS BM - MS Res ) + W MS BP
where is defined as in the Type A single measure confidence interval, with
ICC ( A,1, r ) replaced by ICC ( A, k , r ) .
Hypothesis Testing
The test statistic for H0 : ( A,k ,r ) = 0 , where 1 > 0 0 is the hypothesized value,
is
26 RELIABILITY
F ( A, k ,r ) =
MS BP
c0 MS BM + d 0 MS Res
where
c0 =
0
W (1 - 0 )
and
d0 = 1 +
0 (W - 1)
.
W (1 - 0 )
Under the null hypothesis, the test statistic has an F-distribution with W - 1, (0k )
degrees of freedom.
(0k ) =
(c0 MS BM + d 0 MS Res ) 2
(c MS
! k -1
BM )
(d MS Res ) 2
+ 0
(W - 1)( k - 1)
"# .
#$
Type C Single Measure Intraclass Correlation
The type C single measure intraclass correlation is defined as
( C ,1,r )
%K
K +
=&
KK
K' +
2
p
2
pm
2
p
2
p
2
p
2
e
+ 2e
if interaction effect pm ji is present
.
if interaction effect pm ji is absent
Estimate
The type C single measure intraclass correlation coefficient is estimated by
RELIABILITY 27
ICC (C ,1, r ) =
MS BP - MS Res
.
MS BP + ( k - 1) MS Res
Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval
A (1- )100% confidence interval is given by
Fp / r - F / 2 ,W -1,(W -1)( k -1)
Fp / r - F1- / 2 ,W -1,(W -1)( k -1)
< ( C ,1,r ) <
Fp / r + ( k - 1) F / 2,W -1,(W -1)( k -1)
Fp / r + ( k - 1) F1- / 2,W -1,(W -1)( k -1)
where
Fp / r =
MS BP
.
MS Res
Hypothesis Testing
The test statistic for H0 : ( C ,1,r ) = 0 , where 1 > 0 0 is the hypothesized value,
is
F ( C ,1,r ) = Fp / r
1- 0
.
1 + ( k - 1) 0
Under the null hypothesis, the test statistic has an F-distribution with
W - 1,(W - 1)( k - 1) degrees of freedom.
Type C Average Measure Intraclass Correlation
The type C average measure intraclass correlation is defined as
28 RELIABILITY
( C , k ,r )
%K
K
=&
KK
K'
2p
2
p
+ ( 2pm + 2e ) / k
2p
2
p
+ 2e / k
if interaction effect pm ji is present
.
if interaction effect pm ji is absent
Estimate
The type C average measure intraclass correlation coefficient is estimated by
ICC (C , k , r ) =
MS BP - MS Res
.
MS BP
Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval
A (1- )100% confidence interval is given by
Fp / r - F / 2,W -1,(W -1)( k -1)
Fp / r - F1- / 2 ,W -1,(W -1)( k -1)
.
< ( C , k ,r ) <
Fp / r
Fp / r
Hypothesis Testing
The test statistic for H0 : ( C ,k ,r ) = 0 , where 1 > 0 0 is the hypothesized value,
is
F ( C ,k ,r ) = Fp / r (1 - 0 ) .
Under the null hypothesis, the test statistic has an F-distribution with
W - 1,(W - 1)( k - 1) degrees of freedom.
RELIABILITY 29
Two-Way Mixed Effects Model: People Effects Random, Measures Effects Fixed
Let Xji be the response to the i-th measure given by the j-th person, i = 1, , k, j =
1, , W. Suppose that Xji can be expressed as Xji = + pj + mi + pmji + eji, where pj
is the people effect which is normal distributed with zero mean and a variance of
2p , mi is considered as a fixed effect, pmji is the interaction effect which is normal
distributed with zero mean and a variance of 2pm , and eji is the error effect which
is again normal distributed with zero mean and a variance of e2 . Denote m2 as
the expected measure square of between measures effect mi.
Let MS BP and MS Res be the respective between-people Mean Squares and
Residual Mean Squares. Formulas for these quantities can be found on page 479 of
SPSS 7.5 Statistical Algorithms by dividing the corresponding Sum of Squares with
its degrees of freedom.
Type A Single Measure Intraclass correlation
The type A single measure Intraclass correlation is defined as
%K - / (k - 1)
K + + ( +
=&
KK
K' + +
2
p
( A,1,m)
2
p
2
p
2
pm
2
m
2
p
2
m
2
pm
2
e
2
e)
if interaction effect pm ji is present
.
if interaction effect pm ji is absent
Estimate
The type A single measure intraclass correlation is estimated by
ICC ( A,1, m) =
MS BP
MS BP - MS Res
.
+ ( k - 1) MS Res + k ( MS BM - MS Res ) / W
Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval
30 RELIABILITY
A (1- )100% confidence interval for ( A,1,m) is the same as that for ( A,1,r ) , with
ICC ( A,1, r ) replaced by ICC( A,1, m) .
Hypothesis Testing
The test statistic for H0 : ( A,1,m) = 0 , where 1 > 0 0 is the hypothesized value,
is the same as that for ( A,1,r ) , with the same distribution under the null hypothesis.
Type A Average Measure Intraclass Correlation
The type A average measure intraclass correlation is defined as
( A,k ,m)
%K
K
=&
KK
K'
2p - 2pm / ( k - 1)
2
p
if interaction effect pm ji is present
+ ( 2m + 2pm + 2e ) / k
2p
2
p
if interaction effect pm ji is absent
+ ( 2m + 2e ) / k
Estimate
The type A single measure intraclass correlation is estimated by
%KNot estimable
MS - MS
ICC ( A, k , m) = &
K' MS + ( MS - MS
BP
BP
BM
if interaction effect pm ji is present
Res
Res ) / W
if interaction effect pm ji is absent
Confidence Interval
A (1- )100% confidence interval for ( A,k ,m) is the same as that for ( A,k ,r ) ,
with ICC ( A, k , r ) replaced by ICC ( A, k , m) . Notice that the hypothesis test is not
available when the interaction effect pmji is present.
Hypothesis Testing
The test statistic for H0 : ( A,k ,m) = 0 , where 1 > 0 0 is the hypothesized value,
is the same as that for ( A,k ,r ) , with the same distribution under the null
RELIABILITY 31
hypothesis. Notice that the hypothesis test is not available when the interaction
effect pmji is present.
Type C Single Measure Intraclass Correlation
The type C single measure intraclass correlation is defined as
( C ,1,m)
%K -
K + (
=&
KK
K' +
2
p
2
p
2
pm / ( k - 1)
2
2
pm + e )
2
p
2
p
2
e
if interaction effect pm ji is present
.
if interaction effect pm ji is absent
Estimate
The type C single measure intraclass correlation is estimated by
ICC (C ,1, m) =
MS Between people - MS Residual
MS Between people + ( k - 1) MS Residual
Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval
A (1- )100% confidence interval is given by
Fp / r - F / 2 ,W -1,(W -1)( k -1)
Fp / r + ( k - 1) F / 2,W -1,(W -1)( k -1)
where
Fp / r =
Hypothesis Testing
MS BP
.
MS Res
< ( C ,1,m) <
Fp / r - F1- / 2 ,W -1,(W -1)( k -1)
Fp / r + ( k - 1) F1- / 2,W -1,(W -1)( k -1)
32 RELIABILITY
The test statistic for H0 : ( C ,1,m) = 0 , where 1 > 0 0 is the hypothesized value,
is
1- 0
.
1 + ( k - 1) 0
F ( C ,1,m) = Fp / r
Under the null hypothesis, the test statistic has an F-distribution with
W - 1,(W - 1)( k - 1) degrees of freedom.
Type C Average Measure Intraclass Correlation
The type C average measure intraclass correlation is defined as
%K - / ( k - 1)
K + ( + ) / k
=&
KK
K' + / k
2
p
( C ,k ,m)
2
pm
2
pm
2
p
2
e
2
p
2
p
2
e
if interaction effect pm ji is present
.
if interaction effect pm ji is absent
Estimate
The type C average measure intraclass correlation is estimated by
%K
&K
'
Not estimable
ICC (C, k , m) = MS BP - MS Res
MS BP
if interaction effect pm ji is present
if interaction effect pm ji is absent
Confidence Interval
A (1- )100% confidence interval is given by
Fp / r - F / 2,W -1,(W -1)( k -1)
Fp / r - F1- / 2 ,W -1,(W -1)( k -1)
.
< ( C ,k ,m) <
Fp / r
Fp / r
RELIABILITY 33
Notice that the confidence interval is not available when the interaction effect pmji
is present.
Hypothesis Testing
The test statistic for H0 : ( C ,k ,m) = 0 , where 1 > 0 0 is the hypothesized value,
is
F ( C ,1,m) = Fp / r 1 - 0 .
Under the null hypothesis, the test statistic has an F-distribution with
W - 1,(W - 1)( k - 1) degrees of freedom. Notice that the F-test is not available
when the interaction effect pmji is present.
References
Cronbach, L. J. 1951. Coefficient alpha and internal structure of tests.
Psychometrika, 16: 297334.
Dempster, A. P. 1969. Elements of Continuous Multivariate Analysis. Reading,
Mass.: Addison-Wesley.
Guttman, L. 1945. A basis for analysing test-retest reliability. Psychometrika,
10(4): 255282.
Kristof, W. 1963. The statistical theory of stepped-up reliability coefficients when
the test has been divided into several equivalent parts. Psychometrika, 28: 221
238.
Kristof, W. 1969. Estimation of true score and error variance under various
equivalent assumptions. Psychometrika, 34: 489507.
McGraw, K. O., and Wong, S. P. (1996). Forming Inferences About Some
Intraclass Correlation Coefficients. Psychological Methods, 1, p30-46.
Novick, M. R., and Lewis, C. 1967. Coefficient alpha and the reliability of
composite measurements. Psychometrika, 32: 113.
Shrout, P. E., and Fleiss, J. L. (1979). Intraclass Correlations: Uses in Assessing
Rater Reliability. Psychological Bulletin, 86, p420-428.
34 RELIABILITY
Winer, B. J. 1971. Statistical principles in experimental design, 2nd ed. New York:
McGraw-Hill.