Chapter 8:

ANALYSIS
OF
VARIANCE

Chapter 8:
The analysis of variance (ANOVA)

November 30, 2021

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 1 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

The analysis of variance (ANOVA) is a hypothesis-testing

technique used to test the claim that three or more
populations (or treatment) means are equal by examining the
variances of samples that are taken. This is an extension of the
two independent samples t-test.
ANOVA is based on comparing the variance (or variation)
between the data samples to variation within each particular
sample. If the between variation is much larger than the within
variation, the means of different samples will not be equal. If
the between and within variations are approximately have the
same size, then there will be no significant difference between
sample means.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 2 / 37

Chapter 8: In the one and two-way ANOVA models, some of the names and
ANALYSIS
OF what they refer to are listed as follows:
VARIANCE
1 Response: the dependent variable (interval or ratio scale)
2 Factor(s): the independent variable(s) (factors): nominal or
ordinal scale with more than 2 categories. In the One-way
ANOVA: one factor is involved while in the two-way ANOVA:
two factors are involved.
3 Levels: the possible values of a factor.
Treatments: another name for levels in one-way ANOVA, but
there will be a distinction between levels and treatments when
we discuss two-way ANOVA.
The term treatments derive from medicine, where the different
treatments were the drugs or procedures being tested on
patients, and agriculture, where the treatments were the
different fertilizers or pesticides being tested on crops.
4 Unit: person, animal, piece of material, etc. that is subjected
to treatment(s) and provides a response.
One-way ANOVA = one factor = one independent variable with two
or more levels/conditions
Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 3 / 37
Chapter 8:
ANALYSIS In the one-way ANOVA model (Single Factor Analysis), the
OF
VARIANCE dependent (or response) variable is quantitative, but the
independent (or factor) variable is qualitative. Conditions or
Assumptions:
1 All populations involved follow a normal distribution.
2 All populations have the same variance (or standard
deviation).
3 The samples are randomly selected and independent of
one another.
Since ANOVA assumes the populations involved follow a
normal distribution, ANOVA falls into a category of hypothesis
tests known as parametric tests. If the populations involved did
not follow a normal distribution, an ANOVA test could not be
used to examine the equality of the sample means. Instead, one
would have to use a non-parametric test (or distribution-free
test), which is a more general form of hypothesis testing that
does not rely on distributional assumptions.
Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 4 / 37
Chapter 8:
ANALYSIS
OF
VARIANCE HYPOTHESIS TEST:
The null hypothesis for a one-way ANOVA always assumes the
population means for the k samples drawn (one from each
population) are equal. Hence, we may write the null hypothesis
as:
H0 : µ 1 = µ 2 = . . . = µ k
This is equivalent to saying that the k treatments have no
differential effect upon the value of the response. Since the
null hypothesis assumes all the means are equal, we could
reject the null hypothesis if only mean is not equal. Thus, the
alternative hypothesis is:

H1 : At least one of the means different.

The ANOVA doesn’t test that one mean is less than another,
only whether they’re all equal or at least one is different.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 5 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE COMPUTING ONE WAY ANOVA:
We will assume the model

yij = µi + eij ,

where
• i = 1, . . . , k,
• j = 1, . . . , ni , and n = ki=1 ni . ni is the number of units in the
P
i−th treatment and n is the total of observations.
• yij denotes the value of the response variable in the j−th unit
and i−th treatment.
• µi denotes the parameter of the unknown population mean for
the i−th treatment.
• eij is independent random errors with N(0, σ 2 ).

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 6 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE
Treatment
1 2 ... k

y11 y21 ... yk1

..
y12 . ... yk2
.. .. .. ..
. . . .
y1n1 y2n2 ... yknk
sample Mean ȳ1. ȳ2. ... ȳk.
sample Variance S12 S22 ... Sk2

To carry out ANOVA, the mean and standard deviation for

each of the groups involved in the study must be calculated.
The analysis of variance will then tell us three things:

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 7 / 37

Chapter 8:
ANALYSIS
OF 1 Whether any of the specific groups differ from each other.
VARIANCE There are more than one possible pairs of groups when you
have more than two. This is provided by using a comparison
technique.
2 Whether the differences are relatively big or small. Measures of
explained variance will tell us this. The within-group variance is
related to sampling error, subject differences, etc. It’s like the
variance we previously discussed. The between-group variance
is due to the differences between the groups. If the means of
the groups differ significantly, then there will be an associated
between groups variance that is high.
3 Whether there are any significant differences of means among
all of the groups provided by the F-ratio. The F-ratio is similar
to the t-ratio, however, the F-ratio compares the variance
between the groups with the variance within the groups:

(variance between groups)

F =
(variance within groups)

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 8 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE
If the F-ratio is small, then groups are probably not
significantly different from each other. However, if it is big,
then some (two or more) might be significantly different from
each other. The F-table is consulted to determine if there is a
significant difference among the means.
In order to carry out the one-way ANOVA, we will do the
following steps:
1 State the null and alternative hypothesis.
2 Compute SS (sums of squares).
3 Compute degrees of freedom (df).
4 Compute MS (mean squares).
5 Compute F.
6 Take a decision.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 9 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE Example:
An instructor divided the classroom into three rows: first, second,
and third. The first in the front row, the second in the middle row
and the third in the back row. The instructor noticed that the
further the students were from him, the more likely they were to miss
class or use an instant messenger during class. A random sample of
the students in each row was taken. The score for those students on
the second exam was recorded:

At level of significance of α = 0.05, the instructor wanted to see are

the students further away did worse on the exams?

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 10 / 37

Chapter 8:
ANALYSIS Solution:
OF
VARIANCE 1. State the null and alternative hypothesis.
The null hypothesis of this example is:

H0 : µF = µS = µT

The alternative hypothesis is:

HA : At least one of the means differ.

The summary statistics for the grades of each row are shown in the
table below

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 11 / 37

Chapter 8:
ANALYSIS
OF VARIATION:
VARIANCE Variation is the sum of the squares of the deviations between a value
and the mean of the value. Sum of squares, which is another name
for variation, is abbreviated by SS and often followed by a variable in
parentheses such as SS(B) or SS(W) so we know which sum of
squares we’re talking about.
• ARE ALL OF THE VALUES IDENTICAL? No, so there is some
variation in the data. This is called the total variation and
denoted by SS(T).
• ARE ALL OF THE SAMPLE MEANS IDENTICAL? No, so
there is some variation between the groups. This is called
variation between groups. Sometimes called the variation due
to the factor and denoted by SS(B).
• ARE EACH OF THE VALUES WITHIN EACH GROUP
IDENTICAL? No, there is some variation within the groups.
This is called variation within groups. Sometimes called the
error variation and denoted by SS(W).

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 12 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

As a result, there are two sources of variation:

• The variation between the groups, SS(B), or the variation
due to the factor.
• The variation within the groups, SS(W), or the variation
that can’t be explained by the factor so it’s called the
error variation and denoted sometimes by SS(E).

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 13 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE
THE GRAND MEAN:
The grand mean is the average of all the values when the
factor is ignored. It is a weighted average of the individual
sample means.
Pk
ni ȳi.
ȳ.. = Pi=1
k
i=1 ni
n1 ȳ1. + n2 ȳ2. + . . . + nk ȳk.
ȳ.. =
n1 + n2 + . . . + nk
Grand Mean for our example is:

7(75.71) + 9(67.11) + 8(53.50)

ȳ.. = = 65.08
7+9+8

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 14 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE
2. Compute SS (sums of squares)
a. BETWEEN GROUP VARIATION, SS(B): The between
group variation is the variation between each sample mean and
the grand mean. Each individual variation is weighted by the
sample size.
k
X
SS(B) = ni (ȳi. − ȳ.. )2
i=1
= n1 (ȳ1. − ȳ.. )2 + n2 (ȳ2. − ȳ.. )2 + . . . + nk (ȳk. − ȳ.. )2

The Between Group Variation for our example is:

SS(B) = 7(75.71 − 65.08)2 + 9(67.11 − 65.08)2 + 8(53.50 −
65.08)2 = 1900.84.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 15 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

b. WITHIN GROUP VARIATION, SS(W): The Within Group

Variation is the weighted total of the individual variations. The
weighting is done with the degrees of freedom. The df for each
sample is one less than the sample size for that sample. Within
Group Variation is:
k
X
SS(W ) = dfj Sj2
j=1

= df1 S12 + df2 S22 + . . . + dfk Sk2

The within group variation for our example is:

SS(W ) = 6(310.90) + 8(119.86) + 7(80.29) = 3386.31.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 16 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

c. After filling in the sum of squares, we have:

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 17 / 37

Chapter 8:
ANALYSIS 3. Compute degrees of freedom (df)
OF
VARIANCE A degree of freedom occurs for each value that can vary before
the rest of the values are predetermined. For example, if you
had six numbers that had an average of 40, you would know
that the total had to be 240. Five of the six numbers could be
anything, but once the first five is known, the last one is fixed
so the sum is 240. The df in this case would be 6-1=5.
The Between Group df is one less than the number of groups.
We have three groups, so df(B) = 2.
The Within Group DF is the sum of the individual df’s of each
group
The sample sizes are 7, 9, and 8

df (W ) = 6 + 8 + 7 = 21

The total df is one less than the sample size

df (Total) = 24–1 = 23

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 18 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

Filling in the degrees of freedom gives this:

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 19 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE 4. VARIANCES
The variances are also called the Mean of the Squares and
abbreviated by MS, often with an accompanying variable
MS(B) or MS(W). They are an average squared deviation from
the mean and are found by dividing the variation by the
degrees of freedom
SS
MS = df

-MS(B)= 1900.84 / 2 = 950.42

-MS(W)= 3386.31 / 21 = 161.253

Notice that the MS(Total) is NOT the sum of MS(Between)

and MS(Within). This works for the sum of squares SS(Total),
but not the mean square MS(Total). The MS(Total) isn’t
usually shown.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 20 / 37

Chapter 8:
ANALYSIS
OF - Completing the MS gives
VARIANCE

Special Variances
The MS(Within) is also known as the pooled estimate of the
variance since it is a weighted average of the individual
variances. The MS(Total) is the variance of the response
variable, not technically part of ANOVA table.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 21 / 37

Chapter 8: 5. Compute F test statistic
ANALYSIS
OF An F test statistic is the ratio of two sample variances. The MS(B)
VARIANCE and MS(W) are two sample variances and that’s what we divide to
find F.
F = MS(B) / MS(W)
For our data, F = 951.055 / 161.2 = 5.898 Adding F to the table

The F test is a right tail test. The F test statistic has an F

distribution with df(B) numerator df and df(W) denominator df. The
tabulated value is F0.05,2,21 = 3.47. Since the calculated F value,
F=5.894, is greater than the tabulated F value, F0.05,2,21 = 3.47, we
reject H0 .
Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 22 / 37
Chapter 8:
ANALYSIS
OF The p-value is the area to the right of the test statistic:
VARIANCE
P(Fdf (B),df (W ) > F ) Using any statistical package, one have:
P(F2,21 > 5.9) = 0.009.
Completing the table with the p-value

The p-value is 0.009, which is less than the significance level of 0.05,
so we reject the null hypothesis. The null hypothesis is that the
means of the three rows in class were the same, but we reject that,
so at least one row has a different mean.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 23 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

There is enough evidence to support the claim that there is a

difference in the mean scores of the front, middle, and back
rows in class. The ANOVA doesn’t tell which treatment is
different, you would need to look at confidence intervals or run
post hoc tests to determine that. (out of the course)

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 24 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

In General, one-way ANOVA table is as bellow:

Source of variation Sum of squares degree of freedom Mean Square F
(Source) (SS) (df) (MS)

BETWEEN (B) SSB k-1 MSB MSB

FB = MSW (E )
Within(Error) SSW(E) n-k MSW(E)

Total SSTOT n-1

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 25 / 37

Chapter 8:
ANALYSIS
OF One way to improve the precision of the treatment
VARIANCE
comparisons is to reduce variability among the units. We can
group units into blocks so that each block contains relatively
homogeneous units. Within each block, we randomly assign
treatment to units (we do a separate random assignment for
each block). The number of units per block is a multiple of
number of the number of factor combinations. The most
common isto use each treatment onece in each block.
Example: Toxicology Assays
Treatment: 6 levels of toxin
units:24 mice
Blocks: make 4 blocks with 6 mice per block, based on weight.
Clinical trails:
Comparison of 3 treatments for controlling blood pressure
Treatment: 3 drugs units: 30 human subjects
Blocks: make 10 blocks with 3 subjects per block. Match on
weight, sex, smoking status, diet, initial blood pressure data.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 26 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

Benefits of Blocking
• Reduction in variability of estimators for treatment means.

• Improved power for F test.

• smaller values for MSE.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 27 / 37

Chapter 8: The data of Completely Randomized Designs (CRD) are arranged in
ANALYSIS
OF
two way table
VARIANCE
Treatments Blocks Treatment
1 2 ... j ... b means
1 y11 y12 ... y1j ... y1b ȳ1.
2 y21 y22 ... y2j ... y2b ȳ2.
.. .. .. .. .. .. .. ..
. . . . . . . .
i yi1 yi2 ... yij ... yib ȳi.
.. .. .. .. .. .. .. ..
. . . . . . . .
k yk1 yk2 ... ykj ... ykb ȳk.
Block means ȳ.1 ȳ.2 ... ȳ.j ... ȳ.b ȳ..

• yij = the response the response obtained by using treatment i in

block j,
• ȳi. = mean of the response for the i−th treatment level,
• ȳ.j = mean of the response for the j−th block,
• ȳ.. = mean of all kb response observations.
Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 28 / 37
Chapter 8:
ANALYSIS The analysis of variance model in CRD is based on the following:
OF
VARIANCE
yij = µij + eij , i = 1, . . . , k, j = 1, . . . , b.

where yij represents the observation of the i−th treatment in the

j−th block, µij is the mean response of the i−th treatment in the
j−th block and eij are independent random errors with N(0, σ 2 ).
The response mean for the level i of the factor is µi. = E (ȳi. ) and
the response mean for the block j is µ.j = E (ȳ.j ) and the overall
mean is µ.. = E (ȳ.. ). The hypothesis to be tested is as follows:
• for treatment effects

H0 :µ1. = µ2. = . . . = µk.

H1 :At least two of the treatment means are not equal.

• for block effects

H0 :µ.1 = µ.2 = . . . = µ.b

H1 :At least two of the block means are not equal.

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 29 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

Sum of squares quantities:

We have

SST = SSA + SSB + SSE,

where
Pk Pb
SST(TOT)= i=1 j=1 (yij − ȳ.. )2
Pk
SSA= b i=1 (ȳi. − ȳ.. )2
Pb
SSB= k j=1 (ȳ.j − ȳ.. )2
Pk Pb
SSE= i=1 j (yij − ȳi. − ȳ.j + ȳ.. )2

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 30 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

Source degree of freedom Sum of Squares

Treatment k-1 SSA
Block b-1 SSB
Residual(error) (k-1)(b-1) SSE
Total kb-1 SST(TOT)

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 31 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE The null hypothesis of no treatment effect difference

H0 : µ1. = µ2. = . . . = µk.

or

H0treat : µ1 = µ2 = . . . = µk

can be tested can be tested by using the F statistic

SSA/(k−1) MSA
F = SSE /(k−1)(b−1) = MSE ,

where SSA and SSE are the treatment and error sums of
squares.
The F test rejects H0 at level of significance α if the F statistic
value in exceeds Fα,k−1,(k−1)(b−1) (reject H0 if
F > Fα,k−1,(k−1)(b−1) )

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 32 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE
The null hypothesis of no block effect difference

H0 : µ.1 = µ.2 = . . . = µ.b

or

H0block : µ1 = µ2 = . . . = µb

can be tested can be tested by using the F statistic

SSB/(b−1) MSB
F = SSE /(k−1)(b−1) = MSE ,

where SSB and SSE are the block and error sums of squares.
The F test rejects H0 at level of significance α if the F statistic
value in exceeds Fα,b−1,(k−1)(b−1) (reject H0 if
F > Fα,b−1,(k−1)(b−1) )

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 33 / 37

Chapter 8:
ANALYSIS
OF
VARIANCE

Anova Table for Completely Randomized Block Design:

Source of variation Sum of squares degree of freedom Mean Square F
(Source) (SS) (df) (MS)

Treatments (A) SSA k-1 MSA MSA

FA = MSE
Blocks (B) SSB b-1 MSB FB = MSB
MSE
Error SSE (k-1)(b-1) MSE

Total SST kb-1

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 34 / 37

Chapter 8: Example:
ANALYSIS
OF Four methods of manufacturing penicillin were compare in a
VARIANCE
randomized block design. The block are blends of the raw
material, corn steep liquor, known to be quite variable. The
yield of each method for five blends is given below.

blend (block)
Method 1 2 3 4 5
A 89 84 81 87 79
B 88 77 87 92 81
C 97 92 87 89 80
D 94 79 85 84 88

At level α = 0.01,
1 are there significant differences between the blends
(blocks)?
2 are there significant differences between the Methods
(treatments)?
Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 35 / 37
Chapter 8:
ANALYSIS
OF
VARIANCE

blend (block)

Method 1 2 3 4 5 Method mean

A 89 84 81 87 79 ȳ1. = 84
B 88 77 87 92 81 ȳ2. = 85
C 97 92 87 89 80 ȳ3. = 89
D 94 79 85 84 88 ȳ4. = 86

blend mean ȳ.1 = 92 ȳ.2 = 83 ȳ.3 = 85 ȳ.4 = 88 ȳ.5 = 82 ȳ.. = 86

P4 P5
1
We have k = 4, b = 5, ȳ.. = 4×5 yij = 89+84+81+87+79+...+94+79+85+84+88
20
= 1720 .
Pk 2
 i=1
2
j=1
2 2 2
  20
SSA = b (ȳi. −ȳ.. ) = 5 (84−86) +(85−86) +(89−86) +(86−86) = 5 4+1+9+0 = 70.
Pi=1b
 
SSB = k (ȳ.j − ȳ.. )2 = 4 (92 − 86)2 + (83 − 86)2 + (85 − 86)2 + (88 − 86)2 + (82 − 86)2 =
 j=1 
4 36 + 9 + 1 + 4 + 16 = 264.
Pk Pb 2
P4 P5 2
SST = (yij − ȳ.. ) = (yij − ȳ.. ) =
i=1 j=1 i=1 j=1
(89 − 86)2 + (84 − 86)2 + . . . + (84 − 86)2 + (88 − 86)2 = 9 + 4 + . . . + 4 + 4 = 560
SSE = SST − SSA − SSB = 560 − 70 − 264 = 226

Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 36 / 37

Chapter 8: ANOVA of CRD table
ANALYSIS
OF
VARIANCE
(Source) (SS) (df) (MS) F
Methods (A) 70 3 23.333 1.239
blends (B) 264 4 66.000 3.504
Error 226 12 18.833
Total 560 19

1 Test the blend (blocks) effects

H0 :µ.1 = µ.2 = . . . = µ.5
H1 :at least one blend mean is different.
F = 3.504 > F0.05,4,12 = 3.26, then we reject H0 , which means
there do appear to be significant differences between blends.
2 Test the method effects
H0 :µ1. = µ2. = . . . = µ.4
H1 :at least one method mean is different.
F = 1.239 < F0.05,3,12 = 3.49, then we fail to reject H0 , which
means there is no significant differences between methods.
Chapter 8: ANALYSIS OF VARIANCE November 30, 2021 37 / 37