STAT603: Chapter 4

Basic Statistical Inference

Section 4.1: Introduction ……………………….……………………………......................... Slide 2
Section 4.2: Sampling distributions …………………………………………......................... Slide 8
Section 4.3: Confidence intervals ……………………………………………..................... Slide 18
Section 4.4: One-sample hypothesis tests ………………………………..……................. Slide 34
Section 4.5: Two-sample hypothesis tests …………………………………........................ Slide 63
Section 4.6: ANOVA Part 1 - Completely randomized designs (CRD) …..................... Slide 77
Part 2 - Randomized block designs (RBD) ……………………... Slide 117
Part 3 - Two-way ANOVA under a CRD, Multiple Comparisons,
Model assumptions …………………………..................... Slide 140
 STAT603:
Chapter 4
Basic Statistical
Inference

Section 4.1 & 4.2:

Introduction &
Sampling Distributions
4.1 Introduction to Statistical Inference
STATISTICS

Descriptive Inferential
stats stats
The collection, • Involves using samples
organisation, to draw conclusions
summarization and about a population.
presentation of data
• Express the results in
language of probability

3
 e.g. All the Population – all items
customers that with a characteristic Problems
shop at Pick n of interest (size 𝑁𝑁) • Expensive
Pay • Not practical
• Time consuming
Inferential
Generalize the statistics Census – a
findings from the study of all
sample to the e.g. 100 customers
items in the that shop at Pick n
population pop.
Pay

Study only Sample – a subset

Take a of the population
the items in sample
the sample (size 𝑛𝑛)

4
 Parameter vs Statistic

A measure of A measure of
description from a description from a
Population Sample

1) The average weight of

1) The average weight of �
30 elephants selected 𝒙𝒙
an elephant in RSA 𝝁𝝁
from the national parks
in RSA
2) The proportion of
students at UKZN that �
𝒑𝒑
𝒑𝒑 2) The proportion of 60
have a bank account
UKZN students that have
bank accounts.
Parameter vs Statistic

or 𝛽𝛽̂

6
 Drawing conclusions about a
Statistical Inference population parameter by using
a statistic calculated from the
sample.

Hypothesis Tests Estimation

(Covered in section 4.4) (Covered in section 4.3)

Point Estimate Interval Estimate

A single value that A range of values that
estimates the parameter. estimate the parameter.
Associated with some chance
that the parameter lies in this
interval.
7
 4.2 Sampling Distributions
• A sampling distribution arises when repeated samples of the same size are
drawn from a particular population (distribution) and a statistic (numerical
measure of description of sample data, e.g. a mean, variance or
proportion) is calculated for each sample.
• The interest is then focused on the probability distribution (called the
sampling distribution) of the statistic.
• Sampling distributions arise in the context of statistical inference i.e. when
statements are made about a population on the basis of random samples
drawn from it.

8
Suppose, from a population of values, we draw a whole lot of samples of size
𝑛𝑛, and calculate a statistic for each of the obtained samples.

In this section, we will explain the • A sample mean 𝑥𝑥,̅ or

idea of a sampling distribution using • A sample variance 𝑆𝑆 2
for the sample mean.

From example: Suppose we have a

population of size 4 with the sample
space S = {2, 4, 6, 8}.

If a sample of size 𝑛𝑛 = 2 is drawn with replacement (values can be

repeated), then all the possible samples are a follows…

9
 Sample: 1: {2, 2} 𝑥𝑥1̅ =2
2: {2, 4} 𝑥𝑥̅2 =3
Population values: 3: {2, 6} 𝑥𝑥̅3 =4
S = {2, 4, 6, 8} 4: {2, 8} 𝑥𝑥̅4 =5
5: {4, 2} 𝑥𝑥̅5 =3
6: {4, 4} 𝑥𝑥̅6 = 4
These are all the 7: {4, 6} 𝑥𝑥̅7 = 5 The mean of the
possible samples 8: {4, 8} 𝑥𝑥̅8 = 6 samples of size 2 has
of size 2 from the 9: {6, 2} 𝑥𝑥̅9 = 4 different values.
population of 10: {6, 4} 𝑥𝑥10
̅ =5
values above. 11: {6, 6} 𝑥𝑥11
̅ =6
12: {6, 8} 𝑥𝑥12 ̅ =7
This is the definition
13: {8, 2} 𝑥𝑥13̅ =5
of a random
14: {8, 4} 𝑥𝑥14 ̅ =6
15: {8, 6}
variable.
𝑥𝑥15 ̅ =7
16: {8, 8} 𝑥𝑥16 ̅ =8 10
 For this example, our statistic of interest was the mean where:

(The mean can take on any value from 2

𝑥𝑥̅𝑖𝑖 = {2, 3, 4, 5, 6, 7, 8} to 8 depending on which two values were
obtained in the sample.)

• In general, a statistic is always a random variable that can assume

different values depending on the sample of size n obtained.
• Any random variable has a probability distribution.
• A probability distribution of a statistic is referred to as a sampling
distribution.
• Since a statistic is a random variable with a probability distribution, a mean
and variance can be calculated for it.

11
 Consider the previous example again:
�
Let’s denote this sample statistic of interest (the mean) by 𝑋𝑋.
(A capital letter to denote it is a random variable, with a bar above it to denote a
mean.)
The sampling distribution of 𝑋𝑋� for this example will be: 16 = total no. of
possible
2 3 4 5 6 7 8 Sum
samples/outcomes
Count 1 2 3 4 3 2 1 16
1� 2� 3� 4� 3� 2� 1� 1
16 16 16 16 16 16 16

The mean and variance of this random variable 𝑋𝑋� (the sample mean)is:
5 Found using the calculator or Excel (put all 16
� =5
E(𝑋𝑋) and � =
Var(𝑋𝑋) values of 𝑋𝑋� from the previous slide into Excel to
2
find the mean and variance)

12
Recall: the population of values were 2, 4, 6 and 8. Let’s represent these
population values by the random variable 𝑋𝑋.

∴ 𝑋𝑋 = {2, 4, 6, 8}

The population mean and population variance are:

E(𝑋𝑋) = 5 = 𝜇𝜇 The mean and variance

Var(𝑋𝑋) = 5 = 𝜎𝜎 2 of the population values.

Now let’s compare the mean and variance of each random variable, 𝑋𝑋
and 𝑋𝑋� (based on a sample of size 𝑛𝑛 = 2)

� = 5 = E(𝑋𝑋) = 𝜇𝜇
E(𝑋𝑋) 5
� = =
Var( 𝑋𝑋) 𝜎𝜎 2
Var(𝑋𝑋)
2
=
𝑛𝑛 𝑛𝑛

13
In general, for a sample of size 𝑛𝑛 drawn from a population of values (X) that
has a population mean of 𝜇𝜇 and a population variance of 𝜎𝜎 2 , the mean
� will be as
and variance of the sampling distribution of the sample mean (𝑋𝑋),
follows:

� = 𝜇𝜇𝑋𝑋� = 𝜇𝜇
E(𝑋𝑋) (The population mean)
𝜎𝜎 2
� = 𝜎𝜎𝑋𝑋2� =
Var(𝑋𝑋) (The population variance divided by the sample
𝑛𝑛 size 𝑛𝑛)

σ
𝜎𝜎𝑋𝑋� = � known as the
(This is the standard deviation of 𝑋𝑋,
𝑛𝑛
standard error)

14
 The Central Limit Theorem

• In the previous slides, a formula to calculate the mean and variance of

𝑋𝑋� was specified. However, nothing is mentioned about the specific
distribution of this random variable.
• The next part of this section states a theorem that can be used to
� thus allowing us to calculate
specify the approximate distribution of 𝑋𝑋,
�
probabilities for 𝑋𝑋.
• This theorem is called the Central Limit Theorem, or abbreviated to CLT.

15
 The Central Limit Theorem (CLT):

Let 𝑋𝑋1 , 𝑋𝑋2 , … , 𝑋𝑋𝑛𝑛 be a random sample of size n drawn from a population (with
any distribution) with a population mean 𝜇𝜇 and variance 𝜎𝜎 2 .
∑𝑛𝑛𝑖𝑖=1 𝑥𝑥𝑖𝑖
Then for a sufficiently large n, the mean of the sample 𝑋𝑋� = will be
𝑛𝑛
2 𝜎𝜎 2
approximately normally distributed with a mean 𝜇𝜇𝑋𝑋� = 𝜇𝜇 and a variance 𝜎𝜎𝑋𝑋� = .
𝑛𝑛

𝜎𝜎 2
�
i.e. 𝑋𝑋~𝑁𝑁 𝜇𝜇,
𝑛𝑛

Now that we know the specific distribution of this random variable, we can
use it to calculate probabilities. Specifically, since it follows a normal
distribution, we can standardize it in order to use the standard normal table.

16
 Key points on the CLT:
� will be approximately normally distributed if the
• Normality: The sample means (𝑋𝑋)
sample size is large enough, even if the population distribution is not normal.
• Sample size: Generally, a sample size of 30 or more is considered sufficient for the
CLT to hold.
• Statistical inference: The basis of many statistical inference methods (hypothesis
tests, confidence intervals, statistical models) is formed from the normal distribution.
Hence, such methods require normality.
• When the assumption of normality is not met, traditional parametric statistical
methods will not be accurate. However, there are other methods that can be
considered in this case, such as non-parametric methods.
• Many machine learning methods also do not require the assumption of normality
to be met.
17
 STAT603:
Chapter 4
Basic Statistical
Inference

Section 4.3:
Confidence Intervals
 4.3 Confidence Intervals
Some terminology:
• A point estimate of a parameter is a single value (point) that estimates a
parameter.

• An interval estimate of a parameter is a range of values from L (lower value)

to U (upper value) that estimate a parameter. Associated with this range of
values is a percentage of confidence that the range of values will contain
the parameter that is being estimated.

Example:
Suppose the mean time it takes to serve customers at a supermarket
checkout counter is to be estimated.
19
1) The mean service time of 100 customers of (say) 𝑥𝑥̅ = 2.283 minutes is an
example of a point estimate of the parameter µ.
2) If it is stated that we are 95% confident that the mean service time will be
from 1.637 minutes to 4.009 minutes, the interval of values (1.637, 4.009) is
an interval estimate of the parameter μ.

Since, from section 4.2, we have seen that a point estimate (a statistic) can
differ each time depending on the sample obtained, it is more appropriate
(and useful) to obtain an interval estimate for a parameter rather than just
one single value.

Therefore, in this section, we will go through an overview of interval estimates,

known as confidence intervals (for the purpose of this module, none of the
theory will be shown).
20
 Overview of the form of a confidence interval
A confidence interval (CI) is typically expressed in the form:

Estimate ± Margin of Error

Where:
• Estimate: This is the sample statistic (point estimate) that serves as the best guess
for the population parameter.
• Margin of Error: This accounts for the uncertainty in the estimate and is calculated
as: Margin of Error = Critical Value × Standard Error
− The critical value is based on the desired confidence level (%) and comes
from a probability distribution (e.g. the normal distribution).
− The standard error measures the variability of a sample statistic, it is the
standard deviation of the estimate and is impacted by the sample size.
21
 So what do confidence intervals tell us?
• A confidence interval is a range of values (from a lower limit to an upper
limit), derived from sample data, that is likely to contain the true value of
an unknown population parameter (such as a mean, proportion, or
difference between means) with a certain level of confidence.
• We typically see 95% confidence intervals reported but what does this
mean?
• E.g. If we calculate a 95% confidence interval for a population mean
based on a sample taken from the population of interest, we can be 95%
confident that our interval contains the population mean.
• It is how sure, or confident, we want to be that our interval contains the
true population mean.
22
 So what do confidence intervals tell us?
• A 95% confidence interval means that if we were to take all possible
samples of the same size (𝑛𝑛) from the same population and calculate the
confidence interval using each sample, we would expect that 95% of these
intervals would contain the true value of the population parameter and 5%
of the intervals won’t contain the true value of the population parameter.
• However, we will never know whether the confidence interval we have
obtained is one of those 5%.
• If we want to be highly confident our interval captures the population
mean we could calculate a 99% confidence interval. We would then be
99% confident that our population mean was captured within our
confidence interval.

23
 So what do confidence intervals tell us?
• Not a Probability Statement: It's important to note that the 95% confidence
level does not mean there is a 95% chance that the true value of the
population parameter is within the interval for this particular sample. The
true value is either in the interval or it is not. The confidence level reflects
the long-term reliability of the method used to construct the interval.

Why are confidence intervals useful?

1. Quantifying Uncertainty:
Confidence intervals provide a range of values that reflect the uncertainty
around a sample estimate. Instead of offering a single point estimate, they give
a sense of the variability and precision of the estimate, helping to understand
the reliability of the result.
24
 Why are confidence intervals useful…

2. Making Inferences about Population Parameters:

They allow researchers to infer the possible range of a population parameter
based on sample data. This is especially valuable when it is impractical or
impossible to measure the entire population.

3. Assessing Statistical Significance:

Confidence intervals can be used to assess statistical significance. For
example, if a confidence interval for the difference between two population
means does not include zero, it suggests that there is a statistically significant
difference between the groups at the given confidence level.

25
4. Robustness to Sample Size:
Confidence intervals naturally adjust for sample size. Larger samples lead to
narrower intervals (more precise estimates), while smaller samples result in wider
intervals, reflecting greater uncertainty.
5. Guiding Decision-Making:
Confidence intervals provide valuable information for decision-makers, helping
them assess the range within which the true effect or parameter might lie, and
thereby make more informed choices based on the degree of uncertainty.
6. Alternative to Hypothesis Testing:
While traditional hypothesis tests give a binary decision (reject or fail to reject a
null hypothesis – more about this next), confidence intervals offer more
nuanced information. They not only tell us whether an effect is significant but
also provide insight into the magnitude and direction of the effect.
26
 Which confidence interval is better?

Out of the two 95% confidence intervals below for the average time taken for
a customer to be served at a supermarket, which would you think is more
informative?
(1.283 ; 3.283) vs (0.421 ; 8.765)

This CI is more precise. It is

narrower and therefore is more
informative about the possible
value of the mean.

In general, the narrower the CI, the better it is.

27
 Example
Suppose the 95% confidence interval for the average height of a man is
given by:
95% CI: [1.76m, 1.87m]

99% CI: [1.72m, 1.91m] 90% CI: [1.79m, 1.84m]

More confident, wider interval Less confident, narrower interval

28
 What affects the width of a confidence interval?
𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 ± 𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸

The width of the CI is affected by the error. The smaller the error, the better!
The error is made up of
• The critical value, which is dependent on the confidence level (90%, 95%, 99%) –
we can’t decrease this too much as we then will be less confident in the CI. So this
cannot change by much to obtain a narrower CI.
• The standard error which is made up of
o the standard deviation of the population. This is a fixed constant – cannot
change it.
o 𝑛𝑛, the sample size – we can change this (as much as resources allow).
As 𝑛𝑛 increases, the standard error decreases which in turn decreases the error and the
CI becomes narrower (this is why we prefer to obtain as large a sample as possible).
29
 Example
Consider the two confidence intervals below for the average height of a
man, which have each been calculated using the same level of
confidence (95%):

Which one was based on a larger sample size (more men)?

Wider interval: Calculated based on a

95% CI: [1.76m, 1.87m] smaller sample size.

95% CI: [1.79m, 1.84m] Narrower interval: Calculated based on a

much larger sample size.

30
 Example
Suppose a 95% confidence interval for the difference in the average height
of a man and woman is given by:

95% CI: [12cm, 16cm]

• We can conclude, with 95% confidence that there is a statistically

significant difference between the average height of a man and a
woman, since the 95% CI does not contain zero.
• Zero is the reference point because if their difference is zero, then they are
equal.

31
 Example
Suppose a 95% confidence interval for the difference in the average IQ of a
man and woman is given by:

95% CI: [-5, 5]

We CANNOT conclude that there is a statistically significant difference

between the average IQ of a man and a woman, since the 95% CI contains
zero i.e. zero difference.

32
 Other use-cases of confidence intervals

• Drug effectiveness: Assessing the difference in white blood cell count

among two groups of patients being treated with different drugs.
• Stock market analysis: Assessing the difference in average monthly returns
on two different stocks or investments.
• Customer Satisfaction Surveys: A company surveys customers to find hat
% are satisfied with their service. A confidence interval (e.g., 68%-72%)
helps estimate the true satisfaction level in the entire customer base.
• Quality Control: A factory tests a sample of products for defects. A
confidence interval helps determine the defect rate in all products.

33
 STAT603:
Chapter 4
Basic Statistical
Inference

Section 4.4:
One-sample
Hypothesis Tests
 4.4 One-sample Hypothesis Tests
Introduction and Terminology
• We now know what a confidence interval is and how it can aid us in
answering research questions.
• However, no information/knowledge is known about the possible value of
the parameter before the confidence interval is constructed.

Now we look at methods for carrying out hypothesis tests concerning claims
made about the possible value (or values) of the population parameter.
• Therefore, in these sections, some knowledge about the specific value (or
range of values) of the parameter is known/assumed.

35
• Hypothesis testing is a fundamental concept in statistics that allows us to
make decisions or inferences about a population based on sample data.
• It is widely used in research, business, medicine, and many other fields to
test assumptions or claims about population parameters (such as means,
proportions, or variances).

Statistical Hypothesis:
• A statistical hypothesis is an assertion (claim) made about the value(s) of a
population parameter.

36
Purpose of a Statistical Hypothesis:
• The purpose of testing of hypotheses is to determine whether a claim that is
made could be true.
• The conclusion about the truth of such a claim is not stated with absolute
certainty, but rather in terms of the language of probability.

Examples of claims to be tested:

1) A supermarket receives complaints that the mean content of “1 kilogram”
sugar bags that are sold by them is less than 1 kilogram.
2) An electrical firm manufacturing light bulbs claims that the average lifetime
(in hours) of their light bulbs is more than 780 hours.
3) A construction company believes that the average compressive strength
of its concrete is at the required level of 4000psi.
37
Null and Alternate Hypotheses:
Null hypothesis:
A statement concerning the exact value of the population parameter of
interest (𝜃𝜃) from the claim that is made.

The general notation of representing the null hypothesis:

𝐻𝐻0 : 𝜃𝜃 = 𝜃𝜃0 (𝜃𝜃 = 𝜇𝜇, 𝜎𝜎 2 𝑜𝑜𝑜𝑜 𝑝𝑝 and 𝜃𝜃0 is the specified value made in the claim)

Alternative hypothesis:
A statement concerning the possible range of values of the population
parameter 𝜃𝜃 that is believed to be true if 𝐻𝐻0 is not true.

The general notation of representing an alternative hypothesis:

𝐻𝐻1𝑎𝑎 : 𝜃𝜃 < 𝜃𝜃0 𝐻𝐻1𝑏𝑏 : 𝜃𝜃 > 𝜃𝜃0 𝐻𝐻1𝑐𝑐 : 𝜃𝜃 ≠ 𝜃𝜃0
(Only one alternative hypothesis is tested at a time)
38
Examples:
Consider the examples of the claims from slide 3:

1) A supermarket receives complaints that the mean content of “1 kilogram”

sugar bags that are sold by them is less than 1 kilogram.

The parameter of interest is the mean ∴ 𝜃𝜃 = 𝜇𝜇

𝐻𝐻0 : 𝜇𝜇 = 1 The specified value in the claim is 𝜇𝜇0 = 1

𝐻𝐻1 : 𝜇𝜇 < 1 This is the claim

For this example, the alternative hypothesis 𝐻𝐻1 represents the claim that is
being made (the mean is less than 1).

However, the null hypothesis still specifies that the mean is exactly equal to 1.
39
2) An electrical firm manufacturing light bulbs claims that the average lifetime
(in hours) of their light bulbs is more than 780 hours.

The parameter of interest is the mean ∴ 𝜃𝜃 = 𝜇𝜇

𝐻𝐻0 : 𝜇𝜇 = 780 The specified value in the claim is 𝜇𝜇 = 780

𝐻𝐻1 : 𝜇𝜇 > 780 The average lifetime is more than 780

For this example, the alternative hypothesis 𝐻𝐻1 once again represents the
claim that is being made.

Either of the hypotheses (null or alternative) can represent the claim to be

tested.

40
3) A construction company believes that the average compressive strength
of its concrete is at the required level of 4000psi.

The parameter of interest is the mean ∴ 𝜃𝜃 = 𝜇𝜇

𝐻𝐻0 : 𝜇𝜇 = 4000 The specified value in the claim is 𝜇𝜇0 = 4000

𝐻𝐻1 : 𝜇𝜇 ≠ 4000
In this example, the claim is being made about the exact value of the
parameter.
Therefore, the claim is represented by the null hypothesis 𝐻𝐻0 (that the mean is
exactly 4000).
If the claim is made about the exact value of the parameter, then the
alternative hypothesis will always be that the parameter is NOT equal to the
specified value. i.e. 𝐻𝐻1𝑐𝑐 : 𝜃𝜃 ≠ 𝜃𝜃0
41
One-sided Alternative:
This is a hypothesis that specifies the alternative values in a direction that is
either less than or greater than that specified by the null hypothesis.

i.e. the following alternative hypotheses result in a one-sided hypothesis test:

• This alternative hypothesis results in a left-tailed test.

𝐻𝐻1𝑎𝑎 : 𝜃𝜃 < 𝜃𝜃0 • This is because we need to prove the parameter 𝜃𝜃 is
less than 𝜃𝜃0 (i.e. it lies to the LEFT of 𝜃𝜃0 ).

• This alternative hypothesis results in a right-tailed test.

𝐻𝐻1𝑏𝑏 : 𝜃𝜃 > 𝜃𝜃0 • This is because we need to prove the parameter 𝜃𝜃 is
more than 𝜃𝜃0 (i.e. it lies to the RIGHT of 𝜃𝜃0 ).

42
Two-sided Alternative:

This is a hypothesis that specifies the alternative values (to the null hypothesis)
in directions that can be either less than or more than that specified by the
null hypothesis. (No specific direction is given)

i.e. the following alternative hypothesis results in a two-sided hypothesis test:

• This alternative hypothesis results in a two-tailed test.

𝐻𝐻1𝑎𝑎 : 𝜃𝜃 ≠ 𝜃𝜃0 • This is because we need to prove the parameter 𝜃𝜃 is
less than OR greater than 𝜃𝜃0 in order to prove it is not
equal to it (i.e. it lies to the LEFT or the RIGHT of 𝜃𝜃0 ).

43
Due to the results of a hypothesis test being based on a sample (which may
not be representative of the population, i.e. it may be bias) there is always
some probability associated with making an error/incorrect decision.

Types of Errors:

Type I Error

A type I error is made when the null hypothesis is rejected when in reality it is
actually true.

Type II Error

A type II error is made when the null hypothesis NOT is rejected when in reality
it is actually false.
44
H0: Not pregnant

45
 Table summarizing the errors:
Conclusion of hypothesis test

True situation\
𝑯𝑯𝟎𝟎 not rejected 𝑯𝑯𝟎𝟎 rejected
Conclusion

𝐇𝐇𝟎𝟎 true Correct decision Type I Error

Reality
𝐇𝐇𝟎𝟎 not true Type II Error Correct decision

P(type I error) = P(Reject 𝐻𝐻𝑜𝑜 |𝐻𝐻𝑜𝑜 is true) = 𝛼𝛼

P(type II error) = P(Do not reject 𝐻𝐻𝑜𝑜 |𝐻𝐻𝑜𝑜 is false) = 𝛽𝛽

These probabilities work opposite to each other. As 𝛼𝛼 increases, 𝛽𝛽

decreases…
46
• Therefore, there is no way to set the probability of making one type of
error equal to 0 without making the probability of making the other type
of error vey large!
• This is why the results of a hypothesis test are stated in terms of a language
of probability (due to there being some probability associated with
making an error).

• Although we want to keep both probabilities 𝛼𝛼 and 𝛽𝛽 as small as possible,

we cannot keep one small without the other getting very large.
• For this reason, we only focus on 𝛼𝛼, as a type I error is considered to be
more serious.
• Therefore, a hypothesis test is performed according to a prespecified
value of 𝛼𝛼.

47
 Procedure of Hypothesis Testing
• Hypothesis tests are carried out by using a set level of significance (𝛼𝛼) and
by assuming the null hypothesis is true.
• We then need to use the collected sample to try and prove it is impossible
for 𝐻𝐻0 to be true (in other words, we need to see if we have enough
evidence to prove that 𝐻𝐻0 is false).
• Thus, the conclusions for the hypothesis test are expressed as follows:
• When 𝐻𝐻0 is rejected: We have sufficient evidence to conclude 𝐻𝐻1 (the
alternative hypothesis).
• When 𝐻𝐻0 is not rejected: We do NOT have sufficient evidence to
conclude 𝐻𝐻1 , however, we do NOT accept 𝐻𝐻0 .
The absence of evidence is not evidence of absence

48
Critical value approach:
• A test statistic is calculated from the sample data and represents how far
the observed data from the sample diverges from the null hypothesis (𝐻𝐻0 ).
• A large test statistic means the sample data is far from what is expected
under 𝐻𝐻0 , making it more likely to reject 𝐻𝐻0 .
• A small test statistic means the data is close to what is expected under 𝐻𝐻0 ,
making it less likely to reject 𝐻𝐻0 .
• But how do we know when the test statistic is large or small?
It is compared to a critical value
• A critical value is a threshold derived from the theoretical distribution
(standard normal distribution, t-distribution, Chi-square etc.) of the test
statistic under the assumption that the null hypothesis is true.

49
Critical value approach…
• The critical value corresponds to the chosen significance level (𝛼𝛼 – usually
1%, 5% or 10%) and marks the boundary of the rejection region.
• If the test statistic lies in the rejection region, the null hypothesis is rejected,
suggesting that the alternative hypothesis (𝐻𝐻1 ) is more plausible.
• Conversely, if the test statistic lies outside of this rejection region (as defined
by the critical value), we fail to reject 𝐻𝐻0 , indicating insufficient evidence
against it.
• If the test statistic falls far enough away from the expected value under 𝐻𝐻0
(i.e., it exceeds the critical value), it indicates that the observed data is
unlikely to have occurred by random chance under the null hypothesis.

50
 Types of hypothesis tests for population means
One-sample tests: 𝐻𝐻0 : 𝜇𝜇 = 𝜇𝜇0
• Purpose: Tests whether the mean of a single sample differs from a known or
hypothesized population mean.
• Example: Testing if the average height of a group of students is different from
the national average height.

Two-sample tests (Independent samples): 𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 𝐻𝐻0 : 𝜇𝜇1 − 𝜇𝜇2 = 0
• Purpose: Compares the means of two independent groups to see if they are
significantly different from each other. (more on this in section 4.5)
• Example: A retail company wants to compare the average sales between
two of its store locations to determine if one store is outperforming the other.

51
Paired t-Test: 𝐻𝐻0 : 𝜇𝜇𝑑𝑑 = 0
• Purpose: Compares the means of two related groups (e.g., before-and-after
measurements) to see if there is a significant change.
• Example: Testing the effect of a training program by comparing the test
scores of participants before and after the program.

ANOVA (Analysis of Variance): 𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 = ⋯ = 𝜇𝜇𝑘𝑘

𝐻𝐻1 : At least two 𝜇𝜇𝑖𝑖′ 𝑠𝑠 differ
• Purpose: Tests whether there are significant differences between the means
of three or more independent groups. (more on this in section 4.6)
• Example: A marketing team wants to compare the effectiveness of three
different advertising campaigns (TV, online, and print) on product sales.

52
P-value approach:
• This method involves comparing a p-value to the specified level of
significance (𝛼𝛼).
• A p-value is the probability, when assuming the null hypothesis is true, of
obtaining by chance a result as or more extreme than that observed in the
sample.
For example, if we want to test:
𝐻𝐻0 : 𝜇𝜇 = 15
𝐻𝐻1 : 𝜇𝜇 < 15

53
 𝐻𝐻0 : 𝜇𝜇 = 15 𝑣𝑣𝑣𝑣 𝐻𝐻1 : 𝜇𝜇 < 15

Suppose we have obtained a sample such that the sample mean is 𝑥𝑥̅ = 8. We
know there are two possibilities:

1. The sample we obtained could, by chance, have consisted of lower

values to obtain a lower sample mean.
2. All the values in the population could be lower therefore resulting in a
lower population mean, and thus 𝜇𝜇 should be less than 15.

• A p-value is the probability of outcome 1 above, under the assumption

𝐻𝐻0 : 𝜇𝜇 = 15 is true.
• Therefore, it is a measure for the chance of obtaining 𝑥𝑥̅ = 8 or something
smaller/more extreme.

54
• For small P-values, we can conclude there is a small chance of outcome 1
occurring if the null hypothesis is true (i.e. a small chance of getting such a
sample if the null hypothesis is true, and therefore is cannot possibly be true),
and thus we can reject the null hypothesis.

• How small must it be to reject? We compare it to the level of significance (α):

True for any
 If P-value < α, reject 𝐻𝐻0 (this is referred to as a significant P-value) hypothesis
test ever
 If P-value > α, do not reject 𝐻𝐻0 carried out!

55
• The calculation of the P-value depends on the distribution of the test
statistic as well as the alternative hypothesis.
• More specifically, it depends on whether the alternative is a one-sided or
two-sided alternative.
• The P-value for a two-sided alternative is always double that of a one-sided
test.

56
 One-sample tests for population means
Z-Test for One Sample Mean:
When
• The population standard deviation (𝜎𝜎) is known.
• The sample size is large (𝑛𝑛 ≥ 30) OR the population is normally distributed.
The test statistic follows a standard normal (Z) distribution. Therefore, the critical
value(s) and p-values are based on this distribution. This is known as a Z-test.

t-Test for One Sample Mean:

When
• The population standard deviation (𝜎𝜎) is unknown, use the sample estimate 𝑆𝑆.
• The sample size is small (𝑛𝑛 < 30) but the population is normally distributed.
The test statistic follows a t-distribution. Therefore, the critical value(s) and p-values are
based on this distribution. This is known as a t-test.
57
 Power of Test
• The power of a hypothesis test, often referred to as statistical power, is the
probability that the test correctly rejects a false null hypothesis (𝐻𝐻0 ).
• It measures the test's ability to detect an effect or difference when one truly
exists. The power ranges from 0 to 1, with a higher value indicating a greater
ability of the test to identify true positives.
Several factors influence the power of a hypothesis test:
1. Effect Size: The larger the true effect or difference between groups, the
more powerful the test will be in detecting that difference.
2. Sample Size: Larger sample sizes generally increase the test's power
because they provide more information about the population, reducing the
effect of random variation.

58
3. Significance Level ( 𝛼𝛼 ): The threshold for rejecting the null hypothesis.
Lowering 𝛼𝛼 (making the test more stringent in declaring significance)
reduces power, whereas increasing 𝛼𝛼 (accepting more risk of a Type I error)
increases power.
4. Variability in the Data: Less variability (noise) in the data increases the
power, as it makes true effects more discernible against the background of
random fluctuations.
Optimally, researchers aim for high power (typically 0.8 or 80%) to be confident
in their ability to detect meaningful effects, recognizing that increasing power
often involves trade-offs, such as needing larger sample sizes or accepting a
higher risk of Type I errors.

59
• It's important to note that while large sample sizes can make it easier to find
statistically significant results, this does not necessarily mean those results
are practically significant or meaningful.
• Very small differences may not be of practical importance, even if they are
statistically significant.
• This highlights the importance of considering both statistical significance (p-
value) and effect size (the magnitude of the difference or relationship)
when interpreting research findings.

60
 Significance of Test
• Recall: Statistical significance is a mathematical concept that measures the
probability that an observed difference or relationship between variables in
a study is due to chance.
• It's usually assessed using a p-value or other statistical tests. A result is typically
considered statistically significant if the p-value is below a predetermined
threshold, commonly 0.05. This means there is less than a 5% probability that
the observed results occurred by chance, assuming the null hypothesis (no
effect or no difference) is true.
• When a result is significant statistically, practical or biological significance
should also be considered.

61
• Biological significance refers to the practical importance or relevance of the
findings in a real-world biological context.
• It considers whether the magnitude of the effect observed in a study is large
enough to have a noticeable impact on organisms, populations, or
ecosystems.
• In clinical research, the term "clinical significance" is used to denote whether
the difference or effect observed has a meaningful impact on patient
health, treatment outcomes, or medical practice.
• While statistical significance helps to understand the reliability of the findings
from a mathematical perspective, biological significance addresses the
practical importance of these findings in the real world. Both are crucial for
interpreting research results and making informed decisions based on
scientific evidence.
62
 STAT603:
Chapter 4
Basic Statistical
Inference

Section 4.5:
Two-sample
Hypothesis Tests
 • Discussed hypothesis tests concerning the
Section 4.4: parameter 𝜇𝜇 (population mean) of a single
population.
• These hypothesis tests were based on one
random sample drawn from a single population
of interest.

• Discusses hypothesis tests for comparing two

population means of two different populations.
This section:
• These hypothesis tests are based on two random
samples, each drawn from one of the populations
of interest.

For example: Are the mean salaries the same for males and females with the
same educational qualifications and work experience?
64
 4.5 Two-sample Hypothesis Tests
Theses hypotheses
Null Hypothesis: can be represented
𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 𝐻𝐻0 : 𝜇𝜇1 − 𝜇𝜇2 = 0 in the general form
of 𝐻𝐻0 : 𝜃𝜃 = 𝜃𝜃0 …
Alternative Hypotheses:
𝐻𝐻1𝑎𝑎 : 𝜇𝜇1 < 𝜇𝜇2 𝐻𝐻1𝑎𝑎 : 𝜇𝜇1 − 𝜇𝜇2 < 0 (left-tailed)

𝐻𝐻1𝑏𝑏 : 𝜇𝜇1 > 𝜇𝜇2 𝐻𝐻1𝑏𝑏 : 𝜇𝜇1 − 𝜇𝜇2 > 0 (right-tailed)

𝐻𝐻1𝑐𝑐 : 𝜇𝜇1 ≠ 𝜇𝜇2 𝐻𝐻1𝑐𝑐 : 𝜇𝜇1 − 𝜇𝜇2 ≠ 0 (two-tailed)

parameter specified
𝜃𝜃 = 𝜇𝜇1 − 𝜇𝜇2 value 𝜃𝜃0 = 0
65
 There are a four cases when testing 𝑯𝑯𝟎𝟎 : 𝝁𝝁𝟏𝟏 − 𝝁𝝁𝟐𝟐 = 𝟎𝟎:

Population variances Population variances

𝜎𝜎12 and 𝜎𝜎22 are known 𝜎𝜎12 and 𝜎𝜎22 are unknown

𝑍𝑍-test Case A
𝑛𝑛1 AND 𝑛𝑛2 ≥ 30 𝑛𝑛1 and/or 𝑛𝑛2 < 30

Special case of Case A

This bit of information may be
given/assumed, or a
𝜎𝜎12 = 𝜎𝜎22 and 𝜎𝜎12 ≠ 𝜎𝜎22 and
hypothesis test can be carried unknown unknown
out to determine this.
Case B: Both 𝑡𝑡-tests Case C:
66
 Case D: Test for Difference Between Population
Means for Paired (dependent) samples

The cases on the previous slide assume that 𝜇𝜇1 and 𝜇𝜇2 are independent.
The last case that will be considered (Case D) is when 𝜇𝜇1 and 𝜇𝜇2 are NOT
independent.
Examples
A group of patients going on a diet is weighed before going on the diet and
again after having been on the diet for one month. A test to determine
whether the diet has reduced their weight is to be performed.
The aptitudes of boys and girls for mathematics are to be compared. In order
to eliminate the effect of social factors, pairs of brothers and sisters are used
in the comparison. Each (brother, sister) pair is given the same test and the
mean marks of boys and girls compared.
67
 In each of the examples on the previous slide, the two samples cannot be
regarded as independent.
 In the first example, two readings (before and after readings) are
made on the same subject.
 In the second example the two samples are matched via a common
factor (family connection).
 The development of the test statistics in the previous cases were based on
the fact that the two populations being studied are independent.
 When this independency is violated, the same statistical theory that is used
to develop the test statistics cannot be used.
 One way to overcome the violation in the case of studying
paired/dependent populations, is to ‘transform’ the two data sets (two
samples collected) into one.
68
 This is done by finding the difference of each paired observation.
 This ‘difference’ then becomes the new data set, which has been reduced
to one sample.

The general layout of the data for paired samples is as follows:

Sample 1 𝑥𝑥1 𝑥𝑥2 … 𝑥𝑥𝑛𝑛
Sample 2 𝑦𝑦1 𝑦𝑦2 … 𝑦𝑦𝑛𝑛
Difference 𝑑𝑑1 = 𝑥𝑥1 − 𝑦𝑦1 𝑑𝑑2 = 𝑥𝑥2 − 𝑦𝑦2 … 𝑑𝑑𝑛𝑛 = 𝑥𝑥𝑛𝑛 − 𝑦𝑦𝑛𝑛

 Using this new data set (of size 𝑛𝑛) made up of the differences (𝑑𝑑𝑖𝑖 ’s) the
mean and the standard deviation of this sample can be calculated,
denoted by 𝑑𝑑̅ and 𝑆𝑆𝑑𝑑 , respectively.

69
 The mean of the paired differences of the (𝑥𝑥, 𝑦𝑦) values of the two dependent
populations is defined as 𝜇𝜇𝑑𝑑 = 𝜇𝜇1 − 𝜇𝜇2 .

 Therefore, testing 𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 for dependent populations is equivalent to

testing:
Note: We can perform this test for a specific difference:
𝐻𝐻0 : 𝜇𝜇𝑑𝑑 = 0 𝐻𝐻0 : 𝜇𝜇𝑑𝑑 = 𝑑𝑑0 rather than just a difference of 0.

 Under the assumption that the differences are sampled from a normal
population, hypotheses concerning the mean of the differences 𝜇𝜇𝑑𝑑 can be
tested by performing a one sample t-test.

 The hypothesis tests for all 4 cases are performed in the same manner as
that for a one-sample test ⇒ compare a test statistic
⇒ compare a P-value to 𝛼𝛼 (level of significance)
70
 Confidence interval for 𝜇𝜇1 − 𝜇𝜇2

Recall from section 4.3: the general form of a confidence interval for the mean
(𝜇𝜇) is:
Point estimate ± error
Made up of a 𝑍𝑍 -value or
𝑡𝑡 -value (“critical value”)
multiplied by the standard
error of the point estimate.
• A confidence interval for 𝜇𝜇1 − 𝜇𝜇2 takes on the same form.
• The value of the standard error and whether a 𝑍𝑍 or 𝑡𝑡-value is used is based
on which case is being considered (case A, B, C or D).

71
 Interpretation of a confidence interval for 𝜇𝜇1 − 𝜇𝜇2

The confidence interval represents a possible range for 𝜇𝜇1 − 𝜇𝜇2 .

• If the range is positive, this implies that 𝜇𝜇1 > 𝜇𝜇2 , and the confidence interval
gives a possible range of how much bigger 𝜇𝜇1 is to 𝜇𝜇2 .
• If the range is negative, this implies that 𝜇𝜇1 < 𝜇𝜇2 , and the confidence interval
gives a possible range of how much smaller 𝜇𝜇1 is to 𝜇𝜇2 .
• If the confidence interval contains 0, then that indicates that 𝜇𝜇1 and 𝜇𝜇2 could
possibly be equal (in more formal terms: there is no significant difference
between them).
• Performing a hypothesis test for 𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 vs 𝐻𝐻1 : 𝜇𝜇1 ≠ 𝜇𝜇2 at 𝛼𝛼 % level of
significance gives the same conclusions concerning the significance of the
difference as what can be concluded from a 1 − 𝛼𝛼 100% confidence
interval.
72
 Testing the Equality of Two Population Variances
Recall:
• When testing 𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 , there are various cases depending on the
available information.
• When the population variances 𝜎𝜎12 and 𝜎𝜎22 are unknown AND the sample
sizes are SMALL (𝑛𝑛1 and/or 𝑛𝑛2 < 30), there are two cases B and C to
choose from (both t-tests):
If the population variances If the population variances
are equal: i.e. 𝜎𝜎12 = 𝜎𝜎22 are unequal: i.e. 𝜎𝜎12 ≠ 𝜎𝜎22

Case B Case C
• Question: But how do we know if 𝜎𝜎12 = 𝜎𝜎22 ?
• Answer: a) We either know from past studies or research.
b) Or we can perform a hypothesis test to determine if 𝜎𝜎12 = 𝜎𝜎22 .
73
Therefore, this section presents methods for carrying out a hypothesis test to
determine if two population variances are equal.

Null Hypothesis:
𝐻𝐻0 : 𝜎𝜎12 = 𝜎𝜎22

Alternative Hypothesis:
𝐻𝐻1 : 𝜎𝜎12 ≠ 𝜎𝜎22 (two-tailed test)

Note: We can test the other one-sided alternatives 𝐻𝐻1 : 𝜎𝜎12 < 𝜎𝜎22 or 𝐻𝐻1 : 𝜎𝜎12 > 𝜎𝜎22 ,
however for this course, we are only interested in the testing of population
means. Thus, we only use this test to determine if we use case B or case C to
test the difference in two population means. Therefore, we only use the two-
sided alternative above.
74
Recall: The general form of representing the null hypothesis is 𝐻𝐻0 : 𝜃𝜃 = 𝜃𝜃0 …
𝜎𝜎12
𝐻𝐻0 : 𝜎𝜎12 = 𝜎𝜎22 𝐻𝐻0 : 2 = 1
𝜎𝜎2

specified value 𝜃𝜃0 = 1

𝜎𝜎12
𝐻𝐻1 : 𝜎𝜎12 ≠ 𝜎𝜎22 𝐻𝐻1 : 2 ≠1
𝜎𝜎2
𝜎𝜎12
parameter 𝜃𝜃 = 2
𝜎𝜎2
• Due to the properties of variances, and in order to formulate a test statistic, we use
the ratio of 𝜎𝜎12 and 𝜎𝜎22 (instead of using their difference, as in the case with means).
• This test statistic now follows a new distribution called the F-distribution.
• The test is performed in the usual manner of either comparing the test statistic to a critical
value(s), or comparing the P-value to 𝛼𝛼 (𝐻𝐻0 is rejected if P-value < 𝛼𝛼).

75
 Important:
• N.B. the purpose of this test is to determine which case (B or C) is used when
testing 𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 where the population variances are unknown.
• Statistical software (such as SAS) automatically produces the results of both
cases as well as that for the test of equality of population variances.
• Based on the test of equal variances, we then consider the results of case B
or case C accordingly in order to test 𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 :

Use case B to carry out the test for

If 𝐻𝐻0 : 𝜎𝜎12 = 𝜎𝜎22 is NOT rejected
𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 - the pooled variance t-test

Use case C to carry out the test for

If 𝐻𝐻0 : 𝜎𝜎12 = 𝜎𝜎22 is rejected 𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 - the Satterthwaite t-test
76
 STAT603:
Chapter 4
Basic Statistical
Inference

Section 4.6:
ANOVA
(Part 1)
 One-Way Analysis of Variance

• Recall: We can carry out hypothesis tests for a single population mean
using a single sample (𝐻𝐻𝑜𝑜 : 𝜇𝜇 = 𝜇𝜇0 ) – referred to as a one-sample test.
• We can also carry out hypothesis tests for comparing population means
of two groups (𝐻𝐻𝑜𝑜 : 𝜇𝜇1 = 𝜇𝜇2 ) – referred to as a two-sample test.
• An Analysis of Variance (ANOVA) is a statistical method of comparing
two or more groups of population means:

𝐻𝐻0: 𝜇𝜇1 = 𝜇𝜇2 = ⋯ = 𝜇𝜇𝑘𝑘 (The mean responses of 𝑘𝑘 groups are all equal)

𝐻𝐻1: At least two 𝜇𝜇𝑖𝑖′ 𝑠𝑠 differ, 𝑖𝑖 = 1, 2, … , 𝑘𝑘 (only one alternate hypothesis)

78
For a one-way ANOVA, you have the following:
• one continuous dependent/response variable.
• one categorical independent variable (also called a predictor or
explanatory variable). This is known as a factor.

Assumptions of an ANOVA:
The following conditions must be satisfied in order to use an ANOVA:
 Each sample must be randomly selected from a normal or approximately
normal population.
 The samples must be independent of each other (results of one sample
do not affect those of other samples).
 Each population must have the same variance (constant variance).

79
 One-way ANOVA
An ANOVA consists of drawing 𝒌𝒌 independent samples from 𝒌𝒌 populations:
Population 1 Population 2 Population 𝒌𝒌

Mean = 𝜇𝜇1 Mean = 𝜇𝜇2 Mean = 𝜇𝜇𝑘𝑘

Variance = 𝜎𝜎 2 Variance = 𝜎𝜎 2
…… Variance = 𝜎𝜎 2

Sample size = 𝑛𝑛1 Sample size = 𝑛𝑛2 Sample size = 𝑛𝑛𝑘𝑘

Note: These populations are referred to as treatments or factor levels for

the factor of interest.
It is not a requirement that 𝑛𝑛1 = 𝑛𝑛2 = ⋯ = 𝑛𝑛𝑘𝑘 .
80
1 Research Questions for One-Way ANOVA 1. Response variable = T-cell count

Do people treated with one of two new drugs have higher Factor = The drug type
average T-cell counts than people in the control group?
Treatments: Placebo, treatment 1,
treatment 2

2 Research Questions for One-Way ANOVA

Placebo Treatment 1
Does the type of fertilizer used affect the average weight
of garlic grown at the Montana Gourmet Garlic Ranch?

Treatment 2

2. Response variable = weight of garlic

Factor = Type of fertilizer

Treatments: Fertilizer 1, fertilizer 2, …
81
3 Research Questions for One-Way ANOVA
Do people spend different amounts depending on which
type of credit card they have?

3. Response variable = amount spent

Factor = Credit card type

Treatments: Visa, master Card….
82
4

4. Response variable = income

Factor = Profession
Treatments: Teacher, accountant

• This is an example of testing for a difference between two population means.

• If you analyze the difference between two means using ANOVA, you reach the same
conclusions as you reach using a pooled, two-sample t-test (Case B).

83
 Experimental Designs
• An ANOVA is commonly used in the analysis of data generated from an
experiment.
• In such an experiment, we have to ensure that the maximum information
is obtained from the data that was generated at the lowest possible cost.
• This process is called an experimental design.

Terminology:
 Experiment: is the means of getting an answer to a question. It can either
be observational or designed.
E.g. Decide which of the several pain relieving drugs are most effective.
Compare different methods of teaching.
84
 Terminology…
 Treatment: Different procedures under comparison in an experiment are
different treatments. Also known as factor levels.
E.g: In agriculture, different varieties of a crop or different fertilizers are
treatments.
In a medical experiment, different diets or medicines are the treatments.

 Experimental Units: An experimental unit is the material on which the

treatment is applied and it is the one on which the response variable is
measured.
E.g: In a medical experiment comparing the effectiveness of 3 different
drugs, the experimental units are the patients.

85
• There are different types of experimental designs ranging in complexity.
• We will consider only two experimental designs:
 A completely randomized design (CRD) and
 A randomized block design (RBD).
• Both experimental designs make use of an ANOVA in the analysis of the
data and both study the effect of one factor (whose factor levels make
up the different treatments) on the response variable.

Completely Randomized Designs

o A completely randomized design (CRD) is the simplest of the designs
using the two essential principles of
 Randomization and
 Replication
86
 Completely Randomized Designs…
o In this design, the treatments are assigned to the experimental units
completely at random.
o This allows every experimental unit, i.e., plot, animal, patient, soil sample,
etc., to have an equal probability of receiving a treatment.
o Experimental units are similar or homogeneous in their make up.
o More reliable estimates are obtained if a treatment is repeated a
number of times.
o Replication is a helpful in reducing errors by reducing random variation
within the treatments.
o Randomization may be done using a table of random numbers, or by
pulling numbered slips out of a hat, or by using a computer program.

87
 Assumptions of a CRD:
o As the analysis of the data from a CRD is done using an ANOVA, the
assumptions of an ANOVA hold (from slide 79):
 Samples are independent and random (results of one sample do not
affect those of other samples).
 Samples are obtained from a population of values that are normally
distributed with each population having the same variance 𝝈𝝈𝟐𝟐 .
o In addition to the assumptions above, the following also pertain to a CRD:
 The experimental units are identical w.r.t. their characteristics that can
affect the response of interest.
 There are no external variables that can affect the response of
interest.
88
o The last two assumptions of a CRD are very important because if there
are differences in the experimental units, or if there are other external
factors that affect the response variable, then we cannot attribute the
differences we are seeing in the mean responses of the treatments
groups to the different treatments that are being applied.

For example, suppose you are researching how exercise can change a
person’s weight.
• Exercise is your factor of interest (let’s say it consists of 3 treatments: no
exercise, little exercise and regular exercise)
• The change in weight is your response variable.
• Let’s say you test 200 volunteers that randomly get assigned a
treatment (exercise regime).

89
What are the experimental units?
 The individuals
What assumptions must be satisfied in order for this to be a CRD?
 The individuals (the experimental units) are all as similar as possible with
regards to characteristics that can affect weight change:
 They are all the same gender and a similar age.
 They all have a similar starting weight.
 They all have a similar diet.
 There are no external factors that can affect their weight change.
These assumptions ensure us that the differences we see in the weight
change are ONLY due to the treatments (exercise regime).

90
 New Terminology
 𝑌𝑌 is the response variable, and its values are responses.

 𝑌𝑌𝑖𝑖𝑖𝑖 refers to the 𝑗𝑗𝑡𝑡𝑡 observation/response in the sample from the 𝑖𝑖𝑡𝑡𝑡
treatment group.

E.g. 𝑌𝑌35 is the 5th observation of the sample from the 3rd treatment group.

 ∑∑𝑦𝑦𝑖𝑖𝑖𝑖 = Grand total of all the 𝑛𝑛 observations

 𝑇𝑇𝑖𝑖 = Total of all the responses in the sample from treatment group 𝑖𝑖
 𝑛𝑛𝑖𝑖 = Number of observations in the sample from treatment group 𝑖𝑖
 𝑇𝑇�𝑖𝑖 = Sample mean of the responses of the sample from treatment group 𝑖𝑖
 𝑛𝑛 = 𝑛𝑛1 + 𝑛𝑛2 + ⋯ + 𝑛𝑛𝑘𝑘 = The total number of observations in the full sample
 The grand mean, 𝑇𝑇, � is the mean of all the observations,
91
 Structure and Notation of a CRD:
 𝑘𝑘 treatments each with 𝑛𝑛𝑖𝑖 experimental units (or replications)
Replication

Treatment 1 2 . . 𝒏𝒏𝒊𝒊 Total Mean

1 𝑌𝑌11 𝑌𝑌12 . . 𝑌𝑌1𝑛𝑛𝑖𝑖 𝑇𝑇1 𝑇𝑇�1

2 𝑌𝑌21 𝑌𝑌22 . . 𝑌𝑌2𝑛𝑛𝑖𝑖 𝑇𝑇2 𝑇𝑇�2
. . . . . . .
𝒌𝒌 𝑌𝑌𝑘𝑘𝑘 𝑌𝑌𝑘𝑘𝑘 . . 𝑌𝑌𝑘𝑘𝑛𝑛𝑖𝑖 𝑇𝑇𝑘𝑘 𝑇𝑇�𝑘𝑘

 𝑌𝑌𝑖𝑖𝑖𝑖 is the 𝑗𝑗𝑡𝑡𝑡 response/replication in the 𝑖𝑖𝑡𝑡𝑡 treatment group.

 𝑖𝑖 = 1, 2, … , 𝑘𝑘 ; 𝑗𝑗 = 1, 2, … , 𝑛𝑛𝑖𝑖

 𝑛𝑛 = 𝑛𝑛1 + 𝑛𝑛2 + ⋯ + 𝑛𝑛𝑘𝑘 = total number of observations

92
 Statistical Model for a CRD:

 A statistical model is a specification of the statistical distribution for the

data, and can be used to describe/predict or estimate certain
variables/responses.
In the analysis of experimental data, there may be several different
models that can be fitted to the data, all with unknown parameters.
 The objectives of the experiment can often be
• To determine which model fits the data the best.
• To find the estimates of the parameters for that model, and
• To make inferences about the parameters using only a sample of
observations.

93
 Statistical Model for a CRD:
The general linear statistical model for a CRD has the form:
𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖
This is also known as
= 𝜇𝜇𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖 an ANOVA model
Where
• 𝜇𝜇 is the overall population mean (constant);
• 𝜏𝜏𝑖𝑖 (the Greek symbol Tau) is the effect of the 𝑖𝑖𝑡𝑡𝑡 treatment;
• 𝜇𝜇𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 is the mean of the 𝑖𝑖𝑡𝑡𝑡 treatment;
• 𝜀𝜀𝑖𝑖𝑖𝑖 is the random error for the 𝑗𝑗𝑡𝑡𝑡 response in the 𝑖𝑖𝑡𝑡𝑡 treatment. These
errors are assumed to be independent and normally distributed with a
mean of 0 and a variance of 𝜎𝜎 2 , i.e: 𝜀𝜀𝑖𝑖𝑖𝑖 ~𝑁𝑁(0, 𝜎𝜎 2 )
• 𝜇𝜇, 𝜏𝜏𝑖𝑖 and 𝜎𝜎 2 are the parameters of the model that are to be estimated.

𝜇𝜇 and 𝜏𝜏 are referred to as parameters of the statistical model.

94
 Hypotheses for a CRD:
The hypotheses for a CRD are:

𝐻𝐻0 : There are no differences in the treatment effects

𝐻𝐻1 : The effects of at least two treatments differ

This is the same as testing the following hypotheses:

𝐻𝐻0 : The means of all the treatment groups equal

𝐻𝐻1 : At least two treatment group means differ.

The hypotheses above are for that of any ANOVA, which is why the analysis
of data from a CRD can be done using an ANOVA.
These hypotheses can be tested by carrying out tests on the parameters in
the statistical model…
95
 Hypotheses for a CRD:
Recall: 𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖
= 𝜇𝜇𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖

If there is no significant different in the mean responses for the groups, then
𝜇𝜇1 = 𝜇𝜇2 = ⋯ = 𝜇𝜇𝑘𝑘 (all means equal). This also means that 𝜏𝜏1 = 𝜏𝜏2 = ⋯ = 𝜏𝜏𝑘𝑘 = 0
(the treatment effects are 0).
Therefore, the we can test: Notice this tests if
the treatment
𝐻𝐻0 : 𝜏𝜏1 = 𝜏𝜏2 = … = 𝜏𝜏𝑘𝑘 = 0
effects equal zero
𝐻𝐻1 : Not all 𝜏𝜏𝑖𝑖′ s are equal to zero

Or equivalently, since 𝜇𝜇𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 : Notice this tests if

𝐻𝐻0 : 𝜇𝜇1 = 𝜇𝜇2 = … = 𝜇𝜇𝑘𝑘 the treatment
means are equal
𝐻𝐻1 : At least two 𝜇𝜇𝑖𝑖′ s differ
96
 Example of a CRD
 Suppose the effects of two types of fertilizers (A and B) on the yield of a certain crop
(measured in kilograms per hectare) are to be compared. Each type of fertilizer was
randomly applied to 10 subplots of land of equal size on the same farm.
 Each subplot received the same water treatment, as well as the same amount of
fertilizer. After one month, the yield of the crop was measured. The results are:
Subplot
Fertilizer 1 2 3 4 5 6 7 8 9 10 Total Mean
A 2.5 3.1 2.7 4.9 1.5 2.4 1.8 3.1 3.7 2.2 27.9 2.79
B 3.4 4.8 2.9 5.7 2.8 3.6 3.1 4.2 2.6 2.1 35.2 3.52

 Let Fertilizer A = Treatment 1 and Fertilizer B = Treatment 2 ∴ 𝑘𝑘 = 2, 𝑛𝑛𝑖𝑖 = 10

 𝑇𝑇1. = 27.9 and 𝑇𝑇2. = 35.2
 𝑇𝑇�1. = 2.79 and 𝑇𝑇�2. = 3.52
 𝑛𝑛 = 2 10 = 20 total number of observations
97
 This is a CRD as the experimental units (the subplots of land) are exactly
identical with respect to all factors that may influence the response: i.e.
the water treatment, the amount of fertilizer, the size of the plot and the
location.
 Therefore, any differences in the response of the yield of crop can be
attributed to the different effects of the fertilizers.
 Therefore, the CRD statistical model can be used:
𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖
= 𝜇𝜇𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖
 𝜇𝜇 is the common (population) mean, i.e: If we measured the yield of
crops under the same conditions of the experiment without applying any
fertilizers, naturally there would be growth in the crop.
This value of 𝜇𝜇 is basically the average of the yield of the crop without the
fertilizer.
98
 𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖
= 𝜇𝜇𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖
 𝜏𝜏𝑖𝑖 would be the effect (on the yield or growth of the crop) that is due to
the fertilizer being applied to that subplot of land, i.e. how much extra a
crop is growing due to the fertilizer.

The mean for each treatment group (mean of all responses for a
treatment) is 𝜇𝜇𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖

Notice in this example (the data in the table on slide 19), even with the
exact same fertilizer being used under the same conditions, the yield of
crop in each of the 10 subplots of land varied
In general, the exact same experiment doesn’t always obtain the exact
same results. This is known as measurement error or random variation. This
random variation is represented by the 𝜀𝜀𝑖𝑖𝑖𝑖 term in the statistical model.
99
 Logic Behind the Analysis of Variance
 Even though we are comparing means, the ANOVA test procedure
considers the variation (differences) in the observations.
 Briefly speaking, if there is a large variation in the observations between
the treatment groups, then we can conclude that the treatment means
differ.
The test procedure of an ANOVA is based on a comparison of two sources
of variation:
 Variation attributed to the differences among the observations between
the treatment groups (i.e. due to the effect of the treatments)
 Variation attributed to the differences among the observations within the
treatment groups (i.e. due to random error or random variation/
measurement error)
100
 Logic Behind the Analysis of Variance…
 If these two sources of variation are in proportion (i.e. if their ratio is
close to 1), then it can be concluded there are no differences in the
treatment means.
 However, if the variation between the treatment groups is much larger
than the variation within the treatment groups, then it can be
concluded the treatment means differ.
For example:
Scores of 22 children taught by 3 different methods A, B, C are obtained.
Test at a 5% level of significance that the mean scores for the 3 methods
are equal.
Because one independent variable is being studied (method of
teaching), the process is called one-way analysis of variance.
𝐻𝐻0: 𝜇𝜇1 = 𝜇𝜇2 = 𝜇𝜇3 𝐻𝐻1: At least two means differ
101
 Logic Behind the Analysis of Variance…
Variation between samples

A B C
116 132 108
117 137 96
Variation within samples

138 131 131

100 108 130
125 111 111
130 130 126
134 140
124
114

102
 Logic Behind the Analysis of Variance…
The totals, sample A B C
sizes and sample 𝑻𝑻𝟏𝟏 = 1098 𝑻𝑻𝟐𝟐 = 889 𝑻𝑻𝟑𝟑 = 702
means of each 𝒏𝒏𝟏𝟏 = 9 𝒏𝒏𝟐𝟐 = 7 𝒏𝒏𝟑𝟑 = 6
group:
�𝟏𝟏 = 122
𝑻𝑻 �𝟐𝟐 = 127
𝑻𝑻 �𝟑𝟑 = 117
𝑻𝑻
o If we examine the means of three samples, we see variability BETWEEN
means.
o This variability is due to the combined effect of the natural differences
and possible treatment differences between the groups (differences due
to the different treatments being applied).
o So our problem is to decide whether the differences among sample
means are due to chance or if they occur because the three
populations/treatments are actually different.
103
 Logic Behind the Analysis of Variance…
Consider two situations showing the sample mean of each treatment
group with the individual observations around the mean.
Which situation shows evidence to suggest there is a difference among the
four population means? Situation 2
Situation 1 shows little variation/difference between the treatment group
means compared to the variation within each group, whereas situation 2
shows a larger variation between the means compared to within the
groups.

104
 Logic Behind the Analysis of Variance…
Variation within samples < variation between samples Reject Ho

Variation within samples > variation between samples Do not reject Ho

The following conditions must be TRUE in order to use an ANOVA (recall

from slide 3):
1. Each sample must be randomly selected from a normal or
approximately normal population.
2. The samples must be independent of each other.
3. Each population must have the same variance (constant variance).

We can use statistical software to check the above assumptions for our data.
105
 One-way ANOVA Test Procedure
𝐻𝐻0: 𝜇𝜇1 = 𝜇𝜇2 = ⋯ = 𝜇𝜇𝑘𝑘 𝐻𝐻1: At least two 𝜇𝜇𝑖𝑖′ 𝑠𝑠 differ (𝑖𝑖 = 1, 2, … , 𝑘𝑘)

A measure of
Variation between samples variability in an
Test Statistic =
Variation within samples ANOVA is a MEAN
SUM OF SQUARES
Mean Sum of Squares Between
𝐹𝐹 calc =
Mean Sum of Square Within 𝜶𝜶 = Level of significance

This test statistic follows an F-distribution that is defined by two degrees of

freedom (that of the numerator and that of the denominator)
Terminology: Degrees of freedom (df) refer to the number of independent
values in a statistical calculation that are free to vary while estimating a
parameter. They are crucial in many statistical tests.
Reject the null hypothesis if : 𝐹𝐹 calc > the critical value or P-value < 𝛼𝛼
106
 Calculating the F-Test Statistic
 Recall from the previous slide: The test statistic is calculated by dividing
the variation between the groups by the variation within the groups.
 Any measure of variation is found by dividing a sum of squares by its
degrees of freedom.
A measure of variation between the groups is called the mean sum of
squares of the treatments (𝑀𝑀𝑀𝑀𝑀𝑀):
𝑆𝑆𝑆𝑆𝑆𝑆 𝑑𝑑𝑑𝑑 of SST
𝑀𝑀𝑀𝑀𝑀𝑀 =
𝑘𝑘 − 1 𝑘𝑘
𝑇𝑇𝑖𝑖2
Where 𝑆𝑆𝑆𝑆𝑆𝑆 is the treatment sum of squares given by: 𝑆𝑆𝑆𝑆𝑆𝑆 = � − 𝐶𝐶𝐶𝐶 (FYI)
𝑛𝑛𝑖𝑖
𝑖𝑖=1
• 𝑇𝑇𝑖𝑖 = the total of the responses in group 𝑖𝑖
• 𝑛𝑛𝑖𝑖 = the number of responses in group 𝑖𝑖 sum across 𝑘𝑘 treatment groups
𝑛𝑛𝑖𝑖 2
𝑘𝑘
∑𝑖𝑖 ∑𝑗𝑗 𝑌𝑌𝑖𝑖𝑖𝑖
• 𝐶𝐶𝐶𝐶 = is the grand squared mean of all the responses (FYI)
𝑛𝑛
107
 Calculating the F-Test Statistic
A measure of variation within the groups is called the mean sum of squares
of the error (𝑀𝑀𝑀𝑀𝑀𝑀):
𝑆𝑆𝑆𝑆𝑆𝑆 𝑑𝑑𝑑𝑑 of SSE
𝑀𝑀𝑀𝑀𝑀𝑀 =
𝑛𝑛 − 𝑘𝑘
Where 𝑆𝑆𝑆𝑆𝑆𝑆 is the error sum of squares given by: 𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑆𝑆𝑆𝑆𝑆𝑆
𝑘𝑘 𝑛𝑛𝑖𝑖

Where 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 is the TOTAL sum of squares: 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = � � 𝑌𝑌𝑖𝑖𝑖𝑖2 − 𝐶𝐶𝐶𝐶 (FYI)
𝑖𝑖=1 𝑗𝑗=1

• 𝑛𝑛 = 𝑛𝑛1 + 𝑛𝑛2 + ⋯ + 𝑛𝑛𝑘𝑘 the total number of observations Square of every

response 𝑌𝑌𝑖𝑖𝑖𝑖 , summed
• 𝑛𝑛𝑖𝑖 = the number of responses in group 𝑖𝑖 over the full data set
𝑛𝑛𝑖𝑖 2
𝑘𝑘
• 𝐶𝐶𝐶𝐶 = ∑𝑖𝑖 ∑𝑗𝑗 𝑌𝑌𝑖𝑖𝑖𝑖 is the grand squared mean of all the responses (FYI)
𝑛𝑛
108
 The results can be summarized in an ANOVA table:
Source of Sum of Mean sum of
𝑑𝑑𝑑𝑑 F-statistic
Variation Squares squares
𝑆𝑆𝑆𝑆𝑆𝑆
Treatment 𝑆𝑆𝑆𝑆𝑆𝑆 𝑘𝑘 − 1 𝑀𝑀𝑀𝑀𝑀𝑀 =
+ 𝑘𝑘 − 1
+ 𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 =
𝑀𝑀𝑀𝑀𝑀𝑀
𝑆𝑆𝑆𝑆𝑆𝑆 𝑀𝑀𝑀𝑀𝑀𝑀
Error 𝑆𝑆𝑆𝑆𝑆𝑆 𝑛𝑛 − 𝑘𝑘 𝑀𝑀𝑀𝑀𝑀𝑀 =
= = 𝑛𝑛 − 𝑘𝑘
Total 𝑆𝑆𝑆𝑆𝑆𝑆𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜 𝑛𝑛 − 1
𝑘𝑘 𝑛𝑛𝑖𝑖 𝑘𝑘 2
𝑘𝑘 𝑛𝑛𝑖𝑖
(FYI) 𝑇𝑇𝑖𝑖2 ∑𝑖𝑖 ∑𝑗𝑗 𝑌𝑌𝑖𝑖𝑖𝑖
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = � � 𝑌𝑌𝑖𝑖𝑖𝑖2 − 𝐶𝐶𝐶𝐶 𝑆𝑆𝑆𝑆𝑆𝑆 = � − 𝐶𝐶𝐶𝐶 𝐶𝐶𝐶𝐶 =
𝑛𝑛𝑖𝑖 𝑛𝑛
𝑖𝑖=1 𝑗𝑗=1 𝑖𝑖=1
(You will not be expected to calculate
𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑆𝑆𝑆𝑆𝑆𝑆 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 and 𝑆𝑆𝑆𝑆𝑆𝑆 from scratch)
109
Example:
Scores of 22 children taught by 3 different methods A, B, C are obtained.
Test at a 5% level of significance that the mean scores for the 3 methods
are equal. 𝑘𝑘 = 3 A B C
𝐻𝐻0: 𝜇𝜇1 = 𝜇𝜇2 = 𝜇𝜇3 116 132 108
Solution:
117 137 96
𝐻𝐻1: At least two means differ
138 131 131
The statistical model we wish to fit to this CRD:
100 108 130
𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖 𝑖𝑖 = 1, 2,3 ; 𝑗𝑗 = 1, … , 𝑛𝑛𝑖𝑖
125 111 111
𝑛𝑛1 = 9 𝑛𝑛2 = 7 𝑛𝑛3 = 6 130 130 126
𝑌𝑌𝑖𝑖𝑖𝑖 = score of 𝑗𝑗𝑡𝑡𝑡 child taught by 𝑖𝑖𝑡𝑡𝑡 method, 134 140
𝜏𝜏𝑖𝑖 = effect of teaching method 𝑖𝑖 124
Suppose we obtain the ANOVA table on 114
the next slide…
110
Source of
Sum of Squares 𝑑𝑑𝑑𝑑 Mean sum of squares F-statistic
Variation
𝑆𝑆𝑆𝑆𝑇𝑇
Treatment 𝑆𝑆𝑆𝑆𝑆𝑆 = 323.86 ÷ 𝑘𝑘 − 1 = 2 = 𝑀𝑀𝑀𝑀𝑀𝑀 = = 161.93 𝑀𝑀𝑀𝑀𝑇𝑇
+ 𝑘𝑘 − 1 𝐹𝐹 =
+ ÷ = 𝑀𝑀𝑀𝑀𝑀𝑀
𝑆𝑆𝑆𝑆𝑆𝑆 = 1.02
Error 𝑆𝑆𝑆𝑆𝑆𝑆 = 3026 ÷ 𝑛𝑛 − 𝑘𝑘 = 19 = 𝑀𝑀𝑀𝑀𝑀𝑀 = = 159.26
= = 𝑘𝑘 − 𝑛𝑛

Total 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 3349.86 𝑛𝑛 − 1 = 21 (this is left blank)

Notice how values in the table add up: The corresponding P-value = .3795
…323.86 + 3026 = 3349.86 (obtained from software)
2 + 19 = 21 ∴ since P-value > 0.05, 𝐻𝐻0 is NOT rejected at a
323.86 ÷ 2 = 161.93 5% level of significance.
13026 ÷ 19 = 159.26 Thus, there is insufficient evidence to conclude
161.93 ÷ 159.26 = 1.02 that the mean responses of at least two
treatments differ. 111
 Completely Randomized Designs Recap……
Assumptions:
INDEPENDENT; RANDOM; NORMALLY DISTRIBUTED POP; SAME VARIANCE
A B C
Replication
116 132 108
117 137 96
Treatment 1 2 . . 𝒏𝒏𝒊𝒊 Total Mean
138 131 131
100 108 130
1 𝑌𝑌11 𝑌𝑌12 . . 𝑌𝑌1𝑛𝑛𝑖𝑖 𝑇𝑇1 𝑇𝑇�1
125 111 111
2 𝑌𝑌21 𝑌𝑌22 . . 𝑌𝑌2𝑛𝑛𝑖𝑖 𝑇𝑇2 𝑇𝑇�2 130 130 126
. . . . . . . 134 140
124
𝒌𝒌 𝑌𝑌𝑘𝑘𝑘 𝑌𝑌𝑘𝑘𝑘 . . 𝑌𝑌𝑘𝑘𝑛𝑛𝑖𝑖 𝑇𝑇𝑘𝑘 𝑇𝑇�𝑘𝑘
114

𝐻𝐻0: 𝜇𝜇1 = 𝜇𝜇2 = ⋯ = 𝜇𝜇𝑘𝑘 𝐻𝐻1: At least two 𝜇𝜇𝑖𝑖′ 𝑠𝑠 differ (𝑖𝑖 = 1, 2, … , 𝑘𝑘)
𝑛𝑛𝑖𝑖 → 𝑛𝑛
Variation between samples 𝐹𝐹 calc
Test Statistic = 𝑇𝑇𝑖𝑖 → 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡
Variation within samples

Variation within samples < variation between samples Reject Ho

Variation within samples > variation between samples Do not reject Ho

1. In one-way ANOVA, the amount of total variation that is unexplained is measured by the:

a. sum of squares for treatments.

b. sum of squares for error.
c. total sum of squares.
d. degrees of freedom.

2. Consider the following partial ANOVA table:

1. In one-way ANOVA, the amount of total variation that is unexplained is measured by the:

a. sum of squares for treatments.

b. sum of squares for error.
c. total sum of squares.
d. degrees of freedom.

2. Consider the following partial ANOVA table:

 STAT603:
Chapter 4
Basic Statistical
Inference

Section 4.6:
ANOVA
(Part 2)
 Randomized Block Designs
o Recall: the assumptions of a CRD require the experimental units to be
homogeneous with no external factors affecting the response variable.
o However, it is not always possible to design an experiment under such
assumptions.
o If these assumptions are not satisfied, then there is an extra source of
variation in the response variable due to the differences in the
experimental units/or due to an external factor.
o It is possible to control for this extra source of variation by ‘blocking’ for it
(isolating it).
o This is done by introducing a blocking factor into the design, and such a
design is referred to as a randomized block design (RBD).
 Randomized Block Designs…
o E.g. consider the example of how exercise affects a person’s weight
Suppose 100 men and 100 women were randomly assigned to a
treatment (exercise regime)
o In this case, the individuals who are the experimental units are not
homogenous as their gender differs.
o Thus, gender would be included as a blocking factor (in addition to the
main factor of interest – the exercise regime). This would then result in a
randomized block design.
o It is possible to include additional blocking factors (for example age as
well), however in this course we will only be considering a RBD with one
blocking factor.
 Structure and Notation of a RBD:
 𝑘𝑘 treatments and 𝑏𝑏 blocks each. (N.B: The rows and columns
Treatment can be interchanged.)

Block 1 2 . 𝒌𝒌 Total for blocks Mean of blocks

1 𝑌𝑌11 𝑌𝑌21 . 𝑌𝑌𝑘𝑘𝑘 𝐵𝐵1 𝐵𝐵�1

2 𝑌𝑌12 𝑌𝑌22 . 𝑌𝑌𝑘𝑘𝑘 𝐵𝐵2 𝐵𝐵�2
. . . . . .
𝒃𝒃 𝑌𝑌1𝑏𝑏 𝑌𝑌2𝑏𝑏 . 𝑌𝑌𝑘𝑘𝑘𝑘 𝐵𝐵𝑏𝑏 𝐵𝐵�𝑏𝑏
Total for treatments 𝑇𝑇1 𝑇𝑇2 . 𝑇𝑇𝑘𝑘 ∑∑𝑌𝑌𝑖𝑖𝑖𝑖 = 𝐺𝐺

Mean of treatments 𝑇𝑇�1 𝑇𝑇�2 . 𝑇𝑇�𝑘𝑘

 𝑌𝑌𝑖𝑖𝑖𝑖 is the response for the 𝑖𝑖𝑡𝑡𝑡 treatment and the 𝑗𝑗𝑡𝑡𝑡 block.
 ∑∑𝑌𝑌𝑖𝑖𝑖𝑖 = 𝐺𝐺 is the grand total 𝑛𝑛 = 𝑏𝑏 × 𝑘𝑘 is the total no. of observations.
 𝑏𝑏 responses in each treatment & 𝑘𝑘 responses in each block.
 Statistical Model for an RBD:
The general linear model for a RBD has the form:
𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 + 𝛽𝛽𝑗𝑗 + 𝜀𝜀𝑖𝑖𝑖𝑖
Where
• 𝜇𝜇 is the overall population mean (constant);
• 𝜏𝜏𝑖𝑖 is the effect of the 𝑖𝑖𝑡𝑡𝑡 treatment, 𝑖𝑖 = 1, … , 𝑘𝑘;
• 𝛽𝛽𝑗𝑗 is the effect of the 𝑗𝑗𝑡𝑡𝑡 block, 𝑗𝑗 = 1, … , 𝑏𝑏 ;
• 𝜇𝜇𝑖𝑖. = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 is the mean of the 𝑖𝑖𝑡𝑡𝑡 treatment;
• 𝜇𝜇.𝑗𝑗 = 𝜇𝜇 + 𝛽𝛽𝑗𝑗 is the mean of the 𝑗𝑗𝑡𝑡𝑡 block;
• 𝜀𝜀𝑖𝑖𝑖𝑖 is the random error for the response of the 𝑖𝑖𝑡𝑡𝑡 treatment in the 𝑗𝑗𝑡𝑡𝑡
block. These errors are assumed to be independent and normally
distributed with a mean of 0 and a variance of 𝜎𝜎 2 , i.e: 𝜀𝜀𝑖𝑖𝑖𝑖 ~𝑁𝑁(0, 𝜎𝜎 2 )
 Hypotheses for a RBD:
As usual, in carrying out a test to determine if there are differences in the
treatment effects, we test the following hypotheses:

𝐻𝐻0 : 𝜏𝜏1 = 𝜏𝜏2 = … = 𝜏𝜏𝑘𝑘 = 𝟎𝟎 versus 𝐻𝐻1 : Not all 𝜏𝜏𝑖𝑖′ s are equal to zero

Or equivalently
𝐻𝐻0 : 𝜇𝜇1. = 𝜇𝜇2. = … = 𝜇𝜇𝑘𝑘. versus 𝐻𝐻1 : At least two 𝜇𝜇𝑖𝑖.′ s differ

However, we may also wish to determine if there is a blocking effect, i.e. if

blocking was effective. Therefore, we may also test the following
hypotheses:
𝐻𝐻0 : 𝛽𝛽1 = 𝛽𝛽2 = … = 𝛽𝛽𝑏𝑏 = 𝟎𝟎 versus 𝐻𝐻1 : Not all 𝛽𝛽𝑗𝑗′ s are equal to zero

Or equivalently
𝐻𝐻0 : 𝜇𝜇.1 = 𝜇𝜇.2 = … = 𝜇𝜇.𝑏𝑏 versus 𝐻𝐻1 : At least two 𝜇𝜇.𝑗𝑗′ s differ
 Assumptions of an RBD:

o As in the case of a CRD, the analysis of the data from a RBD is done using
an ANOVA,
o Therefore, the assumptions of an ANOVA hold:
 Samples are independent and random (results of one sample do not
affect those of other samples).
 Samples are obtained from a population of values that are normally
distributed with each population having the same variance 𝝈𝝈𝟐𝟐 .
o In the statistical model for a RBD, it is assumed there are no joint effects
(interactions) between the treatments and blocks.
 Analysis of an RBD:
o An ANOVA is used to analyze the data from a RBD.
o Recall from part 1 of this chapter, the total sum of squares 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 is
partitioned into the various sources of variation.
o The same procedure is applied in an ANOVA for a RBD, where there are
now 3 sources of variation:
1. Variation due to the treatments
2. Variation due to the blocks
3. Variation due to random error (natural variation)
o Thus, the total sum of squares is partitioned into 3 components:
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑆𝑆𝑆𝑆𝑆𝑆 + 𝑆𝑆𝑆𝑆𝑆𝑆 + 𝑆𝑆𝑆𝑆𝑆𝑆
Where 𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 is the block sum of squares (𝑆𝑆𝑆𝑆𝑆𝑆 & 𝑆𝑆𝑆𝑆𝑆𝑆 are the same
from a CRD)
The degrees of freedom can be obtained for each sum of squares:
𝑑𝑑𝑑𝑑 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑏𝑏𝑏𝑏 − 1 (total number of observations less 1)

𝑑𝑑𝑑𝑑 𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑘𝑘 − 1 (number of treatment groups less 1)

𝑑𝑑𝑑𝑑 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑏𝑏 − 1 (number of blocks less 1)

𝑑𝑑𝑑𝑑 𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑑𝑑𝑑𝑑 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑑𝑑𝑑𝑑 𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑑𝑑𝑑𝑑(𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆)

= 𝑏𝑏𝑏𝑏 − 1 − 𝑘𝑘 − 1 − (𝑏𝑏 − 1)
= (𝑏𝑏 − 1)(𝑘𝑘 − 1)
The mean sum of squares can now be obtained for each source of variation:
𝑆𝑆𝑆𝑆𝑆𝑆 𝑆𝑆𝑆𝑆𝑆𝑆𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙 𝑆𝑆𝑆𝑆𝑆𝑆
𝑀𝑀𝑀𝑀𝑀𝑀 = 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 = 𝑀𝑀𝑀𝑀𝑀𝑀 =
𝑑𝑑𝑑𝑑(𝑆𝑆𝑆𝑆𝑆𝑆) 𝑑𝑑𝑑𝑑(𝑆𝑆𝑆𝑆𝑆𝑆) 𝑑𝑑𝑑𝑑(𝑆𝑆𝑆𝑆𝑆𝑆)
 Hypothesis Tests for a RBD:
Since we are now considering two factors in a RBD (the treatments and
the blocks), we can now perform two hypothesis tests:

1. Comparing the means of the treatments:

𝐻𝐻0: 𝜇𝜇1 = 𝜇𝜇2 = ⋯ = 𝜇𝜇𝑘𝑘 (treatment means are equal)
𝐻𝐻1: At least two treatment means differ

2. Comparing the means of the blocks:

𝐻𝐻0: 𝜇𝜇1 = 𝜇𝜇2 = ⋯ = 𝜇𝜇𝑏𝑏 (block means are equal)
𝐻𝐻1: At least two block means differ

The test procedure for each hypothesis test above is similar to that of a CRD
or one-factor ANOVA (compare the P-value to 𝛼𝛼).
 ANOVA Table for a RBD:

Source of Sum of
𝑑𝑑𝑑𝑑 Mean sum of squares F-statistic
Variation Squares
𝑆𝑆𝑆𝑆𝑆𝑆 𝑀𝑀𝑀𝑀𝑀𝑀
Treatments 𝑆𝑆𝑆𝑆𝑆𝑆 𝑘𝑘 − 1 𝑀𝑀𝑀𝑀𝑀𝑀 = 𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 =
𝑘𝑘 − 1 𝑀𝑀𝑀𝑀𝑀𝑀
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀
Blocks 𝑆𝑆𝑆𝑆𝐵𝐵 𝑏𝑏 − 1 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 = 𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 =
𝑏𝑏 − 1 𝑀𝑀𝑀𝑀𝑀𝑀
𝑆𝑆𝑆𝑆𝑆𝑆
Error 𝑆𝑆𝑆𝑆𝑆𝑆 (𝑏𝑏 − 1)(𝑘𝑘 − 1) 𝑀𝑀𝑀𝑀𝑀𝑀 =
(𝑏𝑏 − 1)(𝑘𝑘 − 1)
Total 𝑆𝑆𝑆𝑆𝑆𝑆𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜 𝑏𝑏𝑏𝑏 − 1
 Example 1:
The cellphone industry is involved in a battle for customers, each with its
own pricing plan for customers. A consumer watchdog group decides to
compare the average cost of data for 4 cellphone companies using 3
different usage levels as blocks. Results are given in the table.
1. Construct the ANOVA table for this experiment.

Company Treatments

Usage Level A B C D 𝑘𝑘 = 4
𝑏𝑏 = 3 Low 27 24 31 23
Blocks Middle 68 76 65 67
High 308 326 312 300
The usage level can affect the price of data, therefore it must be blocked for (the more
data you buy, the cheaper it is.
The statistical model we wish to fit to this RBD:

𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝜏𝜏𝑖𝑖 + 𝛽𝛽𝑗𝑗 + 𝜀𝜀𝑖𝑖𝑖𝑖 𝑖𝑖 = 1, … , 4; 𝑗𝑗 = 1, … , 3

𝑌𝑌𝑖𝑖𝑖𝑖 = cost of data for the 𝑖𝑖𝑡𝑡𝑡 cellular company and the 𝑗𝑗𝑡𝑡𝑡 usage level.
𝜏𝜏𝑖𝑖 = effect of cellular company 𝑖𝑖 on the cost of data
𝛽𝛽𝑗𝑗 = effect of usage level 𝑗𝑗 on the cost of data
𝜀𝜀𝑖𝑖𝑖𝑖 = the random error for the response of the 𝑖𝑖𝑡𝑡𝑡 cellular company in the
𝑗𝑗𝑡𝑡𝑡 usage level.
 The ANOVA Table:

Source of Sum of Mean sum of

df F-statistic P-value
Variation Squares squares
Treatments/
222.25 3 𝑀𝑀𝑀𝑀𝑀𝑀 = 74.1 1.84 0.2405
Company
Blocks/ 𝑀𝑀𝑀𝑀𝑀𝑀𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙
189 335.17 2 2349.07 < .0001
Usage = 94667.6

Error 241.50 6 𝑀𝑀𝑀𝑀𝑀𝑀 = 40.3

Total 189 798.92 11

2. Do the data provide sufficient evidence to indicate a difference in the
average monthly cell phone cost depending on the company the
customer uses? (the treatments)

𝐻𝐻0: There is no difference in the average cost among companies

𝐻𝐻1: The average cost differs for at least two companies

𝛼𝛼 = 0.05 P-value = 0.2405 > 0.05

∴ 𝐻𝐻0 is not rejected at a 5% l.o.s.

Thus, there is insufficient evidence to conclude that the average cost differs
for at least two companies.
2. Do the data provide sufficient evidence to indicate a difference in the
block means (average cost due to level of usage)?

𝐻𝐻0: There is no difference in the average cost due to usage levels.

𝐻𝐻1: The average cost differs for at least two usage levels.

𝛼𝛼 = 0.05 P-value < 0.0001 < 0.05

∴ 𝐻𝐻0 is rejected at a 5% l.o.s.

Thus, there is sufficient evidence to conclude that the average cost differs for
at least two usage levels (the mean of at least two blocks differ)
 Randomized Block Designs Recap……

Source of Sum of
𝑑𝑑𝑑𝑑 Mean sum of squares F-statistic
Variation Squares
𝑆𝑆𝑆𝑆𝑆𝑆 𝑀𝑀𝑀𝑀𝑀𝑀
Treatments 𝑆𝑆𝑆𝑆𝑆𝑆 𝑘𝑘 − 1 𝑀𝑀𝑀𝑀𝑀𝑀 = 𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 =
𝑘𝑘 − 1 𝑀𝑀𝑀𝑀𝑀𝑀
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀
Blocks 𝑆𝑆𝑆𝑆𝐵𝐵 𝑏𝑏 − 1 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 = 𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 =
𝑏𝑏 − 1 𝑀𝑀𝑀𝑀𝑀𝑀
𝑆𝑆𝑆𝑆𝑆𝑆
Error 𝑆𝑆𝑆𝑆𝑆𝑆 (𝑏𝑏 − 1)(𝑘𝑘 − 1) 𝑀𝑀𝑀𝑀𝑀𝑀 =
(𝑏𝑏 − 1)(𝑘𝑘 − 1)
Total 𝑆𝑆𝑆𝑆𝑆𝑆𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜 𝑏𝑏𝑏𝑏 − 1
 Questions:
1

2
 Questions:
1

2
3

True or false?
3

True or false?
4

Calculate a to g
4

a= 6720 [12600 -3120-2760] d= 520 [3120/6]

b= 1680 [6720/4] e= 4.5217 [520/115]
c= 14.6087 [1680/115] f= 2760 [24*115]
g= 24 [34-6-4]
 STAT603:
Chapter 4
Basic Statistical
Inference

Section 4.6:
ANOVA
(Part 3)
 Two-way ANOVA
o So far we have considered a one-way ANOVA, where the effect of one
factor (independent categorical variable) on the continuous response
variable is of interest.
o We considered this under two different types of experimental designs: a
completely randomized design (CRD) and a randomized block design
(RBD).
o Even though the analysis of a RBD included two factors, one of the factors
was a the blocking factor which controlled for differences in the
experimental units or external factors affecting the response variable.
o The effect of the blocking factor on the response variable was also not the
main interest in a RBD.
o Furthermore, no joint effects or interactions were allowed in a RBD
between the factor of interest and the blocking factor.
141
 Two-Way ANOVA
o Now, in a two-way ANOVA, we are interested in the effects of TWO
factors/categorical variables (each consisting of at least two
categories/levels) on the continuous response variable.
o These two factors are allowed to have joint effects or interactions.
o Interaction effects occur when the effect of one factor on the response
depends on the value or level of another factor.
o However, interaction effects do not occur if the two variables/factors
are independent of each other.
o A two-way ANOVA can be carried out in a CRD as well as a RBD (i.e.
the same assumptions would hold in these experimental designs as the
one-way ANOVA. A blocking factor can also be introduced which
results in a RBD).
o A two-way ANOVA is also known as a factorial design.
142
 Treatments in a Two-Way ANOVA
o The primary objective of a two-way ANOVA is to study the effects of the
combination of the levels of the factors that are of interest.
o The treatment groups in a two-way ANOVA are formed by making all
possible combinations of the levels/categories of the two factors.
o For example, if the first factor has 3 levels and the second factor has 2
levels, then there will be 3x2=6 different treatment groups.
o Question: if there are two main factors of interest, factors A and B with 𝑎𝑎
and 𝑏𝑏 levels, respectively, then how many treatments (factor level
combinations) will be applied/investigated in the experiment?
 Answer: 𝑎𝑎𝑎𝑎 treatments in total.
o The treatment effects are made up of the effects of factor A, the effects of
factor B and the effects of the interaction between factor A and factor B (if
there is an interaction between the factors).
143
 Structure and Notation of a two-way ANOVA (CRD):
Factor B
𝐵𝐵1 𝐵𝐵2 … 𝐵𝐵𝑏𝑏

𝐴𝐴1 𝑌𝑌111 , 𝑌𝑌112 , … , 𝑌𝑌11𝑟𝑟 𝑌𝑌121 , 𝑌𝑌122 , … , 𝑌𝑌12𝑟𝑟 … 𝑌𝑌1𝑏𝑏𝑏 , 𝑌𝑌1𝑏𝑏𝑏 , … , 𝑌𝑌1𝑏𝑏𝑏𝑏
Factor A

𝐴𝐴2 𝑌𝑌211 , 𝑌𝑌212 , … , 𝑌𝑌21𝑟𝑟 𝑌𝑌221 , 𝑌𝑌222 , … , 𝑌𝑌22𝑟𝑟 … 𝑌𝑌2𝑏𝑏𝑏 , 𝑌𝑌2𝑏𝑏𝑏 , … , 𝑌𝑌2𝑏𝑏𝑏𝑏
⋮ ⋮ … ⋮
𝐴𝐴𝑎𝑎 𝑌𝑌𝑎𝑎𝑎𝑎 , 𝑌𝑌𝑎𝑎𝑎𝑎 , … , 𝑌𝑌𝑎𝑎𝑎𝑎𝑎 𝑌𝑌𝑎𝑎𝑎𝑎 , 𝑌𝑌𝑎𝑎𝑎𝑎 , … , 𝑌𝑌𝑎𝑎𝑎𝑎𝑎 … 𝑌𝑌𝑎𝑎𝑎𝑎𝑎 , 𝑌𝑌𝑎𝑎𝑎𝑎𝑎 , … , 𝑌𝑌𝑎𝑎𝑎𝑎𝑎𝑎

There are 𝑟𝑟 repeats/observations for each treatment group.

𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 is the 𝑘𝑘 𝑡𝑡𝑡 response to the 𝑖𝑖𝑡𝑡𝑡 level of factor A and the 𝑗𝑗𝑡𝑡𝑡 level of factor
B.
E.g. 𝑌𝑌352 is the 2𝑛𝑛𝑛𝑛 response to the 3𝑟𝑟𝑟𝑟 level of factor A and the 5𝑡𝑡𝑡 level
of factor B.
 𝑖𝑖 = 1, … , 𝑎𝑎 ; 𝑗𝑗 = 1, … , 𝑏𝑏 ; 𝑘𝑘 = 1, … , 𝑟𝑟 ; 𝑛𝑛 = 𝑎𝑎𝑎𝑎𝑎𝑎 is the total number of observations
 Structure and Notation of a two-way ANOVA (CRD):
Factor B
𝐵𝐵1 𝐵𝐵2 … 𝐵𝐵𝑏𝑏

𝐴𝐴1 𝑌𝑌111 , 𝑌𝑌112 , … , 𝑌𝑌11𝑟𝑟 𝑌𝑌121 , 𝑌𝑌122 , … , 𝑌𝑌12𝑟𝑟 … 𝑌𝑌1𝑏𝑏𝑏 , 𝑌𝑌1𝑏𝑏𝑏 , … , 𝑌𝑌1𝑏𝑏𝑏𝑏
Factor A

𝐴𝐴2 𝑌𝑌211 , 𝑌𝑌212 , … , 𝑌𝑌21𝑟𝑟 𝑌𝑌221 , 𝑌𝑌222 , … , 𝑌𝑌22𝑟𝑟 … 𝑌𝑌2𝑏𝑏𝑏 , 𝑌𝑌2𝑏𝑏𝑏 , … , 𝑌𝑌2𝑏𝑏𝑏𝑏
⋮ ⋮ … ⋮
𝐴𝐴𝑎𝑎 𝑌𝑌𝑎𝑎𝑎𝑎 , 𝑌𝑌𝑎𝑎𝑎𝑎 , … , 𝑌𝑌𝑎𝑎𝑎𝑎𝑎 𝑌𝑌𝑎𝑎𝑎𝑎 , 𝑌𝑌𝑎𝑎𝑎𝑎 , … , 𝑌𝑌𝑎𝑎𝑎𝑎𝑎 … 𝑌𝑌𝑎𝑎𝑎𝑎𝑎 , 𝑌𝑌𝑎𝑎𝑎𝑎𝑎 , … , 𝑌𝑌𝑎𝑎𝑎𝑎𝑎𝑎

The cells in the body of the table each contain responses corresponding to one
treatment group.
These are the responses in treatment group 1 (when factor A is at level 1 and
factor B is at level 1)
Each treatment group is made up of 𝑟𝑟 responses known as replications.
Factor A consists of 𝑎𝑎 categories/levels and factor B of 𝑏𝑏 categories/levels.
 Statistical Model for a two-way ANOVA (CRD):
𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝛼𝛼𝑖𝑖 +𝛽𝛽𝑗𝑗 + 𝛼𝛼𝛼𝛼𝑖𝑖𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 Note: If there is only 1
replicate per treatment
= 𝜇𝜇𝑖𝑖𝑖𝑖 + 𝜖𝜖𝑖𝑖𝑖𝑖𝑖𝑖 group, then 𝑟𝑟 = 1 and
sometimes the subscript 𝑘𝑘
 𝜇𝜇 is the usual overall population mean; falls away.

 𝛼𝛼𝑖𝑖 is the effect of the 𝑖𝑖 𝑡𝑡𝑡 level of factor A (known as the main effect of factor A);
 𝛽𝛽𝑗𝑗 is the effect of the 𝑗𝑗𝑡𝑡𝑡 level of factor B (known as the main effect of factor B);
 𝛼𝛼𝛼𝛼𝑖𝑖𝑖𝑖 = 𝛼𝛼𝛼𝛼 𝑖𝑖𝑖𝑖 is the interaction effect of the 𝑖𝑖 𝑡𝑡𝑡 level of factor A and 𝑗𝑗𝑡𝑡𝑡 level of
factor B;
 𝜇𝜇𝑖𝑖𝑖𝑖 is the mean of the combination of the 𝑖𝑖 𝑡𝑡𝑡 level of factor A and the 𝑗𝑗𝑡𝑡𝑡 level of
factor B, i.e., the 𝐴𝐴𝑖𝑖 𝐵𝐵𝑗𝑗 treatment mean;
′
 The 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 𝑠𝑠 are the random error terms which are assumed to be independently
and normally distributed with a mean of 0 and constant variance 𝜎𝜎 2 , i.e:
𝜀𝜀𝑖𝑖𝑖𝑖𝑘𝑘 ~𝑁𝑁(0, 𝜎𝜎 2 )
 Hypotheses for a Two-way ANOVA (CRD):
Because the two-way ANOVA considers the effect of two categorical
factors and their interaction effect, there are three pairs of null or
alternative hypotheses for the two-way ANOVA:

𝐻𝐻0 : There are no differences in the means of factor A

𝐻𝐻0 : There are no differences in the means of factor B
𝐻𝐻0 : There is no interaction between factors A and B

The first two hypotheses above are the same as testing the effects of each
factor:
𝐻𝐻0 : There are no main effects of factor A
𝐻𝐻0 : There are no main effects of factor B
These hypotheses can also be represented in terms of the parameters of
the statistical model…
147
 Hypotheses for a Two-way ANOVA (CRD):
𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝛼𝛼𝑖𝑖 + 𝛽𝛽𝑗𝑗 + 𝛼𝛼𝛽𝛽𝑖𝑖𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖

For the main effects of factor A:

𝐻𝐻0 : 𝛼𝛼1 = 𝛼𝛼2 = … = 𝛼𝛼𝑎𝑎 = 0 versus 𝐻𝐻1 : Not all 𝛼𝛼𝑖𝑖′ s are equal to zero

For the main effects of factor B:

𝐻𝐻0 : 𝛽𝛽1 = 𝛽𝛽2 = … = 𝛽𝛽𝑏𝑏 = 0 versus 𝐻𝐻1 : Not all 𝛽𝛽𝑗𝑗′ s are equal to zero

For the interaction effects of factor A and B:

′
𝐻𝐻0 : 𝛼𝛼𝛼𝛼11 = 𝛼𝛼𝛼𝛼12 = … = 𝛼𝛼𝛼𝛼𝑎𝑎𝑎𝑎 = 0 versus 𝐻𝐻1 : Not all 𝛼𝛼𝛼𝛼𝑖𝑖𝑖𝑖 s are equal to zero

148
 Assumptions of a Two-way ANOVA (CRD):

o The same assumptions for a CRD must hold:

 The experimental units are identical w.r.t. their characteristics that can
affect the response of interest.
 There are no external variables that can affect the response of
interest.
o As well as the assumptions of an ANOVA must hold:
 Samples are independent and random (results of one sample do not
affect those of other samples).
 Samples are obtained from a population of values that are normally
distributed with each population having the same variance 𝝈𝝈𝟐𝟐
(constant variance).

149
 Analysis of a Two-way ANOVA:
o As usual in an ANOVA, the total sum of squares 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 is partitioned into
the various sources of variation.
o The same procedure is applied in a two-way ANOVA for a CRD, where
there are still two sources of variation:
1. Variation due to the treatments
2. Variation due to random error (natural variation)
o However, since the treatments are made up of a combination of
effects of factors A and B, the variation due to the treatments can be
partitioned as follows:
 The main effects of factor A
 The main effects of factor B
 The interaction effects between factor A and factor B
150
o Therefore, the total sum of squares is partitioned into 4 components:

𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑆𝑆𝑆𝑆𝑆𝑆 + 𝑆𝑆𝑆𝑆𝑆𝑆 + 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 + 𝑆𝑆𝑆𝑆𝑆𝑆 ~ 𝑎𝑎𝑎𝑎𝑎𝑎 − 1 df

𝑛𝑛 = 𝑎𝑎𝑎𝑎𝑎𝑎 = the total
𝑆𝑆𝑆𝑆𝑆𝑆 number of
observations
𝑆𝑆𝑆𝑆𝑆𝑆 ~ 𝑎𝑎 − 1 df
𝑆𝑆𝑆𝑆𝐵𝐵 ~ 𝑏𝑏 − 1 df

⇒𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑆𝑆𝑆𝑆𝑆𝑆 ~ 𝑎𝑎𝑎𝑎 − 1 − 𝑎𝑎 − 1 − 𝑏𝑏 − 1 = (𝑎𝑎 − 1)(𝑏𝑏 − 1) df

⇒𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 ~ 𝑎𝑎𝑎𝑎𝑎𝑎 − 1 − 𝑎𝑎𝑎𝑎 − 1 = 𝑎𝑎𝑎𝑎(𝑟𝑟 − 1) df

or 𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 − 𝑆𝑆𝑆𝑆𝑆𝑆

151
 The sum of squares on the previous slide are divided by their respective
degrees of freedom to obtain the mean sums of squares for each source
of effect.
 Each mean sum of squares is then divided by the mean error sum of
squares (𝑀𝑀𝑀𝑀𝑀𝑀) to obtain the F-test statistic used in carrying out each of
the hypothesis tests concerning the main and interaction effects.

As usual, the critical value corresponding to each F-statistic will have a
degrees of freedom of 𝑑𝑑𝑑𝑑1 = the numerator df and 𝑑𝑑𝑑𝑑2 = 𝑎𝑎𝑎𝑎(𝑟𝑟 − 1) = the
df of 𝑀𝑀𝑀𝑀𝑀𝑀.
Each null hypothesis is rejected if the respective calculated F-statistic is
greater than the corresponding critical value, or if the P-value is less than
the level of significance.
152
 A table for a two-way ANOVA CRD:

Source of Sum of Mean sum

df F-statistic
Variation Squares of squares
𝑀𝑀𝑀𝑀𝑀𝑀
Factor A 𝑆𝑆𝑆𝑆𝑆𝑆 𝑎𝑎 − 1 𝑀𝑀𝑀𝑀𝑀𝑀 𝐹𝐹𝐴𝐴 =
𝑀𝑀𝑀𝑀𝑀𝑀
𝑀𝑀𝑀𝑀𝑀𝑀
Factor B 𝑆𝑆𝑆𝑆𝑆𝑆 𝑏𝑏 − 1 𝑀𝑀𝑀𝑀𝑀𝑀 𝐹𝐹𝐵𝐵 =
𝑀𝑀𝑀𝑀𝑀𝑀
𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀
Interaction AB 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 (𝑎𝑎 − 1)(𝑏𝑏 − 1) 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 𝐹𝐹𝐴𝐴𝐴𝐴 =
𝑀𝑀𝑀𝑀𝑀𝑀
Error 𝑆𝑆𝑆𝑆𝑆𝑆 𝑎𝑎𝑎𝑎(𝑟𝑟 − 1) 𝑀𝑀𝑀𝑀𝑀𝑀

Total 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝑎𝑎𝑎𝑎𝑎𝑎 − 1

153
Example:
The yield of a chemical process is being studied. The two most important variables that
are thought to affect the yield of the chemical process are the pressure and the
temperature. Three levels of each factor are selected and a factorial experiment in a
CRD with two replications is performed. The yield data follow. The numbers in ( ) are the
cell totals. Pressure
200 215 230

90.4 90.2 90.7 90.6 90.2 90.4

20
(180.6) (181.3) (180.6)
90.1 90.3 90.5 90.6 89.9 90.1
Temperature 30
(180.4) (181.1) (180.0)
90.5 90.7 90.8 90.9 90.4 90.1
40
(181.2) (181.7) (180.5)

𝑎𝑎 = 3 ; 𝑏𝑏 = 3 ; 𝑟𝑟 = 2 𝑖𝑖 = 1,2, 3 ; 𝑗𝑗 = 1, 2 , 3 ; 𝑘𝑘 = 1, 2
𝑛𝑛 = 𝑎𝑎𝑎𝑎𝑎𝑎 = 3 × 3 × 2 = 18 is the total number of observations
154
NOTE:
 The analysis of a two-way ANOVA usually begins by checking the
presence of a factor A by factor B interaction (or B by A interaction)
using a graph of the average responses versus the levels of the factors.
 We can do this by plotting the average response of a treatment for
each level of factor B according to factor A (or visa versa).
 Let’s do this for the pressure and temperature example from the previous
slide. The average response was calculated for each treatment as
follows: Pressure
200 215 230

20 90.30 90.65 90.30

Temperature 30 90.20 90.55 90.00

40 90.60 90.85 90.25

155
 In constructing the graph in order to check for the presence of an
interaction, the 3 different levels of pressure are represented on the x-axis.
 The average responses are represented Average Response
on the y-axis using an appropriate scale. 91.00

 For each level of temperature, the average 90.80

response is plotted corresponding to each
level of pressure. 90.60

Pressure
90.40
200 215 230

20 90.30 90.65 90.30 90.20

Temperature 30 90.20 90.55 90.00 90.00

200 215 230
40 90.60 90.85 90.25
Pressure
156
  Continue to plot these points for each temperature level…

Average Response
91.00
Temp = 40
Pressure
90.80
200 215 230
90.60
20 90.30 90.65 90.30
Temperature 30 90.20 90.55 90.00 90.40
Temp = 20
40 90.60 90.85 90.25
90.20

Temp = 30
90.00
200 215 230

Pressure

157
 From this plot, we can see that there is possibly a pressure by temperature
interaction effect.
 i.e. the treatments for temperature =
Average Response 40 perform the best, except when
91.00 pressure is at 230.
Temp = 40
 Thus, the performance of a treatment
90.80
at a particular temperature may be
90.60
dependent on the pressure – which is
due to the interaction between
Temp = 20 temperature and pressure.
90.40

90.20 However, we would need to confirm

whether this interaction effect is
Temp = 30
90.00 significant using a hypothesis test.
200 215 230

Pressure 158
If no interaction was present, the lines for each temperature would be parallel
(never intersecting). For example:

Average Response  This means that the effect of the

91.00 pressure on temperature remains
Temp = 40
the same no matter what the
90.80 temperature level, and thus there is
NO interaction effect.
90.60

90.40 Temp = 20

90.20 Temp = 30

90.00
200 215 230

Pressure
159
Let’s specify the statistical model for this example (specific to the question):

𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝛼𝛼𝑖𝑖 + 𝛽𝛽𝑗𝑗 + 𝛼𝛼𝛽𝛽𝑖𝑖𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 𝑖𝑖 = 1, 2, 3; 𝑗𝑗 = 1, 2, 3; 𝑘𝑘 = 1, 2

NOTE: When you specify the statistical model, you must also indicate the ranges
of the subscripts, i.e. 𝑖𝑖, 𝑗𝑗, 𝑘𝑘

 𝜇𝜇 is the usual overall population mean

 𝛼𝛼𝑖𝑖 is the effect of the 𝑖𝑖 𝑡𝑡𝑡 level of temperature (factor A)
 𝛽𝛽𝑗𝑗 is the effect of the 𝑗𝑗𝑡𝑡𝑡 level of pressure (factor B)
 𝛼𝛼𝛼𝛼𝑖𝑖𝑖𝑖 = 𝛼𝛼𝛼𝛼 𝑖𝑖𝑖𝑖 is the interaction effect of the 𝑖𝑖 𝑡𝑡𝑡 level of temperature and 𝑗𝑗𝑡𝑡𝑡
level of pressure
′
 The 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 𝑠𝑠 are the random error terms, where 𝜀𝜀𝑖𝑖𝑖𝑖𝑘𝑘 ~𝑁𝑁(0, 𝜎𝜎 2 )
160
The ANOVA table for this example of a two-way ANOVA for a CRD:

Source of Sum of Mean sum of

df F-statistic
Variation Squares squares

2 0.301 𝑀𝑀𝑀𝑀𝑀𝑀𝑒𝑒𝑒𝑒𝑒𝑒
Temperature 0.301 = 0.151 𝐹𝐹 = = 8.389
2 𝑀𝑀𝑀𝑀𝑀𝑀
0.768 𝑀𝑀𝑀𝑀𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃
Pressure 0.768 2 = 0.384 𝐹𝐹 = = 21.333
2 𝑀𝑀𝑀𝑀𝑀𝑀
0.069 𝑀𝑀𝑀𝑀𝑀𝑀𝑒𝑒𝑒𝑒𝑒𝑒 ∗ 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃
Temp*Press 0.069 4 = 0.017 𝐹𝐹 = = 0.944
4 𝑀𝑀𝑀𝑀𝑀𝑀
0.160
Error 0.160 9 = 0.018
9
Total 1.298 17

161
Before carrying out a hypothesis test concerning the main effects of each
factor, we first need to determine if there is a significant interaction effect
between the factors:

′
𝐻𝐻0 : All the 𝛼𝛼𝛽𝛽𝑖𝑖𝑖𝑖 𝑠𝑠 equal zero (there are no interaction effects)

′
𝐻𝐻1 : Not all the 𝛼𝛼𝛽𝛽𝑖𝑖𝑖𝑖 𝑠𝑠 equal zero (there are interaction effects)

The individual effects of factor A and factor B are only meaningful if

there are NO interaction effects between the two factors.
Therefore, only if the null hypothesis concerning the interaction effects is
NOT rejected, can we carry out hypothesis tests concerning the main
effects of each of the factors.

162
 For the interaction effects (using a 5% level of significance):

The P-value corresponding to this F-value is 0.4816 (from software). Since

P-value > 0.05, 𝐻𝐻0 is NOT rejected at a 5% l.o.s.

Therefore, we can conclude that there are no interaction effects present

between temperature and pressure.

 Since there are NO interaction effects present between temperature and

pressure, we are now able to perform hypothesis tests concerning the
effects of temperature and pressure on the yield of the chemical process
separately – the results of which will be meaningful and interpretable.
163
Main Effects:
Main effects of factor A (Temperature):

𝐻𝐻0 : All the 𝛼𝛼𝑖𝑖 ′𝑠𝑠 equal zero (The temperature has no effect on the response)
𝐻𝐻1 : Not all the 𝛼𝛼𝑖𝑖 ′𝑠𝑠 equal zero (The temperature has a significant effect on
the response)

The P-value corresponding to this F-value is .0088 (obtained from software).

Therefore, since P-value < 0.05, 𝐻𝐻0 is rejected at a 5% l.o.s.

Therefore, we can conclude the factor ‘temperature’ has a significant

effect on the response (yield of the chemical process).

164
Main effects of factor B (Pressure):

𝐻𝐻0 : All the 𝛽𝛽𝑗𝑗 ′𝑠𝑠 equal zero (The pressure has no effect on the response)

𝐻𝐻1 : Not all the 𝛽𝛽𝑗𝑗 ′𝑠𝑠 equal zero (The pressure has a significant effect on the
response)

The P-value corresponding to this F-value is 0.0004 (from software), therefore

since P-value < 0.05, 𝐻𝐻0 is rejected at a 5% l.o.s.

Therefore, we can conclude the factor ‘pressure’ has a significant effect

in the response.

165
 Which Treatment Means Differ?
Recall from an ANOVA test:

𝐻𝐻0: 𝜇𝜇1 = 𝜇𝜇2 = ⋯ = 𝜇𝜇𝑘𝑘 (The mean responses of 𝑘𝑘 groups are all equal)

𝐻𝐻1: At least two 𝜇𝜇𝑖𝑖′ 𝑠𝑠 differ (𝑖𝑖 = 1, 2, … , 𝑘𝑘)

• Thus, when the null hypothesis is rejected, we can conclude at least two
means differ.
• However, the alternative hypothesis doesn’t tell us which means differ.
• We can determine which pairs of means differ significantly by:
 Estimating differences in treatment means using confidence intervals,
 Performing pairwise comparisons.
 Confidence Intervals
 Recall in Section 4.5, we could use a confidence interval for 𝜇𝜇𝑖𝑖 − 𝜇𝜇𝑗𝑗 to
determine if 𝜇𝜇𝑖𝑖 and 𝜇𝜇𝑗𝑗 are significantly different (if 0 is not contained in the
confidence interval).
 A confidence interval can then be constructed for every pair of means to
determine which two differ significantly.
 The problem with this procedure is that the probability of making a Type I
error (concluding that two means differ when in fact they are equal) is 𝛼𝛼 for
each confidence interval.
 So if you compare a large number of pairs of means, the probability of
detecting at least one difference in the means, when in fact none exist (i.e.
making a Type I error), is quite large.
 Multiple Comparisons Tests
 A simple way to avoid the high risk of making a Type I error is to use a
multiple comparisons test (also known as post-hoc tests) which adjusts for
these multiple pairwise comparisons of means that are being made.
 Examples of such pairwise comparisons are
o Tukey’s studentized range test (HSD - Honest Significant Difference) – only
used when the sample sizes in the groups being compared are equal.
o Tukey-Kramer test, which is used when the sample sizes between the
groups are unequal
o Bonferroni t-tests
 These methods make the overall probability of making a Type I error equal 𝛼𝛼
so that each individual comparison has a probability of a Type I error lower
than 𝛼𝛼.
 These multiple comparisons methods are able to group the means
according to which are significantly different or not.
 In addition, these methods can be used to obtain confidence intervals for
the difference between all pairs of means, where the overall probability of
making a Type I error is adjusted accordingly.

Checking Model Assumptions

 One of the most important assumptions of an analysis of variance is that the
data is adequately described by the statistical model discussed and that
the errors of the model (𝜀𝜀𝑖𝑖𝑖𝑖 ’𝑠𝑠) are independently and normally distributed
with a mean of 0 and constant variance 𝜎𝜎 2 .
 If these assumptions are not met, then the results of the ANOVA are invalid.
• There are graphical and statistical methods of checking these assumptions.
The most common methods (to be discussed here) are based on the
estimation of the errors called residuals (denoted by 𝜀𝜀).
̂
• 𝜀𝜀̂ is calculated by finding the difference between the actual observation
and the predicted observation.
Normality Assumption
• A plot of a histogram of the calculated residuals can help detect skewness
in the distribution of the residuals.
• If the errors are normally distributed with mean 0, then one would expect the
histogram of the residuals to be approximately bell-shaped about 0.
• If this is not the case, then the normality assumption is violated.
• Another graphical technique is the normal probability plot (also a QQ Plot).
If the errors are normally distributed then these plots should be
approximately a straight line.
Constant Variance Assumption
• This assumption can be checked by plotting the residuals (𝜀𝜀)̂ against the
estimated treatment means/predicted responses.
• If there is no obvious pattern in this plot (an increase or decrease of the
residuals with the means) then this assumption is not violated.
• A formal statistical test, such as Levene’s test for equal variances, can also
be used to check this assumption. The null hypothesis for these tests is that
all the population variances are equal.

Independence Assumption

• Based on the domain of study, the researcher can ensure they are not
working with samples that are dependent. For example data from siblings.
• In addition, when observations are collected at successive time points, the
errors in the responses may become related through time.
 Independence Assumption…

• E.g: the skill of an experimenter may change as the experiment progresses

resulting in a systematic change in the error variance over time, thus the
assumption of independent errors would be violated.
• This violation can be detected by plotting the residuals against time.
• If there is no obvious pattern in this plot (an increase or decrease of the
residuals with the time) then this assumption is not violated.