Week 4 tutorial

Outline
• Questions regarding quiz 1

• 1-way ANOVA

• Jamovi
Quiz 1 grade is out
1-way ANOVA
ANOVA
• ANOVA, stands for Analysis of Variance, is an inferential
parametric statistical test for comparing the means of three or
more groups.

• Focus on Variance
• ANOVA compares the amount of variance caused by manipulation (IV) vs
random error
How does 1-way ANOVA work?
• Instead of t-test, ANOVA relies on the F-ratio test, or F-test for short.
The F-ratio for 1-way ANOVA

𝐵𝑒𝑡𝑤𝑒𝑒𝑛 𝑔𝑟𝑜𝑢𝑝𝑠 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒

𝐹=
𝑊𝑖𝑡ℎ𝑖𝑛 𝑔𝑟𝑜𝑢𝑝𝑠 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒

• The between-groups variance is an estimate of the effect of the independent

variable and error variance.
• Variance due to manipulation + error variance
• The within-groups variance is an estimate of the population error variance.
• Error variance
How to interpret the F-ratio?
The F-ratio for 1-way ANOVA
𝐵𝑒𝑡𝑤𝑒𝑒𝑛 𝑔𝑟𝑜𝑢𝑝𝑠 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑀𝑎𝑛𝑖𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 + 𝐸𝑟𝑟𝑜𝑟
𝐹= =
𝑊𝑖𝑡ℎ𝑖𝑛 𝑔𝑟𝑜𝑢𝑝𝑠 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝐸𝑟𝑟𝑜𝑟

• When manipulation (IV) did not have any effect:

• F = (0 + error) / error = 1
• When manipulation (IV) has an effect:
• F = (large number + error) / error > 1
• If the F-ratio score is significantly above 1, we can claim IV has an effect
 Summary formula for ANOVA table

Source SS df MS Fcalc Fcrit

Between (σ 𝑋𝑔 )2 (σ 𝑋)2 𝑆𝑆𝐵 𝑀𝑆𝐵

𝑆𝑆𝐵 = σ[ ] − k–1 F(dfB,dfW)
(B) 𝑛𝑔 𝑁 𝑑𝑓𝐵 𝑀𝑆𝑊

Within σ 𝑋𝑔
2
𝑆𝑆𝑊
2
𝑆𝑆𝑊 = σ 𝑋 − σ[ ] N-K - -
(W) 𝑛𝑔 𝑑𝑓𝑊

(σ 𝑋)2 𝑆𝑆𝐴
Total 𝑆𝑆𝑇 = σ 𝑋2 − N-1 2
ɳ =
𝑆𝑆𝑇
𝑁
 1-way ANOVA practice
• A researcher wonders whether students’ test scores are affected by
the clothing their profs wear. One group is taught by a prof who
wears blue jeans, another group is taught by a prof who wears
suits, and a third group is taught by a prof who wears a lab coat. Is
there any difference in test scores based on the profs’ attire?
Jeans (J) Suits (S) Lab Coat (L)
6 3 1
4 3 0
3 0 1
4 2 2
3 2 1
 J S L

Solving 1-way ANOVA, steps 1-3 6

4
3
3
1
0
3 0 1
𝟐 𝟐 𝟐 𝟐 4 2 2
σ 𝒙𝑱 σ 𝒙𝑺 σ 𝒙𝑳 σ 𝒙𝒕𝒐𝒕𝒂𝒍
𝑺𝑺𝑩 = + + − 3 2 1
𝒏𝑱 𝒏𝑺 𝒏𝑳 𝒏𝒕𝒐𝒕𝒂𝒍
σ 𝒙𝑱 = 20 σ 𝒙𝒔 = 10 σ 𝒙𝑳 = 5
20 2 10 2 5 2 20+10+5 2
= + + − = [80 + 20 + 5] – 81.67 = 23.33
5 5 5 15

𝟐
σ 𝒙𝑱 σ 𝒙𝑺 𝟐 σ 𝒙𝑳 𝟐
𝑺𝑺𝒘 = σ 𝒙𝟐 − 𝒏𝑱
+ 𝒏𝑺
+ 𝒏𝑳
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 20 2 10 2 5 2
= 6 +4 +3 +4 +3 +3 +3 +0 +2 +2 +1 +0 +1 +2 +1 − + +
5 5 5
= 119 – [80 + 20 + 5 ] = 14

σ𝒙 𝟐
𝑺𝑺𝑻𝒐𝒕𝒂𝒍 = σ 𝒙𝟐 −
𝒏𝒕𝒐𝒕𝒂𝒍
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 20+10+5 2
= 6 +4 +3 +4 +3 +3 +3 +0 +2 +2 +1 +0 +1 +2 +1 -
15
= 119 – 81.67 = 37.33
Steps 4, find the df for each row
dfB = k – 1 = 3 – 1 =2

dfW = N – k = 15 – 3 = 12

dfT = N – 1 = 15 – 1 = 14

Source SS df MS Fcalc Fcrit

Between (B) 23.33 2
Within (W) 14 12
Total 37.33 14
Steps 5 and 6, find MS and Fcalc
MSbetween = SSbetween / dfbetween = 23.33/2 = 11.67
MSwithin = SSwithin / dfwithin = 14/12 = 1.17

𝑴𝑺𝑨
𝑭𝒄𝒂𝒍𝒄 = = 11.67 / 1.17 = 9.97
𝑴𝑺𝑾

Source SS df MS Fcalc Fcrit

Between (B) 23.33 2 11.67 9.97
Within (W) 14 12 1.17
Total 37.33 14
 Step 7, find Fcrit
• dfb = 2 (horizontal)
• dfw = 12 (vertical)
• Alpha = 0.05

Fcrit = F(2,12) = 3.89

SS df MS Fcalc Fcrit
B 23.33 2 11.67 9.97 * 3.89
W 14 12 1.17
T 37.33 14
Step 8, find effect size
𝑆𝑆𝐵 23.33
• ɳ2 = = = 0.62
𝑆𝑆𝑇 37.33
• 62% of variance in DV explained by the IV

SS df MS Fcalc Fcrit
B 23.33 2 11.67 9.97 3.89
W 14 12 1.17
T 37.33 14
Conclusion
SS df MS Fcalc Fcrit

B 23.33 2 11.67 9.97 3.89

W 14 12 1.17
T 37.33 14 ɳ2 = 0.62

Results showed that profs wearing different attires had a significant

effect on the student test scores, F(2,12) = 9.97, p < 0.05, ɳ2 =
0.62. This means that 62% of the variance in test score was
associated with the differences in attires.
Tukey’s Post Hoc Test
• Tukey’s honestly significant difference (HSD) can make pairwise
comparisons when the different levels have equal sample size (n).
• It identifies the smallest difference between any two means that is significant

Q: from the Studentized Range Statistics (Q table) from

HSD Table A.5 (page 423 – 424 in textbook)
* based on level (k) and dfw
𝑀𝑆𝑤
𝐻𝑆𝐷𝑎𝑙𝑝ℎ𝑎 = 𝑄 𝑘, 𝑑𝑓𝑊 ∗ dfw = degree of freedom in the within row
𝑛
MSw = mean squares of the within row

n = sample size in each group (assuming same n)

From our professor’s attire example
𝑀𝑆𝑤 1.17
• 𝐻𝑆𝐷𝑎𝑙𝑝ℎ𝑎 = 𝑄 𝑘, 𝑑𝑓𝑊 ∗ = 3.77 * = 3.77 * 0.48 = 1.82
𝑛 5
• K = 3 , dfw = 12
• Q = 3.77

• MSw = 1.17, n = 5

• HSD0.05 = 1.82
How to use this HSD value?
1. Find the mean of each group
2. For each pair of comparison, if the value of difference > HSD,
then there is significant difference between this pair of means
(ignore the sign)
Jeans vs. Suits: Mj – Ms = 4 – 2= 2 > 1.82 Sig J S L
Jeans vs. Lab coat: Mj – Ml = 4 – 1 = 3 > 1.82 Sig 6 3 1
Suits vs. Lab coat: Ms – Ml = 2 – 1 = 1 < 1.82 Not sig 4 3 0
3 0 1
The mean scores in Jeans condition is significantly 4 2 2
higher than that in the suits condition (p < 0.05) and the 3 2 1
𝑴=4 𝑴=2 𝑴=𝟏
lab coat condition (p < 0.05). However, there is no
significant difference between suit and lab coat condition
(p > 0.05).
Scheffe’s test
• Similar to HSD, can compare pairs of groups under ANOVA
• Does not require the sample size to be same across conditions

• Scheffe’s test is more conservative than HSD – it is very strict,

and it is hard to pass the test to reach significance
• Can be used when the consequences is important (i.e., do not want to
risk making type I error)
𝑥ҧ1 − 𝑥ҧ2 2
𝐹𝑠𝑐ℎ =
1 1
𝑀𝑆𝑤 𝑛 + 𝑛
1 2

Where Fsch crit = (𝑘 − 1)(𝐹𝑐𝑟𝑖𝑡 )

From our professor’s attire example
𝑥ҧ1 − 𝑥ҧ2 2
1. Find the new critical value for Scheffe’s test 𝐹𝑠𝑐ℎ =
1 1
𝑀𝑆𝑤 𝑛 + 𝑛
K = 3, Fcrit = 3.89, Fsch crit = (3 − 1)(3.89) = 7.78 1 2

Where Fsch crit = (𝑘 − 1)(𝐹𝑐𝑟𝑖𝑡 )

2. Plug in the means, MSw and n for each pair of comparison
MSw = 1.17, n = 5
4 −2 2
Jeans vs. Suits: 𝐹𝑠𝑐ℎ = = 8.55 > 7.78 Sig
1 1
1.17 5+5 The mean scores in Jeans condition is
4 −1 2
significantly higher than that in the suits
Jeans vs. Lab coats: 𝐹𝑠𝑐ℎ = 1 1 = 19.23 > 7.78 Sig condition (Fsch = 8.55, p < 0.05) and the
1.17 +
5 5 lab coat condition (Fsch = 19.23, p < 0.05).
2 −1 2 However, there is no significant difference
Suits vs. Lab coats: 𝐹𝑠𝑐ℎ = 1 1 = 2.14 < 7.78 Not sig.
1.17 5+5 between suit and lab coat condition (Fsch =
2.14, p > 0.05).
Practice question
A kindergarten teacher is curious to know if the children respond quicker to cleaning
the classrooms based on type of instructions given. They tested three different
classes (n = 15 in each) and found the mean responses below.
Demand Polite Stare
mean time (s) 16 19 22
The teacher was half-way finished on the ANOVA table before got distracted:
Source SS df MS Fcalc Fcrit ɳ2
Between
Within 168
Total 400

Can you help them figure out if type of instruction matters? and if so which one is
better? Use Tukey’s HSD for multiple comparison, alpha = 0.05.
Completing the ANOVA table
SSbetween + SSwithin = SStotal MSbetween = SSB/ dfB = 232/2 = 116
SSB + 168 = 400 , SSb = 400 – 168 = 232 MSwithin = SSw / dfw = 168/42 = 4
𝑴𝑺𝑩
dfB = k – 1 =2, 𝑭𝒄𝒂𝒍𝒄 = = 116 / 4 = 29
𝑴𝑺𝑾
dfW = N – k = 45 – 3 = 42, Fcrit(2,42) = 2.84
𝑆𝑆 232
dfT = N – 1 = 45 – 1 = 44 ɳ2 = 𝐵 = = 0.58
𝑆𝑆𝑇 400

Source SS df MS Fcalc Fcrit ɳ2

Between 232 2 116 29 * 2.84 0.58
Within 168 42 4
Total 400 44
In class activity (week 4 participation)
• Go to D2L/Assessment/Quizzes

• Find Week 4 participation

• Password: anova (all lower case)

• Complete and submit the task

- you will get the grade as long as you submitted on time.
 Multiple comparisons using HSD

𝐻𝑆𝐷𝑎𝑙𝑝ℎ𝑎 = 𝑄 𝑘, 𝑑𝑓𝑊 ∗
𝑀𝑆𝑤 D vs. P: MD – MP = 16 – 19= -3 > 1.79 Sig
𝑛
D vs. S: MD – MS = 16 – 22 = -6 > 1.79 Sig
Q(3, 42) = 3.44 P vs. S: MP – MS = 19 – 22 = -3 > 1.79 Sig
4
HSD0.05 = 3.44 * = 3.44 * 0.52 = 1.79
15

Demand Polite Stare

mean time (s) 16 19 22
 Conclusion
The type of instruction significantly affected the time it took for children to
clean the classroom, F(2,42) = 29, p < 0.05, ɳ2 = 0.58. This means 58% of the
changes in response time was associated with instruction type. Specifically,
children was faster under demanding instruction than polite instruction (p <
0.05) and staring (p <0.05), and they were faster under polite instruction than
staring (p <0.05).
Demand Polite Stare
mean time (s) 16 19 22
D vs. P: Sig
D vs. S: Sig
P vs. S: Sig
One-way ANOVA in Jamovi
 Solve the attire problem using Jamovi
• Let’s revisit the practice question we just had

Jeans (J) Suits (S) Lab Coat (L)

6 3 1
4 3 0
3 0 1
4 2 2
3 2 1
Set up the data
• The data should consist of:
• One categorical variable: the IV
• Defines the different groups in the
sample
• Example: Attire type

• At least one interval or ratio variable

(Jamovi called it continuous) : the DV
• Set as “continuous”
• The measure that you are comparing
• Example: students’ scores on test
Run the ANOVA test
• Analyses – ANOVA –
ANOVA
• Although there is an option
for “one-way ANOVA”,
choose the second ANOVA,
as it offers more
functionalities
 DV goes here

IV goes here (grouping

variable)

Make sure the following boxes are

checked:
1. Effect size – eta-squared
2. Assumption checks
1. Homogeneity test
2. Normality test (optional)
3. Post hoc tests
1. Move DV (attire) to the right box
2. Check Tukey and/or Scheffe
Reading the output
• ANOVA table
• Jamovi do not show the total
row
• Attire (IV) is the Between
• Residuals is the within
• Sum of squares = SS
• F shown is Fcalc

SS df MS Fcalc Fcrit • p = 0.003 < 0.05 sig.

B 23.33 2 11.67 9.97 3.89

W 14 12 1.17
Our hand calculation had some
error due to rounding
T 37.33 14 ɳ2 = 0.62
Assumption tests
• Homogeneity of Variance
• Levene’s test
• P > 0.05, did not violate assumption
of homogeneity of variance

• Normality Test
• Shapiro-Wilk’s test
• P > 0.05, overall distribution is normal
Post hoc tests

• Jeans is significantly different (higher) than suits and lab coats (p <0.05)
• There is no sig. difference between suits and lab coats (p > 0.05)
Next week
• Will learn Repeated measures ANOVA
• Pages 253 - 261

• Remember to submit assignment 1 (due Feb 10)