COMPARING TWO RELATED SAMPLES: THE WILCOXON SIGNED

RANK AND THE SIGN TEST

INTRODUCTION
In educational research and classroom assessment, teachers often deal with data that do not
meet the assumptions required for traditional statistical tests such as the t-test. These situations
arise when sample sizes are small, data are not normally distributed, or measurements are
ordinal rather than numerical and to address these challenges, nonparametric tests are used.
These tests are flexible, practical, and highly applicable in real classroom settings.
This module focuses on two important nonparametric methods which are Sign Test and
Wilcoxon Signed Rank Test. These tools allow future educators to analyze paired data such as
pre-test and post-test scores, before-and-after interventions and student performance changes
over time.

LEARNING OUTCOMES

At the end of this module, you should be able to:

1. apply the Sign Test and interpret results,

2. apply the Wilcoxon Signed Rank Test in solving problems and
3. analyze and interpret paired data in educational contexts

SIGN TEST

The Sign Test is a simple nonparametric test used to determine whether there is a significant
difference between paired observations by considering only the direction of change.

It answers “Did performance improve or decline?” NOT “By how much did it change?”

When to Use

• Paired data (before vs after)

• Focus is on increase or decrease
• Data may not be numerical or normally distributed

Application

• Measuring effectiveness of teaching strategies

• Pre-test vs post-test analysis
• Behavioral or attitude changes

1
STEPS IN HYPOTHESIS TESTING

1. State the Null and Research Hypotheses

HO: p = 0.5 and H1: p ≠ 0.5

2. Set the Level of Risk (or the Level of Significance) and Choose the Appropriate
Test Statistic

3. Compute the Test Statistic

First, decide if there is a difference in 2 scores and determine if the difference is
positive or negative and put the sign of the difference in the sign column. If we count
the number of ties or “0” differences among the group, we find only two with no
difference from 2 scores. Ties are discarded.

Next, we find the X-score at and beyond where the area under our binomial
probability function is α = 0.05. Since we are performing a two-tailed test, we use
0.025 for each tail. We will calculate the probabilities associated with the binomial
distribution for p and n. Formula is given below.

𝑛!
𝑃(𝑋) = ∙ 𝑝𝑥 ∙ (1 − 𝑝)𝑛−𝑋 ,
(𝑛 − 𝑋)! 𝑋!

𝐼𝑓 𝑛𝑝 + 𝑛𝑛 ≥ 25, we use

max(𝑛𝑝 , 𝑛𝑛 ) − 0.5 (𝑛𝑝 + 𝑛𝑛 ) − 0.5

𝑧=
0.5 √𝑛𝑝 + 𝑛𝑛

Note: np is the number of positive differences and nn is the number of negative

differences.

4. Determine the Value Needed for Rejection of the Null Hypothesis

5. Interpret the Results

2
EXAMPLE 1: SMALL DATA SAMPLES

Hearing of School District A’s success with their antibullying program, School District B
has implemented the program this year to reduce bullying in their own elementary schools. The
School District B evaluates their program’s effectiveness by comparing the percentage of
successful interventions last year before the program began with the percentage of successful
interventions this year with the program in place. In the table below, the 12 elementary school
counselors, or participants, reported the percentage of successful interventions last year and
the percentage this year.

Percentage of Successful Intervention

Participant Last year This year
1 31 31
2 14 14
3 53 50
4 18 30
5 21 28
6 44 48
7 12 35
8 36 32
9 22 23
10 29 34
11 17 27
12 40 42

1. State the Null and Research Hypotheses

HO: p = 0.05 and H1: p ≠ 0.05

2. Set the Level of Risk (or the Level of Significance) and Choose the Appropriate
Test Statistic
The level of risk is at 0.05. the data samples are related or paired and sample sizes
are relatively small. Since we are comparing two related samples, we will use the
sign test

3. Compute the Test Statistic

First, decide if there is a difference in intervention score from year 1 to year 2.
Determine if the difference is positive or negative and put the sign of the difference in
the sign column. Ties are discarded. We have done this in the table below

3
 Percentage of Successful
Intervention
Participant Last year This year Difference Sign
1 31 31 (31 - 31) = 0
2 14 14 (14 - 14) = 0
3 53 50 (50 - 53) = - 3 -
4 18 30 (30 - 18) = 12 +
5 21 28 (28 - 21) = 7 +
6 44 48 (48 - 44) = 4 +
7 12 35 (35 - 12) = 23 +
8 36 32 (32 - 36) = -4 -
9 22 23 (23 - 22) = 1 +
10 29 34 (29 - 34) = 5 +
11 17 27 (27 - 17) = 10 +
12 40 42 (42 - 40) = 2 +

Now, we count the number of positive and negative differences between last year
and this year. Count the number of “+” or positive differences. When we look at the
table above, we see that eight participants showed positive differences, np = 8 and
the number of “−” or negative differences (nn) is 2.

We will calculate the probabilities associated with the binomial distribution for p =
0.5 and n = 10. We will demonstrate one of the calculations, but list the results for
each value.

𝑛!
𝑃(𝑋) = ∙ 𝑝𝑥 ∙ (1 − 𝑝)𝑛−𝑋
(𝑛 − 𝑋)! 𝑋!
10!
𝑃(0) = ∙ 0.50 ∙ (1 − 0.5)10−0 = 0.0010
(10 − 0)! 0!
10!
𝑃(1) = ∙ 0.51 ∙ (1 − 0.5)10−1 = 0.0098
(10 − 1)! 1!
10!
𝑃(2) = ∙ 0.52 ∙ (1 − 0.5)10−2 = 0.0439
(10 − 2)! 2!
10!
𝑃(3) = ∙ 0.53 ∙ (1 − 0.5)10−3 = 0.1172
(10 − 3)! 3!
10!
𝑃(4) = ∙ 0.54 ∙ (1 − 0.5)10−4 = 0.2051
(10 − 4)! 4!
10!
𝑃(5) = ∙ 0.55 ∙ (1 − 0.5)10−5 = 0.2461
(10 − 5)! 5!
10!
𝑃(6) = ∙ 0.56 ∙ (1 − 0.5)10−6 = 0.2051
(10 − 6)! 6!
10!
𝑃(7) = ∙ 0.57 ∙ (1 − 0.5)10−7 = 0.1172
(10 − 7)! 7!

4
 10!
𝑃(8) = ∙ 0.58 ∙ (1 − 0.5)10−8 = 0.0439
(10 − 8)! 8!
10!
𝑃(9) = ∙ 0.59 ∙ (1 − 0.5)10−9 = 0.0098
(10 − 9)! 9!
10!
𝑃(10) = ∙ 0.510 ∙ (1 − 0.5)10−10 = 0.0010
(10 − 10)! 0!

Finally, calculate the obtained value p by combining the two tails:

𝑃(8,9 𝑜𝑟 10) + 𝑃(0,1 𝑜𝑟 1) = 0.0439 + 0.0098 + 0.0010 + 0.0010 + 0.0098 + 0.0439 = 0.1094
𝒑 = 𝟎. 𝟏𝟎𝟗𝟒

4. Determine the Value Needed for Rejection of the Null Hypothesis

The critical value for rejecting the null hypothesis is α = 0.05 and the obtained p-
value is p = 0.1094. Since the critical value is less than the obtained value, we failed to
reject the null hypothesis.

5. Interpret the Results

We did not reject the null hypothesis, suggesting that no difference exists
between last year’s and this year’s percentages. There was no evidence of positive or
negative intervention.

5
EXAMPLE 2: LARGE DATA SAMPLES
Hearing of School District A’s success with their antibullying program, School District B
has implemented the program this year to reduce bullying in their own elementary schools. The
School District B evaluates their program’s effectiveness by comparing the percentage of
successful interventions last year before the program began with the percentage of successful
interventions this year with the program in place. In the table below, the 25 elementary school
counselors, or participants, reported the percentage of successful interventions last year and
the percentage this year.

Percentage of Successful Intervention

Participant Last year This year
1 31 31
2 14 14
3 53 50
4 18 30
5 21 28
6 44 48
7 12 35
8 36 32
9 22 23
10 29 34
11 17 27
12 40 42
13 19 33
14 37 50
15 28 20
16 15 27
17 25 27
18 38 30
19 40 51
20 30 50
21 23 45
22 41 20
23 31 49
24 28 43
25 14 30

1. State the Null and Research Hypotheses

HO: p = 0.05 and H1: p ≠ 0.05

2. Set the Level of Risk (or the Level of Significance) and Choose the Appropriate
Test Statistic
The level of risk is at 0.05. Since we are comparing two related samples, we will use
the Sign test.

6
3. Compute the Test Statistic
First, we determine the sign of the differences between last year and this year.
Table below includes the column for the sign of the difference for each participant.

Percentage of Successful
Intervention
Participant Last year This year Difference Sign
1 53 50 (50 - 53) = -3 -
2 18 43 (43 - 18) = 25 +
3 21 28 (28 - 21) = 7 +
4 44 48 (48 - 44) = 4 +
5 12 35 (35 - 12) = 23 +
6 36 32 (32 - 36) = -4 -
7 22 23 (23 - 22) = 1 +
8 29 34 (34 - 29) = 5 +
9 17 27 (27 - 17) = 10 +
10 10 42 (42 - 10) = 32 +
11 38 44 (44 - 38) = 6 +
12 37 16 (16 - 37) = - 21 -
13 19 33 (33 - 19) = 14 +
14 37 50 (50 - 37) = 13 +
15 28 20 (20 - 28) = - 8 -
16 15 27 (27 - 15) = 12 +
17 25 27 (27 - 25) = 2 +
18 38 30 (30 - 38) = - 8 -
19 40 51 (51 - 40) = 11 +
20 30 50 (50 - 30) = 20 +
21 23 45 (45 - 23) = 22 +
22 41 20 (20 - 41) = - 21 -
23 31 49 (49 - 31) = 18 +
24 28 43 (43 - 28) = 15 +
25 14 30 (30 - 14) = 16 +

Next, we count the numbers of positive and negative differences. We find six
negative differences, nn = 6, and 19 positive differences, np = 19. Since the sample
size is n ≥ 25, we will use a z-score approximation of the binomial distribution. Using
the formula below.

max(𝑛𝑝, 𝑛𝑛 ) − 0.5 (𝑛𝑝 + 𝑛𝑛 ) − 0.5 19 − 0.5 (19 + 6) − 0.5

𝑧= = = 2.40
0.5 √𝑛𝑝 + 𝑛𝑛 0.5 √19 + 6

Next, we find the one-sided p-value and multiply two times the one-sided p-value to find the
two-sided p-value:

𝑝1 = 1 − 𝐴𝑟𝑒𝑎 𝑜𝑓 𝑧 𝑠𝑐𝑜𝑟𝑒
𝑝1 = 1 − 0.9918 = 0.0082

7
 𝑝 = 2𝑝1 = 2(0.0082) = 0.016
𝑝 = 0.016
4. Determine the Value Needed for Rejection of the Null Hypothesis

Since 0.016 < 0.05. Therefore, we reject the null hypothesis.

5. Interpret the Results

We rejected the null hypothesis, suggesting that there is a difference exists
between last year’s percentages and this year’s percentages. It provides evidence that
the new bullying program is providing positive benefits toward the improvement of
student behavior as perceived by the school counselors.

8
WILCOXON SIGNED RANK TEST

The Wilcoxon Signed Rank Test is a nonparametric test used to compare paired data by
considering both:

• Direction
• Magnitude of differences

Key Idea

It improves the Sign Test by asking “How large is the change and in what direction?”

When to Use

• Paired data
• Differences are not normally distributed
• More accurate analysis is required

Application

• Evaluating learning gains

• Classroom action research
• Comparing teaching methods
• Student performance tracking

STEPS IN HYPOTHESIS TESTING

1. State the Null and Research Hypotheses

HO: μD = 0 and H1: μD ≠ 0
2. Set the Level of Risk (or the Level of Significance) and Choose the Appropriate
Test Statistic
3. Compute the Test Statistic
First, compute the difference between each sample pair. Then, rank the absolute
value of those computed differences. Using this method, the differences of zero are
ignored when ranking. Compute the sum of ranks with positive differences. Compute
the sum of ranks with negative differences. Finally, the obtained T value is the
smaller of the two rank sums.

If the sample size is larger than 20, we will approximate it to a normal distribution.
Therefore, we will find a z-score for our data using a normal approximation. We must
find the mean and the standard deviation for the data.

𝑛(𝑛 + 1) 𝑛(𝑛 + 1)(2𝑛 + 1) 𝑇 − 𝑥̅ 𝑇

𝑥̅ 𝑇 = , 𝑠𝑇 = √ , 𝑧=
4 24 𝑠𝑇

9
 4. Determine the Value Needed for Rejection of the Null Hypothesis
5. Interpret the Results

EXAMPLE 1: SMALL DATA SAMPLES

Hearing of School District A’s success with their antibullying program, School District B
has implemented the program this year to reduce bullying in their own elementary schools. The
School District B evaluates their program’s effectiveness by comparing the percentage of
successful interventions last year before the program began with the percentage of successful
interventions this year with the program in place. In the table below, the 12 elementary school
counselors, or participants, reported the percentage of successful interventions last year and
the percentage this year.

Percentage of Successful Intervention

Participant Last year This year
1 31 31
2 14 14
3 53 50
4 18 30
5 21 28
6 44 48
7 12 35
8 36 32
9 22 23
10 29 34
11 17 27
12 40 42

6. State the Null and Research Hypotheses

HO: μD = 0 and H1: μD ≠ 0

10
7. Set the Level of Risk (or the Level of Significance) and Choose the Appropriate
Test Statistic
The level of risk is at 0.05. the data samples are related or paired and sample sizes
are relatively small. Since we are comparing two related samples, we will use the
Wilcoxon signed rank test. For critical value, we use Table A (Tables at the last part
of this module), which lists the critical values for the Wilcoxon T. In the given, the
two participants with score differences of zero were discarded. This reduces our
sample size to n = 10. In this case, we look for the critical value under the two-tailed
test for n = 10 and α = 0.05(two-tailed). The critical value for the Wilcoxon test of is
8.

8. Compute the Test Statistic

First, compute the difference between each sample pair. Then, rank the absolute
value of those computed differences. Using this method, the differences of zero are
ignored when ranking. We have done this in the table below

Percentage of Successful
Rank
Intervention
Without
Participant Last year This year Difference Sign
Zero
1 31 31 (31 - 31) = 0
2 14 14 (14 - 14) = 0
3 53 50 (50 - 53) = - 3 3 -
4 18 30 (30 - 18) = 12 9 +
5 21 28 (28 - 21) = 7 7 +
6 44 48 (48 - 44) = 4 4.5 +
7 12 35 (35 - 12) = 23 10 +
8 36 32 (32 - 36) = -4 4.5 -
9 22 23 (23 - 22) = 1 1 +
10 29 34 (29 - 34) = 5 6 +
11 17 27 (27 - 17) = 10 8 +
12 40 42 (42 - 40) = 2 2 +

Compute the sum of ranks with positive differences.

∑ 𝑅+ = 9 + 7 + 4.5 + 10 + 1 + 6 + 8 + 2 = 47.5
Compute the sum of ranks with negative differences.
∑ 𝑅− = 3 + 4.5 = 7.5

Finally, the obtained T value is the smaller of the two rank sums.
𝑇 = 7.5

11
 9. Determine the Value Needed for Rejection of the Null Hypothesis

The critical value for rejecting the null hypothesis is 8 and the obtained value is T
= 7.5. If the critical value equals or exceeds the obtained value, we must reject the null
hypothesis. If instead, the critical value is less than the obtained value, we must not
reject the null hypothesis. Since the critical value exceeds the obtained value, we must
reject the null hypothesis.

10. Interpret the Results

We rejected the null hypothesis, suggesting that there is a difference exists
between last year’s percentages and this year’s percentages. It provides evidence that
the new bullying program is providing positive benefits toward the improvement of
student behavior as perceived by the school counselors.

EXAMPLE 2: LARGE DATA SAMPLES

Hearing of School District A’s success with their antibullying program, School District B
has implemented the program this year to reduce bullying in their own elementary schools. The
School District B evaluates their program’s effectiveness by comparing the percentage of
successful interventions last year before the program began with the percentage of successful
interventions this year with the program in place. In the table below, the 25 elementary school
counselors, or participants, reported the percentage of successful interventions last year and
the percentage this year.

Percentage of Successful Intervention

Participant Last year This year
1 31 31
2 14 14
3 53 50
4 18 30
5 21 28
6 44 48
7 12 35
8 36 32
9 22 23
10 29 34
11 17 27
12 40 42
13 19 33
14 37 50
15 28 20
16 15 27
17 25 27
18 38 30
19 40 51
20 30 50
21 23 45
22 41 20
23 31 49
24 28 43
25 14 30

12
Note: We will use the same nonparametric procedure to analyze the data. However, use a large sample
(n ≥ 20) approximation.

1. State the Null and Research Hypotheses

HO: μD = 0 and H1: μD ≠ 0

2. Set the Level of Risk (or the Level of Significance) and Choose the Appropriate
Test Statistic
The level of risk is at 0.05. Since we are comparing two related samples, we will use
the Wilcoxon signed rank test. For a two-tailed test with α = 0.05, we must not reject
the null hypothesis if −1.96 ≤ z ≤ 1.96.

3. Compute the Test Statistic

First, compute the difference between each sample pair. Then, rank the absolute
value of those computed differences. Using this method, the differences of zero are
ignored when ranking. We have done this in the table below

Percentage of Successful
Rank
Intervention
Participant Last year This year Difference Without Zero Sign
1 53 50 (50 - 53) = -3 3 -
2 18 43 (43 - 18) = 25 24 +
3 21 28 (28 - 21) = 7 8 +
4 44 48 (48 - 44) = 4 4.5 +
5 12 35 (35 - 12) = 23 23 +
6 36 32 (32 - 36) = -4 4.4 -
7 22 23 (23 - 22) = 1 1 +
8 29 34 (34 - 29) = 5 6 +
9 17 27 (27 - 17) = 10 11 +
10 10 42 (42 - 10) = 32 25 +
11 38 44 (44 - 38) = 6 7 +
12 37 16 (16 - 37) = - 21 20.5 -
13 19 33 (33 - 19) = 14 15 +
14 37 50 (50 - 37) = 13 14 +
15 28 20 (20 - 28) = - 8 9.5 -
16 15 27 (27 - 15) = 12 13 +
17 25 27 (27 - 25) = 2 2 +
18 38 30 (30 - 38) = - 8 9.5 -
19 40 51 (51 - 40) = 11 12 +
20 30 50 (50 - 30) = 20 19 +
21 23 45 (45 - 23) = 22 22 +
22 41 20 (20 - 41) = - 21 20.5 -
23 31 49 (49 - 31) = 18 18 +
24 28 43 (43 - 28) = 15 16 +
25 14 30 (30 - 14) = 16 17 +

13
 Compute the sum of ranks with positive differences.

∑ 𝑅+ = 24 + 8 + 4.5 + 23 + 1 + 6 + 11 + 25 + 7 + 15 + 14 + 13 + 2 + 12 +

19 + 22 + 18 + 16 + 17 = 257 .5
Compute the sum of ranks with negative differences.
∑ 𝑅− = 3 + 4.5 + 9.5 + 9.5 + 20.5 + 20.5 = 67.5

Then, the obtained T value is the smaller of the two rank sums.
𝑇 = 67.5
Since the sample size is larger than 20, we will approximate it to a normal
distribution. Therefore, we will find a z-score for our data using a normal
approximation. We must find the mean and the standard deviation for the data.

𝑛(𝑛 + 1) 25(25 + 1)
𝑥̅ 𝑇 = = = 162.50
4 4

𝑛(𝑛 + 1)(2𝑛 + 1) 25 (25 + 1)(2(25) + 1)

𝑠𝑇 = √ = √ = 37.17
24 24

𝑇 − 𝑥̅ 𝑇 67.5 − 162.50
𝑧= = = −2.56
𝑠𝑇 37.17

4. Determine the Value Needed for Rejection of the Null Hypothesis

Since −2.56 < −1.96. Therefore, we reject the null hypothesis.

5. Interpret the Results

We rejected the null hypothesis, suggesting that there is a difference exists
between last year’s percentages and this year’s percentages. It provides evidence that
the new bullying program is providing positive benefits toward the improvement of
student behavior as perceived by the school counselors.

14
ASSESSMENT

TASK 1: “Analyze My Data”

You are given a dataset of students’ pre-test and post-test scores. Perform the both
methods. Show your complete solution.

Student Pre-Test Post-Test

1 65 70
2 70 68
3 75 80
4 60 65
5 68 72
6 72 75

Rubric for grading

Score Level Description
18–20 Excellent Complete and accurate solutions for both Sign Test and Wilcoxon Test;
all steps are clearly shown; interpretation is correct, insightful, and
demonstrates strong statistical reasoning
14–17 Proficient Minor computational errors; most steps are clear and correct;
interpretation is generally accurate with acceptable reasoning
10–13 Developing Partial or inconsistent solutions; some steps missing or unclear;
interpretation is weak or partially incorrect
0–9 Beginning Incomplete or incorrect computations; lacks understanding of
procedures; interpretation is missing or incorrect

TASK 2: Classroom Research Simulation

A group of teachers wants to determine whether a new teaching strategy improved

student performance. The following are the pre-test and post-test scores of students.

Student Before After Student Before After

1 60 65 16 70 70
2 75 78 17 68 72
3 70 68 18 75 77
4 80 85 19 64 69
5 68 70 20 79 82
6 72 74 21 67 65
7 65 67 22 72 74
8 78 80 23 76 80
9 69 72 24 63 68
10 74 76 25 74 77
11 66 64 26 69 73
12 71 75 27 71 76
13 77 79 28 66 70
14 62 66 29 78 81
15 73 78 30 70 75

15
Each group consist of 3-5 members will:

1. Perform the 2 methods

2. Compare Results

• Do both tests give the same conclusion?

• Which test is more reliable in this case?
4. Interpretation & Recommendation
Answer:

• Is the teaching strategy effective?

• Should it be implemented in other classes? Why?

Output

• Complete solution (hard copy)

• Short presentation/report

Rubric

Score Level Description

18–20 Excellent Highly accurate computations; strong teamwork and collaboration;
presentation is clear, organized, and engaging; comparison of tests is
insightful with well-justified recommendation
14–17 Proficient Mostly accurate work; good collaboration; presentation is clear;
interpretation and recommendation are reasonable
10–13 Developing Some computational errors; uneven participation; presentation lacks
clarity; interpretation is limited or unclear
0–9 Beginning Incorrect or incomplete analysis; poor collaboration; unclear or missing
presentation; lacks understanding of concepts

16
TASK 3: “From Numbers to Meaning”
Write a short reflection (150–200 words) answering:
1. What is the difference between the Sign Test and Wilcoxon Test?
2. Which test do you think is more useful in classroom research? Why?
3. How can you apply these tests as a future teacher?

Score Level Description

18–20 Excellent Deep and insightful reflection; clearly explains differences
between tests; strong connection to teaching practice; ideas are
well-organized and meaningful
14–17 Proficient Good reflection; shows understanding of concepts; some
connection to teaching application; ideas are generally clear
10–13 Developing Limited reflection; explanations are unclear or incomplete; weak
connection to teaching practice
0–9 Beginning Minimal, off-topic, or unclear response; shows little to no
understanding of the concepts

REFERENCES
Conover, W. J. (1999). Practical Nonparametric Statistics. Wiley.
Siegel, S., & Castellan, N. J. (1988). Nonparametric Statistics for the Behavioral
Sciences. McGraw-Hill.
Walpole, R. E., Myers, R. H., Myers, S. L., & Ye, K. (2012). Probability and
Statistics for Engineers and Scientists. Pearson.

17
TABLE A Critical Values for the Wilcoxon Signed Rank Test Statistics T.

18
19