Non-parametric Methods
Parametric Methods:
• Involve estimating or testing the value of parameters --usually, population means or proportions
• Many statistical tests require that your data follow a normal distribution.
• Sometimes it is possible to transform data to make them follow a normal distribution.
• Transformation is not possible or
• the sample size is too small that to ascertain whether the data a normally distributed or not.
• Need a statistical test that does not require the data to follow a particular distribution – non-parametric or
distribution free test.
• procedures that work without reference to specific parameters.
• do not rely on the estimation of parameters.
• do not require that the data fit a normal distribution.
• allow for the analysis of categorical as well as ranked data.
• For this reason, they are often used in place of parametric tests when one feels that the assumptions of the
parametric test have been violated.
• There is non-parametric equivalent for each parametric test.
Sign test
• If there are differences in the central tendency of 2 independent groups
• If median of a collection of numbers is greater/lesser than a specified value
Mann-Whitney U Test
• compare differences between two independent groups when the DV is either ordinal or continuous, but
not normally distributed.
• Because the samples are independent, they can be of different sizes.
• This test can also be applied when the observations in a sample of data are ranks, i.e., ordinal data rather
than direct measurements
• E.g. whether salaries, measured on a continuous scale, differed based on educational level
• Attitudes towards pay discrimination, where attitudes are measured on an ordinal scale, differ based on
gender (i.e., your dependent variable would be "attitudes towards pay discrimination" and your
independent variable would be "gender"
Assumptions
• 1. DV measured at the ordinal or continuous level.
• 2. IV should consist of two categorical, independent groups.
• 3. Independence of observations in each group or between the groups themselves. Failing this assumption,
requires use of another statistical test like Wilcoxon signed-rank test.
• 4. 2 variables are not normally distributed.
• The effectiveness of a new drug to reduce symptoms of asthma in children. A total of n=10 participants are
randomized to receive either the new drug or a placebo. Participants are asked to record the number of
episodes of shortness of breath over a 1 week period following receipt of the assigned treatment. The data are
shown below.
• Is there a difference in the number of episodes of shortness of breath over a 1 week period in participants
receiving the new drug as compared to those receiving the placebo?
Placebo 7 5 6 4 12
New Drug 3 6 4 2 1
• Participants receiving the placebo appear to have more episodes of shortness of breath, but is this statistically
significant?
• Step 1: Assign Ranks to order the data
• Step 2: Sum ranks in each group
• Step 3: Sum of the ranks = n(n+1) /2
• Step 4: Calculate U
where R1 = sum of the ranks for group 1.
R2 = sum of the ranks for group 2.
Total Sample Ranks
(Ordered Smallest to
Largest)
Placebo New Drug Placebo New Drug Placebo New Drug
7 3 1 1
5 6 2 2
6 4 3 3
4 2 4 4 4.5 4.5
12 1 5 6
6 6 7.5 7.5
7 9
12 10
• Sum of ranks = 37+18 = 55.
• Smaller values of U support the research hypothesis, and larger values of U support the null hypothesis.
• Range of U is from 0 (complete separation between groups, H0 most likely false and H1 most likely true) to
n1*n2 (little evidence in support of H1).
• U1+U2 is always equal to n1*n2.
• The critical value of U can be found in the table using sample sizes.
Wilcoxon Signed rank test
• Non-parametric counterpart to the dependent samples t-test
• When assumptions of scale used and normal distribution are not met
• Useful when dealing with ranked or ordinal data
• The Wilcoxon test compares two paired groups and comes in two versions, the rank sum test, and signed rank
test.
• The goal of the test is to determine if two or more sets of pairs are different from one another in a statistically
significant manner.
• Both versions of the model assume that the pairs in the data come from dependent populations, i.e., following
the same person or share price through time or place.
• E.g. effectiveness of a novel teaching method on improving children’s literacy levels. They conduct a study
involving 20 children, measuring literacy on a 0 to 10 scale both before and after implementing the teaching
method. The pre-intervention average literacy score stands at 5.9, which then rises to 7.6 post-intervention.
• Step 1: Rank the Differences between the paired observations &
rank differences disregarding the sign
• Step 2: Assign Signs to Ranks based on the direction of the
difference
• Step 3: Calculate the W-Statistic with smaller of the two used to
assess the significance of the observed differences.
• Step 4: Interpret the Results using tables
W+ = 4.5 + 4.5 = 9
W- = 1+ 2 + 4.5 + 4.5 + 7.5 + 7.5 + 9 + 10.5 + 10.5 + 12.5 + 12.5 + 14 +
17+ 17 + 17 + 17 + 17 = 181
Make a compensation for ties, 4, 2, 2, 2, and 5, so we must reduce the
variance by
• critical z-value for a 95% CI
(or a 5% level of
significance) which is z =
1.96 for a two-tailed test and
directionality.
• Whenever a test is based the
normal distribution the
sample z value needs to be
1.96 or higher to reject the
null hypothesis.
Assumptions
• DV measured at the ordinal or continuous level.
• IV should consist of two categorical, "related groups" or "matched pairs".
• Distribution of the differences between the two related groups needs to be symmetrical in shape
McNemar test
• Used to determine if there are differences on a dichotomous DV between two related groups.
• Similar to the paired-samples t-test, but for a dichotomous DV.
• Unlike the paired-samples t-test, it tests 2 different properties of a repeated measure dichotomous variable
• E.g. Participants who had low vs high self-esteem before a series of counselling sessions decreased after the
intervention
• DV would be "level of self-esteem“ - "low" and "high"
• The proportion of participants who felt/did not feel safe differed when wearing a cycling helmet as opposed to
wearing no cycling helmet
• DV – "sense of safety", 2 categories - "safe" and "not safe").
Assumptions
• 1 categorical DV with two categories (dichotomous variable) and 1 categorical IV with 2 related groups.
• The two groups of DV must be mutually exclusive.
• The cases (e.g., participants) are a random sample from the population of interest. However, in practice, this is
not always how sampling took place.
• It is applied to 2 × 2 contingency tables with a dichotomous trait, with matched pairs of subjects
Kruskal-Wallis H Test
• Also called the "one-way ANOVA on ranks" is a rank-based nonparametric test that can be used to determine
if there are statistically significant differences between two or more groups of an IV on a continuous or
ordinal DV.
• It is considered the nonparametric alternative to the one-way ANOVA, and an extension of the Mann-Whitney
U test to allow the comparison of more than two independent groups.
• Eg exam performance, measured on a continuous scale, differed based on test anxiety levels
• Cannot tell which specific groups are statistically significantly different from each other; only that at least two
groups were different.
• Assumptions
• DV should be measured at the ordinal or continuous level (i.e., interval or ratio).
• IV should consist of two or more categorical, independent groups.
• Independence of observations. But does not assume normality in the data and is much less sensitive to
outliers
• To interpret the results the distributions in each group of the IV have the same shape and variability.
If distributions have a different shape, the Kruskal-Wallis H test can only compare mean ranks. Having similar
distributions allows the medians to be compared and represent a shift in location between the groups
• E.g. Does physical exercise alleviate depression?
• Each person was allocated randomly to one of three groups: no exercise; 20 minutes of jogging per day; or 60
minutes of jogging per day. At the end of a month, each participant was asked to rate how depressed they felt,
on a Likert scale that runs from 1 ("totally miserable") through to 100 (ecstatically happy").
• Step 1: Rank all of the scores, ignoring which group they belong to. The procedure for ranking is as follows:
the lowest score gets the lowest rank. If two or more scores are the same then they are "tied". "Tied" scores
get the average of the ranks that they would have obtained, had they not been tied.
• Step 2: Find "Tc", the total of the ranks for each group. Just add together all of the ranks for each group in
turn. Here, Tc1 (the rank total for the "no exercise" group) is 76.5. Tc2 (the rank total for the "20 minutes"
group) is 79.5. Tc3 (the rank total for the "60 minutes" group) is 144.
• Step 3: Find H
Step 4: the degrees of freedom is the number of groups
minus one, so df = 2
Step 5:
Assess significance of H depends on the number of
participants and the number of groups.
• Friedman test
• An extension of the Wilcoxon signed-rank test and the nonparametric analog of one-way repeated-measures
(Kruskal-Wallis).
• Friedman tests the null hypothesis that k related variables come from the same population. For each case,
the k variables are ranked from 1 to k.
• The test statistic is based on these ranks.
• Alternative to the one-way ANOVA with repeated measures.
• Used to test for differences between groups when the DV being measured is ordinal.
• It can also be used for continuous data that has violated the assumptions necessary to run the one-way
ANOVA with repeated measures (e.g., data that has marked deviations from normality).
Assumptions
• One group that is measured on three or more different occasions.
• Group is a random sample from the population.
• DV should be measured at the ordinal or continuous level.
• Samples do NOT need to be normally distributed.
• Blocks are mutually independent (i.e. all of the pairs are independent — one doesn’t affect the other),
• Observations are ranked within blocks with no ties.
Example 1
• To examine whether music has an effect on the perceived psychological effort required to perform an exercise
session. DV is "perceived effort to perform exercise" and the IV is "music type", which consists of three
groups: "no music", "classical music" & "dance music". 12 runners were recruited & each ran 3 times on a
treadmill for 30 minutes. The treadmill speed was the same for all three runs. Each subject ran: (a) listening to
no music at all; (b) listening to classical music; & (c) listening to dance music. At the end of each run,
subjects were asked to record how hard the running session felt on a scale of 1 to 10, with 1 being easy and 10
extremely hard.
Example
• Sort your data into blocks (columns in a spreadsheet).for this example, we have 12 patients getting three
different treatments.
• Rank each row separately (compare treatment 1, 2 & 3). The smallest score should get a rank of 1.
• Sum the ranks (find a total for each column).
Calculate the test statistic:
• n: the number of subjects (12)
• k: the number of treatments (3)
• R: The total ranks for each of the three columns (32, 27, 13).
Insert these into the formula:
• Find the FM critical value from the table of critical values for Friedman.
• Use the k=3 table &an alpha level of 5%.
• Looking up n-12 in that table, we find a FM critical value of 6.17.
• Compare the calculated FM test statistic to the FM critical value. Reject the null hypothesis if the calculated F
value is larger than the FM critical value.:
Calculated FM Test Statistic = 15.526.
FM Critical value from table = 6.17.
• The calculated FM statistic is larger, so you would reject the null hypothesis.
Spearman’s (rank-order) correlation
• nonparametric version of the Pearson product-moment correlation.
• Spearman's correlation coefficient, (ρ, also signified by rs) measures the strength and direction of association
between two ranked variables.
Assumptions
• 2 variables that are either ordinal, interval or ratio
• Can be used when the assumptions of the Pearson correlation are markedly violated.
• Determines the strength and direction of the monotonic relationship between 2 variables rather than the
linear relationship (Pearson's correlation)
• Monotonic relationship does one of the following: (1) as one variable increases, the other also increases; or
(2) as one variable increases, the other decreases.
•
Similar to linear relationship
but not exactly a straight line
as seen in the figures 1 & 2.
Monotonicity is "less
restrictive" than a linear
relationship.
• Example data regarding the marks achieved in a maths and English
exam:
English (mark) Maths (mark) Rank (English) Rank (maths)
56 66 9 4
75 70 3 2
45 40 10 10
71 60 4 7
61 65 6.5 5
64 56 5 9
58 59 8 8
80 77 1 1
76 67 2 3
61 63 6.5 6
• The formula for when there are no tied ranks is:
• di = difference in paired ranks and n = number of cases. The formula to use when there are tied ranks is:
• Can take values from +1 to -1.
• The closer rs is to zero, the weaker the association between the ranks.
English Maths Rank Eng Rank Mat d d2
56 66 9 4 5 25
75 70 3 2 1 1
45 40 10 10 0 0
71 60 4 7 3 9
62 65 6 5 1 1
64 56 5 9 4 16
58 59 8 8 0 0
• This indicates a strong
80 77 1 1 0 0 positive rel. the higher you
ranked in maths, the higher
76 67 2 3 1 1
you ranked in English also,
61 63 7 6 1 1 and vice versa.
Chi -square test
• A test of significance in which the test statistic has a chi-square distribution, when the null hypothesis is true.
• It has the assumption that the discrete probability of observed frequencies can be approximated by the chi-
squared distribution
Assumptions:
• The sample must be randomly drawn from the population.
• Data must be reported in raw frequencies (not percentages)
• The variables must be independent.
• Categories of the variables must be mutually exclusive & exhaustive.
• Expected frequencies cannot be too small, each cell should have
minimum 5 observations in it.
Example
• In a movie theatre, suppose we made a list of movie genres & whether or not the people who came to watch
those genres of movies have bought snacks at the theatre. Here the null hypothesis is that the genre of the film
and whether people bought snacks or not are unrelatable. If this is true, the movie genres don’t impact snack
sales.
Goodness-Of-Fit
• Determines whether a variable is likely to come from a given distribution or not. We must have a set of data
values and the idea of the distribution of this data. We can use this test when we have value counts for
categorical variables. This test demonstrates a way of deciding if the data values have a “ good enough” fit for
our idea or if it is a representative sample data of the entire population.
• Suppose we have bags of balls with five different colours in each bag. The given condition is that the bag
should contain an equal number of balls of each colour. The idea we would like to test here is that the
proportions of the five colours of balls in each bag must be exact.
• Let's say you want to know if gender has anything to do with political party preference. You poll 440 voters in
a simple random sample to find out which political party they prefer. The results of the survey are shown in
the table below:
• Define the Hypothesis
• H0: There is no link between gender and political party preference.
• H1: There is a link between gender and political party preference.
• Calculate the Expected Values
• Now you will calculate the expected frequency.
• Calculate (O-E)2 / E for Each Cell in the
Table
• Now you will calculate the (O - E)2 / E
for each cell in the table.
• Where
• O = Observed Value
• E = Expected Value
• Calculate the Test Statistic λ2
• λ2 is the sum of all the values in the last
table
• = 0.743 + 2.05 + 2.33 + 3.33 + 0.384 +
1
• = 9.837
• The degrees of freedom in this case are equal to the table's number of columns minus one multiplied by the
table's number of rows minus one, or (r-1) (c-1). We have (3-1)(2-1) = 2.
• Compare obtained statistic to the critical statistic found in the chi-square table.
• For an alpha level of 0.05 and two degrees of freedom, the critical statistic is 5.991, which is less than the
obtained statistic of 9.83.
• You can reject our null hypothesis because the obtained statistic is higher than the critical statistic.
Limitations of Chi-Square Test
• The chi-square test is extremely sensitive to sample size. Even insignificant relationships can appear
statistically significant when a large enough sample is used. “Statistically significant" does not always imply
"meaningful" when using the chi-square test.
• It can only determine whether 2 variables are related. It does not necessarily follow that one variable has a
causal relationship with the other.
Advantages of nonparametric procedures.
• Nonparametric test make less stringent demands of the data.
• Nonparametric methods can be used when there is no universally recognized scale for the original data and
there is some concern that the results of standard parametric techniques would be criticized.
• There is no parameter estimation involved.
• If the sample size is very small, there may be no alternative to using a nonparametric statistical test.
Disadvantages
• There are no parameters to describe and it becomes more difficult to make quantitative statements about the
actual difference between populations.
• Nonparametric procedures throw away information.
• Some tests use only the signs of the observations. Ranks preserve information about the order of the data but
discard the actual values.
• Because information is discarded, nonparametric procedures can never be as powerful (able to detect existing
differences) as their parametric counterparts.