NON PARAMETRIC TESTS
Most of the statistical tests described earlier require an important assumption to be met if they
are to be correctly applied. This assumption is that population of data from which a sample or
samples are drawn is normally distributed. If the distribution from which a sample is drawn is
badly skewed or is otherwise grossly non normal, however, for smaller samples these statistical
tests will not yield meaningful result.
A second assumption upon which most of the tests rest is that meaningful sample statistics, such
as the mean and standard deviation, can be derived from the sample(s) and used to estimate the
corresponding population parameters. Data which are nominal in nature (such as “increase,
decrease, no change “), or ordinal (ranked) do not yield such meaningful results.
Statisticians have devised alternate procedures which can be used to test hypotheses about data
which are non-normal or for which meaningful sample statistics cannot be calculated. Since
these tests do not depend on the shape of the distribution, they are called distribution-free tests.
These tests do not depend upon the population parameters, such as the mean and the variance;
they are also called non-parametric tests. One of the most important non-parametric techniques is
the Chi-Square test formulated by Karl Pearson in 1900.
A large number of non parametric tests exist. These are:
The Sign Test
A Rank Sum Tests
The One Sample Runs Test
The Kruskal Wallis or H Test
The Spearman Rank Correlation Procedure
ADVANTAGES OF NON PARAMETRIC TESTS
1. Non-parametric tests are distribution free i.e they do not require any assumptions to be
made about population following normal or any other distribution.
2. Generally they are simple to understand and easy to apply when the sample sizes are
small.
3. Most non-parametric tests do not require lengthy and laborious computations and hence
are less time consuming. If significant results are obtained no further work is necessary
4. Non-parametric tests are applicable to all types of data-qualitative (nominal scaling) data
in rank form (ordinary scaling) as well as data that have been measured more precisely
(internal or ratio scaling).
5. Many non-parametric methods make it possible to work with very small samples. This is
particularly helpful to the researcher collecting pilot study data or to the medical
researcher working with a rare disease.
6. Non – parametric methods make fewer and less stringent assumptions (that are more
easily met) than do the classical procedures.
THE MANN WHITNEY U TEST (A RANK SUM TESTS)
The sign test for comparing two population distributions ignores the actual magnitude of the paired
observations and thereby discards information that would be useful in detecting a departure from the
null hypothesis. A statistical test that partially circumvents this loss by utilizing the relative
magnitudes of the observations was proposed by Mann and Whitney and is equivalent to a test
proposed independently by Wilcoxon.
With this test we can test the null hypothesis μ1 = μ 2 without assuming whether the populations
sampled have roughly the shape of normal distribution.
This test helps us to determine whether two samples have come from identical populations. If it is
true that the samples have come from the same populations it is reasonable to assume that the means
of the ranks assigned to the values of the two samples are more or less the same.
The test of the null hypothesis that the two samples come from identical populations may either be
based on R1, the sum of the ranks of the values of first sample, or on R2, the sum of the ranks of the
values of second sample.
If the sample sizes are n1 & n2 the sum of R1 & R2 is simply the sum of first n1+n2 positive integers,
which is known to be {(n1+n2)(n1+n2+1)}/2. This formula enables us to find R2 if we know R1 and
vice versa.
When the use of the rank sums was first proposed as a non parametric alternative to the two sample t
test, the decision was based on R1 & R2, but now the decision is usually based on either of the related
statistics:
U1= n1n2 + {n1 (n1+1)}/2 - R1
U2= n1n2 + {n2 (n2+1)}/2 - R2
where n1 & n2 are the size of the samples and R1 & R2 are the rank sums of the corresponding
samples. If U is smaller than the critical value, H0 can be related to the standard normal curve.
McNemar's test
In statistics, McNemar's test is a statistical test used on paired nominal data. It is applied to
2 × 2 contingency tables with a dichotomous trait, with matched pairs of subjects, to determine
whether the row and column marginal frequencies are equal (that is, whether there is "marginal
homogeneity"). It is named after Quinn McNemar, who introduced it in 1947.[1]
McNemar's Test [1] (sometimes also called "within-subjects chi-squared test") is a statistical test
for paired nominal data. In context of machine learning (or statistical) models, we can use
McNemar's Test to compare the predictive accuracy of two models. McNemar's test is based on a
2 times 2 contigency table of the two model's predictions.
McNemar's Test Statistic
In McNemar's Test, we formulate the null hypothesis that the
probabilities p(b)p(b) and p(c)p(c) are the same, or in simplified terms: None of the two models
performs better than the other. Thus, the alternative hypothesis is that the performances of the
two models are not equal.
The McNemar test statistic ("chi-squared") can be computed as follows
An application of the test in genetics is the transmission disequilibrium test for detecting linkage
disequilibrium.[2] The commonly used parameters to assess a diagnostic test in medical sciences
are sensitivity and specificity. Sensitivity is the ability of a test to correctly identify the people
with disease. Specificity is the ability of the test to correctly identify those without the disease.
Now presume two tests are performed on the same group of patients. And also presume that
these tests have identical sensitivity and specificity. In this situation one is carried away by these
findings and presume that both the tests are equivalent. However this may not be the case. For
this we have to study the patients with disease and patients without disease (by a reference test).
We also have to find out where these two tests disagree with each other. This is precisely the
basis of McNemar's test. This test compares the sensitivity and specificity of two diagnostic tests
on the same group of patients.
If the sum of cell c and b is sufficiently large, the χ2χ2 value follows a chi-squared distribution
with one degree of freedom. After setting a significance threshold, e.g,. α=0.05α=0.05 we can
compute the p-value -- assuming that the null hypothesis is true, the p-value is the probability of
observing this empirical (or a larger) chi-squared value. If the p-value is lower than our chosen
significance level, we can reject the null hypothesis that the two model's performances are equal.
Kruskal–Wallis one-way analysis of variance
The Kruskal–Wallis test by ranks, Kruskal–Wallis H test (named after William
Kruskal and W. Allen Wallis), or one-way ANOVA on ranks is a non-parametric method for
testing whether samples originate from the same distribution. It is used for comparing two or
more independent samples of equal or different sample sizes. It extends the Mann–
Whitney U test, which is used for comparing only two groups. The parametric equivalent of the
Kruskal–Wallis test is the one-way analysis of variance (ANOVA).
A significant Kruskal–Wallis test indicates that at least one sample stochastically dominates one
other sample. The test does not identify where this stochastic dominance occurs or for how many
pairs of groups stochastic dominance obtains. For analyzing the specific sample pairs for
stochastic dominance, Dunn's test, pairwise Mann–Whitney tests with Bonferroni correction, or
the more powerful but less well known Conover–Iman test are sometimes used.
Since it is a nonparametric method, the Kruskal–Wallis test does not assume a normal
distribution of the residuals, unlike the analogous one-way analysis of variance. If the researcher
can make the assumptions of an identically shaped and scaled distribution for all groups, except
for any difference in medians, then the null hypothesis is that the medians of all groups are equal,
and the alternative hypothesis is that at least one population median of one group is different
from the population median of at least one other group.
The test determines whether the medians of two or more groups are different. Like most
statistical tests, you calculate a test statistic and compare it to a distribution cut-off point. The test
statistic used in this test is called the H statistic. The hypotheses for the test are:
Assumptions For The Kruskal Wallis Test
Your variables should have:
One independent variable with two or more levels (independent groups). The test is more
commonly used when you have three or more levels. For two levels, consider using
the Mann Whitney U Test instead.
Ordinal scale, Ratio Scale or Interval scale dependent variables.
Your observations should be independent. In other words, there should be no relationship
between the members in each group or between groups.
All groups should have the same shape distributions. Most software (i.e. SPSS, Minitab)
will test for this condition as part of the test.
To Calculate the H statistic: