Main Statistics Used in Experimentation
Experimental research relies on statistical techniques to examine whether an
independent variable (IV) produces an effect on a dependent variable (DV).
These techniques help researchers make evidence-based decisions, test
hypotheses, and evaluate the significance and strength of observed effects.
This document elaborates in depth the primary statistical tools used in
experimental research, supported with clear explanations and examples.
1. Descriptive Statistics
Descriptive statistics summarize data collected from the experimental
sample. They help researchers understand central trends, variation,
distribution patterns, and the general characteristics of participants before
conducting inferential analyses.
Key Descriptive Measures
Mean (Average): Represents the central value of the dataset.
Median: The middle score; useful when data are skewed.
Mode: Most frequently occurring score.
Standard Deviation (SD): Indicates how much scores vary around
the mean.
Variance: Square of the SD; shows spread of data.
Range: Highest minus lowest value.
Confidence Intervals (CI): Estimate of where the true population
mean lies.
Graphs and Plots: Histograms, bar charts, boxplots enable visual
inspection.
Example
In an experiment comparing memory recall under music vs silent
conditions, descriptive statistics might show: - Music condition: Mean =
18.2, SD = 3.5 - Silent condition: Mean = 15.6, SD = 4.1 These statistics
provide an initial understanding of group tendencies before inferential tests.
2. Inferential Statistics
Inferential statistics allow researchers to test hypotheses about whether
differences between groups are statistically meaningful or due to chance.
They help generalize findings from a sample to a population.
A. Parametric Tests
Used when assumptions of normality, equal variance, and measurement
level are met.
1. t-Tests
Used to compare means between two groups or conditions.
Types of t-Tests
Independent t-test: Compares two different groups (between-
subjects design). Example: caffeine vs placebo.
Paired t-test: Compares the same participants in two conditions
(within-subjects design). Example: pre-test vs post-test performance.
Example
Comparing reaction times between a caffeine group and a placebo group
uses an independent t-test.
2. ANOVA (Analysis of Variance)
ANOVA is used when comparing three or more groups or when the
experiment has multiple independent variables.
Types of ANOVA
One-way ANOVA: One independent variable with three or more
levels.
Two-way ANOVA: Two independent variables.
Factorial ANOVA: Tests interaction effects between multiple
variables.
Repeated Measures ANOVA: Participants experience all conditions.
Mixed ANOVA: Combines between-subjects and within-subjects
variables.
Example
Studying the effect of: - Teaching Method (Traditional, Interactive, Online) -
Motivation Level (High, Low)
on Learning Score = 2 × 3 Factorial ANOVA.
3. ANCOVA (Analysis of Covariance)
ANCOVA adjusts for the influence of an uncontrolled variable (covariate). It
helps increase precision.
Example
Comparing performance across teaching methods while statistically
controlling for prior GPA.
4. MANOVA (Multivariate ANOVA)
Used when there are multiple dependent variables.
Example
Examining the impact of meditation training on: - Anxiety - Stress - Sleep
Quality
MANOVA analyzes all outcomes together, maintaining statistical power.
B. Nonparametric Tests
Nonparametric tests are used when assumptions of normality or equal
variances are violated, or when data are ordinal or categorical.
Common Nonparametric Tests
Mann–Whitney U: Alternative to independent t-test.
Wilcoxon Signed-Rank: Alternative to paired t-test.
Kruskal–Wallis: Alternative to one-way ANOVA.
Friedman Test: Alternative to repeated measures ANOVA.
Chi-Square Test: Used for categorical data.
Example
If student satisfaction is rated on a 5-point ordinal scale across three
teaching methods, Kruskal–Wallis test is appropriate.
3. Effect Size Measures
Effect sizes indicate the magnitude of the difference or relationship. They
complement p-values and are essential for scientific reporting.
Common Effect Sizes
Cohen’s d: Used with t-tests.
Eta squared (η²) / Partial eta squared: Used with ANOVA.
Correlation coefficients (r, r²): Represent associative strength.
Odds Ratio: Used with categorical variables.
Example
A Cohen’s d = 0.80 indicates a large effect of an intervention on
performance.
4. Assumption Testing
Before running parametric tests, key assumptions must be verified.
Common Assumption Tests
Normality: Shapiro–Wilk, Kolmogorov–Smirnov.
Homogeneity of Variance: Levene’s test.
Sphericity: Mauchly’s test (for repeated measures ANOVA).
Example
If Levene’s test is not significant (p > .05), variances are equal, and ANOVA
may be used.
5. Post-Hoc Tests
Post-hoc tests identify which specific groups differ after a significant ANOVA.
Common Post-Hoc Tests
Tukey’s HSD
Bonferroni
Scheffé
Duncan’s
Example
After finding a significant one-way ANOVA for teaching methods, Tukey’s test
may show: - Interactive > Traditional - Online = Traditional
6. Correlation and Regression in Experiments
These are used when experiments have predictive components.
Correlation
Pearson: Interval/ratio, normal data
Spearman: Ordinal or non-normal data
Regression
Simple Regression: One predictor
Multiple Regression: Multiple predictors
Example
Motivation predicting test scores, while controlling for age, uses multiple
regression.
7. Advanced Statistics in Experimentation
These are used in complex or real-world experimental settings.
Includes:
Linear Mixed Models (LMMs): Handle nested and repeated
measures data.
Logistic Regression: For binary outcomes (e.g., success vs failure).
Bayesian Analysis: Probability-based inference.
Mediation and Moderation Models: Explore underlying
mechanisms.
Structural Equation Modelling (SEM): For complex theoretical
models.
Example
Testing whether the effect of a teaching method on performance works
through engagement → Mediation.
Summary Table of Statistical Techniques
Purpose Statistical Tool
Describe data Mean, SD, CI, graphs
Compare 2 groups t-test / Mann–Whitney
Compare 3+ groups ANOVA / Kruskal–Wallis
Repeated measures Repeated-Measures ANOVA /
Purpose Statistical Tool
Friedman
Interaction effects Factorial ANOVA
Control covariates ANCOVA
Multiple dependent variables MANOVA
Categorical comparisons Chi-square
Effect strength Cohen’s d, η²
Predictive analysis Correlation, Regression
Test assumptions Shapiro–Wilk, Levene’s
This elaborated document provides a comprehensive understanding of the
main statistics used in experimentation with clear examples and structured
explanations.