Applied Statistical Methods
Complete Notes — From Zero to Applied Proficiency
Compiled Study Notes
oday
Contents
1 1. Introduction — What Statistics Actually Is 3
1.1 1.1 Population vs. Sample . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 1.2 Types of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 2. Descriptive Statistics 4
2.1 2.1 Measures of Central Tendency . . . . . . . . . . . . . . . . . . . . 4
2.2 2.2 Measures of Spread (Dispersion) . . . . . . . . . . . . . . . . . . . 4
2.3 2.3 Shape of a Distribution . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.4 2.4 The Five-Number Summary & Boxplots . . . . . . . . . . . . . . . . 4
2.5 2.5 Common Visualizations . . . . . . . . . . . . . . . . . . . . . . . . 4
3 3. Probability Foundations 5
3.1 3.1 Core Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 3.2 Rules of Probability . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3 3.3 Conditional Probability & Bayes’ Theorem . . . . . . . . . . . . . . 5
3.4 3.4 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4 4. Key Probability Distributions 6
4.1 4.1 Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2 4.2 Continuous Distributions . . . . . . . . . . . . . . . . . . . . . . . . 6
5 5. Sampling & the Central Limit Theorem 7
5.1 5.1 Why Sampling Works . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5.2 5.2 Sampling Distribution of the Mean . . . . . . . . . . . . . . . . . . 7
5.3 5.3 The Central Limit Theorem (CLT) . . . . . . . . . . . . . . . . . . . 7
6 6. Estimation 8
6.1 6.1 Point Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
6.2 6.2 Confidence Intervals (CI) . . . . . . . . . . . . . . . . . . . . . . . 8
6.3 6.3 Factors Affecting CI Width . . . . . . . . . . . . . . . . . . . . . . . 8
7 7. Hypothesis Testing 9
7.1 7.1 The Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
7.2 7.2 The p-value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1
7.3 7.3 Types of Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
7.4 7.4 One-Tailed vs. Two-Tailed Tests . . . . . . . . . . . . . . . . . . . . 9
8 8. Common Hypothesis Tests 10
8.1 8.1 Choosing the Right Test — Decision Guide . . . . . . . . . . . . . . 10
8.2 8.2 t-Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
8.3 8.3 Chi-Square Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
8.4 8.4 ANOVA (Analysis of Variance) . . . . . . . . . . . . . . . . . . . . . 11
9 9. Correlation & Regression 12
9.1 9.1 Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
9.2 9.2 Simple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . 12
9.3 9.3 Multiple Linear Regression . . . . . . . . . . . . . . . . . . . . . . 12
9.4 9.4 Regression Assumptions (checked via residual plots) . . . . . . . . 12
9.5 9.5 Logistic Regression (brief) . . . . . . . . . . . . . . . . . . . . . . . 13
10 10. Non-Parametric Methods 14
11 11. Practical Applied Workflow 15
11.111.1 Effect Size — Why It Matters as Much as the p-value . . . . . . . . 15
11.211.2 Multiple Comparisons Problem . . . . . . . . . . . . . . . . . . . 15
12 12. Quick-Reference Formula Sheet 16
13 13. Glossary of Must-Know Terms 17
14 14. Where to Go Next 17
2
1 1. Introduction — What Statistics Actually Is
Statistics is the science of collecting, organizing, analyzing, and interpreting
data in order to make decisions under uncertainty. Every method in this document
exists to answer one of two questions:
1. Description — What does the data look like? (descriptive statistics)
2. Inference — What can I conclude about a larger population from a sample, and
how confident can I be? (inferential statistics)
1.1 1.1 Population vs. Sample
• Population: the entire group you want to understand (e.g., all customers, all
patients, all possible measurements).
• Sample: a subset of the population that you actually observe.
• Parameter: a numerical summary of the population (e.g., population mean 𝜇,
population proportion 𝑝). Usually unknown.
• Statistic: a numerical summary of the sample (e.g., sample mean 𝑥,̄ sample
proportion 𝑝̂). Used to estimate parameters.
1.2 1.2 Types of Data
Type Description Examples
Categorical (Nominal) Named categories, no Blood type, color, gender
order
Categorical (Ordinal) Categories with order Satisfaction
(low/med/high)
Numerical (Discrete) Countable values Number of children
Numerical (Continuous) Any value in a range Height, weight, time
The data type you have determines which statistical test or visualization is valid —
this single idea underlies most of applied statistics.
3
2 2. Descriptive Statistics
2.1 2.1 Measures of Central Tendency
1 𝑛
• Mean: 𝑥̄ = ∑ 𝑥 — sensitive to outliers.
𝑛 𝑖=1 𝑖
• Median: the middle value when data is sorted — robust to outliers.
• Mode: the most frequent value — only measure usable for nominal data.
Rule of thumb: if the mean and median differ a lot, your data is skewed and the
median is the better summary.
2.2 2.2 Measures of Spread (Dispersion)
• Range = max − min.
1 𝑛
• Variance (sample): 𝑠2 = ∑𝑖=1 (𝑥𝑖 − 𝑥)̄ 2
𝑛√− 1
• Standard deviation: 𝑠 = 𝑠2 — same units as the data, most commonly re-
ported.
• Interquartile Range (IQR) = 𝑄3 − 𝑄1 — robust measure of spread, used to
detect outliers.
• Coefficient of Variation: 𝐶𝑉 = 𝑠/𝑥̄ — useful for comparing variability across
different units/scales.
Why divide by 𝑛 − 1 and not 𝑛? Using 𝑛 − 1 (Bessel’s correction) makes 𝑠2 an
unbiased estimator of the true population variance 𝜎2 , correcting for the fact that 𝑥̄
is itself estimated from the same data.
2.3 2.3 Shape of a Distribution
• Skewness: measures asymmetry. Positive (right) skew = long tail to the right
(mean > median). Negative (left) skew = long tail to the left.
• Kurtosis: measures “tailedness” relative to a normal distribution. High kurtosis
= heavier tails, more outliers.
2.4 2.4 The Five-Number Summary & Boxplots
Minimum, 𝑄1 , Median, 𝑄3 , Maximum. A boxplot visualizes this directly and flags
outliers as points beyond 𝑄1 − 1.5 × 𝐼𝑄𝑅 or 𝑄3 + 1.5 × 𝐼𝑄𝑅.
2.5 2.5 Common Visualizations
Chart Best for
Histogram Shape/spread of one numeric variable
Boxplot Comparing spread & outliers across groups
Bar chart Categorical frequencies
Scatterplot Relationship between two numeric variables
Line chart Trends over time
4
3 3. Probability Foundations
Statistics inference is built entirely on probability theory, so a working grasp of the
basics is essential.
3.1 3.1 Core Definitions
• Experiment: a process with an uncertain outcome (e.g., rolling a die).
• Sample space (𝑆 ): all possible outcomes.
• Event (𝐴): a subset of the sample space.
• 𝑃 (𝐴) satisfies: 0 ≤ 𝑃 (𝐴) ≤ 1, and 𝑃 (𝑆) = 1.
3.2 3.2 Rules of Probability
• Complement: 𝑃 (𝐴𝑐 ) = 1 − 𝑃 (𝐴)
• Addition rule: 𝑃 (𝐴 ∪ 𝐵) = 𝑃 (𝐴) + 𝑃 (𝐵) − 𝑃 (𝐴 ∩ 𝐵)
• Multiplication rule: 𝑃 (𝐴 ∩ 𝐵) = 𝑃 (𝐴) 𝑃 (𝐵 ∣ 𝐴)
• Independence: 𝐴 and 𝐵 are independent iff 𝑃 (𝐴 ∩ 𝐵) = 𝑃 (𝐴)𝑃 (𝐵), equiva-
lently 𝑃 (𝐴|𝐵) = 𝑃 (𝐴).
3.3 3.3 Conditional Probability & Bayes’ Theorem
𝑃 (𝐴 ∩ 𝐵)
𝑃 (𝐴 ∣ 𝐵) =
𝑃 (𝐵)
Bayes’ Theorem:
𝑃 (𝐵 ∣ 𝐴) 𝑃 (𝐴)
𝑃 (𝐴 ∣ 𝐵) =
𝑃 (𝐵)
This is the backbone of medical testing, spam filtering, and Bayesian statistics: it lets
you update a prior belief about 𝐴 after observing evidence 𝐵 .
3.4 3.4 Random Variables
A random variable 𝑋 assigns a number to each outcome of a random process.
• Discrete: takes countable values, described by a probability mass function
(PMF), 𝑃 (𝑋 = 𝑥).
• Continuous: takes any value in an interval, described by a probability density
function (PDF), 𝑓(𝑥), where probabilities are areas under the curve.
Expected value (mean): 𝐸[𝑋] = ∑ 𝑥 𝑃 (𝑋 = 𝑥) (discrete) or ∫ 𝑥𝑓(𝑥) 𝑑𝑥 (continu-
ous).
Variance: 𝑉 𝑎𝑟(𝑋) = 𝐸[(𝑋 − 𝐸[𝑋])2 ] = 𝐸[𝑋 2 ] − (𝐸[𝑋])2 .
5
4 4. Key Probability Distributions
4.1 4.1 Discrete Distributions
Bernoulli: one trial, success probability 𝑝. 𝐸[𝑋] = 𝑝, 𝑉 𝑎𝑟(𝑋) = 𝑝(1 − 𝑝).
Binomial (𝑛, 𝑝): number of successes in 𝑛 independent Bernoulli trials.
𝑛
𝑃 (𝑋 = 𝑘) = ( )𝑝𝑘 (1 − 𝑝)𝑛−𝑘 , 𝐸[𝑋] = 𝑛𝑝, 𝑉 𝑎𝑟(𝑋) = 𝑛𝑝(1 − 𝑝)
𝑘
Use for: pass/fail counts, conversion counts, defect counts (fixed trials).
Poisson (𝜆): number of events in a fixed interval of time/space, with a known average
rate.
𝜆𝑘 𝑒−𝜆
𝑃 (𝑋 = 𝑘) = , 𝐸[𝑋] = 𝑉 𝑎𝑟(𝑋) = 𝜆
𝑘!
Use for: arrivals, rare events, call-center volume.
4.2 4.2 Continuous Distributions
Uniform (𝑎, 𝑏): every value in [𝑎, 𝑏] equally likely.
Normal (Gaussian) (𝜇, 𝜎2 ): the bell curve — the single most important distribution
in applied statistics, because of the Central Limit Theorem (Section 5).
1 (𝑥−𝜇)2
𝑓(𝑥) = √ 𝑒− 2𝜎2
𝜎 2𝜋
Empirical Rule for a normal distribution: about 68% of data lies within 1 SD of the
mean, 95% within 2 SD, 99.7% within 3 SD.
Standard Normal (Z): normal with 𝜇 = 0, 𝜎 = 1. Any normal variable can be
standardized:
𝑋−𝜇
𝑍=
𝜎
This lets you use one universal Z-table for any normal distribution.
Exponential (𝜆): time between Poisson events; memoryless.
t-distribution: like the normal but with heavier tails; used instead of Z when the
population standard deviation is unknown and estimated from a small sample. As
sample size (degrees of freedom) grows, 𝑡 → 𝑍 .
Chi-square (𝜒2 ): sum of squared standard normals; used for variance tests and
categorical (goodness-of-fit / independence) tests.
F-distribution: ratio of two chi-square distributions; used in ANOVA and regression
F-tests.
6
5 5. Sampling & the Central Limit Theorem
5.1 5.1 Why Sampling Works
We rarely observe an entire population, so we rely on random samples to make valid
inferences. Good sampling avoids bias — a systematic tendency to over- or under-
estimate the true value (e.g., selection bias, non-response bias).
Common sampling designs: simple random sampling, stratified sampling, cluster
sampling, systematic sampling.
5.2 5.2 Sampling Distribution of the Mean
If you repeatedly drew samples of size 𝑛 and computed 𝑥̄ each time, those 𝑥̄ values
would themselves form a distribution — the sampling distribution of the mean.
• Its mean equals the population mean: 𝐸[𝑋]̄ = 𝜇
𝜎
• Its standard deviation, called the standard error, is: 𝑆𝐸 = √
𝑛
Notice that SE shrinks as 𝑛 grows — larger samples give more precise estimates.
5.3 5.3 The Central Limit Theorem (CLT)
The single most important theorem in applied statistics: for a sufficiently large
sample size 𝑛 (rule of thumb 𝑛 ≥ 30), the sampling distribution of 𝑥̄ is approximately
normal, regardless of the shape of the original population distribution.
This is why so many methods (confidence intervals, t-tests, regression inference) rely
on the normal or t-distribution even when raw data isn’t normally distributed — be-
cause it’s the sample mean’s distribution that matters, not the raw data’s.
7
6 6. Estimation
6.1 6.1 Point Estimation
A point estimate is a single best guess for a parameter (e.g., 𝑥̄ estimates 𝜇). A good
estimator is:
• Unbiased: its expected value equals the true parameter.
• Efficient: has the smallest possible variance among unbiased estimators.
• Consistent: converges to the true value as 𝑛 → ∞.
6.2 6.2 Confidence Intervals (CI)
A point estimate alone hides uncertainty. A confidence interval gives a range of
plausible values for the parameter, with a stated confidence level (typically 90%, 95%,
99%).
CI for a mean (large sample or known 𝜎):
𝜎
𝑥̄ ± 𝑧 ∗ √
𝑛
CI for a mean (small sample, 𝜎 unknown — the usual real-world case):
𝑠
𝑥̄ ± 𝑡∗𝑛−1 √
𝑛
CI for a proportion:
𝑝(1
̂ − 𝑝)̂
𝑝̂ ± 𝑧 ∗ √
𝑛
Correct interpretation (this is the most commonly misunderstood idea in statistics):
“If we repeated this sampling process many times and built a CI each time, 95% of
those intervals would contain the true parameter.” It is not “there is a 95% probability
the true parameter is in this specific interval” — the true parameter is fixed; the
interval is what varies.
6.3 6.3 Factors Affecting CI Width
• Larger sample size 𝑛 → narrower interval (more precise).
• Higher confidence level → wider interval (more certain, less precise).
• Larger variability 𝑠 → wider interval.
8
7 7. Hypothesis Testing
7.1 7.1 The Logic
Hypothesis testing asks: “Is what I observed in my sample consistent with random
chance alone, or is there real evidence of an effect?”
1. Null hypothesis (𝐻0 ): the “no effect / no difference / status quo” statement.
2. Alternative hypothesis (𝐻1 or 𝐻𝑎 ): what you’re trying to find evidence for.
3. Collect data, compute a test statistic, and calculate a p-value.
4. Compare the p-value to a pre-chosen significance level 𝛼 (commonly 0.05).
7.2 7.2 The p-value
The p-value is the probability of observing a test statistic as extreme or more extreme
than the one obtained, assuming 𝐻0 is true. It is not the probability that 𝐻0 is true.
• If 𝑝 ≤ 𝛼: reject 𝐻0 (“statistically significant”).
• If 𝑝 > 𝛼: fail to reject 𝐻0 (this does not prove 𝐻0 is true — absence of evidence
isn’t evidence of absence).
7.3 7.3 Types of Error
𝐻0 True 𝐻0 False
Reject 𝐻0 Type I Error (𝛼) Correct (Power)
Fail to Reject 𝐻0 Correct Type II Error (𝛽 )
• Type I error: false positive — concluding there’s an effect when there isn’t.
Controlled by 𝛼.
• Type II error: false negative — missing a real effect. Related to statistical
power = 1 − 𝛽 , the probability of correctly detecting a true effect.
• Power increases with: larger sample size, larger true effect size, lower variability,
and higher 𝛼.
7.4 7.4 One-Tailed vs. Two-Tailed Tests
• Two-tailed: 𝐻1 ∶ 𝜇 ≠ 𝜇0 — tests for a difference in either direction.
• One-tailed: 𝐻1 ∶ 𝜇 > 𝜇0 or 𝜇 < 𝜇0 — tests for a difference in one specific
direction. Use only when a directional hypothesis is justified before seeing the
data.
9
8 8. Common Hypothesis Tests
8.1 8.1 Choosing the Right Test — Decision Guide
Goal Data type Test
Compare 1 sample mean Numeric One-sample t-test
to a value
Compare 2 independent Numeric Independent (two-sample)
group means t-test
Compare 2 Numeric Paired t-test
paired/matched means
Compare 3+ group means Numeric One-way ANOVA
Test a proportion Categorical (binary) One-proportion z-test
Compare 2 proportions Categorical (binary) Two-proportion z-test
Test association between Categorical Chi-square test of
2 categorical vars independence
Test if data fits a Categorical Chi-square goodness-of-fit
distribution
Relationship between 2 Numeric Correlation / Regression
numeric variables
Non-normal data, 2 Numeric (ranked) Mann-Whitney U
groups
Non-normal data, paired Numeric (ranked) Wilcoxon signed-rank
Non-normal data, 3+ Numeric (ranked) Kruskal-Wallis
groups
8.2 8.2 t-Tests
One-sample t-test: tests whether a sample mean differs from a known value 𝜇0 .
𝑥 ̄ − 𝜇0
𝑡= √ , 𝑑𝑓 = 𝑛 − 1
𝑠/ 𝑛
Independent two-sample t-test: tests whether two group means differ.
𝑥1̄ − 𝑥2̄
𝑡= 2 𝑠22
√ 𝑛𝑠1 + 𝑛2
1
(Welch’s version, which doesn’t assume equal variances, is the safer default.)
Paired t-test: used when observations are naturally linked (before/after on the same
subject). Reduces the problem to a one-sample t-test on the differences 𝑑𝑖 = 𝑥1𝑖 −𝑥2𝑖 .
Assumptions: independence of observations, approximate normality (less critical
with large 𝑛 due to CLT), and (for pooled version) roughly equal variances.
10
8.3 8.3 Chi-Square Tests
Goodness-of-fit: does one categorical variable follow a hypothesized distribution?
Test of independence: are two categorical variables associated?
(𝑂 − 𝐸)2
𝜒2 = ∑
𝐸
where 𝑂 = observed count, 𝐸 = expected count under independence. Requires ex-
pected cell counts generally ≥ 5.
8.4 8.4 ANOVA (Analysis of Variance)
Used to compare means across 3 or more groups without inflating Type I error
(which would happen if you ran many pairwise t-tests).
Logic: partition total variability into between-group variance and within-group vari-
ance.
Mean Square Between
𝐹 =
Mean Square Within
A large F suggests group means differ more than expected by chance. A significant
ANOVA tells you that some groups differ, not which — follow up with post-hoc tests
(e.g., Tukey’s HSD) to find which pairs differ, while still controlling the overall error
rate.
Assumptions: independence, normality within groups, homogeneity of variance
(test with Levene’s test).
11
9 9. Correlation & Regression
9.1 9.1 Correlation
Pearson’s correlation coefficient 𝑟 measures the strength and direction of a linear
relationship between two numeric variables, ranging from −1 to +1.
∑(𝑥𝑖 − 𝑥)(𝑦 ̄ 𝑖 − 𝑦)̄
𝑟=
√∑(𝑥𝑖 − 𝑥)̄ 2 √∑(𝑦𝑖 − 𝑦)̄ 2
• 𝑟 near +1: strong positive linear relationship.
• 𝑟 near −1: strong negative linear relationship.
• 𝑟 near 0: weak or no linear relationship (nonlinear relationships can still exist!).
Critical caution: correlation does not imply causation. A relationship between 𝑋
and 𝑌 may be due to a confounding variable, reverse causation, or pure coinci-
dence.
9.2 9.2 Simple Linear Regression
Models the relationship between a predictor 𝑋 and response 𝑌 as a straight line:
𝑦 ̂ = 𝑏 0 + 𝑏1 𝑥
• 𝑏1 (slope): predicted change in 𝑌 for a one-unit increase in 𝑋 .
• 𝑏0 (intercept): predicted value of 𝑌 when 𝑋 = 0.
• Estimated via Ordinary Least Squares (OLS), which minimizes the sum of
squared residuals ∑(𝑦𝑖 − 𝑦𝑖̂ )2 .
𝑅2 (coefficient of determination): the proportion of variance in 𝑌 explained by 𝑋 ,
ranges 0 to 1. 𝑅2 = 𝑟2 in simple regression.
9.3 9.3 Multiple Linear Regression
Extends to several predictors:
𝑦 ̂ = 𝑏 0 + 𝑏 1 𝑥1 + 𝑏 2 𝑥2 + ⋯ + 𝑏 𝑘 𝑥𝑘
Each 𝑏𝑖 represents the effect of 𝑥𝑖 holding all other predictors constant. Watch
for multicollinearity (predictors highly correlated with each other), which makes
individual coefficients unstable — diagnosed via Variance Inflation Factor (VIF).
9.4 9.4 Regression Assumptions (checked via residual plots)
• Linearity: relationship between predictors and response is linear.
• Independence: residuals are not correlated with each other.
• Homoscedasticity: residuals have constant variance across all fitted values (no
“fanning out”).
• Normality: residuals are approximately normally distributed (mainly matters
for small-sample inference).
• No influential outliers (checked via leverage / Cook’s distance).
12
9.5 9.5 Logistic Regression (brief)
When the response 𝑌 is binary (yes/no), linear regression is inappropriate. Logistic
regression models the log-odds of the outcome:
𝑝
log ( ) = 𝑏 0 + 𝑏 1 𝑥1 + …
1−𝑝
Coefficients are interpreted via odds ratios (𝑒𝑏𝑖 ).
13
10 10. Non-Parametric Methods
Non-parametric tests make fewer assumptions about the underlying distribution (par-
ticularly, they don’t require normality) and are used when sample sizes are small or
data is ordinal/skewed.
Parametric test Non-parametric alternative
One-sample t-test Wilcoxon signed-rank test
Independent t-test Mann-Whitney U test
Paired t-test Wilcoxon signed-rank test
One-way ANOVA Kruskal-Wallis test
Pearson correlation Spearman’s rank correlation
These generally work on the ranks of the data rather than raw values, which makes
them robust to outliers and non-normal shapes, at some cost in statistical power when
the data actually is normal.
14
11 11. Practical Applied Workflow
A typical end-to-end applied statistics project follows this sequence:
1. Define the question — what decision does this analysis inform?
2. Design data collection — sampling method, sample size (power analysis),
avoid bias.
3. Clean the data — handle missing values, outliers, incorrect entries.
4. Explore (EDA) — descriptive statistics, visualizations, check assumptions.
5. Choose the right method — use the decision guide in Section 8.1.
6. Test assumptions — normality (Shapiro-Wilk, Q-Q plots), equal variance (Lev-
ene’s test), independence.
7. Run the analysis — compute test statistic, p-value, effect size, confidence in-
terval.
8. Interpret in context — statistical significance is not the same as practical
significance; always report effect size alongside p-values.
9. Communicate clearly — visuals, plain-language conclusions, and honest state-
ment of limitations/uncertainty.
11.1 11.1 Effect Size — Why It Matters as Much as the p-value
A tiny, practically meaningless difference can be “statistically significant” with a large
enough sample. Always report an effect size alongside significance:
𝑥1̄ − 𝑥2̄
• Cohen’s d (mean comparisons): 𝑑 = — small ≈ 0.2, medium ≈ 0.5,
𝑠𝑝𝑜𝑜𝑙𝑒𝑑
large ≈ 0.8.
• 𝑅2 (regression/ANOVA): proportion of variance explained.
• Odds ratio (logistic regression / 2x2 tables).
11.2 11.2 Multiple Comparisons Problem
Running many tests inflates the chance of a false positive somewhere. If you run 20
independent tests at 𝛼 = 0.05, you’d expect roughly one false “significant” result
by chance alone. Corrections include the Bonferroni correction (divide 𝛼 by the
number of tests) and the False Discovery Rate (Benjamini-Hochberg) for less
conservative control.
15
12 12. Quick-Reference Formula Sheet
Descriptive
1 1 √
𝑥̄ = ∑ 𝑥𝑖 𝑠2 = ∑(𝑥𝑖 − 𝑥)̄ 2 𝑠= 𝑠2
𝑛 𝑛−1
Standardization
𝑥−𝜇
𝑧=
𝜎
Standard Error of the Mean
𝑠
𝑆𝐸 = √
𝑛
Confidence Interval (general form)
estimate ± (critical value) × (standard error)
One-sample t-statistic
𝑥 ̄ − 𝜇0
𝑡= √
𝑠/ 𝑛
Two-sample t-statistic (Welch)
𝑥1̄ − 𝑥2̄
𝑡=
√𝑠21 /𝑛1 + 𝑠22 /𝑛2
Chi-square statistic
(𝑂 − 𝐸)2
𝜒2 = ∑
𝐸
ANOVA F-statistic
𝑀 𝑆𝑏𝑒𝑡𝑤𝑒𝑒𝑛
𝐹 =
𝑀 𝑆𝑤𝑖𝑡ℎ𝑖𝑛
Correlation
∑(𝑥𝑖 − 𝑥)(𝑦
̄ 𝑖 − 𝑦)̄
𝑟=
√∑(𝑥𝑖 − 𝑥)̄ 2 ∑(𝑦𝑖 − 𝑦)̄ 2
Simple regression slope
𝑠𝑦
𝑏1 = 𝑟 ⋅ 𝑏0 = 𝑦 ̄ − 𝑏 1 𝑥 ̄
𝑠𝑥
16
13 13. Glossary of Must-Know Terms
• Bias: systematic error that pushes estimates away from the truth.
• Confounding variable: an unmeasured variable affecting both predictor and
outcome, creating a spurious association.
• Degrees of freedom (df): the number of independent values free to vary in a
calculation; typically 𝑛 − 1 or related.
• Effect size: a standardized measure of the magnitude of a finding, independent
of sample size.
• Heteroscedasticity: non-constant variance of residuals/errors.
• Outlier: an observation far from the bulk of the data; may be a data error or a
genuine extreme case — investigate, don’t automatically delete.
• Power: probability of correctly detecting a true effect (1 − 𝛽 ).
• Robustness: a method’s insensitivity to violations of its assumptions or to out-
liers.
• Significance level (𝛼): the pre-chosen threshold for rejecting 𝐻0 , typically
0.05.
14 14. Where to Go Next
Once these fundamentals are solid, natural next steps include: multiple/logistic re-
gression in depth, time series analysis (ARIMA, seasonality), experimental design
(A/B testing, randomized controlled trials), Bayesian statistics, and machine learning
as an extension of applied statistical modeling. Practicing on real datasets — using
R, Python (pandas/statsmodels/scipy), or a spreadsheet tool — is the fastest way to
make these concepts stick.
17