STATISTICS

INTERVIEW
CRACK GUIDE
Complete Conceptual Handbook with Live Examples

Distributions Hypothesis Testing Probability Regression

Bayesian Stats A/B Testing CLT & Sampling Time Series

Covering every concept asked in Data Science & Analyst interviews

Table of Contents

1 Descriptive Statistics — Mean, Median, Variance, Skewness 3

2 Probability Foundations — Rules, Conditional, Bayes 5

3 Distributions — Which One to Use & When 7

4 Sampling & Central Limit Theorem 12

5 Hypothesis Testing — Full Framework 14

6 Confidence Intervals & p-values 17

7 Correlation vs Causation & Regression 19

8 Bayesian Statistics 22

9 A/B Testing End-to-End 24

10 Common Interview Questions & Traps 27

Statistics Interview Guide • Page 2

 CHAPTER 1 Descriptive Statistics
Mean, Median, Mode, Variance, Std Dev, Skewness, Kurtosis

The Five-Number Summary

Every dataset's personality is captured by five numbers: minimum, Q1, median, Q3, maximum. These form the
skeleton of a boxplot and reveal spread, central tendency, and outliers at a glance.

Measure Formula When to Use Sensitive to Outliers?

Mean (µ) Sum / n Symmetric, no outliers YES — highly

Median Middle value Skewed data, outliers present No

Mode Most frequent Categorical or nominal data No

Variance (σ²) Σ(x−µ)²/n Spread of data YES

Std Dev (σ) √Variance Same units as data YES

IQR Q3 − Q1 Robust spread measure No

Example: Mean vs Median — Why it matters

Salaries at a startup: [40K, 42K, 45K, 48K, 50K, 500K (CEO)]

Mean = (40+42+45+48+50+500)/6 = 120.8K ← misleading!

Median = (45+48)/2 = 46.5K ← much more representative

Rule: When you see income/salary/house price data, always report MEDIAN.

Variance vs Standard Deviation

)² / N Sample Variance: s² = Σ(xi − x■)² / (n−1)
essel's correction Use n−1 for sample to get an unbiase

Interview Trap: Why divide by n−1 for sample variance? Because using n underestimates the population variance
(biased). The n−1 correction (Bessel's correction) makes the sample variance an unbiased estimator. Interviewers
love this question!

Skewness & Kurtosis

Type Skewness Value Shape Mean vs Median Real Example

Right (Positive) >0 Long right tail Mean > Median Income, house prices

Left (Negative) <0 Long left tail Mean < Median Exam scores (easy exam)

Symmetric ≈0 Bell shaped Mean ≈ Median Heights, IQ scores

Kurtosis measures tail heaviness. Leptokurtic (kurtosis > 3): fat tails, more outliers — e.g., stock returns.
Platykurtic (kurtosis < 3): thin tails — e.g., uniform distribution. Mesokurtic (= 3): normal distribution.

Statistics Interview Guide • Page 3

 CHAPTER 2 Probability Foundations
Rules, Conditional Probability, Bayes' Theorem

Core Rules
P(A ∪ B) = P(A) + P(B) − P(A ∩ Ex: P(card is red OR face card) = 26/52 +
Addition Rule
B) 12/52 − 6/52 = 32/52

Multiplication
P(A ∩ B) = P(A) × P(B|A) Ex: P(two aces) = 4/52 × 3/51 = 1/221
Rule

Ex: P(at least one head in 3 flips) = 1 − P(no

Complement Rule P(A') = 1 − P(A)
heads) = 1 − (0.5)³ = 0.875

Mutually P(A ∩ B) = 0, so P(A ∪ B) =

Ex: Rolling a 2 and a 5 on one die
Exclusive P(A) + P(B)

Independent Ex: Flipping two coins: P(H,H) = 0.5 × 0.5 =

P(A ∩ B) = P(A) × P(B)
Events 0.25

Conditional Probability
B) = P(A ∩ B) / P(B) Probability of A given B has oc

Example: Medical Diagnosis (Classic Interview Question)

A disease affects 1% of population. Test is 95% accurate (sensitivity).

Specificity = 90% (true negative rate for healthy people).

You test positive. What is the probability you actually have the disease?

Setup: P(Disease) = 0.01, P(No Disease) = 0.99

P(Positive | Disease) = 0.95, P(Positive | No Disease) = 0.10

P(Positive) = 0.95×0.01 + 0.10×0.99 = 0.0095 + 0.099 = 0.1085

P(Disease | Positive) = 0.0095 / 0.1085 ≈ 8.75%

Key insight: Even with a 95% accurate test, you only have ~9% chance of being sick!

This is why low base-rate diseases require confirmatory tests.

Bayes' Theorem
= P(E|H) × P(H) / P(E) Posterior = Likelihood × Prior / E

Term Meaning In the Disease Example

Statistics Interview Guide • Page 4

Prior P(H) Belief before seeing evidence 1% — base rate of disease

Likelihood P(E|H) Probability of evidence if H true 95% — test positive if sick

Evidence P(E) Total probability of evidence 10.85% — anyone tests positive

Posterior P(H|E) Updated belief after evidence 8.75% — you're actually sick

Statistics Interview Guide • Page 5

CHAPTER 3 Probability Distributions
Which Distribution to Use & When — The Complete Decision Guide

This chapter is the most tested topic in data science interviews. You must be able to: (1) identify which distribution fits
a scenario, (2) state its parameters, (3) give a real-world example, and (4) know its mean and variance.

The Distribution Decision Tree

Question Answer Distribution to Use

Is data continuous or discrete? Discrete (counts) Binomial / Poisson / Geometric

Continuous Normal / Exponential / Uniform / Beta

Are events binary (success/fail)? Fixed n trials Binomial

Rare events per interval Poisson

Until first success Geometric

Until k-th success Negative Binomial

Is data symmetric & bell-shaped? Yes Normal (Gaussian)

Is it waiting time between events? Yes, memoryless Exponential

Is data bounded between 0 and 1? Yes Beta

Is data a ratio of variances? Yes F-distribution

Is it a small sample mean? Yes, unknown σ t-distribution

DISCRETE DISTRIBUTIONS
1. Binomial Distribution — B(n, p)

P(X=k) = C(n,k) × p^k × (1−p)^(n−k) Mean = np Variance = np(1−p)

Use Binomial When: Do NOT Use When:

✓ Fixed number of trials (n) ✗ Trials are not independent

✓ Only 2 outcomes per trial (success/fail) ✗ p changes across trials

✓ Each trial is independent ✗ Sampling without replacement (use Hypergeometric)

✓ Probability p is constant ✗ Rare events with large n (use Poisson)

Example: Binomial — 3 Classic Interview Scenarios

Scenario 1: You flip a fair coin 10 times. P(exactly 6 heads)?

X ~ B(n=10, p=0.5), P(X=6) = C(10,6) × 0.5^6 × 0.5^4 = 0.205

Scenario 2: An email campaign has 20% click rate. 100 emails sent.

Expected clicks = np = 100×0.20 = 20, Std Dev = √(100×0.2×0.8) = 4

Statistics Interview Guide • Page 6

 Scenario 3: Drug trial: 70% success rate. 15 patients. P(12+ succeed)?

X ~ B(15, 0.7), P(X≥12) = P(12)+P(13)+P(14)+P(15) ≈ 0.515

2. Poisson Distribution — P(λ)

P(X=k) = (e^−λ × λ^k) / k! Mean = λ Variance = λ (Mean = Variance — key property!)

Poisson is for RATES: number of events in a fixed interval of time or space, when events are rare and independent.
Parameter λ = average rate.

Example: Poisson — Real World Scenarios

When n is large and p is small, Binomial ≈ Poisson with λ = np

• Calls to call center: avg 5 calls/hour → P(X=8 calls) with λ=5

• Website traffic: avg 100 visits/day → P(X>120) with λ=100

• Accidents on highway: avg 2/month → P(no accident this month) = e^-2 ≈ 0.135

• Typos per page: rare event, fixed interval → Poisson

• Mutations per gene per generation → Poisson

Rule of thumb: Use Poisson when n>20 and p<0.05 (and ideally λ<10)

3. Geometric Distribution — G(p)

P(X=k) = (1−p)^(k−1) × p (k = trial of first success) Mean = 1/p Variance = (1−p)/p²

Example: Geometric — Waiting for First Success

Key property: MEMORYLESS — past failures don't affect future probability.

• Coin flip: Expected flips to get first head = 1/0.5 = 2 flips

• Sales call: 10% conversion rate → expected calls until first sale = 1/0.1 = 10

• Interview question: 'How many interviews until I get my first offer?' (p = 0.3)

Expected = 1/0.3 ≈ 3.3 interviews

Interview Trap: Geometric vs Negative Binomial: Geometric asks 'trials until FIRST success'. Negative Binomial
asks 'trials until k-th success'. If k=1, they are the same.

Statistics Interview Guide • Page 7

CONTINUOUS DISTRIBUTIONS
4. Normal Distribution — N(µ, σ²)

f(x) = (1/σ√2π) × exp(−(x−µ)²/2σ²) Standard Normal: Z = (X − µ) / σ ~ N(0, 1)

Range % of Data Real Example (IQ: µ=100, σ=15)

µ ± 1σ 68.27% IQ between 85 and 115

µ ± 2σ 95.45% IQ between 70 and 130

µ ± 3σ 99.73% IQ between 55 and 145

> 3σ away 0.27% (3-sigma event) Extremely rare (used in 6-sigma quality)

Example: Normal Distribution - Interview Scenarios

Q: Heights of adults are N(u=170cm, s=10cm). What % are taller than 190cm?

Z = (190-170)/10 = 2.0 => P(Z>2) = 1 - 0.9772 = 2.28%

Q: A process produces bolts with length N(u=50mm, s=0.5mm).

Specs: 49mm to 51mm. What % are defective?

P(49 less than X less than 51) = P(-2 less than Z less than 2) = 95.45% => Defective = 4.55%

Q: SAT scores N(u=1000, s=200). What score is in the 90th percentile?

Z(90th) = 1.28 => X = 1000 + 1.28 x 200 = 1256

5. Exponential Distribution — Exp(λ)

f(x) = λ × e^(−λx) for x ≥ 0 Mean = 1/λ Variance = 1/λ² CDF: F(x) = 1 − e^(−λx)

Key property: MEMORYLESS — P(X > s+t | X > s) = P(X > t). The exponential distribution is the only continuous
memoryless distribution. It naturally pairs with the Poisson: if events occur at rate λ (Poisson), the waiting TIME
between events is Exponential(λ).

Example: Exponential — Live Examples

• Call center: avg 5 calls/hr arrive (Poisson λ=5).

Time between calls ~ Exp(λ=5) → Mean wait = 1/5 hr = 12 minutes

• Server crashes on average once every 100 hours.

P(no crash in first 50 hours) = e^(−50/100) = e^(−0.5) ≈ 0.607 = 60.7%

• Memoryless property: If server has been running 50 hours without crash,

P(runs another 50 hours) is still 60.7% — past has no effect!

6. Beta Distribution — Beta(α, β)

Defined on [0,1] Mean = α/(α+β) Variance = αβ/[(α+β)²(α+β+1)]

Example: Beta — Bayesian Probability Modeling

Statistics Interview Guide • Page 8

 Use Beta to model a PROBABILITY (a value between 0 and 1).

• Click-through rate: Unknown true CTR → model as Beta(α, β)

α = successes (clicks), β = failures (no-clicks)

After 30 clicks in 100 views: Beta(α=30, β=70) → Mean = 30/100 = 0.30

• A/B testing prior: Beta(1,1) = Uniform (no prior belief)

After seeing data, Beta updates to Beta(1+successes, 1+failures)

• Disease prevalence: If 3 of 10 tested positive → Beta(3,7) → 95% CI: [0.07, 0.65]

7. Other Key Distributions — Quick Reference

Distribution Parameters Use Case Key Property

Uniform U(a,b) min=a, max=b Random number generation, simulation All outcomes equally likely

t-distribution df (degrees of freedom) Small sample means, unknown σ Heavier tails than Normal

Chi-squared χ² df = k Goodness of fit, independence tests Sum of k squared std normals

F-distribution df1, df2 Comparing variances, ANOVA Ratio of two χ² variables

Log-Normal µ, σ (of log X) Stock prices, income, file sizes X = e^(Normal), always positive

Hypergeometric N, K, n Sampling without replacement Like Binomial but no replacement

Negative Binomial r, p Waiting for r-th success Generalizes Geometric (r=1)

Weibull shape k, scale λ Reliability, survival analysis Generalizes Exponential

Multinomial n, p1…pk Multiple category outcomes Generalizes Binomial (k=2)

Statistics Interview Guide • Page 9

 CHAPTER 4 Sampling & Central Limit Theorem
The Foundation of All Statistical Inference

Central Limit Theorem (CLT)

Central Limit Theorem
The most important theorem in statistics: Regardless of the population's distribution shape, the sampling
distribution of the sample mean approaches a Normal distribution as sample size n increases, provided observations
are i.i.d.

∞ Standard Error of Mean: SE = σ / √n Typically n ≥ 30 is sufficient for CLT

Example: CLT — Why It's Magical

Population: Exponential (right-skewed), µ=2, σ=2

Take 10,000 samples of size n=5: Distribution of means looks somewhat normal

Take 10,000 samples of size n=30: Distribution of means looks very normal!

Take 10,000 samples of size n=100: Nearly perfect bell curve.

Real application: Quality control

Biscuit factory: individual biscuit weight varies (not normal). But the mean weight

of a box of 30 biscuits IS approximately normal → can use z-tests on batch means!

Note: For highly skewed data, you may need n>50 or even n>100.

Sampling Methods Comparison

Method How It Works Use When Pros / Cons

Simple Random
Every individual has equal probability Homogeneous population Unbiased / Needs complete list
Sampling

Stratified
Divide into strata, sample from each Subgroups need representation More precise / Complex
Sampling

Cluster
Divide into clusters, sample whole clusters
Population spread geographically Cost-effective / More error
Sampling

Systematic
Every k-th element Large sequential list Easy to implement / Periodic bias
Sampling

Convenience
Easiest to reach individuals Pilot studies only Fast / Highly biased
Sampling

Bias vs Variance Trade-off in Sampling

Statistics Interview Guide • Page 10

Bias: Systematic error — your estimate consistently misses the true value in one direction. Variance: Random error
— estimates scatter widely. A good estimator minimizes both. MSE = Bias² + Variance. Increasing sample size
reduces variance but not bias.

Statistics Interview Guide • Page 11

CHAPTER 5 Hypothesis Testing
Full Framework — Setup, Test Selection, Interpretation

The 5-Step Hypothesis Testing Framework

Step 1: State H■ (null): No effect / Status quo. H■ (alternative): What you want to prove. Example:
Hypotheses H■: µ = 50 (drug has no effect), H■: µ ≠ 50 (two-tailed) or µ > 50 (one-tailed)

Step 2: Set Significance α = Type I error rate. Common: α = 0.05 (5%). This means you accept 5% chance of
Level rejecting H■ when it's actually true (false positive rate).

Step 3: Choose the See the Test Selection Guide below. Key factors: sample size, known/unknown σ,
Right Test number of groups, data type (continuous vs categorical).

Step 4: Calculate Test Compute the test statistic (z, t, F, χ²) and find the probability of seeing your result (or
Statistic & p-value more extreme) IF H■ were true.

Step 5: Make Decision If p-value < α → Reject H■ (result is statistically significant) If p-value ≥ α → Fail to
reject H■ (insufficient evidence against H■)

Test Selection Guide

Situation Test to Use Assumptions

1 sample mean, σ known Z-test Normal population or n≥30

1 sample mean, σ unknown One-sample t-test Normal population or n≥30

2 independent groups means Two-sample t-test Approx equal variances; use Welch's if not

Paired samples (before/after) Paired t-test Differences are normal

3+ group means One-way ANOVA (F-test) Normality, homogeneity of variance

1 sample proportion Z-test for proportion np≥5 and n(1−p)≥5

2 proportions Two-proportion Z-test Large samples

Categorical association Chi-square test Expected count ≥5 in each cell

Goodness of fit Chi-square GOF test Observed vs expected counts

Non-normal, 2 groups Mann-Whitney U (Wilcoxon) Non-parametric alternative to t-test

Non-normal, 3+ groups Kruskal-Wallis test Non-parametric alternative to ANOVA

Correlation significance Pearson r / Spearman ρ test Pearson: bivariate normal; Spearman: any

Type I and Type II Errors

H■ is TRUE (No Effect) H■ is FALSE (Effect Exists)

Statistics Interview Guide • Page 12

 TYPE I ERROR (α) CORRECT!
Reject H■
False Positive True Positive (Power = 1−β)
(Positive result)
'Alarm when nothing wrong' 'Caught the real effect'

CORRECT! TYPE II ERROR (β)

Fail to Reject H■
True Negative False Negative
(Negative result)
'Correctly no alarm' 'Missed a real effect'

Example: Type I & II Error — COVID Test Analogy

H■: Patient does NOT have COVID

Type I Error (α): Test says POSITIVE but patient is HEALTHY → False Alarm → Unnecessary isolation

Type II Error (β): Test says NEGATIVE but patient IS SICK → Miss → Spreads disease!

Trade-off: Lowering α (stricter test) increases β. You choose based on consequences.

In medical diagnosis: Type II errors are usually worse → use lower β (higher power).

In spam filtering: Type I errors are worse (miss real emails) → use lower α.

Statistics Interview Guide • Page 13

CHAPTER 6 Confidence Intervals & p-values
The Two Most Misunderstood Concepts in Statistics

Confidence Intervals
CI = X■ ± Z*(σ/√n) [known σ] CI = X■ ± t*(s/√n) [unknown σ, use t-distribution]

Interview Trap: WRONG interpretation: 'There is a 95% probability the true parameter is in this interval.' The true
parameter is fixed — it's either in the interval or not. CORRECT interpretation: 'If we repeated this experiment 100
times, approximately 95 of the resulting intervals would contain the true parameter.'

Confidence Level α Z* (two-tailed) Use For

90% 0.10 1.645 Less stringent, exploratory

95% 0.05 1.96 Standard in most research

99% 0.01 2.576 Medical/safety critical decisions

99.9% 0.001 3.291 Six Sigma, nuclear safety

Example: CI — Step by Step Calculation

Problem: Survey of 100 users. Mean session time = 12 min. Std Dev = 4 min.

Construct a 95% CI for the true mean session time.

SE = s/√n = 4/√100 = 4/10 = 0.4 minutes

For 95% CI: Z* = 1.96

CI = 12 ± 1.96 × 0.4 = 12 ± 0.784 = (11.22, 12.78) minutes

Interpretation: We are 95% confident the true mean session time is between

11.22 and 12.78 minutes.

Width of CI = 2 × 1.96 × 0.4 = 1.57 min. To halve the width, need 4× sample size!

p-values — The Complete Truth

Definition: The p-value is the probability of observing a test statistic as extreme as (or more extreme than) the one
computed, ASSUMING H■ is true. It is NOT the probability that H■ is true. It is NOT the probability of making a
mistake.

Common Misconception Reality

p=0.05 means 5% chance H■ is true p is P(data | H■ true), not P(H■ | data)

p < 0.05 means result is important Statistical significance ≠ practical significance

Large p means H■ is definitely true Absence of evidence ≠ evidence of absence

p-value measures effect size Small effects with large n give tiny p-values

Statistics Interview Guide • Page 14

p=0.049 is very different from p=0.051 The 0.05 cutoff is arbitrary — use judgment

Statistics Interview Guide • Page 15

CHAPTER 7 Correlation vs Causation & Regression
Understanding Relationships Between Variables

Correlation
Pearson r = Σ[(xi − x■)(yi − ■)] / [√Σ(xi−x■)² × √Σ(yi−■)²] Range: −1 ≤ r ≤ +1

r value Strength & Direction Example

0.9 to 1.0 Very strong positive Height vs weight in same population

0.7 to 0.9 Strong positive Study hours vs exam score

0.4 to 0.7 Moderate positive Income vs spending

0.1 to 0.4 Weak positive Shoe size vs IQ (spurious!)

≈0 No linear relationship TV hours vs job satisfaction

−0.4 to −0.1 Weak negative Stress level vs sleep quality

−0.7 to −0.4 Moderate negative Price vs demand

−1.0 to −0.9 Very strong negative Speed vs fuel efficiency

Interview Trap: Correlation DOES NOT imply causation! Classic example: Ice cream sales and drowning rates are
positively correlated (r≈0.8). Why? Confounding variable: hot weather. Always ask: Is there a lurking variable? Is it
reverse causation? Is it coincidence?

Simple Linear Regression

■ = β■ + β■x where: β■ = r × (sy/sx) = Σ[(xi−x■)(yi−■)] / Σ(xi−x■)² β■ = ■ − β■x■

Concept Meaning Interview Answer

R² (R-squared) % variance in y explained by x R²=0.75 means model explains 75% of variance in y

Adjusted R² R² penalized for # predictors Use instead of R² when comparing models with different # vars

Residual Actual − Predicted (ei = yi − ■i) Residuals should be random (no pattern) if model is good

OLS Assumption 1 Linearity y and x have linear relationship

OLS Assumption 2 Independence Observations are independent

OLS Assumption 3 Homoscedasticity Residual variance is constant (no funnel shape)

OLS Assumption 4 Normality Residuals are normally distributed

Multicollinearity Predictors are correlated with each other

Check VIF > 10 indicates serious multicollinearity

Overfitting Model fits training data too well, failsUse

on new
cross-validation;
data penalize complexity (Ridge/Lasso)

Statistics Interview Guide • Page 16

CHAPTER 8 Bayesian Statistics
Updating Beliefs with Evidence

Frequentist vs Bayesian — The Fundamental Divide

Aspect Frequentist Bayesian

What is probability? Long-run frequency of events Degree of belief / uncertainty

Parameters are... Fixed, unknown constants Random variables with distributions

Uses prior info? No — data only Yes — prior + data → posterior

Output Point estimate + CI Full posterior distribution

p-value Core tool Not used (use posterior probability)

Small sample Struggles Works well with informative priors

Key equation Likelihood Posterior ∝ Likelihood × Prior

Common use Traditional research Machine learning, A/B testing, NLP

Example: Bayesian Updating — Live Example: Spam Filter

Prior: P(Spam) = 0.30 (30% of emails are spam — our prior belief)

New email contains the word 'FREE'.

P('FREE' | Spam) = 0.60 (60% of spam emails contain 'FREE')

P('FREE' | Not Spam) = 0.05 (5% of legit emails contain 'FREE')

P('FREE') = 0.60×0.30 + 0.05×0.70 = 0.18 + 0.035 = 0.215

Posterior: P(Spam | 'FREE') = (0.60 × 0.30) / 0.215 = 0.18/0.215 ≈ 0.837

We updated from 30% to 83.7% probability of spam just from one word!

Bayesian spam filters chain these updates across multiple words.

Conjugate Priors — Quick Reference

Likelihood Conjugate Prior Posterior Use Case

Binomial Beta(α, β) Beta(α+k, β+n−k) Click rates, conversion rates

Poisson Gamma(α, β) Gamma(α+Σx, β+n) Arrival rates, counts

Normal (µ unknown) Normal(µ■, σ■²) Normal (weighted avg) Mean estimation

Exponential Gamma(α, β) Gamma(α+n, β+Σx) Survival analysis

Statistics Interview Guide • Page 17

 CHAPTER 9 A/B Testing End-to-End
The Complete Framework Used at Every Tech Company

A/B Testing Pipeline

1. Define the Metric Choose ONE primary metric (e.g., conversion rate, revenue/user, CTR). Also define
guardrail metrics (things that must not get worse).

2. Form a Hypothesis H■: New feature has no effect on conversion rate. H■: New feature increases
conversion rate.

3. Calculate Sample Use power analysis before starting. n = 2 × (Z_α + Z_β)² × p(1−p) / δ² where δ =
Size minimum detectable effect (MDE).

4. Run the Experiment Randomly assign users to control (A) and treatment (B). Collect data for the
pre-determined period. Avoid peeking!

5. Analyze Results Compute test statistic. Check p-value vs α. Also compute confidence interval for the
effect size.

6. Make a Decision Statistically significant + practically significant → Ship it. Check for novelty effects,
segment interactions, long-term effects.

Sample Size Formula & Power Analysis

× p■(1−p■) / δ² where: p■ = average of control &
_α/2 = 1.96 (α=0.05), Z_β = 0.84 (80% power) or
1.28 (90% power) Power = 1 − β = probability of detecting an

Example: Sample Size Calculation

Current conversion rate: 10% (control). Want to detect a +2% lift (MDE = 2%).

α = 0.05 (two-tailed), Power = 80% → Z_β = 0.84

p■ = (0.10 + 0.12)/2 = 0.11

n = 2 × (1.96 + 0.84)² × 0.11×0.89 / (0.02)²

n = 2 × 7.84 × 0.0979 / 0.0004 = 2 × 767 ≈ 3,840 per group → 7,680 total

To get results faster: Increase MDE (detect only bigger effects)

To get more reliable results: Increase power (need more users)

Common A/B Testing Pitfalls

Interview Trap: Peeking Problem: Checking results before reaching planned sample size inflates Type I error rate.
Use Sequential testing (O'Brien-Fleming) or Bayesian methods if you must peek.

Statistics Interview Guide • Page 18

Interview Trap: Multiple Testing: Testing 10 metrics at α=0.05 gives ~40% chance of a false positive. Use
Bonferroni correction: α* = α/m, or control False Discovery Rate (Benjamini-Hochberg).

Interview Trap: Network Effects: Users influence each other (social network). Cluster randomize instead of
individual randomize.

Interview Trap: Novelty Effect: Users interact more with anything new. Run experiment for 2+ weeks to distinguish
from real effect.

Interview Trap: SRM (Sample Ratio Mismatch): Actual ratio deviates from planned ratio. Investigate before
interpreting results. Indicates logging bugs or assignment errors.

Interview Trap: Simpson's Paradox: Overall trend reverses when data is segmented. Always check segments
separately.

Statistics Interview Guide • Page 19

CHAPTER 10
Top Interview Questions & Model
Answers
Curated from Google, Meta, Amazon, Netflix, Airbnb interviews

Q1: What is the difference between Type I and Type II errors? Which is worse?

Type I (false positive): Rejecting a true null hypothesis. Rate = α. Type II (false negative): Failing to reject a false null.
Rate = β. Which is worse depends on context. Medical: Type II (missing disease) is worse. Legal: Type I (convicting
innocent) is worse. Spam: Type I (blocking legit mail) is worse.

Q2: Explain p-value to a non-technical stakeholder.

Imagine the product change actually did nothing. The p-value tells you: how often would we see results this extreme
just by random chance? p=0.03 means: only 3% of the time would we see this by accident. So we're fairly confident
the change had a real effect.

Q3: When would you use median instead of mean?

Use median when data is skewed or has outliers: income distributions, house prices, response times, any data where
extreme values exist. Median is robust to outliers; mean gets pulled toward them. Always visualize first — boxplots
and histograms tell you which to use.

Q4: What is the Law of Large Numbers vs Central Limit Theorem?

LLN: As n→∞, the sample mean converges TO the true population mean. About accuracy of a single estimate. CLT:
The DISTRIBUTION of sample means is approximately Normal, regardless of population shape. About the SHAPE of
the sampling distribution. Both require independence.

Q5: How do you handle multiple testing?

Three approaches: (1) Bonferroni correction — α* = α/m (conservative, good for few tests). (2) Benjamini-Hochberg
— controls False Discovery Rate (better for many tests). (3) Pre-register hypotheses — decide which tests matter
before collecting data.

Q6: What's the difference between correlation and covariance?

Covariance = Σ(xi−x■)(yi−■)/n — shows direction of relationship but magnitude depends on units. Correlation =
Covariance / (σx × σy) — standardized version, always in [−1, +1], unit-free. Use correlation for comparing
relationships across different variables.

Q7: You have 2 groups. How do you decide which statistical test to use?

Ask: (1) Is data continuous or categorical? (2) Are groups independent or paired? (3) Is n large (n>30) or small? (4) Is
σ known? (5) Is data normal? Large n, continuous, independent, unknown σ → Two-sample t-test (Welch's). Paired
continuous → Paired t-test. Non-normal or ordinal → Mann-Whitney U test. Categorical → Chi-square test.

Statistics Interview Guide • Page 20

Q8: What is Power in hypothesis testing?

Power = 1 − β = P(reject H■ | H■ is false) = P(detecting an effect that truly exists). Typical target: 80% power. Power
increases with: larger sample size, larger effect size, larger α (but more false positives), lower variance. Power
analysis tells you how many observations you need.

Q9: Explain overfitting and how to prevent it.

Overfitting: Model learns noise in training data and fails to generalize. Signs: very low training error but high
validation error. Prevention: (1) More data, (2) Cross-validation, (3) Regularization (Ridge L2 / Lasso L1), (4) Simpler
model, (5) Dropout (neural nets), (6) Early stopping.

Q10: What is the difference between Pearson and Spearman correlation?

Pearson r: Measures LINEAR relationship. Assumes both variables are normally distributed. Sensitive to outliers.
Use for continuous, normally distributed data. Spearman ρ: Measures MONOTONIC relationship. Uses ranks, not
raw values. Non-parametric — no normality assumption. Better for ordinal data or when outliers exist.

Statistics Interview Guide • Page 21

APPENDIX: The Ultimate Cheat Sheet
Formula What It Does

Z = (X−µ)/σ Standardize any normal variable

SE = σ/√n Standard error of sample mean

CI = X■ ± Z*(σ/√n) Confidence interval for mean

p■ ± Z*√[p■(1−p■)/n] CI for proportion

χ² = Σ(O−E)²/E Chi-square test statistic

t = (X■−µ)/(s/√n) One-sample t-test statistic

r = Cov(X,Y)/(σx·σy) Pearson correlation coefficient

β■ = Cov(X,Y)/Var(X) Regression slope

R² = 1 − SSres/SStot Coefficient of determination

MSE = Bias² + Variance Decomposition of prediction error

P(A|B) = P(A∩B)/P(B) Conditional probability

Power = 1 − β Probability of detecting true effect

MDE δ = X■_B − X■_A Minimum detectable effect

Good luck with your interview! Remember: Interviewers care more about how you THINK through a problem than
whether you recall every formula. Explain your assumptions, check edge cases, and always relate statistics back to
the business problem.

Statistics Interview Guide • Page 22