Scribd
Upload
2 views7 pages

Sample Size Calculation

The document outlines sample size calculation for discrete and continuous variables using a standard formula. It details how to calculate variance for binary metrics and continuous metrics, providing examples for signup completion rates and average revenue per user. The document also explains the significance of alpha (α) and power in determining sample size.

Uploaded by

dasarojit
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views7 pages

Sample Size Calculation

The document outlines sample size calculation for discrete and continuous variables using a standard formula. It details how to calculate variance for binary metrics and continuous metrics, providing examples for signup completion rates and average revenue per user. The document also explains the significance of alpha (α) and power in determining sample size.

Uploaded by

dasarojit
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

SAMPLE SIZE

CALCULATION
DISCRETE VS CONTINUOUS
VARIABLES
STANDARD FORMULA
(Both Types)

n = 16σ² / δ²
n = sample size per group

σ² = variance

δ = minimum detectable effect (MDE)

The key difference: How you calculate variance (σ²)


Examples: Conversion rate, click-
through rate, signup completion, any
yes/no outcome

DISCRETE How to Calculate Variance:


(BINARY) For binary metrics: σ² = p(1-p)
Where p = baseline conversion rate

VARIABLES
LinkedIn Metric: Signup completion rate (yes/no)

Signup Given:
Baseline rate (p) = 70% = 0.70
Completion MDE (δ) = 2 percentage points = 0.02
α = 0.05, Power = 0.8
(Discrete)
Step 1: Calculate Variance
σ² = p(1-p)
σ² = 0.70 × 0.30
σ² = 0.21

Step 2: Calculate Sample Size


n = 16σ² / δ²
n = 16 × 0.21 / (0.02)²
n = 3.36 / 0.0004
n = 8,400 users per group

Total needed: 16,800 users (8,400 control + 8,400 treatment)


Examples: Revenue per user, time
on site, session duration, cart value

CONTINUOUS How to Calculate Variance:

VARIABLES For continuous metrics: σ² =


standard deviation²
Get from historical data or A/A
tests.
Metric: Average revenue per user

Revenue Per Given:


Baseline: $50/user

User Standard deviation: $200 (very high - some buy $0, some buy $500+)
MDE (δ) = $5 (want to detect $5 increase)

(Continuous) α = 0.05, Power = 0.8

Step 1: Variance is Already Given


σ² = (standard deviation)²
σ² = $200²
σ² = $40,000

Step 2: Calculate Sample Size


n = 16σ² / δ²
n = 16 × 40,000 / (5)²
n = 640,000 / 25
n = 25,600 users per group

Total needed: 51,200 users


Where α and Power Show Up in the Formula:
Standard Formula:
n = (σ² + σ²) × (z₁₋α/₂ + z₁₋β)² / δ²
Simplified Formula (assuming equal variance):
n = 16σ² / δ²
That "16" comes from α and Power!

You might also like