Probability & Statistics — Complete Study Guide
Built from your notes, in plain language with analogies and worked examples.
1. Random Variables (RV) — The Foundation
The core idea: A random variable is just a rule that turns the outcome of a random
experiment into a number.
Analogy: Imagine a vending machine. You put in an “experiment” (a coin toss, a dice roll, a
customer walking in) and the machine spits out a number. That machine is the random
variable.
Example from your notes — tossing 3 coins:
Sample space (all possible outcomes) = {HHH, TTT, HHT, HTH, THH, TTH, THT, HTT} →
2³ = 8 outcomes
Let X = number of Heads
So X can be 0, 1, 2, or 3 — these are the “real numbers” X assigns to each outcome
P(X = 0) = 1/8 → meaning: out of all 8 equally likely ways the coins could land, exactly 1
way gives zero heads (TTT). So there’s a 1-in-8 chance of getting no heads at all.
Two types of random variables:
Type What it means Analogy Example
Rolling a die (1–
Countable outcomes — Counting people in a room
6), number of
Discrete you can list them 1, 2, — you’ll never have 2.5
defective
3… people
screws
Measuring water in a glass
Uncountable outcomes
— could be 200ml, Human height,
Continuous — any value in a range,
200.001ml, endlessly time to fix a bug
including decimals
precise
Why this distinction matters: It determines which formulas and which distributions
(Binomial/Poisson vs Normal) you’re allowed to use later. This is the first fork in the road for
every problem you’ll see.
2. Probability Distribution Function — “The Menu”
The core idea: This is just a table (or formula) that lists every possible outcome next to its
probability — like a restaurant menu listing every dish with its price.
For a fair die: f(x) = 1/6 for each of x = 1,2,3,4,5,6 — outcomes on top, probabilities
underneath.
PMF (Probability Mass Function): the menu for discrete variables (dice, coins, defect
counts)
PDF (Probability Density Function): the “menu” for continuous variables (height,
time) — except because there are infinite possible values, we can’t list them one by one,
so instead we describe probability using a curve, and area under the curve = probability.
Analogy: PMF is like a jar of individual candies — you can count exact probabilities for each
candy. PDF is like a pitcher of juice — you can’t count individual “molecules” of probability,
only measure a volume (area) between two points.
CDF (Cumulative Distribution Function): answers “what’s the probability of getting this
value or less?” It’s a running total.
Analogy: If PMF/PDF is “how many people are exactly this height,” CDF is “what percentage
of people are this height or shorter.” It always climbs upward from 0 to 1 (0% to 100%)
because you keep adding more people as height increases.
Example from your notes: rolling a die, CDF at 3 = P(X≤3) = 1/6+1/6+1/6 = 3/6 = 0.5 →
meaning: there’s a 50% chance of rolling a 3 or lower.
3. Expected Value & Variance — “Center and Spread”
Expected Value E[X] — the long-run average
Formula (discrete): E[X] = Ʃ xᵢ·P(xᵢ) Formula (continuous): E[X] = ∫ x·f(x) dx
The core idea: If you repeated the experiment thousands of times and averaged the results,
E[X] is the number you’d converge to. It’s not necessarily a value X can actually take — it’s
the “center of gravity” of the distribution.
Analogy: If you balanced a seesaw with weights placed at each possible outcome (weight =
probability), E[X] is the exact point where the seesaw balances perfectly.
Variance — how spread out the data is
Discrete: Var(X) = E[X²] − (E[X])²
Continuous: same idea, using the integral version
The core idea: Variance tells you how far, on average, the outcomes wander from the
expected value. A low variance means outcomes cluster tightly around the mean
(predictable). A high variance means outcomes are scattered widely (unpredictable).
Analogy: Two archers can both average a bullseye score of 8/10 (same E[X]), but one
archer’s arrows are tightly grouped near the center (low variance/consistent), while the
other’s are scattered all over the target (high variance/erratic). The average doesn’t tell you
consistency — variance does.
4. Discrete Probability Distributions
4.1 Bernoulli Distribution — “One Coin Flip”
Concept: Models a single trial with exactly two outcomes (success/failure, yes/no, 1/0).
Formula: P(X=x) = pˣ(1−p)¹⁻ˣ, where p = probability of “success”
Analogy: Flipping one coin once. Did a light bulb work when tested — yes or no? That’s it,
one shot.
4.2 Binomial Distribution — “Many Coin Flips”
Concept: Binomial is Bernoulli repeated n times, and you want to know the probability of
getting exactly x successes out of those n tries.
Formula: P(X=x) = C(n,x)·pˣ·(1−p)ⁿ⁻ˣ
C(n,x) = “n choose x” = number of different ways to arrange x successes among n trials
pˣ = probability all the successes happen
(1−p)ⁿ⁻ˣ = probability all the failures happen
Analogy: Instead of flipping one coin, you flip 20 coins. Binomial tells you: “what’s the
chance exactly 3 land heads?” It accounts for the fact that there are many different orders
those 3 heads could show up in among the 20 flips — that’s what C(n,x) is counting.
Worked example (from your notes): A factory produces screws with a 2% defect rate. If
an inspector checks 20 screws, what’s the probability exactly 2 are defective?
P(X=2) = C(20,2)·(0.02)²·(0.98)¹⁸ = 190 × 0.0004 × 0.98¹⁸ ≈ 0.05 = 5%
What this means in plain English: If you repeated this “check 20 screws” process over and
over, about 5% of the time you’d land on exactly 2 defective screws — not more, not fewer,
not zero. It’s a fairly unlikely but not rare outcome.
Another worked example — “at least” problems: A drug works for 80% of patients. If 10
patients are treated, what’s the probability at least 8 recover?
Key concept: “At least 8” means 8 OR 9 OR 10 — you must calculate each individually and
add them:
P(X=8) ≈ 0.302
P(X=9) ≈ 0.268
P(X=10) ≈ 0.107
P(at least 8) = P(8)+P(9)+P(10) ≈ 0.68 = 68%
(Note: your handwritten notes had some arithmetic slips here — the individual binomial
terms must each be ≤1, and probabilities can never exceed 1 or be negative. Always sanity-
check: if a probability calculation gives you a number bigger than 1, you’ve made an
arithmetic error somewhere, usually in the factorial/combination step.)
Why this matters conceptually: “At least” and “at most” are direction words. At least 8 =
8,9,10 (everything from 8 upward). At most 8 = 0,1,2…8 (everything from 8 downward).
Getting this backwards is the #1 mistake students make — always draw a number line and
shade the region the question is asking about.
4.3 Poisson Distribution — “Rare Events Over Time/Space”
Concept: Poisson counts how many times a rare, random event happens in a fixed window
of time or space, when you know the average rate (µ).
Formula: P(x, µ) = (e⁻ᵘ · µˣ) / x!
µ = the average number of events (this is given/known)
x = the specific number of events you’re asking about
e ≈ 2.71828 (a fixed constant, like π)
Analogy: Think of raindrops falling on a 1-square-meter patch of pavement during a light
drizzle. You know the average is, say, 10 drops per minute. Poisson answers: “what’s the
probability that in the next minute, exactly 15 drops fall?” — even though drops fall randomly
and independently.
When to use Poisson vs Binomial: Binomial needs a fixed number of trials (n) with a clear
success/failure per trial. Poisson is for counting rare events over a continuous interval where
there’s no natural “n” — you can’t count how many “chances” there were for a raindrop to
fall, only the rate.
Worked example (school closures): Average school closures due to snow = 10
days/winter. What’s the probability schools close for exactly 15 days this winter?
P(15,10) = (e⁻¹⁰ × 10¹⁵) / 15! ≈ 3.4%
What this means: Even though the average is 10 closure days, getting exactly 15 is not
impossible — but it’s uncommon (about a 1-in-30 chance). Poisson naturally captures that
actual counts fluctuate around the average.
Worked example (call center): Average = 100 calls/hour → what’s the probability of exactly
5 calls in the next minute?
Key conceptual step: convert the rate to match your time window. 100 calls/hour → µ =
100/60 ≈ 1.67 calls/minute (your notes used µ=100 directly against a 1-minute window,
which is a common mistake — always match units of µ to the interval in the question).
4.4 Poisson as an Approximation to Binomial
Concept: When n is very large (n ≥ 100) and p is very small (p ≤ 0.01), calculating Binomial
directly involves huge factorials that are a pain to compute. Poisson gives you a close, much
simpler approximation.
Analogy: It’s like using a shortcut formula instead of doing long division by hand — the
answer comes out nearly identical, but far less effort.
Rule: Set µ = n × p, then plug into the Poisson formula instead of Binomial.
Worked example: A factory makes 500 footballs with 1% defect rate. Probability exactly 10
are defective?
µ = n×p = 500×0.01 = 5
P(10, 5) = (e⁻⁵ × 5¹⁰)/10! ≈ 1.8% (your notes’ final arithmetic had an error yielding a
negative number — probabilities can never be negative; a negative result always signals
a computational mistake, not a valid answer)
5. Normal Distribution — The Bell Curve
Concept: Models continuous data that clusters around a center and tapers off
symmetrically on both sides — the most common pattern in nature (heights, test scores,
measurement errors, etc.)
Key property: Mean = Median = Mode — all three “center” measures land in the same
spot, at the peak of the curve.
Analogy: Think of a dartboard where most people’s throws land near the bullseye, with
fewer landing far away, and it’s equally likely to overshoot left as it is right. That symmetric
pile-up around the center is a normal distribution.
The Empirical Rule (68–95–99.7 Rule)
This is one of the most useful shortcuts in statistics:
68% of data falls within 1 standard deviation (σ) of the mean (µ−σ to µ+σ)
95% falls within 2 standard deviations (µ−2σ to µ+2σ)
99.7% falls within 3 standard deviations (µ−3σ to µ+3σ)
Analogy: Imagine standard deviation as “rings” around a target, like an archery target’s
rings. The 68% ring is close to the bullseye, 95% is a bigger ring, and 99.7% nearly covers
the entire board. Almost everything lands somewhere on the board.
Worked example (DevOps bug fix times): Mean time to resolve a bug = 45 hours,
standard deviation = 5 hours.
(A) P(40 ≤ X ≤ 50) — probability the fix takes between 40 and 50 hours. Notice: 40 = µ−σ,
and 50 = µ+σ. So this is exactly the 68% empirical rule range! Using numerical integration
(Simpson’s rule, as in your notes) confirms: ≈ 68%
What this means: If you watched 100 different bugs get fixed, about 68 of them would take
somewhere between 40 and 50 hours. It’s the “typical” range — neither unusually fast nor
unusually slow.
(B) P(X < 40) — probability the fix takes less than 40 hours. Since 68% is the middle chunk
(symmetric around the mean), the remaining 32% is split evenly between the two tails: (1 −
0.68) / 2 = 0.16 → 16%
What this means: Only 16% of bugs get fixed unusually fast (under 40 hours) — it’s a
below-average, “lucky” outcome, but not rare.
The Z-score — your universal translator
Z = (X − µ) / σ
Concept: Z-score converts any normal distribution into a standard scale, telling you “how
many standard deviations away from the mean is this value?” This lets you compare wildly
different scales (heights in cm vs. exam scores out of 100) using the same yardstick.
Analogy: It’s like converting different currencies into a single reference currency (like USD)
so you can directly compare prices from different countries.
6. Sampling Distribution & Central Limit Theorem (CLT)
Concept: If you take many different samples from a population and compute the mean of
each sample, those sample means themselves form their own distribution — the sampling
distribution of the mean.
The Central Limit Theorem’s big claim: No matter what shape the original population data
has (skewed, weird, non-normal) — as long as your sample size is n ≥ 30, the distribution of
sample means will approximate a normal distribution.
Analogy: Imagine a bag of unevenly-shaped rocks (population). If you scoop out random
handfuls (samples) and calculate the average weight of each handful, and do this thousands
of times — those handful-averages will form a nice bell curve, even though individual rocks
are all shapes and sizes. Averaging smooths out the underlying weirdness.
Why this matters so much: It’s the reason we’re allowed to use the normal distribution
(and Z-scores) for so many real-world statistics problems, even when we don’t know the
population’s true shape. It’s the bridge that connects “messy real data” to “clean,
predictable math.”
7. Estimation of Parameters (Confidence Intervals)
The core idea: We usually can’t measure an entire population (e.g., every battery a
company ever made). Instead, we take a sample, calculate a statistic (like the sample
mean), and use it to make an educated guess — with a margin of error — about the true
population value.
Two types of estimates:
Point estimate: a single number, your “best guess” (e.g., “average battery life is 4.75
years”) — but it’s 100% likely to be at least slightly off.
Interval estimate (Confidence Interval): a range of values you’re fairly confident (say
95%) contains the true population value.
Analogy: Guessing someone’s weight by looking at them once (point estimate) vs. saying
“I’m 95% confident they weigh between 65–75kg” (confidence interval). The range
acknowledges your uncertainty honestly.
Z-statistics (when population standard deviation σ is known, n ≥ 30)
CI = x̄ ± Zα/2 · (σ/√n)
x̄ = sample mean
σ/√n = standard error — how much sample means typically vary from the true mean
Zα/2 = a lookup value based on your confidence level (e.g., 1.65 for 90%, 1.96 for 95%)
Worked example (network load): Sample mean = 12.5ms, n=40, σ=2, confidence level =
90%
CI = 12.5 ± 1.65 × (2/√40) = 12.5 ± 0.522 → (11.978, 13.022)
What this means in plain English: We are 90% confident that the true average network
load (across all network traffic, not just our sample) lies somewhere between 11.978ms and
13.022ms. It does not mean there’s a 90% chance the true value is in this specific range
(the true value is fixed, not random) — it means if we repeated this sampling process many
times, 90% of the intervals we’d construct would capture the true mean.
t-statistics (when population standard deviation is UNKNOWN, n < 30)
t = x̄ ± t(confidence, df) · (S/√n), where df (degrees of freedom) = n − 1
Why a different formula? When you don’t know σ and have to estimate it from your small
sample (using S, the sample standard deviation), there’s extra uncertainty — small samples
are less reliable. The t-distribution has “fatter tails” than the normal distribution to account
for that added uncertainty. As your sample size grows, the t-distribution starts to look just
like the normal distribution.
Analogy: Z-statistics is like measuring with a laser-precise ruler (you know the true
variability). t-statistics is like measuring with a slightly wobbly ruler you had to calibrate
yourself from a small sample — you build in extra caution because your tool itself is less
certain.
Worked example: A study on political party supporters: n=15, mean age=35, S=4,
confidence=99%. df = 15−1 = 14 t = 35 ± t(0.99, df=14) × (4/√15)
Concept check: Notice n=15 < 30, so we MUST use t-statistics, not Z-statistics — this is
the very first decision to make in any confidence interval problem.
8. Hypothesis Testing
The core idea: You start with a claim (“the average battery lasts 5 years”) and use sample
data to decide: does the evidence support this claim, or contradict it?
Two competing hypotheses:
Null Hypothesis (H₀): the “status quo” claim, assumed true until proven otherwise (like
“innocent until proven guilty” in court)
Alternate Hypothesis (H₁): what you’d conclude if the evidence strongly contradicts H₀
Analogy — courtroom trial: H₀ = “the defendant is innocent” (assumed by default). You
only reject H₀ (declare guilty) if the evidence is overwhelming. You never “prove” innocence
— you just fail to find enough evidence of guilt.
Choosing your test
Condition Test to use
Population σ known, n ≥ 30 Z-test
Population σ unknown, n < 30 t-test
Two types of errors
Type I Error: rejecting H₀ when it was actually TRUE (a “false alarm” — convicting an
innocent person)
Type II Error: failing to reject H₀ when it was actually FALSE (a “missed detection” —
letting a guilty person go free)
Analogy: A smoke detector. Type I error = alarm goes off but there’s no fire (annoying false
alarm). Type II error = there IS a fire but the alarm stays silent (dangerous miss).
Statisticians try to balance the risk of both, but usually prioritize keeping Type I error low
(typically at 5%, called α).
One-tailed vs Two-tailed tests
Right-tailed test: H₀: µ ≤ 5, H₁: µ > 5 — you’re only checking if the true value is
significantly higher
Left-tailed test: H₀: µ ≥ 5, H₁: µ < 5 — only checking if it’s significantly lower
Two-tailed test: H₀: µ = 5, H₁: µ ≠ 5 — checking if it’s significantly different in either
direction
Analogy: A two-tailed test is like checking if a coin is “unfair” in either direction (too many
heads OR too many tails). A one-tailed test is like only worrying about too many heads
specifically, and not caring if there are too many tails.
Worked example: A company claims average battery life = 5 years, σ=2 years (known
population std dev, so Z-test applies). Inspector samples 50 batteries, finds mean = 4.75
years. At 95% confidence, does the inspector reject or fail to reject the claim?
Z = (x̄ − µ) / (σ/√n) = (4.75 − 5) / (2/√50) ≈ −0.88
(Your notes had a computational error giving −0.017 — double-check: 2/√50 ≈ 0.283, and
−0.25/0.283 ≈ −0.88)
Decision rule: For a two-tailed test at 95% confidence, the “rejection region” starts beyond
±1.96. Since −0.88 is well inside that range (between −1.96 and +1.96), we fail to reject H₀.
What this means in plain English: The sample average (4.75) is lower than the claimed 5
years, but not lower by a statistically significant amount — it’s well within the range of
natural sample-to-sample variation you’d expect even if the company’s claim were true. The
inspector doesn’t have strong enough evidence to say the company is lying.
9. Covariance & Correlation
Covariance — direction of a relationship
Cov(x,y) = Ʃ(xᵢ−x̄ )(yᵢ−ȳ) / (n−1) [for sample data]
Concept: Covariance tells you whether two variables tend to move together (both up, or
both down) or oppositely.
Positive covariance: when X goes up, Y tends to go up too (e.g., height and weight)
Negative covariance: when X goes up, Y tends to go down (e.g., price and demand)
Near-zero covariance: no consistent relationship
Analogy: Think of two dancers. Positive covariance = they move in sync, both stepping
forward together. Negative covariance = one steps forward exactly as the other steps back.
Zero covariance = their movements are unrelated, like two strangers dancing to different
songs.
Worked example (height & weight): Using your data (6 people’s height & weight), Cov(x,y)
≈ 70.9 (positive)
What this means: As height increases across this group, weight tends to increase too. The
sign (positive) tells us the direction — but the raw number 70.9 is hard to interpret on its
own because it depends on the units (cm × kg). This is covariance’s weakness — which
brings us to correlation.
Correlation — direction AND strength, on a fixed scale
Corr(x,y) = Cov(x,y) / (Sx · Sy), always between −1 and +1
Concept: Correlation is covariance, but “standardized” so it always falls between −1 and +1,
no matter what units your original data was in. This makes it directly comparable and
interpretable.
Value Meaning
+1 Perfect positive relationship (as X rises, Y rises predictably)
0 No linear relationship at all
−1 Perfect negative relationship (as X rises, Y falls predictably)
Analogy: Covariance is like saying “these two dancers are somewhat in sync” without a
number. Correlation is like giving them a “sync score out of 100%” — precise, comparable,
and easy to interpret regardless of who’s dancing.
Correlation ≠ Causation
Your notes’ example: Ice cream sales go up ↑ AND shark attacks go up ↑ at the same
time. They’re correlated — but ice cream doesn’t cause shark attacks!
What’s really happening: A hidden third factor (hot summer weather) drives both — more
people swim (more shark encounters) AND more people buy ice cream. This is called a
confounding variable.
Why this matters: Just because two things move together doesn’t mean one causes the
other. This is one of the most important lessons in all of statistics — always ask “could
something else be driving both of these?”
10. Linear Regression
Concept: While correlation just tells you how strongly two variables relate, linear regression
goes a step further — it finds the actual equation of the best-fit straight line through your
data, so you can predict Y from X.
Equation: Y = mx + c
m = slope — how much Y changes for every 1-unit increase in X
c = intercept — the value of Y when X = 0 (where the line crosses the Y-axis)
Analogy: If correlation tells you “yes, taller people tend to weigh more,” regression gives
you an actual formula: “for every extra cm of height, expect about 0.9kg more weight” —
letting you predict a specific person’s weight from their height.
Formulas:
m = Ʃ(xᵢ−x̄ )(yᵢ−ȳ) / Ʃ(xᵢ−x̄ )²
c = ȳ − m·x̄
Worked example from your notes:
x y
1 5
2 6
3 7
8 10
11 15
m = 68/74 ≈ 0.92
What this means: For every 1-unit increase in x, y increases by about 0.92 units on
average. This is the “rate of change” — the heart of the prediction.
Error (Residuals): the difference between what the line predicts and what actually
happened: Error = (ŷ − y) for each point, averaged.
Analogy: Think of the regression line as a weather forecaster’s prediction model. Error
measures how far off the forecast was from the actual weather each day. A good model has
small, evenly-scattered errors; a bad model has large, systematic errors.
11. Statistical Quality Control (Control Charts)
Concept: Factories take repeated small samples of their product over time and track
whether measurements stay within an acceptable range. If measurements drift outside the
range, it signals something has gone wrong in the process — before it becomes a bigger,
costlier problem.
Analogy: Think of a doctor tracking your temperature every day. Normal body temp
fluctuates slightly, but if it suddenly spikes far outside the normal range, that’s a signal
something’s wrong — you don’t wait for a full-blown fever to investigate.
Two charts working together:
Mean Chart (x̄ -chart) — tracks the center of your process
Upper Control Limit (UCL) = x̄ + A₂·R̄
Lower Control Limit (LCL) = x̄ − A₂·R̄
Range Chart (R-chart) — tracks the consistency/spread of your process
UCL = D₄·R̄
LCL = D₃·R̄
(A₂, D₃, D₄ are lookup constants based on your sample size n)
Worked example (from your notes): 5 samples of size n=5, grand mean x̄ =23.3, average
range R̄ =1.54, A₂=0.577, D₃=0, D₄=2.114
Mean chart UCL = 23.3 + (0.577)(1.54) ≈ 24.18
Mean chart LCL = 23.3 − (0.577)(1.54) ≈ 22.44
Range chart UCL = (2.114)(1.54) ≈ 3.25
Range chart LCL = (0)(1.54) = 0
What this means in plain English: As long as each sample’s average stays between 22.44
and 24.18 (mean chart), AND the spread within each sample stays under 3.25 (range chart),
the production process is considered “in control” — behaving normally, with only natural
random variation. If a point ever falls outside these lines, it’s a red flag that something
specific changed (a machine miscalibrated, a new batch of raw material, etc.) and needs
investigating — this is different from normal random noise.
Why two charts, not one? The mean chart alone could miss a process that’s becoming
more erratic even if its average stays the same (like an archer whose shots are still centered
on the bullseye on average, but now scattered wildly around it). The range chart catches
that increased inconsistency.
Quick-Reference: “Which Tool Do I Use?”
Is your data COUNTABLE (discrete) or MEASURED (continuous)?
│
├── DISCRETE
│ ├── One trial, 2 outcomes → Bernoulli
│ ├── Fixed n trials, 2 outcomes each → Binomial
│ └── Rare events over time/space, known average rate → Poisson
│
└── CONTINUOUS
└── Bell-shaped, symmetric data → Normal Distribution
└── Need to estimate the population mean from a sample?
├── σ known, n ≥ 30 → Z-statistics (CI or Z-test)
└── σ unknown, n < 30 → t-statistics (CI or t-test)
Studying the relationship between TWO variables?
├── Just the direction & raw relationship → Covariance
├── Standardized strength (−1 to 1) → Correlation
└── Want to PREDICT one from the other → Linear Regression
Monitoring an ongoing PROCESS over time?
└── Control Charts (Mean chart + Range chart)
The Golden Rule for Every Word Problem
1. What kind of variable is this? (discrete or continuous) → picks your distribution family
2. What EXACTLY is being asked? (“exactly,” “at least,” “at most,” “between”) → this
changes which probabilities you add up
3. After you get a number, ask: “what does this number mean in the real world?” — a
probability of 0.05 isn’t just “5%,” it’s “if we repeated this 100 times, we’d expect this
outcome about 5 of those times.” Always translate the math back into a sentence about
the actual scenario. This is exactly what your teacher is grading you on — not just
getting the number, but understanding what it represents.