Probability & Statistics

~ Course Intro · Steve Brunton (UW) ~

What is this course about?

Probability & statistics is one of the most powerful tools to describe the
complexity of the real world.

Sits alongside the foundations of math:

★ calculus
★ linear algebra
★ differential equations
★ probability & statistics ← (this course!)

ine learning today!

→ essential for data & mach

Why? — The Real World is UNCERTAIN

Many systems are too complex to:

→ measure every detail

→ simulate every degree of freedom
→ model every physical effect

So we build probability models instead. ✿

Example 1 — Gas molecules → Thermodynamics

few numbers!

~10²³ molecules
—1—
~10²³ molecules → just a few stats numbers!
More Applications

2. Turbulence

A turbulent fluid (air over a wing, coffee being stirred) has tons of degrees of
freedom. Same idea as gas — too complex to track, so we build statistical /

probabilistic closures.

3. Measurement Error

Repeat any experiment 30 times → 30 slightly different values. The error often
follows a Normal (Gaussian) distribution.

"bell curve"
Normal /
Gaussian

x
μ−2σ μ−σ μ μ+σ μ+2σ

Historical note: Pierre-Simon Laplace was a founder of modern probability —

he independently rediscovered & massively generalized Bayesian stats.

-Laplace stats"!
→ should arguably be "Bayes

4. Kalman Filter (control / dynamics)

Merges dynamics + control with probability:

→ measurements have Gaussian error

→ unknown disturbances (wind, slope, rain…)

→ leads naturally into Stochastic Differential Eqs (SDEs)

5. Weather & Human Behavior

Catch-all for complexity + uncertainty — humans have wrestled with these

forever. (See next page!)
—2—
A Bit of Histor y — From Divination to Probability

For millennia humans have tried to handle uncertainty using divination :

★ astrology · augury
★ tea-leaf reading
★ rolling animal bones

🤔
★ pyromancy (flame patterns)

process with another!

→ trying to model one random
Big sea change with Laplace & co.:

BEFORE AFTER

astrology astronomy

alchemy chemistry

divination probability & stats

Random ≠ Always Truly Random

A coin flip is technically deterministic — F = ma, air drag, gravity. In principle
predictable!

heads

tails
Deterministic physics, but too complex to model in our heads → treat as random.

Key idea: Many "random" systems (turbulence, weather, coin flips) are
deterministic but chaotic .

Past a finite prediction horizon → use probability models. ✓

—3—
Probability vs. Statis tics — Two Sides of a Coin

Distribution Data / Samples

PROBABILITY

STATISTICS

(model is KNOWN) (observed)

PROBABILITY STATISTICS
✓ Distribution is KNOWN ✓ Data / samples KNOWN

✗ Future samples UNKNOWN ✗ Distribution UNKNOWN

→ "Given a fair coin, what's the → "Given 10 flips, is this coin fair?"
chance of 7 heads in 10 flips?"

RST so we have models to use later.

→ We'll learn probability FI

Cours e Outlin e ( ≈ 10 hrs of probability)

✓ Intro to probability — intuition + lots of examples

✓ Random variables & distributions

✓ Functions of RVs — expectation, variance, median

✓ Central Limit Theorem — bridge to statistics

Topic 1 · Intro to Probability

If you boil it down — probability is advanced counting :

★ chance of 7 heads in 10 flips?

★ how many poker hands from a 52-card deck?
★ 3 dice → P(sum = 13)?

—4—
Topic 2 · Random Variables & Distributions

A random variable X is like a regular variable, but it has a probability of taking

each value.

P( X = v ; θ )
↑ probability that X equals v, given parameters θ

Example: a Gaussian needs 2 parameters → mean μ and std deviation σ.

The "Big 5" distributions we'll meet:

① Bernoulli — single yes/no, heads/tails, success/fail

② Binomial — sum of n independent Bernoullis

e.g. how many heads in 10 flips?

③ Normal / Gaussian — large-n limit of Binomial

measurement error, the bell curve

④ Poisson — large-n limit for rare events

light bulb failures, radioactive decay

⑤ Exponential — waiting times between Poisson events

Central Limit Theorem in action!

—5—
Binomial → Normal as n grows (CLT preview!)
More on Distributions

Poisson — rare events Exponential — wait times

Poisson Exponential
(rare events) P(t) = λe^(−λt)

t (waiting)

0 1 2 3 4 5 6 7 8 9
k (number of events)

Sometimes the distribution is UNKNOWN — too complex (turbulence, weather,

human behavior).

→ Learn it from data = Machine Learning! 🤖

Topic 3 · Functions of Random Variables

Once we have a distribution, we summarize it with a few key numbers:

→ Expected value E[X] = μ ← what value do I expect?

→ Variance Var(X) = σ² ← how spread out?

→ Standard deviation σ ← spread, in original units

→ Median ← robust "middle" value

Why these matter → They're exactly what we'll estimate from data in the
statistics half of the course! Mean & variance ≈ "fingerprint" of a distribution.

sa me id ea as ga s: 10 ²³ mo lecules → just T, S, ρ.
→

—6—
Topic 4 · The Centr al Limit Theorem ★
One of the most important results in all of probability — and the
cornerstone of statistics.

The CLT (informally):

If X₁, X₂, …, Xₙ are i.i.d. random variables (any distribution!), their sample
mean

x̄ = (1/n) · ( X₁ + X₂ + … + Xₙ )

tends toward a Normal distribution as n grows.

Average enough samples → bell curve, no matter the source!

This explains the "uncanny" fact that:

★ measurement errors look Gaussian

★ noisy averages stabilize
★ large n → Normal works almost everywhere

Where Probability Meets Statis tics

In probability we computed:

P( data | θ )

In statistics we flip it:

P( θ | data )

↑ "what model best explains my observations?" —7—

Sneak Peek — Statis tics (next half)

① Hypothesis Testing

e.g. clinical trial of a new cancer drug → does it work? We collect data and
conclude things like:

"95% confident the drug has an effect"

② Survey Sampling

From a small sample (e.g. 1,000 people) infer about a huge population (e.g. 300M+
in the US).

po wered by Centra l Limit Theorem & std deviation!

→

③ Bayesian Statistics

Combine prior knowledge + data → posterior distribution.

"Bayes-Laplace" — Laplace independently rediscovered & generalized Bayes'

work.

④ Machine Learning Connection

θ doesn't have to be a "named" parameter — it can be the weights of a neural

network ! Learning weights from data = a statistics problem.

Big Pictur e — What to Take Away

✓ Probability gives us models for uncertainty.

✓ Statistics uses data to fit / test those models.

✓ The Normal distribution shows up everywhere — thanks to the CLT.

✓ ML = learning probability distributions from data.

~ Onward to Probability! ~
★ examples · intuition · models · CLT ★
—8—