Math for AI

Probability, Bayes & First Models

Tutorial — with full worked solutions

July 2026
How this session works

The method. Every concept gets a one-line idea and a tiny warm-up on easy numbers. The
warm-ups chain together, so by the end we can solve the hard problems that appear later in
this session.

Part 1 — foundations Part 2 — building blocks

1. Probability in a table 7. Maximum likelihood (MLE)
2. Bayes’ rule → medical screening 8. Sigmoid & softmax
3. Bayes, 3 outcomes → battery QC 9. Multivariate Gaussians
4. Densities & Gaussian → continuous score 10. Generative classification / GDA
5. Least squares → used-car regression
6. Convolution → kernel quiz

1/35
Part 1 — Probability, Bayes &
First Models
Task 1 — Probability in a table

Idea. A table of counts splits a group into buckets. Row/column totals are marginals; one
cell over the total is a joint; restrict to one row and re-divide to get a conditional.

200 commuters — weather vs. umbrella 40

Joint p(Rain, Umb) = = 0.20
Umbrella None Total 200
50
Rain 40 10 50 Marginal p(Rain) = = 0.25
200
No rain 30 120 150 40
Conditional p(Umb | Rain) = = 0.80
Total 70 130 200 50
(stay in the Rain row, then re-divide)

2/35
Task 1 — the two rules fall straight out

Sum rule — a marginal is a row/column total:

40 + 10 The seed of Bayes. Flipping the condition

p(Rain) = = 0.25.
200 changes the answer:
Product rule — joint = conditional × p(Umb | Rain) = 0.80
marginal:
40
p(Rain | Umb) = ≈ 0.57.
40 40 50 70
= · = 0.20. ✓
200 |{z}
50 |{z}
200 Bayes’ rule is exactly this flip.
0.80 0.25

3/35
Task 2 — Bayes’ rule, the easy way

Idea. Bayes flips a conditional. We usually know p(test | disease) but want p(disease | test):

p(B | A) p(A)
p(A | B) = .
p(B)

0.80 × 0.25
Same table as a check: p(Rain | Umb) = = 0.57 ✓
0.35
The denominator uses total probability (add up every way the evidence happens):

p(Umb) = 0.80 × 0.25 + 0.20 × 0.75 = 0.35.

| {z } | {z }
rain no rain

4/35
Main problem — medical screening (Day 3)

Problem
Screen a population for cancer. 1% actually have cancer. Of healthy people, 3% test
positive (false positives). Of people with cancer, 10% test negative (false negatives).

With C = cancer state, T = test result:

p(C =1) = 0.01, p(C =0) = 0.99,

p(T =1 | C =1) = 0.90, p(T =1 | C =0) = 0.03.

Q1. Chance a random person tests positive? Q2. Given a positive test, chance they truly have
cancer?

Try it yourself first — the solution is on the next slide.

5/35
Medical screening — solution

Q1 — total probability

p(T =1) = 0.90 × 0.01 + 0.03 × 0.99 = 0.009 + 0.0297

| {z } | {z }
has cancer healthy

p(T =1) = 0.0387

Q2 — Bayes

0.90 × 0.01 0.009

p(C =1 | T =1) = = = 0.233
0.0387 0.0387

Prior → posterior: cancer chance rises from 1% to ∼ 23% — not higher, because healthy people are 99×
more common, so their false positives dominate.

6/35
Medical screening — the same answer, in counts

Natural frequencies. The same problem in whole people — the form most minds find
intuitive.

90% 90 test +
100
cancer
1% 10%
10 test −
10,000
people
99% 297 test +
3%
9,900
healthy
97% 9,603 test −

90
Positives = |{z}
90 + 297 = 387, P(cancer | +) = = 0.233.
|{z} 387
cancer false alarms
The 297 false alarms swamp the 90 real ones — that is the whole story.
7/35
Task 3 — Bayes with 3 outcomes, and independence

Idea. The same engine works with more than two outcomes. Two variables are independent
only if conditioning on one leaves the other’s probabilities unchanged.

Tiny example — the independence test.

ˆ Two fair coins: p(2nd=H | 1st=H) = 0.5 = p(2nd=H) ⇒ independent.

ˆ Two cards, no replacement: p(2nd ace | 1st ace) ̸= p(2nd ace) ⇒ dependent.

Test: does conditioning change the number?

8/35
Main problem — battery quality control (Day 4)

Problem
A scanner inspects batteries. D ∈ {def, not def}, R ∈ {pass, manual, reject}. 5% are
defective. Each row gives P(R | D).

P(def) = 0.05, P(not def) = 0.95

pass manual review reject

not defective 0.94 0.05 0.01
defective 0.10 0.30 0.60

Ask: marginal P(R); P(def | manual); P(def | reject); P(def | pass); are D, R independent?

Try it yourself first — the solution is on the next slide.

9/35
Battery QC — the three marginals

Total probability, one scanner outcome at a time:

P(pass) = 0.94(0.95) + 0.10(0.05) = 0.893 + 0.005

P(manual) = 0.05(0.95) + 0.30(0.05) = 0.0475 + 0.015

P(reject) = 0.01(0.95) + 0.60(0.05) = 0.0095 + 0.030

P(pass) = 0.898 P(manual) = 0.0625 P(reject) = 0.0395

(they sum to 1 — a good live check)

10/35
Battery QC — posteriors and independence

Bayes for each outcome:

0.015 0.030
P(def | manual) = = 0.240 P(def | reject) = = 0.759
0.0625 0.0395
0.005
P(def | pass) = = 0.0056
0.898

Independent? Compare a conditional to its marginal:

P(pass | defective) = 0.10 ̸= P(pass) = 0.898 ⇒ not independent.

(as it must be — a useful scanner’s output depends on the true quality)

11/35
Task 4 — from counts to curves (continuous variables)

Idea. For a continuous score, probability is the area under a density curve. The bell curve
is the Gaussian N (µ, σ 2 ); areas come from the CDF Φ(z) = P(Z ≤ z) of the standard
N (0, 1).

a − µ
Mechanic to drill — the z-score. P(X > a) = 1 − Φ
σ
a−µ
= how many standard deviations a sits above the mean.
σ
X ∼ N (0, 1) : X ∼ N (3, 1) :

P(X > 2) = 1 − Φ(2) ≈ 0.0228 P(X > 2) = Φ(1) ≈ 0.8413

(Φ comes from a standard-normal table or software; these two areas are what the main problem needs)

12/35
The Gaussian — what µ and σ 2 mean

µ is the centre (where the bell sits); σ 2 is the spread (how wide it is).

σ 2 =0.5: low variance, narrow

σ 2 =1
σ 2 =4: high variance, wide
density

µ=−3: same shape, shifted

−6 −4 −2 0 2 4 6
x

Same µ: larger σ 2 ⇒ wider and flatter; as σ 2 → 0 the bell collapses to a spike at µ (every value equals
the mean). Changing µ just slides the curve (amber).

13/35
Main problem — cancer screening, continuous score (Day 4)

Problem
A blood test gives a continuous score X (larger = more suspicious). Cancer prevalence 1%:

X | no cancer ∼ N (0, 1), X | cancer ∼ N (3, 1).

A patient is high risk if X > 2.

Ask: (1) probability of a high-risk result; (2) given high risk, probability of cancer; (3) probability of a
false negative.

Try it yourself first — the solution is on the next slide.

14/35
Continuous screening — solution

Q1 — total probability

P(X > 2) = 0.02275 × 0.99 + 0.8413 × 0.01 = 0.02252 + 0.00841

| {z } | {z }
no cancer cancer

0.00841
P(X > 2) = 0.0309 P(cancer | X > 2) = = 0.272 FN = Φ(−1) = 0.159
0.0309

(a false negative is a cancer patient scoring ≤ 2: P(X ≤ 2 | cancer) = Φ(−1))

Same moral as Day 3, now continuous: a high-risk flag still means only ∼ 27% chance of cancer, because
cancer is rare.

15/35
Task 5 — fitting a line (least squares)

Idea. Linear regression predicts a number as intercept + a weighted sum of features. “Best”
weights minimise total squared error; the solution is one formula.

Toy: 3 cars, feature age (yrs), price (k-SAR): (1, 18), (2, 15), (4, 9). Put a ones column first:
   
1 1 " # 18
θ0
X = 1 2 , θ= , y = 15
   
θ1
1 4 (3×2) (2×1) 9 (3×1)

Minimise ∥X θ − y ∥2 via the normal equations θ̂ = (X ⊤ X )−1 X ⊤ y = (21, −3):

d = 21 − 3 · age.
price

16/35
Main problem — used-car regression (Quiz a)

Problem
n = 1500 used-car sales. For car i: yi resale price (k-SAR), xi1 age (yrs), xi2 odometer
(10,000 km), xi3 ∈ {0, 1} accident, xi4 number of previous owners. Build a linear regression
to predict price.

Ask:

ˆ formulate parameter estimation (state the dimensions of every quantity);

ˆ write the solution in Python pseudocode;
ˆ how would you evaluate the model, and how might you improve it?

Try it yourself first — the solution is on the next slide.

17/35
Regression — formulation and dimensions

Stack the data; a leading ones column carries the intercept:

X ∈ R1500×5 , θ ∈ R5 , y ∈ R1500 .

ŷi = θ0 + θ1 xi1 + θ2 xi2 + θ3 xi3 + θ4 xi4 , θ̂ = arg min ∥X θ − y ∥2 .

θ

Closed-form solution — the normal equations, with X ⊤ X ∈ R5×5 , X ⊤ y ∈ R5 :

θ̂ = (X ⊤ X )−1 X ⊤ y

18/35
Regression — Python pseudocode

import numpy as np
n = 1500 # number of cars

ones = [Link]((n, 1)) # intercept column, (n,1)

X = [Link]([ones, age, odo, acc, own]) # design matrix, (n,5)
# y : resale prices, shape (n,)

# normal equations: theta = (X^T X)^(-1) X^T y

theta = [Link](X.T @ X, X.T @ y) # (5,) -- @ is matmul

# numerically safer (QR / least squares):

# theta, *_ = [Link](X, y, rcond=None)

y_pred = X @ theta # predictions, (n,)

19/35
Regression — evaluate and improve

Evaluate (on held-out data) Improve

ˆ train/test split or k-fold cross-validation ˆ standardise features (comparable scales)
ˆ RMSE, MAE (in k-SAR — interpretable) ˆ add nonlinear / interaction terms (log
ˆ R (variance explained)
2 odometer)

ˆ plot residuals — should be patternless ˆ ridge / lasso regularisation

ˆ check age–odometer collinearity
ˆ remove outliers; add richer features

20/35
Task 6 — sliding a window (convolution)

Idea. A kernel slides along the signal; at each stop, multiply overlapping entries and add. On
P
a flat patch the output is ( kernel) × c: sum = 1 → averaging/blur; sum = 0 → detector.

Tiny example. signal [2, 2, 2, 7, 7], kernel [−1, 1]:

[2, 2] → 0, [2, 2] → 0, [2, 7] → 5, [7, 7] → 0 ⇒ [0, 0, 5, 0].

Flat regions give 0; a single spike appears exactly at the 2 → 7 edge.

21/35
Main problem — convolution quiz (Quiz b)

Problem
" #
0.25 0.25 h i
B1 = (entries sum to 1); B2 = −1 1 (entries sum to 0).
0.25 0.25

A1. Output of each kernel on a flat region, and why? A2. Which is a feature detector, and what does
it respond to?
h i h i
For X = 1 1 1 0 1 1 1 and B = −1 2 −1 , compute X ∗ B over the 5 positions where
B fits inside X .
B1. the 5 output values. B2. where is the largest-magnitude value, and what feature sits there?

Try it yourself first — the solution is on the next slide.

22/35
Convolution — parts A1 and A2

P
A1 — flat region. Output on a constant c is ( kernel) × c:

B1 (sum 1) → c: reproduces the constant (a blur). B2 (sum 0) → 0: flats vanish.

A2 — feature detector.

B2 is the detector: it zeroes flats and fires only where brightness changes — i.e. it detects
edges. B1 merely averages.

23/35
Convolution — parts B1 and B2

window of X compute out

[1, 1, 1] −1+2−1 0
B1: [ 0, 1, −2, 1, 0 ]
[1, 1, 0] −1+2−0 1
[1, 0, 1] −1+0−1 −2 B2: largest magnitude −2 at the middle, cen-
[0, 1, 1] 0+2−1 1 tred on the isolated 0 (the dip). [−1, 2, −1]
[1, 1, 1] −1+2−1 0 is a spot/edge detector; it fires hardest at the
sharpest local change.

(kernel is symmetric, so convolution = this

slide-and-multiply)

24/35
Part 2 — Likelihoods, Squashing
Functions & Generative
Classification
Task 7 — maximum likelihood (MLE)

Idea. The likelihood is how probable the data is for a candidate (µ, σ 2 ). MLE picks the
values that maximise it — for a Gaussian, just the data’s mean and variance.

N N
1 X 2 1 X
µML = xn , σML = (xn − µML )2 .
N n=1 N n=1
Why the log? It turns the product of N tiny numbers into a sum (easier, no underflow) and the
maximiser is unchanged. The value minimising (xn − µ)2 is the mean.
P

25/35
MLE — tiny example

Data x = [2, 4, 4, 4, 5, 5, 7, 9].

2+4+4+4+5+5+7+9 40
µML = = = 5.
8 8
Squared deviations from 5: 9, 1, 1, 1, 0, 0, 4, 16 (sum 32), so

2 32
σML = = 4, σML = 2.
8

Aside: dividing by N is the MLE (the 1/N form above); dividing by N − 1 is the unbiased estimator. For
large N the difference is tiny.

26/35
Task 8 — sigmoid and softmax (scores → probabilities)

Idea. Models output raw scores; these functions squash them into valid probabilities. Sigmoid
→ one yes/no probability; softmax → a distribution over K classes.

1 e zk
σ(x) = ∈ (0, 1), softmax(z)k = PK .
1 + e −x j=1 e zj
1
Sigmoid. σ(0) = 0.5 (no idea); σ(2) = = 0.881 (leans yes).
1 + e −2

27/35
Softmax — example and a neat connection

Scores z = (2, 1, 0): e z = (7.39, 2.72, 1.00), sum = 11.11:

softmax(z) = (0.665, 0.245, 0.090) — in (0, 1), sums to 1, biggest score wins.

Softmax with K = 2 is the sigmoid:

e z1 1
softmax(z1 , z2 )1 = = = σ(z1 − z2 ).
e z1 + e z2 1 + e 1 −z2 )
−(z

Used as: sigmoid = output of logistic regression (binary); softmax = output layer of a multiclass network.

28/35
Task 9 — multivariate Gaussians

Idea. The bell curve in D dimensions — a mean vector µ (centre) and a covariance matrix
Σ (spread and tilt). Contours are ellipses.

1 h i
N (x | µ, Σ) = exp − 12 (x − µ)⊤ Σ−1 (x − µ)
(2π)D/2 |Σ|1/2
Reading Σ: diagonal = variance along each axis; off-diagonal = covariance. Zero off-diagonal →
axis-aligned (diagonal); equal variances too → circle (spherical); non-zero off-diagonal → tilted ellipse
(full).

29/35
Multivariate MLE — tiny example

Four points (0, 1), (2, 1), (2, 3), (4, 3) — same MLE formulas, now with vectors:
1
 
µML = 4 (0, 1) + (2, 1) + (2, 3) + (4, 3) = (2, 2).

Deviations (−2, −1), (0, −1), (0, 1), (2, 1) give

" #
1 X ⊤ 2 1
ΣML = (xn − µ)(xn − µ) =
N 1 1

More spread in x1 (variance 2) than x2 (1), and a positive off-diagonal ⇒ the cloud tilts
up-and-to-the-right.

30/35
Task 10 — generative classification and GDA

Idea. Model each class with its own Gaussian, then use Bayes to ask “which class most
likely generated this point?” Since p(x) is shared, compare numerators:

ŷ = arg max p(x | y =c) p(y =c) .

c | {z } | {z }
class Gaussian prior Nc /N

GDA = fit one Gaussian N (µc , Σc ) per class by MLE (Task 9).

31/35
GDA — a worked 2-class example

Training points: Class 0: (1, 1), (1, 3), (3, 1), (3, 3); Class 1: (4, 6), (8, 6), (6, 5), (6, 7).

Fit each class (equal priors, N0 = N1 = 4):

" # " #
1 0 2 0
µ0 = (2, 2), Σ0 = ; µ1 = (6, 6), Σ1 = ; p(y =0) = p(y =1) = 0.5.
0 1 0 0.5

Compare the log-posteriors (shared constants cancel):

gc (x) = log p(y =c) − 1

2
log |Σc | − 21 (x − µc )⊤ Σ−1
c (x − µc ).

this is log N (x | µc , Σc ) p(y =c) with the (2π)D/2 factor (the same for every class) dropped.
 

32/35
GDA — classify a new point

Test point x ∗ = (4, 4):

Class 0: x − µ0 = (2, 2), Σ−1
0 = I , |Σ0 | = 1 Class 1: x − µ1 = (−2, −2), Σ−1
1 = diag(0.5, 2),
|Σ1 | = 1
g0 = log 0.5 − 0 − 21 (4+4) = −4.69
g1 = log 0.5 − 0 − 21 (2+8) = −5.69

g0 > g1 ⇒ classify x ∗ as class 0 (it sits nearer the class-0 cluster).

33/35
GDA — why the boundary is quadratic

The boundary is where g0 (x) = g1 (x). The x-dependent quadratic parts differ:
" #
1 ⊤ −1 −1 1 ⊤ 0.5 0
− 2 x (Σ0 − Σ1 ) x = − 2 x x = −0.25 x12 + 0.5 x22 .
0 −1

Because Σ0 ̸= Σ1 this does not cancel — the boundary carries x12 , x22 terms, so it is a curve
(quadratic). Equal covariances ⇒ a straight line.

Trade-offs. Generative models learn each class’s structure — good with little data, can generate
samples, handle missing data — but they assume the class really is Gaussian, and mislead when it
isn’t.

34/35
Summary — five problems, one toolkit

Solved today The one engine

ˆ Medical screening — p(cancer | +) = 23%
Total probability builds the denominator; Bayes
ˆ Battery QC — marginals, posteriors,
flips the conditional — whether the pieces are
dependence
table cells, Gaussian areas, or per-class models.
ˆ Continuous score — 27%, FN = 16%
ˆ Used-car regression — normal equations
ˆ Convolution — [0, 1, −2, 1, 0], dip at centre

Each problem sits one slide before its solution — try it first, then turn the page.

35/35