Normal (Gaussian) Random Variable
mean
variance
Support:
Piech & Cain, CS109, Stanford University
Normal (Gaussian) Random Variable
mean
variance
Support:
PDF:
Piech & Cain, CS109, Stanford University
Anatomy of a The Normal PDF
distance to the mean
(makes the PDF symmetric
around the mean)
…normalized by
a constant: the variance
makes the integral
over all possible
outcomes sum to 1
Piech & Cain, CS109, Stanford University
Expected value of a normal distribution
Variance
v du
Properties
13
Properties
• Median = mean (why?)
• Because of symmetry of the pdf about the mean
• Mode = mean – can be checked by setting the first derivative of the pdf
to 0 and solving, and checking the sign of the second derivative.
14
Carl Friedrich Gauss (1777-1855)
• German mathematician
• Sort-of invented the normal
distribution
• Also astronomer, geologist, physicist
• Super influential in a lot of fields
Piech & Cain, CS109, Stanford University
Why the Normal?
• Common for natural phenomena: human height, weight, shoe sizes, etc.
Piech & Cain, CS109, Stanford University
Why the Normal?
• Common for natural phenomena: human height, weight, shoe sizes, etc.
• A lot of noise in the world is Normal
• E.g. random errors in measurements, residuals in linear regression
Piech & Cain, CS109, Stanford University
Why the Normal?
• Common for natural phenomena: human height, weight, shoe sizes, etc.
• A lot of noise in the world is Normal
• E.g. random errors in measurements, residuals in linear regression
• The sum of many random variables often looks Normal (spoilers)
• Sample means are distributed normally – important for statistics
Piech & Cain, CS109, Stanford University
Why the Normal?
• Common for natural phenomena: human height, weight, shoe sizes, etc.
• A lot of noise in the world is Normal
• E.g. random errors in measurements, residuals in linear regression
• The sum of many random variables often looks Normal (spoilers)
• Sample means are distributed normally – important for statistics
• Even things that aren’t Normal might fit a normal-related distribution
Piech & Cain, CS109, Stanford University
Why the Normal?
• Common for natural phenomena: human height, weight, shoe sizes, etc.
• A lot of noise in the world is Normal
• E.g. random errors in measurements, residuals in linear regression
• The sum of many random variables often looks Normal (spoilers)
• Sample means are distributed normally – important for statistics
• Even things that aren’t Normal might fit a normal-related distribution
People also just assume things are normally distributed a lot.
• They can do this in part because the Normal is so common
• But there’s a deeper reason to it…
Piech & Cain, CS109, Stanford University
“The simplest explanation is usually the best one”
When We Fit Models To Data, WeTry To Keep It Simple
Likelihood
Value
Piech & Cain, CS109, Stanford University
When We Fit Models To Data, We Try To Keep It Simple
Likelihood
Value
This curve fits the data well, but does it really represent the distribution?
Or is it “overfit”, so that the curve captures too much of the noise?
Piech & Cain, CS109, Stanford University
When We Fit Models To Data, We Try To Keep It Simple
Likelihood μ
σ2
Value
This curve fits the data about as well, but appears to overfit less.
We could say that this simpler distribution makes fewer assumptions.
The formal concept for this idea is entropy
Piech & Cain, CS109, Stanford University
When We Fit Models To Data, We Try To Keep It Simple
Likelihood μ
σ2
For a fixed mean and variance, the unique distribution that maximizes the
entropy is the normal distribution.
[Link]
[Link]