Statistical Estimation, MLE, Summaries and
Correlation Analysis
Ajmal Mohammed
Department of ECE
September 23, 2025
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 1 / 15
Outline
1 Statistical Estimation
2 Maximum Likelihood Estimation (MLE)
3 Statistical Summaries
4 Correlation Analysis
5 Summary
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 2 / 15
Estimator
An estimator is a rule or formula used to estimate an unknown
population parameter.
Example:
n
1X
µ̂ = Xi estimates µ.
n
i=1
Properties: unbiasedness, consistency, efficiency.
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 3 / 15
MLE Concept
Given sample x1 , . . . , xn from distribution f (x; θ)
n
Y n
X
L(θ) = f (xi ; θ), ln L(θ) = ln f (xi ; θ)
i=1 i=1
MLE θ̂ maximizes ln L(θ).
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 4 / 15
Problem 1: Bernoulli Distribution
Problem: A coin is tossed n times with probability of head p. Derive the
MLE of p.
Solution:
P P X X
xi
L(p) = p (1 − p)n− xi
, ln L(p) = xi ln p + (n − xi ) ln(1 − p).
Differentiating:
P P
d xi n − xi
−
ln L(p) = = 0.
dp p 1−p
P
xi
⇒ p̂MLE = = X̄ .
n
Thus, the MLE of p is the sample mean.
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 5 / 15
Problem 2: Poisson Distribution
Problem: Let X1 , . . . , Xn ∼ Poisson(λ). Find the MLE of λ.
Solution:
n
Y e −λ λxi P Y 1
L(λ) = = e −nλ λ xi
xi ! xi !
i=1
X
ln L(λ) = −nλ + xi ln λ + constant.
P P
d xi xi
ln L = −n + = 0 ⇒ λ̂ = = X̄ .
dλ λ n
So, MLE of λ is the sample mean.
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 6 / 15
Problem 3: Exponential Distribution
Problem: Let X1 , . . . , Xn ∼ Exponential(θ) with pdf
1
f (x; θ) = e −x/θ , x > 0.
θ
Find the MLE of θ.
Solution: n P
1
L(θ) = e − xi /θ
θ
1X
ln L(θ) = −n ln θ − xi .
θ
Differentiating: P
d n xi
ln L = − + 2 = 0.
dθ θ θ
1X
⇒ θ̂ = xi = X̄ .
n
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 7 / 15
Problem 4: Normal Distribution (Mean unknown, Variance
known)
Problem: X1 , . . . , Xn ∼ N(µ, σ 2 ) with known σ 2 . Find the MLE of µ.
Solution:
(xi − µ)2
1
f (xi ; µ) = √ exp −
2πσ 2 2σ 2
n 1 X
ln L(µ) = − ln(2πσ 2 ) − 2 (xi − µ)2 .
2 2σ
Differentiate wrt µ:
d 1 X
ln L = 2 (xi − µ) = 0 ⇒ µ̂ = X̄ .
dµ σ
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 8 / 15
Problem 5: Normal Distribution (Variance unknown)
Problem: X1 , . . . , Xn ∼ N(µ, σ 2 ) with mean µ known. Find the MLE of
σ2.
Solution:
n 1 X
ln L(σ 2 ) = − ln(2πσ 2 ) − 2 (xi − µ)2 .
2 2σ
Differentiate:
d n 1 X
ln L = − + (xi − µ)2 = 0.
dσ 2 2σ 2 2σ 4
1X
⇒ σ̂ 2 = (xi − µ)2 .
n
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 9 / 15
Statistical Summaries
1P
Mean: x̄ = xi
n
1 P
Variance: s 2 = (xi − x̄)2
n−1
√
Standard deviation: s = s 2
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 10 / 15
Problem: Mean and Variance
Problem: Find mean and variance for marks: 10, 20, 30, 40, 50.
10 + 20 + 30 + 40 + 50
x̄ = = 30
5
400 + 100 + 0 + 100 + 400
s2 = = 250, s ≈ 15.81.
4
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 11 / 15
Correlation
Measures linear relationship between x and y .
Pearson’s correlation coefficient:
P
(x − x̄)(y − ȳ )
r = pP
(x − x̄)2 (y − ȳ )2
P
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 12 / 15
Correlation: Detailed calculations (step-by-step)
Given data:
x = {2, 4, 6, 8, 10, 12}, y = {40, 55, 65, 70, 85, 95}.
Sample means:
x̄ = 7.00, ȳ = 68.333333 (approx 68.33).
x y x − x̄ y − ȳ (x − x̄)(y − ȳ ) (x − x̄)2 (y − ȳ )2
2 40 −5.00 −28.33 141.67 25.00 802.78
4 55 −3.00 −13.33 40.00 9.00 177.78
6 65 −1.00 −3.33 3.33 1.00 11.11
8 70 1.00 1.67 1.67 1.00 2.78
10 85 3.00 16.67 50.00 9.00 277.78
12 95 5.00 26.67 133.33 25.00 711.11
Sums: 370.00 70.00 1983.33
Now compute Pearson’s correlation coefficient r :
P
(x − x̄)(y − ȳ )
r = pP P .
(x − x̄)2 (y − ȳ )2
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 13 / 15
Correlation
Substitute the sums:
370.00 370.00
r = √ = √ .
70.00 × 1983.3333 138833.33
Evaluate:
√ 370.00
138833.33 ≈ 372.86 ⇒ r≈ ≈ 0.9930.
372.86
Interpretation: r ≈ 0.993 indicates a very strong positive linear
correlation between study hours and marks.
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 14 / 15
Summary
MLE is a powerful method of parameter estimation.
Many distributions (Bernoulli, Poisson, Exponential, Normal) have
simple MLEs linked to sample mean/variance.
Statistical summaries describe central tendency and spread.
Correlation quantifies strength of linear relationship.
Ajmal Mohammed (Department of ECE) Statistical Estimation September 23, 2025 15 / 15