Scribd
Upload
4 views5 pages

IE4497 Readiness Assessment 1

The document discusses the Maximum Likelihood Estimation (MLE) of the mean (µ) for a normally distributed dataset with known variance, deriving the MLE as the sample mean. It further explores the Maximum A Posteriori (MAP) estimate of µ when a prior distribution is assumed, resulting in a formula that combines the sample mean and prior information. Additionally, it provides explanations for a code snippet related to linear regression and polynomial features.

Uploaded by

Wong Zhunhao
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views5 pages

IE4497 Readiness Assessment 1

The document discusses the Maximum Likelihood Estimation (MLE) of the mean (µ) for a normally distributed dataset with known variance, deriving the MLE as the sample mean. It further explores the Maximum A Posteriori (MAP) estimate of µ when a prior distribution is assumed, resulting in a formula that combines the sample mean and prior information. Additionally, it provides explanations for a code snippet related to linear regression and polynomial features.

Uploaded by

Wong Zhunhao
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Readiness Assessment 1

Readiness Assessment 1


1. Suppose x1 , . . . , xn ∼ N · µ, σ 2 i.i.d. with known variance σ 2 . Find the
MLE of µ.
The log likelihood is
n n
Y 1 X
N xi µ, σ 2 = − 2 (xi − µ)2 + const .

log p(D | µ) = log
i=1
2σ i=1

The first order derivative w.r.t. µ is


n n
d 1 X 1 X
log p(D | µ) = − 2 2(xi − µ)(−1) = 2 (xi − µ).
dµ 2σ i=1 σ i=1

Tay W.P. IE4497/EE6497 Pattern Recognition and Deep Learning 1/5


Readiness Assessment 1

Readiness Assessment 1 (cont.)

Finding the root of the first order derivative, we have


n
1 X
(xi − µML ) = 0
σ 2 i=1
n
X
nµML = xi
i=1
n
1X
µML = xi .
n i=1

Tay W.P. IE4497/EE6497 Pattern Recognition and Deep Learning 2/5


Readiness Assessment 1

Readiness Assessment 1 (cont.)

2. Continuing from the previous question, suppose that the prior distribution of
µ is N (· | µ0 , 1). Find the MAP estimate of µ.

log p(µ | D) = log p(D | µ) + log p(µ) + const


n
1 X 1
=− 2 (xi − µ)2 − (µ − µ0 )2 + const .
2σ i=1 2

The first order derivative w.r.t. µ is


n
d 1 X
log p(µ | D) = − 2 2(xi − µ)(−1) − (µ − µ0 )
dµ 2σ i=1
n
1 X
= (xi − µ) − µ + µ0 .
σ 2 i=1

Tay W.P. IE4497/EE6497 Pattern Recognition and Deep Learning 3/5


Readiness Assessment 1

Readiness Assessment 1 (cont.)


Finding the root of the first order derivative, we have
n
1 X
(xi − µMAP ) − µMAP + µ0 = 0
σ 2 i=1
1
Pn
σ2 i=1 xi + µ0
µMAP = n .
σ2 + 1

3. Explain each line of the following code snippet.

Line 1: Creates a Linear Regression object.


Line 2: Creates a polynomial features class of degree d with bias term.

Tay W.P. IE4497/EE6497 Pattern Recognition and Deep Learning 4/5


Readiness Assessment 1

Readiness Assessment 1 (cont.)

Line 3: Generates the training feature matrix.


Line 4: Fits the linear regression model.

Tay W.P. IE4497/EE6497 Pattern Recognition and Deep Learning 5/5

You might also like