不知 體参 牧 M on
Ch8, p.1
Point Estimation 點估計
利 觀察到 的 樣本教 振 推
• What is point estimation? M 52
細胞膜 的 電流
肌 across muscle cell membrane, TBp. 257-258)
Example 6.1 (current
筆資料
每個資料點都來 同 個 Normal distribution
問 如何 Xi Xaa52 去推估 M 52
放射性粒 的發射
Ch8, p.2
Example 6.2 (emission of alpha particles, TBp. 255-256)
發射頻率 常牧
粒
a
降雨量
Example 6.3 (rainfall amount, TBp. 258-259)
Ch8, p.3
不管
孜 速率参 教 rate
形
實際降雨量直 圖
點 估計 的 三 步驟 Ch8, p.4
Summary (procedure of fitting a particular distribution to data, i.e. point estimation)
收集資料 ˇ
射次數
電流
降雨量
統計建 模
將觀察到的資料 祝 作 rv
samplemean
sample var
點估計 La
Ch8, p.5
Notes
未知系統 性 模式 规律 ex u
隨 機 授動 ex var
All information
contained in data
空牧
未來资料
理論上的 樣本 Ch8, p.6
京到的實際值
統計推論
哀笑
ex 放射性
Ch8, p.7
Definition 6.1 (parameter)
固定但未知 參教 常改 M 02 入 等
unnnn
Definition 6.2 (statistic, estimator, estimate, sampling distribution, TBp.260)
統計量
ex 樣本平均 x 樣本䟝異孜
估計
羼值iǎmnnner
ò 抽樣分布
Definition 6.3 (standard error, estimated standard error, TBp. 262)
標準誤差 ò
Definition 6.4 (bias, unbiased estimator, TBp. 262)
偏族 estimator的期望
無
偏 參
差
影
Reading: textbook, 8.1, 8.2, 8.3
的
Ch8, p.8
• Method of finding estimators I --- method of moments
Definition 6.5 (sample moment, TBp. 260)
Definition 6.6 (method of moments, TBp. 261)
Ch8, p.9
Example 6.4 (Poisson distribution, TBp. 261)
Ch8, p.10
To evaluate the stability/uncertainty of
an estimation procedure, it is required to
know what the sampling distribution is.
Ch8, p.11
Example 6.5 (cont. Ex.6.4, TBp. 262)
Ch8, p.12
realization
function point
estimation
simulation histogram
sampling distribution
Ch8, p.13
Example 6.6 (Normal distribution, TBp. 263)
Ch8, p.14
Example 6.7 (Gamma distribution, TBp. 263-264)
Ch8, p.15
Definition 6.7 (parametric bootstrap, TBp. 264-265)
Example 6.8 (cont. Ex.6.7, bootstrap, TBp. 265-266)
Ch8, p.16
Reading: textbook, 8.4
Ch8, p.17
• method of finding estimators II --- Maximum Likelihood Estimator (MLE)
Definition 6.8 (likelihood, log likelihood, TBp. 267, 268)
Ch8, p.18
Definition 6.9 (maximum likelihood estimator, TBp. 267)
Note.
Ch8, p.19
Theorem 6.1 (invariance property of MLE)
Example 6.10 (i.i.d Poisson distribution, TBp. 268)
Ch8, p.20
Example 6.11 (i.i.d normal distribution, TBp. 269)
Ch8, p.21
Ch8, p.22
Example 6.12 (i.i.d restricted normal distribution)
Example 6.13 (i.i.d uniform(0, θ) distribution)
Ch8, p.23
Example 6.14 (multinomial distribution, TBp. 272)
Ch8, p.24
Example 6.15 (Hardy-Weinberg Equilibrium, TBp. 273)
Mother
A [1−θ ] a [θ ]
Father A [1−θ ] AA [(1−θ)2] Aa [θ(1−θ)]
a [θ ] Aa [θ(1−θ)] aa [θ 2]
Ch8, p.25
Ch8, p.26
Ch8, p.27
Example 6.16 (Muon Decay, TBp. 266 & 271)
Ch8, p.28
Example 6.17 (i.i.d. Gamma distribution, TBp. 270)
Ch8, p.29
Ch8, p.30
Summary (advantages of MLE)
Reading: textbook, 8.5, 8.5.1
Ch8, p.31
• Large sample (asymptotic) theory for method of moment estimator and MLE
Definition 6.10 (consistent, TBp. 266)
method of moment estimator
Theorem 6.2 (consistency of method of moment estimator, TBp. 266)
Ch8, p.32
Theorem 6.3 (justification for estimating standard errors, TBp. 266-267)
MLE
Ch8, p.33
Definition 6.11 (score equation, score function)
Definition 6.12 (Fisher information for one-dimensional parameter, TBp.263)
Ch8, p.34
Theorem 6.4 (TBp. 276)
Ch8, p.35
Theorem 6.4 (TBp. 276)
Ch8, p.36
Note (TBp. 278, 279).
Example 6.18 (Fisher information of i.i.d. Bernoulli B(θ))
Ch8, p.37
Ch8, p.38
Theorem 6.5 (consistency of MLE, TBp. 275)
What is the difference between
Ch8, p.39
Theorem 6.6 (asymptotic normality of MLE for one-dimensional parameter, TBp. 277)
Ch8, p.40
Ch8, p.41
Notes (TBp. 277)
Theorem 6.7 (Fisher information under reparameterization)
Ch8, p.42
Theorem 6.8 (asymptotic normality of MLE under reparameterization)
Example 6.19 (information and asymptotic distribution of MLE for Poisson mean, TBp.282)
Ch8, p.43
Theorem 6.9 (multidimensional parameters, information and asymptotic normality of MLE, TBp.279)
Reading: textbook, 8.5.2; Further reading: Hogg et al., 6.1, 6.2
Ch8, p.44
• Data reduction --- the concepts of sufficiency, minimal sufficiency, and completeness
Question 6.1 (information and data reduction)
order statistics
raw (original) data
histogram
sample variance
sample mean
Ch8, p.45
Summary (formulation of information and data reduction problem, TBp. 305)
Ch8, p.46
n
T = i=1 Xi
(X1 , . . . , Xn )
Definition 6.13 (sufficient, TBp. 305)
Ch8, p.47
Example 6.20 (sufficient statistics of i.i.d. Bernoulli distribution, TBp. 306)
Theorem 6.10 (factorization theorem, TBp. 306)
Ch8, p.48
Theorem 6.11 (MLE and sufficient statistics, TBp.309)
Example 6.21 (cont. Ex. 6.20, sufficient statistic of i.i.d. Bernoulli distribution, TBp.309)
Ch8, p.49
Example 6.22 (sufficient statistics of i.i.d. Normal distribution, TBp.308)
Ch8, p.50
Question 6.2
Definition 6.14 (minimal sufficient statistic)
Example 6.23 (minimal sufficient statistics for i.i.d. Uniform distribution U(θ, θ+1))
Ch8, p.51
Example 6.24 (cont. Ex. 6.23, ancillary statistics)
Ch8, p.52
Question 6.3
Definition 6.15 (completeness, TBp.310)
u1(S): non-constant function u2(S): constant function
Ch8, p.53
Example 6.25 (sufficient and complete statistics of i.i.d. Uniform distribution U(0,θ))
Ch8, p.54
Example 6.26 (sufficient and complete statistic of i.i.d. Poisson distribution)
Theorem 6.12
Definition 6.16 (one-parameter exponential family of probability distributions, TBp.308)
Ch8, p.55
Theorem 6.13 (sufficient and complete statistics, one-parameter exponential family, TBp.309)
Example 6.27 (some one-parameter exponential families, TBp.309)
Ch8, p.56
Definition 6.17 (regular k-parameter exponential family, TBp.309)
Ch8, p.57
Theorem 6.14 (sufficient and complete statistics for regular k-parameter exponential family)
Example 6.28 (some regular k-parameter exponential families)
Ch8, p.58
Reading: textbook, 8.8, 8.8.1; Further reading: Hogg et al., 7.2, 7.4, 7.5, 7.7, 7.8, 7.9
• criteria for evaluating estimators
Question 6.4 (choice among different estimators of the same parameter, TBp.298)
Ch8, p.59
Question 6.5 (TBp.298)
Definition 6.18 (mean square error, TBp.298)
Definition 6.19 (relative efficiency, TBp.298)
Ch8, p.60
Notes (interpretation of relative efficiency, TBp. 298)
Ch8, p.61
Example 6.29 (Muon Decay, TBp.299)
Ch8, p.62
Question 6.6 (TBp.300)
Theorem 6.15 (Cramer-Rao inequality, TBp.300)
Ch8, p.63
Notes
Ch8, p.64
Definition 6.20 (efficiency, efficient, asymptotically efficient, TBp.302)
Notes (TBp.302)
Ch8, p.65
Example 6.30 (Poisson distribution, TBp.302)
Theorem 6.16 (generalization of Cramer-Rao inequality)
Example 6.31 (generalized negative binomial, TBp.302-305)
Ch8, p.66
Ch8, p.67
Ch8, p.68
Reading: textbook, 8.7; Further reading: Hogg et al., 6.2
Ch8, p.69
• method of finding estimator III --- UMVUE (or MVUE)
Question 6.7 (“best” unbiased estimator)
Definition 6.21 (UMVUE)
Example 6.32 (cont. Ex. 6.30, UMVUE of Poisson mean, TBp.302)
Ch8, p.70
Question 6.8
Theorem 6.17 (Rao-Blackwell theorem, TBp.310)
S=s
all unbiased estimators × ×
unbiased estimators and × × ×
×
function of S
Ch8, p.71
Question 6.9 (uniqueness)
Theorem 6.18
Ch8, p.72
Theorem 6.19 (Lehmann-Scheffe theorem)
Summary (methods of finding UMVUE)
Ch8, p.73
Example 6.33 (cont. Ex. 6.32, UMVUE of Poisson mean)
Example 6.34 (UMVUE of Normal mean and variance)
Example 6.35 (UMVUE may not attain Cramer-Rao lower bound)
Ch8, p.74
Example 6.36 (UMVUE of Poisson zero probability)
Example 6.37 (cont. Ex. 6.25, UMVUE of Uniform upper bound, c.f. Ex. 6.13)
Theorem 6.20
Reading: textbook, 8.8.2; Further reading: Hogg et al., 7.1, 7.3, 7.6