STATISTICS: Tutorial 2 - Solutions

MSc Economics & MSc Finance and Banking

Department of Economics and Finance
University of Rome Tor Vergata

TA: Francesca Centofanti

[Link]@[Link]

September 27th, 2022

First of all, let’s have a look at a brief recap of sufficiency and minimal sufficiency.
In order to find sufficient statistics for a parameter θ, we can make use of the Fisher-
Neyman Factorization theorem:

A statistic T (·) is sufficient for θ if functions g and h can be found such that:
f (x|θ) = h(x)g[θ, T (x)]

We are interested in investigating whether we can factorize the probability distribution in

two functions: one, h(x), which depends exclusively on x, and g[θ, T (x)], which depends
on θ and on x only through the statistic T (x).
Theorem: for i.i.d. observations from an exponential family
k
X 
f (x|θ) = h(x)c(θ) exp ωi (θ)ti (x) (1)
i=1

the statistic
n
X 
T (X) = T1 (xj ), . . . , Tk (xj ) (2)
j=1

is minimal sufficient for θ.

Recall that a statistic T (x) is minimal sufficient for a parametric model F if:

• it is itself sufficient

• it can be expressed as a function of any other sufficient statistic; i.e. for any other
sufficient statistic T ′ (x), there is some function g such that T = g(T ′ ).

The Lehmann-Scheffe Theorem offers a characterization for minimal sufficient statistics.

A statistic T (·) is minimal sufficient for the parameter θ if the following property holds:
f (x|θ)
given two data points x and y, the ratio f (y|θ) does not depend on θ if and only if
T (x) = T (y)

1
Problem 1
Let x1 , . . . , xn be an i.i.d. sample from the Poisson distribution

exp{−λ}λx 1
f (x|λ) = = exp{−λ} exp{x ln λ}
x! x!
Find a sufficient statistic for λ by
a) using the factorization theorem
b) using the property of exponential family. Are they the same? If not, why are both of
them sufficient?

SOLUTION

a) Assuming independence, we can write the joint pdf as the product of the n individual
density function:
n
Y exp{−λ}λxi
f (x1 , . . . , xn |λ) = (3)
i=1
xi !

We can manipulate this expression and show that we can factorize it in a way that sat-
isfies the Fisher-Neyman theorem.

1 Pn
f (x1 , . . . , xn |λ) = Qn exp{−nλ}λ( i=1 xi )

i=1 xi !
 n
X  
1
= Qn exp −nλ + xi ln λ (4)
i=1 xi ! i=1
1
= Qn exp{−nλ + nx̄ ln λ}
i=1 xi !
Notice that now we can identify two functions:
1
h(x) = Qn (5)
i=1 xi !

g[θ, T (x)] = exp{−nλ + nx̄ ln λ} (6)

The function h depends on x only while g[θ, T (x)] depends on θ and on x only through
the statistic T1 (x1 , . . . , xn ) = x̄.
Then, the statistic x̄ is sufficient for λ since:
1
f (x1 , . . . , xn |λ) = Qn exp{−nλ + nx̄ ln λ} = h(x)g[θ, T (x)] (7)
i=1 xi !

b) We know that the Poisson distribution belongs to the exponential family, so we can
rearrange the expression of the joint density as:
1 1
f (x1 , . . . , xn |λ) = Qn exp{−nλ + nx̄ ln λ} = Qn exp{−nλ} exp{x̄n ln λ} (8)
i=1 xi ! i=1 xi !

with
 −1
Qn
• h(x) = i=1 xi !

2
 • c(λ) = exp{−nλ}

• ω1 (λ) = n ln λ

• t1 (x) = x̄
By Factorization theorem we proved that T1 (x1 , . . . , xn ) = x̄ is a sufficient statistic. The
theorem for exponential family distributions states that the statistic T2 (x1 , . . . , xn ) =
Pn
i ti (x) = x̄ is minimal sufficient for θ. We observe that T1 (x1 , . . . , xn ) = T2 (x1 , . . . , xn ) = x̄
are equal and, hence, both sufficient.

Problem 2
Let x1 , . . . , xn be an i.i.d. sample from the Pareto distribution with parameters θ and x0

f (x| θ, x0 ) = θxθ0 x−θ−1 , x ≥ x0 θ > 1

Assume that x0 > 0 and is given. Find a sufficient statistic for θ by

a) using the factorization theorem
b) using the property of exponential family. Are they the same? If not, why are both of
them sufficient?

SOLUTION

a) We start writing down the joint pdf for the identically and independently distributed
sample as:
n
Y n
Y
f (x1 , . . . , xn | θ, x0 ) = θxθ0 x−θ−1
i = θn xnθ
0 x−θ−1
i (9)
i=1 i=1
We can clearly identify two functions that satisfy the factorization theorem:
n
Y
f (x1 , . . . , xn | θ, x0 ) = θn xnθ
0 x−θ−1
i = h(x)g[θ, T (x)] (10)
i=1

where we have
Qn
• h(x) = i=1 x−1
i
Qn
• g[θ, T (x)] = θn xnθ
0 i=1 x−θ
i
Qn
Hence, the statistic T1 (x1 , . . . , xn ) = i=1 xi is sufficient for θ.

b) The joint density for the Pareto i.i.d. sample belongs to the exponential family as
well, so we can rearrange the expression of the density function as:
n
Y  Yn 
−1 n nθ −θ
f (x1 , . . . , xn | θ, x0 ) = xi θ x0 exp ln xi (11)
i=1 i=1

Using properties of logarithms, we can rewrite it as:

Yn 
ln x−θ
i = ln(x−θ −θ −θ
1 x 2 . . . xn ) (12)
i=1
n
X
= −θ ln xi (13)
i=1

3
 Finally, we have :
n
Y  X n 
f (x1 , . . . , xn | θ, x0 ) = x−1
i θ n nθ
x 0 exp −θ ln x i (14)
i=1 i=1
Qn
• h(x) = i=1 x−1
i

• c(θ) = θn xnθ
0

• ω1 (θ) = −θ
Pn
• ti (x) = i=1 ln xi
Pn
The statistic T2 (x1 , . . . , xn ) = i=1 ln xi is minimal sufficient statistic for θ.
We found two different statistics for the parameter θ to be inferred from our Pareto sam-
Qn Pn
ple: T1 (x1 , . . . , xn ) = i=1 xi and T2 (x1 , . . . , xn ) = i=1 ln xi . We have already shown that
T1 is sufficient for θ and we complete the answer noticing that T2 is simply an identity
transformation of T1 , so it is minimal sufficient according to the definition.

Problem 3
Find the minimal sufficient statistic for the Beta distribution with only β = 2 and α un-
known.

SOLUTION

To identify the minimal sufficient statistic for α we have to look at the ratio
f (x|θ)
(15)
f (y|θ)
and investigate whether it is constant with respect to θ, if and only if we observe T (x) =
T (y).
A Beta random variable has pdf:

Γ(α + β)
f (x | α, β) = xα−1 (1 − x)β−1 (16)
Γ(α)Γ(β)
Given two arbitrary realizations x = (x1 , . . . , xn ) and y = (y1 , . . . , yn ), we can write the
ratio: Qn
f (x|α) xα−1 (1 − xi )
= Qi=1
n
i
α−1 (17)
f (y|α) i=1 yi (1 − yi )
This is independent from α if and only if i=1 xi = i=1 yi . The minimal sufficient statis-
Q Q
Qn
tic for α is then i=1 xi .

Problem 4
Find the minimal sufficient statistic for the Gamma distribution with only β known and
α unknown.

4
SOLUTION

Again, we apply the Lehmann-Scheffe Theorem to identify the minimal sufficient statistic
for a Gamma random variable with pdf:
1
f (x | α, β) = xα−1 e−x/β (18)
Γ(α)β α
For two realizations x and y the ratio of the densities is:
1
Qn α−1 −xi /β
f (x|α) Γ(α)n β nα i=1 xi e
= 1 n α−1 (19)
e−yi /β
Q
f (y|α) Γ(α)n β nα i=1 yi

We simplify the expression, so we can rewrite:

 Qn α−1  Pn Pn 
f (x|α) xi i=1 yi − i=1 xi
= Qi=1
n exp (20)
f (y|α) i=1 yi β

This is independent from α if and only if yi . The minimal sufficient statis-

Q Q
i=1 xi = i=1
Qn
tic for α is then i=1 xi .

Problem 5
Let x1 , . . . , xn be an i.i.d. sample from the Uniform distribution on [a, b]

(b − a)−1 if a ≤ x ≤ b
f (x|a, b) =
0 otherwise

Find a sufficient and minimum sufficient statistic(s) for a and b.

SOLUTION

a) For a sufficient statistic, we use the Factorization theorem. First, we write the joint
density and investigate whether it can be factorized to satisfy f (x; θ) = h(x)g[θ, T (x)]:

(b − a)−n n
if a ≤ xi ≤ b Y
f (x1 , . . . , xn | a, b) = = (b − a)−n I[a,b] (xi ) (21)
0 otherwise i=1

Since a ≤ xi ≤ b, we can surely write a ≤ min(xi ) = x1 and xn = max(xi ) ≤ b. The joint

density is then
n
Y
f (x1 , . . . , xn | a, b) = (b − a)−n I[a≤x1 ,...,xn ≤b] (22)
i=1
Qn
We can set h(x) = 1 and g[θ, T (x)] = (b − a)−n i=1 I[a≤x1 ,...,xn ≤b] .

Note that g[θ, T (x)] depends on θ and on x only through (x1 ; xn ) = [min(xi ), max(xi )],
hence, according to the factorization theorem, the sufficient statistic for (a; b) is precisely

5
[min(xi ); max(xi )]

b) To identify minimal sufficient statistic(s), we can use the Lehmann-Scheffe Theorem.

We look at the ratio of the densities for two arbitrary realizations x and y:
Qn
f (x | a, b) (b − a)−n i=1 I[a≤x1 ,...,xn ≤b]
= Qn (23)
f (y | a, b) (b − a)−n i=1 I[a≤y1 ,...,yn ≤b]

Eq. (23) is independent from θ = (a, b) if and only if min(xi ) = min(yi ) and max(xi ) =
max(yi ). Therefore, minimal sufficient statistics for (a, b) are jointly [min(xi ); max(xi )].