Chapter 4: Bayesian inference for

Binomial proportion

Instructor: Truong Buu Chau

Email: truongbuuchau@[Link]

Faculty of Mathematics and Statistics

August 18, 2025

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 1 / 24

Contents

1 Using a Uniform prior

2 Using a Beta prior
3 Choosing your prior
4 Summarizing the posterior distribution
5 Estimating the proportion
6 Bayesian credible interval

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 2 / 24

1. Using a Uniform prior
▶ The likelihood function:
 
n
f (y |π) = y π y (1 − π)n−y

for 0 ≤ π ≤ 1
▶ The posterior distribution is proportional to prior
distribution times likelihood
g (π)f (y |π)
g (π|y ) = R 1
0 g (π)f (y |π)d π

∝ g (π)f (y |π)

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 3 / 24

1. Using a Uniform prior

▶ Using a Uniform prior:

g (π) = 1
for 0 ≤ π ≤ 1, the posterior density is:
 
n
g (π|y ) ∝ y π y (1 − π)n−y

for 0 ≤ π ≤ 1
▶ π|y ∼ beta(a, b ) where a = y + 1 and b = n − y + 1.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 4 / 24

2. Using a Beta prior
▶ Using a Beta prior: beta(a, b )
Γ(a + b) a−1
g (π; a, b ) = π (1 − π)b−1
Γ(a)Γ(b)
for 0 ≤ π ≤ 1, the posterior density is:
g (π|y ) ∝ π y +a−1 (1 − π)n−y +b−1
Γ(n + a + b )
= π y +a−1 (1 − π)n−y +b−1
Γ(y + a)Γ(n − y + b )
for 0 ≤ π ≤ 1
▶ π|y ∼ beta(a′ , b ′ ) with parameters a′ = a + y and
b′ = b + n − y .
Truong Buu Chau C03031 - Chapter 4 August 18, 2025 5 / 24
2. Using a Beta prior
The shapes of beta(a; b ) densities for values of
a = 0.5; 1; 2; 3 and b = 0.5; 1; 2; 3.

This shows the variety of shapes that members of the

beta(a; b ) family can take.
Truong Buu Chau C03031 - Chapter 4 August 18, 2025 6 / 24
2. Using a Beta prior

When a < b, the density has more weight in the lower

half.
The opposite is true when a > b.
When a = b, the beta(a; b ) density is symmetric.
When a = 21 much more weight is given to values near
0, and when b = 12 much more weight is given to values
near 1.
The Uniform prior is a special case of the beta(a; b )
prior where a = 1 and b = 1.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 7 / 24

2. Using a Beta prior

▶ Fisher’s information
∂ 2 log f (y |θ)
 
I (θ|y ) = −E
∂θ2
▶ Jeffreys’ prior: q
g (θ) ∝ I (θ|y )

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 8 / 24

2. Using a Beta prior
▶ Using a Jeffreys’ prior: beta( 12 , 12 )
1 1 Γ(1) 1 1
g (π; , ) = 1 1 π − 2 (1 − π)− 2
2 2 Γ( 2 )Γ( 2 )
for 0 ≤ π ≤ 1, the posterior density is:
1 1
g (π|y ) ∝ π y + 2 −1 (1 − π)n−y + 2 −1

Γ(n + 1) y − 21 n−y − 12
= π ( 1 − π)
Γ(y + 12 )Γ(n − y + 12 )
for 0 ≤ π ≤ 1
1
▶ π|y ∼ beta(a′ , b ′ ) with parameters a′ = 2
+ y and
b ′ = 12 + n − y .
Truong Buu Chau C03031 - Chapter 4 August 18, 2025 9 / 24
3. Choosing your prior

▶ Choosing a prior: Conjugate priors

Definition
A class P of prior distributions for θ is called conjugate for
a sampling model f (y |θ) if g (θ) ∈ P ⇒ g (θ|y ) ∈ P

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 10 / 24

3. Choosing your prior
When you have vague prior knowledge, for example π is
very small, then beta(0.5; 1), beta(0.5; 2), beta(0.5; 3),
beta(1; 2), or beta(1; 3) would all be satisfactory priors.
All of these conjugate priors offer easy computation of
the posterior, together with putting most of the prior
probability at small values of π . It doesn’t matter very
much which one you chose; the resulting posteriors
given the data would be very similar.
When you have real prior knowledge by matching
location and scale
Prior: π ∼ beta(a, b ) with the mean of the proportion
π0 = a+a b and the standard deviation for the proportion
q
(1−π0 )
σ0 = πa0+ b+1
neq = a + b + 1
Truong Buu Chau C03031 - Chapter 4 August 18, 2025 11 / 24
3. Choosing your prior

▶ Example 8.1 in page 156

Three students are constructing their prior belief about
π , the proportion of Hamilton residents who support
building a casino in Hamilton.
Anna thinks that her prior mean is 0.2, and her prior
standard deviation is 0.08. The beta(a, b ) prior that
satisfies her prior belief is found by 0a.+2×b+0.18 = 0.082
Therefore her equivalent sample size is a + b + 1 = 25.
For Anna’s prior, a = 4.8 and b = 19.2.
Bart is a newcomer to Hamilton, so he is not aware of
the local feeling for or against the proposed casino. He
decides to use a uniform prior. For him, a = b = 1. His
equivalent sample size is a + b + 1 = 3.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 12 / 24

3. Choosing your prior
▶ Example 8.1 in page 156
Chris cannot fit a beta(a, b ) prior to match his belief.
He believes his prior probability has a trapezoidal shape.
He gives heights of his prior in below table and he
linearly interpolates between them to get his continuous
prior.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 13 / 24

3. Choosing your prior

▶ Example 8.1 in page 156

When we interpolate between these points, we see that
Chris’s prior is given by
for 0 ≤ π ≤ 0.1

 20π
g (π) = 2 for 0.1 ≤ π ≤ 0.3
5 − 10π for 0.3 ≤ π ≤ 0.5


Note that Chris’s prior is not actually a density since it

does not have area equal to one. However, this is not a
problem since the relative weights given by the shape of
the distribution are all that is needed since the constant
will cancel out.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 14 / 24

3. Choosing your prior

▶ Example 8.1 in page 156

The three priors are shown in the Figure

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 15 / 24

3. Choosing your prior

▶ Example 8.1 in page 156

The three students take a random sample of n = 100
Hamilton residents and find their views on the casino.
Out of the random sample, y = 26 said they support
building a casino in Hamilton.
Anna’s posterior is beta(4.8 + 26, 19.2 + 74).
Bart’s posterior is beta(1 + 26, 1 + 74).
We need to evaluate Chris’ prior numerically.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 16 / 24

3. Choosing your prior
▶ Example 8.1 in page 156
The three posteriors are shown in Figure

We see that the three students end up with very similar

posteriors, despite starting with priors having quite different
shapes.
Truong Buu Chau C03031 - Chapter 4 August 18, 2025 17 / 24
4. Summarizing the posterior distribution
▶ Summarizing the posterior distribution when
π|y ∼ beta(a′ , b ′ )
Measures of location
Posterior mode:
a′ − 1
mode = ′
a + b′ − 2
Posterior median: It is the solution of
Z median
g (π|y )d π = 0.5
0
Posterior mean:
a′
m′ =
a′ + b ′
Truong Buu Chau C03031 - Chapter 4 August 18, 2025 18 / 24
4. Summarizing the posterior distribution

▶ Summarizing the posterior distribution when

π|y ∼ beta(a′ , b ′ )
Measures of spread
Posterior variance:
a′ × b ′
var (π|y ) = ′
(a + b′ )2 × (a′ + b′ + 1)
Posterior standard deviation:This is the square root of
posterior variance.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 19 / 24

4. Summarizing the posterior distribution

▶ Example 8.1 in page 156

Anna, Bart, and Chris computed some measures of
location and spread for their posterior distributions.

We see clearly that the posterior distributions have similar

summary statistics, despite the different priors used.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 20 / 24

5. Estimating the proportion

A point estimate π̂ is a statistic calculated from the data

used as an estimate of the parameter π .
▶ The posterior mean square of an estimate
Z 1
PMSE[π̂] = (π − π̂)2 g (π|y )d π
0

PMSE[π̂] = Var [π|y ] + (m′ − π̂)2

The posterior mean is the optimum estimator post-data.
That’s a good reason to use the posterior mean as the
estimate, and it explains why the posterior mean is the
most widely used Bayesian estimate. We will use the
posterior mean as our estimate for π .

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 21 / 24

6. Bayesian credible interval
▶ Bayesian Credible Interval for π when
π|y ∼ beta(a′ , b ′ )
We approximate the beta(a′ , b ′ ) posterior distribution by
the normal distribution having the same mean and variance:
(π|y ) is approximately N [m′ ; (s ′ )2 ]
with
′ a′
m = ′
a + b′
and
a′ × b ′
(s ′ )2 =
(a′ + b′ )2 × (a′ + b′ + 1)
Truong Buu Chau C03031 - Chapter 4 August 18, 2025 22 / 24
6. Bayesian credible interval

The (1 − α) × 100% credible region for π is

approximately
m′ ± zα/2 × s ′
where zα/2 is the value found from the standard normal
table.
For a 95% credible interval, z0.025 = 1.96. The
approximation works very well if we have both a′ ≥ 10
and b ′ ≥ 10.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 23 / 24

6. Bayesian credible interval

▶ Example 8.1 in page 156

Anna, Bart, and Chris calculated 95% credible intervals
for π having equal tail areas two ways: using the exact
(beta) density function and using the Normal
approximation.

Truong Buu Chau C03031 - Chapter 4 August 18, 2025 24 / 24