Received: 00 Month 0000 Revised: 00 Month 0000 Accepted: 00 Month 0000

DOI: xxx/xxxx

CHAPTER

Random Variables

Chang, Chih-Hao

Department of Statistics, National Chengchi

University, Taipei, Taiwan Summary
This chapter provides a comprehensive introduction to the fundamental concepts of
discrete probability distributions and their associated measures of expectation and
variance. We begin with the definition of expectation as a weighted average and
extend it to functions of random variables, emphasizing both intuitive interpretations
and formal derivations. Variance and standard deviation are introduced as mea-
sures of spread, with illustrative examples demonstrating their roles in quantifying
variability. The chapter explores key discrete distributions—including the bino-
mial, Poisson, geometric, negative binomial, and hypergeometric—and highlights
important applications such as quality control, ecological estimation, and classical
probability puzzles. Properties of the cumulative distribution function (CDF) are dis-
cussed in detail, emphasizing monotonicity, limits, and right-continuity. Finally, we
establish the linearity of expectation and illustrate its use in analyzing sums of ran-
dom variables, both independent and dependent. Throughout, examples and problem
contexts are chosen to build intuition, connect theory to practice, and prepare stu-
dents for more advanced probabilistic reasoning.

KEYWORDS:
Expectation, Variance, Poisson Approximation, Hypergeometric Distribution, Linearity of Expectation

1 RANDOM VARIABLES

1. In many experiments, our interest often lies not in the exact outcome itself, but rather in some numerical characteristic
that can be expressed as a function of the outcome.

(a) For example, when tossing two dice, we are usually interested in the sum of the two dice rather than the specific
ordered pair of values.
(b) That is, we might care about the event that the sum equals 7, without distinguishing whether the outcome was (1, 6),
(2, 5), (3, 4), (4, 3), (5, 2), or (6, 1).
(c) Similarly, when flipping a coin several times, we may only be concerned with the total number of heads, instead of
the detailed head–tail sequence.

2. Such quantities of interest, which map each outcome of the sample space into a real number, are formally called random
variables.

3. Since the value of a random variable depends on the random outcome of the experiment, it is natural to assign probabilities
to the possible values taken by the random variable. This assignment induces a probability distribution on the real line.

0
2

4. Example:

(a) Consider the experiment of tossing 3 fair coins.

(b) Define the random variable 𝜔 to be the number of heads obtained. Then 𝜔 can take values 0, 1, 2, or 3. The
probabilities can be computed as follows:
𝜀 ({𝜔 = 0}) = 𝜀 ({(𝜗 , 𝜗 , 𝜗 )}) = 18 ,
𝜀 ({𝜔 = 1}) = 𝜀 ({(𝜗 , 𝜗 , 𝜛), (𝜗 , 𝜛, 𝜗 ), (𝜛, 𝜗 , 𝜗 )}) = 38 ,
(1)
𝜀 ({𝜔 = 2}) = 𝜀 ({(𝜗 , 𝜛, 𝜛), (𝜛, 𝜛, 𝜗 ), (𝜛, 𝜗 , 𝜛)}) = 38 ,

𝜀 ({𝜔 = 3}) = 𝜀 ({(𝜛, 𝜛, 𝜛)}) = 18 .

(c) Since 𝜔 must take one of the values 0, 1, 2, 3, the total probability satisfies
⌋ 3 ⌉ 3
⌈ {
1=𝜀 {𝜔 = 𝜚} = 𝜀 ({𝜔 = 𝜚}), (4.5)
𝜚=0 𝜚=0
which agrees with the preceding calculations.

5. Strictly speaking, a probability measure 𝜀 (⋛) is defined on events, so writing 𝜀 ({𝜔 = 0}) emphasizes that the event
{𝜔 = 0} is a subset of the sample space.

6. Hereafter, for convenience of notation, we will simply write

𝜀 (𝜔 = 0) instead of 𝜀 ({𝜔 = 0}),
and similarly for other events. This simplification is common in probability theory, but it is important to remember that
𝜀 (⋛) is always defined on events.

7. Example:

(a) Three balls are randomly selected without replacement from an urn containing 20 balls numbered 1 through 20.
(b) Suppose we bet that at least one of the selected balls has a number greater than or equal to 17. What is the probability
of winning this bet?
(c) Let 𝜍 denote the largest number selected. Then 𝜍 is a random variable taking values in {3, 4, … , 20}.
} ⦃
20
(d) Since all possible selections are equally likely, the probability mass function of 𝜍 is
3
} ⦃
𝜚ω1
2
𝜀 (𝜍 = 𝜚) = } ⦃ , 𝜚 = 3, 4, … , 20. (2)
20
3
This follows because for 𝜍 = 𝜚, the ball numbered 𝜚 must be chosen, together with two balls from {1, 2, … , 𝜚 ω 1}.
(e) From (2), the probabilities of interest are
} ⦃ } ⦃
19 18
2 3 2 51
𝜀 (𝜍 = 20) = } ⦃= , 𝜀 (𝜍 = 19) = } ⦃= ,
20 20 20 380
3 3
} ⦃ } ⦃ (3)
17 16
2 34 2 2
𝜀 (𝜍 = 18) = } ⦃ = , 𝜀 (𝜍 = 17) = } ⦃ = .
20 285 20 19
3 3
 3

(f) Since the event 𝜍 ∱ 17 is the union of the disjoint events {𝜍 = 𝜚} for 𝜚 = 17, 18, 19, 20, the probability of winning
the bet is
𝜀 (𝜍 ∱ 17) = 𝜀 (𝜍 = 17) + 𝜀 (𝜍 = 18) + 𝜀 (𝜍 = 19) + 𝜀 (𝜍 = 20)
2 34 51 3
= + +
19 285 380 20
+ (4)
29
= .
57
8. Example:

(a) Independent trials are performed by repeatedly flipping a coin that shows heads with probability 𝜑, until either the
first head occurs or a total of 𝛻 flips is made.
(b) Let 𝜍 denote the number of times the coin is flipped. Then 𝜍 is a random variable taking values in {1, 2, … , 𝛻}.
(c) The probability mass function of 𝜍 is given by
𝜀 (𝜍 = 1) = 𝜀 ({𝜛}) = 𝜑,
𝜀 (𝜍 = 2) = 𝜀 ({𝜗 , 𝜛}) = (1 ω 𝜑)𝜑,
𝜀 (𝜍 = 3) = 𝜀 ({𝜗 , 𝜗 , 𝜛}) = (1 ω 𝜑)2 𝜑,
(5)
⋜
𝜀 (𝜍 = 𝛻 ω 1) = 𝜀 ({𝜗 , 𝜗 , … , 𝜗 , 𝜛}) = (1 ω 𝜑) 𝛻ω2 𝜑,
𝜀 (𝜍 = 𝛻) = 𝜀 ({𝜗 , 𝜗 , … , 𝜗 , 𝜛} ε {𝜗 , 𝜗 , … , 𝜗 , 𝜗 }) = (1 ω 𝜑) 𝛻ω1 𝜑 + (1 ω 𝜑) 𝛻 .
Here the case 𝜍 = 𝛻 occurs either when the first head appears exactly on the 𝛻-th trial, or when all 𝛻 flips result in
tails.
(d) As a check, note that the total probability is
⌋ 𝛻 ⌉
⌈ {
𝛻
𝜀 {𝜍 = 𝜚} = 𝜀 (𝜍 = 𝜚)
𝜚=1 𝜚=1

{
𝛻ω1 ⦄ ⟨
= (1 ω 𝜑) 𝜚ω1 𝜑 + (1 ω 𝜑) 𝛻ω1 𝜑 + (1 ω 𝜑)𝛻
𝜚=1
(6)
} ⦃
1 ω (1 ω 𝜑) 𝛻ω1
=𝜑 + (1 ω 𝜑) 𝛻ω1 𝜑 + (1 ω 𝜑)𝛻
1 ω (1 ω 𝜑)

= 1.
9. Example: Coupon Collector Problem - I

(a) Suppose there are 𝜕 distinct types of coupons, and that each time one obtains a coupon, it is independent of the past
and equally likely to be any one of the 𝜕 types.
(b) Let 𝜗 denote the number of coupons that must be collected until a complete set, containing at least one of each type,
is obtained.
(c) Instead of directly deriving 𝜀 (𝜗 = 𝛻), we first consider the complementary probability 𝜀 (𝜗 > 𝛻), the probability
that after 𝛻 coupons we still do not have a complete set.
(d) To proceed, fix 𝛻 and define the events ℵ1 , ℵ2 , … , ℵ𝜕 as follows:
ℵℶ = {no coupon of type ℶ appears among the first 𝛻 coupons}, ℶ = 1, 2, … , 𝜕. (7)
4

(e) Then, by the inclusion–exclusion principle,

⌋𝜕 ⌉
⌈ {
𝜕
{
𝜀 (𝜗 > 𝛻) = 𝜀 ℵℶ = 𝜀 (ℵℶ ) ω 𝜀 (ℵℶ1 ϑ ℵℶ2 ) + ⋝
ℶ=1 ℶ=1 ℶ <ℶ
(8)
1 2
{
+ (ω1) ℷ+1
𝜀 (ℵℶ1 ϑ ⋝ ϑ ℵℶℷ ) + ⋝
ℶ1 <⋝<ℶℷ

+ (ω1) 𝜕+1
𝜀 (ℵ1 ϑ ℵ2 ϑ ⋝ ϑ ℵ𝜕 ).
(f) For each ℵℶ , the event occurs if none of the 𝛻 coupons is of type ℶ. Since each coupon avoids type ℶ with probability
𝜕 ω1
, independence yields
𝜕
⟩ ⟪
𝜕 ω1 𝛻
𝜀 (ℵℶ ) = . (9)
𝜕
(g) Similarly, for distinct types ℶ1 , ℶ2 , the probability that none of the first 𝛻 coupons is of either type ℶ1 or ℶ2 is
⟩ ⟪
𝜕 ω2 𝛻
𝜀 (ℵℶ1 ϑ ℵℶ2 ) = . (10)
𝜕
(h) In general, for any ℷ distinct types ℶ1 , … , ℶℷ ,
⟩ ⟪
𝜕 ωℷ 𝛻
𝜀 (ℵℶ1 ϑ ⋝ ϑ ℵℶℷ ) = . (11)
𝜕
(i) Substituting back into the inclusion–exclusion expansion, we obtain, for 𝛻 > 0,
⟩ ⟪ } ⦃⟩ ⟪ } ⦃⟩ ⟪
𝜕 ω1 𝛻 𝜕 𝜕 ω2 𝛻 𝜕 𝜕 ω3 𝛻
𝜀 (𝜗 > 𝛻) = 𝜕 ω + ω⋝
𝜕 2 𝜕 3 𝜕
} ⦃ ⟩ ⟪𝛻
𝜕 1
+ (ω1)𝜕
𝜕 ω1 𝜕 (12)
𝜕ω1 } ⦃⟩ ⟪𝛻
{ 𝜕 𝜕 ω𝜚
= (ω1) 𝜚+1 .
𝜚=1
𝜚 𝜕
(j) Finally, the distribution of 𝜗 follows from the relation
𝜀 (𝜗 = 𝛻) = 𝜀 (𝜗 > 𝛻 ω 1) ω 𝜀 (𝜗 > 𝛻)
𝜕ω1 } ⦃ ⟫⟩ ⟪𝛻ω1 ⟩ ⟪𝛻 ❲
{ 𝜕 𝜕 ω𝜚 𝜕 ω𝜚
= ω (ω1) 𝜚+1
𝜚=1
𝜚 𝜕 𝜕 (13)
𝜕ω1 } ⦃⟩ ⟪⟩ ⟪𝛻ω1
{ 𝜕 𝜚 𝜕 ω𝜚
= (ω1) 𝜚+1 .
𝜚=1
𝜚 𝜕 𝜕

10. Example: Coupon Collector Problem – II

(a) Suppose there are 𝜕 distinct types of coupons, and that each time one obtains a coupon, it is independent of the past
and equally likely to be any one of the 𝜕 types.
(b) Let ℸ𝛻 denote the number of distinct types of coupons that appear among the first 𝛻 selections.
(c) To compute 𝜀 ({ℸ𝛻 = ℷ}), fix attention on a particular set of ℷ distinct types, and determine the probability that this
set constitutes the distinct types observed in the first 𝛻 selections.
(d) For this to occur, it is necessary and su!cient that:
(i) all of the first 𝛻 coupons belong to these ℷ types (event ℵ), and
(ii) each of the ℷ types is represented at least once (event ⊳).
(e) Since each coupon has probability of belonging to the chosen ℷ types, the probability of ℵ is
ℷ
𝜕
⟩ ⟪𝛻
(14)
ℷ
𝜀 (ℵ) = .
𝜕
 5

(f) Conditional on ℵ, each coupon is equally likely to be any of the ℷ types. Thus, 𝜀 (⊳ ϖ ℵ) equals the probability that
𝛻 independent draws from ℷ equally likely types contain a complete set of all ℷ types.
(g) Equivalently, this is the probability that the coupon collector time for ℷ types is at most 𝛻, which (by analogy with
the result in (13) but replacing 𝜕 with ℷ) is
ℷω1 } ⦃ ⟩ ⟪
{ ℷω𝜚 𝛻
(15)
ℷ
𝜀 (⊳ ϖ ℵ) = 1 ω (ω1)𝜚+1 .
𝜚=1
𝜚 ℷ
} ⦃
(h) Since there are possible choices for the set of ℷ types, we obtain
𝜕
ℷ
} ⦃
𝜕
𝜀 (ℸ𝛻 = ℷ) = 𝜀 (ℵ ϑ ⊳)
ℷ
} ⦃
𝜕
=
ℷ
𝜀 (⊳ ϖ ℵ)𝜀 (ℵ) (16)
} ⦃ ⟩ ⟪𝛻 ⌋ } ⦃ ⟩ ⟪
⌉
𝜕 ℷ {
ℷω1
ℷ ℷω𝜚 𝛻
= 1ω (ω1)𝜚+1 .
ℷ 𝜕 𝜚=1
𝜚 ℷ

11. Remark:

(a) In the coupon collector problem, at least 𝜕 coupons must be collected to obtain a complete set. Hence, for 𝛻 < 𝜕,
𝜀 (𝜗 > 𝛻) = 1. (17)

(b) Therefore, by (13), we arrive at the following combinatorial identity: for integers 1 ∲ 𝛻 < 𝜕,
𝜕ω1 } ⦃ ⟩ ⟪
{ 𝜕 𝜕 ω𝜚 𝛻
(ω1) 𝜚+1 = 1. (18)
𝜚=1
𝜚 𝜕
(c) Equivalently, this can be written as
𝜕ω1 } ⦃⟩
1 ∲ 𝛻 < 𝜕.
{ ⟪𝛻
𝜕 ω𝜚
(19)
𝜕
(ω1) 𝜚+1 = 0,
𝜚=0
𝜚 𝜕

(d) Index transform to a binomial–power identity: multiplying through by (ω1)𝜕 𝜕 𝛻 and substituting ℶ = 𝜕 ω 𝜚, we
obtain the well-known identity
𝜕 } ⦃
ℶ (ω1) ℶω1 = 0, 1 ∲ 𝛻 < 𝜕.
{
(20)
𝜕 𝛻
ℶ=1
ℶ

i. Multiply both sides of (19) by (ω1)𝜕 𝜕 𝛻 . Term by term, this gives

𝜕ω1 } ⦃ ⟩ ⟪ 𝜕ω1 } ⦃
{ 𝜕 𝜕 ω𝜚 𝛻 { 𝜕
(ω1) 𝜕
𝜕 𝛻
(ω1) 𝜚+1
= (𝜕 ω 𝜚)𝛻 (ω1) 𝜚+1+𝜕 = 0. (21)
𝜚=0
𝜚 𝜕 𝜚=0
𝜚
Here the factor 𝜕 𝛻 cancels the denominator, and the signs combine as (ω1)𝜚+1+𝜕 .
ii. Perform the index change ℶ = 𝜕 ω 𝜚. As 𝜚 runs from 0 to 𝜕 ω 1, ℶ runs from 𝜕 down to 1. Reversing the order
of summation, we obtain
𝜕 } ⦃
{
(22)
𝜕
ℶ 𝛻 (ω1) (𝜕ωℶ)+1+𝜕 = 0.
ℶ=1
𝜕 ω ℶ
iii. Simplify each component: } ⦃ } ⦃
A. By binomial symmetry: .
𝜕 𝜕
=
𝜕 ωℶ ℶ
B. The power term: (𝜕 ω 𝜚) = ℶ .
𝛻 𝛻
6

C. The sign factor:

(ω1)(𝜕ωℶ)+1+𝜕 = (ω1) 2𝜕ωℶ+1 ∳ (ω1) ℶω1 (since 2𝜕 is even, exponents are equivalent mod 2).
(Alternatively: note that ℶ + 1 and ℶ ω 1 di"er by 2, and (ω1)2 = 1, so (ω1)ℶ+1 = (ω1)ℶω1 .)
iv. Putting everything together yields (20), the desired identity.

12. Distribution function of a random variable:

(a) For a random variable 𝜍, the function ⊲ defined by

⊲ (0) = 𝜀 (𝜍 ∲ 0), ωϱ < 0 < ϱ, (23)
is called the cumulative distribution function, or more simply, the distribution function, of 𝜍.
(b) Thus, the distribution function specifies, for all real values 0, the probability that the random variable is less than or
equal to 0.
(c) Now, suppose that 1 ∲ 2. The event {𝜍 ∲ 1} is contained in the event {𝜍 ∲ 2}, it follows that ⊲ (1), the probability
of the former, is less than or equal to ⊲ (2), the probability of the latter.
(d) In other words, ⊲ (0) is a nondecreasing function of 0; other general properties of the distribution function are given
in Section 10

2 DISCRETE RANDOM VARIABLES

1. A random variable that can take on at most a countable number of possible values is said to be discrete.

2. For a discrete random variable 𝜍, the probability mass function (pmf) 𝜑(1) of 𝜍 is defined by
𝜑(1) = 𝜀 (𝜍 = 1), ωϱ < 1 < ϱ. (24)

3. The pmf 𝜑(1) is strictly positive for at most a countable number of values of 1.

4. More precisely, if 𝜍 must assume one of the values 01 , 02 , …, then

𝜑(0𝜚 ) ∱ 0, 𝜚 = 1, 2, … ,
(25)
𝜑(0) = 0, for all other values of 0.

5. Since 𝜍 must take one of the values 0𝜚 , the probabilities sum to one:
ϱ
{
𝜑(0𝜚 ) = 1. (26)
𝜚=1

6. It is often useful to display the pmf in a graphical format, by plotting 𝜑(0𝜚 ) on the 3-axis against 0𝜚 on the 0-axis.
1 1 1
(a) For instance, if the pmf of 𝜍 is given by 𝜑(0) = , 𝜑(1) = , and 𝜑(2) = , we can represent it graphically as shown
4 2 4
in Figure 1 (a).
(b) Similarly, the graph of the pmf of the random variable representing the sum when two dice (denoted by 𝜔 ) are rolled
is illustrated in Figure 1 (b).

7. Example:

(a) Suppose the pmf of a random variable 𝜍 is given by

5𝜚
𝜑(𝜚) = 4 , 𝜚 = 0, 1, 2, … , (27)
𝜚!
where 5 > 0 is a parameter and 4 is a normalizing constant.
 7

(a) (b)

FIGURE 1 (a) PMF of 𝜍 and (b) PMF of 𝜔 .

ϱ
{
(b) Since 𝜑(𝜚) = 1 and
𝜚=0
ϱ
{ 0𝜚
60 = , (28)
𝜚=0
𝜚!
we obtain
ϱ
{ 5𝜚
4 = 4 ⋛ 65 = 1. (29)
𝜚=0
𝜚!

(c) Hence 4 = 6ω5 , and the pmf becomes

5𝜚
𝜑(𝜚) = 𝜀 (𝜍 = 𝜚) = 6ω5 , 𝜚 = 0, 1, 2, … (30)
𝜚!
which is known as the Poisson distribution with parameter 5.
(d) For example,
50
𝜀 (𝜍 = 0) = 6ω5 = 6ω5 ,
0!
51 (31)
𝜀 (𝜍 = 1) = 6ω5 = 56ω5 ,
1!
𝜀 (𝜍 > 1) = 1 ω 𝜀 (𝜍 = 0) ω 𝜀 (𝜍 = 1) = 1 ω 6ω5 ω 56ω5 .
8. The distribution function of a discrete random variable 𝜍 is a step function.

(a) For a discrete random variable, this can be written in terms of the probability mass function 𝜑(0) as
⊲ (1) = 𝜀 (𝜍 ∲ 1) =
{
𝜑(0). (32)
0 ς 0∲1

(b) In words, ⊲ (1) is the total probability of all outcomes whose value is less than or equal to 1.
(c) If 𝜍 is a discrete random variable taking values 01 , 02 , 03 , … with 01 < 02 < 03 < ⋝, then ⊲ is a step function:
• between 0𝜚ω1 and 0𝜚 , the value of ⊲ remains constant,
• at the point 0𝜚 , the function “jumps” upward by an amount equal to 𝜑(0𝜚 ).
(d) Each probability mass 𝜑(0𝜚 ) corresponds to a jump size in the cdf.
(e) For instance, suppose the pmf of 𝜍 is
𝜑(1) = 0.25, 𝜑(2) = 0.5, 𝜑(3) = 0.125, 𝜑(4) = 0.125. (33)
8

(f) Then the cdf of 𝜍 is

❳0, 1 < 1,
/
/0.25,
/ 1 ∲ 1 < 2,

2 ∲ 1 < 3,
/
⊲ (1) = \0.75, (34)

3 ∲ 1 < 4,
/
/0.875,
/
1 ∱ 4.
/
(1,
(g) Observe that the jumps in the cdf occur exactly at the possible values of 𝜍, and the size of each jump matches the
corresponding probability.
(h) This function is depicted graphically in Figure 2, where the flat portions represent intervals with no new probability
mass, and the vertical jumps correspond to 𝜑(1), 𝜑(2), 𝜑(3), 𝜑(4).

FIGURE 2 The distribution function of the discrete random variable 𝜍 given in (33).

3 EXPECTED VALUE

1. One of the most important concepts in probability theory is the expectation (or expected value) of a random variable.
2. If 𝜍 is a discrete random variable with pmf 𝜑(0), then the expectation of 𝜍, denoted by 7(𝜍), is defined as
{
E(𝜍) = 0 ⋛ 𝜑(0). (35)
0ς𝜑(0)>0

(a) In words, the expected value of 𝜍 is a weighted average of the possible values of 𝜍, where each value is weighted
by its probability.
(b) For example, suppose 𝜑(0) = 𝜑(1) = 0.5. Then
E(𝜍) = 0 ⋛ 0.5 + 1 ⋛ 0.5 = 0.5, (36)
which is simply the ordinary average of the two possible values, 0 and 1.
(c) On the other hand, if 𝜑(0) = 1φ3 and 𝜑(1) = 2φ3, then
1 2 2
E(𝜍) = 0 ⋛ +1⋛ = , (37)
3 3 3
which is a weighted average of the two possible values, 0 and 1, where the value 1 is given twice as much weight as
0, reflecting the fact that 𝜑(1) = 2𝜑(0).
 9

3. Another way to motivate this definition is through the frequency interpretation of probability.

(a) This interpretation states that if an experiment is repeated independently a very large number of times, then for any
event 7, the proportion of times that 7 occurs will approach 𝜀 (7).
(b) Now, suppose 𝜍 takes values 01 , 02 , … , 0𝛻 with probabilities 𝜑(01 ), 𝜑(02 ), … , 𝜑(0𝛻 ). Think of 𝜍 as the payo" (or
winnings) in a single play of a game of chance.
(c) With probability 𝜑(0𝜚 ), the outcome of a play is 0𝜚 units, for 𝜚 = 1, 2, … , 𝛻. If the game is repeated many times, then
by the frequency interpretation, the proportion of plays resulting in 0𝜚 converges to 𝜑(0𝜚 ).
(d) Therefore, in the long run, the average payo! per play is
{
𝛻
0𝜚 𝜑(0𝜚 ) = E(𝜍). (38)
𝜚=1

(e) This shows that the expected value represents the long-run average outcome when the random experiment is repeated
many times.

4. Example: Fair Die.

(a) Let 𝜍 be the outcome when rolling a fair six-sided die.

(b) Since each face is equally likely, we have
1
𝜑(1) = 𝜑(2) = ⋝ = 𝜑(6) = . (39)
6
(c) The expected value is therefore
1 1 1 1+2+3+4+5+6 7
E(𝜍) = 1 ⋛ +2⋛ +⋝+6⋛ = = . (40)
6 6 6 6 2
(d) In words, 7(𝜍) = 3.5 is the long-run average outcome of repeatedly rolling a fair die, even though 3.5 is not itself
a possible value of 𝜍.

5. Example: Indicator Variable.

(a) An indicator variable 8 for an event ℵ is defined by

)
1, if ℵ occurs,
8= (41)
0, if ℵ4 occurs.
(b) Since 𝜀 (8 = 1) = 𝜀 (ℵ) and 𝜀 (8 = 0) = 1 ω 𝜀 (ℵ), the expected value is
E(8) = 0 ⋛ (1 ω 𝜀 (ℵ)) + 1 ⋛ 𝜀 (ℵ) = 𝜀 (ℵ). (42)

(c) Thus, the expectation of an indicator variable equals the probability of the event it represents.
(d) This fact makes indicator variables especially useful in probability and statistics, as they link events with numerical
analysis.

6. Example: Choosing the Order of Questions.

(a) A contestant on a quiz show is presented with two questions, labeled 1 and 2, which he may attempt to answer in
any order.
(b) If he chooses to try question 𝜚 first, then he will be allowed to proceed to question ℶ (ℶ ⨋ 𝜚) only if his answer to
question 𝜚 is correct.
(c) If his first answer is incorrect, he is not permitted to attempt the other question.
(d) The contestant will receive 9𝜚 dollars if he answers question 𝜚 correctly, for 𝜚 = 1, 2.
(e) For example, he will receive 91 + 92 dollars if both answers are correct.
10

(f) Suppose the probability that he knows the answer to question 𝜚 is 𝜀𝜚 , 𝜚 = 1, 2, and assume that the events
7𝜚 = {he knows the answer to question 𝜚}, 𝜚 = 1, 2,
are independent.
(g) Case 1: Attempt question 1 first. In this case, the possible winnings are
0, with probability 1 ω 𝜀1 ,
91 , with probability 𝜀1 (1 ω 𝜀2 ), (43)
91 + 92 , with probability 𝜀1 𝜀2 .
Hence, the expected winnings are
91 𝜀1 (1 ω 𝜀2 ) + (91 + 92 )𝜀1 𝜀2 . (44)

(h) Case 2: Attempt question 2 first. By symmetry, the expected winnings are
92 𝜀2 (1 ω 𝜀1 ) + (91 + 92 )𝜀1 𝜀2 . (45)

(i) Comparison. The contestant should try question 1 first if

91 𝜀1 (1 ω 𝜀2 ) ∱ 92 𝜀2 (1 ω 𝜀1 ). (46)
Equivalently,

∱
91 𝜀1 92 𝜀2
. (47)
1 ω 𝜀1 1 ω 𝜀2

4 EXPECTATION OF A FUNCTION OF A RANDOM VARIABLE

1. Often we are interested not in 𝜍 itself, but in some function of 𝜍, say .(𝜍), and we wish to compute its expected value.

2. There are two equivalent ways to accomplish this:

(a) Method 1 (via the distribution of .(𝜍)):

i. Since .(𝜍) is itself a random variable, it has its own probability mass function (pmf), which can be derived
from the pmf of 𝜍.
ii. Once the pmf of .(𝜍) is determined, we can compute E(.(𝜍)) directly from the definition of expectation.
(b) Method 2 (direct weighted average):
i. Note that .(𝜍) takes the value .(0) whenever 𝜍 = 0.
ii. Thus it is natural to compute E(.(𝜍)) as a weighted average of the values .(0), where each .(0) is weighted by
the probability 𝜀 (𝜍 = 0).
iii. This leads to the general formula
{
E(.(𝜍)) = .(0𝜚 )𝜑(0𝜚 ). (48)
𝜚∱1

3. Example:

(a) Let 𝜍 be a random variable taking the values ω1, 0, 1 with probabilities
𝜀 (𝜍 = ω1) = 0.2, 𝜀 (𝜍 = 0) = 0.5, 𝜀 (𝜍 = 1) = 0.3. (49)

(b) We wish to find E(𝜍 2 ). Define 𝜔 = 𝜍 2 . Then 𝜔 takes the values 0 and 1.
(c) Using Method 1, the pmf of 𝜔 is
𝜀 (𝜔 = 1) = 𝜀 (𝜍 = 1 or 𝜍 = ω1) = 𝜀 (𝜍 = 1) + 𝜀 (𝜍 = ω1) = 0.3 + 0.2 = 0.5,
(50)
𝜀 (𝜔 = 0) = 𝜀 (𝜍 = 0) = 0.5.
 11

(d) Hence,
E(𝜍 2 ) = E(𝜔 ) = 1 ⋛ 0.5 + 0 ⋛ 0.5 = 0.5. (51)

(e) Using Method 2, we can compute directly:

E(𝜍 2 ) = (ω1)2 ⋛ 0.2 + 02 ⋛ 0.5 + 12 ⋛ 0.3 = 0.2 + 0 + 0.3 = 0.5. (52)
Both methods agree.
(f) Note that
E(𝜍 2 ) = 0.5 ⨋ (7[𝜍])2 . (53)
In fact,
E(𝜍) = (ω1) ⋛ 0.2 + 0 ⋛ 0.5 + 1 ⋛ 0.3 = 0.1, (E(𝜍))2 = 0.01. (54)
This shows that the expectation of a square is not the same as the square of the expectation.

4. General formula for 7(.(𝜍))

Proposition 1. If 𝜍 is a discrete random variable taking values 0𝜚 with probabilities 𝜑(0𝜚 ), then for any real-valued
function .,
{
E(.(𝜍)) = .(0𝜚 )𝜑(0𝜚 ). (55)
𝜚∱1

Proof. (a) Let 3ℶ denote the distinct values of .(0𝜚 ).

(b) Group terms according to which 3ℶ they correspond to:
{ { {
.(0𝜚 )𝜑(0𝜚 ) = .(0𝜚 )𝜑(0𝜚 )
𝜚∱1 ℶ∱1 𝜚ς.(0𝜚 )=3ℶ
{ {
= 3ℶ 𝜑(0𝜚 )
(56)
ℶ∱1 𝜚ς.(0𝜚 )=3ℶ
{ {
= 3ℶ 𝜑(0𝜚 ).
ℶ∱1 𝜚ς.(0𝜚 )=3ℶ

(c) The inner sum is simply 𝜀 (.(𝜍) = 3ℶ ). Thus,

{ {
.(0𝜚 )𝜑(0𝜚 ) = 3ℶ 𝜀 (.(𝜍) = 3ℶ ) = E(.(𝜍)). (57)
𝜚∱1 ℶ∱1

5. Example: The Optimal Stocking Problem.

(a) A product is sold seasonally. Each unit sold yields a net profit of 2 dollars, while each unit left unsold at the end of
the season results in a net loss of ⋞ dollars.
(b) The random variable 𝜍 denotes the number of units actually demanded during the season, with pmf 𝜑(𝜚), 𝜚 ∱ 0.
(c) The store must stock in advance and wants to determine the number of units , that maximizes its expected profit.
(d) If , units are stocked, then the profit <(,) is

if 𝜍 ∲ ,,
❳
/2𝜍 ω (, ω 𝜍)⋞,
<(,) = \ (58)
/,2, if 𝜍 > ,.
(
12

(e) Thus the expected profit is

ϱ
{
,
⦅ ⦆ {
E(<(,)) = 2𝜚 ω (, ω 𝜚)⋞ 𝜑(𝜚) + ,2 𝜑(𝜚)
𝜚=0 𝜚=,+1

{
,
{
,
= (2 + ⋞) 𝜚𝜑(𝜚) ω (2 + ⋞), 𝜑(𝜚) + ,2 (59)
𝜚=0 𝜚=0

{
,
= ,2 + (2 + ⋞) (𝜚 ω ,)𝜑(𝜚).
𝜚=0

(f) To identify the optimal ,, compare the e"ect of increasing the stock by one unit:
{
,
E(<(, + 1)) ω E(<(,)) = 2 ω (2 + ⋞) 𝜑(𝜚). (60)
𝜚=0

(g) Therefore, it is better to increase the stock from , to , + 1 whenever

{
,
(61)
2
𝜑(𝜚) < .
𝜚=0
2+⋞
[,
(h) Let ,∇ be the largest , satisfying (61). Because the cdf 𝜚=0
𝜑(𝜚) is nondecreasing in ,, this ,∇ exists and is unique.
(i) From the inequality, we conclude:
i. Stocking ,∇ + 1 units strictly increases expected profit relative to stocking ,∇ units.
ii. Stocking more than ,∇ + 1 units reduces expected profit, since then the incremental gain is negative.
(j) Hence, the optimal stocking level is
,opt = ,∇ + 1, (62)
which corresponds to the so-called critical ratio rule in inventory theory:

Order up to the smallest , such that 𝜀 (𝜍 ∲ ,) ∱ (63)

2
.
2+⋞
6. A simple but important consequence of Proposition 1 is the linearity of expectation.
Corollary 1. If 1 and 2 are constants, then
E(1𝜍 + 2) = 1E(𝜍) + 2. (64)

Proof. (a) By definition,

{
E(1𝜍 + 2) = (10 + 2)𝜑(0) (65)
0ς𝜑(0)>0
{ {
=1 0𝜑(0) + 2 𝜑(0) (66)
0ς𝜑(0)>0 0ς𝜑(0)>0

= 1E(𝜍) + 2. (67)
[
where the last equality follows from 0ς𝜑(0)>0 𝜑(0) = 1.

7. This property shows that expectation is a linear operator: multiplying a random variable by a constant multiplies its
expectation by the same constant, and adding a constant simply shifts the expectation.

8. The expected value of a random variable 𝜍, E(𝜍), is also referred to as the mean or the first moment of 𝜍.

9. More generally, the quantity E(𝜍 𝛻 ) for 𝛻 ∱ 1 is called the 𝛻th moment of 𝜍:
{
E(𝜍 𝛻 ) = 0𝛻 𝜑(0). (68)
0ς𝜑(0)>0
 13

10. In particular:

(a) E(𝜍) is the first moment (the mean).

(b) E(𝜍 2 ) is the second moment, which plays an important role in defining the variance.
(c) Higher-order moments (𝛻 = 3, 4, …) provide information about skewness and kurtosis of the distribution.

5 VARIANCE

1. Given a random variable 𝜍 with distribution function ⊲ , it is often desirable to summarize the key features of ⊲ using
suitably defined numerical measures.

2. One such measure is the mean E(𝜍), which represents the “center” or long-run average value of 𝜍.

3. However, although E(𝜍) captures the location of 𝜍, it tells us nothing about the degree of variation, or the spread, in the
possible values of 𝜍.

4. Illustration. Consider the following random variables:

ℏ = 0, with probability 1,
)
ω1, with probability 0.5,
𝜔 =
+1, with probability 0.5, (69)
)
ω100, with probability 0.5,
>=
+100, with probability 0.5.
Although ℏ , 𝜔 , and > all have the same expectation, namely E(ℏ ) = E(𝜔 ) = E(>) = 0, their spreads are very di"erent:
ℏ is constant, 𝜔 fluctuates moderately, while > fluctuates dramatically.

5. Since we expect 𝜍 to take values “around” its mean ⋆ = E(𝜍), it seems natural to measure variation using the average
distance from the mean:
E(]𝜍 ω ⋆]). (70)
However, the absolute value is mathematically inconvenient for algebraic manipulations.

6. Instead, a more tractable and widely used measure is the expected squared deviation from the mean.
Definition 1. If 𝜍 is a random variable with mean ⋆, then the variance of 𝜍, denoted Var(𝜍), is defined by
⟦ ⟧
Var(𝜍) = E (𝜍 ω ⋆)2 . (71)

Variance measures the typical squared distance of 𝜍 from its mean. A larger variance indicates greater spread in the
possible values of 𝜍.

7. An alternative but very useful formula for the variance can be derived as follows:
⟦ ⟧
Var(𝜍) = E (𝜍 ω ⋆)2
⟦ ⟧
= E 𝜍 2 ω 2⋆𝜍 + ⋆ 2
(72)
= E(𝜍 2 ) ω 2⋆E(𝜍) + ⋆2

= E(𝜍 2 ) ω ⋆2 .
This identity shows that variance can be computed directly from the first and second moments of 𝜍.

8. Example: Variance of a Fair Die.

14

(a) Let 𝜍 be the outcome when rolling a fair six-sided die. We wish to compute Var(𝜍).
(b) First note that
1+2+3+4+5+6 7
E(𝜍) = = . (73)
6 2
(c) Next, compute the second moment:
12 + 22 + 32 + 42 + 52 + 62 91
E(𝜍 2 ) = = . (74)
6 6
(d) Therefore, the variance is
⦅ ⦆2 91 ⟩ 7 ⟪2 35
Var(𝜍) = E(𝜍 2 ) ω E(𝜍) = ω = . (75)
6 2 12
9. A useful identity is that for any constants 1 and 2,
Var(1𝜍 + 2) = 12 Var(𝜍). (76)

Proof. (a) Let ⋆ = E(𝜍). Then 7(1𝜍 + 2) = 1⋆ + 2.

(b) Hence,
⟩⦅ ⦆2 ⟪
Var(1𝜍 + 2) = E 1𝜍 + 2 ω (1⋆ + 2)
⦅ ⦆
= E 12 (𝜍 ω ⋆)2
(77)
⦅ ⦆
= 12 E (𝜍 ω ⋆)2

= 12 Var(𝜍).

10. The square root of the variance is called the standard deviation of 𝜍, denoted Sd(𝜍):
⌊
Sd(𝜍) = Var(𝜍). (78)
The standard deviation is in the same units as 𝜍, making it easier to interpret as an absolute measure of spread.

11. Having established the notions of expectation and variance, we now turn to specific families of discrete random variables,
classified by their probability mass functions, in the following sections.

6 THE BERNOULLI AND BINOMIAL RANDOM VARIABLES

1. Consider an experiment (or trial) whose outcome can be classified into one of two categories: success or failure.

2. Define a random variable 𝜍 such that

)
1, if the outcome is a success,
𝜍=
0, if the outcome is a failure.
The probability mass function (pmf) of 𝜍 is then
𝜀 (𝜍 = 0) = 1 ω 𝜑,
(79)
𝜀 (𝜍 = 1) = 𝜑,
where 𝜑, with 0 ∲ 𝜑 ∲ 1, represents the probability of success.

3. A random variable 𝜍 with pmf given by (79) is called a Bernoulli random variable (named after the Swiss mathematician
Jacob Bernoulli). Intuitively, it models a single “yes/no” or “success/failure” outcome.

4. Now suppose we repeat such trials independently 𝛻 times, each trial resulting in success with probability 𝜑 and failure
with probability 1 ω 𝜑.
 15

5. Let 𝜍 denote the total number of successes observed in these 𝛻 trials. Then 𝜍 is said to follow a binomial distribution
with parameters (𝛻, 𝜑).

6. Notice that a Bernoulli random variable is just a special case of the binomial distribution when 𝛻 = 1. In other words,
Bernoulli(𝜑) ∳ Binomial(1, 𝜑). (80)

7. The pmf of a binomial random variable with parameters (𝛻, 𝜑) is given by

} ⦃
(81)
𝛻 𝜚
𝜀 (𝜍 = 𝜚) = 𝜑 (1 ω 𝜑) 𝛻ω𝜚 , 𝜚 = 0, 1, … , 𝛻.
𝜚
8. To see why (81) holds, note that:

• The probability of any specific sequence of 𝛻 outcomes containing 𝜚 successes and 𝛻 ω 𝜚 failures is 𝜑𝜚 (1 ω 𝜑)𝛻ω𝜚 , since
the trials are independent.
⦅⦆
• There are 𝛻𝜚 such distinct sequences, corresponding to the di"erent ways of choosing which 𝜚 of the 𝛻 trials are
successes.

Multiplying these together yields the formula in (81).

9. Finally, we can verify that the probabilities sum to 1 using the binomial theorem:
𝛻 } ⦃
{ 𝛻
{
(82)
𝛻 𝜚
𝜀 (𝜍 = 𝜚) = 𝜑 (1 ω 𝜑) 𝛻ω𝜚 = (𝜑 + (1 ω 𝜑))𝛻 = 1.
𝜚=0 𝜚=0
𝜚
Thus, the binomial distribution is a valid probability distribution.

10. Example:

(a) Suppose screws produced by a certain company are defective with probability 0.01, independently of each other.
(b) The company sells the screws in packages of 10 and o"ers a money-back guarantee: a package will be replaced if it
contains two or more defective screws. In other words, a package is acceptable only if it has 0 or 1 defective screw.
(c) What proportion of packages sold must the company replace?
(d) Let 𝜍 denote the number of defective screws in a package. Then 𝜍 follows a binomial distribution with parameters
(𝛻 = 10, 𝜑 = 0.01).
(e) The probability that a package must be replaced is the probability of having 2 or more defectives:
𝜀 (replace) = 𝜀 (𝜍 ∱ 2)
= 1 ω 𝜀 (𝜍 = 0) ω 𝜀 (𝜍 = 1)
} ⦃ } ⦃
10 0 10 10
=1ω (0.01) (0.99) ω (0.01)1 (0.99)9
0 1
∂ 0.004. (83)

(f) Thus, only about 0.4% of the packages will have to be replaced under the company’s guarantee.

11. Example:

(a) Consider the following gambling game, often called the wheel of fortune (or chuck-a-luck), which is popular at
carnivals and casinos:
i. A player chooses and bets on one of the numbers 1 through 6.
ii. Three dice are rolled. If the chosen number appears 𝜚 times (𝜚 = 1, 2, 3), the player wins 𝜚 units.
iii. If the chosen number does not appear at all, the player loses 1 unit.
(b) In actual practice, the game is played with a spinning wheel labeled with triples of numbers between 1 and 6.
However, this version is mathematically equivalent to rolling three independent dice.
(c) The natural question is: Is this game fair to the player?
16

(d) Assuming the dice are fair and independent, the number of times the player’s chosen number appears follows a
binomial distribution with parameters
1
𝛻 = 3, 𝜑 = .
6
(e) Let 𝜍 denote the player’s net winnings from a single play. Then
} ⦃ ⟩ ⟪0 ⟩ ⟪3
3 1 5 125
𝜀 (𝜍 = ω1) = = ,
0 6 6 216
} ⦃ ⟩ ⟪1 ⟩ ⟪2
3 1 5 75
𝜀 (𝜍 = 1) = = ,
1 6 6 216
} ⦃ ⟩ ⟪2 ⟩ ⟪1 (84)
3 1 5 15
𝜀 (𝜍 = 2) = = ,
2 6 6 216
} ⦃ ⟩ ⟪3 ⟩ ⟪0
3 1 5 1
𝜀 (𝜍 = 3) = = .
3 6 6 216
(f) To determine fairness, compute the expected value of 𝜍:
ω1 ⋛ 125 + 1 ⋛ 75 + 2 ⋛ 15 + 3 ⋛ 1 ω17
E(𝜍) = = . (85)
216 216
(g) Therefore, the game is not fair. On average, the player loses about 17 units for every 216 plays, which corresponds
to a loss of roughly 0.079 units per play.

12. Example:

(a) Consider a simple genetic model in which a particular trait (such as eye color or left-handedness) is determined by
a pair of genes.
(b) Let ≨ denote a dominant gene and 𝐴 denote a recessive gene.
(c) An individual with the gene pair ≨≨ is purely dominant, one with 𝐴𝐴 is purely recessive, and one with 𝐴≨ (or ≨𝐴) is
hybrid.
(d) In terms of outward appearance (phenotype), both the purely dominant (≨≨) and the hybrid (𝐴≨) individuals exhibit
the dominant trait, while only 𝐴𝐴 individuals exhibit the recessive trait.
(e) Each child inherits one gene from each parent.
(f) Suppose two hybrid parents (𝐴≨ and 𝐴≨) have a total of four children. What is the probability that exactly three of
the four children show the dominant trait in appearance?
(g) To answer this, note that for each child:
• With probability 1φ4, the gene pair is ≨≨,
• With probability 1φ4, the gene pair is 𝐴𝐴,
• With probability 1φ2, the gene pair is 𝐴≨.
(h) Therefore, the probability that a child exhibits the dominant trait (either ≨≨ or 𝐴≨) is
1 1 3
𝜀 (dominant phenotype) = + = . (86)
4 2 4
(i) Let 𝜍 denote the number of dominant-appearing children among the four. Then
⟩ ⟪
3
𝜍 − Binomial 𝛻 = 4, 𝜑 = . (87)
4
(j) The desired probability is
} ⦃ ⟩ ⟪3 ⟩ ⟪1
4 3 1 27
𝜀 (𝜍 = 3) = = . (88)
3 4 4 64
13. Example:

(a) Consider a jury trial in which there are 12 jurors, and it takes at least 8 votes of “guilty” to convict the defendant.
 17

(b) That is, in order for the defendant to be convicted, at least 8 of the 12 jurors must vote “guilty.” Otherwise, the
defendant is acquitted.
(c) Suppose each juror acts independently, and regardless of the defendant’s actual guilt, each juror makes the correct
individual decision with probability 𝐵. What is the probability that the jury as a whole renders a correct decision?
(d) As stated, the problem cannot be solved yet, since we must distinguish two possible situations: whether the defendant
is truly guilty or truly innocent.
(e) If the defendant is innocent, then the correct jury decision is acquittal, which occurs provided that fewer than 8
jurors mistakenly vote “guilty.” Equivalently, at least 5 jurors must vote “not guilty” (the correct vote). Thus, the
probability of a correct decision is
12 } ⦃
{ 12 𝜚
𝐵 (1 ω 𝐵)12ω𝜚 , (89)
𝜚=5
𝜚
where 𝜚 represents the number of jurors who correctly vote “not guilty.”
(f) If the defendant is guilty, then the correct jury decision is conviction, which occurs provided that at least 8 jurors
vote “guilty” (the correct vote). Hence, the probability of a correct decision is
12 } ⦃
{ 12 𝜚
𝐵 (1 ω 𝐵)12ω𝜚 , (90)
𝜚=8
𝜚
where 𝜚 represents the number of jurors who correctly vote “guilty.”
(g) Finally, to obtain the unconditional probability that the jury renders a correct verdict, we must account for the prior
probability 𝐶 that the defendant is guilty. Conditioning on whether or not the defendant is guilty, we have
12 } ⦃ 12 } ⦃
{ 12 𝜚 { 12 𝜚
𝜀 (jury correct) = 𝐶 𝐵 (1 ω 𝐵)12ω𝜚 + (1 ω 𝐶) 𝐵 (1 ω 𝐵)12ω𝜚 . (91)
𝜚=8
𝜚 𝜚=5
𝜚
14. Example:
(a) A communication system consists of 𝛻 independent components, each functioning with probability 𝜑.
(b) The system is said to be e!ective if at least one-half of its components function.
(c) Question: For what values of 𝜑 is a 5-component system more likely to be e"ective than a 3-component system?
i. Since the number of functioning components follows a Binomial(𝛻, 𝜑) distribution, the probability that a 5-
component system is e"ective (i.e., at least 3 of 5 work) is
} ⦃ } ⦃
5 3 5 4
𝜀5 (e"ective) = 2
𝜑 (1 ω 𝜑) + 𝜑 (1 ω 𝜑) + 𝜑5 . (92)
3 4
ii. Similarly, the probability that a 3-component system is e"ective (i.e., at least 2 of 3 work) is
} ⦃
3 2
𝜀3 (e"ective) = 𝜑 (1 ω 𝜑) + 𝜑3 . (93)
2
iii. The 5-component system is better when
𝜀5 (e"ective) > 𝜀3 (e"ective). (94)
After simplification, this inequality reduces to
3(𝜑 ω 1)2 (2𝜑 ω 1) > 0, (95)
1
which holds if and only if 𝜑 >.
2
(d) More generally, when is a (2ℷ + 1)-component system better than a (2ℷ ω 1)-component system?
i. Consider a system of 2ℷ + 1 components. Let 𝜍 be the number of functioning components among the first
2ℷ ω 1. Then
𝜀2ℷ+1 (e"ective) = 𝜀 (𝜍 ∱ ℷ + 1)
⟦ ⟧
+ 𝜀 (𝜍 = ℷ) 1 ω (1 ω 𝜑)2 (96)
+ 𝜀 (𝜍 = ℷ ω 1) 𝜑2 ,
18

since the system works in three cases:

• 𝜍 ∱ ℷ + 1 (already e"ective among the first 2ℷ ω 1 components),
• 𝜍 = ℷ and at least one of the remaining two components functions,
• 𝜍 = ℷ ω 1 and both of the remaining two components function.
ii. For a (2ℷ ω 1)-component system,
𝜀2ℷω1 (e"ective) = 𝜀 (𝜍 ∱ ℷ) = 𝜀 (𝜍 = ℷ) + 𝜀 (𝜍 ∱ ℷ + 1). (97)
iii. Subtracting gives
𝜀2ℷ+1 (e"ective) ω 𝜀2ℷω1 (e"ective) = 𝜀 (𝜍 = ℷ ω 1)𝜑2 ω (1 ω 𝜑)2 𝜀 (𝜍 = ℷ). (98)
} ⦃ } ⦃
2ℷ ω 1 2ℷ ω 1
Substituting the binomial probabilities and using = , we obtain
ℷω1 ℷ
} ⦃
2ℷ ω 1 ℷ ⦅ ⦆
𝜀2ℷ+1 (e"ective) ω 𝜀2ℷω1 (e"ective) = 𝜑 (1 ω 𝜑)ℷ 𝜑 ω (1 ω 𝜑) . (99)
ℷ
iv. Since the binomial coe!cient and 𝜑ℷ (1ω𝜑)ℷ are nonnegative, the sign depends on 𝜑ω(1ω𝜑) = 2𝜑ω1. Therefore,
1
𝜀2ℷ+1 (e"ective) > 𝜀2ℷω1 (e"ective)  𝜑 > . (100)
2
(e) Conclusion: Adding two extra components improves system reliability only when each component is more likely
1 1
to work than to fail (i.e., 𝜑 > ). If 𝜑 < , additional components actually reduce the system’s chance of being
2 2
e"ective.

6.1 Properties of Binomial Random Variables

1. Up to this point, we have introduced the definition of a binomial random variable 𝜍 with parameters (𝛻, 𝜑). We now turn
to some of its important properties, such as its expectation and variance.

2. These quantities are fundamental because they summarize the “center” and the “spread” of the distribution, and they
appear repeatedly in both theoretical results and applied problems.

3. In general, the ℷ-th moment of 𝜍 is defined as

} ⦃
{
𝛻
(101)
𝛻 0
E(𝜍 ℷ ) = 0ℷ 𝜑 (1 ω 𝜑)𝛻ω0 .
0=0
0
Since the term with 0 = 0 vanishes, this sum may also be written as
} ⦃
{𝛻
E(𝜍 ) =
ℷ
0 ℷ 𝛻
𝜑0 (1 ω 𝜑)𝛻ω0 . (102)
0=1
0

4. To simplify this expression, we use the combinatorial identity

} ⦃ } ⦃
𝛻ω1
(103)
𝛻
0 =𝛻 .
0 0ω1
Intuitively, this identity says: if we want to count the number of ways of choosing 0 objects out of 𝛻 and then designating
one of them as “special,” this is equivalent to first choosing the special object in 𝛻 ways and then selecting the remaining
0 ω 1 objects from the remaining 𝛻 ω 1.

5. Applying this identity, we obtain

}
⦃
{
𝛻
𝛻 ω 1 0ω1
E(𝜍 ) = 𝛻𝜑
ℷ
0 ℷω1
𝜑 (1 ω 𝜑)𝛻ω0
0=1
0ω1
} ⦃
{
𝛻ω1
𝛻ω1 3 (104)
= 𝛻𝜑 (3 + 1)ℷω1 𝜑 (1 ω 𝜑)(𝛻ω1)ω3
3=0
3
⦅ ⦆
= 𝛻𝜑 E (𝜔 + 1)ℷω1 ,
 19

where 𝜔 is a binomial random variable with parameters (𝛻 ω 1, 𝜑).

6. This recursive formula is particularly useful for small ℷ. For example:

E(𝜍) = 𝛻𝜑, (ℷ = 1)
2
⦅ ⦆ (105)
E(𝜍 ) = 𝛻𝜑 E(𝜔 + 1) = 𝛻𝜑 (𝛻 ω 1)𝜑 + 1 , (ℷ = 2).
7. Finally, the variance of 𝜍 is
⦅ ⦆2
Var(𝜍) = E(𝜍 2 ) ω E(𝜍)
⦅ ⦆
= 𝛻𝜑 (𝛻 ω 1)𝜑 + 1 ω (𝛻𝜑)2 (106)
= 𝛻𝜑(1 ω 𝜑).
This elegant result tells us that the variability of a binomial random variable increases with the number of trials 𝛻 but
decreases as the success probability 𝜑 approaches 0 or 1.

8. The following proposition explains why the binomial probability mass function has a “hump-shaped” curve: it first
increases and then decreases as ℷ grows.
Proposition 2. Let 𝜍 − Binomial(𝛻, 𝜑) with 0 < 𝜑 < 1. Then as ℷ goes from 0 to 𝛻, the probability 𝜀 (𝜍 = ℷ) increases
monotonically until ℷ reaches the largest integer less than or equal to (𝛻+1)𝜑, and then decreases monotonically afterwards.
In other words, the binomial pmf attains its maximum at ℷ = ⌋(𝛻 + 1)𝜑⌈.

Proof. (a) To see how 𝜀 (𝜍 = ℷ) changes with ℷ, it is natural to look at the ratio
𝜀 (𝜍 = ℷ)
.
𝜀 (𝜍 = ℷ ω 1)
If this ratio is greater than 1, then 𝜀 (𝜍 = ℷ) > 𝜀 (𝜍 = ℷ ω 1), meaning the sequence is increasing at ℷ. If the ratio
is less than 1, the sequence is decreasing.
(b) Compute the ratio:
} ⦃
𝛻 ℷ
𝜑 (1 ω 𝜑)𝛻ωℷ
𝜀 (𝜍 = ℷ) ℷ
=} ⦃
𝜀 (𝜍 = ℷ ω 1) 𝛻
𝜑ℷω1 (1 ω 𝜑)𝛻ωℷ+1 (107)
ℷω1
(𝛻 ω ℷ + 1)𝜑
= .
ℷ(1 ω 𝜑)
(c) Therefore, 𝜀 (𝜍 = ℷ) ∱ 𝜀 (𝜍 = ℷ ω 1) if and only if

∱ 1,
(𝛻 ω ℷ + 1)𝜑
(108)
ℷ(1 ω 𝜑)
which simplifies to
ℷ ∲ (𝛻 + 1)𝜑. (109)

(d) This means:

• As long as ℷ ∲ (𝛻 + 1)𝜑, each probability is at least as large as the previous one, so the sequence is increasing.
• Once ℷ > (𝛻 + 1)𝜑, the ratio becomes less than 1, so the sequence starts to decrease.
Thus, the maximum occurs at ℷ = ⌋(𝛻 + 1)𝜑⌈, the largest integer less than or equal to (𝛻 + 1)𝜑.

9. Example: Voting Power

(a) In a U.S. presidential election, the candidate who receives the most votes in a state wins all of that state’s electoral
college votes.
(b) The number of electoral votes is roughly proportional to the state’s population. For simplicity, suppose a state with
𝛻 citizens has about 𝛻4 electoral votes for some constant 4.
20

(c) We want to measure the average power of a single voter in a state of size 𝛻. A natural way is to ask: in a very close
election, what is the probability that one voter is decisive?
(d) If 𝛻 = 2ℷ + 1 (odd case), then one voter is decisive exactly when the other 2ℷ voters split evenly between the two
candidates.
(e) Assuming independence and that each of the other voters has probability 1φ2 of voting for either candidate, this
probability equals
} ⦃ ⟩ ⟪2ℷ
2ℷ 1 (2ℷ)!
𝜀 ({voter is decisive}) = = . (110)
ℷ 2 (ℷ!)2 22ℷ
(f) To approximate this expression for large ℷ, we use Stirling’s approximation
⌊
ℷ! − ℷℷ+1φ2 6ωℷ 2𝐷. (111)
Substituting this into the formula gives
1
𝜀 ({voter is decisive}) − ⌊ . (112)
ℷ𝐷
⌊
Thus, the chance of being decisive shrinks roughly like 1φ 𝛻 as the population grows.
(g) If a voter is decisive, he or she influences all 𝛻4 electoral votes. Therefore, the average power of a voter in a state of
size 𝛻 is
⌉
2𝛻
average power = 𝛻4 ⋛ 𝜀 ({voter is decisive}) − 4 . (113)
𝐷
⌊
(h) This result shows that a voter’s influence is not proportional to 𝛻 (the population), but rather to 𝛻. In other words:
• Larger states give their voters more influence than voters in smaller states.
• However, the advantage grows slowly (only like a square root). For example, a state 100 times larger gives its
voters only about 10 times more power.

6.2 Computing the Binomial Distribution Function

1. Suppose that 𝜍 − Bin(𝛻, 𝜑). Its cumulative distribution function is
0 } ⦃
𝜀 (𝜍 ∲ 0) =
{
(114)
𝛻 ℷ
𝜑 (1 ω 𝜑) 𝛻ωℷ , 𝜚 = 0, 1, … , 𝛻.
ℷ=0
ℷ

2. A key to e!cient computation is the relationship between adjacent point probabilities:

𝜑 𝛻ωℷ
𝜀 (𝜍 = ℷ + 1) = ⋛ ⋛ 𝜀 (𝜍 = ℷ). (115)
1ω𝜑 ℷ+1
Starting from 𝜀 (𝜍 = 0) = (1 ω 𝜑)𝛻 , one can build up 𝜀 (𝜍 = 1), 𝜀 (𝜍 = 2), … and then obtain 𝜀 (𝜍 ∲ 0) by cumulative
summation.
1
3. Numerical tip. When 𝜑 > , it is sometimes more stable to work backward, starting from 𝜀 (𝜍 = 𝛻) = 𝜑𝛻 and using
2
(115). Another common practice is to start from the mode ⌋(𝛻 + 1)𝜑⌈ and expand in both directions.

4. Example (𝛻 = 6, 𝜑 = 0.4):

(a) Initial value:

𝜀 (𝜍 = 0) = (1 ω 0.4)6 = 0.66 ∂ 0.046656. (116)
 21

(b) Using (115) recursively:

0.4 6ω0
𝜀 (𝜍 = 1) = ⋛ ⋛ 𝜀 (𝜍 = 0) ∂ 0.186624,
0.6 0+1
0.4 6ω1
𝜀 (𝜍 = 2) = ⋛ ⋛ 𝜀 (𝜍 = 1) ∂ 0.311040,
0.6 1+1
0.4 6ω2
𝜀 (𝜍 = 3) = ⋛ ⋛ 𝜀 (𝜍 = 2) ∂ 0.276480,
0.6 2+1
(117)
0.4 6ω3
𝜀 (𝜍 = 4) = ⋛ ⋛ 𝜀 (𝜍 = 3) ∂ 0.138240,
0.6 3+1
0.4 6ω4
𝜀 (𝜍 = 5) = ⋛ ⋛ 𝜀 (𝜍 = 4) ∂ 0.036864,
0.6 4+1
0.4 6ω5
𝜀 (𝜍 = 6) = ⋛ ⋛ 𝜀 (𝜍 = 5) ∂ 0.004096.
0.6 5+1

7 THE POISSON RANDOM VARIABLE

1. A random variable 𝜍 taking values 0, 1, 2, … is said to follow a Poisson distribution with parameter 5 > 0 if
50
𝜑(0) = 𝜀 (𝜍 = 0) = 6ω5 , 0 = 0, 1, 2, … . (118)
0!
2. It is easily verified that the probabilities sum to one:
ϱ ϱ
{ { 50
𝜑(0) = 6ω5 = 6ω5 65 = 1. (119)
0=0 0=0
0!

3. The Poisson distribution is important because it arises as an approximation to the binomial distribution when 𝛻 is large
and 𝜑 is small, with 5 = 𝛻𝜑 being of moderate size.

(a) If 𝜍 − Bin(𝛻, 𝜑), then

} ⦃
𝛻 0
𝜀 (𝜍 = 0) = 𝜑 (1 ω 𝜑)𝛻ω0
(120)
0
⟩ ⟪0 ⟩ ⟪
𝛻! 5 5 𝛻ω0
= 1ω .
0!(𝛻 ω 0)! 𝛻 𝛻
(b) Rewriting,
𝛻(𝛻 ω 1) ⋝ (𝛻 ω 0 + 1) 50 (1 ω 5φ𝛻)𝛻
𝜀 (𝜍 = 0) = ⋛ ⋛ . (121)
𝛻0 0! (1 ω 5φ𝛻)0
(c) For 𝛻 large, 5 fixed, and 0 constant,
⟩
⟪
5 𝛻
1ω 6ω5 , (122)
𝛻
𝛻(𝛻 ω 1) ⋝ (𝛻 ω 0 + 1)
 1, (123)
𝛻0 ⟩ ⟪0
(124)
5
1ω  1,
𝛻
so that
50 ω5
𝜀 (𝜍 = 0) ∂
6 . (125)
0!
4. In words: if 𝛻 independent trials are performed, each with a small success probability 𝜑, then the number of successes
is approximately Poisson with parameter 5 = 𝛻𝜑. The value 5 is often determined empirically as the average number of
events observed.

5. Typical examples of Poisson-type random variables include:

(a) the number of misprints on a page of a book,

22

(b) the number of people in a community surviving to age 100,

(c) the number of wrong telephone numbers dialed in a day,
(d) the number of packages of dog biscuits sold in a store per day,
(e) the number of customers entering a post o!ce in a day,
(f) the number of vacancies arising in a federal judicial system in a year,
(g) the number of 𝐶-particles emitted in a fixed period of time from radioactive material.

6. These situations are all approximately Poisson because they can be modeled as many independent trials with small
probability of success.

(a) For instance, each letter typed on a page has a small chance of being misprinted, so the total misprints are Poisson
with 5 = 𝛻𝜑, where 𝛻 is the number of letters on the page.
(b) Similarly, each person in a community has a small chance of reaching age 100.
(c) Each customer entering a store has a small chance of buying a given product, and so forth.

7. Example:

(a) Suppose that the probability that an item produced by a certain machine is defective is 0.1. We are interested in the
probability that a sample of 10 items contains at most 1 defective item.
(b) Exact computation (Binomial). Let 𝜍 − Binomial(𝛻 = 10, 𝜑 = 0.1). Then
𝜀 (𝜍 ∲ 1) = 𝜀 (𝜍 = 0) + 𝜀 (𝜍 = 1)
} ⦃ } ⦃
10 10
= 0 10
(0.1) (0.9) + (0.1)1 (0.9)9 (126)
0 1
= 0.3487 + 0.3874 ∂ 0.7361.
(c) Approximation (Poisson). Since 𝛻 = 10 is moderately large and 𝜑 = 0.1 is small, the Poisson approximation with
parameter 5 = 𝛻𝜑 = 1 is reasonable. If 𝜔 − Poisson(5 = 1), then
𝜀 (𝜔 ∲ 1) = 𝜀 (𝜔 = 0) + 𝜀 (𝜔 = 1)
10 11 (127)
= 6ω1 + 6ω1 = 6ω1 + 6ω1 ∂ 0.7358.
0! 1!
(d) Comparison. The exact Binomial result is 0.7361, and the Poisson approximation gives 0.7358, which are extremely
close. This illustrates why the Poisson distribution is widely used as a practical approximation to the Binomial when
𝛻 is large and 𝜑 is small.

8. Example:

(a) Consider an experiment in which we count the number of 𝐶-particles emitted in a one–second interval by 1 gram of
radioactive material.
(b) From past observations, the average number of 𝐶-particles emitted in one second is 3.2. We want to approximate
the probability that at most 2 𝐶-particles are observed in a given second.
(c) To model this, suppose the material contains a very large number 𝛻 of atoms. Each atom has a small probability
of disintegrating in one second. If the expected number of disintegrations per second is 3.2, then each atom has
probability roughly 3.2φ𝛻 of decaying in that second.
(d) The total number of emitted particles can then be viewed as a binomial random variable with parameters (𝛻, 𝜑 =
3.2φ𝛻). Since 𝛻 is huge and 𝜑 is tiny, with 𝛻𝜑 = 3.2 fixed, the Poisson approximation applies. Thus, the count of
𝐶-particles can be modeled by
𝜍 − Poisson(5 = 3.2).
 23

(e) The desired probability is

𝜀 (𝜍 ∲ 2) = 𝜀 (𝜍 = 0) + 𝜀 (𝜍 = 1) + 𝜀 (𝜍 = 2)
(3.2)0 ω3.2 (3.2)1 ω3.2 (3.2)2
=
0!
6 +
1!
6 +
2! (128)
⟩ (3.2)2 ⟪
ω3.2
=6 1 + 3.2 + .
2
(f) Numerically,
𝜀 (𝜍 ∲ 2) ∂ 0.3799. (129)

So in about 38% of intervals we see 0–2 particles, while in the other 62% of intervals we see 3 or more.

9. Recall that a Poisson random variable provides an approximation to a binomial random variable with parameters (𝛻, 𝜑)
when 𝛻 is large, 𝜑 is small, and 5 = 𝛻𝜑.

10. Since a binomial random variable has expected value 𝛻𝜑 = 5 and variance 𝛻𝜑(1 ω 𝜑) = 5(1 ω 𝜑) ∂ 5 (when 𝜑 is small), we
might anticipate that both the mean and the variance of a Poisson random variable equal its parameter 5. We now verify
this directly.

(a) For the mean:

ϱ
{ 50 ω5
E(𝜍) = 0⋛ 6
0=0
0!
ϱ
{ 50ω1
=5 6ω5 (130)
0=1
(0 ω 1)!
ϱ
{ 53
= 56ω5 = 5.
3=0
3!

(b) For the variance, we first compute the factorial moment:

ϱ
{ 50 ω5
E[𝜍(𝜍 ω 1)] = 0(0 ω 1) ⋛ 6
0!
(131)
0=0
ϱ
{ 53
2 ω5 2
=5 6 =5 .
3=0
3!
Hence,
Var(𝜍) = E[𝜍(𝜍 ω 1)] + E(𝜍) ω (E(𝜍))2 = 52 + 5 ω 52 = 5. (132)

(c) Thus, the Poisson distribution is unique in that both its mean and variance are equal to the same parameter 5.

11. The Poisson(5 = 𝛻𝜑) distribution is an excellent approximation to the distribution of the number of successes in 𝛻 indepen-
dent Bernoulli trials, each with success probability 𝜑, provided 𝛻 is large and 𝜑 is small. Remarkably, the approximation
remains valid even when the trials are not fully independent, as long as the dependence is weak.

(a) Example (The Hat-Check Problem). Suppose 𝛻 men randomly select hats from a set of 𝛻 hats, one belonging to each
person.
(b) Define a “success” as a man selecting his own hat. Then the number of successes can be viewed as the number of
matches in 𝛻 trials.
(c) For each person 𝜚,

, ℶ ⨋ 𝜚.
1 1
𝜀 (7𝜚 ) = , 𝜀 (7𝜚 ϖ 7ℶ ) = (133)
𝛻 𝛻ω1
(d) Clearly, the events {7𝜚 } are not independent. However, as 𝛻 grows large, the dependence between them becomes
weak, since the probability that one match occurs hardly a"ects the probability of another match.
24

(e) Consequently, the total number of matches approximately follows a Poisson(1) distribution, i.e., with mean 1. In fact,
the exact distribution of the number of matches is given by the so-called rencontres numbers, and the Poisson(1)
distribution serves as an excellent asymptotic approximation.

12. Poisson Paradigm:

(a) Suppose we have 𝛻 events, where event 𝜚 occurs with probability 𝜑𝜚 , for 𝜚 = 1, … , 𝛻.
(b) If each 𝜑𝜚 is small (so that no single event dominates) and the events are independent or only weakly dependent, then
the total number of events that occur is approximately distributed as a Poisson random variable.
(c) The mean of this approximating Poisson distribution is
{
𝛻
5= 𝜑𝜚 , (134)
𝜚=1
which represents the expected total number of events.
(d) This general principle is known as the Poisson paradigm (or law of rare events), and it provides the theoretical
justification for modeling many rare-event phenomena using the Poisson distribution.
(e) Examples:
i. The number of misprints on a page: each letter has a very small probability of being misprinted, but there are
many letters.
ii. The number of accidents in a large city on a given day: each individual driver has a small accident probability,
and the total number is well-approximated by a Poisson law.
iii. The number of telephone calls arriving at an exchange in a short time interval: each caller acts independently
with small probability in that interval, leading again to a Poisson distribution.

13. Example: Consecutive Heads

(a) A coin is flipped 𝛻 times independently, with each flip landing heads with probability 𝜑.
(b) What is the probability that there exists a string of ℷ consecutive heads?
(c) A first (but flawed) attempt at a Poisson approximation:
i. For 𝜚 = 1, … , 𝛻 ω ℷ + 1, let 𝜛𝜚 denote the event that flips 𝜚, 𝜚 + 1, … , 𝜚 + ℷ ω 1 are all heads.
{𝛻ωℷ+1
ii. Then the desired event is 𝜚=1 𝜛𝜚 , i.e., that at least one of the 𝜛𝜚 occurs.
iii. Since 𝜀 (𝜛𝜚 ) = 𝜑ℷ , one might be tempted to approximate the number of such events by a Poisson distribution
with mean (𝛻 ω ℷ + 1)𝜑ℷ .
iv. However, this reasoning fails because the events 𝜛𝜚 are strongly dependent when the sequences overlap.
v. For instance,
𝜀 (𝜛2 ϖ 𝜛1 ) = 𝜀 ({flips 2, … , ℷ + 1 are heads} ϖ {flips 1, … , ℷ are heads}) = 𝜑, (135)
which is much larger than the unconditional probability 𝜀 (𝜛2 ) = 𝜑ℷ .
vi. Hence the naive Poisson approximation does not hold.
(d) A practical Poisson approximation:
i. The key observation is that a run of ℷ heads can be uniquely identified either
• by a string of ℷ consecutive heads immediately followed by a tail, or
• by the last ℷ flips of the sequence all being heads.
ii. Define events
7𝜚 = {flips 𝜚, … , 𝜚 + ℷ ω 1 are heads and flip 𝜚 + ℷ is a tail}, 𝜚 = 1, … , 𝛻 ω ℷ, (136)
7𝛻ωℷ+1 = {flips 𝛻 ω ℷ + 1, … , 𝛻 are all heads}. (137)
iii. Then
𝜀 (7𝜚 ) = 𝜑ℷ (1 ω 𝜑), 𝜚 = 1, … , 𝛻 ω ℷ, 𝜀 (7𝛻ωℷ+1 ) = 𝜑ℷ . (138)
 25

iv. Each 7𝜚 has a small probability of occurring when 𝜑ℷ is small. Moreover, when 𝜚 ⨋ ℶ:
• if 7𝜚 and 7ℶ involve non-overlapping sequences, then 𝜀 (7𝜚 ϖ 7ℶ ) = 𝜀 (7𝜚 ) (approximately independent);
• if they overlap, then 𝜀 (7𝜚 ϖ 7ℶ ) = 0.
v. In both cases, the dependence is weak enough that the number of events
{
𝛻ωℷ+1
𝜕= 𝛚7𝜚
𝜚=1
can be approximated by a Poisson random variable.
vi. Its mean is
{
𝛻ωℷ+1
E(𝜕) = 𝜀 (7𝜚 ) = (𝛻 ω ℷ)𝜑ℷ (1 ω 𝜑) + 𝜑ℷ . (139)
𝜚=1
vii. Because there is no run of ℷ heads if and only if 𝜕 = 0, we obtain
𝜀 ({no run of ℷ heads}) = 𝜀 (𝜕 = 0) ∂ 6ω(𝛻ωℷ)𝜑 (140)
ℷ (1ω𝜑)ω𝜑ℷ
.
viii. Consequently,
𝜀 ({at least one run of ℷ heads}) ∂ 1 ω 6ω(𝛻ωℷ)𝜑 (141)
ℷ (1ω𝜑)ω𝜑ℷ
.

14. Another important use of the Poisson distribution arises in modeling the occurrence of random events in time.

(a) For example:

• the occurrence of earthquakes,
• the arrival of customers at a bank, post o!ce, or gas station,
• the outbreak of wars, or
• the emission of radioactive particles.
(b) To model such situations, suppose events occur at random points in time, and assume that for some constant 5 > 0,
the following conditions hold:
i. The probability that exactly one event occurs in an interval of length 𝐸 is
5𝐸 + 𝐹(𝐸),
where 𝐹(𝐸) denotes any function such that lim𝐸0 𝐹(𝐸)φ𝐸 = 0. (For example, 𝐸2 = 𝐹(𝐸), but 𝐸 ⨋ 𝐹(𝐸).)
ii. The probability of two or more events occurring in an interval of length 𝐸 is 𝐹(𝐸).
iii. For any finite collection of disjoint intervals, the numbers of events occurring in these intervals are mutually
independent.
(c) Intuitively:
• Assumptions (i) and (ii) state that for very small intervals, the probability of a single event is proportional to
the interval length, while the probability of multiple events is negligible.
• Assumption (iii) states that disjoint time intervals behave independently.

15. Under these assumptions, we now show that the number of events in an interval of length 𝐺 follows a Poisson distribution
with parameter 5𝐺.

(a) Let 𝜕(𝐺) denote the number of events occurring in [0, 𝐺].
(b) Divide [0, 𝐺] into 𝛻 subintervals of equal length 𝐺φ𝛻.
(c) Define ℵℷ = the event that exactly ℷ subintervals each contain 1 event (and the rest contain none), and ⊳ℷ = the
event that 𝜕(𝐺) = ℷ but at least one subinterval contains two or more events.
(d) Clearly,
𝜀 (𝜕(𝐺) = ℷ) = 𝜀 (ℵℷ ) + 𝜀 (⊳ℷ ).
26

(e) By assumption (ii),

𝜀 (⊳ℷ ) ∲ 𝜀 ({𝜚th subinterval contains ∱ 2 events})

{
𝛻

𝜚=1 (142)
𝐹(𝐺φ𝛻)
= 𝛻 ⋛ 𝐹(𝐺φ𝛻) = 𝐺 ⋛  0 as 𝛻  ϱ.
𝐺φ𝛻
(f) For 𝜀 (ℵℷ ), we use the binomial form:
} ⦃⟩ ⟩ ⟪⟪ℷ ⟩ ⟩ ⟪⟪𝛻ωℷ
(143)
𝛻 5𝐺 𝐺 5𝐺 𝐺
𝜀 (ℵℷ ) = +𝐹 1ω +𝐹 .
ℷ 𝛻 𝛻 𝛻 𝛻
(g) As 𝛻  ϱ,
⟩
⟩ ⟪⟪
(144)
5𝐺 𝐺
+𝐹
𝛻  5𝐺,
𝛻 𝛻
so the above expression converges (by the same reasoning as the Poisson limit of the binomial) to
(5𝐺)ℷ ω5𝐺
𝜀 (ℵℷ )  6 . (145)
ℷ!
(h) Since 𝜀 (⊳ℷ )  0, we conclude
(5𝐺)ℷ ω5𝐺
𝜀 (𝜕(𝐺) = ℷ) = 6 , ℷ = 0, 1, 2, … . (146)
ℷ!
16. Hence, under assumptions (i)–(iii), the number of events in any time interval of length 𝐺 is a Poisson random variable with
mean 5𝐺. We then say that events occur according to a Poisson process with rate 5.

17. The rate 5 represents the expected number of events per unit time and must typically be estimated from data.

18. Example: Earthquakes and Waiting Times.

(a) Suppose that earthquakes in the western United States occur according to a Poisson process with rate 5 = 2 per week.
(b) Consider the event 7 = “at least 3 earthquakes occur in the next 2 weeks.” Since 𝜕(2) − Poisson(5⋛2) = Poisson(4),
we have
𝜀 (7) = 𝜀 (𝜕(2) ∱ 3)
= 1 ω 𝜀 (𝜕(2) = 0) ω 𝜀 (𝜕(2) = 1) ω 𝜀 (𝜕(2) = 2)
42 ω4 (147)
= 1 ω 6ω4 ω 46ω4 ω 6
2
= 1 ω 136ω4 .
This illustrates how the Poisson distribution can be used to compute event probabilities over a fixed time window.
(c) Now, let us ask a di"erent question: what is the probability distribution of the waiting time until the next earthquake?
Let 𝜍 denote the time (in weeks) until the first event of the Poisson process.
(d) Observe that 𝜍 > 𝐺 if and only if no events occur in the time interval [0, 𝐺]. Therefore,
𝜀 (𝜍 > 𝐺) = 𝜀 (𝜕(𝐺) = 0) = 6ω5𝐺 , 5 = 2. (148)

(e) Thus, the distribution function of 𝜍 is

⊲ (𝐺) = 𝜀 (𝜍 ∲ 𝐺) = 1 ω 6ω2𝐺 , 𝐺 ∱ 0, (149)
which is exactly the exponential distribution with rate 5 = 2. This connection between the Poisson process and the
exponential distribution is fundamental: the inter-arrival times in a Poisson process are i.i.d. exponential random
variables.

19. Computing Poisson Probabilities:

 27

(a) For a Poisson random variable 𝜍 − Poisson(5), the probabilities satisfy the recursion
50+1 ω5
6
𝜀 (𝜍 = 0 + 1) (0 + 1)!
(150)
5
= = .
𝜀 (𝜍 = 0) 50 ω5 0+1
6
0!
(b) Starting from 𝜀 (𝜍 = 0) = 6ω5 , we can generate all subsequent probabilities:
𝜀 (𝜍 = 1) = 5𝜀 (𝜍 = 0),
5
𝜀 (𝜍 = 2) = 𝜀 (𝜍 = 1),
2 (151)
⋜
5
𝜀 (𝜍 = 0 + 1) = 𝜀 (𝜍 = 0).
0+1
(c) This recursive relationship is extremely useful for computing Poisson probabilities in practice, especially when 5 is
large, as it avoids repeated factorial calculations.

8 OTHER DISCRETE PROBABILITY DISTRIBUTIONS

1. In addition to the Binomial and Poisson distributions, several other discrete probability laws appear frequently in
applications.

2. They can be viewed as natural extensions or variations of the ideas we have already studied:

• Geometric distribution: models the number of trials until the first success; the simplest waiting-time distribution.
• Negative Binomial distribution: generalizes the geometric case to the number of trials until the 𝐴th success.
• Hypergeometric distribution: similar to the Binomial, but models sampling without replacement, leading to
dependent trials.

8.1 The Geometric Random Variable

1. Suppose that independent Bernoulli trials, each having a probability 𝜑, 0 < 𝜑 < 1, of being a success, are performed until
the first success occurs.

2. If we let 𝜍 equal the total number of trials required, then 𝜍 has probability mass function (pmf)
𝜀 (𝜍 = 0) = (1 ω 𝜑)0ω1 𝜑, 0 = 1, 2, … . (152)
This says: the first 0ω1 trials must be failures (probability (1ω𝜑)0ω1 ), and the 0th trial must be the first success (probability
𝜑).

3. Since
ϱ ϱ
{ { 𝜑
𝜀 (𝜍 = 0) = 𝜑 (1 ω 𝜑)0ω1 = = 1, (153)
0=1 0=1
1 ω (1 ω 𝜑)
it follows that, with probability 1, a success will eventually occur.

4. Any random variable 𝜍 whose pmf is given by (152) is called a geometric random variable with parameter 𝜑. Its mean
and variance are
1 1ω𝜑
7(𝜍) = , Var(𝜍) = , (154)
𝜑 𝜑2
which emphasize that the rarer the success (small 𝜑), the longer we expect to wait.
28

(a) Computing the mean. Let 𝐻 = 1 ω 𝜑. Then

ϱ ϱ
{ {
7(𝜍) = 0(1 ω 𝜑) 0ω1
𝜑=𝜑 0𝐻 0ω1 . (155)
0=1 0=1
Notice that
ϱ
{ 𝐻
𝐻0 = , ]𝐻] < 1. (156)
0=1
1ω𝐻
Di"erentiating with respect to 𝐻 gives
ϱ
{ 1
0𝐻 0ω1 = . (157)
0=1
(1 ω 𝐻)2
Hence
1 1
7(𝜍) = 𝜑 ⋛ = . (158)
(1 ω 𝐻)2 𝜑
(b) Computing the second moment. To obtain the variance, it is convenient to compute 7(𝜍(𝜍 + 1)):
ϱ
{
7(𝜍(𝜍 + 1)) = 𝜑 0(0 + 1)𝐻 0ω1 . (159)
0=1
Observe that
ϱ
{ 𝐻2
𝐻 0+1 = , (160)
0=1
1ω𝐻
and by di"erentiating twice with respect to 𝐻,
ϱ
{ 2
0(0 + 1)𝐻 0ω1 = . (161)
0=1
(1 ω 𝐻)3
Therefore,
2 2
7(𝜍(𝜍 + 1)) = 𝜑 ⋛ = 2. (162)
(1 ω 𝐻)3 𝜑
(c) Computing the variance. Recall that
Var(𝜍) = 7(𝜍 2 ) ω (7(𝜍))2 . (163)
Since 7(𝜍 2 ) = 7(𝜍(𝜍 + 1)) ω 7(𝜍), we have
⟩ ⟪ ⟩ ⟪2
2 1 1
Var(𝜍) = 2
ω ω
(164)
𝜑 𝜑 𝜑
1ω𝜑
= .
𝜑2
5. Example (Urn Model):

(a) An urn contains 𝜕 white and 𝐼 black balls. Balls are randomly selected, one at a time, until a black one is obtained.
Assume sampling with replacement.
(b) Let 𝜍 denote the number of draws until the first black ball. Then 𝜍 − Geometric(𝜑) with
𝐼
𝜑= .
𝐼 +𝜕
(c) The pmf is
⟩ ⟪𝛻ω1
(165)
𝜕 𝐼
𝜀 (𝜍 = 𝛻) = (1 ω 𝜑)𝛻ω1 𝜑 = .
𝐼 +𝜕 𝐼 +𝜕
(d) The probability that at least ℷ draws are needed is
𝜀 (𝜍 ∱ ℷ) = (1 ω 𝜑)ℷω1 . (166)
 29

This is called the survival function of the geometric distribution: the chance of surviving ℷ ω 1 consecutive failures
before the first success.
(e) Equivalently, plugging into the urn model:

𝜀 (𝜍 ∱ ℷ) =
ϱ ⟩ ⟪0ω1
𝐼 { 𝜕
𝐼 + 𝜕 0=ℷ 𝐼 + 𝜕
(167)
⟩ ⟪ℷω1
𝜕
= .
𝐼 +𝜕
6. Remark: The geometric distribution is the simplest waiting-time distribution. It measures the number of trials until the
first success. Later, when we consider the negative binomial distribution, we will generalize this idea to the number of
trials until the 𝐴th success.

8.2 The Negative Binomial Random Variable

1. Suppose that independent trials, each having probability 𝜑, 0 < 𝜑 < 1, of being a success are performed until a total of 𝐴
successes is accumulated.

2. If we let 𝜍 equal the total number of trials required, then 𝜍 has probability mass function
} ⦃
0ω1 𝐴
𝜀 (𝜍 = 0) = 𝜑 (1 ω 𝜑)0ω𝐴 , 0 = 𝐴, 𝐴 + 1, … . (168)
𝐴ω1
Why this formula? The last trial (the}0th) must
⦃ be a success (probability 𝜑), and among the first 0 ω 1 trials we must have
0 ω 1 𝐴ω1
exactly 𝐴 ω 1 successes (probability 𝜑 (1 ω 𝜑)0ω𝐴 ). Multiplying gives (168).
𝐴ω1
3. To verify that a total of 𝐴 successes will eventually occur with probability 1, note that
ϱ ϱ } ⦃
{ { 0ω1 𝐴
𝜀 (𝜍 = 0) = 𝜑 (1 ω 𝜑)0ω𝐴 = 1. (169)
0=𝐴 0=𝐴
𝐴 ω 1

4. Alternatively, we can argue probabilistically:

(a) Let 𝜔1 be the number of trials required for the first success, 𝜔2 the additional number of trials after that until the
second success, and so on.
(b) Then 𝜍 = 𝜔1 + ⋝ + 𝜔𝐴 , where each 𝜔𝜚 is a geometric random variable with parameter 𝜑.
(c) Since each 𝜔𝜚 is finite with probability 1, their sum is also finite with probability 1, which confirms (169).

5. Any random variable 𝜍 whose pmf is given by (168) is called a negative binomial random variable with parameters (𝐴, 𝜑).

6. Relationship to the Geometric distribution. When 𝐴 = 1, the negative binomial reduces to the geometric distribution:
𝜀 (𝜍 = 0) = (1 ω 𝜑)0ω1 𝜑, 0 = 1, 2, … .

7. Mean and variance. The expectation and variance of a negative binomial random variable are
𝐴(1 ω 𝜑)
(170)
𝐴
7(𝜍) = , Var(𝜍) = .
𝜑 𝜑2
These formulas agree with intuition: on average it takes 1φ𝜑 trials for one success, so for 𝐴 successes we expect 𝐴φ𝜑 trials
in total; the variance scales similarly.

(a) To derive these results, consider the ℷth moment:

ϱ } ⦃
{ 0ω1 𝐴
E(𝜍 ) =
ℷ
0ℷ
𝜑 (1 ω 𝜑)0ω𝐴 . (171)
0=𝐴
𝐴ω1
30

} ⦃ } ⦃
0ω1
(b) Using the identity 0 , we obtain
0
=𝐴
𝐴ω1 𝐴
ϱ } ⦃
𝐴 { ℷω1 0 𝐴+1
E(𝜍 ) =
ℷ
0 𝜑 (1 ω 𝜑)0ω𝐴 . (172)
𝜑 0=𝐴 𝐴
(c) Letting 3 = 0 + 1 and recognizing that the summation corresponds to another negative binomial distribution with
parameters (𝐴 + 1, 𝜑), we can write
𝐴 ⦅ ⦆
E(𝜍 ℷ ) = E (𝜔 ω 1)ℷω1 , (173)
𝜑
where 𝜔 − NegBin(𝐴 + 1, 𝜑).
(d) For ℷ = 1, we obtain
(174)
𝐴
E(𝜍) = .
𝜑
(e) For ℷ = 2, we have
} ⦃
𝐴+1
(175)
𝐴 𝐴
E(𝜍 2 ) = E(𝜔 ω 1) = ω1 .
𝜑 𝜑 𝜑
(f) Therefore, the variance is
⦅ ⦆2
Var(𝜍) = E(𝜍 2 ) ω E(𝜍)
} ⦃ } ⦃2
𝐴 𝐴+1 𝐴
=
𝜑 𝜑
ω1 ω
𝜑 (176)
𝐴(1 ω 𝜑)
= .
𝜑2
8. Connection to the Binomial distribution:

(a) In a binomial setting, we fix the number of trials 𝛻 and ask: how many successes?
(b) In a negative binomial setting, we fix the number of successes 𝐴 and ask: how many trials?
(c) Thus, the two distributions are natural complements of each other.

9. Example: Competing successes and failures

(a) Suppose we perform independent Bernoulli trials, each resulting in a success with probability 𝜑 and a failure with
probability 1 ω 𝜑.
(b) We ask: what is the probability that 𝐴 successes occur before 𝐽 failures?
(c) Observe that this happens if and only if the 𝐴th success occurs no later than the (𝐴 + 𝐽 ω 1)th trial.
• If the 𝐴th success has already occurred by trial 𝐴 + 𝐽 ω 1, then at most 𝐽 ω 1 failures could have appeared in the
first 𝐴 + 𝐽 ω 1 trials, meaning the 𝐽th failure has not yet occurred.
• Conversely, if 𝐴 successes occur before 𝐽 failures, then the 𝐴th success must have happened by trial 𝐴 + 𝐽 ω 1.
(d) Therefore, letting 𝜍 be a negative binomial random variable with parameters (𝐴, 𝜑) representing the trial number of
the 𝐴th success, the desired probability is
𝐴+𝐽ω1 } ⦃
𝜀 (𝜍 ∲ 𝐴 + 𝐽 ω 1) =
{ 0ω1
𝜑𝐴 (1 ω 𝜑) 0ω𝐴 . (177)
0=𝐴
𝐴 ω 1
(e) In words: we are summing the probabilities of all scenarios where the 𝐴th success occurs on trial 𝐴, 𝐴+1, … , 𝐴+𝐽ω1.
This ensures that 𝐴 successes occur before the 𝐽th failure.

10. Example: The Banach match problem.

(a) A mathematician always carries two matchboxes, one in each pocket, and each time he needs a match, he chooses
randomly from the two pockets with equal probability.
 31

(b) Initially, each matchbox contains 𝜕 matches.

(c) The key subtlety in the problem is this: the mathematician discovers that a box is empty not at the moment he removes
the last match, but at the next time he tries to draw from that box and finds it empty.
(d) We want to find the probability that, at this moment of discovery, the other matchbox contains exactly ℷ matches,
where ℷ = 0, 1, … , 𝜕.
(e) By symmetry, it does not matter which box is discovered empty first. Without loss of generality, let 7 denote the
event that the right-hand matchbox is first discovered to be empty while the left-hand box has ℷ matches remaining.
(f) Event 7 occurs if and only if the (𝜕 + 1)th choice of the right-hand box is made at the (𝜕 + 1 + 𝜕 ω ℷ)th trial
overall. Intuitively, this means: after exactly 𝜕 successful draws from the right box, and (𝜕 ω ℷ) draws from the left
box, the next draw is again from the right box, at which point the right box is found empty.
(g) If we let 𝜍 denote the number of trials needed to accumulate 𝜕 + 1 draws from the right-hand box, then 𝜍 follows
a negative binomial distribution with parameters (𝐴 = 𝜕 + 1, 𝜑 = 1φ2). Thus,
𝜀 (7) = 𝜀 (𝜍 = 𝜕 + 1 + 𝜕 ω ℷ)
} ⦃ ⟩ ⟪2𝜕ωℷ+1
2𝜕 ω ℷ 1 (178)
= .
𝜕 2
(h) By symmetry, the same probability holds if the left-hand box is first discovered empty while the right-hand box
contains ℷ matches. Therefore, the desired probability is
} ⦃ ⟩ ⟪2𝜕ωℷ
2𝜕 ω ℷ 1
2𝜀 (7) = . (179)
𝜕 2

8.3 The Hypergeometric Random Variable

1. Suppose that a sample of size 𝛻 is drawn randomly without replacement from an urn containing 𝜕 balls, of which 𝐽 are
white and 𝜕 ω 𝐽 are black.

2. If 𝜍 denotes the number of white balls selected, then the probability mass function (pmf) is
} ⦃} ⦃
𝐽 𝜕 ω𝐽
0 𝛻ω0
𝜀 (𝜍 = 0) = } ⦃ , 0 = 0, 1, … , 𝛻. (180)
𝜕
𝛻
3. This formula can be understood as follows:
} ⦃
(a) counts the number of ways to choose 0 white balls from the 𝐽 available.
𝐽
0
} ⦃
𝜕 ω𝐽
(b) counts the number of ways to choose the remaining 𝛻 ω 0 balls from the 𝜕 ω 𝐽 black balls.
𝛻ω0
(c) Multiplying gives the number of favorable outcomes (choose 0 white and 𝛻 ω 0 black).
} ⦃
(d) counts the total number of possible samples of size 𝛻 from the urn.
𝜕
𝛻
(e) The ratio is the desired probability.

4. Remark:

(a) Although we have written the support of 𝜍 as 0 = 0, 1, … , 𝛻, not all of these values are feasible. For example, if
𝐽 = 2 (only two white balls exist in the urn), then it is impossible to select 0 = 3 white balls.
(b) In general, 𝜀 (𝜍 = 0) > 0 only if
𝛻 ω (𝜕 ω 𝐽) ∲ 0 ∲ min(𝛻, 𝐽). (181)
32

} ⦃
(c) However, the pmf formula in (180) remains valid because of the convention that = 0 whenever ℷ < 0 or 𝐴 < ℷ.
𝐴
ℷ
This convention makes the formula robust without needing to specify the feasible range separately.

5. Example: Capture–Recapture and MLE.

(a) Suppose an unknown number 𝜕 of animals inhabit a certain region. Ecologists often use the following capture–
recapture experiment to estimate 𝜕:
i. First, 𝐽 animals are caught, marked, and released back into the population.
ii. After su!cient time for mixing, a second catch of size 𝛻 is taken.
iii. Let 𝜍 denote the number of marked animals found in the second sample.
(b) Under the assumptions that (i) the total population remains fixed between the two catches and (ii) each animal has
an equal chance of being caught, 𝜍 follows a hypergeometric distribution:
} ⦃} ⦃
𝐽 𝜕 ω𝐽
𝜚 𝛻ω𝜚
𝜀 (𝜍 = 𝜚) = } ⦃ =ς 𝜀𝜚 (𝜕), 𝜚 = 0, 1, … , min(𝐽, 𝛻). (182)
𝜕
𝛻
(c) If in the second catch we observe 𝜍 = 𝜚, then 𝜀𝜚 (𝜕) is the likelihood of the observed data, viewed as a function of 𝜕.
(d) A natural way to estimate 𝜕 is to choose the value that maximizes this likelihood — the maximum likelihood estimate
(MLE) of 𝜕.
(e) To locate the maximizing 𝜕, consider the ratio
𝜀𝜚 (𝜕) (𝜕 ω 𝐽)(𝜕 ω 𝛻)
𝐴= = . (183)
𝜀𝜚 (𝜕 ω 1) 𝜕(𝜕 ω 𝐽 ω 𝛻 + 𝜚)
(f) The idea is that:
i. If 𝐴 > 1, then 𝜀𝜚 (𝜕) > 𝜀𝜚 (𝜕 ω 1), so the likelihood is still increasing with 𝜕.
ii. If 𝐴 < 1, then 𝜀𝜚 (𝜕) < 𝜀𝜚 (𝜕 ω 1), so the likelihood has started to decrease.
iii. Thus, the maximizer is the largest integer 𝜕 such that 𝐴 ∱ 1.
(g) Solving 𝐴 ∱ 1 gives
𝜕 ∲ (184)
𝐽𝛻
.
𝜚
(h) Therefore, the likelihood 𝜀𝜚 (𝜕) increases up to this point and then decreases, so the MLE is
} ⦃
(185)
𝐽𝛻
𝜕𝐾 = ,
𝜚
the greatest integer not exceeding 𝐽𝛻φ𝜚.
(i) Example (numerical):
i. Suppose 𝐽 = 50 animals are initially captured and marked.
ii. In the second sample of size 𝛻 = 40, we observe 𝜚 = 4 marked animals.
iii. Then the MLE is
} ⦃
50 + 40
𝜕𝐾 = = 500. (186)
4
(j) Remark: This estimate agrees with a simple proportional reasoning: we expect the proportion of marked animals
in the second sample, 𝜚φ𝛻, to roughly match the proportion of marked animals in the population, 𝐽φ𝜕. Solving
𝜚φ𝛻 ∂ 𝐽φ𝜕 also gives 𝜕𝐾 = 𝐽𝛻φ𝜚.

6. Example: Lot Acceptance Sampling

(a) A purchaser of electrical components buys them in lots of size 10.

(b) His inspection policy is to randomly select 3 components from each lot and to accept the lot only if all 3 inspected
components are non-defective.
 33

(c) Suppose that 30% of the lots contain 4 defective components (and hence 6 good ones), while the remaining 70% of
the lots contain only 1 defective component (and 9 good ones). What proportion of lots will be rejected under this
policy?
(d) Let ℵ denote the event that a lot is accepted. By the law of total probability,
𝜀 (ℵ) = 𝜀 (ℵ ϖ lot has 4 defectives) ⋛ 𝜀 (lot has 4 defectives)
(187)
+ 𝜀 (ℵ ϖ lot has 1 defective) ⋛ 𝜀 (lot has 1 defective).
(e) Conditional probabilities:
i. If the lot has 4 defectives and 6 good ones, then the probability that all 3 inspected items are good is
} ⦃} ⦃
4 6
0 3
𝜀 (ℵ ϖ lot has 4 defectives) = } ⦃ . (188)
10
3
ii. If the lot has 1 defective and 9 good ones, then the probability that all 3 inspected items are good is
} ⦃} ⦃
1 9
0 3
𝜀 (ℵ ϖ lot has 1 defective) = } ⦃ . (189)
10
3
(f) Therefore,
} ⦃} ⦃ } ⦃} ⦃
4 6 1 9
0 3 0 3 20 84
𝜀 (ℵ) = } ⦃ + 0.3 + } ⦃ + 0.7 = + 0.3 + + 0.7 = 0.54. (190)
10 10 120 120
3 3
(g) Hence the proportion of lots rejected is 𝜀 (ℵ4 ) = 1 ω 𝜀 (ℵ) = 1 ω 0.54 = 0.46.
(h) In other words, under this sampling policy, approximately 46% of the lots will be rejected.

7. Approximation of the hypergeometric by the binomial.

(a) Recall that if 𝛻 balls are chosen without replacement from an urn of 𝜕 balls, 𝐽 of which are white, then the number
𝜍 of white balls selected follows a hypergeometric distribution:
} ⦃} ⦃
𝐽 𝜕 ω𝐽
0 𝛻ω0
𝜀 (𝜍 = 0) = } ⦃ , 0 = 0, 1, … , 𝛻. (191)
𝜕
𝛻
(b) Now suppose 𝐽 and 𝜕 are very large compared with 𝛻. Intuitively, the di"erence between sampling with and without
replacement should be negligible:
• With replacement: each draw is white with probability 𝜑 = 𝐽φ𝜕.
• Without replacement: the probability changes slightly after each draw, but when 𝜕 is large the change is tiny.
Hence, we expect the hypergeometric distribution to be well approximated by a Binomial(𝛻, 𝜑) with 𝜑 = 𝐽φ𝜕.
34

(c) To make this precise, expand the hypergeometric probability:

} ⦃} ⦃
𝐽 𝜕 ω𝐽
0 𝛻ω0
𝜀 (𝜍 = 0) = } ⦃
𝜕
𝛻
(𝜕 ω 𝐽)! 𝛻!(𝜕 ω 𝛻)!
(192)
𝐽!
= ⋛ ⋛
0!(𝐽 ω 0)! (𝛻 ω 0)!(𝜕 ω 𝐽 ω 𝛻 + 0)! 𝜕!
} ⦃
𝛻 𝐽 𝐽ω1 𝐽ω0+1 (𝜕 ω 𝐽)! (𝜕 ω 𝛻)!
= ⋝ ⋛ ⋛
0 𝜕 𝜕 ω 1 𝜕 ω 0 + 1 (𝜕 ω 𝐽 ω 𝛻 + 0)! (𝜕 ω 0)!
} ⦃
𝛻 𝐽 𝐽ω1 𝐽ω0+1 𝜕 ω𝐽𝜕 ω𝐽ω1 𝜕 ω𝐽ω𝛻+0+1
= ⋝ ⋛ ⋝ .
0 𝜕 𝜕 ω1 𝜕 ω0+1 𝜕 ω0 𝜕 ω0ω1 𝜕 ω𝛻+1
(d) When 𝐽 and 𝜕 are large relative to 𝛻 and 0, each factor can be approximated:
𝐽ωℶ 𝜕 ω𝐽ωℶ 𝜕 ω𝐽
(193)
𝐽
∂ = 𝜑, ∂ = 1 ω 𝜑.
𝜕 ωℶ 𝜕 𝜕 ω0ωℶ 𝜕
(e) Substituting these approximations gives
} ⦃
(194)
𝛻 0
𝜀 (𝜍 = 0) ∂ 𝜑 (1 ω 𝜑)𝛻ω0 ,
0
which is exactly the binomial pmf with parameters (𝛻, 𝜑).

8. Mean and variance of the hypergeometric random variable.

(a) Suppose 𝜍 − Hypergeometric(𝜕, 𝐽, 𝛻), i.e., 𝜍 is the number of white balls in a sample of size 𝛻 drawn without
replacement from an urn of 𝜕 balls containing 𝐽 white and 𝜕 ω 𝐽 black balls.
(b) Formulas. The expectation and variance are
𝐽
7(𝜍) = 𝛻 = 𝛻𝜑,
(195)
𝜕
𝜕 ω𝛻 𝐽
Var(𝜍) = 𝛻𝜑(1 ω 𝜑) , 𝜑= .
𝜕 ω1 𝜕
𝜕 ω𝛻
The extra factor is known as the finite population correction (FPC).
𝜕 ω1
(c) Interpretation. Compared with the binomial variance 𝛻𝜑(1 ω 𝜑), the hypergeometric variance is smaller due to the
negative dependence induced by sampling without replacement. The closer 𝛻 is to 𝜕, the more variability is reduced.
(d) Approximation. When 𝜕 and 𝐽 are large relative to 𝛻, the FPC factor satisfies
𝜕 ω𝛻
∂ 1, (196)
𝜕 ω1
so that 7(𝜍) ∂ 𝛻𝜑 and Var(𝜍) ∂ 𝛻𝜑(1 ω 𝜑), i.e., the hypergeometric distribution is well approximated by the
binomial distribution.
(e) Proof.
i. Recall that if 𝜍 − Hypergeometric(𝜕, 𝐽, 𝛻), then
} ⦃} ⦃
𝐽 𝜕 ω𝐽
0 𝛻ω0
𝜀 (𝜍 = 0) = } ⦃ , 0 = 0, 1, … , 𝛻. (197)
𝜕
𝛻
Hence,
} ⦃} ⦃} ⦃ω1
{
𝛻
{
𝛻
𝐽 𝜕 ω𝐽 𝜕
E(𝜍 ℷ ) = 0ℷ 𝜀 (𝜍 = 0) = 0ℷ . (198)
0=0 0=1
0 𝛻ω0 𝛻
 35

ii. Using the combinatorial identities

} ⦃ } ⦃ } ⦃ } ⦃
𝐽ω1 𝜕 ω1
(199)
𝐽 𝜕
0 =𝐽 , 𝛻 =𝜕 ,
0 0ω1 𝛻 𝛻ω1
we obtain
} ⦃} ⦃} ⦃ω1
𝛻𝐽 { ℷω1 𝐽 ω 1
𝛻
𝜕 ω𝐽 𝜕 ω1
E(𝜍 ℷ ) = 0
𝜕 0=1 0ω1 𝛻ω0 𝛻ω1
} ⦃} ⦃} ⦃ω1 (200)
𝛻𝐽 {
𝛻ω1
ℷω1 𝐽 ω 1 (𝜕 ω 1) ω (𝐽 ω 1) 𝜕 ω1
= (3 + 1) .
𝜕 3=0 3 (𝛻 ω 1) ω 3 𝛻ω1
That is,
𝛻𝐽 ⦅ ⦆
E(𝜍 ℷ ) = E (𝜔 + 1)ℷω1 , (201)
𝜕
where 𝜔 − Hypergeometric(𝜕 ω 1, 𝐽 ω 1, 𝛻 ω 1).
iii. For ℷ = 1, we immediately get
(202)
𝛻𝐽
E(𝜍) = .
𝜕
iv. For ℷ = 2,
} ⦃
(𝛻 ω 1)(𝐽 ω 1)
(203)
𝛻𝐽 𝛻𝐽
E(𝜍 2 ) = E(𝜔 + 1) = +1 .
𝜕 𝜕 𝜕 ω1
v. Now write 𝜑 = 𝐽φ𝜕. Using the identity
𝐽ω1 𝜕𝜑 ω 1 1ω𝜑
= =𝜑ω , (204)
𝜕 ω1 𝜕 ω1 𝜕 ω1
we have
⦅ ⦆2
Var(𝜍) = E(𝜍 2 ) ω E(𝜍)
} ⦃
𝛻𝐽 (𝛻 ω 1)(𝐽 ω 1) 𝛻𝐽
= +1ω
𝜕 𝜕 ω1 𝜕
} }
1ω𝜑
⦃ ⦃ (205)
= 𝛻𝜑 (𝛻 ω 1) 𝜑 ω + 1 ω 𝛻𝜑
𝜕 ω1
⟩ ⟪
𝛻ω1
= 𝛻𝜑(1 ω 𝜑) 1 ω .
𝜕 ω1

V
9 EXPECTED VALUE OF SUMS OF RANDOM VARIABLES

1. One of the most important and useful properties of expectation is its linearity: the expected value of a sum of random
variables equals the sum of their expected values.

2. In this section, we prove this result under the simplifying assumption that the sample space ⨌ of the probability experiment
is finite or countably infinite.

3. For each outcome , ± ⨌, let 𝜑(,) = 𝜀 ({,}) denote its probability, so that
{
𝜑(,) = 1.
,±⨌

4. A random variable 𝜍 is simply a function 𝜍 ς ⨌  ℝ, and we write 𝜍(,) for the value of 𝜍 when outcome , occurs.

5. The expected value of 𝜍 is then defined as a weighted average of its possible values, with weights given by their
probabilities:
Proposition 3. For a random variable 𝜍 on a finite or countably infinite sample space ⨌,
{
E(𝜍) = 𝜍(,) 𝜑(,).
,±⨌
36

(a) Intuitively, this says that E(𝜍) averages the possible values 𝜍(,), each weighted by the probability 𝜑(,) of the
outcome ,.
[
(b) This formula is consistent with the more familiar version E(𝜍) = 0 0 𝜀 (𝜍 = 0), since 𝜀 (𝜍 = 0) is the sum of
𝜑(,) over all , such that 𝜍(,) = 0.
(c) Proof: Let 0𝜚 , 𝜚 ∱ 1 be the distinct values of 𝜍 and for each 𝜚, let ⨌𝜚 be the event that 𝜍 is equal to 0𝜚 . Then
{ { {{ {{ {
E(𝜍) = 0𝜚 𝜀 (𝜍 = 0𝜚 ) = 0𝜚 𝜀 (⨌𝜚 ) = 0𝜚 𝜑(,) = 𝜍(,)𝜑(,) = 𝜍(,)𝜑(,). (206)
𝜚∱1 𝜚∱1 𝜚∱1 ,±⨌𝜚 𝜚∱1 ,±⨌𝜚 ,±⨌

6. If 𝜍 and 𝜔 are random variables, then their sum > = 𝜍 + 𝜔 is also a random variable, defined pointwise by >(,) =
𝜍(,) + 𝜔 (,). The next result shows why linearity of expectation holds:
Proposition 4 (Linearity of Expectation). For random variables 𝜍 and 𝜔 on a finite or countably infinite sample space ⨌,
E(𝜍 + 𝜔 ) = E(𝜍) + E(𝜔 ).

Proof. (a) By definition,

{
E(𝜍 + 𝜔 ) = (𝜍(,) + 𝜔 (,)) 𝜑(,). (207)
,±⨌

(b) Distributing and rearranging gives

{ {
E(𝜍 + 𝜔 ) = 𝜍(,) 𝜑(,) + 𝜔 (,) 𝜑(,) = E(𝜍) + E(𝜔 ). (208)
,±⨌ ,±⨌

7. The same argument extends immediately to any finite sum of random variables:
⌋ 𝛻 ⌉
{ {𝛻
E 𝜍𝜚 = E(𝜍𝜚 ). (209)
𝜚=1 𝜚=1

8. Example:

(a) Suppose that two independent flips of a coin are made, each coming up heads with probability 𝜑.
(b) Let 𝜍 denote the total number of heads obtained. Equivalently, if 𝜍1 and 𝜍2 are the indicator random variables of
a head on the first and second flip, then 𝜍 = 𝜍1 + 𝜍2 .
(c) Method 1: Using the distribution of 𝜍. The possible values of 𝜍 are 0, 1, 2 with probabilities
𝜀 (𝜍 = 0) = (1 ω 𝜑)2 , 𝜀 (𝜍 = 1) = 2𝜑(1 ω 𝜑), 𝜀 (𝜍 = 2) = 𝜑2 . (210)
Therefore,
E(𝜍) = 0 ⋛ (1 ω 𝜑)2 + 1 ⋛ 2𝜑(1 ω 𝜑) + 2 ⋛ 𝜑2 = 2𝜑. (211)

(d) Method 2: Using the definition over the sample space. The sample space is {(𝜛, 𝜛), (𝜛, 𝜗 ), (𝜗 , 𝜛), (𝜗 , 𝜗 )}. Then
{
E(𝜍) = 𝜍(,) 𝜑(,) = 2𝜑2 + 1 ⋛ 𝜑(1 ω 𝜑) + 1 ⋛ (1 ω 𝜑)𝜑 + 0 ⋛ (1 ω 𝜑)2 = 2𝜑. (212)
,±⨌

(e) Connection to linearity. Since 𝜍 = 𝜍1 + 𝜍2 and each 𝜍𝜚 − Bernoulli(𝜑) with expectation 𝜑, linearity of expectation
immediately gives
E(𝜍) = E(𝜍1 ) + E(𝜍2 ) = 𝜑 + 𝜑 = 2𝜑, (213)
in agreement with both methods above.

9. Example:

(a) Find the expected total number of successes in 𝛻 trials when the 𝜚th trial is a success with probability 𝜑𝜚 , 𝜚 = 1, … , 𝛻.
 37

(b) Let 𝜍 denote the total number of successes and define

)
1, if trial 𝜚 is a success,
𝜍𝜚 = (214)
0, otherwise.
Then each 𝜍𝜚 is a Bernoulli random variable with E(𝜍𝜚 ) = 𝜑𝜚 .
(c) Clearly,
{
𝛻
𝜍= 𝜍𝜚 , (215)
𝜚=1
since the total number of successes is the sum of the individual indicators.
(d) By linearity of expectation,
{
𝛻
{
𝛻
E(𝜍) = E(𝜍𝜚 ) = 𝜑𝜚 . (216)
𝜚=1 𝜚=1

10. Remark:

(a) The result in (209) does not require the random variables to be independent.
(b) Even when the random variables are dependent, the expectation of their sum still equals the sum of their expectations.
(c) For instance, consider the hypergeometric random variable with 𝜕 balls, of which 𝐽 are white.
(d) This can be interpreted as the number of successes in 𝛻 draws, where trial 𝜚 is defined as a success if the 𝜚th ball
selected is white.
(e) Since each draw is equally likely to be any of the 𝜕 balls, the probability that the 𝜚th draw is white is 𝐽φ𝜕.
(f) Thus the hypergeometric distribution can be viewed as the total number of successes in 𝛻 trials, each with success
probability 𝜑 = 𝐽φ𝜕.
(g) Although these trials are not independent (because sampling is without replacement), linearity of expectation still
applies, and so
(217)
𝐽
E(𝜍) = 𝛻 ⋛ .
𝜕
(h) This illustrates an important principle: expectations add, regardless of dependence.

11. Example:

(a) We now derive an expression for the variance of the hypergeometric random variable 𝜍 with parameters (𝜕, 𝐽, 𝛻).
(b) Define indicator variables
)
1, if the 𝜚th draw is white,
𝜍𝜚 = (218)
0, otherwise,
[𝛻
so that 𝜍 = 𝜚=1
𝜍𝜚 .
(c) Let 𝜑 = 𝐽φ𝜕. Clearly,
E(𝜍𝜚 ) = E(𝜍𝜚2 ) = 𝜑, 𝜚 = 1, 2, … , 𝛻, (219)
because each draw is equally likely to be any of the 𝜕 balls.
(d) Expanding 𝜍 2 gives
⌋⌋ ⌉⌋ 𝛻 ⌉⌉
{
𝛻
{
2
E(𝜍 ) = E 𝜍𝜚 𝜍ℶ
𝜚=1 ℶ=1
{
𝛻
{
𝛻
{
= E(𝜍𝜚2 ) + E(𝜍𝜚 𝜍ℶ ) (220)
𝜚=1 𝜚=1 ℶ⨋𝜚
{
𝛻
{
= 𝛻𝜑 + E(𝜍𝜚 𝜍ℶ ).
𝜚=1 ℶ⨋𝜚
38

(e) Now, 𝜍𝜚 𝜍ℶ = 1 only if both the 𝜚th and ℶth draws are white. Thus
E(𝜍𝜚 𝜍ℶ ) = 𝜀 (𝜍𝜚 = 1, 𝜍ℶ = 1) = 𝜀 (𝜍𝜚 = 1 ϖ 𝜍ℶ = 1)𝜀 (𝜍ℶ = 1). (221)

(f) Since 𝜀 (𝜍ℶ = 1) = 𝐽φ𝜕 = 𝜑 and, conditional on 𝜍ℶ = 1, the chance the 𝜚th draw is also white is (𝐽 ω 1)φ(𝜕 ω 1),
we obtain
} ⦃
𝐽ω1 𝐽 1ω𝜑
E(𝜍𝜚 𝜍ℶ ) = ⋛ = 𝜑ω 𝜑. (222)
𝜕 ω1 𝜕 𝜕 ω1
(g) Substituting back,
} ⦃
1ω𝜑
E(𝜍 2 ) = 𝛻𝜑 + 𝛻(𝛻 ω 1) 𝜑 ω 𝜑. (223)
𝜕 ω1
(h) Therefore,
Var(𝜍) = E(𝜍 2 ) ω (E(𝜍))2
} ⦃
1ω𝜑
= 𝛻𝜑 + 𝛻(𝛻 ω 1) 𝜑 ω 𝜑 ω (𝛻𝜑)2 (224)
𝜕 ω1
⟩ ⟪
𝛻ω1
= 𝛻𝜑(1 ω 𝜑) 1 ω .
𝜕 ω1

10 PROPERTIES OF THE CUMULATIVE DISTRIBUTION FUNCTION

1. Recall that the cumulative distribution function (cdf) ⊲ of a random variable 𝜍 is defined by
⊲ (0) = 𝜀 (𝜍 ∲ 0). (225)

2. The cdf ⊲ has the following fundamental properties:

(a) ⊲ is non-decreasing: if 1 < 2, then ⊲ (1) ∲ ⊲ (2).

(b) lim0ϱ ⊲ (0) = 1.
(c) lim0ωϱ ⊲ (0) = 0.
(d) ⊲ is right-continuous: for any 1 and any decreasing sequence 1𝛻  1,
lim ⊲ (1𝛻 ) = ⊲ (1). (226)
𝛻ϱ

3. These properties follow from the continuity of probability measures:

Proof. (a) Since {𝜍 ∲ 1} 𝐿 {𝜍 ∲ 2} when 1 < 2, we have ⊲ (1) ∲ ⊲ (2).

(b) Let 2𝛻  ϱ. Then the events {𝜍 ∲ 2𝛻 } form an increasing sequence with union {𝜍 < ϱ} = ⨌. By continuity of
probability,
lim 𝜀 (𝜍 ∲ 2𝛻 ) = 𝜀 (𝜍 < ϱ) = 1. (227)

(c) Similarly, if 2𝛻  ωϱ, the events {𝜍 ∲ 2𝛻 } decrease to the empty set, giving
𝛻ϱ

lim 𝜀 (𝜍 ∲ 2𝛻 ) = 0. (228)

(d) If 2𝛻  2, then {𝜍 ∲ 2𝛻 } decrease to {𝜍 ∲ 2}, so

𝛻ϱ

lim 𝜀 (𝜍 ∲ 2𝛻 ) = 𝜀 (𝜍 ∲ 2) = ⊲ (2). (229)

𝛻ϱ

4. All probability statements about 𝜍 can be expressed in terms of ⊲ . For instance:

𝜀 (1 < 𝜍 ∲ 2) = ⊲ (2) ω ⊲ (1), 1 < 2. (230)
 39

To compute 𝜀 (𝜍 < 2), note that

𝜀 (𝜍 < 2) = lim 𝜀 𝜍 ∲ 2 ω
⟩ ⟪ ⟩ ⟪
1 1
= lim ⊲ 2 ω . (231)
𝛻ϱ 𝛻 𝛻ϱ 𝛻
5. Example:

(a) Suppose the cdf of 𝜍 is

❳0, 0 < 0,
/ 0 , 0 ∲ 0 < 1,
/
/2
1 ∲ 0 < 2,
/2
⊲ (0) = \ , (232)
/3
/ 11 , 2 ∲ 0 < 3,
/ 12
0 ∱ 3.
/
(1,
(b) We can compute various probabilities:
⟩ ⟪
1 11
𝜀 (𝜍 < 3) = lim ⊲ 3 ω = ,
𝛻ϱ 𝛻 ⟩ 12⟪
1 2 1 1
𝜀 (𝜍 = 1) = ⊲ (1) ω lim ⊲ 1 ω = ω = ,
⟩ ⟪
𝛻ϱ
⟩ ⟪ 𝛻 3 2 6 (233)
1 1 1 3
𝜀 𝜍> =1ω⊲ =1ω = ,
2 2 4 4
𝜀 (2 < 𝜍 ∲ 4) = ⊲ (4) ω ⊲ (2) = 1 ω
11 1
= .
12 12

15: ) (
11 HOMEWORK
(
)
450
:
:
弄
1. A box contains 5 red and 5 blue marbles. Two marbles are withdrawn randomly. If they are the same color, then you win
$1.10; if the are di"erent colors, then you lose $1.00. Calculate

(a) the expected value of the amount you win;

(b) the variance of the amount you win.

2. If E(𝜍) = 1 and Var(𝜍) = 5. find 5

=
ux )
:-
6 1
以 岩 = 幾 :井
(a) E((2 + 𝜍)2 );EC 4 + X +
4 Xリン 4+ 4f6
=
14
(b) Var(4 + 3𝜍). 45
'

3 x5 =

3. Suppose that, in flight, airplane engines will fail with probability 1ω𝜑, independently from engine to engine. If an airplane
needs a majority of its engines operative to complete a successful flight, for what values of 𝜑 is a 5-engine plane preferable $p3
to a 3-engine plane? { X ≥ 2 )= engineP x } 3)
{3 ) p
3
给 |p ㎡ 1 1) tp
5
>= ” · p
(& ) 4 y /
(
(
p
-
5
㎡
{ - 3 p 2
p . -
cr )
:
+ ( + -

3 engineP P3 10
Cip ~+
“
5
p
4 sp
) fp
52 Po

4. People enter a gambling casino at a rate of 1 every 2 minutes. P 5 P 3 3p Crpc悲

㎡
-
=

.5

℃=
λ U5 2

(a) What is the probability that no one enters between 12:00 and 12:05? 5 -
e 1f
C

(b) What is the probability that at least 4 people enter the casino during that time? .
PCX 5+ 湾 )
:3
5. A total of 2𝛻 people, consisting of 𝛻 married couples, are randomly seated (all possible orderings being equally likely) at
a round table. Let 𝑀𝜚 denote the event that the members of couple 𝜚 are seated next to each other, 𝜚 = 1, ∓; ≨𝐹𝐺,, 𝛻.
!
yl "
=alerncj
2m

csnum =
zt2 S 2 n ☆) !
(a) Find 𝜀 (𝑀𝜚 ).
"
4
"
=
(b) For ℶ ⨋ 𝜚, find 𝜀 (𝑀ℶ ϖ 𝑀𝜚 ). 幽質箭
sm - =前幽 最

)
(c) Approximate the probability, for 𝛻 large, that there are no married couples who are seated next to each other.
儿
二
二
一
一
40

6. If 𝜍 has distribution ⊲ , what is the distribution function of 6𝜍 ?

7. Let 𝜍 be such that 𝜀 (𝜍 = 1) = 𝜑 = 1 ω 𝜀 (𝜍 = ω1). Find 4 ⨋ 1 such that E(4 𝜍 ) = 1.

8. Show that 𝜍 is a Poisson random variable with parameter 5, then of

value
E(𝜍 ) = 5E((𝜍 + 1)
𝛻 𝛻ω1
). Expect
9. If 𝜍 is a geometric random variable, show that
𝜀 (𝜍 = 𝛻 + ℷ ϖ 𝜍 > 𝛻) = 𝜀 (𝜍 = ℷ).

10. For a hypergeometric random variable, determine

𝜀 (𝜍 = ℷ = 1)φ𝜀 (𝜍 = ℷ).

References

1. Sheldon Ross. A First Course in Probability. 8th edition, Pearson, Upper Saddle River, NJ, 2010. (See especially Chapter 4
on random variables.)
 ex exxx
=
π
of X" is 區 xnl 不梁
8. Expected value ETX1 λ!
=

*
(x 1) !
+
=λ
X 區 ×^
州
州 M州 x
λ
= x * ,
=
E [ × 11 ]
X :
^ %
7 .
let Y = C
loyy Xlogo
PC! ) p
≠ 影 % 品箱 If ECM 1 then
:

=
1p
P( 1 ) =

1
ECCEECP + OCH P )
- =

61 - p ) 1 ( 同 XC )
℃CEp +
=

㎡
CtS 1 p ) : 0
PC
.
-

| t 1 -

4 P ( tP ) | ± rp )
= .

C= 1 or
p ( C# 1 ) C “ 答 CP 井
6 Let keX P Y≤ Y} PEXEY }
.

, Fycy ) = =

Case 1 /
:
zo since expo for all real X ,
the event Lex <Y] is impossible
FyY に Ple ×≤ Y } = 0
for yEo
Case 2:
y10 , take In ex
of =
Y e
x
<
YE ,
X *
enY
( y) P { e ey } = { x < eny } = ( lny ) for Y 70
*

Fy = P F

ifyc
FyY に { 微 yl ify 7
ntkank X > n) PCx mek )
P(X
-
=

9 P ( X ntkl X 2n )
= Ʃ
÷

P(X (n) p - P(
. )
X

Cr-pyutk 1
) )
(+H
+

p ( 1- P P
(l - p)k
-
=
- n
PCr 61 p3 )
=

(l -p(n p PSx
=

()
Crplitp
= =

に
-

= (
Hi - p
1

公 6“
N
↑ )
5
) 公)
/
1
10 P(X k+ 1)/P(x k)
=
= = =

.
[ )
!
M! ) I- M :
( ) : (N ! 1 NM
. - )! k ! /I-() ! (n -4 ! (N -

M -n+ k) !
=
ntk + 1 ) !
=

Mi
(ktll ! ( 1- - 1 ) :( nt 11 ! ( 11-M
11- 17 :
M
k
- -

K! M -

K] : ): ( N -M - k) :
「
 i 州 utk 1 )
-
#
=

+
《