0% found this document useful (0 votes)

22 views119 pages

Probability and Statistics Lecture Notes

Q: How does the document elaborate on finding the joint m.g.f. of a random vector?

The joint moment generating function (m.g.f.) of a random vector X = (X1, X2, ..., Xp) is defined over a set A where each element t in A ensures the expectation E(exp(Σ(ti*Xi))) is finite. The function MX on this set is described as the expectation of the exponential sum, and it can be used to derive the independence of the variables when the set A allows factorization into individual m.g.f.s .

Q: What solution is given for a problem involving a continuous random variable with mixed types of p.d.f.?

For a continuous random variable with a mixed p.d.f. of the form f(x) = x² for 0 < x < 1 and f(x) = ce^{-x} for x ≥ 1, c is calculated such that the integral over the entire domain equals 1. Calculations for the probability over specific intervals are done by integrating the p.d.f. over those intervals, determining the support, and deriving the distribution function by integrating the p.d.f. .

Q: In the context of generating functions, what condition implies independence of random variables?

The independence of random variables X1, X2, ..., Xp is implied if their joint moment generating function MX(t) equals the product of the individual moment generating functions MXi(ti) for all t in the domain A. Conversely, if the joint m.g.f. can be expressed as such a product over A, then the random variables are independent .

Q: Describe the conditional density function for a continuous random vector as presented in the document.

For a continuous random vector X = (X1, X2), the conditional density function fX2|X1(x|x1) is given by the ratio of the joint p.d.f. fX1,X2(x1, x2) to the marginal p.d.f. fX1(x1), provided fX1(x1) > 0. This function describes the probability density of X2 given that X1 is equal to some value x1 .

Uploaded by

jsmrkrohan555

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

22 views119 pages

Probability and Statistics Lecture Notes

Uploaded by

jsmrkrohan555

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

MAL403/IC105/MA503: Probability and Statistics 2025-26-M

Lecture 0: Lecture Notes

Instructor: Dr. Kuldeep Kumar Kataria Scribe:

0.1. Introduction

Example 0.1. The production manager of a bulb manufacturing company wishes to study the effect of new manufac-
turing process on the lifetimes of bulbs produced through it.
Here the population under study is the following:
P: Collection of lifetimes of all electric bulbs produced using new manufacturing process.
In most practical situation P is generally large (e.g. collection of lifetimes of all electric bulbs that would be produced
using new manufacturing process) and it is not (due to time/cost contraints) to get complete information about P.
Thus a representative sample (a sample that in certain sense is a true representative of the population) is taken from P
and using this representative sample inferences regarding various population characteristics of P (such as population
mean, population variance etc.) are made. Note that the sample contains only partial information about P and the
goal is to make inferences about various population characteristics based on partial information in the sample drawn
from P.
X: Lifetime of a typical electric bulbs manufactured using new manufacturing process (a typical element of P).
X is random (called a random variable) and its value varies across P according to some law.

Probability Theory: A mathematical tool for modelling uncertainty (e.g. to describe the law according to which
values of X vary across P).
Statistics: Concerns with procedures for analyzing data (sample) and drawing inferences about various characteristics
of the population P.
For understanding of statistics, one must have a sound background in probability theory.
The only way to collect information about any random phenomenon is to perform experiments (e.g. selecting a set of
bulbs manufactured by the new manufacturing process and putting them on test for measuring their lifetimes). Each
experiment terminates in an outcome which cannot be predicted in advance prior to the performance of experiment
(e.g. lifetimes of the bulbs put on test cannot be predicted before they are put on test).
Definition 0.2 (Random Experiment). A random experiment is an experiment in which
(a) all possible outcomes of the experiment are known in advance,
(b) outcome of a particular performance (trial) of the experiment cannot be predicted in advance,
(c) the experiment can be repeated under identical conditions.

We will generally denote a random experiment by E.

0-1
Lecture 0 Lecture Notes 0-2

Definition 0.3 (Sample Space). The collection of all possible outcomes of a random experiment is called its sample
space. A sample space will usually be denoted by Ω.
Example 0.4. (i) E: Tossing a coin once. Sample space Ω = {H, T }, where H: Heads and T : Tails.
(ii) E: Throwing a die. Sample space Ω = {1, 2, 3, 4, 5, 6}.
(iii) E: Birth of a child. Sample space Ω = {M, F }. If we consider his/her weight then Ω = (0, 7).
(iv) E: Age at the death of a person. Sample space Ω = (0, 120).
(v) E: Putting an electric bulbs produced by new manufacturing process into test and measuring its lifetime. Sample
space Ω = [0, ∞).
(vi) E: Throwing two dice. Sample space

Ω = {(1, 1), (1, 2), . . . , (1, 6), (2, 1), (2, 2), . . . , (2, 6), . . . , (6, 1), (6, 2), . . . , (6, 6)}
= {(i, j) : i, j ∈ {1, 2, . . . , 6}}.

(vii) E: Putting two electric bulbs produced by new manufacturing process into test and measuring their lifetimes.
Sample space Ω = {(x1 , x2 ) : x1 ≥ 0, x2 ≥ 0} = [0, ∞) × [0, ∞).
(viii) E: Casting one red die and white die. Sample space

Ω = {(r, w) : r is number of spots on the red die and w is number of spots on the white die }
= {(1, 1), (1, 2), . . . , (1, 6), (2, 1), (2, 2), . . . , (2, 6), . . . , (6, 1), (6, 2), . . . , (6, 6)}
= {(i, j) : i, j ∈ {1, 2, . . . , 6}}
= {1, 2, . . . , 6} × {1, 2, . . . , 6} → has 36 elements.

Definition 0.5 (Event). An event is any subset of the sample space. If the outcome of a random experiment is a member
of the set E ⊆ Ω, we say that event E has occured.
Example 0.6. In Example 0.4 (vi), A = {(1, 5), (6, 2), (2, 2)} is an event. Also, in Example 0.4 (vii), A = {(x1 , x2 ) :
x1 ≤ 6, x2 ≥ 8} = [0, 6] × [8, ∞) may be an event.

Impossible Event: φ.
Sure Event: Ω.
Sn
Exhaustive Events: If i=1 Ai = Ω then we call A1 , A2 , . . . , An to be exhaustive events.
Mutually Exclusive Events: If A ∩ B = φ then A and B are called mutually exclusive events i.e., happening or
occcurrence of one of them excludes the possiblity of occurrence of other.
Pairwise Disjoint Events: Let A1 , A2 , . . . be events such that Ai ∩ Aj = φ, i 6= j. Then, we say that A1 , A2 , . . .
are pairwise disjoint or mutually exclusive.
Let A and B be two events. Then,
(i) A ∪ B → occurrence of at least one of the event A and B.
S∞
(ii) i=1 Ai → occurrence of at least one Ai , i = 1, 2, . . . , n.
(iii) A ∩ B → simultaneous occurrence of A and B.
T∞
(iv) i=1 Ai → simultaneous occurrence of Ai , i = 1, 2, . . . , n.
(v) Ac → not happenning of A.
Lecture 0 Lecture Notes 0-3

(vi) A − B → happenning of A not B. Thus, A − B = A ∩ B c .

Generally, we are interested in specific subsets of Ω, called event. So the event space (events under consideration) F
is a subset of power set of Ω.
So the event space is F ⊆ P(Ω). Here, P(Ω) denotes the power set of Ω. In many situations and for almost all practical
purposes F = P(Ω).
The choice of F is an important one:
(i) If Ω contains atmost a countable number of points we can always take F to be the P(Ω). (This is certainly a σ-field).
In this case each point set is a member of F and is the fundamental object of interest. Every subset of Ω is an event.
(ii) If Ω = R or any interval then Ω is uncountable. In this case we would like to consider all one point subsets of Ω,
all intervals (closed, open or semi-closed) to be events. We consider the Borel σ-field B generated by the class of all
semi closed intervals (a, b], which is a σ-field in R.
We say that the event space F ⊆ P(Ω) contains all subsets of Ω actually encountered in ordinary analysis and proba-
bility. It is large enough for all practical purposes.
Definition 0.7 (σ-Field/ σ-Algebra). A class of subset F of the sample space Ω is called a σ-field if it satisfies the
following conditions:
(a) Ω ∈ F,
(b) If A ∈ F then Ac ∈ F,
S∞
(c) If A1 , A2 , · · · ∈ F then i=1 Ai ∈ F.

The algebra of set theory is applicable in probability theory. Probability is a measure of uncertainty. We are interested
in quantifying uncertainty associated with various outcomes of a random experiment by assigning probability to these
outcomes.
Here, we will not discuss how probabilities are assigned (which is a part of probability modelling) rather we will
discuss properties of a probability as a measure.

0.2. Probability Measure

Recall that E denotes a random experiment, Ω denotes the sample space of E and F denotes event space. For all
practical purposes one may take F = P(Ω).
A set function is a function whose domain is a collection of sets (called a class of sets).
Definition 0.8 (Probability Function or Probability Measure). A probability function (or probability measure) is a real
valued set function, defined on the event space F satisfying the following axioms:
(a) P (Ω) = 1 (certainty),
(b) P (A) ≥ 0 ∀ A ∈ F (positivity),
(c) If A1 , A2 ∈ F be mutually exclusive/disjoint sets (i.e. A1 ∩ A2 = φ, the empty set) then

P (A1 ∪ A2 ) = P (A1 ) + P (A2 ).

Lecture 0 Lecture Notes 0-4

More generally, if {An }n≥1 is a sequence of mutually exclusive (disjoint) sets in F i.e., Ai ∩ Aj = φ, i 6= j, then
∞ ∞
!
[ X
P Ai = P (Ai ) (countable additivity).
i=1 i=1

We call P (A) the probability of event A. The triplet (Ω, F, P ) is called probability space.
Remark 0.9. Axiom (b) and (c) are desirable for any measure (such as area, volume, probability etc.). Since the
sample space Ω consists of all possible outcomes its occurrence is certain (100% chance of occurrence) and therefore
Axiom (a) (P (Ω) = 1) is also reasonable.

Elementary Properties of Probability Function/ Measure:

Let (Ω, F, P ) be a probability space.
(P1) P (φ) = 0.

∞
S
Proof. Let A1 = Ω and Ai = φ, i = 2, 3, . . . Also, we have A1 = Ai , Ai ∩ Aj = φ, ∀ i 6= j. Therefore,
i=1

∞
!
[
P (Ω) = P Ai
i=1
∞
X
=⇒ 1 = P (Ai ), (Axioms (a) and (c))
i=1
n
X
=⇒ 1 = lim P (Ai )
n→∞
i=1
=⇒ 1 = lim [P (Ω) + (n − 1)P (φ)]
n→∞
=⇒ 1 = 1 + lim [(n − 1)P (φ)]
n→∞
=⇒ P (φ) = 0.

This completes the proof.

n
n
(P2) For some natural number n, let A1 , A2 , . . . , An ∈ F be mutually exclusive. Then, P
S P
Ai = P (Ai ).
i=1 i=1

n
S ∞
S
Proof. Let Ai = φ, i = n + 1, n + 2, . . . Then Ai ∩ Aj = φ, ∀ i 6= j and Ai = Ai . This implies
i=1 i=1

∞
n
! !
[ [
P Ai =P Ai
i=1 i=1
∞
X
= P (Ai ), (Axioms (c))
i=1
Xn
= P (Ai ), (P (Ai ) = P (φ) = 0, ∀ i = n + 1, n + 2, . . .).
i=1

This completes the proof.

Lecture 0 Lecture Notes 0-5

(P3) For all A ∈ F, 0 ≤ P (A) ≤ 1 and P (Ac ) = 1 − P (A).

Proof. Note that Ω = A ∪ Ac and A ∩ Ac = φ. Therefore,

1 = P (Ω) = P (A ∪ Ac ) = P (A) + P (Ac ) ≥ P (A), (using Axioms (a), (b) and (P2)).

Thus, 0 ≤ P (A) ≤ 1 and P (Ac ) = 1 − P (A).

(P4) Let A1 , A2 ∈ F be such that A1 ⊆ A2 . Then, P (A2 − A1 ) = P (A2 ) − P (A1 ) and P (A1 ) ≤ P (A2 ).

Proof. A2 = A1 ∪ (A2 − A1 ) and A1 ∩ (A2 − A1 ) = φ. Thus,

P (A2 ) = P (A1 ) + P (A2 − A1 ) =⇒ P (A2 − A1 ) = P (A2 ) − P (A1 ).

By Axiom (b), we have P (A2 − A1 ) ≥ 0 =⇒ P (A2 ) ≥ P (A1 ), that is, P (·) is monotone.

(P5) Let A1 , A2 ∈ F. Then,

P (A1 ∪ A2 ) = P (A1 ) + P (A2 ) − P (A1 ∩ A2 ) (Inclusion-Exclusion principle for two events).

Proof. Note that A1 ∪ A2 = A1 ∪ (A2 − A1 ) and A1 ∩ (A2 − A1 ) = φ. This implies

P (A1 ∪ A2 ) = P (A1 ∪ (A2 − A1 )) = P (A1 ) + P (A2 − A1 ) (using (P2)). (0.1)

Also, we have
(A1 ∩ A2 ) ∩ (A2 − A1 ) = φ and A2 = (A1 ∩ A2 ) ∪ (A2 − A1 ),
which implies

P (A2 ) = P (A1 ∩ A2 ) + P (A2 − A1 )

=⇒ P (A2 − A1 ) = P (A2 ) − P (A1 ∩ A2 ). (0.2)

Using (0.2) in (0.1), we get

P (A1 ∪ A2 ) = P (A1 ) + P (A2 ) − P (A1 ∩ A2 ).
This completes the proof.
Remark 0.10. (a) If P (A) = 0 and B ⊆ A, then P (B) = 0 (using (P4) and Axiom (b)).
Similarly, if P (C) = 1 and C ⊆ D, then P (D) = 1 (using (P3) and (P4)).
(b) Exercise: If P (D) = 1, then P (A) = P (A ∩ D), ∀A ∈ F.
Similarly, if P (D) = 0, then P (A) = P (A ∩ Dc ), ∀A ∈ F.
(c) Let A1 , A2 ∈ F. Then, using (P5) and Axiom (b), we get

P (A1 ∪ A2 ) ≤ P (A1 ) + P (A2 ) (Boole’s inequality for two events).

(d) Let A1 , A2 ∈ F. Then, using (P3), (P5) and Axiom (b), we get

P (A1 ∩ A2 ) ≥ max {P (A1 ) + P (A2 ) − 1, 0} (Bonferroni’s inequality for two events).

Lecture 0 Lecture Notes 0-6

Theorem 0.11 (Inclusion-Exclusion Principle). For A1 , A2 , . . . , Ak ∈ F, (k ≥ 2 is an integer), let

k
X
p1,k = P (A1 ) + P (A2 ) + · · · + P (Ak ) = P (Ai )
i=1
p2,k = P (A1 ∩ A2 ) + P (A1 ∩ A3 ) + · · · + P (A1 ∩ Ak ) + P (A2 ∩ A3 ) + · · · + P (A2 ∩ Ak ) + · · · + P (Ak−1 ∩ Ak )
XX
= P (Ai ∩ Aj )
1≤i<j≤k

(sum of probabilities of all possible intersections involving 2 events out of the k events A1 , . . . , Ak )
..
.
XX X
pi,k = ··· P (Aj1 ∩ Aj2 ∩ · · · ∩ Aji )
1≤j1 <j2 <···<ji ≤k

(sum of probabilities of all possible intersections involving i events out of k events A1 , . . . , Ak , i = 1, . . . , k).

Then, !
k
[
P Ai = p1,k − p2,k + p3,k − p4,k + · · · + (−1)k−1 pk,k .
i=1

Proof. Note that, for k = 2, p1,2 = P (A1 ) + P (A2 ), p2,2 = P (A1 ∩ A2 ) and

P (A1 ∪ A2 ) = P (A1 ) + P (A2 ) − P (A1 ∩ A2 ) = p1,2 − p2,2 .

Thus the result is true for k = 2. Now suppose that the result is true for k = 2, 3, . . . , m, that is,
k
!
[
P Ai = p1,k − p2,k + p3,k − p4,k + · · · + (−1)k−1 pk,k ∀ k = 2, 3, . . . , m.
i=1

Then,
m+1
! m
! !
[ [ [
P Ai =P Ai Am+1
i=1 i=1
m
! m
! !
[ [ \
=P Ai + P (Am+1 ) − P Ai Am+1 , (using result for k = 2)
i=1 i=1
m m
! m
X [ [
= (−1)j−1 pj,m + P (Am+1 ) − P (Ai ∩ Am+1 ) , (using the result for k = m on Ai )
j=1 i=1 i=1
Xm m
X m
[
= (−1)j−1 pj,m + P (Am+1 ) − (−1)j−1 tj,m , (using the result for k = m on (Ai ∩ Am+1 )),
j=1 j=1 i=1

where
m
X
t1,m = P (Ai ∩ Am+1 )
i=1
XX
t2,m = P (Ai ∩ Aj ∩ Am+1 )
1≤i<j≤m
XX X
tj,k = ··· P (Ai1 ∩ Ai2 ∩ · · · ∩ Aij ∩ Am+1 ), j = 1, 2, . . . , m
1≤i1 <i2 <···<ij ≤m
Lecture 0 Lecture Notes 0-7

tm,m = P (A1 ∩ A2 ∩ · · · ∩ Am ∩ Am+1 )

Therefore,
m+1
!
[
P Ai = (p1,m + P (Am+1 )) − (p2,m + t1,m ) + (p3,m + t2,m ) + · · · + (−1)m−1 (pm,m + tm−1,m ) + (−1)m tm,m
i=1
= p1,m+1 − p2,m+1 + p3,m+1 + · · · + (−1)m−1 pm,m+1 + (−1)m pm+1,m+1 ,

as
m
X
p1,m + P (Am+1 ) = P (Aj ) + P (Am+1 ) = p1,m+1 ,
j=1
XX m
X
p2,m + t1,m = P (Ai ∩ Aj ) + P (Ai ∩ Am+1 )
1≤i<j≤m i=1
XX
= P (Ai ∩ Aj ) = p2,m+1 ,
1≤i<j≤m+1
..
.
XX X
pm,m + tm−1,m = P (A1 ∩ A2 ∩ · · · ∩ Am ) + ··· P (Ai1 ∩ Ai2 ∩ · · · ∩ Aim−1 ∩ Am+1 )
1≤i1 <i2 <···<im−1 ≤m

= pm,m+1

and tm,m = P (A1 ∩ A2 ∩ · · · ∩ Am ∩ Am+1 ) = pm+1,m+1 . The result now follows by induction.
Remark 0.12. Let A1 , A2 , A3 ∈ F. Then

P (A1 ∪ A2 ∪ A3 ) = p1,3 − p2,3 + p3,3

= P (A1 ) + P (A2 ) + P (A3 ) − P (A1 ∩ A2 ) − P (A1 ∩ A3 ) − P (A2 ∩ A3 ) + P (A1 ∩ A2 ∩ A3 ).

Theorem 0.13. For some positive integer k ≥ 2, let A1 , A2 , . . . , Ak ∈ F. Then

k
!
[
p1,k − p2,k ≤ P Ai ≤ p1,k .
i=1

Proof. Note that for k = 2, p1,2 = P (A1 ) + P (A2 ), p2,2 = P (A1 ∩ A2 ) and

P (A1 ∪ A2 ) = P (A1 ) + P (A2 ) − P (A1 ∩ A2 ) ≤ P (A1 ) + P (A2 ).

This implies p1,2 − p2,2 = P (A1 ∪ A2 ) ≤ P (A1 ) + P (A2 ). Thus the result is true for k = 2. Now suppose that for
some positive integer m(≥ 2)
k
!
[
p1,k − p2,k ≤ P Ai ≤ p1,k ∀ k = 1, 2, . . . , m.
i=1

Then,
m+1
! m
! !
[ [ [
P Ai =P Ai Am+1
i=1 i=1
Lecture 0 Lecture Notes 0-8

m
!
[
≤P Ai + P (Am+1 ), using result for k = 2, A = ∪m
i=1 Ai and B = Am+1 ,
i=1

then P (A ∪ B) ≤ P (A) + P (B)
≤ p1,m + P (Am+1 )
= p1,m+1 . (0.3)

Also using the result for k = m, we get

m
!
[
P Ai ≥ p1,m − p2,m
i=1

and !
m
[ m
X
P (Ai ∩ Am+1 ) ≤ P (Ai ∩ Am+1 )
i=1 i=1

Thus,
m+1
! m
! !
[ [ [
P Ai =P Ai Am+1
i=1 i=1
m
! m
!
[ [
=P Ai + P (Am+1 ) − P (Ai ∩ Am+1 )
i=1 i=1
m
X
≥ p1,m − p2,m + P (Am+1 ) − P (Ai ∩ Am+1 )
i=1
m
!
X
= (p1,m + P (Am+1 )) − p2,m + P (Ai ∩ Am+1 ) (0.4)
i=1

Using (0.3) and (0.4), we get !

m+1
[
p1,m+1 − p2,m+1 ≤ P Ai ≤ p1,m+1
i=1

and the result follows using principle of mathematical induction.

Remark 0.14. It can also be shown that
k
!
[
p1,k − p2,k + p3,k − p4,k ≤P Ai ≤ p1,k − p2,k + p3,k
i=1
..
.
k
!
[
p1,k − p2,k + · · · + p2m−1,k − p2m,k ≤P Ai ≤ p1,k − p2,k + · · · + p2m−1,k ,
i=1

for m = 1, 2, . . . , [ k2 ].
Theorem 0.15 (Bonferroni’s Inequality). Let A1 , A2 , . . . , Ak ∈ F. Then
k
! ( k )
\ X
P Ai ≥ max P (Ai ) − (k − 1), 0 .
i=1 i=1
Lecture 0 Lecture Notes 0-9

Proof. We have
k
! k
!c !
\ [
P Ai =P Aci , (De-Morgan’s law)
i=1 i=1
k
!
[
=1−P Aci
i=1
k
X
≥1− P (Aci ), (Boole’s inequality)
i=1
k
X
=1− (1 − P (Ai ))
i=1
k
X
= P (Ai ) − (k − 1). (0.5)
i=1

Also, !
k
\
P Ai ≥ 0. (0.6)
i=1

Combining (0.5) and (0.6), we get

k
! ( k )
\ X
P Ai ≥ max P (Ai ) − (k − 1), 0 .
i=1 i=1

This completes the proof.

Probability as a Continuous Set Function:

A sequence of events {En , n ≥ 1} is said to be an increasing sequence if E1 ⊂ E2 ⊂ · · · ⊂ En ⊂ En+1 . . . whereas
it is said to be decreasing sequence if E1 ⊃ E2 ⊃ · · · ⊃ En ⊃ En+1 ⊃ . . . .
S∞
If {En } is an increasing sequence of events, then we
T∞ define limn→∞ En = i=1 Ei . Similarly, if {En } is decreasing
sequence of events, then we define limn→∞ En = i=1 Ei .
Theorem 0.16. If {En } is either increasing or decreasing sequence of events, then

lim P (En ) = P lim En .
n→∞ n→∞

Proof. Let {En } be an increasing sequence and define Fn , n ≥ 1 as

F1 = E1 ,
F2 = E2 − E1 ,
..
.
Fn = En − En−1 .
Sn Pn
Then, {Fn } is a disjoint sequence of events and En = i=1 Fi =⇒ P (En ) = i=1 P (Fi ). Now
n
[ ∞
[
lim En = lim Fi = Fn .
n→∞ n→∞
i=1 n=1
Lecture 0 Lecture Notes 0-10

So,
∞ ∞
! n n
!
[ X X [
P lim En = P Fn = P (Fn ) = lim P (Fi ) = lim P Fi = lim P (En ).
n→∞ n→∞ n→∞ n→∞
n=1 n=1 i=1 i=1

Similarly, we can prove for other case.

Example 0.17. Random experiment E: casting a red and white die.
Sample space: Ω = {(i, j) : i ∈ {1, 2, . . . , 6}, j ∈ {1, 2, . . . , 6}}.
For (i, j) ∈ Ω, i: number of spots up on the red die; j: number of spots up on the white die.
Event space F = power set of Ω.
For A ∈ F, define Q : F → R as
|A|
Q(A) = , where |A| = number of elements in A.
36
Then
|Ω| 36
(a) Q(Ω) = = = 1.
36 36
|A|
(b) Q(A) = ≥ 0, ∀ A ∈ F.
36
(c) For mutually exclusive events A1 , A2 , . . .
∞ ∞ ∞
! S∞ P∞
[ | i=1 Ai | |Ai | X |Ai | X
Q Ai = = i=1 = = Q(Ai ).
i=1
36 36 i=1
36 i=1

Thus, (Ω, F, Q) is a probability space.

Equally Likely Probability Models for Finite Sample Space:

Suppose that the sample space Ω = {ω1 , ω2 , . . . , ωk } is finite (has k elements). Here singletons {ωi } are called
[k
elementary events and Ω = {ωi }. Suppose that
i=1

1
P ({ωi }) = , i = 1, 2, . . . , k (each elementary event is equally likely).
k

Sr event E ⊆ Ω, we have E = {ωi1 , ωi2 , . . . , ωir }, for some i1 , i2 , . . . , ir ∈ {1, 2, . . . , k}, 1 ≤ r ≤ k. Then,
For any
E = j=1 {ωij } and
 
[r X r

P (E) = P  {ωij } = P {ωij }
j=1 j=1
r
X 1 r number of ways favourable to event E
= = = .
j=1
k k total number of ways in which the random experiment can terminate

1
Here the assumption of equally likely P ({ωi }) = , i = 1, 2, . . . , k is a part of probability modelling.
k
“At random”: In a random experiment with finite sample space Ω, whenever we say that the experiment has been
performed at random it means that all the outcomes in the sample space are equally likely.
Lecture 0 Lecture Notes 0-11

Example 0.18 (Birthday Problem). Suppose that a college has n students, including you. Each of them were born on
non-leap years.
(a) Find the probability that at least two of them have the same birthday. For what values of n this probability is more
than 0.5, 0.8, 0.95?
(b) For what value of n the probability that you will find someone who shares your birthday is 0.5.

365 × 364 × · · · × (365 − n + 1)

Solution: Required probability = 1−P (all of them have different birthdays) = 1−
365n
364n−1
Required probability = 1 − P (no one shares the same birthday as mine) = 1 − .
365n−1
364n−1
For 1 − ≈ 0.5, n ≈ 253.
365n−1
Example 0.19. Five cards are drawn at random and without replacement from a deck of 52 cards. Find the probability
that
(i) each card is spade (event E1 ),
(ii) at least one card is spade (event E2 ),
(iii) exactly three cards are king and two cards are queen (event E3 ),
(iv) exactly two kings, two queens and one jack are drawn (event E4 ).

13

5
Solution: (i) P (E1 ) = 52 ,

5
39

5
(ii) P (E2 ) = 1 − P (E2c ) = 1 − P (no card is spade) = 1 − 52 ,

5
4
4
3
(iii) P (E3 ) = 52
2 ,
5
4
4 4
2 2 1
(iv) P (E4 ) = 52
.
5

Example 0.20 (Capture/Recapture Method for Estimating Population Size). In a wildlife population suppose that the
population size n is unknown. To estimate the population size n, 20 animals are captured, tagged and then released
back. Thereafter 40 animals are captured at random and it is found that 8 of them are tagged. Find an estimate of the
population size n based on the given data.

Solution: We have

n = total number of animals,

20 = number of tagged animals in the population,
n − 20 = number of untagged animals in the population.

Data: Sample of 40 animals yield

number of tagged animals = 8,

number of untagged animals = 32.
Lecture 0 Lecture Notes 0-12

The probability of obtaining this data is

20 n−20

8 32
l(n) = n , n ≥ 52.
40

n−19 n−20

32 32
l(n + 1) > l(n) ⇐⇒ n+1
> n
40 40
n − 19
⇐⇒ >1
(n − 51)(n + 1)
⇐⇒ n < 99.

Similarly l(n + 1) < l(n) ⇐⇒ n > 99. Thus l is maximized at n = 99, that is, for n = 99, the observe data (among
the captured animals 8 are tagged and 32 are untagged) is most probable.
Thus an estimate of n is n̂ = 99 (Maximum likelihood estimator).

0.3. Conditional Probability

Consider a probability space (Ω, F, P ) where Ω = {ω1 , ω2 , . . . , ωn } is finite and

1
P ({ωi }) = , i = 1, 2, . . . , n (equally likely probability model).
n
Then, for any A ∈ F
number of cases favourable to A |A| |A|
P (A) = = = .
total number of cases |Ω| n
Now suppose it is known a priori that event A has occured (i.e. outcome of the experiment is an element of A), where
|A| ≥ 1 (so that P (A) = |A|/n > 0). Given this prior information (that the event A has occured) we want to define
probability function say P (B|A) on the event space F. A natural way to define P (B|A) is

|A ∩ B| |A ∩ B|/n P (A ∩ B)
P (B|A) = = = , B ∈ F.
|A| |A|/n P (A)

Definition 0.21. Let (Ω, F, P ) be a probability space and let A ∈ F be such that P (A) > 0. Then

P (A ∩ B)
P (B|A) = , B ∈ F,
P (A)
is called the conditional probabilty of event B given the event A.
Remark 0.22. (a) In the above definition the event A (with P (A) > 0) is fixed and for this fixed A ∈ F, P (·|A) is a
set function defined on F. Is it a probability function/ measure?
(b) P (A ∩ B) = P (A)P (B|A) = P (B)P (A|B) for A, B ∈ F.
Theorem 0.23. Let (Ω, F, P ) be a probability space and let A ∈ F be such that P (A) > 0 be fixed. Then
P (·|A) : F → R is a probability function (called the conditional probabilty function) on F (so that (Ω, F, P (·|A)) is
a probability space).

P (A ∩ B) P (A ∩ Ω)
Proof. Note that P (B|A) = ≥ 0 for all B ∈ F and P (Ω|A) = = 1.
P (A) P (A)
Lecture 0 Lecture Notes 0-13

Let {Bn }n≥1 be a sequence of disjoint events in F. Then,

∞
! S∞ S∞
[ P (( n=1 Bn ) ∩ A) P ( n=1 (Bn ∩ A))
P Bn A = = .
n=1
P (A) P (A)

Since {Bn }n≥1 are disjoint then subsets {Bn ∩ A}n≥1 are also disjoint. Since P (·) is a probability measure, we get
∞ ∞ ∞
! P
∞
[ P (Bn ∩ A) X P (Bn ∩ A) X
P Bn | A = n=1 = = P (Bn |A).
n=1
P (A) n=1
P (A) n=1

It follows that P (·|A) is a probabilty function on F for any fixed A ∈ F with P (A) > 0.
Example 0.24. Five cards are drawn at random (without replacement) from a deck of 52 cards. Define events

B : all spade in hand and A : at least 4 spade in hand.

Find P (B|A).

Solution: We have
P (A ∩ B) P (B)
P (B|A) = = (since B ⊆ A)
P (A) P (A)
13
/ 52

= 13 395 5
13
52 = 0.441.
4 1 + 5 / 5

Remark 0.25 (Multiplication Law). (i) P (A ∩ B) = P (A)P (B|A), if P (A) > 0.

(ii) P (A ∩ B ∩ C) = P (A ∩ B)P (C|A ∩ B) = P (A)P (B|A)P (C|A ∩ B), provided P (A ∩ B) > 0 (which ensures
that P (A) > 0 as A ∩ B ⊆ A).
(iii) Using principle of mathematical induction, we have
n
!
\
P Ci = P (C1 ∩ C2 ∩ · · · ∩ Cn )
i=1
= P (C1 ∩ C2 ∩ · · · ∩ Cn−1 )P (Cn |C1 ∩ C2 ∩ · · · ∩ Cn−1 )
= P (C1 ∩ C2 ∩ · · · ∩ Cn−2 )P (Cn−1 |C1 ∩ C2 ∩ · · · ∩ Cn−2 )P (Cn |C1 ∩ C2 ∩ · · · ∩ Cn−1 )
..
.
= P (C1 )P (C2 |C1 )P (C3 |C1 ∩ C2 ) . . . P (Cn |C1 ∩ C2 ∩ · · · ∩ Cn−1 )

provided P (C1 ∩ C2 ∩ · · · ∩ Cn−1 ) > 0 (which also ensures that P (C1 ∩ C2 ∩ · · · ∩ Ci ) > 0, i = 1, 2, . . . , n − 2).
Due to symmetry, if (α1 , α2 , . . . , αn ) is a permutation of (1, 2, . . . , n), then
n
!
\
P Ci = P (Cα1 ∩ Cα2 ∩ · · · ∩ Cαn )
i=1
= P (Cα1 )P (Cα2 |Cα1 )P (Cα3 |Cα1 ∩ Cα2 ) . . . P (Cαn |Cα1 ∩ Cα2 ∩ · · · ∩ Cαn−1 )

provided P (Cα1 ∩Cα2 ∩· · ·∩Cαn−1 ) > 0 (which also ensures that P (Cα1 ∩Cα2 ∩· · ·∩Cαi ) > 0, i = 1, 2, . . . , n−2).
Lecture 0 Lecture Notes 0-14

Example 0.26. A bowl contains 3 red and 5 blue chips. All chips that are of the same colour are identical. Two chips
are drawn successively at random and without replacement. Define events

A : first draw resulted in a red chip,

B : second draw resulted in a blue chip.

Find P (A ∩ B), P (A) and P (B).

3 5
Solution: P (A) = , P (B|A) = and
8 7
5 3 4 5 35
P (B) = P (A ∩ B) + P (Ac ∩ B) = P (B|A)P (A) + P (B|Ac )P (Ac ) = × + × = .
7 8 7 8 56
Note that here the outcomes of second draw is dependent on outcome of first draw (P (B|A) 6= P (B)). Also,
3 5
P (A ∩ B) = P (A)P (B|A) = × = 0.2679.
8 7
Theorem 0.27 (Theorem of Total Probability). For a countable set ∆ (that is elements of ∆ can either be put in 1-1
correspondence with N = {1, 2, . . . } or with {1, 2, . . . , n} for some n ∈ N), let {ESα : α ∈ ∆} be a countable
collection of mutually exclusive (i.e., Eα ∩ Eβ = φ, ∀ α 6= β ) and exhaustive (i.e., P α∈∆ Eα = 1) events. Then,
for any E ∈ F, X X
P (E) = P (E ∩ Eα ) = P (E|Eα )P (Eα ).
α∈∆ α∈∆
P (Eα )>0

S
Proof. Since P α∈∆ Eα = 1, we have
!! !
\ [ [
P (E) = P E Eα =P (E ∩ Eα )
α∈∆ α∈∆
X
= P (E ∩ Eα ), (Eα ’s are disjoint =⇒ their subsets (E ∩ Eα )’s are disjoint)
α∈∆
X
= P (E ∩ Eα ), (P (Eα ) = 0 =⇒ P (E ∩ Eα ) = 0, α ∈ ∆)
α∈∆
P (Eα )>0
X
= P (E|Eα )P (Eα ).
α∈∆
P (Eα )>0

This completes the proof.

Example 0.28. A population comprises of 40% female and 60% male. Suppose that 15% of female and 30% of male
in the population smoke. A person is selected at random from the population.
(a) Find the probability that he/she is a smoker.
(b) Given that the selected person is smoker, find the probability that he is male.

Solution: Define the events

M : selected person is a male,

F = M c : selected person is a female,
Lecture 0 Lecture Notes 0-15

S : selected person is a smoker,

T = S c : selected person is a non-smoker.

We have P (F ) = 0.4, P (M ) = 0.6, P (F ∪M ) = P (F )+P (M ) = 1, P (S|F ) = 0.15, P (T |F ) = 0.85, P (S|M ) =

0.30, P (T |M ) = 0.70.
(a) By using Theorem of total probability, we get

P (S) = P (S ∩ F ) + P (S ∩ M ) = P (S|F )P (F ) + P (S|M )P (M ) = 0.15 × 0.4 + 0.30 × 0.6 = 0.24.

(b)
P (M ∩ S) P (S|M )P (M ) 0.30 × 0.60 3
P (M |S) = = = = .
P (S) P (S) 0.24 4
Theorem 0.29 (Bayes’ Theorem). Let {Eα : α ∈ ∆} be a countable collection of mutually exclusive and exhaustive
events and let E be any event P (E) > 0. Then, for j ∈ ∆ with P (Ej ) > 0,

P (E|Ej )P (Ej )
P (Ej |E) = P .
P (E|Eα )P (Eα )
α∈∆
P (Eα )>0

Proof. For j ∈ ∆,

P (Ej ∩ E) P (E|Ej )P (Ej )

P (Ej |E) = = P , (using Theorem of total probability).
P (E) P (E|Eα )P (Eα )
α∈∆
P (Eα )>0

This completes the proof.

Remark 0.30. (a) Suppose that occurrence of any of the mutually exclusive and exhaustive events {Eα : α ∈ ∆}
(where ∆ is a countable set) may cause the occurrence of an event E. Given that the event E has occurred (i.e., given
the effect), Bayes’ Theorem provides the conditional probability that the event E (effect) is caused by occurrence of
event Ej , j ∈ ∆.
(b) In Bayes’ Theorem {P (Ej ) : j ∈ ∆} are called prior probabilities and {P (Ej |E) : j ∈ ∆} are called posterior
probabilities.
Example 0.31. Bowl C1 contains 3 red and 7 blue chips. Bowl C2 contains 8 red and 2 blue chips. Bowl C3 contains
5 red and 5 blue chips. All chips of the same colour are identical.
A die is cast and a bowl is selected as per the following schemes:

Bowl C1 is selected if 5 or 6 spots show on the upper side,

Bowl C2 is selected if 2,3 or 4 spots show on the upper side,
Bowl C3 is selected if 1 spots show on the upper side.

The selected bowl is handed over to another person who drawns two chips at random from this bowl. Find the
probability that:
(a) Two red chips are drawn.
(b) Given that drawn chips are both red, find the probability that it came from bowl C3 .
Lecture 0 Lecture Notes 0-16

Solution: Define the events

Ai : selected bowl is Ci , i = 1, 2, 3, and R: the chips drawn from the selected bowl are both red.
2 1 3 1 1
Then P (A1 ) = = , P (A2 ) = = , P (A3 ) = . Note that {A1 , A2 , A3 } are mutually exclusive and
6 3 6 2 6
exhaustive.
(a)
3 8 5

2 1 2 1 2 1 10
P (R) = P (R|A1 )P (A1 ) + P (R|A2 )P (A2 ) + P (R|A3 )P (A3 ) = 10
× + 10
× + 10
× = .
2
3 2
2 2
6 27

(b)
(52) 1
×
P (R|A3 )P (A3 ) (10
2) 6 1
P (A3 |R) = = 10 = .
P (R) 27
10
Remark 0.32. In the above example,

(32) 1
×
P (R|A1 )P (A1 ) (10
2)
3 3
P (A1 |R) = = 10 = ,
P (R) 27
50
(82) 1
×
P (R|A2 )P (A2 ) 2(1021
2)
P (A2 |R) = = 10 = ,
P (R) 27
25
3 1
P (A1 |R) = < = P (A1 ) ⇐⇒ P (A1 ∩ R) < P (A1 )P (R) ←→ R has negative information about A1 ,
50 3
21 1
P (A2 |R) = > = P (A2 ) ⇐⇒ P (A2 ∩ R) > P (A2 )P (R) ←→ R has positive information about A2 ,
25 2
1 1
P (A3 |R) = < = P (A3 ) ⇐⇒ P (A3 ∩ R) < P (A3 )P (R) ←→ R has negative information about A3 .
10 6
Note that proportion of red chips in C2 > proportion of red chips in Ci , i = 1, 3.

Independent Events:
Definition 0.33. Let {Ej : j ∈ ∆} be a collection of events.
(i) Events {Ej : j ∈ ∆} are said to be pairwise independent if for any pair of events Eα and Eβ (α, β ∈ ∆, α 6= β)
in the collection {Ej : j ∈ ∆}, we have

P (Eα ∩ Eβ ) = P (Eα )P (Eβ ).

(ii) Events {E1 , E2 , . . . , En } are said to be independent if for any subcollection {Eα1 , Eα2 , . . . , Eαk } of {E1 , E2 , . . . , En }
(k = 1, 2, . . . , n), we have  
\k Yk
P Eαj  = P (Eαj ).
j=1 j=1

(iii) Let ∆ ⊆ R be an arbitrary index set so that {Eα : α ∈ ∆} is an arbitrary collection of events. Events
{Eα : α ∈ ∆} are said to be independent if any finite subcollection of events in {Eα : α ∈ ∆} forms a collection of
independent events.
Lecture 0 Lecture Notes 0-17

Theorem 0.34. Let E1 , E2 , . . . be collection of independent events. Then

∞ ∞
!
\ Y
P Ek = P (Ek ).
k=1 k=1

Tn T∞ T∞ T∞
Proof. Let Bn = k=1 Ek , n = 1, 2, . . . . Then Bn ↓ and P ( n=1 Bn ) = lim P (Bn ). But n=1 Bn = k=1 Ek
Tn Qn n→∞
and P (Bn ) = P ( k=1 Ek ) = k=1 P (Ek ). Thus,
∞ ∞
! n
\ Y Y
P Ek = lim P (Ek ) = P (Ek ).
n→∞
k=1 k=1 k=1

This completes the proof.

Remark 0.35. (i) To verify that n events E1 , E2 , . . . , En are independent one must verify

n n n
+ + ··· + = 2n − n − 1 conditions.
2 3 n

For an example to conclude that three events E1 , E2 , E3 are independent, the following four (as 23 − 3 − 1 = 4)
conditions must be verified:

P (E1 ∩ E2 ) = P (E1 )P (E2 ), P (E1 ∩ E3 ) = P (E1 )P (E3 ), P (E2 ∩ E3 ) = P (E2 )P (E3 ),

and
P (E1 ∩ E2 ∩ E3 ) = P (E1 )P (E2 )P (E3 ).

(ii) Any subcollection of independent events is independent. In particular, the independence of a collection of events
implies their pairwise independence.
(iii) If E1 and E2 are independent events (P (E1 ) > 0, P (E2 ) > 0), then

P (E1 ∩ E2 ) P (E1 )P (E2 )

P (E1 |E2 ) = = = P (E1 ),
P (E2 ) P (E2 )

that is, conditional probability of E1 given E2 is the same as unconditional probability of E1 .

Similarly, if E1 , E2 and E3 are independent events then P (E1 |E2 ∩ E3 ) = P (E1 ).
Example 0.36. Consider the probability space (Ω, F, P ) with Ω = {1, 2, 3, 4} and P ({i}) = 1/4, i = 1, 2, 3, 4. Let
A = {1, 4}, B = {2, 4}, C = {3, 4}. Then, show that A, B and C are pairwise independent but not independent.

Solution: We have P (A) = P (B) = P (C) = 1/2. Also, P (A ∩ B) = P (A ∩ C) = P (B ∩ C) = P ({4}) = 1/4.

Thus,
P (A ∩ B) = P (A)P (B), P (A ∩ C) = P (A)P (C), P (B ∩ C) = P (B)P (C),
which implies that A, B and C are pairwise independent. However,

P (A ∩ B ∩ C) = P ({4}) = 1/4 6= 1/8 = P (A)P (B)P (C),

which implies that A, B and C are not independent although they are pairwise independent.
Lecture 0 Lecture Notes 0-18

Example 0.37. Let E1 , E2 , . . . , En be a collection of independent events. Show that,

(a) for any permutation (α1 , . . . , αn ) of (1, . . . , n), Eα1 , Eα2 , . . . , Eαn are independent;
c
(b) E1 , E2 , . . . , Ek , Ek+1 , . . . , Enc are independent for any k ∈ {0, 1, . . . , n − 1};
(c) E1c and E2 ∪ E3c ∪ E5 are independent.
(d) E1 ∪ E2c , E3c and E4 ∩ E5c are independent.
Remark 0.38. When we say that the two random experiments are performed independently, it means that the events
associated with two random experiments are independent.

0.4. Random Variables and their Distribution Functions

Let (Ω, F, P ) be a given probability space. In some situations we may not be directly interested in the sample space
Ω; rather we may be interested in some numerical aspect of Ω.
Example 0.39. A fair coin (head and tail are equally likely) is tossed three times independently. Then,

Ω = {HHH, HHT, HT H, HT T, T T T, T T H, T HT, T HH}

and P ({ω}) = 1/8 for all ω ∈ Ω. Suppose that we are interested in number of heads in three tosses, i.e., we are
interested in the function X : Ω → R defined as

 0, if ω = T T T,

1, if ω ∈ {HT T, T HT, T T H},

X(ω) =
2, if ω ∈ {HHT, HT H, T HH},



3, if ω = HHH.

Clearly the values assumed by X are random with

P (X = 0) = P (X = 3) = 1/8 and P (X = 1) = P (X = 2) = 3/8.

Hence P (X ∈ {0, 1, 2, 3}) = 1.

Definition 0.40. Let (Ω, F, P ) be a given probability space. A real valued measurable function X : Ω → R (defined
on sample space Ω) is called a random variable (r.v.).

Note: From rigorous mathematical point of view a random variable is a real valued function with some technical
condition. In this course we are ignoring these technical details. For all practical purpose r.v. is a real valued function
defined on Ω.
For a function Y : Ω → R and A ⊆ R, define

Y −1 (A) = {ω ∈ Ω : Y (ω) ∈ A}.

Then it is straightforward to prove the following result:

Proposition 0.41. Let A ⊆ R, B ⊆ R and Aα ⊆ R, α ∈ Λ, where Λ is an arbitrary index set. Let Y : Ω → R be a
given function. Then
(a) If A ∩ B = φ, then Y −1 (A) ∩ Y −1 (B) = φ;
(b) Y −1 (Ac ) = (Y −1 (A))c (that is, Y −1 (R − A) = Y −1 (R) − Y −1 (A) = Ω − Y −1 (A));
Lecture 0 Lecture Notes 0-19

(c) Y −1 Aα = α∈Λ Y −1 (Aα );

S S
α∈Λ

(d) Y −1 −1
T T
α∈Λ Aα = α∈Λ Y (Aα ).

For a probability space (Ω, F, P ) and a r.v. X : Ω → R, note that ∀ B ⊆ B

X −1 (B) = {ω ∈ Ω : X(ω) ∈ B} ∈ F.
Thus, one can define a set function PX : B → [0, 1] by
PX (B) = P (X −1 (B)) = P ({ω ∈ Ω : X(ω) ∈ B}) , B ∈ B,
where B is some class of subsets of R. Here, also for all practical purpose we will take B to be a sigma algebra
formed by open subsets of R.
We simply write
PX (B) = P ({ω ∈ Ω : X(ω) ∈ B}) = P (X ∈ B), B ∈ B.
X
We have the following scenario (Ω, F, P ) −→ (R, B, PX ).
Theorem 0.42 (Induced probability space / measures). (R, B, PX ) (as defined above) is a probability space, i.e.,
PX (·) is a probability function defined on B.

Proof. (i) PX (R) = P (X ∈ R) = P (X −1 (R)) = P (Ω) = 1.

(ii) For any B ∈ B, PX (B) = P (X −1 (B)) ≥ 0.
(iii) Let {Bn } be a collection of mutually exclusive events in B. Then,
∞ ∞
! !!
[ [
−1
PX Bn = P X Bn
n=1 n=1
∞
!
[
−1
=P X (Bn ) , (Proposition 0.41(c))
n=1
∞
X
= P (X −1 (Bn )), (P is a probability measure and using Proposition 0.41(a))
n=1
X∞
= PX (Bn ).
n=1

This completes the proof.

Definition 0.43. The probability function PX defined above is called the probability function/ measure induced by
r.v. X and (R, B, PX ) is called the probability space induced by r.v. X.

The induced probability measure PX describes the random behaviour of X.

Example 0.44. Toss a coin three times independently. Then,
Ω = {HHH, HHT, HT H, T HH, HT T, T HT, T T H, T T T } and P ({ω}) = 1/8, ∀ ω ∈ Ω
and X : Ω → R (number of heads in three tosses) is defined by
0, if ω ∈ {T T T },



1, if ω ∈ {HT T, T HT, T T H},

X(ω) =

 2, if ω ∈ {HHT, HT H, T HH},

3, if ω ∈ {HHH}.

Lecture 0 Lecture Notes 0-20

Obviously, X : Ω → R is r.v. with induced probability space given by (R, B, PX ), where

PX ({0}) = P ({T T T }) = 1/8,
PX ({1}) = P ({HT T, T HT, T T H}) = 3/8,
PX ({2}) = P ({HHT, HT H, T HH}) = 3/8,
PX ({3}) = P ({HHH}) = 1/8.
Now for any B ∈ B,
X
PX (B) = P (X −1 (B)) = P ({ω ∈ Ω : X(ω) ∈ B}) = PX ({i}).
i∈B∩{0,1,2,3}

Definition 0.45. Let X be a r.v. defined on probability space (Ω, F, P ) and let (R, B, PX ) denote the probability
space induced by X. Define the function FX : R → R by
FX (x) = P (X ≤ x) = P (X −1 (−∞, x]) = PX ((−∞, x]), x ∈ R.
The function FX is called the cumulative distribution function (c.d.f.) or simply the distribution function (d.f.) of r.v.
X.

Note: Whenever there is no ambiguity we will drop subscript X in FX to represent d.f. of a r.v. by F . It can be shown
(in advanced courses) that the c.d.f. FX (·) of a r.v. X determines the induced probability measure PX (·) uniquely.
Thus to study the random behaviour of r.v. X it suffices to study its d.f. F .
Example 0.46. In the previous example
P (X = 0) = PX ({0}) = 1/8, P (X = 1) = PX ({1}) = 3/8 = P (X = 2) = PX ({2})
and P (X = 3) = PX ({3}) = 1/8. Then, the d.f. of X is obtained as


0, x < 0,

1/8, 0 ≤ x < 1,


X 
FX (x) = P (X ≤ x) = P ({ω : X(ω) ≤ x}) = PX ({i}) = 1/8 + 3/8 = 1/2, 1 ≤ x < 2,

7/8, 2 ≤ x < 3,
i∈{0,1,2,3} 


i≤x 
1, x ≥ 3.


Theorem 0.47. Let F (·) be the c.d.f. of a r.v. X defined on a probability space (Ω, F, P ) and let (R, B, PX ) be the
probability space induced by X. Then
(i) F is non-decreasing,
(ii) F (x) is right continuous,
(iii) F (−∞) = lim F (−n) = 0 and F (∞) = lim F (n) = 1.
n↑∞ n↑∞

Conversely, any function G(·) satisfying properties (i)-(iii) is a d.f. of some r.v. Y defined on a probability space
(Ω∗ , F∗ , P ∗ ).

Proof. (i) Let −∞ < x < y < ∞. Then (−∞, x] ⊆ (−∞, y] =⇒ PX ((−∞, x]) ≤ PX ((−∞, y]). This implies
that F (x) ≤ F (y).
(ii) Since F is monotone and bounded below (by 0), lim F (x + h) = F (x+) exists ∀ x ∈ R. Therefore,
h↓0

1
F (x+) = lim F (x + h) = lim F x+ = lim PX ((−∞, x + 1/n]) .
h↓0 n→∞ n n→∞
Lecture 0 Lecture Notes 0-21

T∞
Let An = (−∞, x + 1/n], n = 1, 2, . . . . Then An ↓ and n=1 (−∞, x + 1/n] = (−∞, x]. Thus,
∞
!
\
F (x+) = PX (−∞, x + 1/n] = PX ((−∞, x]) = F (x).
n=1

(iii) Note that

∞
!
\
F (−∞) = lim F (−n) = lim PX ((−∞, −n]) = PX (−∞, −n] , ((−∞, −n] ↓)
n→∞ n→∞
n=1
∞
!
\
= PX (φ), (−∞, −n] = φ
n=1
= 0.

Also,
∞
!
[
F (+∞) = lim F (n) = lim PX ((−∞, n]) = PX (−∞, n] , ((−∞, n] ↑)
n→∞ n→∞
n=1
∞
!
[
= PX (R), (−∞, n] = R
n=1
= 1.

This completes the proof.

Remark 0.48. (i) Since any distribution function is monotone and bounded above (by 1), lim F (x − h) = F (x−)
h↓0
exists ∀x ∈ R. Moreover,

F (x−) = lim F (x − h) = lim F (x − 1/n) = lim PX ((−∞, x − 1/n])

h↓0 n→∞ n→∞
∞
!
[
= PX (−∞, x − 1/n] , ((−∞, x − 1/n] ↑)
n=1
= PX ((−∞, x)) = P (X < x).

(ii) From the calculus we know that any monotone function is either continuous on R or it has atmost countable number
of discontinuities. Thus any c.d.f F (x) is either continuous on R or has atmost countable number of discontinuities.
Since, for any x ∈ R, F (x+) and F (x−) exist, F has only jump discontinuities (F (x) = F (x+) > F (x−)).
(iii) A distribution function F is continuous at a ∈ R iff F (a) = F (a−).

(iv) For any a ∈ R, P (X = a) = P (X ≤ a) − P (X < a) = F (a) − F (a−). Thus, a d.f. F is continuous at a ∈ R

iff P (X = a) = F (a) − F (a−) = 0.

(v) For −∞ < a < b < ∞, P (X ≤ b) = P (X ≤ a) + P (a < X ≤ b).

P (a < X ≤ b) = P (X ≤ b) − P (X ≤ a) = F (b) − F (a).

Similarly, for −∞ < a < b < ∞,

P (a < X < b) = P (X < b) − P (X ≤ a) = F (b−) − F (a),

Lecture 0 Lecture Notes 0-22

P (a ≤ X ≤ b) = P (X ≤ b) − P (X < a) = F (b) − F (a−),

P (a ≤ X < b) = P (X < b) − P (X < a) = F (b−) − F (a−),
P (a < X < b) = P (X < b) − P (X ≤ a) = F (b−) − F (a),
P (X > a) = 1 − P (X ≤ a) = 1 − F (a),
P (X ≥ a) = 1 − P (X < a) = 1 − F (a−).

Example 0.49. Consider the function G : R → R defined by




 0, if x < 0,
x
3, if 0 ≤ x < 1,



G(x) = 12 , if 1 ≤ x < 2,
2

3, if 2 ≤ x < 3,





1, if x ≥ 3.
(a) Show that G is d.f. of some r.v. X,
(b) Find P (X = a) for various values of a ∈ R,
(c) Find P (X < 3), P X ≥ 12 , P (2 < X ≤ 4), P (1 ≤ X < 2), P (2 ≤ X ≤ 3) and P 1

2 <X<3 .

Solution: (a) Clearly G is non-decreasing in (−∞, 0), (0, 1), (1, 2), (2, 3) and (3, ∞). Moreover,
1 1
G(0) − G(0−) = 0 ≥ 0, G(1) − G(1−) =
− > 0,
2 3
2 1 2
G(2) − G(2−) = − > 0, G(3) − G(3−) = 1 − > 0.
3 2 3
It follows that G is non-decreasing.
Clearly G is continuous ( and hence right continuous) on (−∞, 0), (0, 1), (1, 2), (2, 3) and (3, ∞). Moreover,

G(0+) − G(0) = 0 − 0 =0 
G(1+) − G(1) = 1/2 − 1/2 = 0

=⇒ G is right continuous on R.
G(2+) − G(2) = 2/3 − 2/3 = 0  
G(3+) − G(3) = 1 − 1 =0


Also, G(+∞) = lim G(x) = 1 & G(−∞) = lim G(−x) = 0. Thus, G is a d.f. of some random variable X.
x→∞ x→∞

(b) The set of discontinuity points of F is D = {1, 2, 3}. Thus,

P (X = a) = G(a) − G(a−) = 0, ∀ a 6= 1, 2, 3,
1 1 1
P (X = 1) = G(1) − G(1−) = − = ,
2 3 6
2 1 1
P (X = 2) = G(2) − G(2−) = − = ,
3 2 6
2 1
P (X = 3) = G(3) − G(3−) = 1 − = .
3 3

(c) Note that

2
P (X < 3) = G(3−) = ,
3
Lecture 0 Lecture Notes 0-23

1 1 1 5
P X≥ =1−G − = , =1−
2 2 6 6
2 1
P (2 < X ≤ 4) = G(4) − G(2) = 1 − = ,
3 3
1 1 1
P (1 ≤ X < 2) = G(2−) − G(1−) = − = ,
2 3 6
1 1
P (2 ≤ X ≤ 3) = G(3) − G(2−) = 1 − = ,
2 2
1 1 2 1 1
P < X < 3 = G(3−) − G = − = .
2 2 3 6 2

0.5. Discrete Random Variables

Let (Ω, F, P ) be a probability space and let X : Ω → R be a r.v. with induced probability space (R, B, PX ) and d.f.
F.
Definition 0.50. The r.v. X is said to be a discrete r.v. if there exists a countable set S (finite or infinite) such that
P (X = x) = F (x) − F (x−) > 0 ∀ x ∈ S, and P (X ∈ S) = 1.
The set S is called the support of r.v. X.
Remark 0.51. (i) If S is the support of a discrete r.v. X, then clearly
S = {x ∈ R : F (x) − F (x−) > 0} = set of discontinuity points of F.

(ii) If x is a discontinuity point of d.f. F then

F (x) − F (x−) = size of jump of F at x.
Thus, a r.v. X is of discrete type ⇐⇒ sum of jump points of F equals 1, i.e.,
X X
P (X ∈ S) = P (X = x) = [F (x) − F (x−)] = 1.
x∈S x∈S

Example 0.52. In Example 0.49 the set of discontinuity points of G is D = {1, 2, 3} and
X
[G(x) − G(x−)] = 1/6 + 1/6 + 1/3 = 2/3 < 1 =⇒ X is not a discrete r.v.
x∈D

Example 0.53. Consider the d.f. (see Example 0.46)



 0, if x < 0,

1/8, if 0 ≤ x < 1,



FX (x) = 1/2, if 1 ≤ x < 2,

7/8, if 2 ≤ x < 3,





1, if x ≥ 3.


The set of discontinuity points of F is D = {0, 1, 2, 3} with

X 1 1 1 7 1 7
[F (x) − F (x−)] = + − + − + 1− = 1,
8 2 8 8 2 8
x∈D

which implies that X is a discrete r.v. with support S = D = {0, 1, 2, 3}.

Lecture 0 Lecture Notes 0-24

Definition 0.54. Let X be a r.v. with c.d.f. FX and support SX . Define the function fX : R → R by
(
P (X = x) = FX (x) − FX (x−) > 0, if x ∈ SX ,
fX (x) =
0, otherwise.

The function fX is called the probability mass function (p.m.f.) of r.v. X.

Whenever there is no ambiguity we will drop subscript X in FX , SX and fX to represent the d.f. of X by F , the
support of X by S and the p.m.f. of X by f .
Remark 0.55. (i) Let X be a discrete r.v with p.m.f. f and d.f F . Then, for any A ⊆ R
X
P (X ∈ A) = P (X ∈ A ∩ S) = f (x), (A ∩ S ⊆ S and thus A ∩ S is a countable set),
x∈A∩S

where S is the support of X.

X
Moreover, F (x) = f (y). Also, for any x ∈ S, f (x) = F (x) − F (x−).
y∈S∩(−∞,x]

(ii) Clearly a d.f. determines the p.m.f. uniquely and vice-versa. Thus it suffices to study the p.m.f. of discrete r.v.
(iii) Let X be a discrete r.v. with p.m.f. f and support S. Then, f : R → R satisfies
X
(i) f (x) > 0, ∀ x ∈ S, (ii) f (x) = 1.
x∈S

Conversely, suppose that g : R → R is a function such that, for some countable set T
X
(i) g(x) > 0, ∀ x ∈ T and (ii) g(x) = 1.
x∈T

Then, g(·) is the p.m.f. of some discrete r.v. having support T .

Example 0.56. Let X be a r.v. having d.f.


 0, if x < 0,

1/8, if 0 ≤ x < 1,



F (x) = 1/2, if 1 ≤ x < 2,

7/8, if 2 ≤ x < 3,





1, if x ≥ 3.


We have seen in Example 0.53 that X is a discrete r.v with support S = {0, 1, 2, 3}. Then, the p.m.f. of X is
f : R → R, where

f (0) = F (0) − F (0−) = 1/8, f (1) = F (1) − F (1−) = 1/2 − 1/8 = 3/8,
f (2) = F (2) − F (2−) = 7/8 − 1/2 = 3/8 and f (3) = F (3) − F (3−) = 1 − 7/8 = 1/8.

Thus, the p.m.f. of X is 

1/8,
 x = 0, 3,
f (x) = 3/8, x = 1, 2,

0, otherwise.

Lecture 0 Lecture Notes 0-25

Example 0.57. A fair die (all outcomes are equally likely) is tossed repeatedly and independently until a 6 is observed.
Then X is a discrete r.v. with support S = {1, 2, 3, . . . }.
 x−1
 5
 1
, if x = 1, 2, 3, . . . ,
p.m.f. f (x) = P (X = x) = 6 6

0, otherwise


and d.f. 
0,


if x < 1,
1/6, if 1 ≤ x < 2,




11/36, if 2 ≤ x < 3,

F (x) =

 ..



 .
Pi ( 5 )j−1 1 = 1 − 5 i ,


j=1 6 6 6 if i ≤ x < i + 1.

0.6. Continuous Random Variable

Let X be a random variable with d.f. F .

Definition 0.58. The r.v. X is said to be a continuous r.v. if there exists a non-negative integrable function f : R →
[0, ∞) such that, for any x ∈ R, Zx
F (x) = P (X ≤ x) = f (t)dt.
−∞

The function f (·) is called the probability density function (p.d.f.) of X. The support of the continuous r.v X is the
Z x+h
set S = {x ∈ R : F (x + h) − F (x − h) > 0 ∀ h > 0}, that is, S = {x ∈ R : f (t)dt > 0 ∀ h > 0}.
x−h

Remark 0.59. (i) From the fundamental theorem of calculus, we know that the definite integral
Z x
F (x) = f (t)dt
−∞

is a continuous function on R. Thus, the d.f F of any continuous r.v X is continuous everywhere on R. In particular,

P (X = x) = F (x) − F (x−) = 0, ∀ x ∈ R.

Generally, if A is any countable subset of R then for any continuous r.v. X

X
P (X ∈ A) = P (X = x) = 0.
x∈A

(ii) If X is a continuous r.v. then

(a) P (X < x) = P (X ≤ x) = F (x) ∀ x ∈ R,
(b) P (X ≥ x) = 1 − P (X < x) = 1 − F (x) ∀ x ∈ R,
(c) For any a, b ∈ R, −∞ < a < b < ∞,

P (a < X < b) = P (a ≤ X < b) = P (a < X ≤ b) = P (a ≤ X ≤ b)

Lecture 0 Lecture Notes 0-26

= F (b) − F (a)
Z b Z a Z b
= f (t)dt − f (t)dt = f (t)dt.
−∞ −∞ a

(iii) Let f (·) be the p.d.f. of a continuous r.v. X and let E ⊆ R be any countable subset of R. Define g : R → [0, ∞)
by (
f (x), if x ∈ R ∩ E c ,
g(x) =
Cx , if x ∈ E,
where Cx ≥ 0 are arbitrary. Then
Z x Z x
F (x) = f (t)dt = g(t)dt ∀ x ∈ R
−∞ −∞

and, thus, g is also a p.d.f. of X. Thus, the p.d.f. of a continuous r.v. is not unique.
(iv) There are random variables that are neither discrete nor continuous (see Example 0.49). Such random variables
will not be studied here.

We state the following theorem without proof.

Theorem 0.60. Let X be a r.v. with Zd.f. F . Suppose that F is differentiable everywhere except (possibly) on a
∞
countable set E. Further suppose that F 0 (t)dt = 1. Then, X is a continuous r.v with p.d.f.
−∞
(
F 0 (x), x ∈ Ec,
f (x) =
0, x ∈ E.

Remark 0.61. (i) The p.d.f. determines the d.f. uniquely. Converse is not true. However, the d.f. determines the p.d.f.
almost uniquely (they may vary on sets that have no length (or have zero content)). Thus it is enough to study the p.d.f.
of a continuous r.v.
(ii) Let X be continuous r.v. with p.d.f. f (x). Then,
Z ∞
(a) f (x) ≥ 0 ∀ x ∈ R and (b) f (t)dt = 1.
−∞

Conversely, suppose that g : R → R is a function such that

Z ∞
(a) g(x) ≥ 0 ∀ x ∈ R, (b) g(t)dt = 1.
−∞
( )
Z x+h
Then, g(·) is the p.d.f. of some continuous r.v. having support T = x∈R: g(t)dt > 0 ∀ h > 0 .
x−h

Example 0.62. Let X be a r.v. with d.f.



 0, if x < 0,

x/4, if 0 ≤ x < 1,



F (x) = x/3, if 1 ≤ x < 2,

3x/8, if 2 ≤ x < 5/2,





1, if x ≥ 5/2.


Examine whether X is a continuous r.v. or a discrete r.v. or none?

Lecture 0 Lecture Notes 0-27

Solution: Let D be the set of discontinuity points of F . Then D = {1, 2, 5/2}. So, D 6= φ =⇒ X is not a
continuous r.v. So

X 1 1 3 2 15 11
[F (x) − F (x−)] = − + − + 1− = < 1 =⇒ Xis not a discrete r.v.
3 4 4 3 16 48
x∈D

Thus, X is neither a discrete nor a continuous r.v.

Example 0.63. Let X be a r.v with d.f.

 0, if x < 0,

x2 /2,

if 0 ≤ x < 1,
F (x) =

 x/2, if 1 ≤ x < 2,

if x ≥ 2.

1,

Show that X is a continuous r.v. Find the p.d.f. of X and support of X.

Solution: Clearly F is continuous everywhere. Moreover, F is differentiable everywhere except at two (countable)
points 1, 2, and 
 0, if x < 0,


x, if 0 < x < 1,
F 0 (x) =

 1/2, if 1 < x < 2,

if x ≥ 2.

0,
Z ∞ Z 1 Z 2
1
Also, F 0 (x)dx = xdx + dx = 1 =⇒ X is continuous r.v. with p.d.f.
−∞ 0 1 2

x,
 if 0 < x < 1,
f (x) = 1/2, if 1 < x < 2,

0, otherwise.


The support of X is
( )
Z x+h
S = {x ∈ R : F (x + h) − F (x − h) > 0 ∀ h > 0} = x∈R: f (t)dt > 0 ∀ h > 0 = [0, 2].
x−h

Example 0.64. Let X be a continuous r.v. with p.d.f.

 2
x ,
 if 0 < x < 1,
−x
f (x) = ce , if x ≥ 1, where c ≥ 0 is a constant,

0, otherwise.


(a) Find the value of c,

(b) Find P (1/2 ≤ X ≤ 2),
(c) Find the support of X,
(d) Find the d.f. of X.

Solution: (a) We have

Z b Z 1 Z ∞
2e
f (x)dx = 1 =⇒ x2 dx + ce−x dx = 1 =⇒ 1/3 + ce−1 = 1 =⇒ c = .
a 0 1 3
Lecture 0 Lecture Notes 0-28

(b) Observe that,

Z 2 Z 1 Z 2
P (1/2 ≤ X ≤ 2) = f (x)dx = 2
x dx + c e−x dx
1/2 1/2 1
1 7 2
= (1 − 1/8) + c(e−1 − e−2 ) = + (1 − e−1 ).
3 24 3
( )
Z x+h
(c) The support of X is S = x∈R: f (t)dt > 0 ∀ h > 0 = [0, ∞).
x−h
Z x
(d) The d.f. of X is F (x) = f (t)dt. For x < 0, clearly F (x) = 0. For 0 ≤ x < 1,
−∞
Z x
F (x) = t2 dt = x3 /3.
0
For x ≥ 1, Z 1 Z x
1 2
F (x) = t2 dt + c e−t dt = + (1 − e−(x−1) ).
0 1 3 3
Thus, 

 0, if x < 0,

 x3

F (x) = , if 0 ≤ x < 1,
 3
 1 + 2 (1 − e−(x−1) ), if x ≥ 1.



3 3
Remark 0.65. Let X be a continuous r.v. with p.d.f. f (·). If f is continuous at x ∈ R, then
1 x+δ/2
Z
f (x) = lim f (t)dt =⇒ P (x − δ/2 ≤ X ≤ x + δ/2) ≈ δf (x), for small δ > 0,
δ↓0 δ x−δ/2

that is, P (x − dx ≤ X ≤ x + dx) ≈ f (x)dx.

0.7. Probability Distribution of a Function of Discrete Random Variable

Let (Ω, F, P ) be a probability space and let X : Ω → R be a r.v. with d.f. F , p.m.f. f and support S. Let h : R → R
be a given function. Define Z : Ω → R as
Z(ω) = h(X(ω)), ω ∈ Ω.
Then Z is a r.v. and it is a function of r.v. X. Since we are only interested in values of random variables X and Z and
not in the original probability space (Ω, F, P ), we simply write X(ω), ω ∈ Ω as X and Z(ω), ω ∈ Ω as Z.
We have F (x) = P (X ≤ x), f (x) = P (X = x), x ∈ R, P (X ∈ S) = 1 and P (X = x) > 0 for all x ∈ S.
Define T = h(S) = {h(x) : x ∈ S}. For any set A ⊆ R, define
h−1 (A) = {x ∈ S : h(x) ∈ A}.
Then T is a countable set. Also, P (Z = z) > 0, ∀ z ∈ T (since P (X = x) > 0, ∀x ∈ S) and P (Z ∈ T ) = 1 (since
P (X ∈ S) = 1). It follows that Z is a discrete r.v. Moreover, for z ∈ T ,
X X X
P (Z = z) = P (h(X) = z) = P (X = x) = P (X = x) = f (x),
{x∈S:h(x)=z} x∈h−1 ({z}) x∈h−1 (z)

and for any z 6∈ T , P (Z = z) = 0. Thus, we have the following theorem:

Lecture 0 Lecture Notes 0-29

Theorem 0.66. Let X be a discrete r.v. with support S, d.f. F and p.m.f. f . Let h : R → R be a given function. Then,
Z = h(X) is a discrete r.v. with support T = {h(x) : x ∈ S} and p.m.f.
 X

 f (x), if z ∈ T,
g(z) = x∈h ({z})
−1

0, otherwise,


and d.f. X X X
G(z) = P (Z ≤ z) = g(t) = f (x) = f (x).
{t∈T :t≤z} {x∈S:h(x)≤z} x∈h−1 ((−∞,z])∩S

In particular, if h : S → R is one-one then

(
f (h−1 (z)), if z ∈ T,
g(z) =
0, otherwise.

Example 0.67. Let X be a discrete r.v. with p.m.f.


1/7, if x ∈ {−2, −1, 0, 1},

f (x) = 3/14, if x ∈ {2, 3},

0, otherwise.


Find the p.m.f. and d.f. of Y = X 2 .

Solution: Here, the support of X is S = {−2, −1, 0, 1, 2, 3}. By Theorem 0.66, Y = X 2 is discrete r.v. with support
T = {0, 1, 4, 9} and p.m.f.
 

 P (X = 0), if z = 0, 
 1/7, if z = 0,
 
P (X = −1) + P (X = 1), if z = 1, 2/7, if z = 1,

 

 
2
g(z) = P (X = z) = P (X = −2) + P (X = 2), if z = 4, = 5/14, if z = 4,
 
P (X = 3), if z = 9, 3/14, if z = 9,

 

 
 
0, otherwise. 0, otherwise.
 

The d.f. of Y is 

 0, if z < 0

1/7, if 0 ≤ z < 1



G(z) = P (Y ≤ z) = 3/7, if 1 ≤ z < 4

11/14, if 4 ≤ z < 9





1, if z ≥ 9.


Example 0.68. In Example 0.67, directly find the d.f. of Y = X 2 (i.e. find d.f. of Y before finding the p.m.f. of Y ).
Hence find the p.m.f. of Y .

Solution: By Theorem 0.66, Y is a discrete r.v. with support T = {0, 1, 4, 9}. Thus the d.f. of Y is


 0, z < 0,

P (X 2 = 0), 0 ≤ z < 1,



G(z) = P (Y ≤ z) = P (X 2 ≤ z) = P (X 2 = 0) + P (X 2 = 1), 1 ≤ z < 4,
 2 2 2
P (X = 0) + P (X = 1) + P (X = 4), 4 ≤ z < 9,




1, z ≥ 9.

Lecture 0 Lecture Notes 0-30

 
0, z < 0,  0, z < 0,
 
1
 


7
 , 0 ≤ z < 1, 


 1/7, 0 ≤ z < 1,
1 1 1
= 7 + 7 + 7 , 1 ≤ z < 4, = 3/7, 1 ≤ z < 4,
 
1 1 1 1 3
+ 7 + 7 + 7 + 14 , 4 ≤ z < 9,  11/14, 4 ≤ z < 9,

 
7

 

 
z ≥ 9. 1, z ≥ 9.
 
1,

The p.m.f. of Y is 

 1/7, if z = 0,

2/7, if z = 1,
( 

G(z) − G(z−), if z ∈ T,

g(z) = = 5/14, if z = 4,
0, otherwise. 
3/14, if z = 9,





0, otherwise.


0.8. Probability Distribution of a Function of Continuous Random Variable

Z x+h
Let X be a continuous r.v. with d.f. F , p.d.f. f (·) and support S = {x ∈ R : F (x + h) − F (x − h) = f (t)dt >
x−h
0, ∀ h > 0}. For convenience assume that S = [a, b] and {x ∈ R : f (x) > 0} = (a, b), for some −∞ ≤ a < b ≤ ∞
(with the convention that [−∞, b] ≡ (−∞, b), ∀ b ∈ R, [a, ∞] ≡ (a, ∞), ∀ a ∈ R and [−∞, ∞] ≡ (−∞, ∞)).
Let h : R → R be a function such that h is strictly monotone and differentiable function on S. Then Z = h(X) is a
r.v. with d.f. G(z) = P (Z ≤ z) = P (h(X) ≤ z), z ∈ R.
For any sets A ⊆ R and B ⊆ R, define h(A) = {h(x) : x ∈ A} and h−1 (B) = {x ∈ R : h(x) ∈ B}. Clearly
P (X ∈ (a, b)) = 1 and therefore P (h(X) ∈ h((a, b)) = 1). Consider the following cases:
Case I: h(·) is strictly increasing on S
We have P (h(a) < Z < h(b)) = 1. Therefore, for z < h(a), P (Z ≤ z) = 0 and for z ≥ h(b), P (Z ≤ z) = 1. For
h(a) < z < h(b),
Z h−1 (z) Z h−1 (z) Z z
d −1
G(z) = P (h(X) ≤ z) = P (X ≤ h−1 (z)) = f (t)dt = f (t)dt = f (h−1 (y)) h (y) dy.
−∞ a h(a) dy

Thus, 
0, if z < h(a),

Z z

d −1
G(z) = f (h−1 (y)) h (y) dy, if h(a) ≤ z < h(b),

 h(a) dy

1, if z ≥ h(b).


Since f is continuous on (a, b) it follows that G(z) is differentiable everywhere except possibly at z = h(a) and
z = h(b). Moreover, 
f (h−1 (z)) d h−1 (z) , if h(a) < z < h(b),

0 dz
G (z) =

0, otherwise,

and Z ∞ Z h(b) Z b
d −1
G0 (z)dz = f (h−1 (z)) h (z) dz = f (t)dt = 1.
−∞ h(a) dz a
Lecture 0 Lecture Notes 0-31

It follows that Z is a continuous r.v. with p.d.f.


f (h−1 (z)) d h−1 (z) , if h(a) < z < h(b),

g(z) = dz

0, otherwise

and support T = [h(a), h(b)].

Case II: h(·) is strictly decreasing on S
Here P (h(b) < h(X) < h(a)) = 1 and G(z) = P (h(X) ≤ z), z ∈ R. Clearly, for z < h(b), G(z) = 0 and for
z ≥ h(a), G(z) = 1. For h(b) < z < h(a),
Z ∞ Z b Z z
−1 d −1
G(z) = P (X ≥ h (z)) = f (t)dt = f (t)dt = f (h−1 (y)) h (y) dy.
h−1 (z) h−1 (z) h(b) dy

Thus, 
 0, if z < h(b),

Z z
 d −1
G(z) = f (h−1 (y)) h (y) dy, if h(b) ≤ z < h(a),

 h(b) dy

1, if z ≥ h(a).


Since f is continuous on (a, b), it follows that G(·) is differentiable everywhere except possibly at h(a) and h(b).
Moreover, 
f (h−1 (z)) d h−1 (z) , if h(b) < z < h(a),

0 dz
G (z) =

0, otherwise

and Z ∞ Z h(a) Z b
d −1
G0 (z)dz = f (h−1 (z)) h (z) dz = f (t)dt = 1.
−∞ h(b) dz a

Consequently, Z is a continuous r.v. with p.d.f.


f (h−1 (z)) d h−1 (z) , if h(b) < z < h(a),

g(z) = dz

0, otherwise

and support T = [h(b), h(a)].

Combining Case I and Case II, we get the following result:
Theorem 0.69. Let X be a continuous r.v. with p.d.f. f (·) and support S = [a, b] for some −∞ ≤ a < b ≤ ∞.
Suppose that {x ∈ R : f (x) > 0} = (a, b) and that f is continuous on (a, b). Let h : R → R be a function that is
differentiable and strictly monotone on (a, b). Then, Z = h(X) is a continuous r.v. with p.d.f.

f (h−1 (z)) d h−1 (z) , if z ∈ h((a, b)),

dz

0, otherwise,

and support T = [min{h(a), h(b)}, max{h(a), h(b)}].

The following theorem is a generalization of the above result and can be proved on similar lines.
Lecture 0 Lecture Notes 0-32

S
Theorem 0.70. Let X be a continuous r.v. with p.d.f. f (·) and support S =S i∈Λ [ai , bi ], where Λ is a countable set
and [ai , bi ]’s are disjoint intervals. Suppose that {x ∈ R : f (x) > 0} = i∈Λ (ai , bi ) and that f is continuous in
each (ai , bi ), i ∈ Λ. Let h : R → R be a function that is differentiable and strictly monotone in each (ai , bi ), i ∈ Λ (h
may be monotonic increasing in some (ai , bi ) and monotonic decreasing in some (ai , bi )). Let h−1 i (·) be the inverse
function of hi on (ai , bi ), i ∈ Λ. Then, Z = h(X) is a continuous r.v. with p.d.f.
(
X
−1 d −1 1, z ∈ hj ((aj , bj )),
g(z) = f (hj (z)) hj (z) Ihj ((aj ,bj )) (z), where Ihj ((aj ,bj )) (z) = .
dz 0, otherwise.
j∈Λ

Remark 0.71. Theorem 0.69 and Theorem 0.70 hold even in situations where the function h is differentiable every-
where except possibly at a finite number of points in S.
Example 0.72. Let X be a r.v. with p.d.f.
(
3x2 , 0 < x < 1,
f (x) =
0, otherwise.

Find the p.d.f. and d.f. of Y = 1/X 2 . What is the support of d.f. of Y .

Solution: The support of F is [0, 1] and {x ∈ R : f (x) > 0} = (0, 1). Moreover, f is continuous on (0, 1) and
h(x) = 1/x2 is differentiable and strictly monotone on (0, 1).
h((0, 1)) = (1, ∞). Now

1 1 1 d −1 1
y= =⇒ x = √ , i .e., h−1 (y) = √ =⇒ h (y) = − √ , y ∈ (1, ∞).
x2 y y dy 2y y

Thus, Y = 1/X 2 is continuous r.v. with p.d.f. g(y) given by

d −1
g(y) = f (h−1 (y)) h (y) Ih((0,1)) (y)
dy
d −1
= f (h−1 (y)) h (y) I(1,∞) (y)
dy
3
 · 1√ , if y > 1,  3√ , if y > 1,

2
= y 2y y = 2y y
 
0, otherwise, 0, otherwise.

The d.f. of Y is
 
Z y 0, if y < 1, 0, if y < 1,
Z y
G(y) = g(t)dt = 3 = 1
−∞  √ dt, if y > 1, 1 − , if y > 1.
2 t y 3/2
1 2t

Clearly the support of G is [1, ∞).

Example 0.73. Let X be r.v. with p.d.f.

|x|/2, if − 1 < x < 1,

f (x) = x/3, if 1 ≤ x ≤ 2,

0, otherwise


and let Y = X 2 .
Lecture 0 Lecture Notes 0-33

(a) Find the p.d.f. of Y directly and hence find the d.f. of Y .
(b) Find the d.f. of Y and hence find the p.d.f. of Y .
(c) Find the support of d.f. of Y .

Solution: (a) The support of F is S = [−1, 2] and we may take S = [−1, 0] ∪ [0, 2], {x ∈ R : f (x) > 0} =
(−1, 0) ∪ (0, 2). The p.d.f. f is continuous on (−1, 0) ∪ (0, 1) ∪ (1, 2), h(x) = x2 is differentiable on (−1, 0) ∪ (0, 2),
h(·) is strictly decreasing on (−1, 0) and strictly increasing on (0, 2).
√
h(x) = x2 is strictly decreasing on S1 = (−1, 0) with inverse function h−1 1 (y) = − y, y ∈ (0, 1), h(S1 ) = (0, 1).
√
h(x) = x2 is strictly increasing on S2 = (0, 2) with inverse function h−1 2 (y) = y, y ∈ (0, 4), h(S2 ) = (0, 4).
Thus, Y = X 2 is a continuous r.v. with p.d.f.

d −1 d −1
g(y) = f (h−1
1 (y)) h (y) I(0,1) (y) + f (h−1
2 (y)) h (y) I(0,4) (y)
dy 1 dy 2
√ −1 √ 1
= f (− y) √ I(0,1) (y) + f ( y) √ I(0,4) (y)
2 y 2 y
1 √ √
= √ f (− y)I(0,1) (y) + f ( y)I(0,4) (y)
2 y
1
 2 , if 0 < y < 1,

= 61 , if 1 < y < 4,

0, otherwise.


The d.f. of Y is

0, if y < 0, 


Z y  0, if y < 0,
dt
 


, if 0 ≤ y < 1,

 y
 , if 0 ≤ y < 1,
Z y 
 


0 2 2
G(y) = P (X 2 ≤ y) = g(t)dt = Z 1 Z y = y+2
−∞  dt dt 
, if 1 ≤ y < 4,

 + , if 1 ≤ y < 4, 

6
2 1 6

 

 0 
1, if y ≥ 4.

 
1, if y ≥ 4.


(b) The d.f. of Y is

(
2 0, if y < 0,
G(y) = P (X ≤ y) = √ √ .
P {− y ≤ X ≤ y}, if y > 0.
For 0 ≤ y < 1, √
y
√ √ |x|
Z
y
G(y) = P {− y ≤ X ≤ y} = √
dx = .
− y 2 2
√ √
For 1 ≤ y < 4 (so that −2 < − y ≤ −1 and 1 ≤ y ≤ 2)
√
1 y
√ √ |x|
Z Z
x y+2
G(y) = P {− y ≤ X ≤ y} = dx + dx = .
−1 2 1 3 6
Lecture 0 Lecture Notes 0-34

For y ≥ 4, G(y) = 1. Therefore 


 0, if y < 0,

 y
 , if 0 ≤ y < 1,


2
G(y) = y + 2
, if 1 ≤ y < 4,


6




1, if y ≥ 4.


Clearly G is differentiable everywhere except at finite number of points (0,1 and 4) and we may take

1/2, if 0 < y < 1,

0
G (y) = 1/6, if 1 < y < 4,

0, otherwise.


Z ∞ Z 1 Z 4
1 1
Moreover, G0 (y)dy = dy + dy = 1. Thus, Y is a continuous r.v. with p.d.f.
−∞ 0 2 1 6

1/2, if 0 < y < 1,

g(y) = 1/6, if 1 < y < 4,

0, otherwise.


(c) The support of G is [0, 4].

0.9. Expectation (or Mean) of Random Variables

Let X be a discrete r.v. with p.m.f. f (·) and support S. For any x ∈ S, f (x) gives an idea about proportion
P of
times we will observe the event {X = x} if the experiment is repeated a large number of times. Thus x∈S xf (x)
represents the mean (or expected) value of r.v. X if the experiment is repeated a large number of times.
Z ∞
Similarly, if X is a continuous r.v. with p.d.f. f (·) then xf (x)dx (provided the integral is finite) represents the
−∞
mean (or expected) value of r.v. X.
Definition 0.74. (a) Let X be a discrete r.v. with p.m.f. f (·) and support S. We say that the expected value of X (or
the mean of X, which we denote by E(X)) is finite and equals
X X
E(X) = xf (x), provided |x|f (x) < ∞.
x∈S x∈S

(b) Let X be a continuous r.v. with p.d.f. f (·) and support S. We say that the expected value of X (or the mean of X,
which we denote by E(X)) is finite and equals
Z ∞ Z ∞
E(X) = xf (x)dx, provided |x|f (x)dx < ∞.
−∞ −∞

Example 0.75. (a) Let X be a discrete r.v. with p.m.f.

 1 , if x ∈ {1, 2, 3, . . . },


f (x) = 2x .
0, otherwise.


Show that E(X) is finite. Find E(X).

Lecture 0 Lecture Notes 0-35

(b) Let X be a r.v. with p.m.f.

 3 , if x ∈ {±1, ±2, . . . },


f (x) = π 2 x2 .
0, otherwise.


Show that E(X) is not finite.

e−|x|
(c) Let X be a continuous r.v. with p.d.f. f (x) = , −∞ < x < ∞. Show that E(X) is finite. Find E(X).
2
1
(d) Let X be a continuous r.v. with p.d.f. f (x) = , −∞ < x < ∞. Show that E(X) is not finite.
π(1 + x2 )

Solution: (a) The support of the distribution is S = {1, 2, . . . }. Also,

∞ ∞
X X n X
|x|f (x) = = an (say),
n=1
2n n=1
x∈S

n
where an = > 0, ∀ n = 1, 2, . . . and
2n
an+1 n+1 1
= → < 1, as n → ∞.
an 2n 2
P P∞ n
Thus by the ratio test x∈S |x|f (x) = n=1 < ∞. It can be seen that E(X) = 2 (Exercise).
2n
(b) Here the support of the distribution is S = {±1, ±2, . . . }.
∞
X 6 X1
|x|f (x) = = ∞ =⇒ E(X) is not finite.
π 2 n=1 n
x∈SX

and
∞ ∞
e−|x|
Z Z
E(X) = xf (x)dx = x dx = 0.
−∞ −∞ 2
(d) We have
Z ∞ Z ∞ Z ∞
1 2 x
|x|f (x)dx = |x| 2
dx = dx = ∞ =⇒ E(X) is not finite.
−∞ −∞ π(1 + x ) π 0 1 + x2

Example 0.76 (St. Petersburg Paradox). To make some money a gambler plays a sequence of fair games with the
following strategy:
In the first bet he bet Rs. 1 million. If the first bet is lost he doubles his bet in the second game. He keeps on doubling
his bet until he wins a game. If the gambler has not won by the mth trial he bets Rs. 2m million in the (m + 1)th
game. If he wins in kth game then

Investment=1 + 2 + 4 + · · · + 2k−1 = 2k − 1 million rupee, win=2k million rupee.

Total earning if he wins on the kth game= 1 million rupee.

Lecture 0 Lecture Notes 0-36

The above scheme seems to be foolproof for earning Rs. 1 million rupee. By this logic all gamblers should be
billionaries!
X : the amount of money bet on the last game (the game he wins). Then
∞
1 X 2k
P (X = 2k ) = k+1
, k = 0, 1, 2, . . . , E(X) = = ∞ (E(X) is not finite).
2 2k+1
k=0

This implies enormous amount of money would be required.

Theorem 0.77. Let X be a continuous (discrete) r.v. Then
Z ∞ Z 0
E(X) = P (X > y)dy − P (X < y)dy,
0 −∞

provided E(X) is finite.

Proof. We will provide the proof for the case when X is a continuous r.v. with p.d.f., say f . We have
Z ∞
E(X) = xf (x)dx
−∞
Z 0 Z ∞
= xf (x)dx + xf (x)dx
−∞ 0
Z 0 Z 0 Z ∞ Z x
=− f (x)dydx + f (x)dydx
−∞ x 0 0
Z 0 Z y Z ∞ Z ∞ Z 0 Z ∞
=− f (x)dxdy + f (x)dxdy = − P (X < y)dy + P (X > y)dy.
−∞ −∞ 0 y −∞ 0

This completes the proof.

Corollary 0.78. (a) Suppose that X is a continuous (discrete) r.v. with P (X ≥ 0) = 1. Then
Z ∞
E(X) = P (X > y)dy.
0

(b) Suppose that P (X ∈ {0, ±1, ±2, . . . }) = 1. Then

∞
X ∞
X
E(X) = P (X ≥ n) − P (X ≤ −n).
n=1 n=1

P∞
(c) Suppose that P (X ∈ {0, 1, 2, . . . }) = 1. Then E(X) = n=1 P (X ≥ n).

Proof. Exercise.

The following theorem suggests that for any r.v. X and any function h : R → R, E(h(X)) can be directly found using
p.m.f. / p.d.f. of X.
Theorem 0.79. (a) Let X be a discrete r.v. with p.m.f. f (·) and support S. Let h : R → R be a given function and let
Z = h(X). Then X X
E(Z) = h(x)f (x) provided |h(x)|f (x) < ∞.
x∈S x∈S
Lecture 0 Lecture Notes 0-37

(b) Let X be a continuous r.v. with p.d.f. f (·) and let h : R → R be a given function. If Z = h(X), then
Z ∞ Z ∞
E(Z) = h(x)f (x)dx, provided |h(x)|f (x)dx < ∞.
−∞ −∞

Proof. We will provide the proof of (a) only. The proof of (b) follows on similar lines. The support of Z = h(X) is
T = h(S). We have
X X
E(Z) = tP (Z = t) = tP (h(X) = t)
t∈T t∈T
X X
= t P (X = x)
t∈T {x∈S:h(x)=t}
X X
= tP (X = x)
{x∈S:h(x)=t} t∈T
X X
= h(x)P (X = x)
{x∈S:h(x)=t} t∈T
X X
= h(x)P (X = x)
t∈T {x∈S:h(x)=t}
X X
= h(x)P (X = x) = h(x)P (X = x).
S
{x∈S:h(x)=t} x∈S
t∈T

This completes the proof.

Example 0.80. (a) Let the r.v. X have the p.m.f.

(
1/6, if x = −2, −1, 0, 1, 2, 3,
f (x) =
0, otherwise.

Find E(X 2 ).
(b) Let the r.v. X have the p.d.f. (
2x, if 0 < x < 1,
f (x) =
0, otherwise.

Find E(X 3 ).

X 1 1 1 1 1 1 19
Solution: (a) E(X 2 ) = x2 f (x) = 4 × +1× +0× +1× +4× +9× = .
6 6 6 6 6 6 6
x∈S
Z ∞ Z 1
3 3 2
(b) E(X ) = x f (x)dx = 2 x4 dx = .
−∞ 0 5
Theorem 0.81. Let X be a discrete or continuous r.v. with p.m.f./ p.d.f. f and support S. Let hi : R → R,
i = 1, 2, . . . , m be given functions.
(a) Then, for real constants c1 , c2 , . . . , cm
m
! m
X X
E ci hi (X) = ci E(hi (X)),
i=1 i=1
Lecture 0 Lecture Notes 0-38

provided involved expectations are finite.

(b) Let h1 (x) ≤ h2 (x), ∀ x ∈ S. Then,

E(h1 (X)) ≤ E(h2 (X)), provided involved expectations are finite.

In particular, if E(X) is finite and P (a ≤ X ≤ b) = 1, for some real constants a and b (a < b) then a ≤ E(X) ≤ b.
(c) If P (X ≥ 0) = 1 and E(X) = 0, then P (X = 0) = 1.
(d) If E(X) is finite then |E(X)| ≤ E(|X|).
(e) Let a and b be two real constants. Then,

E(aX + b) = aE(X) + b, provided involved expectations are finite.

Proof. The proofs of (a), (b) and (e) follows from the definition of expectation of a r.v.
(c) We will provide the proof for the case when X is a continuous r.v. Then
∞ !
[ 1
P (X > 0) = P X≥
n=1
n

1 1
= lim P X ≥ , X≥ ↑
n→∞ n n
Z ∞
= lim f (x)dx
n→∞ 1/n
Z∞
≤ lim nxf (x)dx, (x ∈ [1/n, ∞) =⇒ nx ≥ 1)
n→∞ 1/n
Z ∞
≤ lim n xf (x)dx
n→∞ 0
= lim [nE(X)] = 0 =⇒ P (X = 0) = 1.
n→∞

(d) We have
−|X| ≤ X ≤ |X| =⇒ E(−|X|) ≤ E(X) ≤ E(|X|) =⇒ |E(X)| ≤ E(|X|).
This completes the proof.

Some Special Expectations:

(i) h(x) = x, E(X) = µ01 = mean of X.
(ii) h(x) = xr , r = {1, 2, . . . }, E(X r ) = µ0r = rth moment of X about origin.
(iii) h(x) = |x|r , r = {1, 2, . . . }, E(|X|r ) = rth absolute moment of X about origin.
(iv) h(x) = (x − µ01 )r , r = {1, 2, . . . }, E(X − µ01 )r = µr = rth moment of X about its mean or rth central moment.
√
(v) µ2 = E(X − µ01 )2 = σ 2 = variance of X. We also denote it by Var(X). And, µ2 = E(X − µ01 )2 = σ is
p

called the standard deviation of X (positive square root of the variance of r.v. X).
Remark 0.82. (i) Var(X) = σ 2 = E(X − µ01 )2 = E(X 2 − 2µ01 X + (µ01 )2 ) = E(X 2 ) − 2(µ01 )2 + (µ01 )2 =
E(X 2 ) − (E(X))2 .
Lecture 0 Lecture Notes 0-39

(ii) Since (X − µ01 )2 ≥ 0, we have

Var(X) = E(X − µ01 )2 ≥ 0 =⇒ E(X 2 ) ≥ (E(X))2 .

(iii) Var(X) = 0 ⇐⇒ E(X − µ01 )2 = 0 ⇐⇒ P (X = E(X)) = 1.

Theorem 0.83. Let X be a r.v. such that E(|X|s ) < ∞, for some s > 0. Then, E(|X|r ) < ∞, ∀ 0 < r < s.

Proof. Note that |X|r ≤ max{|X|s , 1} ≤ |X|s + 1. This implies that E(|X|r ) ≤ E(|X|s + 1) = E(|X|s ) + 1 < ∞.
Thus, the result follows.

0.10. Moment Generating Function

Let X be a r.v with d.f. F and p.d.f. / p.m.f. f (·).

Definition 0.84. We say that the moment generating function (m.g.f.) of X (denoted by MX (·)) exists and equals

MX (t) = E(etX ), provided E(etX ) is finite in (−h, h) for some h > 0.

Remark 0.85. (i) MX (0) = 1, thus A = {t ∈ R : E(etX ) is finite} =

6 φ.
(ii) MX (t) > 0, ∀ t ∈ A = {s ∈ R : E(esX ) is finite}.
(iii) Suppose that MX (t) exists and is finite on (−h,h) for some
h > 0. For real constants c and d, let Y = cX + d.
h h
Then, the m.g.f. of Y also exists and is finite on − , (with the convention that ± a0 = ±∞, if a > 0).
|c| |c|
Moreover,
h h
MY (t) = E(et(cX+d) ) = etd MX (ct), t ∈ − , .
|c| |c|

(iv) The name m.g.f. to the transform MX is motivated by the fact that MX can be used to generate moments of any
r.v., as illustrated in the following theorem.
Theorem 0.86. Let X be a r.v. with m.g.f. MX that is finite on (−h, h), h > 0. Then,
(a) For each r ∈ {1, 2, . . . }, µ0r = E(X r ) is finite;
(r)
(b) For each r ∈ {1, 2, . . . }, µ0r = E(X r ) = MX (0), where
r
(r) d
MX (0) = MX (t) , the rth derivative of MX at the point 0;
dtr t=0

∞ r
X t tr
(c) MX (t) = µ0r , t ∈ (−h, h), so that µ0r is equal to coefficient of (r = 1, 2, . . . ) in the Maclaurin’s series
r!
r=0
r!
expansion of MX (t) around t = 0.

Proof. (a) We have

E(etX ) < ∞, ∀ t ∈ (−h, h)

Z 0 Z ∞
tx
=⇒ e f (x)dx < ∞ ∀ t ∈ (−h, h) and etx f (x)dx < ∞ ∀ t ∈ (−h, h)
−∞ 0
Lecture 0 Lecture Notes 0-40

Z 0 Z ∞
=⇒ e−t|x| f (x)dx < ∞ ∀ t ∈ (−h, h) and et|x| f (x)dx < ∞ ∀ t ∈ (−h, h)
−∞ 0
Z 0 Z ∞
=⇒ e|t||x| f (x)dx < ∞ ∀ t ∈ (−h, h) and e|t||x| f (x)dx < ∞ ∀ t ∈ (−h, h)
−∞ 0
Z ∞
|tx|
=⇒ e f (x)dx < ∞ ∀ t ∈ (−h, h);
−∞

here f (·) denotes the p.d.f. of r.v. X.

|x|r
Fix r ∈ {1, 2, . . . } and t ∈ (−h, h) − {0}. Then, lim = 0 and therefore ∃ a positive real number Ar,t such
|x|→∞ e|tx|
that |x|r < e|tx| , ∀ |x| > Ar,t . Therefore
Z ∞
r
E(|X| ) = |x|r f (x)dx
−∞
Z Z
= |x|r f (x)dx + |x|r f (x)dx
|x|≤Ar,t |x|>Ar,t
Z Z
≤ Arr,t f (x)dx + e|tx| f (x)dx
|x|≤Ar,t |x|>Ar,t
Z ∞
≤ Arr,t + e|tx| f (x)dx < ∞, r = 1, 2, . . . .
−∞

∞ ∞
dr
Z Z
tx (r)
(b) MX (t) = e f (x)dx, MX (t) = r etx f (x)dx, r = 1, 2, . . . .
−∞ dt −∞

Using the arguments of advanced calculus it can be shown that of MX (t) = E(etX ) < ∞, ∀ t ∈ (−h, h), then the
derivative can be passed through the integral sign. Therefore,
Z ∞ r Z ∞
(r) d tx
xr etx f (x)dx, r = 1, 2, . . .

MX (t) = r
e f (x) dx =
−∞ dt −∞

and Z ∞
(r)
MX (0) = xr f (x)dx = E(X r ).
−∞

∞ r r
!
Z ∞ Z ∞
tx
X t x
(c) MX (t) = e f (x)dx = f (x)dx.
−∞ −∞ r=0
r!

Under the assumption that MX (t) = E(etX ) < ∞, ∀ t ∈ (−h, h), using arguments of advanced calculus, it can be
shown that the summation sign can be passed through the integral sign. Thus,
∞ r Z ∞ ∞ r
X t X t
MX (t) = xr f (x)dx = E(X r ), r = 1, 2, . . . .
r=0
r! −∞ r=0
r!

This completes the proof.

Corollary 0.87. Under the notation and assumption of the above theorem define ψX (t) = ln(MX (t)), t ∈ (−h, h).
Then,
(1) (2)
µ01 = µ = E(X) = ψX (0) and µ2 = σ 2 = Var(X) = ψX (0).
Lecture 0 Lecture Notes 0-41

Proof. For t ∈ (−h, h)

(1)
(1) MX (t) (1) (1)
ψX (t) = =⇒ ψX (0) = MX (0) = E(X) (since MX (0) = 1).
MX (t)
Also,
(2) (1)
(2) MX (t)MX (t) − (MX (t))2 (2) (2) (1)
ψX (t) = =⇒ ψX (0) = MX (0) − (MX (0))2 = E(X 2 ) − (E(X))2 = Var(X).
(MX (t))2
This completes the proof.
Example 0.88. (a) Let X be a discrete r.v. with p.m.f.
 −λ x
e λ
, x = 0, 1, 2, . . . ,
fX (x) = P (X = x) = x!
0,

otherwise,

where λ > 0. Show that the m.g.f. of X exists and is finite on whole R. Find MX (t), mean, variance of X and E(X 3 ).
(b) Let X be a continuous r.v. with p.d.f.
(
λe−λx , x > 0,
fX (x) =
0, otherwise,

where λ > 0. Find m.g.f., mean, variance of X and E(X r ), r = 1, 2, . . . (provided they exist).
1
(c) Let X be a continuous r.v. having the p.d.f. f (x) = , −∞ < x < ∞ (called Cauchy p.d.f. and
π(1 + x2 )
corresponding probability distribution is called Cauchy distribution). Show that the m.g.f. of X does not exist.

Solution: (a) We have

∞ ∞
X e−λ λx X (λet )x t t
MX (t) = etx = e−λ = e−λ eλe = eλ(e −1) , ∀ t ∈ R.
x=0
x! x=0
x!
t
−1)
Thus, m.g.f. of X exists and finite on whole of R and MX (t) = eλ(e , t ∈ R.
(1) (2)
Now ψX (t) = ln(MX (t)) = λ(et − 1) =⇒ ψX (t) = λet = ψX (t), ∀ t ∈ R.
(1) (2)
Thus, E(X) = ψX (0) = λ and Var(X) = ψX (0) = λ. Again,
(1) t (1)
−1)
MX (t) = λet eλ(e = λet MX (t) =⇒ MX (0) = E(X) = λ,
(2) (1) (2)
MX (t) = λet MX (t) + λet MX (t) =⇒ MX (0) = E(X 2 ) = λ2 + λ,
(3) (2) (1) (3)
MX (t) = λet MX (t) + 2λet MX (t) + λet MX (t) =⇒ MX (0) = E(X 3 ) = λ3 + 3λ2 + λ.

Alternatively, for t ∈ R,
t
−1)
MX (t) = eλ(e
λ2 (et − 1)2 λ3 (et − 1)3
= 1 + λ(et − 1) + + + ···
2! 3!
   2  3
∞ j 2 ∞ j 3 ∞ j
X t λ X t λ X t  + ···
= 1 + λ +   + 
j=1
j! 2! j=1
j! 3! j=1
j!
Lecture 0 Lecture Notes 0-42

λ2 2λ2 λ3

2 λ 3 λ
= 1 + λt + t + +t + + + ···
2! 2! 3! (2!)2 3!
Thus,

E(X) = coefficient of t in the expansion of MX (t) = λ,

t2
E(X 2 ) = coefficient of in the expansion of MX (t) = λ2 + λ,
2!
t3
E(X 3 ) = coefficient of in the expansion of MX (t) = λ3 + 3λ2 + λ.
3!
Z ∞ Z ∞
(b) etx fX (x)dx = λ e−λ(1−t/λ)x dx < ∞, if t < λ. Thus the m.g.f. of X exists and, for t < λ,
−∞ −∞
−1
t2 tr

t t
MX (t) = 1− =1+ + 2 + ··· + r + ··· .
λ λ λ λ
For r = 1, 2, . . .
tr r!
µ0r = E(X r ) = coefficient of in the expansion of MX (t) = r , r ∈ {1, 2, . . . }.
r! λ
Alternatively,
−2 −3 −(r+1)
(1) 1 t (2) 2 t (r) r! t
MX (t) = 1− , MX (t) = 2 1 − and MX (t) = r 1 − , t < λ.
λ λ λ λ λ λ
This implies
(r) r! 2 1 1
E(X r ) = MX (0) = , r = 1, 2, . . . and Var(X) = 2 − 2 = 2 .
λr λ λ λ

(c) Since E(X) is not finite, the m.g.f. of X does not exist.
Definition 0.89 (Equality in Distribution). Let X and Y be two r.v.’s with d.f.’s FX and FY , respectively. We say that
d
X and Y have the same distribution (written as X = Y ) if FX (x) = FY (x), ∀ x ∈ R.
Remark 0.90. (i) Let X and Y be two discrete r.v.’s with p.m.f.’s fX and fY , respectively. Then,
d
X = Y ⇐⇒ fX (x) = fY (x), ∀ x ∈ R.
d
(ii) Let X and Y be two continuous r.v.’s. Then, X = Y iff there exist versions of p.d.f.’s fX and fY of X and Y ,
respectively, such that fX (x) = fY (x), ∀ x ∈ R.
d d
(iii) Suppose X = Y , then for any Borel measurable function h : R → R, h(X) = h(Y ) and hence E(h(X)) =
E(h(Y )).
d
Theorem 0.91. Let X and Y be r.v.’s such that for some c > 0, MX (t) = MY (t), ∀ t ∈ (−c, c). Then, X = Y .

Proof. Special Case: Suppose that X and Y are discrete r.v.’s with support SX = SY = {1, 2, . . . }, pk = P (X = k)
and qk = P (Y = k), k = 1, 2, . . . . Then

MX (t) = MY (t), ∀ t ∈ (−c, c), for some c > 0

X∞ ∞
X
=⇒ ekt pk = ekt qk ∀ t ∈ (−c, c)
k=1 k=1
Lecture 0 Lecture Notes 0-43

∞
X ∞
X
=⇒ Λk pk = Λk qk ∀ Λ ∈ (e−c , ec )
k=1 k=1
=⇒ pk = qk ∀ k = 1, 2, . . . ,

d
since if two power series are equal over an interval then their coefficients are the same. Thus, X = Y .
Example 0.92. For any p ∈ (0, 1) and positive integer n, let Xp,n be a discrete r.v. with p.m.f.

 n px (1 − p)n−x , if x = {0, 1, . . . , n},

fp,n (x) = x

0, otherwise.

Here, p ∈ (0, 1) and n ∈ N. (Such a r.v. or probability distribution is called binomial r.v. or distribution with n trials
d
and probability of success p). Define Yp,n = n − Xp,n . Using the m.g.f. of Xp,n , show that Yp,n = X1−p,n . Find
E(X1/2,n ).

Solution: We have
n n
X n x X n
MXp,n (t) = E etXp,n = etx p (1 − p)n−x = (et p)x (1 − p)n−x = (1 − p + pet )n , t ∈ R.
x=0
x x=0
x

Now

MYp,n (t) = E etYp,n = E et(n−Xp,n )

= ent MXp,n (−t) = ent (1 − p + pe−t )n

= (p + (1 − p)et )n = (1 − (1 − p) + (1 − p)et )n = MX1−p,n (t) ∀ t ∈ R.

d
Thus, Yp,n = X1−p,n .
Alternatively,

fYp,n (y) = P (Yp,n = y)

= P (Xp,n = n − y)

n
pn−y (1 − p)n−(n−y) , if n − y = {0, 1, . . . , n},


= n−y

0, otherwise.

 n (1 − p)y (1 − (1 − p))n−y , if y = {0, 1, . . . , n},

= y

0, otherwise.
= fX1−p,n (y) ∀ y ∈ R.

d
Thus, Yp,n = X1−p,n .
d
Now for p = 1/2, X1/2,n = n − X1/2,n . Thus, E(X1/2,n ) = E(n − X1/2,n ) =⇒ E(X1/2,n ) = n/2.

e−|x| d
Example 0.93. Let X be a r.v. with p.d.f. fX (x) = , −∞ < x < ∞ and let Y = −X. Show that Y = X and
2
hence show that E(X) = 0.
Lecture 0 Lecture Notes 0-44

Solution: We have
∞ ∞
e−|x| e−|x|
Z Z
MY (t) = E(etY ) = E(e−tX ) = e−tx dx = etx dx = MX (t) ∀ t ∈ (−1, 1).
−∞ 2 −∞ 2

"
∞ 0 ∞
e−|x| ex e−x
Z Z Z
MX (t) = etx dx = etx dx + etx dx
−∞ 2 −∞ 2 0 2
Z ∞ Z ∞
1
= e−(1+t)x dx + e−(1−t)x dx
2 0 0
#
1 1 1 1 d
= + = ∀ t ∈ (−1, 1) =⇒ X = Y.
2 1+t 1−t 1 − t2

Alternatively, the p.d.f. of Y is

e−|y|/2 d
= fX (y) ∀ − ∞ < y < ∞ =⇒ X = Y.
fY (y) =
2
Z ∞
Thus, E(Y ) = E(X) =⇒ E(−X) = E(X) =⇒ E(X) = 0 (since |x|fX (x)dx < ∞).
−∞

0.11. Inequalities

Inequalities provide estimates of probabilities when they can not be evaluated precisely.
Theorem 0.94. Let X be a r.v. and let g : R → R be a non-negative function such that E(g(X)) is finite. Then, for
any c > 0,
E(g(X))
P (g(X) ≥ c) ≤ .
c

Proof. We will prove it for the case of continuous r.v.

Let A = {x ∈ R : g(x) ≥ c}. Let fX (x) denote the p.d.f. of X. Then,
Z ∞
E(g(X)) = g(x)fX (x)dx
−∞
Z ∞
= g(x)[IA (x) + IAc (x)]fX (x)dx
−∞
Z ∞ Z ∞
= g(x)IA (x)fX (x)dx + g(x)IAc (x) fX (x)dx
−∞ −∞
Z ∞
≥ g(x)IA (x)fX (x)dx
−∞
Z ∞
≥c IA (x)fX (x)dx
−∞
Z
E(g(X))
=c fX (x)dx = cP (g(X) ≥ c) =⇒ P (g(X) ≥ c) ≤ .
A c

This completes the proof.

Lecture 0 Lecture Notes 0-45

Corollary 0.95. (a) Let g : [0, ∞) → R be a non-negative and strictly increasing function such that E(g(X)) is finite.
Then, for any c > 0 such that g(c) > 0,
E(g(|X|))
P (|X| ≥ c) ≤ .
g(c)

(b) Let r > 0 and t > 0. Then,

E(|X|r )
P (|X| ≥ t) ≤ , (Markov’s inequality)
tr
E(|X|)
provided E(|X r |) < ∞. In particular, P (|X| ≥ t) ≤ , provided E(|X|) < ∞.
t

Proof. (a) Note that

P (|X| ≥ c) = P (g(|X|) ≥ g(c)) (since g is strictly increasing)

E(g(|X|))
≤ (by Theorem 0.94).
g(c)

(b) We take g(x) = xr , x ≥ 0, r > 0. Then, g is strictly increasing on [0, ∞) and is non-negative. Using (a) we get

E(g(|X|)) E(|X|r )
P (|X| ≥ t) ≤ = .
g(t) tr

This proves the result.

Theorem 0.96 (Chebyshev Inequality). Let X be a r.v. with finite variance σ 2 and E(X) = µ. Then, for any > 0,
1
P (|X − µ| ≥ σ) ≤ .
2

Proof. Using the above Corollary

E(|X − µ|2 ) E((X − µ)2 ) 1

P (|X − µ| ≥ σ) ≤ 2 2
= = 2.
σ 2 σ 2
This completes the proof.

Example 0.97 (The above bounds are sharp). Let X be a r.v. with p.m.f.
1
 8 , if x = −1, 1,

f (x) = 34 , if x = 0, .

0, otherwise.


1
Then E(X 2 ) = 4 and P (|X| ≥ 1) = 41 .
Using the Markov inequality, P (|X| ≥ 1) ≤ E(X 2 ) = 41 .
Example 0.98. Let X be a r.v. with p.d.f.
( 1
√ √
√
2 3
, if − 3<x< 3,
f (x) = .
0, otherwise.
Lecture 0 Lecture Notes 0-46

√ √
3 3
x2
Z Z
x
Then µ = E(X) = √
√ dx = 0, σ 2 = E(X 2 ) = √
√ dx = 1 and
− 3 2 3 − 3 2 3
Z 3/2
√
3 1 3
P (|X| ≥ ) = 1 − √ dx = 1 − = 0.134.
2 −3/2 2 3 2

Using the Markov inequality P (|X| ≥ 23 ) ≤ 94 E(X 2 ) = 4

9 = 0.444 . . . (considerably conservative).

Definition 0.99. Let −∞ ≤ a < b ≤ ∞. A function ψ : (a, b) → R is said to be a convex function if

ψ(αx + (1 − α)y) ≤ αψ(x) + (1 − α)ψ(y) ∀ x, y ∈ (a, b) and ∀ α ∈ (0, 1).

The function ψ(·) is said to be strictly convex if the above inequality is strict.

We state the following theorem without proof.

Theorem 0.100. (i) Let ψ : (a, b) → R be a convex function. Then, ψ is continuous on (a, b) and is almost everywhere
differentiable (i.e. if D is the set of points where ψ is not differentiable then D does not contain any interval).
(ii) Let ψ : (a, b) → R be a differentiable function. Then, ψ is convex (strictly convex) on (a, b) iff ψ 0 is non-decreasing
(strictly increasing) on (a, b).
(iii) Let ψ : (a, b) → R be a twice differentiable function. Then, ψ is convex (strictly convex) on (a, b) iff

ψ 00 (x) ≥ (>)0, ∀ x ∈ (a, b).

Theorem 0.101 (Jensen’s Inequality). Let ψ : (a, b) → R be a convex function and let X be a r.v. with d.f. F having
support S ⊆ (a, b). Then,

E(ψ(X)) ≥ ψ(E(X)), provided the expectations exist.

Proof. We give the proof for the special case where ψ is twice differentiable on (a, b) so that ψ 00 (x) ≥ 0, ∀ x ∈ (a, b).
Let µ = E(X). Expand ψ(x) into a Taylor series about µ we get

(x − µ)2 00
ψ(x) = ψ(µ) + (x − µ)ψ 0 (µ) + ψ (ξ), ∀ x ∈ (a, b)
2!
for some ξ between µ and x. Thus,

ψ(x) ≥ ψ(µ) + (x − µ)ψ 0 (µ) =⇒ E(ψ(X)) ≥ E(ψ(µ) + (X − µ)ψ 0 (µ)) = ψ(µ) = ψ(E(X)).

This completes the proof.

Example 0.102. (a) For any r.v. X, E(X 2 ) ≥ (E(X))2 [take ψ(x) = x2 , x ∈ R is convex, apply Jensen’s Inequality]
and E(|X|) ≥ |E(X)| [Take ψ(x) = |x|, x ∈ R is convex and apply Jensen’s Inequality].
(b) For any r.v. X with P (X > 0) = 1, E(ln X) ≤ ln E(X) [Take ψ(x) = − ln x is convex on (0, ∞) and apply
Jensen’s Inequality].
(c) For any r.v. X, E(eX ) ≥ eE(X) [Take ψ(x) = ex , x ∈ R is convex and apply Jensen’s Inequality].
(d) For any r.v. X with P (X > 0) = 1, E(X)E(1/X) ≥ 1 [Take ψ(x) = 1/x, x > 0 is convex and apply Jensen’s
Inequality].
Lecture 0 Lecture Notes 0-47

Pn
Example 0.103. Let a1 , a2 , . . . , an , w1 , w2 , . . . , wn be positive constants such that i=1 wi = 1. Prove the AM-
GM-HM inequality
n n
X Y 1
ai wi ≥ awi ≥ Pn
i
wi , (AM ≥ GM ≥ HM ).
i=1 i=1 i=1 ai

Solution: Let X be a r.v. with p.m.f.

(
wi , if x = ai , i = 1, 2, . . . , n,
f (x) = .
0, otherwise.

Then ψ(x) = − ln x, x > 0 is a convex function. Therefore

E(ψ(X)) ≥ ψ(E(X))
=⇒ E(− ln X) ≥ − ln E(X)
n n
!
X X
=⇒ − (ln ai )wi ≥ − ln ai wi
i=1 i=1
n
! n
! n n
X Y X Y
=⇒ ln ai wi ≥ ln aw
i
i
=⇒ ai wi ≥ aw
i .
i

i=1 i=1 i=1 i=1

n n
X wi Y
Replacing ai ’s by 1
ai ’s, we get ≤ 1/ aw i
i . Therefore,
i=1
ai i=1

n n
X Y 1
ai wi ≥ aw
i ≥ Pn
i
wi .
i=1 i=1 i=1 ai

0.12. Summary of Probabilty Distributions

Let X be a r.v. defined on a probability space (Ω, F, P ) associated with a random experiment E . Let FX (·) be its
distribution function and fX (·) be its p.m.f. / p.d.f.
The probabilty distribution of X (i.e., p.m.f. / p.d.f.) describes the manner in which the r.v. X takes values in various
sets. It may be desirable to have a set of numerical measures that provide a summary of the prominent features of
the probability distribution of X. We call these measures as descriptive measures. Four prominently used descriptive
measures are:
(1) Measures of Central Tendency or Location (also called Averages):
This gives us the idea about central value of the probability distribution around which the values of r.v. X are clustered.
Commonly used measures of central tendency are:
(a) Mean:
Z ∞ X
µ= µ01 = E(X) = xfX (x)dx or xfX (x) → may or may not exist.
−∞ x∈SX

Whenever it exists it gives us the idea about average observed value of X when E is repeated a large number of times.
d
Note that if distribution of X is symmetric about µ (i.e., X − µ = µ − X), then E(X) = µ, provided it exists.
Lecture 0 Lecture Notes 0-48

Mean seems to be the best suited measure of central tendency for symmetric distribution. Because of its simplicity
mean is the most commonly used average. However mean may be affected by a few extreme values and also it may
not be defined.
(b) Median:
Before defining the median we first inroduce the concept of quantile function or quantile.
The quantile function of r.v. X is a function QX : (0, 1) → R defined by

QX (p) = inf{x ∈ R : FX (x) ≥ p}, p ∈ (0, 1).

For a fixed p ∈ (0, 1) the quantity ξp = QX (p) is called the quantile of order p. Note that

FX (ξp −) ≤ p ≤ FX (ξp ), (Exercise)

and FX (ξp ) = p provided FX is continuous at ξp . Also note that:

· QX (FX (x)) ≤ x, provided 0 < FX (x) < 1;
· FX (QX (p)) ≥ p, ∀ 0 < p < 1;
· FX is continuous =⇒ FX (QX (p)) = p;
· QX (p) ≤ x ⇐⇒ FX (x) ≥ p;
−1 −1
· QX (p) = FX (p), provided FX (p) exists;
· QX (p1 ) ≤ QX (p2 ), ∀ 0 < p1 < p2 < 1.
The quantile of order 0.5 is called the median of (distribution) of X. If me is the median of X, then
1
FX (me −) ≤ ≤ FX (me ).
2
If the random experiment E is repeated a large number of times about half of the times observed value of X is expected
to be less than me and about half of the times it is expected to be grearter than me .
Suppose that the distribution of X is symmetric about µ. Then
d
X −µ=µ−X
=⇒ P (X − µ ≤ 0) = P (µ − X ≤ 0)
=⇒ FX (µ) = 1 − FX (µ−)
1
=⇒ FX (µ−) ≤ ≤ FX (µ) =⇒ µ = E(X) = me , provided FX is continuous at µ.
2
Merits of Median as a Measure of Central Tendency:
· Unlike mean it is always defined;
· Median is not affected by a few extreme values of X as it takes into account only the probabilities with which
different values occur and not their numerical values.
As a measure of central tendency the median is preferred over the mean if the distribution is asymmetric and a few
extreme observations occur with positive probabilities.
Demerits of Median as a Measure of Central Tendency:
· Does not at all take into account the numerical values assumed by X;
Lecture 0 Lecture Notes 0-49

· For many probability distributions it is not easy to evaluate.

(c) Mode:
Roughly speaking mode m0 of a probability distribution is the value that occurs with highest probability and is defined
by
fX (m0 ) = sup{fX (x) : x ∈ SX }.
If the random experiment E is repeated a large number of times then either mode m0 or a value in the neighborhood
of m0 is observed with maximum frequency.
Note that mode of a distribution may not be unique. A distribution having single / double / triple / multiple mode(s) is
called a unimodal / bimodal / trimodal / multimodal distribution.
Merits of a Mode as a Measure of Central Tendency:
It is easy to understand and easy to calculate. Normally, it can be found by just inspections.
Demerits of Mode as a Measure of Central Tendency:
· A probability distribution may have more than one mode which may be far apart.
As a measure of central tendency, mode is less preferred than mean and median. Clearly for symmetric unimodal
distributions mean=median=mode.
(2) Measures of Dispersion:
Apart from measures of central tendency other measures are often required to describe a probability distribution.
Measures of dispersion give the idea about the scatter (cluster / dispersion) of probability mass of the distribution
about a measure of a central tendency. Some of the measures of dispersion are listed below.
(a) Range:
Let SX = [a, b]. Then range of distribution of X is defined by R = b − a. It does not take into account how the
probability mass is distributed over [a, b]. For this reason it is not a preferred measure of dispersion.
(b) Mean Deviation:
Let A be a suitable measure of central tendency. Define
· M D(A) = E(|X − A|) → called the mean deviation of X about A (provided it exists);
· M D(µ) = E(|X − µ|) → mean deviation about mean µ = E(X);
· M D(me ) = E(|X − me |) → mean deviation about median.
It can be show that M D(me ) ≤ M D(A), ∀ A ∈ R. For this reason M D(me ) seems to be more applicable than
M D(A) for any A ∈ R.
· M D(A) is generally difficult to compute for many distributions;
· M D(A) is sensitive to extreme observations;
· M D(A) may not exist for many distributions.
(c) Standard Deviation (SD):
p p
The standard
p deviation of distribution of X is defined by σ = Var(X) = E(X − µ)2 , where µ ∈ R. Clearly
σ ≤ E(X − A)2 , ∀A ∈ R. It has same unit as that of X.
Standard deviation σ gives us the idea of average spread of values of X around the mean µ.
Lecture 0 Lecture Notes 0-50

· σ is simple to compute for most distributions (unlike M D(A), A ∈ R);

· SD is most widely used measure of dispersion (especially for nearly symmetric distributions);
· For some distributions SD does not exist;
· SD is sensitive to extreme observations.
(d) Quartile Deviation:
Let q1 = ξ0.25 = quantile of order 0.25 (lower quantile of X),
q2 = me = ξ0.5 = quantile of order 0.5=median,
q3 = ξ0.75 = quantile of order 0.75 (upper quantile of X).
So, q1 , q2 , q3 divide the probability distribution of X into 4 parts so that
1 1 3
FX (q1 −) ≤ ≤ FX (q1 ), FX (q2 −) ≤ ≤ FX (q2 ) and FX (q3 −) ≤ ≤ FX (q3 ).
4 2 4
Note that q1 , q2 and q3 divide the p.d.f. / p.m.f. of X into 4 parts so that each of them has 25% probability mass.
q3 − q1
Define IQR = q3 − q1 → inter-quantile range, QD = → quantile deviation or the semi-interquantile range.
2
· Unlike SD, QD is not sensitive to extreme values assumed by X.
· Does not at all take into account numerical values of X.
· Ignores the tail of the probability distribution (constituting 50% of probability diistributin on left side of q1 and right
side of q3 ).
· QD depends on the unit of measurements of X and thus it may not be appropriate for comparing dispersions of two
q3 − q1
probability distributions having different units of measurements. For this purpose one may use CQD = →
q3 + q1
coefficient of quartile deviation. It does not depend on units of measurements.
(d) Coefficient of Variation:
Like QD, the SD σ also depends on units of measurements of r.v. X and thus it is not an appropriate measure of
dispersion for comparing distributions having different units of measurements. For this purpose we consider
σ
CV (coefficient of variation) = ,
µ
p
where µ = E(X), σ = Var(X). Here, we assume µ 6= 0.
· CV measures variation per unit of mean.
· CV does not depend on the unit of measurements of r.v. X.
· CV is very sensitive to small changes in µ when µ is near 0.
(3) Measure of Skewness:
Skewness of a probability distribution is a measure of its asymmetry (lack of symmetry).
d
Recall that: Distribution of X is symmetric about µ ⇐⇒ X − µ = µ − X ⇐⇒ fX (µ + x) = fX (µ − x), ∀ x ∈ R
and in that case
· µ = E(X) = me (median);
· The shape of the p.d.f. / p.m.f. on the left of µ is the mirror image of that on the right side of µ.
Lecture 0 Lecture Notes 0-51

Positively Skewed Distributions:

· Have more probability mass to the right side of p.d.f. / p.m.f.
· Have longer tails on the right side of p.d.f.
For unimodal positively skewed distribution, normally

Mode < Median < Mean

since the positive mass to large values of X pulls up the values of mean µ.
Negatively Skewed Distributions:
· Have more probability mass to the left side of the p.d.f. / p.m.f.
· Have longer tails on the left side of p.d.f.
For unimodal negatively skewed distributions, normally

Mean < Median < Mode.

p X −µ
Let E(X) = µ, Var(X) = σ and Z = : standardized variable (independent of units). Define
σ
E((X − µ)3 ) µ3
Coefficient of skewness = β1 = E(Z 3 ) = 3
= 3/2 , where µr = E((X − µ)r ), r = 1, 2, . . .
σ µ2
· For symmetric distributions β1 = 0. Converse may not be true.
· For positively skewed distributions, normally β1 is large positive quantity.
· For negatively skewed distributions, normally β1 is a small negative quantity.
A measure of skewness can also be based on quantiles. Let q1 : first quantile, me : Median (or second quantile q2 ),
q3 : third quantile, µ : mean.

q1 + q3
· For symmetric distributions: q3 − m = m − q1 m = .
2
· For positively skewed distributions: q3 − m > m − q1 .
· For negatively skewed distributions: q3 − m < m − q1 .
Thus a measure of skewness can be based on (q3 − m) − (m − q1 ) = q3 − 2m + q1 . Define

(q3 − m) − (m − q1 ) q3 − 2m + q1
Yule coefficient of skewness = β2 = = (independent of units).
q3 − q1 q3 − q1
Clearly for positively / negatively skewed distribution β2 > 0/β2 < 0 and for symmetric distributions β2 = 0.
(4) Measures of Kurtosis:
For µ ∈ R and σ > 0, let Yµ,σ be a r.v. having p.d.f.
1 (x−µ)2
fYµ,σ (x) = √ e− 2σ2 , −∞ < x < ∞ (Normal distribution, Yµ,σ ∼ N (µ, σ 2 )).
σ 2π
It can be shown that
· E(Yµ,σ ) = µ, Var(Yµ,σ ) = σ 2 ;
Lecture 0 Lecture Notes 0-52

d
· Yµ,σ − µ = µ − Yµ,σ and hence β1 = 0, E((Yµ,σ − µ)4 ) = 3σ 4 ;
· fYµ,σ (·) is unimodal and symmetric.
Kurtosis of the probability distribution of X is a measure of peakedness and thickness of tails of p.m.f. / p.d.f. of X
relative to that of normal distribution.
A disribution is said to have higher (lower) kurtosis than the normal distribution if its p.m.f. / p.d.f. in comparison
with p.d.f. of a normal distribution, has a sharper (rounded) peak and longer, fatter (shorter, thinner) tails.
X −µ
Define Z = (independent of units)
σ
E((X − µ)4 ) µ4
ν1 = E(Z 4 ) = = 2 → Kurtosis of the probability distribution of X.
σ4 µ2

ν1 is used as a measure of kurtosis for unimodal distributions. For N (µ, σ 2 ) distribution, ν1 = 3. The quantity
ν2 = ν1 − 3 is called the excess kurtosis of the distribution of X. Obviously for normal distributions, ν2 = 0.
Mesokurtic distributions: Distributions with ν2 = 0,
Leptokurtic distributions: Distributions with ν2 > 0 (has sharper peak and longer, fatter tails).
Platykurtic distributions: Distributions with ν2 < 0 (has rounded peak and shorter, thinner tails).
Example 0.104. For α ∈ [0, 1], let Xα has the p.d.f.
(
αex , x < 0,
fα (x) = .
(1 − α)e−x , x ≥ 0.

Recall that for r ∈ {1, 2, . . . }

Z ∞
Ir = xr−1 e−x dx = (r − 1)! (using integration by parts).
0

Thus, for r ∈ {1, 2, . . . }

Z 0 Z ∞
µ0r (α) = E(Xαr ) = αxr ex dx + (1 − α)xr e−x dx
−∞ 0
Z ∞
= ((−1) α + 1 − α)r
xr e−x dx
0
(
(1 − 2α)r!, r ∈ {1, 3, 5, . . . },
= .
r!, r ∈ {2, 4, 6, . . . }.

Let ξp be the quantile of order p ∈ (0, 1). Then Fα (ξp ) = p, where Fα is the d.f. of Xα . Clearly Fα (0) =
Z 0
α ex dx = α. For 0 ≤ α < p, we have
−∞

Z 0 Z ξp
p = Fα (ξp ) = x
αe dx + (1 − α)e−x dx = 1 − (1 − α)e−ξp
−∞ 0

and for α ≥ p
Z ξp
p= αex dx = αeξp .
−∞
Lecture 0 Lecture Notes 0-53

Thus, 
ln 1−α

1−p , if 0 ≤ α < p,
ξp =
− ln αp , if p ≤ α ≤ 1,


ln 4(1−α) , if 0 ≤ α < 1 ,
3 4
q1 (α) = ξ1/4 =
− ln (4α) , if 1 ≤ α ≤ 1,
4
(
ln (2(1 − α)) , if 0 ≤ α < 21 ,
me (α) = ξ1/2 = 1
− ln (2α) , if 2 ≤ α ≤ 1,
(
ln (4(1 − α)) , if 0 ≤ α < 43 ,
q3 (α) = ξ3/4 =
− ln 4α 3

3 , if 4 ≤ α ≤ 1,

µ01 (α) = E(Xα ) = 1 − 2α,

Mode = m0 (α) = sup{fα (x) : −∞ < x < ∞} = max{α, 1 − α},
p p
µ02 (α) = E(Xα2 ) = 2, σ(α) = Var(Xα ) = 1 + 4α − α2 .

Note that, for 0 ≤ α < 12 , me (α) = ln(2(1 − α)) ≥ 0 and for α > 21 , me (α) = − ln(2α) < 0. Thus, for 0 ≤ α < 1
2
(so that me (α) ≥ 0)

M D(me (α)) = E(|X − me (α)|)

Z 0 Z me (α) Z ∞
−x
=α x
(me (α) − x)e dx + (1 − α) (me (α) − x)e dx + (1 − α) (x − me (α))e−x dx
−∞ 0 me (α)

= me (α) + 2α = ln(2(1 − α)) + 2α.

1
Similarly, for 2 ≤ α ≤ 1 (so that me (α) ≤ 0)

M D(me (α)) = E(|X − me (α)|)

Z me (α) Z 0 Z ∞
=α x
(me (α) − x)e dx + α x
(x − me (α))e dx + (1 − α) (x − me (α))e−x dx
−∞ me (α) 0

= 2(1 − α) − me (α) = ln(2α) + 2(1 − α).

Thus, (
ln(2(1 − α)) + 2α, if 0 ≤ α < 21 ,
M D(me (α)) =
ln(2α) + 2(1 − α), if 21 ≤ α ≤ 1,
(
ln 3, if 0 ≤ α < 41 or 34 ≤ α ≤ 1,
IQR ≡ IQR(α) = q3 (α) − q1 (α) =
ln(16α(1 − α)), if 41 ≤ α < 43 ,
 √
 ln 3, if 0 ≤ α < 14 ,
q3 (α) − q1 (α)  p
QD ≡ QD(α) = = ln(4 α(1 − α), if 14 ≤ α < 43 ,
2  √

ln 3, if 43 ≤ α ≤ 1,
Lecture 0 Lecture Notes 0-54


ln 3

 , if 0 ≤ α < 41 ,
16(1−α)2




 ln 3


ln(16α(1 − α))

q3 (α) − q1 (α) 
, if 14 ≤ α ≤ 43 ,
CQD ≡ CQD(α) = =
(1−α)
q3 (α) + q1 (α)  ln


 α

ln 3


3
− 2 , if 4 ≤ α ≤ 1.


ln 16α

3

For α 6= 21 ,
√
σ(α) 1 + 4α − 4α2
CV ≡ CV (α) = 0 = ,
µ1 (α) 1 − 2α
µ3 (α) = E((Xα − µ01 (α))3 ) = µ03 (α) − 3µ01 (α)µ02 (α) + 2(µ01 (α))3 = 2(1 − 2α)3 ,
µ3 (α) 2(1 − 2α)3
β1 ≡ β1 (α) = =√ ,
σ(α) 1 + 4α − 4α2
 4
 ln( 3 ) , if 0 ≤ α < 1 ,



 ln 3 4

ln(4α(1 − α))


− , if 41 ≤ α < 21 ,


q3 (α) − 2m(α) + q1 (α)  ln(16α(1 − α))
β2 ≡ β2 (α) = =
q3 (α) − q1 (α) ln(4α(1 − α))
, if 21 ≤ α ≤ 43 ,



ln(16α(1 − α))




 3

 ln( 4 ) , if 3 ≤ α ≤ 1.


ln 3 4

Clearly, for 0 ≤ α < 21 , βi (α) > 0, i = 1, 2 and for 1

2 < α ≤ 1, βi (α) < 0, i = 1, 2. For α = 21 , βi (α) = 0, i = 1, 2.
Thus,
· for 0 ≤ α < 21 , distribution of Xα is positively skewed;
1
· for 2 < α ≤ 1, distribution of Xα is negatively skewed;
· for α = 12 , distribution of Xα is symmetric (infact in this case fα (x) = fα (−x), ∀ x ∈ R).

µ4 ≡ µ4 (α) = E((Xα − µ01 (α))4 )

= µ04 (α) − 4µ01 (α)µ03 (α) + 6(µ01 (α))2 µ02 (α) − 3(µ01 (α))4 = 24 − 12(1 − 2α)2 − 3(1 − 2α)4

µ4 (α) 24 − 12(1 − 2α)2 − 3(1 − 2α)4

ν1 ≡ ν1 (α) = = 2
(µ2 (α))2 (2 − (1 − 2α)2 )
and
12 − 6(1 − 2α)4
ν2 ≡ ν2 (α) − 3 = 2.
(2 − (1 − 2α)2 )
Clearly, for any α ∈ [0, 1], ν2 (α) > 0. It follows that for any value of α ∈ [0, 1] the distribution of Xα is leptokurtic.
Lecture 0 Lecture Notes 0-55

0.13. Some Special Discrete Distributions

0.13.1. Bernoulli and Binomial Distribution

Bernoulli Experiment: A random experiment with just two possible outcomes (say success (S) and failure (F )).
Each replication of a Bernoulli experiment is called a Bernoulli trial.
Consider a sequence of n independent Bernoulli trials with probability of success (S) in each trial as p ∈ (0, 1) (same
for each trial); here n ∈ N is a fixed natural number.
Define X = the number of success in n trials. Then SX = {0, 1, 2, . . . , n} and for k ∈ SX

P (X = k) = P ( |SS · · · SF
{z F · · · F} ) + P ( SF · · · F F S}
| F S{z ) + ··· + P( F| F · · · F{zSS · · · S} )
k successes and n − k failures k successes and n − k failures k successes and n − k failures

n
(total of terms)
k
= pk (1 − p)n−k + pk (1 − p)n−k + · · · + pk (1 − p)n−k

n k
= p (1 − p)n−k , (independence of trials).
k

Thus, 
 n px (1 − p)n−x , x = 0, 1, 2, . . . , n,

fX (x) = P (X = x) = x

0, otherwise,

is called Binomial distribution with n trials and success probability p (denoted by Bin(n, p) and written as X ∼
Bin(n, p)). {Bin(n, p) : n ∈ N, p ∈ (0, 1)} is the family of probability distributions that has two parameters n ∈ N
and p ∈ (0, 1).
{Bin(1, p) : p ∈ (0, 1)}: Bernoulli distributions. Bin(1, p): Bernoulli distribution with success probability p ∈
(0, 1).
Suppose that X ∼ Bin(n, p), n ∈ N, p ∈ (0, 1). Then
∞ ∞
tX
X
tkn k n−k
X n
m.g.f. MX (t) = E(e )= e p (1 − p) = (pet )k (1 − p)n−k = (1 − p + pet )n , t ∈ R.
k k
k=0 k=0

Let q = 1 − p, so that MX (t) = (q + pet )n , t ∈ R. Then

(1)
MX (t) = n(q + pet )n−1 pet ,
(2)
MX (t) = np(q + pet )n−1 et + n(n − 1)(q + pet )n−2 (pet )2 ,
(1) (2)
E(X) = MX (0) = np, E(X 2 ) = MX (0) = np + n(n − 1)p2 , Var(X) = np(1 − p) = npq.

Note that if X ∼ Bin(n, p) then Variance < Mean. It can be seen that

µ03 = E(X 3 ) = np(1 − 3p + 3np + 2p2 − 3np2 + n2 p2 ),

µ04 = E(X 4 ) = np(1 − 7p + 7np + 12p2 − 18np2 + 6n2 p2 − 6p3 + 11np3 − 6n2 p3 + n3 p3 ),
µ3 = E((X − µ01 )3 ) = np(1 − p)(1 − 2p),
µ4 = E((X − µ01 )4 ) = np(1 − p)(3p2 (2 − n) + 3p(n − 2) + 1),
Lecture 0 Lecture Notes 0-56

symmetric for p = 12 ,


µ3 1 − 2p 
β1 = 3/2 = p = positively skewed for 0 < p < 21 ,
µ2 np(1 − p) 
negatively skewed for p > 21 ,


1 − 6pq µ4
ν2 = ν1 − 3 = , where ν1 = 2 .
npq µ2

Also, for r ∈ {1, 2, . . . }, let X(r) = X(X − 1)(X − 2) · · · (X − r + 1), the rth factorial moment is given by
n
X n
E(X(r) ) = pk (1 − p)n−k k(k − 1)(k − 2) · · · (k − r + 1)
k
k=0
n
X n−r
= n(n − 1)(n − 2) · · · (n − r + 1) pk (1 − p)n−k
k−r
k=r
n−r
X n − r
= n(n − 1)(n − 2) · · · (n − r + 1)pr pk (1 − p)n−r−k
k
k=0
= n(n − 1)(n − 2) · · · (n − r + 1)p (1 − p + p)n−r = n(n − 1)(n − 2) · · · (n − r + 1)pr .
r

Theorem 0.105. Let X1 , X2 , . . . , Xk be independent r.v.’s with Xi ∼ Bin(ni , p), ni ∈ N, p ∈ (0, 1), i = 1, 2, . . . , k.
Pk Pk
Then Y = i=1 Xi ∼ Bin(n, p), where n = i=1 ni .

Proof. For t ∈ R,
k
! k
Pk Y Y
tY
MY (t) = E(e ) = E(e t i=1 Xi )=E e tXi
= E(etXi ), (independent of Xi0 s)
i=1 i=1
k
Y k
Y Pk
= MXi (t) = (1 − p + pet )ni = (1 − p + pet ) i=1 ni

i=1 i=1
k
!
X
→ m.g.f. of Bin ni , p .
i=1
Pk
By uniqueness of m.g.f. Y ∼ Bin(n, p), where n = i=1 ni .
1 d n d
Example 0.106. Let X ∼ Bin(n, 1/2), then X − 2 = 2 − X, since n − X = X (Exercise).
Example 0.107. A fair dice is rolled 5 times independently. Find the probability that on 3 occasions we get a six.

Solution: Consider getting a six as success. Then X = the number of success in 5 trials ∼ Bin(5, 1/6).

5
So, the required probability =P (X = 3) = (1/6)3 (5/6)2 .
3

0.13.2. Negative Binomial Distribution

Consider a sequence of independent Bernoulli trials with probability of success in each trial as p ∈ (0, 1). Let
r ∈ {1, 2, . . . } be a fixed positive integer. Let X denote the number of failures preceding the rth success. Then
SX = {0, 1, 2, . . . } and for k ∈ SX , we have

fX (k) = P (X = k)
Lecture 0 Lecture Notes 0-57

= P (k failures precede rth success)

= P (r − 1 successes in first k + r − 1 trials and success in (k + r)th trial)
= P (r − 1 successes in first k + r − 1 trials) × P (success in (k + r)th trial), (independence of trials)

k + r − 1 r−1 k k+r−1 r
= p (1 − p) p = p (1 − p)k .
r−1 r−1
Thus, 
 x + r − 1 pr (1 − p)x , x = 0, 1, 2, . . . ,

fX (x) = P (X = x) = r−1

0, otherwise.

The distribution of X is called Negative binomial distribution with r success, and success probability p ∈ (0, 1)
(denoted by N B(r, p) and written as X ∼ N B(r, p)) (has two parameters r ∈ N and p ∈ (0, 1)). {N B(r, p) : r ∈
N, p ∈ (0, 1)} is a family of probability distribution.
Remark 0.108. For t ∈ (−1, 1), we have
∞
X k+r−1 (r + 1)r 2 (r + 2)(r + 1)r 3
tk = 1 + rt + t + t + · · · = (1 − t)−r .
r−1 2! 3!
k=0

The m.g.f. of X ∼ N B(r, p) is

∞
tX k+r−1
X
tk
MX (t) = E(e ) = e (1 − p)k pr
r−1
k=0
∞ r
X k+r−1 p
= pr ((1 − p)et )k = , t < − ln(1 − p).
r−1 1 − (1 − p)et
k=0

Thus,

ψX (t) = ln MX (t) = r ln p − r ln(1 − qet ), t < − ln(1 − p),

rqet

(1) 1
ψX (t) = =r − 1 , t < − ln(1 − p),
1 − qet 1 − qet
(2) rqet
ψX (t) = , t ∈ R,
(1 − qet )2
(1) rq (2) rq
E(X) = ψX (0) = , Var(X) = ψX (0) = 2 , Variance > Mean.
p p
Also, for m ∈ {1, 2, . . . }, let X(m) = X(X − 1)(X − 2) · · · (X − m + 1). Then
∞
X k+r−1 r
E(X(m) ) = k(k − 1)(k − 2) · · · (k − m + 1) p (1 − p)k
r−1
k=0
∞
X k+r−1
= pr k(k − 1)(k − 2) · · · (k − m + 1) (1 − p)k
r−1
k=m
∞
X (k + r − 1)!
= r(r + 1)(r + 2) · · · (r + m − 1)pr (1 − p)k
(k − m)!(r + m − 1)!
k=m
∞
X (k + m + r − 1)!
= r(r + 1)(r + 2) · · · (r + m − 1)pr (1 − p)k+m
k!(r + m − 1)!
k=0
Lecture 0 Lecture Notes 0-58

∞
X k+m+r−1
= r(r + 1)(r + 2) · · · (r + m − 1)pr q m qk
m+r−1
k=0

= r(r + 1)(r + 2) · · · (r + m − 1)p q (1 − q)−(m+r) = r(r + 1)(r + 2) · · · (r + m − 1)(q/p)m .

r m

rq rq(1 + rq)
µ01 = E(X) = ; µ02 = E(X 2 ) = .
p p2
It can be seen that
q(rp2 + 3pqr + q 2 r(r + 1)
µ03 = E(X 3 ) = ,
p3
q(rp3 + 7p2 qr + 6pq 2 r(r + 1) + q 3 r(r + 1)(r + 2)
µ04 = E(X 4 ) = ,
p4
µ2 = E((X − µ01 )2 ) = r(1 − p),
r(p − 1)(p − 2)
µ3 = E((X − µ01 )3 ) = ,
p3
r(1 − p)(6 − 6p + p2 + 3r − 3pr)
µ4 = E((X − µ01 )4 ) = ,
p4
µ2 2−p
β1 = 33 = √ > 0 (positively skewed),
µ2 rq
p2 − 2p + 6 µ4
ν2 = ν1 − 3 = , where ν1 = 2 .
rq µ2
N B(1, p) distribution is called a geometric distribution (denoted by Ge(p), 0 < p < 1). The p.m.f. of Y ∼ Ge(p) is
given by (
pq y , y = 0, 1, 2, . . . ,
fY (y) = P (Y = y) =
0, otherwise.
P∞
P (Y ≥ m) = p y=m q y = q m . This implies that

P (Y ≥ m + n, Y ≥ m) P (Y ≥ m + n)
P (Y ≥ m + n|Y ≥ m) = =
P (Y ≥ m) P (Y ≥ m)
m+n
q
= m = q n = P (Y ≥ n), ∀ m, n ∈ {0, 1, . . . }. (0.7)
q
Also,
P (Y ≥ m + n) = P (Y ≥ m)P (Y ≥ n), ∀ m, n ∈ {0, 1, . . . }. (0.8)
Remark 0.109. The property (0.7) possessed by Ge(p) distribution has an interesting interpetation. Suppose that
a device can absorb 0, 1, 2, . . . shocks before failing. Let T denote the random variable representing the number of
shocks that device can absorb before failing.
P (T ≥ m + n|T ≥ m) : conditional probability that a system has absorbed m shocks will absorb atleast n additional
shocks before failing.
P (T ≥ n) : a new device can survive atleast n shocks before failing.
Thus if distribution of T has property (0.7) then the age of the device has no effect as the residual (remaining) life
of the device (implying that an used device is as good as a new device). The property (0.7) (or equivalently (0.8)) is
famously known as Lack of memory (LoM) property.
Theorem 0.110. Let T be a discrete type r.v. with range ST = {0, 1, 2, . . . }. Then T has the lack of memory property
if and only if T ∼ Ge(p), for some p ∈ (0, 1).
Lecture 0 Lecture Notes 0-59

Proof. Obviously, T ∼ Ge(p), for some p ∈ (0, 1) =⇒ T has LoM property. Then P (T ≥ j + k) = P (T ≥
j)P (T ≥ k) ∀ j, k ∈ {0, 1, . . . }. Let P (T = 0) = p. Then p ∈ (0, 1) and for j ∈ {0, 1, . . . }
P (T ≥ j + 1) = P (T ≥ j)P (T ≥ 1)
= P (T ≥ j)(1 − p)
= P (T ≥ j − 1)(1 − p)2
..
.
= P (T ≥ 0)(1 − p)j+1 = (1 − p)j+1
This implies
P (T = k) = P (T ≥ k) − P (T ≥ k + 1) = p(1 − p)k , k = {0, 1, 2, . . . } =⇒ T ∼ Ge(p).
This completes the proof.
Example 0.111. A person repeatedly rolls a fair die independently untill an upper face with two or three dots is
observed twice. Find the probability that the person would require eight rolls to achive this.

Solution: Consider getting 2 or 3 dots as success. Let Z = the number of trials nrequires to o get 2 successes. Then
6
probability of success in each trial is 1/3 and required probability = P (Z = 8) = 71 13 23 448
× 13 = 6561

.

0.13.3. Hypergeometric Distribution

Consider a population comprising of N (≥ 2) units out of which a ∈ {1, 2, . . . , N − 1} are labelled as S (success)
and N − a are labeled as F (failure). A sample of size n is drawn from this population drawing one unit at a time. Let
X denotes the number of successes in drawn sample.
Case-I: Drawn are independent and sampling is with replacement (i.e. after each draw the drawn units is replaced
back into the population)
a
In this case we have sequence of n independent Bernoulli trials with probability of success in each trial as p = N.
Thus X ∼ Bin(n, Na ).
Case-II: Without replacement (i.e. drawn units are not replaced back into the population).
Here,
a
P (obtaining S in first draw) = ,
N
a a−1 N −a a a
P (obtaining S in second draw) = + = .
N N −1 N N −1 N
a
In general, P (obtaining S in ith trial) = , i = 1, 2, . . . , n (Exercise),
N
a a−1
P (obtaining S in first and second trial) =
N N
a a
6= = P (obtaining S in first trial) × P (obtaining S in second trial)
NN
=⇒ Draws are not independent.
Thus, we can not conclude that X ∼ Bin(n, Na ). So,
 a N −a
 x n−x

, x = max{0, n − N + a}, . . . , min{n, a},
N
fX (x) = P (X = x) = n


0, otherwise.
Lecture 0 Lecture Notes 0-60

The distribution is called Hypergeometric distribution (Hyp(a, n, N )). It has three parameters N ∈ {2, 3, . . . }, a, n ∈
{1, 2, . . . , N − 1}.
For r ∈ N, let X(r) = X(X − 1)(X − 2) · · · (X − r + 1). Then

min{n,a}
1 X a N −a
E(X(r) ) = N
k(k − 1)(k − 2) · · · (k − r + 1) .
n−k

n
k
k=max{0,n−N +a}

Clearly for r > min{n, a}, E(X(r) ) = 0. For 1 ≤ r ≤ min{n, a}, we have

min{n,a}
1 X a N −a
E(X(r) ) = N
k(k − 1)(k − 2) · · · (k − r + 1)
n−k

n
k
k=max{r,n−N +a}
min{n,a}
a(r) X a−r N −a
= N
n
k−r n−k
k=max{r,n−N +a}
min{n−r,a−r}
a(r) X a−r N −a
= N n−r−k

n
k
k=max{0,n−N +a−r}
min{n−r,a−r} N −r

a(r) X a−r (N − r) − (a − r) n−r
= N = N
a(r) .
(n − r) − k

n
k n
k=max{0,(n−r)−(N −r)+a−r}

Pmin{m,b} b
M −b
M

Since k=max{0,m−M +b} k m−k = m . Thus, for r ∈ N, we have
 N −r
 ( n−r ) a , if r ≤ min{n, a},
E(X(r) ) = (Nn ) (r)
0, if r > min{n, a}.


In particular,
a a
E(X) = E(X(1) ) = n
= np (say), where p = ,
N N
n(n − 1)
E(X(X − 1)) = E(X(2) ) = a(a − 1),
N (N − 1)
Var(X) = E(X 2 ) − (E(X))2
= E(X(X − 1)) + E(X) − (E(X))2

a a N −n n−1
= n (1 − ) = np(1 − p) 1 − . (0.9)
N N N −1 N −1

Remark 0.112. In case of sampling with replacement we have X ∼ Bin(n, p), E(X) = np and Var(X) = np(1−p),
where p = Na . The factor (1 − N n−1
−1 ) which on multiplying to variance of Bin(n, p) distribution yields the variance
of Hyp(a, n, N ) distribution (see 0.9) is called the finite population correction (f.p.c.). Clearly if the sample size n is
significantly smaller than the population size N (n << N ) then f.p.c. will be close to 1 and variance of Bin(n, p)
and Hyp(a, n, N ) distribution will be very close. Infact when n << N and n << a ≡ aN (say) are such that aNN is a
fixed quantity (i.e. as N → ∞, aN → ∞ and aNN → p ∈ (0, 1), where p ∈ (0, 1) is a fixed quantity) then Bin(n, Na )
distribution provides an approximation to Hyp(a, n, N ) distribution. Regarding choice of sample size n for using this
approximation a guideline based on various empirical studies, is that the sample size n should not exceed 10% of the
population size N .
Lecture 0 Lecture Notes 0-61

Theorem 0.113 (Binomial Approximation to Hypergeometric Distributon). Let XaN ,n,N ∼ Hyp(aN , n, N ), where
aN depends on N and lim aNN = p ∈ (0, 1). Let faN ,n,N (·) denote the p.m.f. of XaN ,n,N . Then
N →∞

 n pk (1 − p)n−k , k ∈ {0, 1, 2, . . . , n},

lim faN ,n,N (k) = lim P (XaN ,n,N = k) = k
N →∞ N →∞ 
0, otherwise,

aN
i.e., for large N and large aN , so that p = N ∈ (0, 1) is a fixed quantity, Hyp(aN , n, N ) probabilities can be
approximated by Bin(n, Na ) probabilities.

n aN
Proof. SX = {m ∈ N : max{0, n − N + aN } ≤ m ≤ min{n, aN }}, n − N + aN = N ( N −1+ N ) → ∞ and
aN
aN = N N → ∞, as N → ∞. Also for k ∈ SX ,
  
k−1
Y aN − j  n−k−1
Y N − aN − j 
n
fX (k) =
k  j=0 N − j   j=0 N −j 
  
k−1  n−k−1  n
N →∞ n
Y Y
−→ (p) (1 − p) = pk (1 − p)n−k
k  j=0   j=0  k

 n pk (1 − p)n−k , k ∈ {0, 1, 2, . . . , n}

=⇒ lim faN ,n,N (k) = k
N →∞ 
0, otherwise.

This completes the proof.

The m.g.f. of X ∼ Hyp(a, n, N ), althouh exists (since SX is finite), can not be expressed in closed form.

0.13.4. Poisson Distribution

Some event E (say number of cars crossing a particular bridge/tunnel) is occuring randomly over a period of time. Let
X denotes the number of times E has occured in an unit interval (say (0, 1]).
To model probability distribution of X, partition the unit interval into a large number (say n where n → ∞) of
infinitesimal subintervals ( i−1 i 1
n , n ], i = 1, 2, . . . , n of length n each. In many situations, it may be relevant to assume
that
(i) For each infinitesimal interval ( i−1 i
n , n ], i = 1, 2, . . . , n, the probability that E will occur in this interval is pn and
that it will not occur in this interval is 1 − pn ; here pn → 0 as n → ∞ and npn → λ ∈ (0, ∞) as n → ∞.
(ii) Chance of two or more occurences of E in any infinitesimal interval ( i−1 i
n , n ], i = 1, 2, . . . , n, is so small that it
can be neglected.
(iii) occurrences of E in two disjoint infinitesimal intervals are independent.
X ≡ Xn = the number of times event E occurs in (0, 1] ∼ Bin(n, pn ) . The p.m.f. of Xn is

n k
fn (k) = p (1 − pn )n−k I{0,1,...,n} (k)
k n

1 1 2 k−1 npn n−k
= 1− 1− ... 1 − (npn )k 1 − I{0,1,...,n} (k)
k! n n n n
Lecture 0 Lecture Notes 0-62

e−λ λk
→ I{0,1,...,n} (k)
k!
 −λ k
e λ
, k ∈ {0, 1, 2, . . . },
= k! → Poisson distribution (P o(λ) : λ > 0)(family of probability distributions).
0, otherwise.


A r.v. X is said to have a Poisson distribution with parameter λ > 0 (written as X ∼ P o(λ)) if its p.m.f. is given by
 −λ k
e λ
, k = 0, 1, 2, . . . ,
fX (k) = P (X = k) = k!
0, otherwise.


Theorem 0.114 (Poisson Approximation to Binomial Distribution). Let Xn ∼ Bin(n, pn ), n = 1, 2, . . . , where

pn ∈ (0, 1), n = 1, 2, . . . and lim (npn ) = λ, for some λ > 0. Then
n→∞
 −λ k
e λ
, k = {0, 1, 2, . . . },
lim fXn (k) = lim P (Xn = k) = k!
n→∞ n→∞
0, otherwise.


Proof. As above.
Remark 0.115. If n is large and p is small (pn → 0 as n → ∞) so that np is a fixed quantity in (0, ∞) (npn → λ > 0)
then Poisson distribution provides a good approximation to Binomial distribution.
Example 0.116. Consider a person who plays a series of 2500 games independently. If the probability of person
winning any game is 0.002, find the probability that the person will win atleast two games.

Solution: Let X denote the number of wins (successes) in 2500 games played by person.
Clearly X ∼ Bin(2500, 0.002), where n = 2500 and np = 5 (=λ, say) is fixed. Therefore,

P (X ≥ 2) ≈ P (Y ≥ 2), where Y ∼ P o(5).

Thus, P (X ≥ 2) ≈ 1 − (P (Y = 0) + P (Y = 1)) = 1 − (e−5 + 5e−5 ) = 0.9596.

Suppose that X ∼ P o(λ), for some λ > 0. Then for r ∈ {1, 2, . . . }, we have

E(X(r) ) = E(X(X − 1) · · · (X − r + 1))

∞
X e−λ λk
= k(k − 1)(k − 2) · · · (k − r + 1)
k!
k=0
∞ ∞ ∞
X λk X λj+r X λj
= e−λ = e−λ = λr e−λ = λr .
(k − r)! j=0
j! j=0
j!
k=r

Thus,

µ1 = E(X) = E(X(1) ) = λ,
E(X 2 ) = E(X(2) ) + E(X) = λ2 + λ, Var(X) = E(X 2 ) − (E(X))2 = λ (σ 2 = µ2 ) (Mean=Variance),
µ03 = E(X 3 ) = λ(λ2 + 3λ + 1),
µ04 = E(X 4 ) = λ(λ3 + 6λ2 + 7λ + 1), µ3 = λ; µ4 = λ(3λ + 1),
µ3 1 λ(3λ + 1) 1
β1 = 3
= √ , ν2 = ν1 − 3 = 2
−3= ,
σ λ λ λ
Lecture 0 Lecture Notes 0-63

∞ ∞
X e−λ λk X (λetk )k t t
MX (t) = E(etX ) = etk = e−λ = e−λ eλe = eλ(e −1) , t ∈ R,
k! k!
k=0 k=0
t r
ψX (t) = ln MX (t) = λ(e − 1), ψX (t) = λet , r = 1, 2, . . . ,
1 2
=⇒ E(X) = ψX (0) = λ, Var(X) = ψX (0) = λ.

Theorem 0.117. Let X1 , X2 , . . . , Xk be independent r.v.’s such that Xi ∼ P o(λi ), for some λi > 0, i = 1, 2, . . . , k.
Pk Pk
Then Y = i=1 Xi ∼ P o(λ), where λ = i=1 λi .

Proof. For t ∈ R,
k
! k
Pk Y Y
tY
MY (t) = E(e ) = E(e t i=1 Xi )=E e tXi
= E(etXi ), (independent of Xi0 s)
i=1 i=1
k k
Y Y t t
−1) −1)
= MXi (t) = eλi (e = eλ(e .
i=1 i=1

Pk
This implies that Y ∼ P o(λ), where λ = i=1 λi .

0.13.5. Discrete Uniform Distribution

Let N be a given positive integer and x1 < x2 < · · · < xN be given real numbers. A r.v. X is said to follow a discrete
uniform distribution on the set {x1 , x2 , . . . , xN } (written as X ∼ U ({x1 , x2 , . . . , xN })) if its p.m.f. is given by

 1 , x ∈ {x , x , . . . , x },

1 2 N
fX (x) = P (X = x) = N
0, otherwise.


Suppose that X ∼ U ({x1 , x2 , . . . , xN }). Then,

N
1 X r
µ0r = E(X ) = r
x ,
N i=1 i
N
1 X
Mean = µ01 = xi ,
N i=1
N
1 X
Var(X) = σ 2 = E((X − µ01 )2 ) = (xi − µ01 )2 ,
N i=1
N
1 X txi
m.g.f. MX (t) = E(etX ) = e .
N i=1

Suppose that Y ∼ U ({1, 2, . . . , N }). Then,

N
1 X N +1
µ01 = Mean = E(Y ) = i= ,
N i=1 2
N
1 X 2 (N + 1)(2N + 1)
µ02 = E(Y 2 ) = i = ,
N i=1 6
Lecture 0 Lecture Notes 0-64

N
1 X 3 N (N + 1)2
µ03 = E(Y 3 ) = i = ,
N i=1 4
N
1 X 4 (N + 1)(2N + 1)(3N 2 + 3N − 1)
µ04 = E(Y 4 ) = i = ,
N i=1 30
N2 − 1
µ2 = E((Y − µ01 )2 ) = ,
12
µ3 = E((Y − µ01 )3 ) = 0,
(3N 2 − 7)(N 2 − 1)
µ4 = E((Y − µ01 )4 ) = ,
240
µ23
Coefficient of skewness = β1 = = 0,
µ32
µ4 3 (3N 2 − 7)
Kurtosis = ν1 = 2 = ,
µ2 5 N2 − 1

t Nt
1 XN  e (e − 1) , t 6= 0,

m.g.f. MY (t) = E(etY ) = etj = N (et − 1)
N j=1 
1, t = 0.

Example 0.118. A person has to open a lock whose key is lost among a set of N keys. Assume that out of these N
keys only one can open the lock. To open the lock the person tries keys one by one by choosing at each attempt one of
the keys at random from the unattempted keys. The unsuuccessful keys are not considered for future attempts. Let Y
denote the number of attempts the person will have to make to open the lock. Show that Y ∼ U ({1, 2, . . . , N }) and
hence find the mean and variance of the r.v. Y .

Solution: For r 6= {1, 2, . . . , N }, we have P (Y = r) = 0. For r ∈ {1, 2, . . . , N }, we have

N −1 N −2 N − (r − 1) 1 1
P (Y = r) = · ··· · = =⇒ Y ∼ U ({1, 2, . . . , N }).
N N −1 N − (r − 2) N − (r − 1) N
N +1 N2 − 1
This implies E(Y ) = and Var(Y ) = .
2 12

0.14. Some Special Continuous Distributions

0.14.1. Uniform or Rectangular Distribution

Let −∞ < α < β < ∞. An absolutely continuous type r.v. X is said to have a uniform (or rectangular) distribution
over the interval (α, β) (written as X ∼ U (α, β)) if its p.d.f. is given by
 1 , α < x < β,


f (x|α, β) = β − α
0, otherwise.


{U (α, β) : −∞ < α < β < ∞} is a family of distributions corresponding to different choices of α and β (−∞ <
α < β < ∞).
Suppose that X ∼ U (α, β)), for some −∞ < α < β < ∞. Then
Z β " 2 r #
0 r xr β r+1 − αr+1 βr α α α
µr = E(X ) = dx = = 1+ + + ··· + ,
α β − α (r + 1)(β − α) r + 1 β β β
Lecture 0 Lecture Notes 0-65

α+β
E(X) = = µ01 ,
2

Z β
α+β
r
1
Z β−α
2 tr 0, r = 1, 3, 5, . . . ,
0 r
µr = E(X − µ1 ) = x− dx = dt = (β − α)r
α 2 β−α − β−α β−α  , r = 2, 4, 6, . . . .
2
2r (r + 1)

Also,
 1

, − β−α
2 <x<
β−α
2 ,

α+β α+β α+β d α+β
fX x − = fX −x = β−α =⇒ X − = − X.
2 2 2 2
0, otherwise


α+β
This implies distribution of X is symmetric about its mean µ01 = 2 .

(β − α)2
Var(X) = µ2 = σ 2 = E((X − µ01 )2 ) = ,
12
µ3
Coefficient of skewness = β1 = 3/2
= 0,
µ2
µ4 9
Kurtosis = ν1 = = = 1.8,
µ22 5

0, x < α,


x − α
The d.f. of X ∼ U (α, β) is given by F (x) = , α ≤ x < β,
β − α


1, x ≥ β.


Theorem 0.119. Let −∞ < α < β < ∞ and let X be a r.v. of continuous type with P (α ≤ X ≤ β) = 1. Then
X ∼ U (α, β) ⇐⇒ P (X ∈ I) = P (X ∈ J), for any pairs of intervals I, J ⊆ (α, β) having the same length.

Proof. Suppose that X ∼ U (α, β). Then, for α ≤ a < b ≤ β, we have

P (X ∈ (a, b)) = P (X ∈ [a, b)) = P (X ∈ (a, b]) = P (X ∈ [a, b])

= F (b|α, β) − F (a|α, β)
b−a
= → dependes only on length b − a of the interval (a, b)/[a, b)/(a, b]/[a, b].
β−α

Conversely, suppose that P (X ∈ I) = P (X ∈ J), for all pairs of intervals I, J ⊆ (α, β) having the same length.
For 0 < s ≤ 1, let G(s) = P (α < X ≤ α + (β − α)s) = F (α + (β − α)s|α, β). Then for 0 < s1 , s2 ≤ 1,
0 < s1 + s2 ≤ 1,

G(s1 + s2 ) = P (α < X ≤ α + (β − α)(s1 + s2 ))

 

= P (α < X ≤ α + (β − α)s1 ) + P  α + (β − α)s1 < X ≤ α + (β − α)(s1 + s2 )

 

| {z }
Depends only on the length (β − α)s2 of (α + (β − α)s1 , α + (β − α)(s1 + s2 ))

= G(s1 ) + P (α < X ≤ α + (β − α)s2 ) = G(s1 ) + G(s2 ).

Pn
By induction, for 0 < si ≤ 1, i = 1, 2, . . . , n, 0 < i=1 si ≤ 1, we have G(s1 + s2 + · · · + sn ) = G(s1 ) + G(s2 ) +
· · · + G(sn ). This implies that
1
G(ms) = mG(s), ∀ 0 < s ≤ , (0.10)
m
Lecture 0 Lecture Notes 0-66

 
s s s s
G(s) = G  + + · · · +  = nG . (0.11)
|n n {z n} n
n times

For m, n ∈ {1, 2, . . . }, m < n, we get

 
m
1 1 1
G = G + + ··· + 
n |n n {z n}
m times

1
= mG , using (0.10)
n
m
= G (1) , using (0.11)
n
m m
= F (β|α, β) = =⇒ G(r) = r, ∀ r ∈ IQ ∩ (0, 1),
n n
where IQ denotes the set of rational numbers. Now let x ∈ (0, 1). Then there exists a sequence {rn }n≥1 in IQ∩(0, 1)
such that rn ↓ x (rationals are dense in (0, 1)). Then, since G is continuous, we have

G(x) = lim G(rn ) = lim rn = x ∀ x ∈ (0, 1).

n→∞ n→∞

This implies

F (α + (β − α)x|α, β) = x, ∀ x ∈ (0, 1)
x−α
=⇒ F (x|α, β) = , x ∈ (α, β)
β−α


0, x < α,

x − α
=⇒ F (x|α, β) = , α ≤ x < β, =⇒ X ∼ U (α, β).

 β−α

1, x ≥ β,


This completes the proof.


tβ tα
Z β
etx  e − e , t 6= 0,

M.g.f. MX (t) = E(etX ) = dx = t(β − α)
α β−α 
1, t = 0.

Theorem 0.120. Let X ∼ U (α, β), −∞ < α < β < ∞. Then,

(i) for a > 0 and b ∈ R, Y = aX + b ∼ U (aα + b, aβ + b),
(ii) for a < 0 and b ∈ R, Y = aX + b ∼ U (aβ + b, aα + b),
X −α
(iii) Z = ∼ U (0, 1).
β−α

Proof. Exercise.

Recall that quantile function is defined by QX (p) = inf{s ∈ R : FX (s) ≥ p}, 0 < p < 1.
Lecture 0 Lecture Notes 0-67

Theorem 0.121. Let X be a r.v. with d.f. F and quantile function Q(·). Then
(i) (Probability Integral Transform)

X is of continuous type =⇒ F (X) ∼ U (0, 1)

d
(ii) U ∼ U (0, 1) =⇒ Q(U ) = X.

Proof. (i) Let G be the d.f. of Y = F (X). Then G(y) = P (F (X) ≤ y), y ∈ R. Clearly, for y < 0, G(y) = 0 and
for y ≥ 1, G(y) = 1. For y ∈ [0, 1),

{s ∈ R : F (s) ≥ y} = {s ∈ R : s ≥ Q(y)}
=⇒ P (F (X) ≥ y) = P (X ≥ Q(y))
=⇒ P (F (X) < y) = P (X < Q(y))
=⇒ P (F (X) < y) = P (X ≤ Q(y)) = F (Q(y)) = y, since X is of continuous type.

Since X is of continuous type P (F (X) = y) = P (x1 ≤ X ≤ x2 ) = 0 for some x1 , x2 with F (x1 ) = F (x2 ). Thus,

P (F (X) ≤ y) = y, ∀ y ∈ (0, 1),


0, if y < 0,

=⇒ G(y) = y, if 0 ≤ y < 1, =⇒ Y ∼ U (0, 1).

1, if y ≥ 1,


(ii) Let U ∼ U (0, 1) and let Z = Q(U ). Then the d.f. of Z is

H(z) = P (Z ≤ z) = P (Q(U ) ≤ z) = P (Q(U ) ≤ z, 0 < U < 1).

Note that for z ∈ (0, 1), {p ∈ R : Q(p) ≤ z} = {p ∈ R : F (z) ≥ p}. Thus, for z ∈ (0, 1)
d
H(z) = P (F (Z) ≥ U, 0 < U < 1) = P (U ≤ F (z)) = F (z) =⇒ Z = Q(U ) = X.

This completes the proof.

Remark 0.122. The above theorem provides a method to generate observations from any arbitrary distributions
using U (0, 1) observations. Suppose that we require an observation X from a distribution having d.f. F and quantile
functions Q. To do so, the above theorem suggests that generate an observation U from U (0, 1) distribution and take
X = Q(U ).

0.14.2. Gamma and Related Distributions

Gamma Function: Γ : (0, ∞) → (0, ∞) is defined as

Z ∞
Γ(α) = e−t tα−1 dt, α > 0.
0

It converges for any α > 0. Integration by parts yields Γ(α + 1) = αΓ(α), α > 0 and Γ(1) = 1. For any n ∈ N,
Z ∞ Z ∞
1 2
Γ(n) = (n − 1)!, Γ = e−t t−1/2 dt = 2 e−x dx
2 0 0
Lecture 0 Lecture Notes 0-68

This implies
2 Z ∞Z ∞
1 2 2
Γ =4 e−(x +y ) dxdy
2 0 0
Z ∞ Z π/2
2
=4 re−r dθdr, (x = r cos θ, y = r sin θ)
0 0
√

1
= π =⇒ Γ = π.
2

Also,
√ √
3 1 1 π 5 3 1 1 1·3 π
Γ = Γ = , Γ = · Γ = ,
2 2 2 2 2 2 2 2 22
√
(2n)! √

2n + 1 1 · 3 · 5 · · · (2n − 1) π
Γ = n
= π, n ∈ N.
2 2 n!4n

Clearly, Z ∞
e−x/θ xα−1 dx = θα Γ(α), α > 0, θ > 0.
0

Definition 0.123. A r.v. X is said to have a gamma distribution with shape parameter α > 0 and scale parameter
θ > 0 (written as X ∼ GAM (α, θ) if its p.d.f. is given by

−x/θ α−1
e
 x
, x > 0,
f (x|α, θ) = θα Γ(α) → family of distributions {GAM (α, θ), α > 0, θ > 0}.

0, otherwise.

X X
Let X ∼ GAM (α, θ) =⇒ ∼ GAM (α, 1) (θ is called scale parameter since the distribution of does not
θ  θ
−z α−1
e z

, z > 0,
depend on θ). The p.d.f. of Z ∼ GAM (α, 1) is f (z) = Γ(α)

0, otherwise.
Also,
Z ∞
1 Γ(α + r)
r
E(Z ) = z α+r−1 e−z dz = , r > −α, α > 0,
Γ(α) 0 Γ(α)
= α(α + 1) · · · (α + r − 1), if r ∈ N.

Mean = µ01 = E(X) = αθ, µ02 = E(X 2 ) = α(α + 1)θ2 , µ2 = σ 2 = Var(X) = αθ2 ,
µ3 = E((X − µ01 )3 ) = µ03 − 3µ01 µ02 + 2(µ01 )3 = 2αθ3 ,
µ4 = E((X − µ01 )4 ) = µ04 − 4µ01 µ03 + 6(µ01 )2 µ02 − 3(µ01 )4 = 3α(α + 2)θ4 ,
µ3 2 µ4 6
Coefficient of skewness = β1 = 2 = √ , Kurtosis = ν1 = 2 = 3 + .
µ2 α µ2 α

For 0 < α ≤ 1, f (x|α, θ) ↓ and for α > 1, f (x|α, θ) ↑ in (0, (α − 1)θ) and ↓ in ((α − 1)θ, ∞).

m.g.f. MX (t) = E(etX ) = E(etθZ ), (Z = X/θ)

Z ∞ Z ∞
1 1 1
= etθz e−z z α−1 dz = e−(1−tθ)z z α−1 dz = (1 − tθ)−α , t < .
Γ(α) 0 Γ(α) 0 θ
Lecture 0 Lecture Notes 0-69

Theorem 0.124. Let X1 , X2 , . . . , Xk be independent r.v.’s such that Xi ∼ GAM (αi , θ), for some αi > 0, θ > 0,
Pk Pk
i = 1, 2, . . . , k. Then Y = i=1 Xi ∼ GAM ( i=1 αi , θ).

Proof. Note that

k k k
Y Y Pk 1 X
MY (t) = MXi (t) = (1 − tθ)−αi = (1 − tθ)− i=1 αi
, t< = m.g.f. of GAM ( αi , θ).
i=1 i=1
θ i=1

This completes the proof.

Theorem 0.125 (Relationship beween Gamma and Poisson distribution). For n ∈ N, θ > 0 and t > 0, let X ∼
GAM (n, θ) and Y ∼ P o(t/θ). Then P (X > t) = P (Y ≤ n − 1), i.e.

∞ n−1
e−t/θ (t/θ)j
Z
1 X
e−x/θ xn−1 dx = .
(n − 1)!θn t j=0
j!

Proof. Use integration by parts.

Remark 0.126. For n ∈ N and θ > 0, let X ∼ GAM (n, θ). Then
∞ n−1
X e−X/θ (X/θ)j X e−X/θ (X/θ)j
∼ U (0, 1) and ∼ U (0, 1) (U ∼ U (0, 1) =⇒ 1 − U ∼ U (0, 1)).
j=n
j! j=0
j!

Definition 0.127. For a θ > 0, a GAM (1, θ) distribution is called exponential distribution with scale parameter θ
(denoted by Exp(θ)).

The p.d.f. of T ∼ Exp(θ) is given by (

1 −t/θ
θe , t > 0,
fT (t) =
0, otherwise.
and its d.f. is given by (
0, t ≤ 0,
FT (t) = P (T ≤ t) = 1 − P (T > t) =
1 − e−t/θ , t > 0.
Mean= E(T ) = θ, variance= θ2 , µ0r = E(T r ) = r!θr , r ∈ N, coefficient of skewness=β1 = 2, Kurtosis=ν1 = 9.
M.g.f.=MT (t) = (1 − tθ)−1 , t < 1/θ and
(
1, t ≤ 0,
P (T > t) =
e−t/θ , t > 0.

For s > 0, t > 0

P (T > s + t)
P (T > s + t|T > s) = = e−t/θ = P (T > t)
P (T > s)
=⇒ P (T > s + t) = P (T > s)P (T > t), ∀ s, t > 0 → Lack of Memory Property.

Let T denote the lifetime of a system. Given that the system has survived s(> 0) units of time the probability that it
will survive t additional units of time is the same as the probability that a fresh system (of age 0) will survive t units
of time. In other words, the system has no memory of its current age or it is not ageing with time.
Lecture 0 Lecture Notes 0-70

Theorem 0.128. Let Y be a r.v. of continuous type with d.f. F such that F (0) = 0. Then Y has LoM property (i.e.
F̄ (s + t) = F̄ (s)F̄ (t) ∀ s, t > 0, where F̄ = 1 − F ) iff Y ∼ Exp(θ), for some θ > 0.

Proof. Let Y ∼ Exp(θ), θ > 0. Then Y has LoM property (already discussed). Now suppose that F (0) = 0 and Y
has LoM property. Then
F̄ (s + t) = F̄ (s)F̄ (t) ∀ s, t > 0,
=⇒ F̄ (s1 + s2 + · · · + sm ) = F̄ (s1 )F̄ (s2 ) · · · F̄ (sm ), si > 0, i = 1, 2, . . . , m,
 
m m
1 1 1 1
=⇒ F̄ = F̄  + + · · · +  = F̄ ∀ m, n ∈ N, (0.12)
n |n n {z n} n
m times
n
1
=⇒ F̄ (1) = F̄ ∀ n ∈ N, (0.13)
n
m m/n
=⇒ F̄ = F̄ (1) ∀ m, n ∈ N. (0.14)
n
Let λ = F̄ (1) so that 0 ≤ λ ≤ 1.

1
λ = 0 =⇒ F̄ = 0 ∀ n ∈ N (using 0.13) =⇒ F̄ (0) = 0 =⇒ F (0) = 1 (contradiction, since F (0) = 0)
n
m
λ = 1 =⇒ F̄ (m) = F̄ (1) = 1 ∀ m ∈ N =⇒ lim F̄ (m) = 1 =⇒ lim F (m) = 0 → contradiction.
m→∞ m→∞

Thus λ ∈ (0, 1). Let λ = e−1/θ , θ > 0 (θ = −1/ ln λ). Then using (0.14), F̄ (r) = e−r/θ ∀ r ∈ IQ ∩ (0, ∞). Let
x ∈ IQ ∩ (0, ∞). Then there exists a sequence {rn }n≥1 in IQ ∩ (0, ∞) such that rn → x. Then

F̄ (x) = F̄ lim rn = lim F̄ (rn ) = lim e−rn /θ = e−x/θ ,
n→∞ n→∞ n→∞
(
0, x < 0,
=⇒ F (x) = =⇒ Y ∼ Exp(θ).
1 − e−x/θ , x ≥ 0,
This completes the proof.
Example 0.129. X : Waiting time for occurrence of an event E. Suppose that X ∼ Exp(3). Then the conditional
probability that the waiting time for occurrences of E is atleast 5 hrs given that it has not occurred in first two hrs
=P (X > 5|X > 2) = P (X > 3) = e−1 .

Chi-squared Distribution: Let n ∈ N. Then GAM n2 , 2 distribution is called Chi-squared distribution with n

degrees of freedom (denoted by χ2n ). Let X ∼ χ2n . The p.d.f. of X is


−x/2 n/2−1
e
 x
, x > 0,
fX (x) = n/2
2 Γ(n/2)

0, otherwise.
q
Mean = E(X) = n, Var(X) = µ2 = σ 2 = 2n, coefficient of skewness = β1 = 2 n2 , Kurtosis = ν1 = 3 + 12
n , m.g.f.
MX (t) = (1 − 2t)−n/2 , t < 12 .
Pk
Theorem 0.130. Let X1 , X2 , . . . , Xk be independent with Xi ∼ χ2ni , ni ∈ N, i = 1, 2, . . . , k. Then i=1 Xi ∼ χ2n ,
Pk
where n = i=1 ni .

For various values of n ∈ N and α ∈ (0, 1), tables for (1 − α)th quantile of χ2n distribution (i.e. τn,α satisfying
P (χ2n ≤ τn,α ) = 1 − α) are available in various textbook.
Lecture 0 Lecture Notes 0-71

0.14.3. Beta Distribution

For α > 0 and β > 0, we have

Z ∞Z ∞
Γ(α)Γ(β) = e−(s+t) sα−1 tβ−1 dsdt
0 0
Z 1 Z ∞
= e−v (uv)α−1 ((1 − u)v)β−1 |v|dvdu,
0 0
making transformation: s = uv, t = (1 − u)v, Jacobian : J = v
Z 1
= Γ(α + β) uα−1 (1 − u)β−1 du
0
Z 1
Γ(α)Γ(β)
=⇒ = uα−1 (1 − u)β−1 du → Beta function (function of (α, β), α > 0, β > 0).
Γ(α + β) 0

Note that B(α, β) = B(β, α), ∀ α, β > 0.

Definition 0.131. For given α > 0 and β > 0, a r.v. X is said to have the beta distribution with parameter (α, β)
(written as X ∼ Be(α, β)) if its p.d.f. is given by

α−1
x
 (1 − x)β−1
, 0 < x < 1,
f (x|α, β) = B(α, β)

0, otherwise.

Suppose that X ∼ Be(α, β)), for some α > 0 and β > 0. Then

B(α + r, β) Γ(α + r)Γ(α + β)

E(X r ) = = , r > −α,
B(α, β) Γ(α)Γ(α + β + r)
α α(α + 1)
Mean = µ01 = E(X) = , µ02 = E(X 2 ) = ,
α+β (α + β)(α + β + 1)
αβ
µ2 = σ 2 = Var(X) = ,
(α + β)2 (α + β + 1)
α−1
Mode = M0 = , if α > 1 and α + β > 2,
α+β−2
√
µ23 2(β − α) α + β + 1
Skewness = β1 = 3 = √ ,
µ2 αβ(α + β + 2)
µ4 6[(α − β)2 (α + β + 1) − αβ(α + β + 2)]
Kurtosis = ν1 = 2 = +3
µ2 αβ(α + β + 2)(α + β + 3)
6[α3 + α2 (1 − 2β) + β 2 (1 + β) − 2αβ(2 + β)]
= .
αβ(α + β + 2)(α + β + 3)

Let X ∼ Be(α, α), α > 0. Then


α−1
x
 (1 − x)α−1
, 0 < x < 1,
f (x|α, β) = B(α, α)

0, otherwise.

d 1 d 1
Clearly, X = 1 − X =⇒ X − 2 = 2 − X. Thus, if X ∼ Be(α, α). Then the distribuion of X is symmetric about
1/2.
Lecture 0 Lecture Notes 0-72

Theorem 0.132 (Relationship between Beta and Binomial Distribution). For m, n ∈ N and x ∈ (0, 1), let X ∼
Be(m, n) and Y ∼ Bin(m + n − 1, x). Then P (X ≤ x) = P (Y ≥ m), i.e.

x m+n−1
X
m+n−1 j
Z
1 m−1 n−1
t (1 − t) dt = x (1 − x)m+n−1−j .
B(m, n) 0 j=m
j

Proof. Fix m, n ∈ N and x ∈ (0, 1). Let

Z x
(m + n − 1)!
Im,n = LHS = tm−1 (1 − t)n−1 dt
(m − 1)!(n − 1)! 0
(m + n − 1)! x m

m+n−1 m
Z
n−1
= x (1 − x) + t (1 − t)n−2 dt
m m!(n − 2)! 0

m+n−1 m
= x (1 − x)n−1 + Im+1,n−1 .
m

Proceeding recursively give the result.

Z 1
1
m.g.f. MX (t) = E(etX ) = etx xα−1 (1 − x)β−1 dx
B(α, β) 0
 
Z 1 X ∞ j j
1 t x  α−1
=  x (1 − x)β−1 dx
B(α, β) 0 j=0
j!
X tj ∞ Z 1 ∞
X B(α + j, β)tj
1 1
= xj+α−1 (1 − x)β−1 dx = , t ∈ R.
B(α, β) j=0 j! 0 B(α, β) j=0 j!

1 2
Example 0.133. Time (in hours) to finish a job follows beta distribution with mean 3 hrs. and variance 63 hrs. Find
the probability that the job will be finished in 30 minutes.

Solution: Define X = time to finish job (in hours) ∼ Be(α, β), say.
αβ
E(X) = 31 =⇒ α+β α
= 31 , Var(X) = 2
63 =⇒ (α+β)2 (α+β+1) = 2
63 . This implies α = 2 and β = 4. Thus,
X ∼ Be(2, 4). Required probability
Z 1/2
1 1 13
P (X < )= x(1 − x)3 dx = .
2 B(2, 4) 0 16

0.14.4. Normal Distribution

Recall that
Z ∞ Z ∞
√ 1 2
π=Γ = e−t t−1/2 dt = 2 e−x dx
2 0 0
Z ∞
2
= e−x dx
−∞
Z ∞
1 2
=√ e−t /2 dt
2 −∞
Lecture 0 Lecture Notes 0-73

Z ∞
1 2
=⇒ √ e−t /2 dt = 1
2π −∞
Z ∞
1 2 2
=⇒ √ e−(t−µ) /2σ dt = 1 ∀ µ ∈ R and σ > 0.
σ 2π −∞
Definition 0.134. Let µ ∈ R and σ > 0 be given constants. An absolutely continuous type r.v. is said to follow a
normal distribution with patrameters µ ∈ R and σ > 0 (written as X ∼ N (µ, σ 2 )) if its p.d.f. is given by
1 1 x−µ 2
f (x|µ, σ) = √ e− 2 ( σ ) , −∞ < x < ∞.
σ 2π

The N (0, 1) distribution is called standard normal distribution. The p.d.f. and d.f. of a standard normal distribution
are denoted by φ(z) and Φ(z), respectively, so that
Z z Z z
1 −z2 /2 1 2
φ(z) = √ e , −∞ < z < ∞, Φ(z) = φ(t)dt = √ e−t /2 dt, z ∈ R.
2π −∞ 2π −∞
1 x2
X ∼ N (µ, σ 2 ) =⇒ f (µ − x|µ, σ) = f (µ + x|µ, σ) = √ e− 2σ2 , x ∈ R.
σ 2π
d 1
This implies X − µ = µ − X (distribution of X is symmetric about µ) =⇒ E(X) = µ and F (µ|µ, σ) = 2.
Moreover,

P (X − µ ≤ x) = P (µ − X ≤ x) =⇒ F (µ + x|µ, σ) = 1 − F (µ − x|µ, σ) ∀ x ∈ R.

In particular,
1
Φ(0) =and Φ(−z) + Φ(z) = 1 ∀ z ∈ R.
2
The p.d.f. f (x|µ, σ) ↑ in (−∞, µ) and ↓ in (µ, ∞) =⇒ mode = m0 = µ. Thus mean = median = mode = µ.
Let X ∼ N (µ, σ 2 ). Then m.g.f. of X is
Z ∞
1 1 x−µ 2 x−µ
MX (t) = E(etX ) = etx √ e− 2 ( σ ) dx, take = z, x = (µ + σz)
−∞ σ 2π σ
Z ∞
1 2
= et(µ+σz) √ e−z /2 dz
−∞ 2π
Z ∞
µt+ 12 σ 2 t2 1 1 2 2 2
=e √ e− 2 (z −2σtz+σ t ) dz
−∞ 2π
Z ∞
µt+ 12 σ 2 t2 1 1 2 1 2 2
=e √ e− 2 (z−σt) dz = eµt+ 2 σ t , t ∈ R.
−∞ 2π
X −µ
· Let X ∼ N (µ, σ 2 ). Then Z = ∼ N (0, 1).
σ
(X−µ) 2 2
MZ (t) = E(et σ ) = e−µt/σ MX (t/σ) = e−µt/σ eµt/σ+t /2
= et /2
∀ t ∈ R →m.g.f. of N (0, 1)
=⇒ Z ∼ N (0, 1).

Theorem 0.135. Let X ∼ N (µ, σ 2 ) and Z ∼ N (0, 1).

(a) For a 6= 0, b ∈ R, Y = aX + b ∼ N (aµ + b, a2 σ 2 ).
d X−µ
(b) Z = σ ∼ N (0, 1).
Lecture 0 Lecture Notes 0-74

(d) Mean = µ01 = E(X) = µ; Variance = µ2 = σ 2 ; coefficient of skewness = β1 = 0; kurtosis = ν1 = 3.

(e) Z 2 ∼ χ21 .

Proof. (a) Note that

MY (t) = E(etY ) = E et(aX+b)

= ebt E e(ta)X = ebt MX (at)
2
1
(at)2
= ebt eµ(at)+ 2 σ
2
1
σ 2 )t2
= e(aµ+b)t+ 2 (a , t ∈ R =⇒ Y ∼ N (aµ + b, a2 σ 2 ).

(b) Follows from (a) by taking a = σ1 and b = − σµ .

2 P∞ 2k
(c) MZ (t) = et /2 = k=0 2tk k! , t ∈ R.
(
tr 0, if r = 1, 3, 5, . . . ,
r
E(Z ) = Coefficient of in the expansion of MZ (t) = r!
r! 2r/2 (r/2)!
, if r = 2, 4, 6, . . . .

d X−µ
(d) Z = σ .

X −µ
E = E(Z) = 0 =⇒ µ01 = E(X) = µ,
σ
2 !
X −µ
E = E(Z 2 ) = 1 =⇒ µ2 = E((X − µ)2 ) = σ 2 ,
σ
3 !
X −µ
E = E(Z 3 ) = 0 =⇒ µ3 = E((X − µ)3 ) = 0,
σ
4 !
X −µ
E = E(Z 4 ) = 3 =⇒ µ4 = 3σ 4 = 3,
σ
µ3 µ4
Coefficient of skewness = β1 = 2 = 0, kurtosis = 2 = 3.
µ2 µ2

(e) Let Y = Z 2 . Then

Z ∞ Z ∞
2 1 2 2 1 (1−2t) 2 1
MY (t) = E(etZ ) = √ etz e−z /2
dz = √ e− 2 z
dz = (1 − 2t)−1/2 , t < =⇒ Z 2 ∼ χ21 .
2π −∞ 2π −∞ 2

This completes the proof.

Corollary 0.136. Let X1 , X2 , . . . , Xk be independent and let Xi ∼ N (µi , σi2 ), −∞ < µi < ∞, σi > 0, i =
Pk 2
1, 2, . . . , k. Then i=1 Xiσ−µ
i
i
∼ χ2k .

Remark 0.137. (i) In N (µ, σ 2 ) distribution the parameters µ ∈ (−∞, ∞) and σ 2 > 0 are respectively, the mean and
variance of the distribution.
Lecture 0 Lecture Notes 0-75

(ii) If X ∼ N (µ, σ 2 ), then

X −µ x−µ x−µ
P (X ≤ x) = P ≤ =Φ , x ∈ R.
σ σ σ

Let τα be the (1 − α)th quantile of Φ then Φ(−τα ) = 1 − Φ(τα ) = α. Tables for values of Φ(x) for different values
of x are available in various text books.
Example 0.138. Let X ∼ N (2, 4). Find P (X ≤ 0), P (|X| ≥ 2), P (1 < X ≤ 3) and P (X ≤ 3|X > 1).

0−2

Solution: P (X ≤ 0) = Φ = Φ(−1) = 1 − Φ(1) = .1587,
2

−2 − 2 2−2
P (|X| ≥ 2) = P (X ≤ −2) + P (X ≥ 2) = Φ +1−Φ
2 2
= Φ(−2) + 1 − Φ(0) = 0.0228 + 0.5 = 0.5228,

3−2 1−2
P (1 < X ≤ 3) = P (X ≤ 3)−P (X ≥ 1) = Φ +1−Φ = 2Φ(0.5)−1 = 2×0.6915−1 = 0.383,
2 2
P (1 < X ≤ 3) .383 0.383
P (X ≤ 3|X > 1) = = = = 0.55599.
P (X > 1) 1 − Φ 1−22
Φ(0.5)

Theorem 0.139. Let X1 , X2 , . . . , Xk be independent r.v.’s and let Xi ∼ N (µi , σi2 ), i = 1, 2, . . . , k. Let a1 , a2 , . . . , ak
Pk Pk Pk Pk
be real constants such that i=1 a2i > 0. Then Y = i=1 ai Xi ∼ N ( i=1 ai µi , i=1 a2i σi2 ).

Proof. Note that

k
! k
Pk Y Y
t i=1 ai Xi tai Xi
MY (t) = E(e )=E e = E(etai Xi ), (independent of Xi ’s)
i=1 i=1
k k
( k 2 2
P
2 2 2 i=1 ai σi )t
Y Y 1
Pk
= MXi (tai ) = eµi tai + 2 σi t ai
= e( i=1 ai µi )t+ 2

i=1 i=1
k k
!
X X
→ m.g.f. of N ai µi , a2i σi2 .
i=1 i=1
Pk Pk
By uniqueness of m.g.f.’s Y ∼ N ( i=1 ai µi , i=1 a2i σi2 ).

0.15. Random Vectors and their Distribution Functions

Let (Ω, F, P ) be a given probability space. In many situations we may be interested in simultaneously studying two
or more numerical characteristics of outcomes of a random experiment. This amounts to define a function

X = (X1 , X2 , . . . , Xp ) : Ω → Rp .

Example 0.140. A fair coin is tossed three times independently. Then

1
Ω = {HHH, HHT, HT H, T HH, HT T, T HT, T T H, T T T } and P ({ω}) = ∀ ω ∈ Ω.
8
Suppose that we are simultaneously interested in:
Lecture 0 Lecture Notes 0-76

· number of heads in three tosses,

· number of heads in first two tosses.
Here we are interested in the function (X, Y ) : Ω → R2 defined by

 (0, 0) if ω = T T T,




 (1, 0) if ω = T T H,


(1, 1) if ω = HT T, T HT,
(X(ω), Y (ω)) =

 (2, 1) if ω = HT H, T HH,




 (2, 2) if ω = HHT,


(3, 2) if ω = HHH.

The values assumed by (X, Y ) are random with

1


 , if (x, y) ∈ {(0, 0), (1, 0), (2, 2), (3, 2)},
8



Pr{(X, Y ) = (x, y)} = 1 , if (x, y) ∈ {(1, 1), (2, 1)},
4




0, otherwise.


Here, Pr((X, Y ) ∈ {(0, 0), (1, 0), (2, 2), (3, 2), (1, 1), (2, 1)}) = 1.
Definition 0.141. Let (Ω, F, P ) be a given probability space. A function X = (X1 , X2 , . . . , Xp ) : Ω → Rp (defined
on the sample space Ω) is called a random vector (p-dimensional random vector). A one dimensional random vector
is simply called a random variable.

For any function Y = (Y1 , Y2 , . . . , Yp ) : Ω → Rp and A ⊆ Rp , define Y −1 = {ω ∈ Ω : Y (ω) ∈ A}. For

probability space (Ω, F, P ) and a p-dimensional random vector X = (X1 , X2 , . . . , Xp ) : Ω → Rp , define PX (B) =
P (X −1 (B)), B ∈ Bp where for all practical purpose we take Bp to be power set of Rp . We will simply write

PX (B) = P ({ω ∈ Ω : X(ω) ∈ B}) = Pr(X ∈ B), B ∈ Bp .

X
The following scenario has emerged: (Ω, F, P ) −→ (Rp , Bp , PX ).
Theorem 0.142. (Rp , Bp , PX ) defined above is a probability space, i.e. PX (·) is a probability function defined on
Bp .

Proof. Similar to the proof of random variable case.

Definition 0.143. The probability function PX (·) defined above is called the probability function / measure induced
by random vector X and (Rp , Bp , PX ) is called the probability space induced by random vector X.
The induced probability measure PX (·) describes the random behaviour of X.
Example 0.144. Consider the sample space defined in Example 0.140, where
1
Ω = {HHH, HHT, HT H, T HH, HT T, T HT, T T H, T T T } and P ({ω}) = ∀ω∈Ω
8
Lecture 0 Lecture Notes 0-77

and (X, Y ) : Ω → R2 is defined by


 (0, 0) if ω = T T T,




 (1, 0) if ω = T T H,


(1, 1) if ω = HT T, T HT,
(X(ω), Y (ω)) =

 (2, 1) if ω = HT H, T HH,




 (2, 2) if ω = HHT,


(3, 2) if ω = HHH.

Here, (X, Y ) : Ω → R2 is a random vector with induced probability space (R2 , B2 , PX ), where

1


 , if (i, j) ∈ {(0, 0), (1, 0), (2, 2), (3, 2)},
8


PX ({(i, j)}) = 1 , if (i, j) ∈ {(1, 1), (2, 1)},
4




0, otherwise,


and for any B ∈ B2

X
PX (B) = PX ({(i, j)}), where S = {(0, 0), (1, 0), (2, 2), (3, 2), (1, 1), (2, 1)}.
(i,j)∈B∩S

Definition 0.145. (a) The joint distribution function of a p-dimensional random vector X = (X1 , X2 , . . . , Xp ) is
defined as

FX (x1 , x2 , . . . , xp ) = Pr(X1 ≤ x1 , X2 ≤ x2 , . . . , Xp ≤ xp ), x = (x1 , x2 , . . . , xp ) ∈ Rp .

(b) The joint d.f. of any subset of random vectors (X1 , X2 , . . . , Xp ) is called a marginal distribution function of FX (·)
(or X = (X1 , X2 , . . . , Xp )).
Example 0.146. FX1 ,X2 (x, y), (x, y) ∈ R2 , FX2 (x), x ∈ R and FX1 ,X2 ,X3 (x, y, z), (x, y, z) ∈ R3 are marginal d.f.s
of FX1 ,X2 ,X3 ,X4 (x1 , x2 , x3 , x4 ), (x1 , x2 , x3 , x4 ) ∈ R4 .
In the sequel we will describe a notation for writing down all the vertices of a p-dimensional rectangle in a compact
form.
For −∞ ≤ ai < bi < ∞, i = 1, 2, a = (a1 , a2 ) and b = (b1 , b2 ), the vertices of two dimensional rectangle

(a, b] = (a1 , b1 ] × (a2 , b2 ] = {(x, y) ∈ R2 : a1 < x ≤ b1 , a2 < y ≤ b2 }

are

{(b1 , b2 ), (a1 , b2 ), (b1 , a2 ), (a1 , a2 )} = {(b1 , b2 )} ∪ {(a1 , b2 ), (b1 , a2 )} ∪ {(a1 , a2 )} = ∆0,2 ∪ ∆1,2 ∪ ∆2,2 , say.

Similarly, for −∞ ≤ ai < bi < ∞, i = 1, 2, 3, a = (a1 , a2 , a3 ) and b = (b1 , b2 , b3 ), the vertices of three
dimensional rectangle

(a, b] = (a1 , b1 ] × (a2 , b2 ] × (a3 , b3 ] = {(x1 , x2 , x3 ) ∈ R3 : ai < xi ≤ bi , i = 1, 2, 3}

are

{(b1 , b2 , b3 ), (a1 , b2 , b3 ), (b1 , a2 , b3 ), (b1 , b2 , a3 ), (a1 , a2 , b3 ), (a1 , b2 , a3 ), (b1 , a2 , a3 ), (a1 , a2 , a3 )}

= {(b1 , b2 , b3 )} ∪ {(a1 , b2 , b3 ), (b1 , a2 , b3 ), (b1 , b2 , a3 )} ∪ {(a1 , a2 , b3 ), (a1 , b2 , a3 ), (b1 , a2 , a3 )} ∪ {(a1 , a2 , a3 )}
Lecture 0 Lecture Notes 0-78

= ∆0,3 ∪ ∆1,3 ∪ ∆2,3 ∪ ∆3,3 , say.

In general, for −∞ ≤ ai < bi < ∞, i = 1, 2, . . . , p, a = (a1 , a2 , . . . , ap ) and b = (b1 , b2 , . . . , bp ) define

p
∆k,p ≡ ∆k,p ((a, b]) = {z ∈ Rp : zi ∈ {ai , bi }, i = 1, 2, . . . , p and exactly k of zi0 s are a0j s} (→ has elements)
k

where (a, b] = (a1 , b1 ] × (a2 , b2 ] × · · · × (ap , bp ].

Sp Pp
Then k=0 ∆k,p is the set of 2p = k=0 kp vectors of p-dimensional rectangle (a, b].

Theorem 0.147. For constants −∞ ≤ ai < bi < ∞, i = 1, 2, . . . , p

p
X X
Pr(ai < Xi ≤ bi , i = 1, 2, . . . , p) = (−1)k FX (z).
k=0 z∈∆k,p ((a,b])

Proof. Special cases:

Case I: p = 1
We have ∆0,1 ((a1 , b1 ]) = {b1 } and ∆1,1 ((a1 , b1 ]) = {a1 }. Then

R.H.S. = FX1 (b1 ) − FX1 (a1 ) = Pr(a1 < X1 ≤ b1 ) = L.H.S.

Case II: p = 2
Here ∆0,2 = {(b1 , b2 )}, ∆1,2 = {(a1 , b2 ), (b1 , a2 )} and ∆2,2 = {(a1 , a2 )}. Thus

R.H.S. = FX (b1 , b2 ) − FX (a1 , b2 ) − FX (b1 , a2 ) + FX (a1 , a2 )

= Pr(X1 ≤ b1 , X2 ≤ b2 ) − Pr(X1 ≤ a1 , X2 ≤ b2 ) − Pr(X1 ≤ b1 , X2 ≤ a2 ) + Pr(X1 ≤ a1 , X2 ≤ a2 )
= Pr(a1 < X1 ≤ b1 , X2 ≤ b2 ) − Pr(a1 < X1 ≤ b1 , X2 ≤ a2 )
= Pr(a1 < X1 ≤ b1 , a2 < X2 ≤ b2 ) = L.H.S.

Case III: p = 3

Pr(a1 < X1 ≤ b1 , a2 < X2 ≤ b2 , a3 < X3 ≤ b3 )

= Pr(a1 < X1 ≤ b1 , a2 < X2 ≤ b2 , X3 ≤ b3 ) − Pr(a1 < X1 ≤ b1 , a2 < X2 ≤ b2 , X3 ≤ a3 )
= Pr(a1 < X1 ≤ b1 , X2 ≤ b2 , X3 ≤ b3 ) − Pr(a1 < X1 ≤ b1 , X2 ≤ a2 , X3 ≤ b3 )
− {Pr(a1 < X1 ≤ b1 , X2 ≤ b2 , X3 ≤ a3 ) + Pr(a1 < X1 ≤ b1 , X2 ≤ a2 , X3 ≤ a3 )}
= Pr(X1 ≤ b1 , X2 ≤ b2 , X3 ≤ b3 ) − Pr(X1 ≤ a1 , X2 ≤ b2 , X3 ≤ b3 ) − Pr(X1 ≤ b1 , X2 ≤ a2 , X3 ≤ b3 )
+ Pr(X1 ≤ a1 , X2 ≤ a2 , X3 ≤ b3 ) − Pr(X1 ≤ b1 , X2 ≤ b2 , X3 ≤ a3 ) + Pr(X1 ≤ a1 , X2 ≤ b2 , X3 ≤ a3 )
+ Pr(X1 ≤ b1 , X2 ≤ a2 , X3 ≤ a3 ) − Pr(X1 ≤ a1 , X2 ≤ a2 , X3 ≤ a3 )
= FX (b1 , b2 , b3 ) − FX (a1 , b2 , b3 ) − FX (b1 , a2 , b3 ) + FX (a1 , a2 , b3 ) − FX (b1 , b2 , a3 ) + FX (a1 , b2 , a3 )
3
X X
+ FX (b1 , a2 , a3 ) − FX (a1 , a2 , a3 ) = (−1)k FX (z).
k=0 z∈∆k,3 ([a,b])

The proof can be completed using method of induction.

The following theorem provides a technique to find marginal distributions.

Lecture 0 Lecture Notes 0-79

Theorem 0.148. Let F (x1 , x2 , . . . , xp ), x = (x1 , x2 , . . . , xp ) ∈ Rp be a d.f. of p-dimensional random vector X =

(X1 , X2 , . . . , Xp ). Then the marginal distribution function of Y = (X1 , X2 , . . . , Xp−1 ) is

G(x1 , x2 , . . . , xp−1 ) = lim F (x1 , x2 , . . . , xp−1 , t), y = (x1 , x2 , . . . , xp−1 ) ∈ Rp−1 .

t→∞

Proof. For y = (x1 , x2 , . . . , xp−1 ) ∈ Rp−1

G(x1 , x2 , . . . , xp−1 ) = Pr(X1 ≤ x1 , . . . , Xp−1 ≤ xp−1 )

= Pr(X1 ≤ x1 , . . . , Xp−1 ≤ xp−1 , Xp < ∞)
∞
!
[
= Pr {X1 ≤ x1 , . . . , Xp−1 ≤ xp−1 , Xp ≤ t}
t=1
= lim Pr(X1 ≤ x1 , . . . , Xp−1 ≤ xp−1 , Xp ≤ t) = lim F (x1 , x2 , . . . , xp−1 , t).
t→∞ t→∞

This completes the proof.

Theorem 0.149. Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional random vector with d.f. F (·). Then
(a) lim
xi →∞
F (x1 , x2 , . . . , xp ) = 1,
i=1,2,...,p

(b) for each i = 1, 2, . . . , p, lim F (x1 , x2 , . . . , xp ) = 0,

xi →−∞

(c) F (x) is right continuous in each argument (keeping other arguments fixed),
(d) for each rectangle (a, b] ⊆ Rp
p
X X
(−1)k F (z) ≥ 0.
k=0 z∈∆k,p ((a,b])

Conversely, any function G : Rp → [0, 1] satisfying conditions (a) − (d) above is a d.f. of some p-dimensional random
vector.

Proof. For simplicity, we provide the proof for p = 2.

(a) Note that

lim F (x1 , x2 ) = lim Pr({X1 ≤ x1 , X2 ≤ x2 })

x1 →∞,x2 →∞ x1 →∞,x2 →∞

= lim Pr({X1 ≤ n, X2 ≤ n}), (since limit exists)

n→∞
[∞
= Pr( {X1 ≤ n, X2 ≤ n}) = Pr({X1 < ∞, X2 < ∞}) = 1.
n=1

(b) For fixed x2 ∈ R,

lim F (x1 , x2 ) = lim Pr({X1 ≤ −n, X2 ≤ x2 })

x1 →−∞ n→∞
\∞
= Pr( {X1 ≤ −n, X2 ≤ x2 }) = Pr(φ) = 0.
n=1

Similarly, lim F (x1 , x2 ) = 0.

x2 →−∞
Lecture 0 Lecture Notes 0-80

(c) Let {hn }n≥1 be a sequence in R such that hn ↓ 0. Then for (x1 , x2 ) ∈ R2
lim F (x1 + hn , x2 ) = lim Pr({X1 ≤ x1 + hn , X2 ≤ x2 })
n→∞ n→∞

1
= lim Pr X1 ≤ x1 + , X2 ≤ x2 , (as limit exists)
n→∞ n
∞ !
\ 1
= Pr X1 ≤ x1 + , X2 ≤ x2 = Pr({X1 ≤ x1 , X2 ≤ x2 }) = F (x1 , x2 ),
n=1
n

i.e. for every fixed x2 ∈ R, F (x1 , x2 ) is right continuous in x1 ∈ R. Similarly, it can be shown that for every fixed
x1 ∈ R, F (x1 , x2 ) is right continuous in x2 ∈ R.
(d) For −∞ < a1 < b1 < ∞ and −∞ < a2 < b2 < ∞, we have
2
X X
(−1)k F (z) = F (b1 , b2 ) − F (b1 , a2 ) − F (a1 , b2 ) + F (a1 , a2 )
k=0 z∈∆k,p ((a,b])

= P (a1 < X1 ≤ b1 , a2 < X2 ≤ b2 ) ≥ 0.

This completes the proof.
Remark 0.150. (a) For p = 1, (d) of the above theorem reduces to F (b) − F (a) ≥ 0, ∀ − ∞ < a < b < ∞, i.e.,
F (·) is monotone on R.
(b) F (·) is clearly non-decreasing in each argument.

0.15.1. Independent Random Variables

For an arbitary (countable or uncountable) set ∆, let {Xλ : λ ∈ ∆} be a family of random variables.
Definition 0.151. The random variables Xλ , λ ∈ ∆ are said to be mutually independent if for any finite subcollection
{Xλ1 , Xλ2 , . . . , Xλp } in {Xλ : λ ∈ ∆}
p
Y
Fλ1 ,λ2 ,...,λp (x1 , x2 , . . . , xp ) = Fλi (xi ) ∀ x = (x1 , x2 , . . . , xp ) ∈ Rp ,
i=1

where Fλ1 ,λ2 ,...,λp (·) denotes the joint d.f. of (Xλ1 , Xλ2 , . . . , Xλp ) and Fλi (·), i = 1, 2, . . . , p denotes the marginal
d.f. of Xλi .

The random variables Xλ , λ ∈ ∆ are said to be pairwise independent if for any λ1 , λ2 ∈ ∆ (λ1 6= λ2 )
Fλ1 ,λ2 (x1 , x2 ) = Fλ1 (x1 )Fλ2 (x2 ) ∀ x = (x1 , x2 ) ∈ R2 .
Remark 0.152. (a) Random variables {Xλ , λ ∈ ∆} are independent iff those in any finite subset of {Xλ : λ ∈ ∆}
are independent.
(b) Let ∆1 ⊆ ∆2 . Then r.v.s {Xλ , λ ∈ ∆2 } are independent =⇒ r.v.s {Xλ , λ ∈ ∆1 } are independent. In particular,
if r.v.s in a collection are independent then they are pairwise independent. The converse may not be true.
Theorem 0.153. For any positive integer p (≥ 2) the random variables X1 , X2 , . . . , Xp are independent iff
p
Y
F (x1 , x2 , . . . , xp ) = FXi (xi ) ∀ x = (x1 , x2 , . . . , xp ) ∈ Rp , (0.15)
i=1

where F (·) is the joint d.f. of X = (X1 , X2 , . . . , Xp ).

Lecture 0 Lecture Notes 0-81

Proof. Obviously, if X1 , X2 , . . . , Xp are independent then (0.15) holds. Conversely suppose that (0.15) holds. Con-
sider a subset of {X1 , X2 , . . . , Xp }. For simplicity let this subset be {X1 , X2 , . . . , Xq }, for some 2 ≤ q ≤ p. Thus
for x = (x1 , x2 , . . . , xq ) ∈ Rq the joint (marginal) d.f. of (X1 , X2 , . . . , Xq ) is
p
Y q
Y
G(x1 , x2 , . . . , xq ) = lim F (x1 , x2 , . . . , xq , xq+1 , . . . , xp ) =
x →∞
lim
x →∞
FXj (xj ) = FXj (xj ),
i i
i=q+1,...,p i=q+1,...,p j=1 j=1

∀ x = (x1 , x2 , . . . , xq ) ∈ Rq . Here FXj (·) is the marginal d.f. of Xj , j = 1, 2, . . . , q.

0.15.2. Discrete Random Vectors

Let X = (X1 , X2 , . . . , Xp ) be p-dimensional random vector with d.f. F (·).

Definition 0.154. (a) The random vector X = (X1 , X2 , . . . , Xp ) is said to be a discrete random vector if there exists
a countable set S (finite or infinite) such that
Pr(X = x) > 0 ∀ x ∈ S, and Pr(X ∈ S) = 1.
The set S is called support of random vector X (or of F ).
(b) The joint p.m.f. of X having support S is defined by
(
Pr(X = x), if x ∈ S,
f (x) =
0, otherwise.

Remark 0.155. (a) Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional discrete random vector with p.m.f. f (·) and d.f.
F (·) and support S. Then, for any A ⊆ Rp
X
Pr(X ∈ A) = Pr(X ∈ A ∩ S) = f (x), (Pr(X ∈ S) = 1, A ∩ S ⊆ S and thus A ∩ S is a countable set).
x∈A∩S
X
Moreover, F (x) = f (y), x ∈ Rp .
y∈S∩(−∞,x]

(b) Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional discrete random vector with p.m.f. f (·) and support S. Then the
p.m.f. f : Rp → R satisfies:
X
(i) f (x) > 0 ∀ x ∈ S and f (x) = 0 ∀ x ∈ S c , (ii) f (x) = 1.
x∈S

Conversely suppose that g : Rp → R is a function such that for some countable set T
X
(i) g(x) > 0 ∀ x ∈ T and g(x) = 0 ∀ x ∈ T c , (ii) g(x) = 1.
x∈T

Then g(·) is a p.m.f. of some p-dimensional discrete random vector having support T .
(c) Marginal distributions of discrete random vector are discrete.
Theorem 0.156. Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional discrete random vector with p.m.f. f (·) and support
S. Then the marginal distribution of any subset of {X1 , X2 , . . . , Xp } (say that of Y = (X1 , X2 , . . . , Xq ), 1 ≤ q < p)
is again discrete with p.m.f.
(P P P
xq+1 xq+2 · · · xp f (x), if x ∈ T,
g(x1 , x2 , . . . , xq ) =
0, otherwise
Lecture 0 Lecture Notes 0-82

and support

T = {y = (y1 , y2 , . . . , yq ) ∈ Rq : (y1 , y2 , . . . , yq , yq+1 , . . . , yp ) ∈ S, for some (yq+1 , yq+2 , . . . , yp ) ∈ Rp−q }.

Proof. Follows using theorem of total probability.

Conditional distribution of discrete random vectors

Let Y = (Y1 , Y2 , . . . , Yp ), Z = (Z1 , Z2 , . . . , Zq ) and X = (Y , Z) = (Y1 , Y2 , . . . , Yp , Z1 , Z2 , . . . , Zq ) be random
vectors with p.m.f. f1 , f2 and f , respectively. Suppose X, Y and Z have support S, S1 , S2 , respectively. For fixed
z ∈ S2 define
Tz = {y = (y1 , y2 , . . . , yp ) ∈ Rp : (y, z) ∈ S}.

For fixed z ∈ S2 , the conditional p.m.f. of Y given Z = z is defined by


 f (y, z)
Pr(X = (y, z))  , y ∈ Tz ,
f (y|z) = Pr(Y = y|Z = z) = = f2 (z)
Pr(Z = z) 
0, otherwise.

Clearly for each z ∈ S2 , f (·|z) is a proper p.m.f. with support Tz . Also for fix z ∈ S2

Pr(Y1 ≤ y1 , . . . , Yq ≤ yq , Z = z) X f (s, z) X
Pr(Y1 ≤ y1 , . . . , Yq ≤ yq |Z = z) = = = f (s|z).
Pr(Z = z) fZ (z)
s∈Tz ,s≤y s∈Tz ,s≤y

Theorem 0.157. Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional random vector with support S and p.m.f. f (·). Let
fi (·) denote the marginal p.m.f. of Xi , i = 1, 2, . . . , p. Then X1 , . . . , Xp are independent iff
p
Y
f (x1 , x2 , . . . , xp ) = fi (xi ) ∀ x ∈ S.
i=1

Proof. (For p = 2)
Suppose that f (x1 , x2 ) = f1 (x1 )f2 (x2 ) ∀ x = (x1 , x2 ) ∈ S. Then the d.f. of X = (X1 , X2 ) is
XX XX
F (x1 , x2 ) = f (y1 , y2 ) = f1 (y1 )f2 (y2 ), (x1 , x2 ) ∈ R2 .
(y1 ,y2 )∈S (y1 ,y2 )∈S
y1 ≤x1 ,y2 ≤x2 y1 ≤x1 ,y2 ≤x2

Let S1 and S2 be supports of X1 and X2 respectively. Then

S = {(y1 , y2 ) ∈ R2 : f (y1 , y2 ) > 0}

= {(y1 , y2 ) ∈ R2 : f1 (y1 )f2 (y2 ) > 0}
= {(y1 , y2 ) ∈ R2 : f1 (y1 ) > 0 and f2 (y2 ) > 0}
= {y1 ∈ R : f1 (y1 ) > 0} × {y2 ∈ R : f2 (y2 ) > 0} = S1 × S2 .

Therefore, for (x1 , x2 ) ∈ R2

  
XX  X  X 
F (x1 , x2 ) = f1 (y1 )f2 (y2 ) = 
 f1 (y1 )

 f2 (y2 )
 = F1 (x1 )F2 (x2 )
y1 ∈S1 ,y2 ∈S2 y1 ∈S1 y2 ∈S2
y1 ≤x1 ,y2 ≤x2 y1 ≤x1 y2 ≤x2
Lecture 0 Lecture Notes 0-83

where F1 and F2 are marginal d.f.s of X1 and X2 respectively. This imples X1 and X2 are independent. Conversely,
suppose that X1 and X2 are independent. Then F (y1 , y2 ) = F1 (y1 )F2 (y2 ) ∀ (y1 , y2 ) ∈ R2 . Then, for (x1 , x2 ) ∈ R2

f (x1 , x2 ) = Pr(X1 = x1 , X2 = x2 )
∞ !
\ 1 1
= Pr x1 − < X1 ≤ x1 , x2 − < X2 ≤ x2
n=1
n n

1 1
= lim Pr x1 − < X1 ≤ x1 , x2 − < X2 ≤ x2
n→∞ n n

1 1 1 1
= lim F (x1 , x2 ) − F x1 − , x2 − F x1 , x2 − + F x1 − , x2 −
n→∞ n n n n

1 1 1 1
= lim F1 (x1 )F2 (x2 ) − F1 x1 − F2 (x2 ) − F1 (x1 )F2 x2 − + F1 x1 − F2 x2 −
n→∞ n n n n
= F1 (x1 )F2 (x2 ) − F1 (x1 −)F2 (x2 ) − F1 (x1 )F2 (x2 −) + F1 (x1 −)F2 (x2 −)
= (F1 (x1 ) − F1 (x1 −))F2 (x2 ) − (F1 (x1 ) − F1 (x1 −))F2 (x2 −)
= (F1 (x1 ) − F1 (x1 −))(F2 (x2 ) − F2 (x2 −)) = f1 (x1 )f2 (x2 ).

This completes the proof for p = 2 case. Similary, it can be proved for other cases.
Remark 0.158. (a) If X = (X1 , X2 , . . . , Xp ) is a discrete r.v. with support S and Xi has support Si , i = 1, 2, . . . , p
then X1 , X2 , . . . , Xp are independent =⇒ S = S1 × S2 × · · · × Sp .
(b) Let X = (X1 , X2 , . . . , Xp ) be a discrete random vector with support S and p.m.f. f (·). Then X1 , . . . , Xp are
independent iff
f (x1 , x2 , . . . , xp ) = g1 (x1 )g2 (x2 ) · · · gp (xp ), x = (x1 , x2 , . . . , xp ) ∈ Rp
and S = A1 × A2 × · · · Ap for some functions A1 , . . . , Ap and Ai = {x ∈ R : gi (x) > 0},Pi = 1, 2, . . . , p. In
that case the marginal p.m.f. of Xi is fi (x) = ci gi (x), x ∈ R for some constant ci such that x∈Ai ci gi (x) = 1,
i = 1, 2, . . . , p.
(c) If X = (Y, Z) is a two-dimensional r.v. then Y and Z are independent iff f (y|z) = f1 (y) ∀ y ∈ R and z ∈ R such
that f2 (z) > 0, here f (y|z) denotes the conditional p.m.f. of Y given Z = z and f1 (·) denotes the marginal p.d.f. of
Y.
(d) One can extend Definition 0.151 to define indepedence of a collection of random vectors. Then analogous of
Theorem 0.153, Remark 0.155, Theorem 0.156, Theorem 0.157 and (c) above holds for random vectors.
Example 0.159. Let X = (X1 , X2 , X3 ) have the joint p.m.f.
(
cx1 x2 x3 , x1 = 1, 2, x2 = 1, 2, 3, x3 = 1, 3,
f (x1 , x2 , x3 ) =
0, otherwise.

where c is a real constant.

(a) Find the value of c.
(b) Find the marginal p.m.f.s of X1 , X2 and X3 .
(c) Are X1 , X2 and X3 independent.
(d) Find marginal p.m.f. of (X1 , X3 ).
(e) Find conditional p.m.f. of X1 given (X2 , X3 ) = (2, 1).
(f) Are X1 and X3 independent.
Lecture 0 Lecture Notes 0-84

(g) Compute Pr(X1 = X2 = X3 ).

Solution: Here the support of random vector X is SX = {1, 2} × {1, 2, 3} × {1, 3}.
1
P
(a) x∈SX f (x) = 1 =⇒ c(1 + 3 + 2 + 6 + 3 + 9 + 2 + 6 + 4 + 12 + 6 + 18) = 1 =⇒ c = 72 . Clearly f (x) ≥ 0
∀ x ∈ R3 .
(b) For x1 6= {1, 2}, clearly fX1 (x1 ) = 0. For x1 ∈ {1, 2}
3
! !
X x1 x2 x3 x1 X X x1
fX1 (x1 ) = = x2 x3 = .
72 72 x2 =1 x1 =1,3
3
(x2 ,x3 )∈{1,2,3}×{1,3}

Thus,
 x1 , x1 ∈ {1, 2},


fX1 (x1 ) = 3
0, otherwise.

Similarly,
 x2 , x2 ∈ 1, 2, 3,  x3 , x3 ∈ 1, 3,
 

fX2 (x2 ) = 6 ; fX3 (x3 ) = 4

0, otherwise. 0, otherwise.

(c) Clearly f (x1 , x2 , x3 ) = g1 (x1 )g2 (x2 )g3 (x3 ), (x1 , x2 , x3 ) ∈ R3 and SX = ∆1 × ∆2 × ∆3 where ∆1 = {1, 2},
∆2 = {1, 2, 3} and ∆3 = {1, 3}.
( ( (
c1 x1 , x1 ∈ ∆1 , c2 x2 , x2 ∈ ∆2 , c3 x3 , x3 ∈ ∆3 ,
g1 (x1 ) = ; g2 (x2 ) = ; g3 (x3 ) =
0, otherwise. 0, otherwise. 0, otherwise.

Obviously c1 = 31 , c2 = 1
6 and c3 = 41 . Thus X1 , X2 and X3 are independent.
Alternatively, using (b), we have f (x1 , x2 , x3 ) = fX1 (x1 )fX2 (x2 )fX3 (x3 ) ∀ (x1 , x2 , x3 ) ∈ R3 .
P x1 x3 x1 x3
(d) Marginal of (X1 , X3 ) is fX1 ,X3 (x1 , x3 ) = x2 fX (x1 , x2 , x3 ) = ×6= . Thus,
72 12
 x1 x3 , (x1 , x3 ) ∈ {1, 2} × {1, 3},


fX1 ,X3 (x1 , x3 ) = 12

0, otherwise

(e) For x1 ∈ {1, 2},

Pr(X1 = x1 , X2 = 2, X3 = 1) x1 2 1 x1
Pr(X1 = x1 |X2 = 2, X3 = 1) = = = .
Pr(X2 = 2, X3 = 1) 72 12 3
Thus
 x1 , x1 ∈ {1, 2},


fX1 |(X2 ,X3 ) (x1 |(2, 1)) = 3

0, otherwise.

Alternatively, since X1 , X2 and X3 are independent and X1 and (X2 , X3 ) are independent (why!), thus for fixed
(x2 , x3 ) ∈ R2 such that fX2 ,X3 (x2 , x3 ) > 0,

fX1 |(X2 ,X3 ) (x1 |(x2 , x3 )) = fX1 (x1 ) ∀ x1 ∈ R

 x1 , x1 ∈ {1, 2},


=⇒ fX1 |(X2 ,X3 ) (x1 |(2, 1)) = 3

0, otherwise.
Lecture 0 Lecture Notes 0-85

(f) By (c), X1 and X3 are independent.

X x1 x2 x3 1
(g) Pr(X1 = X2 = X3 ) = = P (X1 = X2 = X3 = 1) = .
72 72
x∈SX
x1 =x2 =x3

0.15.3. Continuous Random Vectors

Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional random vector with d.f. F .

Definition 0.160. The random vector X is called a continuous random vector if there exists a non-negative function
f : Rp → R such that for any rectangale set A in Rp
Z Z Z
P (X ∈ A) = . . . f (t)dt,
A

where t = (t1 , t2 . . . , tp ) and dt = dt1 dt2 . . . dtp . The function f (·) is called probability density function of X and
the set
S = {x ∈ Rp : Pr(xi − hi < xi ≤ xi + hi , i = 1, 2, . . . , p) > 0 ∀ hi > 0, i = 1, 2, . . . , p}
is called the support of F (or of X).
Remark 0.161. (a) In particular, if for fixed x = (x1 , x2 , . . . , xp ) ∈ Rp if A = (−∞, x1 ]×(−∞, x2 ]×· · ·×(−∞, xp ],
then Z x Z x Z x
1 2 p

F (x1 , x2 , . . . , xp ) = ... f (t1 , t2 , . . . , tp )dt1 dt2 . . . dtp .

−∞ −∞ −∞

(b) If X is continuous random vector then its d.f. F is a continuous function.

(c) For a continuous random vector if its p.d.f. f (x) is a piecewise continuous function then from the fundamental
theorem of multivariable calculus
∂p
f (x) = F (x), x ∈ Rp ,
∂x1 ∂x2 . . . ∂xp
whenever the derivative is defined.
* If f (·) is continuous at x ∈ Rp , then
Z x1 +h Z xp +hp
1 1
f (x) = lim ··· f (t)dx = lim Pr(xi < Xi ≤ xi + hi , i = 1, 2, . . . , p).
hi →0 h1 · · · hp x1 xp hi →0 h1 · · · hp
i=1,2,...,p i=1,2,...,p

For small dx1 , . . . , dxp if f is continuous at x, then

Z x1 +dx1 Z xp +dxp
Pr(xi < Xi ≤ xi + dxi , i = 1, 2, . . . , p) = ··· f (t1 , t2 , . . . , tp )dtp · · · dt1
x1 xp

≈ dx1 · · · dxp f (x1 , x2 , . . . , xp ).

Thus the probability that X is in a small neighborhood of x = (x1 , x2 , . . . , xp ) is proportional to f (x1 , x2 , . . . , xp ).

(d) There are random vectors that are neither discrete nor continuous.
RR R
(e) If X is a continuous random vector with p.d.f. f (·) then P (X = a) = . . . f (t)dt = 0.
x=a

(f) As in the univariate case the p.d.f. of a continuous random vector is not unique and it has different versions.
Lecture 0 Lecture Notes 0-86

(g) It can be shown that if X is a p-dimensional random vector with continuous d.f. F (·) such that
∂p
F (x1 , x2 , . . . , xp )
∂x1 ∂x2 . . . ∂xp
exists everywhere except (possibly) on a set C comprising of countable number of curves (having 0 volume in Rp ) and
∂p
Z
F (x1 , x2 , . . . , xp )dx1 dx2 . . . dxp = 1.
Rp \C ∂x1 ∂x2 . . . ∂xp

Then X is a continuous random vector with p.d.f.

∂p

 F (x1 , x2 , . . . , xp ), if x ∈ Rp \ C,
f (x) = ∂x1 ∂x2 . . . ∂xp

0, if x ∈ C.

(h) Let X = (X1 , X2 , . . . , Xp ) be a continuous random vector with joint p.d.f. fX (x) and d.f. FX (x). Then for
q ∈ {1, 2, . . . , p − 1} and x = (x1 , x2 , . . . , xq ) ∈ Rq ,

FX1 ,X2 ,...,Xq (x1 , x2 , . . . , xq )

= lim
xj →∞
FX1 ,X2 ,...,Xq ,Xq+1 ,...,Xp (x1 , x2 , . . . , xq , xq+1 , . . . , xp )
j=q+1,...,p
Z x1 Z xq Z xq+1 Z xp
= lim
x →∞
... ··· fX1 ,X2 ,...,Xq ,Xq+1 ,...,Xp (t1 , t2 , . . . , tq , tq+1 , . . . , tp )dtp · · · dt1
j
j=q+1,...,p −∞ −∞ −∞ −∞
Z x1 Z xq Z ∞ Z ∞
= ··· ··· fX1 ,X2 ,...,Xq ,Xq+1 ,...,Xp (t1 , t2 , . . . , tq , tq+1 , . . . , tp )dtp · · · dtq+1 tq · · · dt1
−∞ −∞ −∞ −∞

=⇒ (X1 , X2 , . . . , Xq ) is a continuous random vector with p.d.f.

Z ∞ Z ∞
fX1 ,X2 ,...,Xq (x1 , x2 , . . . , xq ) = ··· fX1 ,X2 ,...,Xq ,Xq+1 ,...,Xp (x1 , x2 , . . . , xq , tq+1 , . . . , tp )dtp · · · dtq+1 .
−∞ −∞

Thus, marginal distribution of a continuous random vector X are continuous with p.d.f. of marginal distribution
obtained by integrating out unwanted variables in the p.d.f. of X.

Conditional Distributions of Continuous Random Vector

For simplicity consider p = 2 and let X = (X1 , X2 ) be a random vector (discrete or continuous) with d.f. FX1 ,X2 (x1 ,
x2 ). Suppose that, for x1 ∈ SX1 (the support of distribution of X1 ) we want to define conditional d.f. of X2 given
X1 = x1 . If X1 is a continuous random variable then Pr(X1 = x1 ) = 0 ∀ x1 ∈ R and therefore Pr(X2 ≤ x|X1 = x1 )
is not defined for any x1 ∈ R; although it is defined for discrete random vector X1 when x1 ∈ SX1 . Thus we define
the conditional random vector X1 when x1 ∈ SX1 . Thus we define the conditional d.f. of X2 given X1 = x1 , through
the limiting argument

FX2 |X1 (x|x1 ) = lim Pr(X2 ≤ x|x1 − h < X1 ≤ x1 )

h↓0
Pr(X2 ≤ x, x1 − h < X1 ≤ x1 ) FX1 ,X2 (x1 , x) − FX1 ,X2 (x1 − h, x)
= lim = lim .
h↓0 Pr(x1 − h < X1 ≤ x1 ) h↓0 FX1 (x1 ) − FX1 (x1 − h)

Clearly if X = (X1 , X2 ) is discrete and x1 ∈ SX1 , then

FX1 ,X2 (x1 , x) − FX1 ,X2 (x1 − h, x) Pr(X1 = x1 , X2 ≤ x)

FX2 |X1 (x|x1 ) = lim = = Pr(X2 ≤ x|X1 = x1 ).
h↓0 FX1 (x1 ) − FX1 (x1 − h) Pr(X1 = x1 )
Lecture 0 Lecture Notes 0-87

Also, if X = (X1 , X2 ) is continuous random vector with p.d.f. f (x1 , x2 ) then

1 x
R R x1 Rx
h −∞ x1 −h fX1 ,X2 (y1 , y2 )dy1 dy2 fX1 ,X2 (x1 , y2 )dy2
FX2 |X1 (x|x1 ) = lim F (x )−F (x −h)
= −∞ .
h↓0 X1 1 X1 1 fX1 (x1 )
h

This implies conditional distribution of X2 given X1 = x1 (provided fX1 (x1 ) > 0) is continuous with p.d.f.

fX1 ,X2 (x1 , x)

fX2 |X1 (x|x1 ) = , x∈R
fX1 (x1 )

provided fX1 (x1 ) > 0.

The above discussion easily extends to general p ≥ 2 by defining conditional d.f. of X 2 = (Xq+1 , . . . , Xp ) given
X 1 = (X1 , X2 , . . . , Xq ) = (x1 , x2 , . . . , xq ) = x1 as

FX 2 |X 1 (x2 |x1 ) = lim Pr(Xj ≤ xj , j = q + 1, . . . , p|xi − hi < Xi ≤ xi , i = 1, 2, . . . , q),

h↓0

where x2 = (xq+1 , xq+2 , . . . , xp ) ∈ SX 2 .

Definition 0.162. Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional random vector with joint p.d.f. fX (·). Let q ∈
{1, 2, . . . , p − 1}, X 1 = (X1 , X2 , . . . , Xq ) and X 2 = (Xq+1 , . . . , Xp ). Then the conditional p.d.f. of X 2 given
X 1 = x1 is defined by

fX 1 ,X 2 (x1 , x2 ) fX (x1 , x2 )
fX 2 |X 1 (x2 |x1 ) = = , x2 ∈ Rp−q .
fX 1 (x1 ) fX 1 (x1 )

Theorem 0.163. Let X = (X1 , X2 , . . . , Xp ) be a continuous random vector with joint p.d.f. fX (·) and marginal
p.d.f.s fXi (·), i = 1, 2, . . . , p. Then X1 , X2 , . . . , Xp are independent iff
p
Y
fX1 ,X2 ,...,Xp (x1 , x2 , . . . , xp ) = fXi (xi ), x = (x1 , x2 , . . . , xp ) ∈ Rp .
i=1

Proof. Exercise.
Remark 0.164. (a) Let SX be the support of distribution X = (X1 , X2 , . . . , Xp ) and let SXi be the support of
Qp
distribution of Xi , i = 1, 2, . . . , p. It can be shown that if X1 , X2 , . . . , Xp are independent then SX = i=1 SXi
(cartesian product).
(b) Let X = (X1 , X2 ) be a continuous random vector. Then X1 and X2 are independent iff ∀ x1 ∈ SX1 ,

fX2 |X1 (x2 |x1 ) = fX2 (x2 ) ∀ x2 ∈ R.

Theorem 0.165. Let X = (X1 , X2 , . . . , Xp ) be a continuous random vector with joint p.d.f. fX (·) and marginal
p.d.f.s fXi (·), i = 1, 2, . . . , p. Then X1 , X2 , . . . , Xp are independent iff
p
Y
fX1 ,X2 ,...,Xp (x1 , . . . , xp ) = gi (xi ), x ∈ Rp ,
i=1

for some non-negative functions gi : R → R, i = 1, 2, . . . , p. In that case fXi (x) = ci gi (x), x ∈ R, for some positive
constants ci , i = 1, 2, . . . , p.
Lecture 0 Lecture Notes 0-88

Example 0.166. Let X = (X1 , X2 , X3 ) have the joint p.d.f.

 1 , 0 < x3 < x2 < x1 < 1,



fX (x1 , x2 , x3 ) = x1 x2
0, otherwise.


(a) Show that fX (·) is a proper p.d.f.

(b) Find the marginal p.d.f. of (X2 , X3 ).
(c) Find the marginal p.d.f. of X1 .
(d) Find the conditional p.d.f. of X1 given (X2 , X3 ) = (x2 , x3 ) where 0 < x3 < x2 < 1.
(e) Are X1 , X2 and X3 independent.
(f) Find the conditional p.d.f. of (X1 , X3 ) given X2 = x2 , where 0 < x2 < 1.
(g) Are X1 and X3 independent given X2 = x2 , where 0 < x2 < 1.

Solution: (a) Clearly, fX (x) ≥ 0 ∀ x ∈ R3 . Also,

Z Z 1 Z x1 Z x2
1
fX (x)dx = dx3 dx2 dx1 = 1.
R3 0 0 0 x1 x2

So, fX (x) is a p.d.f.

(b) The marginal p.d.f. of (X2 , X3 ) is obtained as,
Z ∞ Z 1
1 ln x2
fX2 ,X3 (x2 , x3 ) = fX (x)dx1 = dx1 = − , 0 < x3 < x2 < 1.
−∞ x2 x1 x2 x2

So, 
 ln x
− , 0 < y < x < 1,
fX2 ,X3 (x, y) = x
0, otherwise.

(c) For X1 ∈ R, the marginal of X1 is

Z ∞ Z ∞
fX1 (x1 ) = fX (x)dx2 dx3 .
−∞ −∞

Now 0 < x1 < 1, Z x1 Z x2

1
fX1 (x1 ) = dx3 dx2 = 1.
0 0 x1 x2
Thus, (
1, 0 < x < 1,
fX1 (x) =
0, otherwise.

(d) The conditional distribution of X1 given (X2 , X3 ) = (x2 , x3 ) is

fX (x1 , x2 , x3 ) (1/x1 x2 ) 1
fX1 |(X2 ,X3 ) (x1 |x2 , x3 ) = = =− , x2 < x1 < 1.
fX2 ,X3 (x2 , x3 ) (− ln(x2 )/x2 ) x1 ln x2
Lecture 0 Lecture Notes 0-89

So, the conditional distribution of X1 given (X2 , X3 ) = (x2 , x3 ) is

− 1 , x < x < 1,

2 1
fX1 |(X2 ,X3 ) (x1 |x2 , x3 ) = x1 ln x2
 0, otherwise.

(e) We have SX = {x ∈ R3 : 0 ≤ x3 ≤ x2 ≤ x1 ≤ 1} 6= SX1 × SX2 × SX3 = [0, 1] × [0, 1] × [0, 1]. So, X1 , X2
and X3 are not independent.
(f) For fixed x2 ∈ R, fX1 ,X3 |X2 (x1 , x3 |x2 ) ∝ fX1 ,X2 ,X3 (x1 , x2 , x3 ). For fixed 0 < x2 < 1,

 c(x2 ) , 0 < x < x , x < x < 1,
3 2 2 1
fX1 ,X3 |X2 (x1 , x3 |x2 ) = x
 1
0, otherwise.
Z ∞ Z ∞
1
fX1 ,X3 |X2 (x1 , x3 |x2 )dx1 dx3 = 1 =⇒ c(x2 ) = − .
−∞ −∞ x2 ln x2
Thus, for fixed 0 < x2 < 1, fX1 ,X3 |X2 (x1 , x3 |x2 ) = gx2 (x1 )hx2 (x3 ), (x1 , x3 ) ∈ R3 where for fixed x2 ∈ (0, 1)

1
 (
− , x2 < x < 1, 1, 0 < y < x2 ,
gx2 (x) = xx2 ln x2 ; hx2 (y) =
 0, otherwise. 0, otherwise.

=⇒ given X2 = x2 (0 < x2 < 1) X1 and X3 are independently distributed.

0.15.4. Expectation and Moments

Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional random vector with p.m.f. / p.d.f. f (·) and support S. Let g : Rp → R
be a function.

Definition 0.167. We say that the expected value of g(X) (denoted by E(g(X))) is finite and equals
X

 g(x)f (x), if X is discrete,

x∈S

E(g(X)) =

 Z ∞Z ∞ Z ∞


 ... g(x)f (x)dx, if X is continuous,
−∞ −∞ −∞

X Z ∞ Z ∞ Z ∞
provided |g(x)|f (x) < ∞ ... |g(x)|f (x)dx < ∞ .
x∈S −∞ −∞ −∞

X Z ∞
Theorem 0.168. Let Y = g(X). Then Y has finite expectation iff |y|fY (y) < ∞ or |y|fY (y)dy < ∞
y∈SY −∞
and in that case Z ∞
X
E(g(X)) = yfY (y) yfY (y)dy .
y∈SY −∞

Here, SY denotes the support of Y and fY (·) denotes the p.m.f. / p.d.f. Y .
Lecture 0 Lecture Notes 0-90

Some Special Expectations:

k
(a) For non-negative integers k1 , k2 , . . . , kp , µ0k1 ,k2 ,...,kp = E X1k1 X2k2 . . . Xp p , provided it is finite, is called a joint
moment of order k1 + k2 + · · · + kp of X.
(b) For non-negative integers k1 , k2 , . . . , kp ,

µk1 ,k2 ,...,kp = E (X1 − E(X1 ))k1 (X2 − E(X2 ))k2 . . . (Xp − E(Xp ))kp ,

provided it is finite, is called a joint central moment of order k1 + k2 + · · · + kp of X.

(c) The quantity Cov(X1 , X2 ) = E ((X1 − E(X1 ))(X2 − E(X2 ))), provided it is finite, is called covariance between
X1 and X2 .
Remark 0.169. (a)

Cov(X1 , X2 ) = E ((X1 − E(X1 ))(X2 − E(X2 )))

= E ((X1 − µ1 )(X2 − µ2 ))
= E(X1 X2 − X1 µ2 − µ2 X1 + µ1 µ2 ) = E(X1 X2 ) − E(X1 )E(X2 ).

(b) Cov(X1 , X1 ) = Var(X1 ).

(c) Cov(X1 , X2 ) = Cov(X2 , X1 )
Theorem 0.170. Let ai , i = 1, 2, . . . , p and bj , j = 1, 2, . . . , r are real constants and let Xi , i = 1, 2, . . . , p, Yj ,
j = 1, 2, . . . , r be random variables. Then
p p
!
X X
(a) E ai Xi = ai E(Xi ), provided the involved expectations are finite.
i=1 i=1
 
Xp r
X p X
X r
(b) Cov  ai Xi , bj Yj  = ai bj Cov(Xi , Yj ), provided the involved expectations are finite.
i=1 j=1 i=1 j=1

p p p X
p
!
X X X
(c) Var ai Xi = a2i Cov(Xi , Xi ) + ai aj Cov(Xi , Xj )
i=1 i=1 i=1 j=1
j6=i
Pp
a2i Var(Xi ) + 2
PP
= i=1 ai aj Cov(Xi , Xj ).
1≤i<j≤p

Proof. (a) (We will prove for continuous case).

p p p p
! Z !
X ∞ Z ∞ X X Z ∞ Z ∞ X
E ai Xi = ... ai xi fX (x)dx = ai ... xi fX (x)dx = ai E(Xi ).
i=1 −∞ −∞ i=1 i=1 −∞ −∞ i=1

(b) Note that

   !!   
p
X r
X p
X p
X r
X Xr
Cov  ai Xi , bj Yj  = E  ai Xi − E ai Xi  bj Yj − E  bj Yj 
i=1 j=1 i=1 i=1 j=1 j=1
 ! 
p
X p
X r
X r
X
=E ai Xi − ai E(Xi )  bj Yj − bj E(Yj )
i=1 i=1 j=1 j=1
Lecture 0 Lecture Notes 0-91

 ! 
p
X r
X
=E ai (Xi − E(Xi ))  bj (Yj − E(Yj ))
i=1 j=1
 
Xp X
r
= E  ai bj (Xi − E(Xi ))(Yj − E(Yj ))
i=1 j=1
p X
X r p X
X r
= ai bj E [(Xi − E(Xi ))(Yj − E(Yj ))] = ai bj Cov(Xi , Yj ).
i=1 j=1 i=1 j=1

(c) Note that

 
p p p
!
X X X
Var ai Xi = Cov  ai Xi , aj Xj 
i=1 i=1 j=1
p X
X p
= ai aj Cov(Xi , Xj )
i=1 j=1
Xp X p X p
= a2i Cov(Xi , Xi ) + ai aj Cov(Xi , Xj )
i=1 i=1 j=1
j6=i
Xp XXp p X p XX
= a2i Var(Xi ) + ai aj Cov(Xi , Xj ) = a2i Var(Xi ) + 2 ai aj Cov(Xi , Xj ).
i=1 i=1 j=1 i=1 1≤i<j≤p
j6=i

This completes the proof.

Theorem 0.171. Let X1 , X2 , . . . , Xp be independent random variables, let ψi : R → R, i = 1, 2, . . . , p be given
functions. Then,
p p
!
Y Y
(a) E ψi (Xi ) = E(ψi (Xi )), provided the involved expectations are finite,
i=1 i=1

(b) for any A1 , A2 , . . . , Ap ∈ Bp ,

p
Y
Pr(X1 ∈ A1 , X2 ∈ A2 , . . . , Xp ∈ Ap ) = Pr(Xi ∈ Ai ),
i=1

(c) ψ1 (X1 ), ψ2 (X2 ), . . . , ψp (Xp ) are independent random variables.

Proof. (We will prove for p = 2 in continuous case).

(a)
Z ∞ Z ∞
E(ψ1 (X1 )ψ2 (X2 )) = ψ1 (X1 )ψ2 (X2 )fX1 ,X2 (x1 , x2 )dx1 dx2
−∞ −∞
Z ∞ Z ∞
= ψ1 (X1 )ψ2 (X2 )fX1 (x1 )fX2 (x2 )dx1 dx2 (X1 ⊥
⊥X2 )
−∞ −∞
Z ∞ Z ∞
= ψ1 (X1 )fX1 (x1 )dx1 ψ2 (X2 )fX2 (x2 )dx2 = E(ψ1 (X1 ))E(ψ2 (X2 )).
−∞ −∞
Lecture 0 Lecture Notes 0-92

( (
1, if Xi ∈ Ai , 1, if Xi ∈ Ai ,
(b) Take ψi (Xi ) = in (a). Note that ψ1 (X1 )ψ2 (X2 ) =
0, otherwise, 0, otherwise.

E(ψi (Xi )) = Pr(Xi ∈ Ai ), i = 1, 2 and E(ψ1 (X1 )ψ2 (X2 )) = Pr(X1 ∈ A1 , X2 ∈ A2 ). Now the result follows
from (a).
(c) Let Yi = ψi (Xi ), i = 1, 2. For fixed y = (y1 , y2 ) ∈ R2 , define
(
1, if Yi = ψi (Xi ) ≤ yi , i = 1, 2,
gi (Xi ) =
0, otherwise.

Then by (a) E(g1 (X1 )g2 (X2 )) = E(g1 (X1 ))E(g2 (X2 )). Also,
(
1, if ψ1 (X1 ) ≤ y1 , ψ2 (X2 ) ≤ y2 ,
g1 (X1 )g2 (X2 ) =
0, otherwise,
(
1, if Y1 ≤ y1 , Y2 ≤ y2 ,
=
0, otherwise.

So, E(g1 (X1 )g2 (X2 )) = Pr(Y1 ≤ y1 , Y2 ≤ y2 ) and E(gi (Xi )) = Pr(Yi ≤ yi ), i = 1, 2. Consequently, Pr(Y1 ≤
y1 , Y2 ≤ y2 ) = Pr(Y1 ≤ y1 )Pr(Y2 ≤ y2 ) ∀ (y1 , y2 ) ∈ R2 =⇒ Y1 = ψ1 (X1 ) and Y2 = ψ2 (X2 ) are independent
random variables.

Corollary 0.172. Let X1 , X2 , . . . , Xp are independent random variables. Then

(a) Cov(Xi , Xj ) = 0 ∀ i 6= j.
(b) For real constants a1 , a2 , . . . , ap , we have
p p
!
X X
Var ai Xi = a2 Var(Xi ).
i=1 i=1

Proof. (a) For i 6= j, Cov(Xi , Xj ) = E(Xi Xj ) − E(Xi )E(Xj ) = E(Xi )E(Xj ) − E(Xi )E(Xj ) = 0.
Pp Pp Pp Pp Pp
(b) Var ( i=1 ai Xi ) = i=1 a2 Var(Xi ) + i=1 j=1 ai aj Cov (Xi , Xj ) = i=1 a2 Var(Xi ), (using (a)).
j6=i

Definition 0.173. (a) The correlation between random variables X1 and X2 is defined by

Cov(X1 , X2 )
ρ(X1 , X2 ) = p ,
Var(X1 ) Var(X2 )

provided 0 < Var(Xi ) < ∞, i = 1, 2.

(b) Random variables X1 and X2 are said to be uncorrelated if ρ(X1 , X2 ) = 0 (or equivalently Cov(X1 , X2 ) = 0).

Remark 0.174. If X1 and X2 are independent random variables =⇒ X1 and X2 are uncorrelated. converse may
not be true.
Example 0.175 (Uncorrelated random variables may not be independent). Let (X, Y ) have joint p.m.f.
1
 2 , if (x, y) = (0, 0),

f (x, y) = 14 , if (x, y) = (1, −1), (1, 1),

0, otherwise.

Lecture 0 Lecture Notes 0-93

Then 1
 4 , if y = −1, 1,
(
1 
2, if x = 0, 1,
fX (x) = fY (y) = 12 , if y = 0,
0, otherwise, 
0, otherwise.


Clearly, there exists (x, y) ∈ R2 such that fX,Y (x, y) 6= fX (x)fY (y) =⇒ X and Y are not independent (in fact
Pr(X = Y 2 ) = 1).
1
However, E(XY ) = E(Y ) = 0 and E(X) = 2 =⇒ Cov(X, Y ) = 0 =⇒ ρ(X, Y ) = 0.

Theorem 0.176 (Cauchy-Schwarz Inequality). For random variables X and Y

(E(XY ))2 ≤ E(X 2 )E(Y 2 ) (0.16)

provided involved expectations are finite. The equality is attained iff Pr(Y = cX) = 1 or Pr(X = cY ) = 1, for some
real constant c.

Proof. Case I: E(X 2 ) = 0. In this case Pr(X = 0) = 1. Therefore Pr(XY = 0) = 1 and E(XY ) = 0. We have
inequality in (0.16).
Case II: E(X 2 ) > 0. Then

E((Y − cX)2 ) ≥ 0 ∀ c ∈ R
=⇒ c2 E(X 2 ) − 2cE(XY ) + E(Y 2 ) ≥ 0 ∀ c ∈ R
=⇒ Discriminant ≤ 0 =⇒ (2E(XY ))2 − 4(E(X 2 ))E(Y 2 ) ≤ 0 =⇒ (E(XY ))2 ≤ E(X 2 )E(Y 2 ).

Clearly, equality is attained iff E((Y − cX)2 ) = 0 for some c ∈ R =⇒ Pr(Y = cX) = 1 for some c ∈ R. By
symmetry Pr(X = cY ) = 1 for some c ∈ R.
Corollary 0.177. Let X1 and X2 be random variables with E(Xi ) = µi ∈ (−∞, ∞) and Var(Xi ) = σi2 ∈
(0, ∞), i = 1, 2. Then
(a) |ρ(X1 , X2 )| ≤ 1.

X1 − µ1 X2 − µ2 X2 − µ2 X1 − µ1
(b) |ρ(X1 , X2 )| = 1 iff Pr =c = 1 or Pr =c = 1, for some real
σ1 σ2 σ2 σ1
constant c.

X1 − µ1 X2 − µ2
Proof. Let X = and Y = . Using Cauchy-Schwarz inequality (E(XY ))2 ≤ E(X 2 )E(Y 2 ) but
σ1 σ2

E(X1 − µ1 )2 E(X2 − µ2 )2
E(X 2 ) = = 1 and E(Y 2
) = = 1.
σ12 σ22

Thus 2
E((X1 − µ1 )(X2 − µ2 ))
≤ 1 =⇒ ρ2 (X1 , X2 ) ≤ 1 =⇒ |ρ(X1 , X2 )| ≤ 1
σ1 σ2

X1 − µ1 X2 − µ2
and equality is attained iff Pr(X = cY ) = 1, for some real constants c =⇒ Pr =c = 1 for
σ1 σ2
some real constants c.
Lecture 0 Lecture Notes 0-94

0.15.5. Conditional Expectation, Conditional Variance and Conditional Covariance

Definition 0.178. (a) Let X be a p-dimensional random vector and Y be a q-dimensional random vector. Let y ∈ Rq
be such that fY (y) > 0 and let ψ : Rp → R be a given function. Here fY (·) is the p.d.f. / p.m.f. of random vector Y .
Then
(i) The conditional expectation of ψ(X) given Y = y (denoted by E(ψ(X)|Y = y)) is the expectation of ψ(X) under
the conditional distribution of X given Y = y.
(ii) The conditional variance of ψ(X) given Y = y (denoted by Var(ψ(X)|Y = y)) is the variance of ψ(X) under
the conditional distribution of X given Y = y.
(b) Let X1 and X2 be two random variables and Y be a q-dimensional random vector. Then the conditional covariance
between X1 and X2 given Y = y, (denoted by Cov(X1 , X2 |Y = y)) is the covariance between X1 and X2 under the
conditional distribution of (X1 , X2 ) given Y = y.

Notation Let for y ∈ {t ∈ Rq : fY (t) > 0}, ψ1 (y) = E(ψ(X)|Y = y) and ψ2 (y) = Var(ψ(X)|Y = y) and ψ3 (y) =
Cov(X1 , X2 |Y = y). We denote ψ1 (Y ) = E(ψ(X)|Y ) and ψ2 (Y ) = Var(ψ(X)|Y ) and ψ3 (Y ) = Cov(X1 , X2 |Y ).
Theorem 0.179. Under the above notation
(a) E(ψ(X)) = E(E(ψ(X)|Y )),
(b) Var(ψ(X)) = Var(E(ψ(X)|Y )) + E(Var(ψ(X)|Y )),
(c) Cov(X1 , X2 ) = Cov(E(X1 |Y ), E(X2 |Y )) + E(Cov(X1 , X2 |Y )).

Proof. (We will prove for p = q = 1 continuous case).

(b) Follows from (c).

Cov(X1 , X2 ) = E((X1 − E(X1 ))(X2 − E(X2 ))) = E [E [(X1 − E(X1 ))(X2 − E(X2 ))|Y ]] .

Now,

E [(X1 − E(X1 ))(X2 − E(X2 ))|Y ]

This completes the proof.

Lecture 0 Lecture Notes 0-95

0.15.6. Joint Moment Generating Function

Let X =
(X1 , X2 , .. . , Xp ) be a p-dimensional random vector with p.d.f. /p.m.f. fY (·). A = {t = (t1 , t2 , . . . , tp ) ∈
Pp
R : E e i=1 ti Xi < ∞}.
p

Definition 0.180. (a) The function MX : A → R defined by

Pp
MX (t) = E e i=1 ti Xi , t = (t1 , t2 , . . . , tp ) ∈ A

is called the joint moment generating function (m.g.f.) of random vector X = (X1 , X2 , . . . , Xp ).

Notation: For a = (a1 , a2 , . . . , ap ) ∈ Rp , −a = (−a1 , −a2 , . . . , −ap ) and (−a, a) = (−a1 , a1 ) × · · · × (−ap , ap ),
a = (a1 , a2 , . . . , ap ) > 0 ⇐⇒ ai > 0, i = 1, 2, . . . , p.
Remark 0.181. (i) As MX (0) = 1, we have A 6= φ. Moreover MX (t) > 0 ∀ t ∈ A.
(ii) If X1 , X2 , . . . , Xp are independent then
p p p
!
Pp Y Y Y
i=1 ti Xi ti Xi ti Xi

MX (t) = E e =E e = E e = MXi (ti ) ∀ t ∈ A.
i=1 i=1 i=1
Qp
Conversely, suppose that A ⊆ (−a, a) for some a > 0 and MX (t) = i=1 MXi (ti ) ∀ t ∈ A, then it can be shown
that X1 , X2 , . . . , Xp are independent.
p
X
(iii) Let X1 , X2 , . . . , Xp be independent random variables and let Y = Xi , then
i=1

p p p
!
Pp Y Y Y
t Xi tXi
MY (t) = E e i=1 =E e = E etXi = MXi (t), t ∈ A.
i=1 i=1 i=1

In particular, if X1 , X2 , . . . , Xp are independent and identically distributed (iid) with common m.g.f. M (t), then
p
MY (t) = (M (t)) , t ∈ A.
Theorem 0.182. Suppose that the joint m.g.f. MX (t) is finite on a rectangle (−a, a) ∈ Rp , a > 0. Then MX (t)
posseses partial derivatives of all order in (−a, a). Furthermore, for non-negative integers k1 , k2 , . . . , kp
" #

k1 k2 kp
∂ k1 +k2 +···+kp
E X1 X2 . . . Xp = k
MX (t) .
∂tk11 ∂tk22 . . . ∂tpp t=0

Proof. (We give an outline of the proof).

Pp Z Pp
MX (t1 , t2 , . . . , tp ) = E e i=1 ti Xi = e i=1 ti xi fX (x)dx,
Rp
k1 +···+kp Z
∂ P p

k1 kp
MX (t) = xk11 · · · xkpp e i=1 ti Xi fX (x)dx,
∂t1 · · · ∂tp R p
" #
k1 +···+kp Z
∂ k1 kp

k1 k2 kp

k
M X (t) = x 1 · · · x p fX (x)dx = E X 1 X 2 . . . Xp .
∂tk11 · · · ∂tpp t=0
Rp

This completes the proof.

Lecture 0 Lecture Notes 0-96

Let ψX (t) = ln MX (t), t ∈ (−a, a). Then

∂ ∂
E(Xi ) = MX (t) = ψX (t) ,
∂ti t=0 ∂ti t=0
m
m ∂
E(Xi ) = MX (t) , m = 1, 2, . . . , i = 1, 2, . . . , p,
∂tmi t=0
2 !2 2
∂ ∂ ∂
Var(Xi ) = MX (t) − MX (t) = ψX (t) , i = 1, 2, . . . , p,
∂t2i t=0 ∂ti t=0 ∂t2i t=0

provided MX (t) is finite on (−a, a), for some a > 0. For i 6= j, if MX (t) is finite on (−a, a), for some a > 0,
Cov(Xi , Xj ) = E(Xi Xj ) − E(Xi )E(Xj ) = E((Xi − E(Xi ))(Xj − E(Xj )))
∂2 ∂2

∂ ∂
= MX (t) − MX (t) MX (t) = ψX (t) .
∂ti ∂tj t=0 ∂ti t=0 ∂tj t=0 ∂ti ∂tj t=0

Moreover,
MX (0, . . . , 0, ti , 0, . . . , 0) = E(eti Xi ) = MXi (ti ), i = 1, 2, . . . , p,
MX (0, . . . , 0, ti , 0, . . . , 0, tj , 0, . . . , 0) = E(eti Xi +tj Xj ) = MXi ,Xj (ti , tj ),
provided the m.g.f. is finite.

0.15.7. Equality in Distribution

Definition 0.183. Two p-dimensional random vectors X and Y are said to have the same distribution (written as
d
X = Y ) if FX (x) = FY (x) ∀ x ∈ Rp .
Theorem 0.184. (a) Let X and Y be discrete random vectors with p.m.f.s fX (·) and fY (·), respectively. Then
d
X = Y ⇐⇒ fX (x) = fY (x) ∀ x ∈ Rp .
(b) Let X and Y be continuous random vectors. Then
d
X = Y ⇐⇒ fX (x) = fY (x) ∀ x ∈ Rp ,
for some versions fX (·) and fY (·) of p.d.f.s of X and Y , respectively.
(c) Let X and Y be p-dimensional random vectors and let ψ : Rp → Rq be a given function. Then
d d
X = Y ⇐⇒ ψ(X) = ψ(Y )

(d) Let X and Y be p-dimensional random vectors with finite m.g.f.s MX (t) and MY (t) on a rectangle (−a, a), for
some a > 0. Then
d
MX (t) = MY (t) ∀ (−a, a) =⇒ X = Y .

0.15.8. Some Generalizations

Let X i : a pi - dimensional Pprandom vector, i = 1, 2, . . . , m. FX i : d.f. of X i , i = 1, 2, . . . , m, fX i : p.m.f. / p.d.f.

of X i , i = 1, 2, . . . , m, i=1 pi = p, X = (X 1 , X 2 , . . . , X m ): p-dimensional random vector with d.f. FX (·) and
p.m.f. / p.d.f. fX (·).
Lecture 0 Lecture Notes 0-97

Definition 0.185. The random vectors X1 , X2 , . . . , Xm are said to be independent if for any subcollection {Xi1 , Xi2 ,
. . . , Xiq } of {X1 , X2 , . . . , Xm } (2 ≤ q ≤ m)

q
Y Pq
pij
FXi1 ,Xi2 ,...,Xiq (x1 , x2 , . . . , xq ) = FXij (xj ) ∀ x = (x1 , x2 , . . . , xq ) ∈ R j=1 .
j=1

Remark 0.186. X1 , X2 , . . . , Xm are independent =⇒ random variables in any subset of {X1 , X2 , . . . , Xm } are
independent.
Theorem 0.187. (a) The following statements are equivalent:
(i) X1 , X2 , . . . , Xm are independent random vectors.
Qm
(ii) FX1 ,X2 ,...,Xm (x1 , x2 , . . . , xm ) = i=1 FXi (xi ) ∀ x = (x1 , x2 , . . . , xm ) ∈ Rp .
Qm
(iii) fX1 ,X2 ,...,Xm (x1 , x2 , . . . , xm ) = i=1 fXi (xi ) ∀ x = (x1 , x2 , . . . , xm ) ∈ Rp .
Qm
(iv) fX1 ,X2 ,...,Xm (x1 , x2 , . . . , xm ) = i=1 gi (xi ) ∀ x = (x1 , x2 , . . . , xm ) ∈ Rp for some non-negative real valued
function gi : Rp → R, i = 1, 2, . . . , m.
Qm
(v) Pr(Xi ∈ Ai , i =, 2, . . . , m) = i=1 Pr(Xi ∈ Ai ) ∀ Ai ∈ Bpi , i = 1, 2, . . . , m.
(b) If X1 , X2 , . . . , Xm are independent random vectors, then
Qm Qm
(i) E i=1 ψi (Xi ) = i=1 E ψi (Xi ) for any functions ψi , i = 1, 2, . . . , m.

(ii) ψ1 (X1 ), ψ2 (X2 ), . . . , ψm (Xm ) are independent random vectors for any functions ψ1 , ψ2 , . . . , ψm .
Definition 0.188. Let ∆ be an arbitrary index set. The random vectors {X λ : λ ∈ ∆} are said to be independent if
random variables in any finite subcollection of {X λ : λ ∈ ∆} are independent.

Theorem 0.189. Under the notation of Theorem 0.187, X1 , X2 , . . . , Xm are independent random vectors ⇐⇒ for
some a > 0 and ∀ t = (t1 , t2 , . . . , tm ) ∈ (−a, a),
m
Y
MX (t1 , t2 , . . . , tm ) = MXi (ti ).
i=1

0.15.9. Functions of Random Vector

Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional random vector with p.m.f. / p.d.f. f (·). Let g : Rp → Rq , where
1 ≤ q ≤ p be a function defined on Rp and taking values in Rq . Sometimes it may be of interest to derive the
probability distribution of Y = g(X).

Definition 0.190. (a) Let X 1 , X 2 , . . . , X n be a collection of iid random vectors each having the (joint) d.f. F and the
same p.m.f. / p.d.f. f (·). We call X 1 , X 2 , . . . , X n a random sample (r.s.) of size n from a distribution having d.f. F
(p.m.f. / p.d.f. f (·)). In other words a random sample is a collection of iid random vectors.
(b) A function of one or more random vectors that does not depend on any unknown parameter is called a statistic.
Example 0.191. Let X1 , X2 , . . . , Xn be a random sample from a distribution having p.d.f.
(
1 −x/θ
e , x > 0,
fθ (x) = θ
0, otherwise,
Lecture 0 Lecture Notes 0-98

1
Pn
where θ ∈ H = (0, ∞) is unknown. Then X̄ = n i=1 Xi is a statistic (called sample mean) but X1 − θ is not a
statistic. Some other statistic are:
n n
1X 1 X
Sn2 = (Xi − X̄)2 or Sn−1
2
= (Xi − X̄)2 −→ Sample Variance,
n i=1 n − 1 i=1

Xr:n = r-th smallest of X1 , X2 , . . . , Xn , r = 1, 2, . . . , n so that

X1:n ≤ X2:n ≤ · · · ≤ Xn:n −→ r-th order statistic, r = 1, . . . , n,

X[np]:n , 0 < p < 1; [x] = largest integer ≤ x −→ p-th sample quantile,

X[n/4]:n −→ sample lower quantile, X[3n/4]:n −→ sample upper quantile,


X n+1 :n , if n is odd,
2
M= −→ sample median,
 X n2 :n +X n2 +1:n , if n is even,
2

p q
Sn = Sn2 or Sn−1 = 2
Sn−1 −→ sample standard deviation,

1
Pn
n i=1 (Xi − X̄)(Yi − Ȳ )
r= q
1
Pn 1 Pn −→ sample correlation coefficient.
2 − Ȳ )2
n i=1 (Xi − X̄) n i=1 (Yi

Let X1 , X2 , . . . , Xn be a random sample from a distribution having d.f. F and p.m.f. / p.d.f. f (·). Then the joint d.f.
of X = (X1 , X2 , . . . , Xn ) is
n
Y
FX (x) = F (xi ), x = (x1 , x2 , . . . , xn ) ∈ Rn
i=1

and the joint p.m.f. / p.d.f. of X is

n
Y
fX (x) = f (xi ), x = (x1 , x2 , . . . , xn ) ∈ Rn .
i=1

Theorem 0.192. If X1 , X2 , . . . , Xn is a random sample, then

d
(X1 , X2 , . . . , Xn ) = (Xβ1 , Xβ2 , . . . , Xβn )

for any permutation (β1 , β2 , . . . , βn ) of (1, 2, . . . , n).

Example 0.193. Let X1 , X2 , . . . , Xn be a random sample from a given distribution.
1
(a) If X1 is a continuous r.v. then Pr(X1 < X2 < · · · < Xn ) = Pr(Xβ1 < Xβ2 < · · · < Xβn ) = n! , for any
permutation (β1 , β2 , . . . , βn ) of (1, 2, . . . , n).
1
(b) If X1 is a continuous r.v. then for any r ∈ {1, 2, . . . , n}, Pr(Xi = Xr:n ) = n, i = 1, 2, . . . , n.

(c) E X1 +X2X+···+X
i
n
= n1 , i = 1, 2, . . . , n.
Lecture 0 Lecture Notes 0-99

Pn
t
(d) E Xi | j=1 Xj = t = n, i = 1, 2, . . . , n.

Solution (a)

X1 is a continuous r.v. =⇒ X = (X1 , X2 , . . . , Xn ) is a continuous random vector. (Why?)

d
=⇒ (X1 , X2 , . . . , Xn ) = (Xβ1 , Xβ2 , . . . , Xβn ) for any permutation (β1 , β2 , . . . , βn ) of (1, 2, . . . , n) and
Pr(all Xi ’s are distinct) = 1
d
=⇒ (X1 , X2 , . . . , Xn ) = (Xβ1 , Xβ2 , . . . , Xβn ) for any permutation (β1 , β2 , . . . , βn ) of (1, 2, . . . , n) and
X
Pr(Xβ1 < Xβ2 < · · · < Xβn ) = 1, where Sn is the set of all permutation of (1, 2, . . . , n)
β∈Sn
1
=⇒ Pr(Xβ1 < Xβ2 < · · · < Xβn ) = Pr(X1 < X2 < · · · < Xn ) = .
n!
(b)
d
For any i = 1, 2, . . . , n, (X1 , X2 , . . . , Xi , . . . , Xn ) = (Xi , X2 , . . . , X1 , . . . , Xn )
=⇒ Xr:n , r-th smallest of (X1 , X2 , . . . , Xi , . . . , Xn ) = r-th smallest of (Xi , X2 , . . . , X1 , . . . , Xn ) and
Pr(X1 = r-th smallest of (X1 , X2 , . . . , Xi , . . . , Xn )) = Pr(Xi = r-th smallest of (Xi , X2 , . . . , X1 , . . . , Xn ))
=⇒ Pr(X1 = Xr:n ) = Pr(Xi = Xr:n ), i = 1, 2, . . . , n
n
X
since Pr(X1:n < X2:n < · · · < Xn:n ) = 1, (by (a)), we have Pr(Xi = Xr:n ) = 1
i=1
1
=⇒ Pr(X1 = Xr:n ) = Pr(Xi = Xr:n ) = .
n
(c)
d
(X1 , X2 , . . . , Xi , . . . , Xn ) = (Xi , X2 , . . . , X1 , . . . , Xn )

X1 Xi
=⇒ E =E
X1 + X2 + · · · + Xi + · · · + Xn Xi + X2 + · · · + X1 + · · · + Xn
! ! n
! Pn !
X1 Xi X Xi i=1 Xi
=⇒ E Pn = E Pn but E Pn = E Pn =1
j=1 Xj j=1 Xj i=1 j=1 Xj j=1 Xj
! !
Xi X1 1
=⇒ E Pn = E Pn = , i = 1, 2, . . . , n.
j=1 j X X
j=1 j n

(d)
d
(X1 , X2 , . . . , Xi , . . . , Xn ) = (Xi , X2 , . . . , X1 , . . . , Xn )
=⇒ E (X1 |X1 + X2 + · · · + Xi + · · · + Xn = t) = E (Xi |Xi + X2 + · · · + X1 + · · · + Xn = t)
       
Xn Xn Xn Xn n
X n
X
=⇒ E X1 Xj = t = E Xi Xj = t but E Xi Xj = t = E  Xi Xj = t = t.
j=1 j=1 i=1 j=1 i=1 j=1

Therefore    
n n
X X t
E Xi Xj = t = E X1 Xj  = , i = 1, 2, . . . , n.
j=1 j=1
n
Lecture 0 Lecture Notes 0-100

0.15.10. Distribution Function Technique

Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional random vector with d.f. F and p.m.f. / p.d.f. f (·). Also, let
g : Rp → Rq : g = (g1 , g2 , . . . , gq ), Y = (Y1 , Y2 , . . . , Yq ) = (g1 (X), g2 (X), . . . , gq (X)). We are interested in the
distribution of random vector Y .
One can first find the d.f. of Y = (Y1 , Y2 , . . . , Yq )

FY (y1 , y2 , . . . , yq ) = Pr(g1 (X) ≤ y1 , g2 (X) ≤ y2 , . . . , gq (X) ≤ yq ), y = (y1 , y2 , . . . , yq ) ∈ Rq ,

and then find the p.m.f. / p.d.f. of Y = (Y1 , Y2 , . . . , Yq ).

Example 0.194. Let X1 , X2 , . . . , Xn be a random sample from a distribution having d.f. F , p.m.f. / p.d.f. f and
support S. Let Y1 = min{X1 , X2 , . . . , Xn } and Y2 = max{X1 , X2 , . . . , Xn }.
(a) Find the joint d.f. of Y = (Y1 , Y2 ).
(b) Find the marginal d.f.s of Y1 and Y2 using findings of (a).
(c) Find the marginal d.f.s of Y1 and Y2 directly (that is, without using (a)).
(d) Find the marginal p.m.f. / p.d.f. Y = (Y1 , Y2 ) using findings in (b).

Solution: (a) For (y1 , y2 ) ∈ R2 ,

FY (y1 , y2 ) = Pr(Y1 ≤ y1 , Y2 ≤ y2 )
= Pr(min{X1 , X2 , . . . , Xn } ≤ y1 , max{X1 , X2 , . . . , Xn } ≤ y2 )
= Pr(max{X1 , X2 , . . . , Xn } ≤ y2 ) − Pr(min{X1 , X2 , . . . , Xn } > y1 , max{X1 , X2 , . . . , Xn } ≤ y2 )
= Pr(Xi ≤ y2 , i = 1, 2, . . . , n) − Pr(Xi > y1 , i = 1, 2, . . . , n, Xi ≤ y2 , i = 1, 2, . . . , n)
Yn
= Pr(Xi ≤ y2 ) − Pr(y1 < Xi ≤ y2 , i = 1, 2, . . . , n)
i=1
n n
( n
Y Y [F (y2 )]n − [F (y2 ) − F (y1 )] , −∞ < y1 < y2 < ∞,
= Pr(Xi ≤ y2 ) − Pr(y1 < Xi ≤ y2 ) =
i=1 i=1
[F (y2 )]n , −∞ < y2 < y1 < ∞.

(b)
n
FY1 (y1 ) = lim FY (y1 , y2 ) = 1 − [1 − F (y1 )] , −∞ < y1 < ∞,
y2 →∞
n
FY2 (y2 ) = lim FY (y1 , y2 ) = [F (y2 )] , −∞ < y2 < ∞.
y1 →∞

(c)

FY1 (y1 ) = Pr(Y1 ≤ y1 )

= Pr(min{X1 , X2 , . . . , Xn } ≤ y1 )
= 1 − Pr(min{X1 , X2 , . . . , Xn } > y1 )
n
Y n
= 1 − Pr(Xi > y1 , i = 1, 2, . . . , n) = 1 − Pr(Xi > y1 ) = 1 − [1 − F (y1 )] , −∞ < y1 < ∞.
i=1

FY2 (y2 ) = Pr(Y2 ≤ y2 )

= Pr(max{X1 , X2 , . . . , Xn } ≤ y2 )
Lecture 0 Lecture Notes 0-101

n
Y n
= Pr(Xi ≤ y2 , i = 1, 2, . . . , n) = Pr(Xi ≤ y2 ) = [F (y2 )] , −∞ < y2 < ∞.
i=1

(d) Case I: X1 is a discrete r.v. Then SX1 = SY1 = SY2 . For y1 ∈ SX1
n n
fY1 (y1 ) = Pr(Y1 = y1 ) = FY1 (y1 ) − FY1 (y1 −) = [1 − F (y1 −)] − [1 − F (y1 )] .

Thus, ( n n
[1 − F (y1 −)] − [1 − F (y1 )] , if y1 ∈ SX1 ,
fY1 (y1 ) =
0, otherwise.
Similarly, ( n n
[F (y2 )] − [F (y2 −)] , if y2 ∈ SX1
fY2 (y2 ) = FY2 (y2 ) − FY2 (y2 −) =
0, otherwise
Case II: X1 is a continuous r.v.
Let F (·) be differentiable everywhere (except possibly on a set having length zero (that is, it does not contain any open
interval)
d n n−1
fY1 (y) = (1 − [1 − F (y)] ) = n [1 − F (y)] f (y), −∞ < y < ∞,
dy
d n n−1
fY2 (y) = [F (y)] = n [F (y)] f (y), −∞ < y < ∞.
dy
Example 0.195. Let X1 and X2 be iid r.v.s with common p.d.f.
(
2x, 0 < x < 1,
f (x) =
0, otherwise.

Find the d.f. of Y = X1 + X2 . Hence find the p.d.f. of Y .

Solution: The joint p.d.f. of X = (X1 , X2 ) is

(
4x1 x2 , 0 < x1 < 1, 0 < x2 < 1,
fX (x1 , x2 ) = fX1 (x1 )fX2 (x2 ) = f (x1 )f (x2 ) =
0, otherwise.

For y ∈ R, Z 1 Z 1
FY (y) = Pr(Y ≤ y) = Pr(X1 + X2 ≤ y) = 4x1 x2 dx1 dx2 .
0 0
x1 +x2 ≤y

Clearly for y < 0, FY (y) = 0 and for y ≥ 2, FY (y) = 1. Now consider y ∈ [0, 1),
y y−x1
y4
Z Z
FY (y) = 4x1 x2 dx2 dx1 = .
0 0 6

For y ∈ [1, 2),

y−1 1 1 y−x1
(4y − 3) − (y + 3)(y − 1)3
Z Z Z Z
FY (y) = 4x1 x2 dx2 dx1 + 4x1 x2 dx2 dx1 = (y − 1)2 + .
0 0 y−1 0 6
Lecture 0 Lecture Notes 0-102

Thus,
0, y < 0,



 y4 , 0 ≤ y < 1,


FY (y) = 6 3



 (y − 1)2 + (4y−3)−(y+3)(y−1)
6 , 1 ≤ y < 2,

1, y ≥ 2.
Clearly, Y is continuous r.v. with p.d.f.
2 3
y , 0 < y < 1,
3

2
fY (y) = 2(y − 1) + 3[1−(y+2)(y−1)2 ] , 1 < y < 2,


0, otherwise.

0.15.11. Transformation of Variable Technique

Let X = (X1 , X2 , . . . , Xp ) be a p-dimensional discrete random vector with support S and p.m.f. f (·). Let gi : Rp →
R, i = 1, 2, . . . , k and Yi = gi (X), i = 1, 2, . . . , k where 1 ≤ k ≤ p is an integer. Then Y = (Y1 , Y2 , . . . , Yk ) is
discrete random vector with support

T = {(y1 , y2 , . . . , yk ) : yi = gi (x1 , x2 , . . . , xp ), i = 1, 2, . . . , k for some x = (x1 , x2 , . . . , xp ) ∈ S},

d.f. G(y) = G(y1 , y2 , . . . , yk ) = x∈Ay f (x), y ∈ Rk and p.m.f.

(P
x∈By f (x), if y ∈ T,
g(y) =
0, otherwise,

where Ay = {x : (x1 , x2 , . . . , xp ) ∈ S : gi (x) ≤ yi , i = 1, 2, . . . , k} and By = {x : (x1 , x2 , . . . , xp ) ∈ S : gi (x) =

yi , i = 1, 2, . . . , k}.
Example 0.196. Let X1 , X2 , . . . , Xp be independent r.v.s with Xi having the p.m.f. (Binomial distribution), that is,
(
ni x ni −x
x θ (1 − θ) , x ∈ {0, 1, . . . , ni },
fi (x) =
0, otherwise,

i = 1, 2, . . . , k, where θ ∈ (0, 1) and ni ∈ {1, 2, . . . }, i = 1, 2, . . . , k are fixed real constants. Let Y = X1 + X2 +

· · · + Xp . Find the p.m.f. of Y .

Solutions: The joint p.m.f. of X = (X1 , X2 , . . . , Xp ) is

 Pp x Pp
Yp  Qp ni Qp
θ i=1 i (1 − θ)n− i=1 xi , x ∈ i=1 {0, 1, . . . , ni },
i=1 xi
fX (x) = fi (xi ) =
0, otherwise,
i=1

Pp
where n = i=1 ni . Clearly, fY (y) = Pr(X1 + · · · + Xp = y) = 0, if y 6= {0, 1, . . . , n}. For y ∈ {0, 1, . . . , n}

fY (y) = Pr(Y = y)
= Pr(X1 + · · · + Xp = y)
np p
n1 X n2
!
X X Y ni Pp Pp
= ··· θ i=1 xi (1 − θ)n− i=1 xi
x =0 x =0 x =0 i=1
xi
1 2 p
x1 +···+xp =y
Lecture 0 Lecture Notes 0-103

np p
n1 X
n2
!
y n−y
X X Y ni n y
= θ (1 − θ) ··· = θ (1 − θ)n−y .
x1 =0 x2 =0 xp =0 i=1
xi y
x1 +···+xp =y

Thus, ( n
y θy (1 − θ)n−y , y ∈ {0, 1, . . . , n},
fY (y) =
0, otherwise.
Exercise: Let X1 , X2 , . . . , Xp be independent r.v.s with Xi having the p.m.f. (Poisson distribution)
( −λi x
e λi
x! , x ∈ {0, 1, . . . },
fi (x) =
0, otherwise,

where λi > 0, i = 1, 2, . . . , p are fixed real constants. Show that the p.m.f. of Y = X1 + X2 + · · · + Xp is
( −λ y
e λ
y! , y ∈ {0, 1, . . . },
fY (y) =
0, otherwise,
Pp
where λ = i=1 λi .
Theorem 0.197. Let X = (X1 , X2 , . . . , Xp ) be a continuous random vector with support S and joint p.d.f. f (·). Let
Si ⊆ Rp , i ∈ ∆ be a countable partition of S (Si ∩ Sj ∀ i 6= j and ∪i∈∆ Si = S). Suppose that hj : Rp → R,
j = 1, 2, . . . , p are functions such that in each Si0 , h = (h1 , h2 , . . . , hp ) : Si0 → R is one-to-one with inverse
transformation h−1 −1 −1 −1 0
i (t) = (h1,i (t), h2,i (t), . . . , hp,i )(t), i ∈ ∆, here S1 denotes the interior of Si , i ∈ ∆. Further
−1
suppose that hj,i (t), j = 1, 2, . . . , p, i ∈ ∆ have continuous partial derivatives and the Jacobian determinants

∂h−1
1,i (t) ∂h−1
1,i (t)
∂t1 ··· ∂tp
Ji = .. .. .. 6= 0, i ∈ ∆.
. . .
∂h−1
p,i (t) ∂h−1
p,i (t)
∂t1 ··· ∂tp

Define h(Sj0 ) = {h(x) = (h1 (x), h2 (x), . . . , hp (x)) ∈ Rp : x ∈ Sj }, j ∈ ∆ and Tl = hl (X1 , X2 , . . . , Xp ),

l = 1, 2, . . . , p. Then the random vector T = (T1 , T2 , . . . , Tp ) is a continuous random vector with p.d.f.
X
fT (t) = f (h−1 −1 −1
1,j (t), h2,j (t), . . . , hp,j (t))|Jj |Ih(Sj0 ) (t).
j∈∆

Corollary 0.198. Under the notation and assumption of the above theorem suppose that h = (h1 , h2 , . . . , hp ) :
S 0 → Rp is one-to-one with inverse transformation h−1 (t) = (h−1 −1 −1 0
1 (t), h2 (t), . . . , hp )(t) (say), here S denotes
−1
the interior of S. Furthermore suppose that hi (t), i = 1, 2, . . . , p have continuous partial derivatives and the
jacobian determinant
∂h−1
1 (t) ∂h−1
1 (t)
∂t1 ··· ∂tp
.. .. ..
J= . . . 6= 0.
∂h−1
p (t) ∂h−1
p (t)
∂t1 ··· ∂tp

Define h(S 0 ) = {h(x) : x ∈ S} and Tj = hj (X1 , X2 , . . . , Xp ), j = 1, 2, . . . , p. Then the random vector T =

(T1 , T2 , . . . , Tp ) is a continuous random vector with joint p.d.f.

fT (t) = f (h−1 −1 −1
1 (t), h2 (t), . . . , hp (t))|J|Ih(S 0 ) (t).
Lecture 0 Lecture Notes 0-104

(
e−x , x > 0, X1
Example 0.199. Let X1 and X2 be iid r.v.s with common p.d.f. f (x) = Find the p.d.f. of Y = X1 +X2 .
0, otherwise.

Solution: The joint p.d.f. of X = (X1 , X2 ) is

(
e−(x1 +x2 ) , x1 > 0, x2 > 0,
fX (x1 , x2 ) = f (x1 )f (x2 ) =
0, otherwise.
x1
Here S = [0, ∞) × [0, ∞), S 0 = (0, ∞) × (0, ∞). Define Z = X1 + X2 , h1 (x1 , x2 ) = x1 +x2 and h2 (x1 , x2 ) =
x1 + x2 . Then h : S 0 → R2 as 1-1; have h = (h1 , h2 ). We have
x1
h1 (x1 , x2 ) = = y, h2 (x1 , x2 ) = x1 + x2 = z =⇒ x1 = h−1 −1
1 (y, z) = yz and x2 = h2 (y, z) = z(1 − y).
x1 + x2
x ∈ S 0 ⇐⇒ x1 > 0, x2 > 0 ⇐⇒ yz > 0, z(1 − y) > 0 ⇐⇒ 0 < y < 1, z > 0. Thus, h(S 0 ) = (0, 1) × (0, ∞),
∂h−1
1 (y,z) ∂h−1
1 (y,z)
∂y ∂z z y
J= ∂h−1 ∂h−1
= = z.
p (y,z) p (y,z) −z 1−y
∂y ∂z

Thus the joint p.d.f. of (Y, Z) is

(
ze−z , 0 < y < 1, z > 0,
fY,Z (y, z) = fX (yz, z(1 − y))|z|I(0,1)×(0,∞) (y, z) = = fY (y)fZ (z),
0, otherwise

where ( (
1, 0 < y < 1, ze−z , z > 0,
fY (y) = and fZ (z) =
0, otherwise, 0, otherwise.
(
1, 0 < y < 1,
Thus, Y and Z are independent r.v.s with p.d.f.s given above. In particular the p.d.f. of Y is fY (y) =
0, otherwise.
Exercise: Let X1 and X2 be iid r.v.s with common p.d.f.
1
 2 , −2 < x < −1,

f (x) = 61 , 0 < x < 3,

0, otherwise.


Find the p.d.f. of Y1 = |X1 | + |X2 |.

Hint: Define auxiliary variable Y2 = |X1 |. Here S = ([−2, −1] ∪ [0, 3]) × ([−2, −1] ∪ [0, 3]) and

S 0 = (((−2, −1) ∪ (0, 3)) × ((−2, −1) ∪ (0, 3))) = S10 ∪ S20 ∪ S30 ∪ S40 ,

where S10 = (−2, −1) × (−2, −1), S20 = (−2, −1) × (0, 3), S30 = (0, 3) × (−2, −1) and S40 = (0, 3) × (0, 3). On
each S10 , h(x) = (h1 (x1 , x2 ), h2 (x1 , x2 )) = (y1 , y2 ) = (|x1 | + |x2 |, |x1 |) is 1-1. Now proceed.

0.15.12. Moment Generating Function Technique

Let X = (X1 , X2 , . . . , Xn ) be a random vector with p.m.f. / p.d.f. fX (·) and let g : Rp → Rq be a given function.
Suppose that we seek probability distribution (p.m.f. / p.d.f.) of Y = g(X). Under the m.g.f. technique, we try to
identify the m.g.f. MY (t) of random vector Y with the m.g.f. of some known distribution on a rectangle containing
origin. Then the uniqueness of m.g.f. as stated in the following theorem, ascertains that Y has that known distribution.
Lecture 0 Lecture Notes 0-105

Theorem 0.200. Let X and Y be 1-dimensional random vectors. Suppose that there exists an h > 0 such that

MX (t) = MY (t) ∀ t ∈ (−h, h) × (−h, h) × · · · × (−h, h).

d
Then X = Y .

0.15.13. Order Statistics

Let X1 , X2 , . . . , Xn be a random sample (of continuous r.v.s) from a distribution having d.f. F , p.d.f. f and support S.
Let Yr = r-th smallest of X1 , X2 , . . . , Xn , r = 1, 2, . . . , n. The Yr is called the r-th order statistic based on random
sample X1 , X2 , . . . , Xn and Y1 , Y2 , . . . , Yn are called order statistics based on random sample X1 , X2 , . . . , Xn .
Note that if X1 , X2 , . . . , Xn are continuous r.v.s then Pr(Y1 < Y2 < · · · < Yn ) = 1 and thus Y1 , Y2 , . . . , Yn are
uniquely defined with probability one.
Theorem 0.201. Under the above notation,
(a) the joint p.d.f. of Y = (Y1 , Y2 , . . . , Yn ) is
( Qn
n! i=1 f (yi ), −∞ < y1 < y2 < · · · < yn < ∞,
g(y1 , y2 , . . . , yn ) =
0, otherwise,

(b) the marginal p.d.f. of Yr , r = 1, 2, . . . , n is

n! n−r
gr (y) = [F (y)]r−1 [1 − F (y)] f (y), −∞ < y < ∞.
(r − 1)!(n − r)!

Proof. Since X = (X1 , X2 , . . . , Xn ) is a continuous random vector Pr(Y1 < Y2 < · · · < Yn ) = 1 (why?). Define
Sn = S × S × · · · × S, so that support of X = (X1 , X2 , . . . , Xn ) is Sn . Define

S10 = {x ∈ Sn : x1 < x2 < · · · < xn },

S20 = {x ∈ Sn : x1 < x2 < . . . xn < xn−1 },
..
.
0
Sn! = {x ∈ Sn : xn < xn−1 < · · · < x1 }.

On each Si0 , Y = (Y1 , Y2 , . . . , Yn ) = (h1,i (X), h2,i (X), . . . , hn,i (X)) is 1-1 with inverse transformation h−1
i =
(h−1 −1 −1
1,i , h2,i , . . . , hn,i ), i = 1, 2, . . . , n!. Note that as a set

{h−1 −1 −1
1,i , h2,i , . . . , hn,i } = {y1 , y2 , . . . , yn }, i = 1, 2, . . . , n!.

Therefore the Jacobian of inverse transformation in each Si is ±1.

h(S10 ) = {y ∈ Sn : y1 < y2 < · · · < yn } = T1 , say.

Then the joint p.d.f. of Y = (Y1 , Y2 , . . . , Yn ) is

n!
X
g(y) = fX (h−1 −1
1,j (y), . . . , hn,j (y))|Jj |Ih(Sj0 ) (y)
j=1
n! n
! n
X Y Y
= f (h−1
ij (y)) | ± 1|ITj (y) = n! f (yi ), −∞ < y1 < y2 < · · · < yn < ∞.
j=1 i=1 i=1
Lecture 0 Lecture Notes 0-106

(b) The marginal p.d.f. of Yr is

Z y Z yr−1 Z y2 Z ∞ Z ∞ Z ∞
gr (y) = ··· ··· n!f (y1 ) · · · f (yr−1 )f (y)f (yr+1 ) · · · f (yn )dyn · · · dyr+1 dy1 · · · dyr
−∞ −∞ −∞ y yr+1 yn−1
n! n−r
= [F (y)]r−1 [1 − F (y)] f (y), −∞ < y < ∞.
(r − 1)!(n − r)!

Similarly, for 1 ≤ r < s ≤ n, the joint p.d.f. of (Yr , Ys ) is

n! s−r−1 n−s
fYr fYs (y, z) = [F (y)]r−1 [F (z) − F (y)] [1 − F (y)] f (x)f (y),
(r − 1)!(s − r − 1)!(n − s)!
−∞ < x < y < ∞. This completes the proof.
Theorem 0.202. (a) Let X1 and X2 be independent r.v.’s with Xi ∼ GAM (αi , θ), αi > 0, θ > 0, i = 1, 2. Define
Y1 = X1 + X2 and Y2 = X1X+X1
2
. Then Y1 and Y2 are independently distributed with Y1 ∼ GAM (α1 + α2 , θ) and
Y2 ∼ Be(α1 , α2 ).
X1
(b) Let X1 and X2 be iid Exp(θ) r.v.’s. Then Y = X1 +X2 ∼ U (0, 1).

Proof. (a) The joint p.d.f. of X = (X1 , X2 ) is


−(x1 +x2 )/θ α1 −1 α2 −1
2
Y Y2 −xi /θ αi −1
e xi
e
 x1 x2
, if x1 > 0, x2 > 0,
fX (x1 , x2 ) = fXi (xi ) = I(0,∞) (x i ) = θα1 +α2 Γ(α1 )Γ(α2 )
i=1 i=1
θαi Γ(αi ) 
0, otherwise.

X1
Here, SX = (0, ∞)2 . Let h1 (X1 , X2 ) = Y1 = X1 + X2 and h2 (X1 , X2 ) = Y2 = X1 +X2 . Thus h = (h1 , h2 ) :
SX → R2 is 1 − 1 with inverse image (h−1 −1
1 , h2 ), where

y2 y1
h−1 −1
1 (y1 , y2 ) = y1 y2 , h2 (y1 , y2 ) = y1 (1 − y2 ), J = = −y1 .
1 − y2 −y1

h−1 (y) ∈ SX ⇐⇒ y1 y2 > 0, y1 (1 − y2 ) > 0 ⇐⇒ y1 > 0, 0 < y2 < 1 =⇒ h(SX ) = (0, ∞) × (0, 1). Thus the
joint p.d.f. of Y = (Y1 , Y2 ) is

e−(y1 y2 +y1 (1−y2 ))/θ (y1 y2 )α1 −1 (y1 (1 − y2 ))α2 −1

fY (y1 , y2 ) = | − y1 |I(0,∞)×(0,1) (y1 , y2 )
θα1 +α2 Γ(α1 )Γ(α2 )
−y1 /θ α1 +α2 −1
e y1 1 α1 −1 α2 −1
= I(0,∞) (y1 ) y (1 − y2 ) I(0,1) (y2 ) = fY1 (y1 )fY2 (y2 ),
Γ(α1 + α2 ) B(α1 , α2 ) 2

where Y1 ∼ GAM (α1 + α2 , θ) and Y2 ∼ Be(α1 , α2 ). Clearly Y1 and Y2 are independent. Part (b) can similarly be
proved.

0.16. Special Multivariate Distribution

0.16.1. Multinomial Distribution (A generalization of binomial distribution)

E : a random experiment whose each trial results in one (and only one) of p + 1 possible outcomes E1 , E2 , . . . , Ep+1
Pp+1 Pp
where Ei ∩ Ej P = φ and i=1 Ei = Ω. Let P (Ei ) = θi ∈ (0, 1), i = 1, 2 . . . , p and i=1 θi < 1 so that
p
P (Ep+1 ) = 1 − i=1 θi ∈ (0, 1).
Lecture 0 Lecture Notes 0-107

Consider P n independent trials of E . Define XiP= the number of times Ei occurs in n trials, i = 1, 2 . . . , p +
p+1 n
1. Then i=1 Xi = n, that is, Xp+1 = n − i=1 Xi . One may interested in probability distribution of X =
(X1 , X2 , . . . , Xp ). We have
n
X
SX = {x = (x1 , x2 , . . . , xp ) : xi ∈ {0, 1, . . . , n}, i = 1, 2, . . . , p, xi ≤ n}
i=1

and

fX (x1 , x2 , . . . , xp ) = P (X1 = x1 , X2 = x2 , . . . , Xp = xp )
n!
 Pp
x Pp
 Pp θ1x1 θ2x2 · · · θp p (1 − i=1 θi )n− i=1 xi , x ∈ SX ,
= x1 !x2 ! · · · xp !(n − i=1 xi )!
0, otherwise.


−→ Multinomial distribution with n trials and cell probabilities θ1 , . . . , θp (denoted by M ult(n, θ1 , θ2 , . . . , θp ))−→
a family of distribution
Pp with varying n ∈ N and θ = (θ1 , θ2 , . . . , θp ) ∈ Θ = {(t1 , t2 , . . . , tp ) : 0 < ti < 1, i =
1, 2, . . . , p and i=1 ti < 1}.
Remark 0.203. For p = 1, M ult(n, θ1 ) distribution is the same as Bin(n, θ1 ) distribution.

Theorem 0.204. Suppose that X = (X1 , X2 , . . . , Xp ) ∼ M ult(n, θ1 , θ2 , . . . , θp ), where n ∈ N and θ = (θ1 , θ2 ,

. . . , θp ) ∈ Θ. Then
(a) Xi ∼ Bin(n, θi ), i = 1, 2, . . . , p,
(b) Xi + Xj ∼ Bin(n, θi + θj ), i = 1, 2, . . . , p, j = 1, 2, . . . , p, i 6= j,
(c) E(Xi ) = nθi and Var(Xi ) = nθi (1 − θi ), i = 1, 2, . . . , p,
(d) Cov(Xi , Xj ) = −nθi θj , i = 1, 2, . . . , p, j = 1, 2, . . . , p, i 6= j.

Proof. (a) Fix i ∈ {1, 2, . . . , p}. A given trial of the experiment treat the occurrence of Ei as success and its non-
occurrence (that is, occurrence of any other Ej , j 6= i) as failure. Then we have a sequence of independent Bernoulli
trials with probability of success in each trial as P (Ei ) = θi . Thus

Xi = the number of times Ei occurs in n Bernoulli trials ∼ Bin(n, θi ), i = 1, 2, . . . , p.

(b) Fix i, j ∈ {1, 2, . . . , p} i 6= j. In any given trial of E consider occurrence of Ei or Ej as success and occurrence
of any other El (l 6= i, j) as failure. Then we have a sequence of n Bernoulli trials with success probability in each
trials as P (Ei ∪ Ej ) = θi + θj ,

Xi + Xj = the number of success occurs in n Bernoulli trials ∼ Bin(n, θi + θj ).

Var(Xi + Xj ) = n(θi + θj )(1 − θi − θj )

=⇒ Var(Xi ) + Var(Xj ) + 2 Cov(Xi , Xj ) = n(θi + θj )(1 − θi − θj )
=⇒ nθi (1 − θi ) + nθj (1 − θi ) + 2 Cov(Xi , Xj ) = n(θi + θj )(1 − θi − θj ) =⇒ Cov(Xi , Xj ) = −nθi θj .

This completes the proof.

Lecture 0 Lecture Notes 0-108

The m.g.f. of X = (X1 , X2 , . . . , Xp ) is given by

MX (t1 , t2 , . . . , tp ) = E(et1 X1 +t2 X2 +···+tp Xp )
n n n x p
X X X
t1 x1 +t2 x2 +···+tp xp n!θ1x1 θ2x2 · · · θp p X Pp
= ··· e Pp (1 − θi )n− i=1 xi
x1 =0 x2 =0 xp =0
x1 !x2 ! · · · xp !(n − i=1 xi )! i=1
x1 +x2 +···+xp ≤n
n n n p
X X X n!(θ1 et1 )x1 (θ2 et2 )x2 · · · (θp etp )xp X Pp
= ··· Pp (1 − θi )n− i=1 xi
x1 =0 x2 =0 x =0
x1 !x2 ! · · · xp !(n − i=1 xi )! i=1
p
x1 +x2 +···+xp ≤n
p
!n
X
t1 t2 tp
= θ1 e + θ2 e + · · · + θp e + 1 − θi , t ∈ Rp .
i=1

Remark 0.205. The last theorem can also be proved using m.g.f. For example (for i, j ∈ {1, 2, . . . , p}, i 6= j)
MXi +Xj (t) = MX (0, . . . , 0, t , 0, . . . , 0, t , 0, . . . , 0) = ((θi + θj )et + 1 − θi − θj ), t ∈ Rp .
ith position jth position

0.17. Bivariate Normal Distribution

Definition 0.206. A bivariate r.v. X = (X1 , X2 ) is said to follow bivariate normal distribution N2 (µ1 , µ2 , σ12 , σ22 , ρ)
if for some −∞ < µi < ∞, σi > 0, i = 1, 2 and −1 < ρ < 1, the joint p.d.f. of X = (X1 , X2 ) is given by
2 2
1 x1 −µ1 x1 −µ1 x2 −µ2 x −µ
1 − 2(1−ρ 2) σ1 −2ρ σ1 σ2 + 2σ 2
fX1 ,X2 (x1 , x2 ) = p e 2
, ∞ < xi < ∞, i = 1, 2.
2πσ1 σ2 1 − ρ2

Clearly, fX1 ,X2 (x1 , x2 ) ≥ 0 ∀ x ∈ R2 and on making the transformation x1σ−µ 1

1
= z1 and x2σ−µ 2
2
= z2 (so that
J = σ1 σ2 ) we have
Z Z ∞Z ∞
1 − 1
(z 2 −2ρz1 z2 +z22 )
fX1 ,X2 (x1 , x2 )dx1 dx2 = p e 2(1−ρ2 ) 1 dz1 dz2
2
2π 1 − ρ −∞ −∞
R2
Z ∞ 2
Z ∞
1 z2
− 1
(z −ρz )2
= p e− 2 e 2(1−ρ2 ) 1 2 dz2 dz1
2π 1 − ρ2 −∞ −∞
Z ∞ 2
1 z2
=√ e− 2 dz2 = 1 =⇒ fX1 ,X2 (x1 , x2 ) is a p.d.f.
2π −∞
Note that for x = (x1 , x2 ) ∈ R2
2 2
x1 −µ1 x2 −µ2 x2 −µ2
1 1
− 2(1−ρ 2) σ1 −ρ σ2 +(1−ρ2 ) σ2
fX1 ,X2 (x1 , x2 ) = p e
2πσ1 σ2 1 − ρ2
h i2
ρσ
1 − 1
2 (1−ρ2 ) x1 − µ1 + σ 1 (x2 −µ2 ) 1 − 12 (x2 −µ2 )2
=√ p e 2σ1 2
√ e 2σ2
2πσ1 1 − ρ2 2πσ2
= fX1 |X2 (x1 |x2 )fX2 (x2 )

ρσ1
=⇒ X1 |X2 = x2 ∼ N µ1 + (x2 − µ2 ), σ1 (1 − ρ ) , X2 ∼ N (µ2 , σ22 ).
2 2
σ2
By symmetry
ρσ2
X2 |X1 = x1 ∼ N µ2 + (x1 − µ1 ), σ22 (1 − ρ2 ) , X1 ∼ N (µ1 , σ12 ).
σ1
Lecture 0 Lecture Notes 0-109

Clearly, µ1 = E(X1 ), µ2 = E(X2 ), σ12 = Var(X1 ) and σ22 = Var(X2 ).

m.g.f. MX1 ,X2 (t1 , t2 ) = E et1 X1 +t2 X2

= E E et1 X1 +t2 X2 |X2 = E et2 X2 E et1 X1 |X2 , t = (t1 , t2 ) ∈ R2 ,

E et1 X1 |X2 = m.g.f. of conditional distribution X1 |X2 at point t2

2 (1−ρ2 )t2
σ1
ρσ1 1
= e{µ1 + σ2 (X2 −µ2 )}t1 + 2

2 (1−ρ2 )t2
σ1
ρσ1 h 1 ρσ1 i
MX1 ,X2 (t1 , t2 ) = e{µ1 − σ2 µ2 }t1 + 2
E et2 X2 e σ2 t1 X2
σ 2 (1−ρ2 )t2

ρσ
{µ1 − σ 1 µ2 }t1 + 1 2 1 ρσ1
=e 2 MX2 t2 + t1
σ2
2 (1−ρ2 )t2
σ1 σ22
ρσ1 1 ρσ1 ρσ1 2
= e{µ1 − σ2 µ2 }t1 + 2
eµ2 {t2 + σ2 t1 }+ 2 (t2 + σ2 t1 )

2 t2
σ1 2 t2
σ2
1 2
= eµ1 t1 +µ2 t2 + 2 + 2 +ρσ1 σ2 t1 t2
, t = (t1 , t2 ) ∈ R2 .

Thus we have the following theorem.

Theorem 0.207. Suppose that X = (X1 , X2 ) ∼ N2 (µ1 , µ2 , σ12 , σ22 , ρ), −∞ < µi < ∞, σi > 0, i = 1, 2 and
−1 < ρ < 1. Then
(a) X1 ∼ N (µ1 , σ12 ) and X2 ∼ N (µ2 , σ22 );

ρσ1
(b) For fixed x2 ∈ R, X1 |X2 = x2 ∼ N µ1 + σ2 (x2 − µ2 ), σ12 (1 − ρ2 ) and for fixed x1 ∈ R, X2 |X1 = x1 ∼

N µ2 + ρσ σ1
2
(x 1 − µ1 ), σ2
2
(1 − ρ 2
) ;

(c) The m.g.f. of X = (X1 , X2 ) is

2 t2
σ1 2 t2
σ2
1 2
MX1 ,X2 (t1 , t2 ) = eµ1 t1 +µ2 t2 + 2 + 2 +ρσ1 σ2 t1 t2
, t = (t1 , t2 ) ∈ R2 ;

(d) ρ(X1 , X2 ) = Corr(X1 , X2 ) = ρ;

(e) X1 and X2 are independent iff ρ = 0;
(f) For real constants C1 and C2 such that (C1 , C2 ) 6= (0, 0)

C1 X1 + C2 X2 ∼ N (C1 µ1 + C2 µ2 , C12 σ12 + C22 σ22 + 2ρC1 C2 σ1 σ2 )

Proof. (a)-(c) Already done.

(d) For t = (t1 , t2 ) ∈ R2

σ12 t21 σ 2 t2
ψX1 ,X2 (t1 , t2 ) = ln MX1 ,X2 (t1 , t2 ) = µ1 t1 + µ2 t2 + + 2 2 + ρσ1 σ2 t1 t2
2 2
∂
ψX ,X (t1 , t2 ) = µ1 + 2σ12 t1 + ρσ1 σ2 t2
∂t1 1 2
∂2
ψX ,X (t1 , t2 ) = ρσ1 σ2
∂t2 ∂t1 1 2
∂2

=⇒ Cov(X1 , X2 ) = ψX1 ,X2 (t1 , t2 ) = ρσ1 σ2
∂t2 ∂t1 t=0
Cov(X1 , X2 )
=⇒ ρ(X1 , X2 ) = Corr(X1 , X2 ) = p = ρ.
Var(X1 ) Var(X2 )
Lecture 0 Lecture Notes 0-110

(e) Obviously, if X1 and X2 are independent then ρ = Corr(X1 , X2 ) = 0. Now suppose that ρ = 0. Then
2 2
x1 −µ1 x −µ
1 − 21 σ1 + 2σ 2
2
fX1 ,X2 (x1 , x2 ) = e
2πσ1 σ2
1 − 12 [x1 −µ1 ]2 1 − 12 (x2 −µ2 )2
=√ e 2σ1 √ e 2σ2
2πσ1 2πσ2
= fX1 (x1 )fX2 (x2 ) ∀ x = (x1 , x2 ) ∈ R2 =⇒ X1 and X2 are independent.

(f) Let Y = C1 X1 + C2 X2 . Then

MY (t) = E(etY ) = E(et(C1 X1 +C2 X2 ) ) = MX1 ,X2 (tC1 , tC2 )

C12 t2 σ12 C22 t2 σ22

2
= exp C1 tµ1 + C2 tµ2 + + + ρt C1 C2 σ1 σ2
2 2
t2

2 2 2 2
= exp (C1 µ1 + C2 µ2 )t + (C1 σ1 + C2 σ2 + 2ρC1 C2 σ1 σ2 )
2
−→ m.g.f. of N (C1 µ1 + C2 µ2 , C12 σ12 + C22 σ22 + 2ρC1 C2 σ1 σ2 ).

This completes the proof.

Theorem 0.208. Let X = (X1 , X2 ) be a bivariate r.v. with E(Xi ) = µi ∈ (−∞, ∞), Var(Xi ) = σi2 , (σi > 0),
i = 1, 2 and Corr(X1 , X2 ) = ρ ∈ (−1, 1). Then X ∼ N2 (µ1 , µ2 , σ12 , σ22 , ρ) iff for any t = (t1 , t2 ) ∈ R2 − {0},
Y = t1 X1 + t2 X2 ∼ N (t1 µ1 + t2 µ2 , t21 σ12 + t22 σ22 + 2ρt1 t2 σ1 σ2 ).

Proof. Let X ∼ N2 (µ1 , µ2 , σ12 , σ22 , ρ). Then by (f) of last theorem

Y = t1 X1 + t2 X2 ∼ N (t1 µ1 + t2 µ2 , t21 σ12 + t22 σ22 + 2ρt1 t2 σ1 σ2 ) ∀ t ∈ R2 − {0}.

Conversely, suppose that for all t = (t1 , t2 ) ∈ R2 − {0}, Y = t1 X1 + t2 X2 ∼ N (t1 µ1 + t2 µ2 , t21 σ12 + t22 σ22 +
2ρt1 t2 σ1 σ2 ). Then for t = (t1 , t2 ) ∈ R2 − {0}

MX1 ,X2 (t1 , t2 ) = E(et1 X1 +t2 X2 )

2 2 2 2 1
= MY (1) = exp (t1 µ1 + t2 µ2 ) + (t1 σ1 + t2 σ2 + 2ρt1 t2 σ1 σ2 )
2
−→ m.g.f. of N2 (µ1 , µ2 , σ12 , σ22 , ρ) =⇒ X = (X1 , X2 ) ∼ N2 (µ1 , µ2 , σ12 , σ22 , ρ).

This completes the proof.

X2 , . . . , Xn (n ≥ 2) be a random
Theorem 0.209. Let X1 , P sample from N (µ, σ 2 ) distribution, where −∞ < µ < ∞
1 n 1
Pn
and σ > 0. Let X̄ = n i=1 Xi and S = n−1 i=1 (Xi − X̄)2 denote the sample mean and sample variance,
2

respectively. Then
2

(i) X̄ ∼ N µ, σn ;

(ii) X̄ and S 2 are independent r.v.’s;

(n−1)S 2
(iii) σ2 ∼ χ2n−1 ;
q
2σ 4 2 Γ(n/2)
(iv) E(S 2 ) = σ 2 , Var(S 2 ) = n−1 , E(S) = n−1 Γ( n−1 σ.
2 )
Lecture 0 Lecture Notes 0-111

1
Proof. (i) Follows from last theorem by taking k = n, ai = n, µi = µ, σi2 = σ, i = 1, 2, . . . , n.
(ii) Let Yi = Xi − X̄, i = 1, 2, . . . , n and let Y = (Y1 , Y2 , . . . , Yn ). Then
n
X n
X n
X
Yi = (Xi − X̄) = Xi − nX̄ = 0
i=1 i=1 i=1
n
X n
X
(n − 1)S 2 = (Xi − X̄)2 = Yi2 (a function of Y )
i=1 i=1

The joint m.g.f. of (Y , X̄) is given by

Pn
MY ,X̄ (t) = E e i=1 ti Yi +tn+1 X̄ , t = (t1 , t2 , . . . , tn , tn+1 ) ∈ Rn+1
n
X n
X
ti Yi + tn+1 X̄ = ti (Xi − X̄) + tn+1 X̄
i=1 i=1
n Pn n
X (tn+1 − i=1 ti ) X
= ti Xi + Xi
i=1
n i=1
n n
X tn+1 1X
= ti − t̄ + Xi , where t̄ = ti
i=1
n n i=1
n
X tn+1
= ui Xi , where ui = ti − t̄ + , i = 1(1)n.
i=1
n
Pn Pn Pn t2n+1
Then i=1 ui = tn+1 and i=1 u2i = i=1 (ti − t̄)2 + n .
Pn
MY ,X̄ (t) = E e i=1 ui Xi
n
Y
= MXi (ui )
i=1
n
2
1
u2i
Y
= eµui + 2 σ
i=1
σ2
µ n
P Pn 2
=e i=1 ui + 2 i=1 ui

t2 σ 2 t2
n
σ2 σ2
Pn 2 n+1 n+1 Pn 2
o
{ i=1 (ti −t̄) } i=1 (ti −t̄)
= eµtn+1 + 2 + n = eµtn+1 + 2n e2

σ2
Pn 2
MY (t1 , t2 , . . . , tn ) = MY ,X̄ (t1 , t2 , . . . , tn , 0) = e 2 i=1 (ti −t̄) , (t1 , t2 , . . . , tn ) ∈ Rn
σ 2 t2
n+1
MX̄ (tn+1 ) = MY ,X̄ (0, . . . , 0, tn+1 ) = eµtn+1 + 2n , tn+1 ∈ R
=⇒ MY ,X̄ (t) = MY (t1 , t2 , . . . , tn )MX̄ (tn+1 ), ∀ t = (t1 , t2 , . . . , tn , tn+1 ) ∈ Rn+1
=⇒ Y and X̄are independent
Xn
=⇒ (Xi − X̄)2 and X̄ are independent.
i=1
√
(iii) Let Zi = Xiσ−µ , i = 1, 2, . . . , n. Then Z1 , Z2 , . . . , Zn are iid N (0, 1) r.v.’s. Also let Z = n(X̄−µ)
σ ∼ N (0, 1)
(using (i)). Let
n(X̄ − µ)2 (n − 1)S 2
W = Z2 = and T = .
σ2 σ2
Lecture 0 Lecture Notes 0-112

Pn
Then by (ii), W and T are independent r.v.s. Also W ∼ χ21 and V = i=1 Zi2 ∼ χ2n .
n n
X 1 X
V = Zi2 = (Xi − µ)2
i=1
σ 2 i=1
n
1 X
= 2 (Xi − X̄ + X̄ − µ)2
σ i=1
n
1 X 2 n(X̄ − µ)2 (n − 1)S 2 n(X̄ − µ)2
= (Xi − X̄) + = + = T + W.
σ 2 i=1 σ2 σ2 σ2

This implies

MV (t) = MT (t)MW (t)

MV (t) (1 − 2t)−n/2 1
=⇒ MT (t) = = −1/2
= (1 − 2t)−(n−1)/2 , t < → m.g.f. of χ2n−1
MW (t) (1 − 2t) 2
2
(n − 1)S
=⇒ T = ∼ χ2n−1 .
σ2
(n−1)S 2
(iv) T = σ2 ∼ χ2v , where v = n − 1. Thus
Z ∞
1
E(T s ) = ts v/2 e−t/2 tv/2−1 dt
0 2 Γ(v/2)
v+2s v+2s
2 2 Γ( 2 )
Z ∞
2s Γ v+2s

1 −t/2 v+2s −1 2 v
= v/2 e t 2 dt = v/2 = , s>− .
2 Γ(v/2) 0 2 Γ(v/2) Γ(v/2) 2

This implies

2s Γ v+2s

(n − 1)s 2s 2
E(S ) =
σ 2s Γ(v/2)
r/2
Γ v+r

r 2
=⇒ E(S ) = 2
σr ,
n−1 Γ(v/2)
r/2
Γ n−1+r

2
=⇒ E(S r ) = 2
σr , r > 0
n−1 Γ((n − 1)/2)
1/2
Γ n2

2
=⇒ E(S) = σ
n−1 Γ((n − 1)/2)
Γ n−1

2 2 +1
=⇒ E(S 2 ) = σ2 = σ2
n − 1 Γ((n − 1)/2)
2
Γ n−1

4 2 2 +2 n+1 4
=⇒ E(S ) = σ4 = σ
n−1 Γ((n − 1)/2) n−1
2σ 4
Var(S 2 ) = E(S 4 ) − (E(S 2 ))2 = .
n−1
This completes the proof.
Remark
Pn 0.210. Let X1 , XP
2 , . . . , Xn be a random sample from a distribution having p.m.f. / p.d.f. f . Let X̄ =
1 2 1 n 2 2 1
P n
n X
i=1 i and S = n−1 i=1 (X i − X̄) . Let E(X1 ) = µ and Var(X1 ) = σ . Then E( X̄) = E n i=1 Xi =
Lecture 0 Lecture Notes 0-113

σ2
1
Pn
µ, Var(X̄) = Var n i=1 Xi = n .
" n #
X
2 2

E (n − 1)S = E (Xi − X̄)
i=1
" n #
X
2
=⇒ (n − 1)E(S ) = E Xi2 − nX̄ 2

i=1
n
X
= E(Xi2 ) − nE(X̄ 2 )
i=1
= n E(X12 ) − E(X̄ 2 )

= n Var(X1 ) + (E(X1 ))2 − Var(X̄) − (E(X̄))2

σ2
− µ2 ) = (n − 1)σ 2 =⇒ E(S 2 ) = σ 2 .
= n(σ 2 + µ2 −
n
1
Pn Pn
For this reason S 2 = n−1 (Xi − X̄)2 is called sample variance and not S12 = n1 i=1 (Xi − X̄)2 . Note that
i=1
n
E(S12 ) = E n1 i=1 (Xi − X̄)2 = n−1 2 2 2 2
P
n σ < σ , i.e., S1 underestimates σ .

0.18. Distributions Based on Sampling from Normal Distribution

Definition 0.211. (a) For a positive integer m, a random variable X is said to have the student t-distribution with m
degrees of freedom (written as X ∼ tm ) if the p.d.f. of X is given by
− m+1
Γ m+1

2 x2 2

f (x|m) = √ 1+ , −∞ < x < ∞.

mπΓ(m/2) m
(b) For positive integers n1 and n2 a random variable X is said to have the Snedecor F distribution with (n1 , n2 )
degrees of freedom (written as X ∼ Fn1 ,n2 ) if its p.d.f. is given by
n1 −1


n n x

2

 1 1


n2 n2 −(n1 +n2 )
f (x|n1 , n2 ) = 1 + n1 x
, x > 0,
 B(n1 /n2 , n2 /2)
 n2


0, otherwise.


Remark 0.212. (a) Note that

X ∼ tm =⇒ f (x|m) = f (−x|m), ∀ x
d
=⇒ X = −X
=⇒ distribution of X is symmetric about 0 =⇒ me = 0 and E(X) = 0, provided it exists.
(b) X ∼ tm =⇒ f (x|m) ↑ m(−∞, 0), ↓ (0, ∞) =⇒ m0 = 0.
(c) t1 distribution is nothing but Cauchy distribution with p.d.f.
1 1
f (x|1) = , −∞ < x < ∞ =⇒ E(X) does not exist.
π 1 + x2
(d) Let X ∼ Fn1 ,n2 . Then,

n1 n1 x
! n21 −1 !n2 −1
1 − nn12x
−2

 n2 n2 n1 x
1 + , x > 0,

f (x|n1 , n2 ) = B(n1 /2, n2 /2) 1 + nn1 x 1 + nn12x n2
 2

0, otherwise.

Lecture 0 Lecture Notes 0-114

n1 X
n2
=⇒ Y = ∼ Be (n1 /2, n2 /2) .
1 + nn12X

Theorem 0.213. (a) Let Z ∼ N (0, 1) and let Y ∼ χ2m , m ∈ {1, 2, . . . } be independent random variables. Then

Z
T =p ∼ tm .
Y /m

(b) For positive integers n1 and n2 , let X1 ∼ χ2n1 and X2 ∼ χ2n2 be independent random variables, then

X1 /n1
U= ∼ Fn1 ,n2 .
X2 /n2

(c) Let X ∼ tm . Then E(X 2 ) is not finite if r ∈ {m, m + 1, . . . }. For r ∈ {1, 2, . . . , m − 1} (m ≥ r + 1)


0, if r is odd,

r
E(X ) = mr/2 r!Γ((m − r)/2)
 , if r is even.
2r (r/2)!Γ(m/2)


(d) If X ∼ tm then

Mean = µ01 = E(X) = 0, m = 2, 3, . . . ,

m
Var(X) = µ2 = E((X − µ01 )2 ) = , m ∈ {3, 4, . . . },
m−2
Coefficient of skewness = β1 = 0, m = 4, 5, 6, . . . ,
3(m − 2)
Kurtosis = ν1 = , m ∈ {5, 6, . . . }.
m−4
n2
(e) Let n1 , n2 and r be positive integers, and let X ∼ Fn1 ,n2 . Then, for n2 ∈ {1, 2, . . . , 2r} and r ≥ 2 , it follows
that E(X r ) is not finite. For n2 ∈ {2r + 1, 2r + 2, . . . } and r ≥ n22−1 , we have
r
r Y
r n2 n1 + 2(i − 1)
E(X ) = .
n1 i=1
n2 − 2i

(f) If X ∼ Fn1 ,n2 then

n2
Mean = µ01 = E(X) = , if n2 ∈ {3, 4, . . . },
n2 − 2
2n22 (n1 + n2 − 2)
Var(X) = µ2 = E((X − µ01 )2 ) = , if n2 ∈ {5, 6, . . . },
n1 (n1 − 2)2 (n2 − 4)
s
2(2n1 + n2 − 2) 2(n2 − 4)
Coefficient of skewness = β1 = , n2 ∈ {7, 8, . . . },
n2 − 6 n1 (n1 + n2 − 2)

12 (n2 − 2)2 (n2 − 4) + n1 (n1 + n2 − 2)(5n2 − 22)
Kurtosis = ν1 = .
n1 (n2 − 6)(n2 − 8)(n1 + n2 − 2)

Proof. (a) The joint p.d.f. of (Y, Z) is given by

1 y+z 2

m

(m+1)/2
√ e− 2 y 2 −1 , y > 0, −∞ < z < ∞,
fY,Z (y, z) = fY (y)fZ (z) = 2 Γ(m/2) π

0, otherwise.
Lecture 0 Lecture Notes 0-115

q
Y
Let U = m. SY,Z = (0, ∞) × R. Let h = (h1 , h2 ) : (0, ∞) × R → R2 where h1 (y, z) = √ z and
y/m
h2 (y, z) = y/m. The transformation h : SY,Z → R is 1-1 with inverse transformation h−1 = (h−1 −1
p
1 , h2 ), where
0 2mu
h−1 2 −1
1 (t, u) = mu , h2 (t, u) = tu, J = u = −2mu2 .
t

h(SY,Z ) = {(t, u) : mu2 > 0, −∞ < tu < ∞} = {(t, u) : u > 0, t ∈ R} = R × (0, ∞).

The joint p.d.f. of (T, U ) is given by

fT,U (t, u) = fY,Z (h−1 −1

1 (t, u), h2 (t, u))|J|Ih(SY,Z ) (t, u)
1 mu2 +t2 u2 m
= (m+1)/2 √ e− 2 (mu2 ) 2 −1 |2mu2 |IR×(0,∞) (t, u)
2 Γ(m/2) π
mm/2 um (m+t2 )u2
= (m−1)/2
√ e− 2 IR (t)I(0,∞) (u).
2 Γ(m/2) π
The marginal p.d.f. of T is
Z ∞
fT (t) = fT,U (t, u)du
−∞
Z ∞
mm/2 (m+t2 )u2
= (m−1)/2 √ um e− 2 du
2 Γ(m/2) π 0
Z ∞
1 1 m−1
= √ y 2 e−y dy (u2 = y)
Γ(m/2) mπ (1 + t2 /m) m+1
2 0

Γ( m+1
2 ) 1
= √ , t ∈ R −→ p.d.f. of tm .
Γ(m/2) mπ (1 + t2 /m) m+1
2

(b) The joint p.d.f. of X = (X1 , X2 ) is given by

1 x1 +x2 n1
−1
n2
−1
fX1 ,X2 (x1 , x2 ) = fX1 (x1 )fX2 (x2 ) = e− 2 x12 x22 I(0,∞)×(0,∞) (x1 , x2 ).
2(n1 +n2 )/2 Γ(n 1 /2)Γ(n2 /2)

X2
Let V = n2 . SX = (0, ∞) × (0, ∞). Consider the transformation: h = (h1 , h2 ) : (0, ∞) × (0, ∞) → R defined by
x1 /n1 x2
h1 (x1 , x2 ) = x2 /n2 and h2 (x1 , x2 ) = n2 so that U = h1 (X1 , X2 ) and V = h2 (X1 , X2 ).

The transformation h : (0, ∞) × (0, ∞) → R2 is 1-1 with inverse transformation h−1 = (h−1 −1
1 , h2 ), where

n1 v n1 u
h−1 −1
1 (u, v) = n1 uv, h2 (u, v) = n2 v, J = = n1 n2 v,
0 n2

h(SX ) = {(u, v) : n1 uv > 0, n2 v > 0} = {(u, v) : u > 0, v > 0} = (0, ∞) × (0, ∞).
Thus, the joint p.d.f. of (U, V ) is given by

fU,V (u, v) = fX1 ,X2 (h−1 −1

1 (u, v), h2 (u, v))|J|Ih(SX ) (u, v)
n /2 n /2
n1 1 n2 2 (n2 +n1 u)v n1 n1 +n2
= e− 2 u 2 −1 v 2 −1 I(0,∞) (u)I(0,∞) (v)
2(n1 +n2 )/2 Γ(n1 /2)Γ(n2 /2)
The marginal p.d.f. of U is given by
Z ∞
fU (u) = fU,V (u, v)dv
−∞
Lecture 0 Lecture Notes 0-116

n /2 n /2 ∞
n1 1 n2 2
Z
n1 (n2 +n1 u)v n1 +n2
= u 2 −1 e− 2 v 2 −1
dv
2(n1 +n2 )/2 Γ(n1 /2)Γ(n2 /2) 0
n1
Γ( n1 +n2 ) (n1 u/n2 ) 2 −1
= (n +n )/2 2 I(0,∞) −→ p.d.f. of Fn1 ,n2 .
2 1 2 Γ(n1 /2)Γ(n2 /2) (1 + n u/n ) n1 +n
2
2
1 2

d
(c) Fix m ∈ {1, 2, . . . }. Then X = √ Z where Z ∼ N (0, 1) and Y ∼ χ2m are independent. This implies that
Y /m
!r
Z
r
E(X ) = E p = mr/2 E(Z r Y −r/2 ) = mr/2 E(Z r )E(Y −r/2 ) (Y and Z are independent)
Y /m
(
0, if r = 1, 3, 5, . . . ,
r
E(Z ) = r!
2r/2 (r/2)!
, if r = 2, 4, 6 . . . .
Z ∞
1 m−r
E(Y −r/2 ) = y 2 −1 e−y/2 dy = ∞, if r ≥ m.
2m/2 (m/2)! 0

For r < m, we have

m−r
2 2 Γ( m−r 2 ) Γ( m−r
2 )
E(Y −r/2 ) =
m/2
= r/2
2 Γ(m/2) 2 Γ(m/2)

0, if r is odd and r < m,

=⇒ E(X r ) = mr/2 r!Γ(m − r)/2

 r , if r is even and r < n.
2 (r/2)!Γ(m/2)
(d) Exercise.
(e) Fix n1 , n2 ∈ N. Then
d X1 /n1 n2 X1
X= = ,
X2 /n2 n1 X2
where X1 ∼ χ2n1 and X2 ∼ χ2n2 are independent. For r ∈ N,
r r r
r n2 X1 n2 r 1
E(X ) = E = E(X1 )E ,
n1 X2r n1 X2r
Z ∞
1 n1 +2r
E(X1r ) = n /2 x 2 −1 e−x/2 dx
2 1 Γ(n1 /2) 0
n1 +2r r
Γ( n1 +2r
2 2 ) Y
2
= = (n1 − 2(i − 1)), r ∈ {1, 2, . . . },
2n1 /2 Γ(n1 /2) i=1
 n1 −2r
n2 −2r
 2 2 Γ( 2 )
(Qr
i=1 (n2 − 2i), n2 > 2r,

1 , n 2 > 2r,
E = 2n2 /2 Γ(n2 /2) =
X22  ∞, if n2 ≤ 2r,
∞, if n2 ≤ 2r

 r
 n2 Qr n1 +2(i−1)
, n2 > 2r,
n1 i=1 n2 −2i
=⇒ E(X r ) =
∞, if n ≤ 2r.
2

(f) Exercise.
Corollary 0.214. Let X1 , X2 , . . . , Xn (n ≥ 2) be a random sample from N (µ, σ 2 ) distribution, where µ ∈ R and
σ > 0. Let
n n
1X 1 X
X̄ = Xi , S 2 = (Xi − X̄)2
n i=1 n − 1 i=1
Lecture 0 Lecture Notes 0-117

denote the sample mean and sample variance, respectively. Then,

√
n(X̄ − µ)
∼ tn−1 .
S

Proof. We know that

(n − 1)S 2
X̄ ∼ N (µ, σ 2 /n) and ∼ χ2n−1 independent
σ2
√
n(X̄ − µ) (n − 1)S 2
=⇒ ∼ N (0, 1) and ∼ χ2n−1 independent
σ σ2
√ √
n(X̄ − µ)/σ n(X̄ − µ)
=⇒ p ∼ tn−1 , that is, ∼ tn−1 .
2
S /σ 2 S

This completes the proof.

Corollary 0.215. Let X1 , X2 , . . . , Xm (m ≥ 2) and Y1 , Y2 , . . . , Yn (n ≥ 2) be independent random samples (that
is, X = (X1 , X2 , . . . , Xm ) and Y = (Y1 , Y2 , . . . , Ym ) are independent) from N (µ1 , σ12 ) and N (µ2 , σ22 ) distribution,
respectively where µi ∈ R, i = 1, 2, and σi > 0, i = 1, 2. Let
m n m n
1 X 1X 1 X 1 X
X̄ = Xi , Ȳ = Yi , S12 = (Xi − X̄)2 , S22 = (Yi − Ȳ )2 .
m i=1 n i=1 m − 1 i=1 n − 1 i=1

X̄ − Ȳ − (µ1 − µ2 )
Then, (a) p ∼ N (0, 1),
σ12 /m + σ22 /n
1 X̄ − Ȳ − (µ1 − µ2 )
(b) p 2 2
q 2 2 2 2
∼ tm+n−2 ,
σ1 /m + σ2 /n (m−1)S 1 /σ1 +(n−1)S1 /σ2
m+n−2

σ22 S12
(c) ∼ Fm−1,n−1 .
σ12 S22

(m−1)S12 (n−1)S22
Proof. X̄ ∼ N (µ1 , σ12 /m), Ȳ ∼ N (µ2 , σ22 /n), σ12
∼ χ2m−1 and σ22
are independent r.v.s. Thus,

X̄ − Ȳ ∼ N µ1 − µ2 , σ12 /m + σ22 /n

(m − 1)S12 (n − 1)S22
+ ∼ tm+n−2
σ12 σ22
X̄ − Ȳ − (µ1 − µ2 ) 1 X̄ − Ȳ − (µ1 − µ2 )
=⇒ p 2 2
∼ N (0, 1) and p 2 2
q ∼ tm+n−2 .
σ1 /m + σ2 /n σ1 /m + σ2 /n (m−1)S12 /σ12 +(n−1)S12 /σ22
m+n−2

This completes the proof.

Remark 0.216. (a) Note that

X ∼ tm

N (0, 1)
d
=⇒ X = p independent
χ2m /m
(N (0, 1))2

d
=⇒ X 2 = independent
χ2m /m
Lecture 0 Lecture Notes 0-118

χ2

d
= 2 1 independent = F1,m .
χm /m

Thus, X ∼ tm =⇒ X 2 ∼ F1,m .
(b) Note that

X ∼ Fn1 ,n2
χ2 /n1

d
=⇒ X = n2 1 independent
χn2 /n2
1 d χ2n2 /n2

d
=⇒ = 2 independent = Fn2 ,n1 .
X χn1 /n1
1
Thus, X ∼ Fn1 ,n2 =⇒ X ∼ Fn2 ,n1 .

(c) X ∼ tm =⇒ Kurtosis = ν1 = 3(m−2) m−4 , m > 4 =⇒ tm distribution (m > 4) is symmetric and leptokurtic (that
is, it has sharper peak and longer fatter tails compared to N (0, 1) distribution). As m → ∞, ν1 → ∞. This suggests
that for large d.f. m, tm distribution behaves like N (0, 1) distribution.
(d) For various values of m ∈ N and α ∈ (0, 1), the d.f. of tm is tabulated in various text books.
(e) For fixed n1 ∈ N, n2 ∈ N and α ∈ (0, 1) let fn1 ,n2 ,α be the (1 − α)-th quantile of X ∼ Fn1 ,n2 . Thus

1 1 1 1
P (X ≤ fn1 ,n2 ,α ) = 1 − α =⇒ P ≤ = α =⇒ fn2 ,n1 ,1−α = as ∼ Fn2 ,n1 .
X fn1 ,n2 ,α fn1 ,n2 ,α X

Example 0.217. Let X1 , X2 , . . . , Xn be a random sample from N (µ, σ 2 ) distribution, where µ ∈ R, σ > 0 and
n ≥ 2. Let
n n
1X 1 X
X̄ = Xi and S 2 = (Xi − X̄)2
n i=1 n − 1 i=1

be the sample mean and sample variance, respectively. Evaluate E X̄S , for n > 2.

Solution: We have
(n − 1)S 2
X̄ ∼ N (µ, σ 2 /n) and Y = 2
∼ χ2n−1 are independent
σ
√
X̄ n−1
=⇒ E = E(X̄Y −1/2 )
S σ
√
n−1
= E(X̄)E(Y −1/2 ) (indepedence)
σ
√ Z ∞ −y/2 n−2 −1
n−1 e y 2
= µ n−1 n−1 dy
σ 0 2 2 Γ( 2 )
√ n−2
n − 1 2 2 Γ( n−2 (n − 1)/2 Γ( n−2
p
2 ) 2 )µ
= µ n−1 n−1 = n−1 σ .
σ 2 2 Γ( 2 ) σ Γ( 2 )
Pn
Example
Pn 0.218. Let P Z1 , Z2 , . . . , Zn be iid N (0, 1) r.v.s and let ai , bi ∈ R, i = 1, . . . , n be such that i=1 a2i >
n
0, i=1 b2i > 0 and i=1 ai bi = 0. Show that
sP
n Pn
b2 ai Zi
(a) Y1 = Pni=1 i2 Pi=1 n ∼ t1 ;
a
i=1 i | i=1 bi Zi |
Lecture 0 Lecture Notes 0-119

Pn 2 Pn 2
b ai Zi
(b) Y2 = Pni=1 i2 Pi=1
n ∼ F1,1 ;
i=1 ai i=1 bi Zi
sP
n 2
Pn
i=1 bi Pi=1 ai Zi
(c) Y3 = P n 2 n ∼ t1 .
i=1 ai i=1 bi Zi

Solution: Linear combination of Z1 , Z2 , . . . , Zn :

n
X n
X
c1 ai Zi + c2 bi Zi (univariate normal distribution)
i=1 i=1
n n
!
X X
=⇒ ai Zi , bi Z i ∼ N2
i=1 i=1
n
! n
! n
X X X
E ai Zi = 0, Var ai Zi = a2i ,
i=1 i=1 i=1
n
! n
! n
X X X
E bi Zi = 0, Var bi Z i = b2i ,
i=1 i=1 i=1
n n
! n
X X X
Cov ai Zi , bi Z i = ai bi = 0,
i=1 i=1 i=1
n n
! n n
!
X X X X
=⇒ ai Zi , bi Zi ∼ N2 0, 0, a2i , b2i , 0
i=1 i=1 i=1 i=1
n n
! n n
!
X X X X
=⇒ ai Zi ∼ N 0, a2i and bi Zi ∼ N 0, b2i are independent
i=1 i=1 i=1 i=1
Pn Pn
i=1 ai Zi i=1 bi Zi
=⇒ pP n 2
∼ N (0, 1) and pP n 2
∼ N (0, 1) are independent.
i=1 ai i=1 bi

(a)
Pn Pn 2
i=1 ai Zi ( i=1 bi Zi )
pP n 2
∼ N (0, 1) and P n 2 ∼ χ21 are independent
i=1 ia i=1 ib
Pn pPn
2
ai Zi / i=1 ai
=⇒ i=1
r P ∼ t1 , that is, Y1 ∼ t1 .
n 2
( P i=1 bi Zi )
n
i=1 b2i

d
(b) Since t21 = F1,1 , the result follows on using (a).

Z1
(c) FY3 (y) = P (Y3 ≤ y) = P Z 2
≤ y , y ∈ R (Why?). Clearly,

Z1 Z1
FY3 (y) = P ≤ y, Z2 > 0 + P − ≤ y, Z2 < 0
|Z2 | |Z2 |

Z1 Z1 d
=P ≤ y, Z2 > 0 + P ≤ y, Z2 < 0 ((Z1 , Z2 ) = (−Z1 , Z2 ))
|Z2 | |Z2 |

Z1 d Z1
=P ≤ y , ∀ y ∈ R =⇒ Y3 = ∼ t1 , (by (a)) =⇒ Y3 ∼ t1 .
|Z2 | |Z2 |

Common questions

The joint moment generating function (m.g.f.) of a random vector X = (X1, X2, ..., Xp) is defined over a set A where each element t in A ensures the expectation E(exp(Σ(ti*Xi))) is finite. The function MX on this set is described as the expectation of the exponential sum, and it can be used to derive the independence of the variables when the set A allows factorization into individual m.g.f.s .

For a continuous random variable with a mixed p.d.f. of the form f(x) = x² for 0 < x < 1 and f(x) = ce^{-x} for x ≥ 1, c is calculated such that the integral over the entire domain equals 1. Calculations for the probability over specific intervals are done by integrating the p.d.f. over those intervals, determining the support, and deriving the distribution function by integrating the p.d.f. .

The independence of random variables X1, X2, ..., Xp is implied if their joint moment generating function MX(t) equals the product of the individual moment generating functions MXi(ti) for all t in the domain A. Conversely, if the joint m.g.f. can be expressed as such a product over A, then the random variables are independent .

For a continuous random vector X = (X1, X2), the conditional density function fX2|X1(x|x1) is given by the ratio of the joint p.d.f. fX1,X2(x1, x2) to the marginal p.d.f. fX1(x1), provided fX1(x1) > 0. This function describes the probability density of X2 given that X1 is equal to some value x1 .

The support set of a function g, under the conditions that g(x) ≥ 0 for all x and the integral of g over its domain equals 1, determines whether g is a p.d.f. of some continuous random variable. Specifically, the support set T is defined as those values x in the real numbers for which the integral of g from x-h to x+h is greater than 0 for all h > 0 .

The p.d.f. determines the distribution function (d.f.) uniquely, but the converse is not true. The d.f. determines the p.d.f. almost uniquely, as they may vary on sets with no length or zero content. Therefore, it is sufficient to study the p.d.f. of a continuous random variable .

For a discrete random variable X with support S, the distribution of a function Y = h(X) is determined by finding the support set T = {h(x) : x ∈ S} and defining the probability mass function of Y as g(z) = Σ P(X = x) for x ∈ h^{-1}({z}), where z ∈ T. This crafts the distribution of Y as a discrete random variable .

The document defines random variables that are neither discrete nor continuous as those whose distribution functions have discontinuity points that form a non-empty set. An example given is a random variable with discontinuity points at specific locations, indicating that it is not a purely continuous random variable nor a purely discrete one .

To find the marginal p.d.f. of a component of a continuous random vector, one integrates the joint p.d.f. over the other variables. For example, the marginal p.d.f. fX1 of X1 is obtained by integrating the joint p.d.f. fX(x1, x2, ..., xp) over all other variables x2, ..., xp .

The document explains that the sample variance, calculated as S² = 1/(n-1)∑(Xi - X̄)², is used because it has an expectation equal to the population variance σ², unlike the biased estimator S1² = 1/n∑(Xi - X̄)². The unbiased estimator S² compensates to correctly estimate the variance .

Probability and Statistics Lecture Notes
No ratings yet
Probability and Statistics Lecture Notes
54 pages
Probability and Statistics Lecture Notes
No ratings yet
Probability and Statistics Lecture Notes
119 pages
Introduction to Probability and Statistics
No ratings yet
Introduction to Probability and Statistics
7 pages
Set Theory and Probability Basics
No ratings yet
Set Theory and Probability Basics
37 pages
Probability Spaces and Basics Explained
No ratings yet
Probability Spaces and Basics Explained
33 pages
Chap1 Mazumdar
No ratings yet
Chap1 Mazumdar
38 pages
Average Rate of Change in Functions
No ratings yet
Average Rate of Change in Functions
86 pages
Probability Concepts and Definitions
No ratings yet
Probability Concepts and Definitions
14 pages
Probability and Stochastic Processes Guide
No ratings yet
Probability and Stochastic Processes Guide
60 pages
Understanding Probability Axioms and Events
No ratings yet
Understanding Probability Axioms and Events
10 pages
Introduction to Probability Concepts
No ratings yet
Introduction to Probability Concepts
10 pages
Understanding Probability Concepts
No ratings yet
Understanding Probability Concepts
298 pages
MScFE 620 DTSP - Compiled - Video - Transcripts - M1 PDF
No ratings yet
MScFE 620 DTSP - Compiled - Video - Transcripts - M1 PDF
14 pages
Probability Spaces
No ratings yet
Probability Spaces
13 pages
Understanding Sample Space in Probability
No ratings yet
Understanding Sample Space in Probability
4 pages
MScFE 620 DTSP - Video - Transcripts - M1
No ratings yet
MScFE 620 DTSP - Video - Transcripts - M1
14 pages
Probability Theory Basics Explained
No ratings yet
Probability Theory Basics Explained
38 pages
Probability Review for Electrical Engineering
No ratings yet
Probability Review for Electrical Engineering
38 pages
Axiomatic Probability Overview
No ratings yet
Axiomatic Probability Overview
17 pages
Probability Intro 28072024
No ratings yet
Probability Intro 28072024
32 pages
Understanding Outcomes in Probability
No ratings yet
Understanding Outcomes in Probability
4 pages
Lecture Notes From WhatsApp
No ratings yet
Lecture Notes From WhatsApp
93 pages
Understanding Probability Concepts
No ratings yet
Understanding Probability Concepts
33 pages
Probability
No ratings yet
Probability
18 pages
Kolmogorov's Axioms of Probability
100% (1)
Kolmogorov's Axioms of Probability
52 pages
Foundations of Probability Theory
No ratings yet
Foundations of Probability Theory
62 pages
Chapter I Ad 2
No ratings yet
Chapter I Ad 2
43 pages
Introduction to Probability Theory
No ratings yet
Introduction to Probability Theory
14 pages
Probability Theory: Axioms and Theorems
No ratings yet
Probability Theory: Axioms and Theorems
34 pages
Introduction to Probability Concepts
No ratings yet
Introduction to Probability Concepts
5 pages
Understanding Probability Concepts
No ratings yet
Understanding Probability Concepts
22 pages
Probability Concepts and Distributions
No ratings yet
Probability Concepts and Distributions
33 pages
Probability and Statistics Study Guide
No ratings yet
Probability and Statistics Study Guide
266 pages
Introduction to Probability Theory
No ratings yet
Introduction to Probability Theory
24 pages
Understanding Probability Concepts
No ratings yet
Understanding Probability Concepts
238 pages
Understanding Probability Concepts
No ratings yet
Understanding Probability Concepts
109 pages
Lec 1
No ratings yet
Lec 1
21 pages
Probability and Statistics Syllabus
100% (2)
Probability and Statistics Syllabus
174 pages
Basic Concepts of Probability Theory
No ratings yet
Basic Concepts of Probability Theory
21 pages
Understanding Probability Concepts
No ratings yet
Understanding Probability Concepts
18 pages
Introduction to Probability Theory
No ratings yet
Introduction to Probability Theory
7 pages
DE ZG535 (23-S2) - Sessions 10-12 (23 Mar, 6 & 13 Apr 2024)
No ratings yet
DE ZG535 (23-S2) - Sessions 10-12 (23 Mar, 6 & 13 Apr 2024)
11 pages
Lecture 5
No ratings yet
Lecture 5
6 pages
Understanding Random Experiments in Probability
No ratings yet
Understanding Random Experiments in Probability
14 pages
Understanding Sample Space and Events
No ratings yet
Understanding Sample Space and Events
8 pages
Introduction to Probability Concepts
No ratings yet
Introduction to Probability Concepts
24 pages
Introduction to Probability Concepts
No ratings yet
Introduction to Probability Concepts
31 pages
Understanding Probability Concepts
No ratings yet
Understanding Probability Concepts
57 pages
Probability I: Axioms and Concepts
No ratings yet
Probability I: Axioms and Concepts
6 pages
PTSP - Unit - I Part A
No ratings yet
PTSP - Unit - I Part A
8 pages
Basic Probability Concepts Overview
No ratings yet
Basic Probability Concepts Overview
12 pages
Probability Theory Fundamentals
No ratings yet
Probability Theory Fundamentals
60 pages
Understanding Stochastic Experiments
No ratings yet
Understanding Stochastic Experiments
32 pages
Introduction to Probability Concepts
100% (1)
Introduction to Probability Concepts
14 pages
Probability
No ratings yet
Probability
3 pages
Probability Theory and Stochastic Processes Notes
No ratings yet
Probability Theory and Stochastic Processes Notes
81 pages
Axioms of Probability Explained
No ratings yet
Axioms of Probability Explained
21 pages
Cha1hac5 PDF
No ratings yet
Cha1hac5 PDF
34 pages
Understanding Polarization of Light
No ratings yet
Understanding Polarization of Light
22 pages
Yosys: Open-Source RTL Synthesis Guide
No ratings yet
Yosys: Open-Source RTL Synthesis Guide
13 pages
CMOS Fundamentals and Logic Gates Guide
No ratings yet
CMOS Fundamentals and Logic Gates Guide
26 pages
Ministry of Education Contact Directory
No ratings yet
Ministry of Education Contact Directory
12 pages
Probability Problem Set Analysis
No ratings yet
Probability Problem Set Analysis
3 pages
Independence of Computer Reliability
No ratings yet
Independence of Computer Reliability
99 pages
Step-by-Step Risk Assessment Guide
No ratings yet
Step-by-Step Risk Assessment Guide
35 pages
Applied Probability & Random Process - Study List
No ratings yet
Applied Probability & Random Process - Study List
5 pages
Mathematical Foundations of Big Data
No ratings yet
Mathematical Foundations of Big Data
21 pages
Binomial & Normal Distribution Guide
No ratings yet
Binomial & Normal Distribution Guide
26 pages
Decision Theory Under Uncertainty
No ratings yet
Decision Theory Under Uncertainty
8 pages
Grade 11 Statistics BOW Schedule
No ratings yet
Grade 11 Statistics BOW Schedule
3 pages
Probability Questions and Solutions
No ratings yet
Probability Questions and Solutions
4 pages
Understanding Binomial Probability Distribution
No ratings yet
Understanding Binomial Probability Distribution
11 pages
Basic Probability Concepts and Examples
No ratings yet
Basic Probability Concepts and Examples
9 pages
Probability Distribution in Statistics
No ratings yet
Probability Distribution in Statistics
4 pages
Convert Qualitative to Quantitative Risk
No ratings yet
Convert Qualitative to Quantitative Risk
22 pages
Understanding Normal Distribution Basics
No ratings yet
Understanding Normal Distribution Basics
17 pages
Exploring Quantum Physics Concepts
No ratings yet
Exploring Quantum Physics Concepts
43 pages
Transformations of Random Variables
No ratings yet
Transformations of Random Variables
6 pages
Memo: A Probabilistic Language for Theory of Mind
No ratings yet
Memo: A Probabilistic Language for Theory of Mind
26 pages
Naive Bayes Algorithms
No ratings yet
Naive Bayes Algorithms
16 pages
Math 10 - Q3 M14
No ratings yet
Math 10 - Q3 M14
13 pages
Probability and Stochastic Processes Guide
No ratings yet
Probability and Stochastic Processes Guide
699 pages
Uncertainty Problems With Solutions
No ratings yet
Uncertainty Problems With Solutions
2 pages
Understanding Statistical Distribution
No ratings yet
Understanding Statistical Distribution
4 pages
Monte Carlo Simulation for Policy Analysis
No ratings yet
Monte Carlo Simulation for Policy Analysis
8 pages
Mathematical Expectation Explained
No ratings yet
Mathematical Expectation Explained
3 pages
Mean, Variance, and Standard Deviation of Discrete Random Variables
No ratings yet
Mean, Variance, and Standard Deviation of Discrete Random Variables
15 pages
Discrete Probability Distributions Exam
No ratings yet
Discrete Probability Distributions Exam
7 pages
Probability Concepts in Management Statistics
No ratings yet
Probability Concepts in Management Statistics
21 pages
Basic Probability Theory Overview
No ratings yet
Basic Probability Theory Overview
30 pages
Risk Management in Project Teams
No ratings yet
Risk Management in Project Teams
6 pages