Chapter 2: A review on

probability theory
Instructor: Truong Buu Chau
Email: truongbuuchau@[Link]

Faculty of Mathematics and Statistics

August 18, 2025

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 1 / 18

Contents

1 Probability
2 Joint Probability and Independent Events
3 Conditional probability
4 Bayes’theorem
5 Random variables
6 Discrete random variables
7 Continuous random variables
8 Joint discrete and continous random variables

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 2 / 18

1. Probability

The probability assignment for a random experiment is

an assignment of probabilities to all possible events the
experiment generates.
These probabilities are real numbers between 0 and 1.
The assignment of probabilities to events must satisfy
the following axioms.
1 P(A) ≥ 0 for any event A. (Probabilities are nonnegative.)
2 P(U) = 1. (Probability of universe = 1. Some outcome occurs every
time you conduct the experiment.)
3 If A and B are mutually exclusive events, then
P(A ∪ B) = P(A) + P(B). (Probability is additive over disjoint events.)

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 3 / 18

1. Probability

The other rules of probability:

1 P (ϕ) = 0.
(The empty set has zero probability.)
2 P (Ã) = 1 − P (A).
(The probability of a complement of an event.)
3 P (A ∪ B ) = P (A) + P (B ) − P (A B ).
(The addition rule of probability.)

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 4 / 18

2. Joint Probability and Independent Events

The joint probability of events A and B is P (A ∩ B ),

the probability of their intersection.
If event A and event B are independent, then
P (A ∩ B ) = P (A) × P (B ).
The joint probability is the product of the individual
probabilities.
Marginal probability: The probability of one of the
events A, in the joint event setting is called its marginal
probability.
P (A) = P (A ∩ B ) + P (A ∩ B̃ )

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 5 / 18

3. Conditional probability

The conditional probability of event B given event A:

P (A ∩ B )
P (B |A) =
P (A)
Multiplication rule: P (A ∩ B ) = P (B ) × P (A|B )
The conditional probability for independent events.
When A and B are independent events we have:
P (B |A) = P (B )
Note: Distinction between independent events and mutually
exclusive events.

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 6 / 18

4. Bayes’theorem

Definition
A collection of sets {B1 , . . . , Bn } is a partition of another
set U if:
The union B1 ∪ B2 ∪ · · · ∪ Bn = U (the union of the
sets is U)
Every distinct pair of the events are disjoint,
Bi ∩ Bj = ϕ for i = 1, . . . , n; j = 1, . . . , n and i ̸= j.
Suppose {B1 , . . . , Bn } is a partition of U, P (U ) = 1, and A
is some specific event.
The law of total probability: P (A) = nj=1 P (A ∩ Bj ).
P
(The probability of an event A is the sum of the
probabilities of its disjoint parts).
Truong Buu Chau C03031 - Chapter 2 August 18, 2025 7 / 18
4. Bayes’theorem
Using the multiplication rule on each joint probability
gives:
Xn
P (A) = P ( A| Bj ) P (Bj ).
j =1

Bayes’ rule:
P (A ∩ Bi ) P ( A| Bi )P (Bi )
P ( Bi | A) = = P
n .
P (A)
P ( A| Bj )P (Bj )
j =1

P (Bi ) is called the prior probability of event Bi

P (Bi |A) is called the posterior probability of event Bi .
P ( A| Bi ) is called the likelihood function of event Bi .
Truong Buu Chau C03031 - Chapter 2 August 18, 2025 8 / 18
4. Bayes’theorem

Bayes’ theorem in its proportional form as:

posterior ∝ prior × likelihood
The use of Bayes’ theorem for events by the following three
steps:
1 Multiply prior times likelihood for each of the Bi . This
finds the probability of Bi ∩ A by the multiplication rule.
2 Sum them for i = 1, . . . , n. This finds the probability of
A by the law of total probability.
3 Divide each of the prior times likelihood values by their
sum. This finds the conditional probability of that
particular Bi given A.

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 9 / 18

5. Random variables

A number that is determined by the outcome of a

random experiment is called a random variable.
Random variables are denoted by uppercase letters,
e.g.,Y.
The value the random variable takes is denoted by
lowercase letters, e.g.,y.

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 10 / 18

6. Discrete random variables

A discrete random variable, Y, can only take on the

distinct values yk . There can be a finite possible number
of values or there can be a countably infinite number of
possible values.
The probability function:
f (yk ) = P (Y = yk )
Some discrete distributions:
▶ Binomial distribution
▶ Poisson distribution
▶ Hypergeometric distribution

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 11 / 18

7. Continuous random variables
When we have a continuous random variable, we believe
all values over some range are possible if our
measurement device is sufficiently accurate.
It impossible to find the probability function of a
continuous random variable the same way we did for a
discrete random variable.
Probability density function.
Some continuous distributions:
▶ Uniform distribution
▶ Exponential distribution
▶ Beta distribution
▶ Gamma distribution
▶ Normal distribution
Truong Buu Chau C03031 - Chapter 2 August 18, 2025 12 / 18
8. Joint continous and discrete random variables
Joint discrete distributions.
The two numbers are called joint random variables and
denoted X , Y . If both the random variables are discrete,
they each have separated possible values xi for i = 1 . . . , I
and yj for j = 1, . . . , J. The universe for the experiment is
the set of all possible outcomes of the experiment which are
all possible ordered pairs of possible values. The universe of
the joint experiment is shown in table

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 13 / 18

8. Joint discrete and continous random variables
Joint discrete distributions.
The joint probability function of two discrete joint random
variables is defined at each point in the universe:
f (xi , yj ) = P (X = xi , Y = yj ),
i = 1 . . . , I ; j = 1, . . . , J .
Marginal probability of Y can be computed from the joint
density
P (Y = yj ) = P (∪i (X = xi , Y = yj ))
X
= P (X = xi , Y = yj ).
i

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 14 / 18

8. Joint discrete and continous random variables
Joint discrete distributions.
Putting this relationship in terms of the probability
function, we get
X
f (yj ) = f (xi , yj )
i

The joint probability distribution and the marginal

probability distributions are shown in table

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 15 / 18

8. Joint discrete and continous random variables

Joint discrete distributions.

The conditional probability that random variable X = xi ,
given Y = yj is given by
P (X = xi , Y = yj )
f (xi |yj ) = P (X = xi |Y = yj ) =
P (Y = yj )
f (xi , yj )
f (xi |yj ) =
f (yj )

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 16 / 18

8. Joint discrete and continous random variables
Joint continous distributions.
If both X and Y are continuous random variables, they have
joint density f (x , y ), which measures the probability density
at the point (x , y ). This would be found by dividing the
plane into rectangular regions by partitioning both the x
axis and y axis.

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 17 / 18

8. Joint discrete and continous random variables

Joint continous distributions.

The marginal density of Y is found by integrating the joint
density over the whole range of X:
Z ∞
f (y ) = f (x , y )dx
−∞

The conditional density of X given Y = y is given by

f (x , y )
f (x |y ) =
f (y )

Truong Buu Chau C03031 - Chapter 2 August 18, 2025 18 / 18