STAT 2001 Tutorial 2
STAT 2001 Basic Concepts in Statistics and
Probability I
Tutorial 2
September 23 – 27, 2024
In this tutorial note, we will cover the following topics
• Independent Events
• Law of Total Probability and Bayes’ Theorem
• Discrete random variables and Probability Mass Function(p.m.f.)
• Functions of Discrete Random Variable
• Cumulative Distribution Function(c.d.f.)
• Mathematical Expectation
1 Independent Events
Definition 1 (Independent).
Two events, say A and B, are said to be independent if
P (A | B) = P (A), provided P (B) > 0.
A more general definition : A and B are independent if and only if
P (A ∩ B) = P (A)P (B).
Remark 2. Independence vs. Mutually Exclusive: A pair of events A and B cannot
be simultaneously mutually exclusive and independent.
Remark 3. Pairwise Independence ̸⇒ Independence
Exercise 4. Suppose a fair coin is tossed twice. Let A = {The first toss is a head}; B =
{The second toss is a head}; C = {The first toss has the same result with the second toss}.
Show that A, B, and C are pairwise independent but not independent.
1
STAT 2001 Tutorial 2
2 Law of Total Probability and Bayes’ Theorem
2.1 Law of Total Probability
Let B1 , ..., Bn be disjoint events that form a partition of the sample space S and assume that
P (Bi ) > 0, for all i. And
B1 ∪ B2 ∪ ... ∪ Bn = S
For any event A, we have
P (A) = P (S ∩ A)
= P ((B1 ∪ B2 ∪ ... ∪ Bn ) ∩ A)
= P ((B1 ∩ A) ∪ ... ∪ (Bn ∩ A))
= P (B1 ∩ A) + ... + P (Bn ∩ A)
= P (B1 )P (A|B1 ) + ... + P (Bn )P (A|Bn )
2.2 Bayes’ Theorem
If P (A) > 0, then
P (A ∩ Bk ) P (Bk )P (A|Bk )
P (Bk |A) = = Pn .
P (A) i=1 P (Bi )P (A|Bi )
Remark 5. Bayes’ Theorem is a common and useful technique for inference. Without any
ancillary information, the (prior) probability that event Bk occurs is P (Bk ). Now sup-
pose we observe event A happens, and know events B1 , ..., Bn have certain association re-
lationship with event A. We represent this knowledge in terms of conditional probability
2
STAT 2001 Tutorial 2
P (A|B1 ), ..., P (A|Bn ). By using Bayes’ Theorem, the (posterior) probability that Bk occurs
is updated to be P (Bk |A). Briefly speaking, Bayes’ Theorem provides us a framework to
unify the conditional information together to draw statistical inference.
Exercise 6. Suppose that 5% of men and 0.25% of women are color-blind.A person is
chosen at random and that person is color-blind. What is the probability that the person is
male?(Assume males and females to be in equal numbers.)
Exercise 7. A computer can send any one of three characters A, B, C to a printer. During
each transmission, each character has equal chance to be sent. However, the printer may not
print the same character as transmitted for transmission errors. The probability that an A
will be incorrectly printed as B is 0.2, and that of C to be printed as A is 0.5. Other printer
errors are impossible.
1. Suppose two characters are transmitted independently. What is the probability that
two A’s will be printed?
2. Suppose that a single character is transmitted. If B is printed, what is the probability
that a B was transmitted?
3
STAT 2001 Tutorial 2
3 Discrete random variables and Probability Mass Func-
tion(p.m.f.)
Definition 8. (Random Variable)
A random variable(r.v.) is a function X that maps each element s ∈ S to a real
number x, that is X(s) = x.
Discrete random variable:If the set {x : X(s) = x, s ∈ S} has finite or countably
infinite real numbers.
Remark: A random variable X is a function from sample space S to R. X itself is not
an event but "X = some real number" is.
Definition 9. (Probability Mass Function)
The Probability Mass Function (p.m.f.) of a discrete random variable X is a
function f (x) satisfies
1. f (x) > 0, ∀x ∈ S;
P
2. x∈S f (x) = 1.
3. p(x ∈ A) = f (x), ∀A ⊂ S;
P
x∈A
4
STAT 2001 Tutorial 2
Exercise 10. For each of the following discrete random variable X, determine the constant
c such that f (x) is the p.m.f. of X.
1. f (x) = xc , x = 1, 2, ..., n.
x
1
2. f (x) = c 4
, x = 1, 2, 3, ....
4 Functions of Discrete Random Variable
For any real function g, let Y = g(X), Y also a discrete random variable. Moreover, the
p.m.f. of Y is completely determined by the p.m.f. of X.
Remark 11. Notation: Y (s) = g ◦ X(s) = g(X(s)). g ◦ X(s) is a real function of s ∈ Ω.
5 Cumulative Distribution Function(c.d.f.)
Definition 12. (Cumulative Distribution Function)
The cumulative distribution function of a real-valued random variable X is the function
given by
FX (x) = P (X ≤ x),
where the right-hand side represents the probability that the random variable X takes
on a value less than or equal to x.
5
STAT 2001 Tutorial 2
Remark 13. We use FX (x) to emphasize the dependence on the random variable X. You
can simply write F (x) when there is no ambiguity.
Remark 14. Both p.m.f and c.d.f are two equivalent ways to describe a discrete random
variable for we could derive the p.m.f. of a r.v. from its c.d.f. and vice versa.
Exercise 15. Roll a fair dice twice independently. Let X = 0 if the two numbers we get
are the same, X = −1 if the first number is smaller than the second number, and X = 1
otherwise.
1. Find both the p.m.f and c.d.f of X and draw graphs for them.
2. Set Y = |X|, find the p.m.f of Y and give an interpretation of Y being a random
variable.
6 Mathematical Expectation
The Expected value or mean of a random variable is a measure of the "middle part" of the
random variable’s distribution. For a discrete random variable X with p.m.f. f (x), the
expected value of X is defined as:
X X
E(X) = xf (x) = xP (X = x).
x∈S x∈S
6
STAT 2001 Tutorial 2
provided that the sum exists.
If u(X) is a function of X, the mathematical expectation of u(X) is
X X
E[u(X)] = u(x)f (x) = u(x)P (X = x).
x∈S x∈S
6.1 Properties of Expectation
1. If c is a constant, then E(c) = c.
2. If c1 and c2 are two constants, X and Y are two random variables, then
E[c1 X + c2 Y ] = c1 EX + c2 EY.
3. If X and Y are independent, then E(XY ) = E(X)E(Y ).
Exercise 16. Suppose the largest prize of Mark Six lottery is 1500 HKD, and each bet costs
10 HKD. How small the probability is when the expectation income is below zero(which
means we should not buy it).