Complete Notes on Probability Theory (STA 211)

200 LEVEL FIRST SEMESTER

November 4, 2025

Contents

I PROBABILITY I: Foundations 3
1 PROBABILITY II (STA 211) - Course Outline 3

2 Combinatorial Analysis 3
2.1 Introduction to Combinatorics . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Counting Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Partition of a Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4 Combinatorial Identities . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.5 Binomial Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.6 Stirling’s Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Part 2: Foundations of Probability 9

3.1 Basic Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Approaches to Probability . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Axioms of Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.4 Basic Probability Theorems . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 Part 3: Event Operations and Conditional Probability 11

4.1 Types of Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 Laws of Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.3 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.4 Total Probability Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.5 Bayes’ Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

II PROBABILITY II: Random Variables 15

5 Part 4: Random Variables 15
5.1 Definition and Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.2 Discrete R.V. and Probability Mass Functions (PMF) . . . . . . . . . . . 15
5.3 Continuous R.V. and Probability Density Functions (PDF) . . . . . . . . 15
5.4 Cumulative Distribution Function (CDF) . . . . . . . . . . . . . . . . . . 16

1
6 Part 5: Expectation and Variance 18
6.1 Expected Value (Mean) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6.2 Properties of Expected Value . . . . . . . . . . . . . . . . . . . . . . . . 18
6.3 Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6.4 Properties of Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2
Part I
PROBABILITY I: Foundations
1 PROBABILITY II (STA 211) - Course Outline
1. Combinatorial Analysis
2. Probability models for the study of random phenomena in finite sample spaces.
3. Probability distributions of discrete and continuous random variables.
4. Expectations and moments, generating functions.
5. Chebyshev’s Inequality, Bivariate, Marginal distributions and moments.
6. Convolution of two distributions, the Central Limit Theorem and its uses.

2 Combinatorial Analysis
2.1 Introduction to Combinatorics
Combinatorial Analysis is a branch of mathematics concerned with permutations or com-
binations. It is the theory of enumerating, or to solve problems about the probability
of constructing arrangements of objects which satisfy specified conditions, especially in
statistics and probability.
Combinatorics is a branch of mathematics concerning the study of finite or countably
discrete structures.

2.2 Counting Techniques

Factorial Number
The symbol n! (read as ”n factorial”) is defined as the product of all positive integers
from 1 to n, inclusive. Mathematically,

n! = n(n − 1)(n − 2) . . . 3 · 2 · 1

Note: 1! = 1 and by convention, 0! = 1.

Factorial Examples
1. Evaluate 5!

Solution
5! = 5 × 4 × 3 × 2 × 1 = 120
(n+2)!
2. Show that: n!
= n2 + 3n + 2

3
 Solution
(n + 2)! (n + 2)(n + 1)n!
=
n! n!
= (n + 2)(n + 1) = n + n + 2n + 2 = n2 + 3n + 2
2

n!
3. Simplify (n−2)!

Solution
n! n(n − 1)(n − 2)!
= = n(n − 1) = n2 − n
(n − 2)! (n − 2)!

Permutation
A permutation is the number of different ways you can arrange a set of objects, taking
order into account. The number of ways of arranging n objects, taking r at a time, is
given by the formula:
n n!
Pr =
(n − r)!

Permutation Examples
1. Evaluate 7 P4

Solution
7 7! 7! 7 × 6 × 5 × 4 × 3!
P4 = = = = 840
(7 − 4)! 3! 3!

2. If A = {a, b, c}, in how many ways can the elements be arranged in pairs?

Solution
Here n = 3 and r = 2.

3 3! 3! 3×2×1
P2 = = = =6
(3 − 2)! 1! 1
The paired arrangements are: ab, ac, ba, bc, ca, cb.
n+1
3. For what positive value of n is P2 = 6?

Solution
n+1 (n + 1)! (n + 1)!
P2 = = = (n + 1)n
(n + 1 − 2)! (n − 1)!
Given (n + 1)n = 6 =⇒ n2 + n − 6 = 0 =⇒ (n + 3)(n − 2) = 0. The positive
value of n is 2.

4
Combination
Combination means selection of objects without regard to order. The number of ways of
choosing r objects from a set of n objects is:
 
n n n!
Cr or =
r (n − r)!r!

Combination Examples
1. Evaluate 4 C3 and 7 C4 .

Solution
 
4 4 4! 4!
C3 = = = =4
3 (4 − 3)!3! 1!3!
 
7 7 7! 7!
C4 = = = = 35
4 (7 − 4)!4! 3!4!

2. Given that 9 Cr+1 = 9 Cr , find r.

Solution
9 9



If r+1
= r
, then either r + 1 = r (impossible) or r + 1 = 9 − r.

2r = 8 =⇒ r = 4
nP
3. If nC
2
2
= 6, find n.

Solution
Note: The original notes contain contradictory working. The problem as stated,
n!/(n−2)!
n!/((n−2)!2!)
= 6 =⇒ 2! = 6, has no solution. The notes then appear to solve a
different problem, which is transcribed below.
(n−2)!
Solving the problem (n−3)!
= 3:

(n − 2)! (n − 2)(n − 3)!

= =3
(n − 3)! (n − 3)!
n − 2 = 3 =⇒ n = 5

4. Committee Problem 1: In how many ways can a committee of 3 boys and 2 girls
be selected from 6 boys and 4 girls?

5
 Solution
Number of ways = (Ways to choose 3 boys from 6) × (Ways to choose 2 girls from
4)    
6 4
Total = × = 20 × 6 = 120 ways
3 2
5. Committee Problem 2: A committee of 5 men and 4 women is to be formed
from 9 men and 8 women.
(a) No restrictions: Total = 95 × 84 = 126 × 70 = 8820 ways.

(b) If it must contain 2 particular men: We must choose (5 − 2) = 3 more

(9
− 2) = 7 men. We must still choose 4 women from
men from the remaining
8 women. Total = 73 × 84 = 35 × 70 = 2450 ways.
6. Committee Problem 3: A committee of 3 is to be formed from 5 men and 3
women.
(i) With 2 men and 1 woman: Total = 52 × 31 = 10 × 3 = 30 ways.

(ii) With 1 man and 2 women, if a
particular woman must be a member:

We must choose 1 man from 5: 51 . We must
choose (2 − 1)
= 12
more woman
2 5
from the remaining (3 − 1) = 2 women: 1 . Total = 1 × 1 = 5 × 2 =
10 ways.
(iii) If there is no restriction:

8×7×6 Choose any 3 people from the total of 5 + 3 = 8
8
people. Total = 3 = 3×2×1 = 56 ways.

2.3 Partition of a Set

The total number of ways a sample space (S) of size N can be partitioned into k subsets
of sizes K1 , K2 , . . . , Kk is given by:
 
N N!
=
K1 , K2 , . . . , Kk K1 !K2 ! . . . Kk !

where N = ki=1 Ki .
P

Example: Committees
In how many ways can a group of 12 students be put into 3 respective committees such
that 4, 3, 5 students belong to the respective committees? Solution: This is the partition
of a set of 12 (N = 12) students into subsets of sizes 3 (K1 = 3), 4 (K2 = 4), and 5
(K3 = 5).
12!
Required number of ways = = 27, 720
3!4!5!

2.4 Combinatorial Identities

(a) Symmetry Identity: Prove nr = n−rn



Proof:
   
n n! n! n
R.H.S = = = = (L.H.S)
n−r (n − (n − r))!(n − r)! r!(n − r)! r

6
 n n−1 n−1




(b) Pascal’s Identity: Prove r
= r−1
+ r
Proof:
(n − 1)! (n − 1)!
R.H.S = +
(n − r)!(r − 1)! (n − r − 1)!r!
Find common denominator r!(n − r)!:
r · (n − 1)! (n − r) · (n − 1)!
= +
r!(n − r)! r!(n − r)!
 
(n − 1)!(r + n − r) n(n − 1)! n! n
= = = = (L.H.S)
r!(n − r)! r!(n − r)! r!(n − r)! r
(c) Prove: nr = nr n−1



r−1
Proof:
n (n − 1)! n (n − 1)!
R.H.S = · = ·
r ((n − 1) − (r − 1))!(r − 1)! r (n − r)!(r − 1)!
 
n · (n − 1)! n! n
= = = (L.H.S)
r · (r − 1)!(n − r)! r!(n − r)! r
(d) Prove: nr + 2 r−1
n n
= n+2





+ r−2 r
Proof:
       
n n n n
L.H.S = + + +
r r−1 r−1 r−2
Apply Pascal’s Identity to each pair:
   
n+1 n+1
= +
r r−1
Apply Pascal’s Identity again:
   
(n + 1) + 1 n+2
= = (R.H.S)
r r

2.5 Binomial Theorem

n


The binomial coefficient, r
, is used in the binomial theorem:
n  
n
X n n−r r
(x + y) = x y
r=0
r

Examples
Pn n
n n
Pn n
n−r r n
(a) Prove:
Pn n
r=0 r = 2 Let x = 1, y = 1: (1 + 1) = r=0 r (1) (1) =⇒ 2 =
r=0 r .
Pn r n

n
Pn n
n−r
(b) Prove:
Pn r=0 (−1) r
= 0 Let x = 1, y = −1: (1−1) = r=0 r (1) (−1)r =⇒
0 = r=0 (−1)r nr .

(c) Prove: nr=0 P n

(a−1)
r = an Let x = 1, y = a−1: (1+(a−1))n = nr=0 nr (1)n−r (a−
P
P

r
1)r =⇒ an = nr=0 nr (a − 1)r .
Pn n
n
Pn n
n

Pn n
Pn n

(d) Prove: r=1Pr = 2 − 1 From (a), r=0 r
= 0
+ r=1 r
= 1 + r=1 r .
n n n

Pn n
n
So, 2 = 1 + r=1 r =⇒ r=1 r = 2 − 1.

7
2.6 Stirling’s Approximation
When n is very large, evaluating n! is difficult. Stirling’s formula provides an approxima-
tion: √  n n
n! ≈ 2πn
e
Or equivalently: √
n! ≈ 2π · nn+1/2 · e−n

Example
Find the Stirling’s approximation of 8!. Solution:
√
8! ≈ 2π · 88.5 · e−8
p
8! ≈ 2(3.142) · (47745313.28) · (0.00033546)
8! ≈ (2.50679) · (47745313.28) · (0.00033546) ≈ 39902.39
(The exact value is 8! = 40320)

8
3 Part 2: Foundations of Probability
3.1 Basic Terminology
ˆ Probability: Probability is the mathematical term which shows all the degree of
belief which we have about something. It is the measure of the likelihood that an
event would occur.
ˆ Experiment: Any operation or process that is followed by a result. This means
performing an act: for example tossing a coin, throwing a die, dipping your hand
into a bowl, centering balls to pick one etc. It is the process which leads to the
outcome of one (and only one) of several possible observations. It is also seen as
a painstaking test using scientific methods to discover how something in someone
reacts under certain conditions.
ˆ Trial: A trial is just one act performed.
ˆ Outcome: An outcome is one of the possible results that can happen in a trial
of an experiment. However it is a particular result of an experiment.
ˆ Statistical Experiment: It refers to any process that yields a collection of
outcomes.
ˆ Random Experiment: It is an experiment whose outcome may not be the same
even though the conditions of the experiment may be fair. E.g., Tossing a coin,
throwing a fair die.
ˆ Sample Space (U or S): It is the set of all possible outcomes of a sample exper-
iment.
ˆ Finite Sample Space: Contains a definite (countable) number of points. E.g.,
S = {1, 2, 3, 4, 5, 6}.
ˆ Continuous Sample Space: Contains uncountably infinite points. E.g., Human
heights.
ˆ Event: It is a collection of sample points; a subset of the sample space.
ˆ Simple Event (Elementary Event): An event consisting of one sample point.
ˆ Composite Event: An event consisting of two or more sample points.
ˆ Sure Event: An event that contains all elements of the sample space (S).
ˆ Impossible Event (∅): An event that contains no elements of the sample space.

3.2 Approaches to Probability

1. Classical (A Priori) Approach If an event A can occur in k ways out of a total
of N equally likely outcomes:
No. of favorable outcomes k
P (A) = =
Total no. of outcomes N
2. Subjective Approach A probability based on past experience, available opinions,
or human judgement/intuition.

9
3.3 Axioms of Probability
Let P (·) be a real valued function defined on every event A ⊂ U . Then P (A) is the
probability function of event A if:

1. P (A) ≥ 0

2. P (U ) = 1

3. If A1 , A2 , . . . , An are mutually exclusive events (Ai ∩ Aj = ∅ for i ̸= j):

n
X
P (A1 ∪ A2 ∪ · · · ∪ An ) = P (Ai )
i=1

3.4 Basic Probability Theorems

1. Theorem 1: If A1 ⊂ A2 , then P (A1 ) ≤ P (A2 ).
Proof: We can write A2 = A1 ∪ (A2 − A1 ), where A1 and (A2 − A1 ) are mutually
exclusive. By Axiom 3, P (A2 ) = P (A1 )+P (A2 −A1 ). By Axiom 1, P (A2 −A1 ) ≥ 0.
Therefore, P (A2 ) − P (A1 ) ≥ 0 =⇒ P (A2 ) ≥ P (A1 ).

A2

A2 − A1
A1
A1

2. Theorem 2: 0 ≤ P (A) ≤ 1.
Proof: By Axiom 1, P (A) ≥ 0. Since A ⊂ U , by Theorem 1, P (A) ≤ P (U ). By
Axiom 2, P (U ) = 1. Thus, P (A) ≤ 1.

3. Theorem 3: P (∅) = 0.
Proof: U = U ∪ ∅. U and ∅ are mutually exclusive. P (U ) = P (U ) + P (∅) =⇒
1 = 1 + P (∅) =⇒ P (∅) = 0.

4. Theorem 4 (Complement): P (A′ ) = 1 − P (A).

Proof: A ∪ A′ = U . A and A′ are mutually exclusive. P (A ∪ A′ ) = P (A) + P (A′ ) =
P (U ) = 1. Therefore, P (A′ ) = 1 − P (A).

10
U

A A′

11
4 Part 3: Event Operations and Conditional Proba-
bility
4.1 Types of Events
ˆ Mutually Exclusive Events: Two events A and B are mutually exclusive if they
cannot occur simultaneously. A ∩ B = ∅ =⇒ P (A ∩ B) = 0.

ˆ Collectively Exhaustive Events: A set of events is collectively exhaustive if

their union covers the entire sample space. A1 ∪ A2 ∪ · · · = U .

ˆ Complementary Events: Two events A and A′ are complementary if they are

both mutually exclusive (A ∩ A′ = ∅) and collectively exhaustive (A ∪ A′ = U ).

4.2 Laws of Probability

1. Addition Law (General): For any two events A and B:

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

Proof: A ∪ B = (A − B) ∪ (A ∩ B) ∪ (B − A). These 3 sets are mutually exclusive.

P (A∪B) = P (A−B)+P (A∩B)+P (B−A). Since P (A) = P (A−B)+P (A∩B) =⇒
P (A−B) = P (A)−P (A∩B). And P (B) = P (B −A)+P (A∩B) =⇒ P (B −A) =
P (B) − P (A ∩ B). Substitute these: P (A ∪ B) = (P (A) − P (A ∩ B)) + P (A ∩ B) +
(P (B) − P (A ∩ B)) P (A ∪ B) = P (A) + P (B) − P (A ∩ B).

A B

A−B A∩B B−A

2. Theorem 6: P (A) = P (A ∩ B ′ ) + P (A ∩ B). Proof: A = (A ∩ B ′ ) ∪ (A ∩ B).

These two sets are mutually exclusive. By Axiom 3, P (A) = P (A ∩ B ′ ) + P (A ∩ B).

3. Multiplication Law (Independent Events): Two events are independent if the

occurrence of one does not affect the probability of the other.

P (A ∩ B) = P (A)P (B)

4. Multiplication Law (Dependent Events):

P (A ∩ B) = P (A)P (B|A) = P (B)P (A|B)

12
4.3 Conditional Probability
The conditional probability of event B occurring, given that event A has already occurred,
is:
P (A ∩ B)
P (B|A) = provided P (A) > 0
P (A)

Example: 25 Students
A department has 25 students classified by course (A, B) and sex (M, F).

Course A Course B Total

Male (M) 10 5 15
Female (F) 3 7 10
Total 13 12 25

Problem: (1) What is P (F |B) (Prob. is Female, given takes Course B)? (2) What is
P (A|F ) (Prob. takes Course A, given is Female)?
∩B) ∩B)
Solution: (1) P (F |B) = P P(F(B) = n(F
n(B)
7
= 12 ≈ 0.583 (2) P (A|F ) = P P(A∩F
(F )
)
=
n(A∩F ) 3
n(F )
= 10 = 0.3
(Note: The original notes solved for P (B|F ) = 7/10 = 0.7)

4.4 Total Probability Theorem

Let A1 , A2 , . . . , An be a partition of the sample space S (i.e., they are mutually exclusive
and collectively exhaustive). For any event B:

P (B) = P (B ∩ A1 ) + P (B ∩ A2 ) + · · · + P (B ∩ An )
n
X
P (B) = P (Ai )P (B|Ai )
i=1

A1 A2
B

A4 A3

13
Example: Defective Items
Machines A, B, C produce 45%, 35%, 20% of items, respectively. Their defective rates
are 3%, 4%, 5%. Find the probability a random item is defective (D). Solution:

P (D) = P (A)P (D|A) + P (B)P (D|B) + P (C)P (D|C)

P (D) = (0.45)(0.03) + (0.35)(0.04) + (0.20)(0.05)

P (D) = 0.0135 + 0.0140 + 0.0100 = 0.0375

4.5 Bayes’ Theorem

Given a partition A1 , . . . , An and an event B, Bayes’ Theorem finds the probability of a
specific partition Aj given that B occurred:

P (Aj ∩ B) P (Aj )P (B|Aj )

P (Aj |B) = = Pn
P (B) i=1 P (Ai )P (B|Ai )

Example 1: Defective Items (cont.)

Given the item was defective (D), find the probability it came from Machine A (M1 ).
(Note: The original notes had P (D|M1 ) = 0.05, P (D|M2 ) = 0.03, P (D|M3 ) = 0.02. The
calculation below uses those numbers.) P (D) = (0.45)(0.05)+(0.35)(0.03)+(0.20)(0.02) =
0.0225 + 0.0105 + 0.0040 = 0.0370 Solution:
P (M1 )P (D|M1 ) (0.45)(0.05) 0.0225
P (M1 |D) = = = ≈ 0.6081
P (D) 0.0370 0.0370

Example 2: Balls in Boxes

Box 1 has 3 Blue, 2 Red. Box 2 has 2 Blue, 5 Red. A ball is drawn from a random box
and is Blue. What is P (from Box 1|Blue)? Solution: Let B1 = Box 1, B2 = Box 2, B
= Blue ball. P (B1 ) = 1/2, P (B2 ) = 1/2. P (B|B1 ) = 3/5, P (B|B2 ) = 2/7.

P (B1 )P (B|B1 )
P (B1 |B) =
P (B1 )P (B|B1 ) + P (B2 )P (B|B2 )
1 3 3 3
2
× 5 10 10 3 70 21
= 1 3

1 2

= 3 1 = 21+10 = × =
2
× 5
+ 2
× 7 10
+ 7 70
10 31 31

Example 3: Transferring Chips

Bowl I: 6 Red, 4 Blue. 5 chips are moved to empty Bowl II. 1 chip is drawn from Bowl
II and is Blue. Find P (2R, 3B transferred|Blue). Solution: Let W1 = Event (2 Red,
3 Blue) transferred. Let W2 = Event (3 Red, 2 Blue) transferred. Let W3 = Event
(4 Red, 1 Blue) transferred. Let W4 = Event (1 Red, 4 Blue) transferred. Let B =
Event that a blue chip is drawn from bowl II. We want P (W1 |B). Total ways to choose
(62)(43) (63)(42)
5 from 10 is 10 15×4 60
= 20×6 = 120


5
= 252. P (W 1 ) = 252
= 252
= 252
P (W 2 ) = 252 252 252
(64)(41) 15×4 60 (61)(44) 6×1 6
P (W3 ) = 252 = 252 = 252 P (W4 ) = 252 = 252 = 252 (Note: These partitions are
incomplete, e.g., (5R, 0B) is missing, but we follow the notes.)

14
 Conditional probabilities of drawing Blue from Bowl II: P (B|W1 ) (Bowl II has 2R,
3B) = 3/5 P (B|W2 ) (Bowl II has 3R, 2B) = 2/5 P (B|W3 ) (Bowl II has 4R, 1B) = 1/5
P (B|W4 ) (Bowl II has 1R, 4B)P= 4/5 60
× 35 + 252

120 2
60
× 5 + 252 × 15 +


Total
Probability P (B) = P (Wi )P (B|Wi ): P (B) = 252
6
252
× 45 P (B) = 180+240+60+24
1260
504
= 1260 42
= 105 = 25
Now, find P (W1 |B) using Bayes’ Theorem:

P (W1 )P (B|W1 ) (60/252) × (3/5)

P (W1 |B) = =
P (B) 2/5

180/1260 1/7 1 5 5
= = = × =
2/5 2/5 7 2 14
(Note: The notes’ calculation P (W1 |B) = (1/7)/(42/105) = 15/42 = 5/14 is correct.)

15
Part II
PROBABILITY II: Random
Variables
5 Part 4: Random Variables
5.1 Definition and Types
Random Variable (R.V.): A function X which assigns a real number X(s) to every
sample point s in the sample space U .
ˆ Discrete Random Variable: Takes on a finite or countably infinite number of
values.
ˆ Continuous Random Variable: Takes on an uncountably infinite number of
values (e.g., all values in an interval).

5.2 Discrete R.V. and Probability Mass Functions (PMF)

A PMF, p(x) = P (X = x), gives the probability for each discrete value x. It must satisfy:
1. p(x) ≥ 0 for all x
P
2. all x p(x) = 1

Example 1: Coin tossed 3 times

Let X = number of heads. S = {HHH, HHT, HT H, T HH, HT T, T HT, T T H, T T T }
P (X = 0) = 1/8, P (X = 1) = 3/8, P (X = 2) = 3/8, P (X = 3) = 1/8.
Distribution Table (PMF):
X 0 1 2 3
p(X) 1/8 3/8 3/8 1/8

Example 2: PMF as a Geometric Series

P∞
f (x) = C( 23 )x for x = 1, 2, 3, . . . , find C. Solution: We need
If P 2 x
x=1 C( 3 ) = 1.
C ∞ 2 x
x=1 ( 3 ) = 1. The sum is an infinite geometric progression with a = 2/3 and r = 2/3.
a 2/3 2/3
S∞ = 1−r = 1−2/3 = 1/3 = 2. So, C · (2) = 1 =⇒ C = 1/2.

5.3 Continuous R.V. and Probability Density Functions (PDF)

A PDF, f (x), describes the probability distribution for a continuous R.V. It must satisfy:
1. f (x) ≥ 0 for all x
R∞
2. −∞ f (x)dx = 1
Rb
Probability is the area under the curve: P (a ≤ X ≤ b) = a
f (x)dx. Note: For any
continuous R.V., P (X = a) = 0.

16
Example: Continuous PDF
Let f (x) = kx2 for −1 ≤ x ≤ 1. (a) Find k. (b) Compute P (X ≥ 1/2).
Solution: (a) Find k
1 1
x3
Z 
2
kx dx = 1 =⇒ k =1
−1 3 −1
    
1 1 2 3
k − − = 1 =⇒ k = 1 =⇒ k =
3 3 3 2
(b) Compute P (X ≥ 1/2)
1  1
3 x3
Z
3 2 1
P (X ≥ 1/2) = x dx = = [x3 ]11/2
1/2 2 2 3 1/2 2
   
1 3 1 3 1 1 7
= 1 −( ) = 1− =
2 2 2 8 16

5.4 Cumulative Distribution Function (CDF)

The CDF, F (x), gives the total probability up to a point x. F (x) = P (X ≤ x).

ˆ Discrete: F (x) = t≤x p(t). This is a step function.

P

Rx
ˆ Continuous: F (x) = −∞ f (t)dt. This is a continuous function.

Example: CDF for coin tossed twice

P (X = 0) = 1/4, P (X = 1) = 1/2, P (X = 2) = 1/4.


 0 for x<0

1/4 for 0≤x<1
F (x) =


 3/4 for 1≤x<2
x≥2

1 for

3/4
F (x)

1/2

1/4

0
0 1 2
x

17
Example: CDF for f (x) = 32 x2 on [−1, 1]
For x ∈ [−1, 1]:
x  x
3 t3
Z
3 2 1 1
F (x) = t dt = = [x3 − (−1)3 ] = (x3 + 1)
−1 2 2 3 −1 2 2

So the full CDF is: 

0
 x < −1
1
F (x) = (x3 + 1) −1 ≤ x ≤ 1
2
1 x>1


18
6 Part 5: Expectation and Variance
6.1 Expected Value (Mean)
The Expected Value E(X) (or µ) is the long-run average of the R.V.

ˆ Discrete: E(X) = all x x · p(x)

P
R∞
ˆ Continuous: E(X) = −∞ xf (x)dx

Example: E(X) for 2 Dice Roll Sum

The possible sums X range from 2 to 12.
X 2 3 4 5 6 7 8 9 10 11 12
1 2 3 4 5 6 5 4 3 2 1
p(X) 36 36 36 36 36 36 36 36 36 36 36
       
1 2 3 1
E(X) = 2 +3 +4 + · · · + 12
36 36 36 36
1 252
E(X) = [2 + 6 + 12 + 20 + 30 + 42 + 40 + 36 + 30 + 22 + 12] = =7
36 36

6.2 Properties of Expected Value

Let X, Y be R.V.s and a, b, C be constants.

1. E(C) = C

2. E(aX + b) = aE(X) + b

3. E(X + Y ) = E(X) + E(Y )

4. If X, Y are independent, E(XY ) = E(X)E(Y ).

6.3 Variance
The Variance V (X) (or σ 2 ) measures the spread of the distribution.

V (X) = E[(X − µ)2 ] = E[(X − E(X))2 ]

Computational Formula:
V (X) = E(X 2 ) − (E(X))2
p
The Standard Deviation is σ = V (X).

19
Example: V (X) for f (x) = 1/4 on [−2, 2]
1. Find E(X):
 
Z 2
1 1 1
E(X) = x dx = [x2 ]2−2 = [4 − 4] = 0
−2 4 8 8

2. Find E(X 2 ):
 
Z 2
2 1 21 1 16 4
E(X ) = x dx = [x3 ]2−2 = [8 − (−8)] = =
−2 4 12 12 12 3

3. Find V (X):
4 4
V (X) = E(X 2 ) − (E(X))2 = − (0)2 =
3 3
4. Find Standard Deviation (S.D.):
r √
p 4 2 2 3
S.D. = V (X) = =√ =
3 3 3

6.4 Properties of Variance

Let X, Y be R.V.s and a, b, C be constants.

1. V (C) = 0

2. V (X + C) = V (X) Proof: V (X + C) = E[((X + C) − E[X + C])2 ] = E[((X +

C) − (E[X] + C))2 ] = E[(X − E[X])2 ] = V (X).

3. V (aX) = a2 V (X) =⇒ V (aX + b) = a2 V (X)

4. If X, Y are independent: V (X + Y ) = V (X) + V (Y ) Proof: V (X + Y ) = E[(X +

Y )2 ] − (E[X + Y ])2 = E[X 2 + 2XY + Y 2 ] − (E[X] + E[Y ])2 = (E[X 2 ] + 2E[XY ] +
E[Y 2 ])−((E[X])2 +2E[X]E[Y ]+(E[Y ])2 ) = (E[X 2 ]−(E[X])2 )+(E[Y 2 ]−(E[Y ])2 )+
2(E[XY ] − E[X]E[Y ]) Since X, Y are independent, E[XY ] = E[X]E[Y ], so the
last term is zero. = V (X) + V (Y ).

5. If X, Y are independent: V (X − Y ) = V (X) + V (Y ) Proof: V (X − Y ) = E[(X −

Y )2 ]−(E[X −Y ])2 = (E[X 2 ]−2E[XY ]+E[Y 2 ])−((E[X])2 −2E[X]E[Y ]+(E[Y ])2 )
= (E[X 2 ] − (E[X])2 ) + (E[Y 2 ] − (E[Y ])2 ) − 2(E[XY ] − E[X]E[Y ]) Since X, Y are
independent, the last term is zero. = V (X) + V (Y ).

20