Background for Lesson 1
1 Rules of Probability
Lesson 1 will assume that you have previously seen probability notation and the basic rules
of probability. For reference, we review those here.
Probabilities are defined for events. An event is some outcome that we could potentially
or hypothetically observe or experience, such as the result of rolling a fair six-sided die.
(English grammar note: die is singular, dice is plural.) In mathematical notation, we often
write an event as a capital letter, for example, A is the event that we roll a “4” on a fair
six-sided die. This event has probability 1/6, so we would write P (A) = 1/6. We might
want to represent the numerical result of the die roll as the random variable X, and then we
could also write P (X = 4) = 1/6.
Probabilities must be between zero and one, i.e., 0 ≤ P (A) ≤ 1 for any event A.
Probabilities add to one, i.e., if we add up the probabilities of all possible events, those
probabilities must add to one. For the die rolling example:
6
X
P (X = i) = 1 .
i=1
6
X
Recall that the symbol denotes adding up the entries as i goes from 1 to 6.
i=1
The complement of an event, Ac , means that the event does not happen. Since probabilities
must add to one, P (Ac ) = 1 − P (A).
If A and B are two events, the probability that A or B happens (this is an inclusive or,
meaning that either A, or B, or both happen) is the probability of the union of the events:
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
where ∪ represents union (“or”) and ∩ represents intersection (“and”). If a set of events Ai
1
for i = 1, . . . , m are mutually exclusive (only one can happen), then
m
[ m
X
P( Ai ) = P (Ai ) .
i=1 i=1
2 Odds
Probabilities can be re-expressed in terms of odds. Suppose again that we denote rolling a
“4” on a fair six-sided die as the event A. Then P (A) = 1/6. The odds for event A, denoted
O(A) is defined as O(A) = P (A)/P (Ac ) = P (A)/(1 − P (A)). Hence, in this example,
1/6 1
O(A) = = .
5/6 5
This can also be expressed as 1:5 (or 5:1 “odds against”). Thus, an event with probability
3/10 has 3:7 odds (7:3 odds against) and an event with probability 4/5 has 4:1 odds.
Note that we can also calculate probabilities from odds. If an event B has a : b odds (with
a > 0 and b > 0), then P (B)/(1 − P (B)) = a/b =⇒ P (B) · b = a − P (B) · a =⇒ P (B) =
a/(a + b). Thus, an event with 2:5 odds has probability 2/7.
3 Expectation
The expected value of a random variable X is a weighted average of values X can take, with
weights given by the probabilities of those values. If X can take on only a finite number of
values (say, x1 , x2 , . . . , xn ), we can calculate the expected value as
n
X
E(X) = xi · P (X = xi ) .
i=1
For example, the expected value of a fair six-sided die would be
6 6
X X 1 1
E(X) = i · P (X = i) = i· = (1 + 2 + 3 + 4 + 5 + 6) = 3.5 .
i=1 i=1
6 6
Note that the die cannot achieve this value, but if you were to roll the die many times and
average the values, the result would likely be close to 3.5.