Chapter 01 Probability Concepts

and Applications
IIMT3636
Faculty of Business and Economics
The University of Hong Kong
Instructor: Dr. Sundar
 2

All positive cases reported by rapid test must

be re-examined by nucleic acid test.
 3

Introduction
• Probability is a numerical statement about the likelihood
that an event will occur.
▫ 10% chance of rain tomorrow
▫ 20% chance the Hang Seng Index will go up next week
▫ 30% chance the launch of the new iPhone will be delayed
▫ 40% chance our life will go back to normal next year

• Notation: 𝑃 𝐴 = Probability of event A occurring.

• 0 ≤ 𝑃 𝐴 ≤ 1.
 4

Determination of Probability
• Objective approach
▫ Classical or logical method
 𝑃 head = 0.5
 𝑃 spade = 0.25
 𝑃 getting one head and one tail in two tosses = 0.5
▫ Relative frequency
 Use data, experiments, or simulation

• Subjective approach
 5

Relative Frequency: An Example

Daily Sales Statistics of A Newspaper at a Newsstand
120

100

80
Sales Quantity

60

40

20

0
0 10 20 30 40 50
Day
 6

Determination of Probability
• Objective approach
▫ Classical or logical method
 𝑃 head = 0.5
 𝑃 spade = 0.25
 𝑃 getting one head and one tail in two tosses = 0.5
▫ Relative frequency
 Use data, experiments, or simulation

• Subjective approach
▫ Expert opinion
▫ Opinion polls (wisdom of the crowd)
 7

Opinion Polls: An Example

• You developed a new product and there are two possible
selling prices: $40 or $60.
• You surveyed 100 consumers about whether they would
buy the product at each price.
• At price $40, 70 consumers gave positive responses.
• At price $60, 40 consumers gave positive responses.
• The cost of selling the product is $20 per unit.
• Which price is better?

▫ Expected profit at price $40 = 0.7*$(40-20) = $14

▫ Expected profit at price $60 = 0.4*$(60-20) = $16
 8

Events
• Mutually exclusive: Events are said to be mutually exclusive if
only one of the events can occur in any one trial
• Collectively exhaustive: Events are said to be collectively
exhaustive if the list of outcomes includes every possible outcome
Mutually Collectively
Two events: A and B
Exclusive Exhaustive
1. Draw a spade and a club Yes No
2. Draw a face card and a number card Yes Yes
3. Draw an ace and a 3 Yes No
4. Draw a club and a non-club Yes Yes
5. Draw a 5 and a diamond No No
6. Draw a red card and a non-diamond No Yes
 9

Venn Diagram
The area of an event represents the probability.

A B A B

Events that are Events that are not

mutually exclusive mutually exclusive

A: draw a red number card A: draw a number card

B: draw a black number card B: draw a black card
Question: what’s the blue area?
 10

Union and Intersection

• The union of two events is the set of all
outcomes that are contained in either of A B
the two events.
• 𝑃 Union of 𝐴&𝐵 = 𝑃 𝐴 or 𝐵 = 𝑃 𝐴 ∪ 𝐵

• The intersection of two events is the set

of all outcomes that are common to both
events.
• 𝑃 Intersection of 𝐴&𝐵 = 𝑃 𝐴 and 𝐵 = A B
𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴𝐵 ; it is called joint prob.

• 𝑷 𝑨∪𝑩 +𝑷 𝑨∩𝑩 =𝑷 𝑨 +𝑷 𝑩
 11

Conditional Probability
• A conditional probability is the probability of an event 𝐴 occurring
given that another event 𝐵 has already happened.

𝑃 𝐴𝐵
• Notation: 𝑃 𝐴|𝐵 = . Why?
𝑃 𝐵

A B
• 𝑷 𝑨𝑩 = 𝑷 𝑨|𝑩 ∙ 𝑷 𝑩 = 𝑷 𝑩|𝑨 ∙ 𝑷 𝑨 .

• Independent events:
• If 𝐴 ⊥ 𝐵, then 𝑃 𝐴|𝐵 = 𝑃 𝐴 .
• If 𝐴 ⊥ 𝐵, then 𝑷 𝑨𝑩 = 𝑷 𝑨 ∙ 𝑷 𝑩 .
 12

Basic Probability Rules

• 0 ≤ 𝑃 𝐴 ≤ 1 for any event 𝐴.
• 𝑃 𝐴 ∩ 𝐵 = 0 if 𝐴 and 𝐵 are mutually exclusive.
• 𝑃 𝐴 ∪ 𝐵 = 1 if 𝐴 and 𝐵 are collectively exhaustive.
• 𝑃 𝐴∪𝐵 +𝑃 𝐴∩𝐵 =𝑃 𝐴 +𝑃 𝐵 .
• 𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴|𝐵 ∙ 𝑃 𝐵 = 𝑃 𝐵|𝐴 ∙ 𝑃 𝐴 .
• 𝑃 𝐴|𝐵 = 𝑃 𝐴 if 𝐴 and 𝐵 are independent.
 13

In-class Exercises
• Suppose 𝑃 𝐴 = 0.4 and 𝑃 𝐵 = 0.6. Are 𝐴 and 𝐵 mutually exclusive?
• Suppose 𝐴 and 𝐵 are mutually exclusive and 𝑃 𝐴 = 0.4. Then 𝑃 𝐵 = ?
• Suppose 𝐴 and 𝐵 are mutually exclusive. In addition, 𝑃 𝐴 = 0.4 and
𝑃 𝐵 = 0.6. Suppose 𝐶 and 𝐷 are also mutually exclusive and collectively
exhaustive. Further, 𝑃 𝐶|𝐴 = 0.2 and 𝑃 𝐷|𝐵 = 0.4. What are 𝑃 𝐶 and
𝑃 𝐷 ?
• A student taking MS301 at East Haven University will receive one of the five
possible grades for the course: A, B, C, D, or F. The distribution of grades
over the past two years is in the table. Grade Number of Students
What is the probability of a student A 80

receiving at least C? B 75

Given that a student didn’t fail, what is the C 90

D 30
probability of this student receiving C?
F 25
Total: 300
 14

Bayes’ Theorem
• How to revise your probability assessment when you
have new information?

Diagnostic test for the Human Immuno-deficiency Virus (HIV)

Infected Not Infected
Test Positive 90% (conditional)
Test Negative 95% (conditional)
HK Prevalence Rate 0.1% (marginal) 99.9% (marginal)

• 𝑃 Infected|Test Positive =?
 15

Bayes’ Theorem
• 𝐴 = Infected; 𝐴′ = Not Infected.
• 𝐵 = Test Positive; 𝐵′ = Test Negative.

𝑃 𝐴𝐵 𝑃 𝐵|𝐴 𝑃 𝐴 𝑃 𝐵|𝐴 𝑃 𝐴
• 𝑃 𝐴|𝐵 = = =
𝑃 𝐵 𝑃 𝐵𝐴 +𝑃 𝐵𝐴′ 𝑃 𝐵|𝐴 𝑃 𝐴 +𝑃 𝐵|𝐴′ 𝑃 𝐴′

▫ Note: 𝑃 𝐵𝐴 + 𝑃 𝐵𝐴′ = 𝑃 𝐵𝐴 ∪ 𝐵𝐴′ + 𝑃 𝐵𝐴 ∩ 𝐵𝐴′ =𝑃 𝐵 +0

0.9×0.001
• 𝑃 Infected|Test Positive = =1.77%
0.9×0.001+0.05×0.999
 16

Bayes’ Theorem: An Intuitive Way

• Draw a matrix for possible events related to two types
(dimensions) of information.
• Calculate the joint probabilities
• Calculate conditional probabilities

Infected Not Infected

Test Positive 0.9*0.001=0.0009 0.05*0.999=0.04995
Test Negative 0.1*0.001=0.0001 0.95*0.999=0.94905
HK Prevalence Rate 0.1% (marginal) 99.9% (marginal)
 17

Bayes’ Theorem: An Example

• The Monty Hall Problem
▫ Suppose you're on a game show, and you're given the choice of
three doors: Behind one door is a car; behind the others, goats.
You pick a door, say No. 1, and the host, who knows what's
behind the doors, opens another door, say No. 3, which has a
goat. He then says to you, "Do you want to pick door No. 2?" Is it
to your advantage to switch your choice?
 18

Bayes’ Theorem: An Example

(1) (2) (3)

X and A 0 0 0 0
X: YouBpick(1/2)(1/3)
X and door 1 0 (1.0)(1/3) 1/2
A: Host opens door 1 (1): The car is behind door 1
X and
B: HostC opens
(1/2)(1/3)
door 2 (1.0)(1/3) 0 behind door
(2): The car is 1/2 2
C: Host opens 1/3
door 3 (3): The car1/3
1/3 is behind door 3
 19

In-class Exercises
• There are two fortune tellers, A & B. According to
historical data, A’s predictions were correct in 90%
cases, while B’s predictions were correct only in 30%
cases. Now, without communicating with each other,
both A & B predict that Donald Trump will be elected
again in the next election.
• Without any information, your prior belief about Donald
Trump being elected again is 0.5. Now knowing A & B’s
predictions, what should be the correct probability for
you?
 20

Random Variables
• A random variable (RV) assigns a real number to every
possible outcome or event in a statistical experiment
such that the numbers represent mutually exclusive and
collectively exhaustive events.

• A discrete RV can assume only a finite or countable set

of values.
▫ E.g., X = the number of newspapers sold during the day.

• A continuous RV has an uncountable set of possible

values.
▫ E.g., Y = the lifespan of a light bulb.
 21

RV with Qualitative Outcomes

• When the outcome itself is not numerical or quantitative,
it is necessary to define an RV that associates each
outcome with a unique real number.
▫ E.g.,
▫ For tossing a coin, X = 1 if head and 0 if tail;
▫ For consumers’ response to how they like a product, Y = 1 if poor,
2 if average, and 3 if good;
▫ For the brand of soda purchased by a consumer, Z = 1 if Pepsi, 2
if Coca-Cola, and 3 if Dr. Pepper.
 22

Probability Distribution
• X = the number of newspapers sold during a day.

The Histogram Cumulative Probability

0.4 1
0.35 0.9
0.8
0.3
0.7

Probability
Probability

0.25 0.6
0.2 0.5
0.15 0.4
0.3
0.1
0.2
0.05 0.1
0 0
 23

Probability Distribution: Discrete RV

• For each possible outcome 𝑋𝑖 , there is a probability value
𝑃 𝑋𝑖 .
• These values must be between 0 and 1: 0 ≤ 𝑃 𝑋𝑖 ≤ 1.
• They must sum up to 1: σ𝑛𝑖=1 𝑃 𝑋𝑖 = 1.
0.4
0.35
Distribution for Quiz

0.3
Probability

0.25
Scores

0.2
0.15
0.1
0.05
0
𝑋1 = 1 𝑋2 = 2 𝑋3 = 3 𝑋4 = 4 𝑋5 = 5
 24

Probability Distribution: Continuous RV

• The sum of the probability values must equal 1.
• A continuous RV can take on an uncountable set of
values such that the probability of each value must be 0.

• The probability distribution is defined by continuous

mathematical functions, the cumulative distribution
function (CDF), and its derivative, the probability
density function (PDF).
▫ CDF is denoted by 𝐹 ∙ and 𝐹 𝑥 = 𝑃 𝑋 ≤ 𝑥 .
▫ PDF is denoted by 𝑓 ∙ = 𝐹 ′ ∙ and 𝑓 𝑥 ≈ 𝑃 𝑥 < 𝑋 ≤ 𝑥 + ∆ /∆.
 25

Probability Distribution: Continuous RV

• 𝑊 = the weight of a particular machined part
Density

f
𝑃 5.22 ≤ 𝑊 ≤ 5.26
= 𝐹 5.26 − 𝐹 5.22
5.26
=න 𝑓 𝑊 𝑑𝑊
5.22
 26

Summary Statistics
• The expected value is a measure of central tendency of a
probability distribution, and it is defined as
𝑛

𝜇 = 𝐸 𝑋 = ෍ 𝑋𝑖 ∙ 𝑃 𝑋𝑖
𝑖=1
= 𝑋1 ∙ 𝑃 𝑋1 + 𝑋2 ∙ 𝑃 𝑋2 + ⋯ + 𝑋𝑛 ∙ 𝑃 𝑋𝑛

• The variance is a measure of the overall spread or

dispersion of a distribution, and it is defined as
𝑛

𝜎 2 = 𝑉 𝑋 = ෍ 𝑋𝑖 − 𝐸 𝑋 2
∙ 𝑃 𝑋𝑖
𝑖=1
 27

In-class Exercises
• True or False?
▫ We can define X to be a random variable as follows: X = 0 if it
rains tomorrow and X = 1 if it is sunny tomorrow.
▫ Suppose a random variable X has the following three possible
values: -1, 0, and 1. Then the expectation of X is 0.
▫ If 𝑓(𝑥) is the pdf of a random variable X, we must have 𝑓 𝑥 ≤ 1.
▫ If X is a discrete random variable, then the possible values of X
must be integers.
▫ If X is a discrete random variable that can take integer values
from 1 to 5, then 𝑃 2 ≤ 𝑋 ≤ 3 = 𝑃 𝑋 ≤ 3 − 𝑃 𝑋 ≤ 2 .
 28

Binomial Distribution
• A binomial RV represents the number of successes out of
𝑛 independent trials.

• 𝑛 = the number of trials

• 𝑝 = the probability of success on any single trial
• 𝑟 = the number of successes

𝑛!
• 𝑃 𝑟; 𝑛, 𝑝 = ∙ 𝑝𝑟 ∙ 1 − 𝑝 𝑛−𝑟
𝑟! 𝑛−𝑟 !

• If 𝑋~𝐵 𝑛, 𝑝 , then 𝐸 𝑋 = 𝑛𝑝 and 𝑉 𝑋 = 𝑛𝑝 1 − 𝑝 .

 29

Binomial Distribution in Excel

 30

Poisson Distribution
• A Poisson RV describes the number of independent
arrivals during a unit period of time.
▫ E.g., patients arriving at a health clinic

• 𝜆 = average number of arrivals per unit of time

• 𝑒 = 2.718, the base of the natural logarithm
• 𝑥 = the actual number of arrivals (0, 1, 2, …)

𝑒 −𝜆 ∙𝜆𝑥
• 𝑃 𝑥; 𝜆 = (Note: if time length 𝑡 ≠ 1, replace 𝜆 with 𝜆𝑡.)
𝑥!

• If 𝑋~𝑃𝑜𝑖𝑠𝑠𝑜𝑛 𝜆 , then 𝐸 𝑋 = 𝜆 and 𝑉 𝑋 = 𝜆.

 31

Poisson Distribution in Excel

 32

Binomial and Poisson Distributions

• It turns out the Poisson distribution is just a special case of the
binomial — where the number of trials is large, and the probability
of success in any given one is small.
• We can use a Binomial distribution with parameters (𝑛, 𝑝) to
approximate a Poisson distribution with rate 𝜆 by setting 𝑛 to a large
number and 𝑝 = 𝜆/𝑛.

Poisson,  = 5
0.2

0.15

0.1

0.05

0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
 33

Exponential Distribution
f
• The exponential distribution
often describes the time required
to service a customer or the
lifespan of a product (e.g., a light
bulb).

• If 𝑋~𝐸𝑋𝑃 𝜆 , then 𝑓 𝑥 = 𝜆𝑒 −𝜆𝑥

and 𝐹 𝑥 = 1 − 𝑒 −𝜆𝑥 , where 𝜆 = F
1/𝐸 𝑋 .
• 𝐸 𝑋 = 1/𝜆 and 𝑉 𝑋 = 1/𝜆2 .
 34

Exponential and Poisson Distributions

• Exponential and Poisson are related: if the number of
arrival per time period follows Poisson(𝜆0 ), then the time
between arrivals follows an identical exponential
distribution with rate 𝜆1 = 𝜆0 .

• Exercise. Suppose customer arrivals at a retail store per

hour follow Poisson with mean 𝜆 = 3. Given that a
customer arrived at 2:30pm, what is the probability that
the next customer will arrive before 3:00pm?

▫ 𝑃𝑃 𝑋 ≥ 1; 𝜆 = 1.5 or 𝑃𝐸 𝑋 ≤ 0.5; 𝜆 = 3
 35

Exponential Distribution in Excel

 36

Uniform Distribution
• A random variable 𝑋 is uniformly distributed if it takes
any value equally likely from a finite interval 𝑎, 𝑏 .

Probability density function Cumulative distribution function

 37

Uniform Distribution in Excel

• 𝑟𝑎𝑛𝑑() function generates a uniformly distributed random
variable on [0,1]

• For a general uniform distribution on [a,b], we can use “=

𝑎 + 𝑟𝑎𝑛𝑑() ∗ 𝑏 − 𝑎 ”
 38

Normal Distribution
• The normal distribution is the most popular and useful
continuous probability distribution.
▫ E.g., return of a stock portfolio, forecast errors, and test scores.
▫ The probability density function is
1 𝑥−𝜇 2
−
𝑓 𝑥 = ∙ 𝑒 2𝜎2
𝜎 2𝜋

• The normal distribution is specified completely when we

know the mean, 𝜇, and the standard deviation, 𝜎. We
often use the notation 𝑁 𝜇, 𝜎 2 .
 39

Normal Distribution
• Normal distribution with different 𝜇. (shape unchanged)

| | |
40  = 50 60

Smaller , same 

| | |
 = 40 50 60

Larger , same 
| | |
40 50  = 60
 40

Normal Distribution
• Normal distribution with different 𝜎. (always symmetric)

Same , smaller 

Same , larger 


 41

Normal Distribution
 42

Standard Normal Distribution

𝑿−𝝁
If 𝑿~𝑵 𝝁, 𝝈𝟐 , then 𝒁 = follows standard normal distribution.
𝝈
 43

Standard Normal Distribution: Z Table

 44

Using Standard Normal Distribution

• If 𝑋 follows normal distribution with mean 𝜇 = 100 and
s.d. 𝜎 = 22, then 𝑃 𝑋 < 120 = ?

𝑋−𝜇 120−𝜇
• 𝑃 𝑋 < 120 = 𝑃 𝑋 − 𝜇 < 120 − 𝜇 = 𝑃 <
𝜎 𝜎
𝑋−𝜇
• Define 𝑍 = .
𝜎
• 𝑃 𝑋 < 120 = 𝑃 𝑍 < 0.91 = 0.8186

• If we want 𝑋 to be less than a certain value with

probability 0.8, then how to find this value? Or
𝑃 𝑋 < ? = 0.8.
 45

Finding the x value

• Step 1: Finding the z value corresponding to 𝑃 𝑋 < 𝑥 .
▫ If we need 𝑥 for 𝑃 𝑋 < 𝑥 , find 𝑧 corresponding to 𝑃 𝑋 < 𝑥 .
▫ If we need 𝑥 for 𝑃 𝑋 > 𝑥 , find 𝑧 corresponding to 1 − 𝑃 𝑋 > 𝑥 .
▫ 𝑧0.8 = 0.84

• Step 2: Setting 𝑥 to 𝑥𝑧 = 𝜇 + 𝑧 ∙ 𝜎.
𝑋−𝜇
▫ 𝑃 𝑋 < 𝑥𝑧 = 𝑃 𝑋 < 𝜇 + 𝑧 ∙ 𝜎 = 𝑃 <𝑧 =𝑃 𝑍<𝑧 =
𝜎
𝑃 𝑋<𝑥
▫ 𝑃 𝑋 < 100 + 𝑧0.8 × 22 = 𝑃 𝑋 < 118.48 = 0.8
 46

Haynes Construction Company

• Builds three- and four-unit apartment buildings
• Total construction time follows a normal distribution
• For triplexes, μ = 100 days and σ = 20 days
• Contract calls for completion in 125 days
• Late completion will incur a severe penalty fee
• Probability of completing in 125 days?

𝑃 𝑋 < 125 = 𝑃 𝑍 < 1.25 = 0.8944

 = 100 days
 = 20 days
x = 125 days
 47

Haynes Construction Company

• If finished in 75 days or less, bonus = $5,000. Probability of bonus?
• 𝑃 𝑋 < 75 = 𝑃 𝑍 < −1.25
• = 𝑃 𝑍 > 1.25 = 1 − 𝑃 𝑍 < 1.25 = 1 - 0.8944 = 0.1056

• What is the probability of completing between 110 and 125 days?

• 𝑃 110 < 𝑋 < 125 = 𝑃 𝑋 < 125 − 𝑃 𝑋 < 110

110 days 125 days

 48

Haynes Construction Company

• Suppose Haynes can hire more workers to reduce the
mean construction time while keeping the standard
deviation constant. Hence, to ensure a penalty is avoided
with probability 0.95, what should be the new mean?

𝑋 ′ −𝜇
• Objective: find 𝜇 such that 𝑃 𝑋′ < 125 = 𝑃ቀ <
20
125−𝜇
ቁ = 0.95
20
• Step 1: 𝑧0.95 = 1.645
125−𝜇
• Step 2: find 𝜇 such that = 1.645; 𝜇 = 92.1
20
 49

Normal Distribution in Excel

 50

Statistical Learning
• Consider the HIV example again.
• If a man took the test five times and got all negative
results, what is the probability that he is healthy?

Healthy Infected
Positive 5% 90%
Negative 95% 10% 0.1 + 0.85𝛼
𝛼 1−𝛼
 51

Recap
• Probability
• Venn Diagram
• Conditional Probability
• Bayes’ Theorem
• Random Variables
• Probability Distributions
• Statistical Learning (Not required)
 52

In-Class Exercise
• When a man passes the airport security check, they
discover a bomb in his bag. He explains. “Statisticians
show that the probability of a bomb being on an airplane
is 1/10,000. However, the chance that there are two
bombs on one plane is 1/10,000,000. So, I am much
safer …”

• Suppose the statisticians are right and it is impossible to

have more than two bombs on an airplane. Do you agree
with the man?