• To describe the uncertainty of the
phenomena,
what are the key things we must know?
Please provide your answer on Poll Everywhere!
Agenda
• Conditional Probability
• Bayes’Theorem
• Independence
Toy Example
•Assume two dice are tossed independently,
what is the probability of observing the sum
is 3 or 5, given that the sum is a odd number ?
After this lesson
•You can apply “conditional probability”
concepts to analyze the real-word problems
you encounter.
Conditional Probability
• The probability of an event given that
another event has occurred is called a
Conditional Probability.
Misunderstanding 1 – Ignoring Condition
•“The probability of surviving for 5 years with
colorectal cancer is …”
Notation of Conditional Probability
• A conditional probability is often expressed using
the phrase "given" to describe the condition.
• Examples
• 5-year survival probability of a colorectal cancer
patient given that his age is between 55-64 and he
has a Chinese background.
Notation
Prior Probability
• Let A be an event with probability P(A).
• Let B be an event that has occurred that may
change the probability of A.
• The conditional probability of A given B is
denoted by P(A|B).
Posterior Probability
Misunderstanding 2 – Confusing with
Reverse
David is a typical Singaporean
Probability that David has a Chinese
background given that he has colon cancer
≠
Probability that David has colon cancer given
that he has a Chinese background.
Conditional Probability
Computing Conditional Probability
𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵)
𝑃𝑃(𝐴𝐴|𝐵𝐵) =
𝑃𝑃(𝐵𝐵)
Conditional Probability
Venn diagram
B
Conditional Probability
Note that
𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵)
𝑃𝑃(𝐴𝐴|𝐵𝐵) =
𝑃𝑃(𝐵𝐵)
implies:
𝑃𝑃 𝐴𝐴 ∩ 𝐵𝐵 = 𝑃𝑃(𝐵𝐵 ∩ 𝐴𝐴)
𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵) = 𝑃𝑃(𝐴𝐴|𝐵𝐵) 𝑃𝑃(𝐵𝐵)
Common Mistake:
✗
Conditional Probability
Example
Experiment : pick one card from a deck
C : Obtain a Spade D : Obtain a Queen
Conditional Probability
P(C) = 13/52=1/4
Q: P(C|D) =
P(C ∩ D) = 1/52
P(D) = 4/52=1/13 Q: P(D|C) =
C: Obtain a Spade D: Obtain a Queen
Example
• A box contains seven red and thirteen blue balls.
Three balls were selected at random and were discarded
without their colors being seen. Suppose the third ball
drawn is a red ball,
what is the probability that both balls discarded were
blue?
Bayes’ Theorem
Often, we begin analysis with initial or prior probabilities for
certain events.
Then, we obtain additional info (from data collection).
Given this info, we calculate revised or posterior probabilities.
Bayes’ theorem
Prior New Bayes’ Posterior
Probabilities Information Theorem Probabilities
Bayes’ Theorem
Consider n mutually exclusive and collectively exhaustive events
A1, A2 , . . , An with given prior probabilities P(A1), P(A2) , . . , P(An)
Given that an event B has occurred, we can use Bayes’ theorem
to find the posterior probabilities of Ai .
𝑃𝑃(𝑩𝑩|𝑨𝑨𝒊𝒊 ) 𝑃𝑃(𝑨𝑨𝒊𝒊 )
𝑃𝑃(𝑨𝑨𝒊𝒊 |𝑩𝑩) =
𝑃𝑃(𝑩𝑩|𝑨𝑨𝟏𝟏 )𝑃𝑃(𝑨𝑨𝟏𝟏 ) + 𝑃𝑃(𝑩𝑩|𝑨𝑨𝟐𝟐 )𝑃𝑃(𝑨𝑨𝟐𝟐 )+ . . . +𝑃𝑃(𝑩𝑩|𝑨𝑨𝒏𝒏 )𝑃𝑃(𝑨𝑨𝒏𝒏 )
Bayes’ Theorem
𝑃𝑃(𝑨𝑨𝒊𝒊 ∩ 𝑩𝑩) 𝑃𝑃(𝑩𝑩|𝑨𝑨𝒊𝒊 ) 𝑃𝑃(𝑨𝑨𝒊𝒊 )
𝑃𝑃(𝑨𝑨𝒊𝒊 |𝑩𝑩) =
𝑃𝑃(𝑩𝑩)
𝑃𝑃(𝑩𝑩|𝑨𝑨𝒊𝒊 ) 𝑃𝑃(𝑨𝑨𝒊𝒊 )
𝑃𝑃(𝑨𝑨𝒊𝒊 |𝑩𝑩) =
𝑃𝑃(𝑩𝑩|𝑨𝑨𝟏𝟏 )𝑃𝑃(𝑨𝑨𝟏𝟏 ) + 𝑃𝑃(𝑩𝑩|𝑨𝑨𝟐𝟐 )𝑃𝑃(𝑨𝑨𝟐𝟐)+ . . . +𝑃𝑃(𝑩𝑩|𝑨𝑨𝒏𝒏)𝑃𝑃(𝑨𝑨𝒏𝒏 )
Bayes’ Theorem
B
A1 A2 A3
A Quick Example
• A NUS student can be a graduate from JC or Poly or Others.
• Each student can be a BIZ student or non-BIZ student
• The probability that a NUS student is from BIZ?
Pr 𝐵𝐵𝐵𝐵𝐵𝐵 = Pr 𝐵𝐵𝐵𝐵𝐵𝐵 and 𝐽𝐽𝐽𝐽 + Pr 𝐵𝐵𝐵𝐵𝐵𝐵 and 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 + Pr 𝐵𝐵𝐵𝐵𝐵𝐵 and 𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂
Bayes’ Theorem on
Diagnostic Test
Sensitivity = the probability that a person tests positive if the disease is present
= Pr(test positive | has the disease)
Positive Predictive Value
= the probability that a person has disease if tested positive
= Pr(has disease | test positive)
Specificity = the probability that a person tests negative if the disease is absent
= Pr(test negative| disease absent)
Bayes’ Theorem
1% of women at age 40 have breast cancer.
80% of women with breast cancer will get positive mammograms.
9.6% of women without breast cancer will also get positive
mammograms
Q: Amy in this age group had a positive mammogram in a routine
screening. What is the probability that she actually has breast
cancer?
Bayes’ Theorem
BC = woman (at 40) has breast cancer P(BC)=0.01
NBC = woman (at 40) does not have breast P(NBC)=0.99
cancer
PM = woman has a positive mammogram
P(PM|BC) = 0.80 P(PM|NBC) = 0.096
Probability Table
• Let A1, A2 mutually exclusive and collectively exhaustive
events. Likewise for B1, B2.
Joint Marginal
probabilities Probabilities
Probability Table
• Let A1, A2 mutually exclusive and collectively exhaustive
events. Likewise for B1, B2.
+ =
+
+
= + =
=
+ =
Bayes’ Theorem
BC = woman (at 40) has breast cancer P( B C) = 0 .0 1
NBC = woman (at 40) does not have breast cancer
BC NBC
PM
NPM
1
Bayes’ Theorem
PM = woman has a positive mammogram P(PM|BC) = 0.80
P(PM|NBC) = 0.096
BC NBC
PM
NPM
1
Bayes’ Theorem
Now that we have completed the probability
table below, we can answer the question: P(BC|PM) =
BC NBC
PM
NPM
What is your answer of Amy having breast cancer?
Independent Events
Independent Events
Two events A and B are independent if knowing whether one event
has occurred will not change our assessment of the other event.
Events A and B are independent if
P(A|B) = P(A) or P(B|A) = P(B)
Recall : P(A∩B) = P(A|B) P(B)
Thus, for independent events,
P(A∩B) = P(A) P(B)
Independent Events
Experiment : pick one card from a deck
C : Obtain a Spade P(C) =1/4, P(D) = 1/13,
D : Obtain a Queen P(C ∩ D) =1/52
P(C ∩ D) = P(C) x P(D)
Q : are they independent?
Information of the card being a Spade does not change the
probability that it would be a Queen.
P(D | C) = P(D)=1/13
Independent Events
Q : Can mutually exclusive events be
independent?
Take-home Message
• The uncertainty can be reduced by collecting data to
refine the probability statement
• When describing a conditional probability, the given
condition must be correctly stated.
• Don’t be confused with Independence and Mutual
Exclusiveness.