Applied Probability and Statistics for

Information Technology

Conditional Probability

Instructor: Tran Luong Quoc-Dai, Ph.D.

Outline

• Conditional Probability
• Motivation
• Introduction
• Definition & Formula
• Multiplication Rule
• Law of Total Probability
• Bayes’ Theorem
• Independence

March 19, 2024 2

A Motivating Scenario

• Consider a university course where the proportion of students earning a

grade is fixed at 10%.
• Without knowing anything else about a particular student, the probability that
this student gets is: .
• Now imagine you known that this student scored a perfect mark in the final
exam.
Would the chance of the
student getting still be 0.1?

March 19, 2024 3

A Motivating Scenario (cont.)

• This new fact strongly suggests the probability of getting an is higher

than 0.1 — because grades and exam performance are related events.
• Why this information changes things:
• Without extra information: The probability is based on the overall
distribution of grades.
• With extra information: Knowing the student’s final exam score changes our
belief about the likelihood of getting an .
• This modifies the experiment: instead of considering all possible students, we
focus only on those who scored perfectly in the final.
• This leads us to the idea of conditional probability — the probability of an event
given that another event has occurred.
March 19, 2024 4
Introduction to Conditional Probability

• In many probability problems, we begin with a complete description of an

experiment:
• The set of all possible outcomes is known, and each outcome has a specified
probability.
• In such cases, the probability of an event can be calculated directly from this
initial information.
• Before the experiment concludes, or before we actually evaluate the probability of an
event, we might learn new facts that partially reveal the outcome or rule out certain
possibilities.
• This additional information can change the context in which the probability is
computed.
March 19, 2024 5
Two Ways New Information Can Affect the Problem

1. Change in the sample space – The set of possible outcomes is reduced to only
those consistent with the information.
Example: When rolling two dice, if you are told the sum is 8, the original 36
possible outcomes shrink to the 5 pairs .
2. Change in the probability distribution – The probabilities assigned to each
remaining outcome may need to be updated to reflect the new conditions.
Example: In a card-drawing experiment, discovering that the card is red does not
change the total number of cards in the deck (still 52), but it alters the probability
distribution — the entire probability mass is now concentrated on the 26 red cards,
with each assigned a probability of
March 19, 2024 6
Conditional Probability

• Definition: The conditional probability of an event A given an event B is the probability

that A occurs under the condition that B is known to have occurred.
• Notation: .
• Intuition:
• When new information becomes available — such as knowing that event B has
happened — the set of outcomes we consider possible changes.
• Instead of considering all outcomes in the sample space Ω, we only consider the
outcomes that lie in B.
• From this restricted set, we determine the proportion that also satisfies A.

March 19, 2024 7

Formula

• Let A and B be two events in the sample space Ω with P(B)>0.

• Unconditional Probability (no additional information):

• Conditional Probability (given B has occurred):

• The conditional probability of A given B is defined as:

March 19, 2024 8

Example 2.1

• Problem: A fair six-sided die (faces numbered from 1 to 6) is rolled once. Define the
following events:
• Event A: “The outcome is odd.”
• Event B: “ The outcome is smaller than 4.”
Find:
1. P(A) — the probability that the outcome is odd.
2. P(B) — the probability that the outcome is smaller than 4.
3. P(A∣B) — the probability that the outcome is odd given that it is smaller than 4.
4. P(B∣A) — the probability that the outcome is smaller than 4 given that it is odd.

March 19, 2024 9

Example 2.1 – Solution

• The sample space Ω for rolling a fair six-sided die is:

• Event A (odd outcomes):
• Event B (outcomes smaller than 4):
• Intersection
• Calculate P(A) and P(B):

March 19, 2024 10

Example 2.1 – Solution (cont.)

• Calculate :

• Calculate : similarity

March 19, 2024 11

Example 2.2

• At a smartphone shop, among all customers who buy a new phone on the spot: 60% also
buy a screen protector; 40% buy a power bank and 30% buy both a screen protector and
a power bank.
• Consider randomly selecting a customer. Let:
o A is the event “the customer bought a screen protector,”;
o B is the event “the customer bought a power bank.”
• Questions:
1. Given that the selected customer bought a power bank, what is the probability
they also bought a screen protector?
2. Given that the selected customer bought a power bank, what is the probability
they did not buy a screen protector?

March 19, 2024 12

Example 2.2 – Solution

• Given:
1. Probability that they also bought a screen protector:
Since A and Ac
are mutually exclusive
and exhaustive (exactly
2. Probability that they did not buy a screen protector:
one must happen), their
conditional probabilities
given B must sum to 1:

March 19, 2024 13

Example 2.3

• A container holds a total of 25 light bulbs, which are categorized as follows:

• 5 good bulbs: These are in perfect working condition and will last for at least 30
days without failure.
• 10 partially defective bulbs: These initially work when tested, but they will fail
exactly on day 2.
• 10 totally defective bulbs: These do not light up at all when tested.
• A bulb is randomly chosen from the container. It is tested immediately, and it is
observed to light up during the first test.
• Question: Given that the selected bulb lights up when first tested, what is the
probability that it will still be functioning after 7 days (one week)?
March 19, 2024 14
Example 2.3 – Solution

• Let:
• G: bulb is good (lasts ≥ 30 days) → count = 5.
• P: bulb is partially defective (fails on day 2) → count =10.
• T: bulb is totally defective (won’t light) → count = 25−5−10 = 10.
• Prior probabilities:

• Hint:
• “Lights up initially” means the bulb is not totally defective, so it’s either good or
partially defective.
• The goal is to find the probability that the bulb is good given that it lights up
initially.
March 19, 2024 15
Example 2.3 – Solution (cont.)

• Condition on “bulb lights initially”: A bulb lights initially iff it is not totally defective,
event

• We want the probability the bulb still be working after one week given it lights initially:

• Among the bulbs that light at first (the 5 good + 10 partially defective), exactly 5 are
good. So, conditional on lighting initially, the chance it is still working after a week is 1/3
(≈ 33.3%).

March 19, 2024 16

Example 2.4

• A family has exactly two children (each child is equally likely to be male or
female at birth, independently). A telemarketer calls the family’s home, and one
of the two children answers the phone. We consider two distinct scenarios:
1. The child who answers the phone is male and says, “Let me get my older
sibling.”
Question: What is the probability that both children are male?
2. The child who answers the phone is male and says, “Let me get my other
sibling.”
Question: What is the probability that both children are male?

March 19, 2024 17

Example 2.4 – Solution
• Notation:
• Let M = male, F = female.
• We represent the gender of the two children as an ordered pair: (Older, Younger).
• The four equally likely combinations are:
• Each occurs with probability:
• Hint:
• When approaching such problems, identify the conditioning event carefully:
• Scenario 1 specifies exactly which child is male (the younger).
• Scenario 2 specifies only that at least one child is male.
• These two conditions eliminate different subsets of possible outcomes, which is why
the resulting probabilities differ.
March 19, 2024 18
Example 2.4 – Solution (cont.)

• Scenario 1: “Older sibling”

• We are told that the younger sibling is male (since he is the one answering and
refers to the older sibling).
• Given Younger = M, the possible combinations are: MM, FM.
• The conditional probability that both are male is:

• Intuition: Once we know the younger child is male, the gender of the older child
remains independent, with a 50% chance of being male.

March 19, 2024 19

Example 2.4 – Solution (cont.)

• Scenario 2: “Other sibling”

• We are told that the child who answered is male, but it could be either the older
or the younger child. This is equivalent to knowing that at least one child is
male.
• We can therefore eliminate the case FF, leaving: MM, MF, FM.
• The conditional probability that both are male is:

• Intuition: Knowing only that there is at least one male leaves three equally
likely possibilities, only one of which has two males.
March 19, 2024 20
Multiplication Rule

• In probability theory, the intersection of two events A and B, denoted A∩B, refers to the
event that both A and B occur at the same time.
• The general intersection formula comes from the definition of conditional probability:

• This means: The probability that both A and B happen equals the probability of one of
them happening multiplied by the conditional probability that the other happens given
that the first one did.
March 19, 2024 21
General Form of the Multiplication Rule

• Suppose we have n events in a sample space Ω,

• The joint probability that all n events occur is:

• Interpretation:
o First Term — Initial Event Probability: is the probability of the first event
happening without any prior conditions.
o Subsequent Terms — Conditional Probabilities: Each next term is the probability of
the next event occurring given that all previous events have already occurred.
o Sequential Dependence: This structure works whether the events are independent or
dependent; the conditional probabilities naturally adjust for dependencies.

March 19, 2024 22

Example 2.5

• Problem: A box contains 5 light bulbs, of which 2 are defective and 3 are good. A person
tests bulbs one by one without replacement until they find a good bulb. What is the
probability that the testing stops on the second trial?
• Interpretation: “Stops on the second trial” ⇔ the first bulb is defective and the second
bulb is good.
• Solution:
• Di : “The bulb tested in the i-th trial is defective.”
• Gi : “The bulb tested in the i-th trial is good.”

March 19, 2024 23

Outline

• Conditional Probability
• Law of Total Probability
• Introduction
• Partition of a Sample Space
• Theorem
• Bayes’ Theorem
• Independence

March 19, 2024 24

Introduction

• An insightful observation is that any event B can be thought of as having two

distinct parts:
• The part that also lies in A ( ), and
• The part that lies outside A ( ).
• This is because A and Ac are mutually exclusive events whose union covers the entire
sample space. Therefore, the outcomes in B must be found either in A or in Ac, and
these two subsets do not overlap.
• This leads to the following general identity:
• This observation is called the Law of Total Probability.

March 19, 2024 25

Partition of a Sample Space

• Let be non-empty events in a sample space Ω.

• These events are said to form a partition of Ω if they satisfy both of the following
conditions:
1. Exhaustive Coverage:

2. Pairwise Mutual Exclusivity:

For all i ≠ j, .

• Remark: For any event A such that and , the two events A and Ac (the
complement of A) always form a partition of Ω.
March 19, 2024 26
Theorem (Law of Total Probability)

• Let be non-empty events that form a partition of the sample space Ω.

Then, for any event

• By the definition of conditional probability:

• We can replace each of the terms above and get the (typically) more useful formula:

March 19, 2024 27

Example 2.6

• A bookstore fulfills online orders from two warehouses. 40% of the daily orders
are shipped from Warehouse A, and 60% are shipped from Warehouse B.
At Warehouse A, 8% of packages are damaged on arrival; at Warehouse
B, 10% of packages are damaged on arrival.
• If one package is selected at random from today’s deliveries, what is the
probability that it arrives undamaged?

March 19, 2024 28

Example 2.6 – Solution

• Notation:
• A and B: the package is shipped from Warehouse A and Warehouse B, respectively.
• D: package is damaged; Dc: package is not damaged.
• Given:

• Applying the Law of Total Probability:

March 19, 2024 29

Example 2.7

March 19, 2024 30

Outline

• Conditional Probability
• Law of Total Probability
• Bayes’ Theorem
• What is Bayes’ Theorem?
• Bayes’ Theorem with the Law of Total Probability
• Examples
• Independence

March 19, 2024 31

What is Bayes’ Theorem?

• The theorem is named after the 18th-century mathematician Thomas Bayes and forms
the cornerstone of Bayesian statistics.
• Bayes’ Theorem is a core principle in probability theory that describes how to update the
probability of an event when new, relevant information becomes available. It connects:
• Prior probability: the initial belief about the event before new evidence.
• Likelihood: the probability of observing the evidence if the event is true.
• Posterior probability: the updated belief after incorporating the evidence.
• Bayes’ Theorem provides a systematic method to reverse conditional probabilities—
allowing us to infer causes from observed effects.
• The adaptability to new evidence makes it an essential tool for decision-making under
uncertainty in science, business, and technology.
March 19, 2024 32
Bayes’ Theorem with the Law of Total Probability
• Let be events that form a partition of the sample space Ω (mutually
exclusive and exhaustive). Let B be another event with nonzero probability.
• General Case by Bayes’ Theorem:

• Using the Law of Total Probability to expand P(B):

• Interpretation of Terms:
• Posterior: P(A∣B) → The updated probability of A after observing B.
• Likelihood: P(B∣A) → The probability of observing B given that A is true.
• Prior: P(A) → The initial probability of A before seeing B.
March 19, 2024 33
Example 2.8

• A company operates three cab-call phone lines: A, B, and C. The connection

failure rates are 20% for A, 10% for B, and 30% for C. The lines are used with
the following frequencies: 60% for A, 30% for B, and 10% for C.
• A businessperson attempts to call and the connection fails. What is the
probability that they had used line A?
• Hints:
• A city has three cab-call phone lines: A, B, and C.
• Let F be the event “connection fails.” We want

March 19, 2024 34

Example 2.8 – Solution

• Usage (prior) probabilities:

• Failure (busy/no connection) rates:
• Total probability of a failure:

• Posterior for line A given failure

March 19, 2024 35

Example 2.9

• During a midterm, three students—X, Y, and Z—submitted unsigned exams. The

professor knows that the probabilities they produce a good exam are 0.8, 0.7,
and 0.5, respectively. After grading, he finds that exactly two of the unsigned
exams are good and one is bad. Assuming the students worked independently, what
is the probability that the bad exam was written by Z?
• Hints:
• GX, GY, GZ : “X, Y, Z wrote a good exam”.
• BX, BY, BZ are their complements.
• Let E = “exactly two good (and one bad)”, i.e., the multiset outcome GGB.
• We find
March 19, 2024 36
Example 2.9 – Solution

1. Numerator:
• If Z is the bad one, then both X and Y must be good:

• We also have:

• Hence:

March 19, 2024 37

Example 2.9 – Solution (cont.)

2. Denominator:
Exactly two good can happen in three mutually exclusive ways:
• X bad: BXGYGZ with probability 0.2 × 0.7 × 0.5 = 0.07.
• Y bad: GXBYGZ with probability 0.8 × 0.3 × 0.5 = 0.12.
• Z bad: GXGYBZ with probability 0.8 × 0.7 × 0.5=0.28.
Þ
3. P(bad exam is Z ∣ exactly two good):

March 19, 2024 38

Outline

• Conditional Probability
• Law of Total Probability
• Bayes’ Theorem
• Independence
• Definition
• Mutual Independence
• Conditional Independence

March 19, 2024 39

Independence

• Events A and B are independent iff any of the following equivalent conditions
hold:
1.
2.
3.
• Intuitively, means that knowing B does not tell us anything about
A. So, event B has no influence on event A.

March 19, 2024 40

Example 2.10

• Independent events:
• Flip a fair coin twice. Let A={first toss is Heads} and B={second toss is Heads}.

=> so A and B are independent.

• Not independent:
• Roll a fair die once. Let A={even}={2,4,6} and B={≥4}={4,5,6}.

=> so A and B are not independent.

March 19, 2024 41

Mutual Independence

• A collection is mutually independent if for every nonempty index

set , we have:

• They are pairwise independent if every pair of distinct events is independent; i.e., for
all i ≠ j:

• Relationship:
• Mutual independence ⇒ pairwise independence.
• Pairwise independence ⇏ mutual independence.
March 19, 2024 42
Example 2.11

• Pairwise independent but not mutually independent:

• Flip a fair coin twice. Define:
• A={first toss is H} = {HH, HT}.
• B={second toss is H} = {HH, TH}.
• C={exactly one head} = {HT, TH}.
• Each pair among A, B, C is independent, but

so the three events are not mutually independent.

March 19, 2024 43

Conditional Independence

• Events A and B are said to be conditionally independent given C iff any (hence all) of
the following equivalent statements hold:
1. No update given B once C is known:

2. No update given A once C is known:

3. Product form under C:

• Intuitively, after we learn that C occurred, learning B gives no extra information

about A (and vice versa).
March 19, 2024 44
Example 2.12
• A: The person has the flu.
• B: The person has a cough.
• C: Flu test result.
1.
Once the flu test result (C) is known, knowing whether the person has a cough (B)
does not change the probability of having the flu (A).
2.
Once the flu test result (C) is known, knowing whether the person actually has the flu
(A) does not change the probability of having a cough (B).
3.
Given the flu test result (C), having the flu (A) and having a cough (B) are
conditionally independent — the joint probability is the product of the individual
probabilities.

March 19, 2024 45

 Conditional Rule
Name of Rule Original Rule
(define for P(G) > 0)
First axiom of
probability

Complement rule

Chain rule

Bayes’ theorem

March 19, 2024 46

Exercises

1. I roll a fair six-sided die twice. Let X1 be the outcome of the first roll
and X2 the outcome of the second roll. If it is known that X1+X2=7, what is the
probability that either X1=4 or X2=4?
2. Two players, A and B, play the following game:
“They take turns drawing one ball at a time from a box containing 2 white
balls and 4 black balls. The balls are drawn without replacement. The
player who draws the first white ball wins the game.”
Assuming player A draws first, what is the probability that A wins?

March 19, 2024 47

Exercises

3. A student in a fixed-term (annual) program is allowed to retake the exam once

if they fail on the first attempt (the two attempts are independent). The
probabilities of passing the first and second attempts are 60% and 80%,
respectively. Find the probability that the student passes the course.
4. Two customers, A and B, independently place orders for a limited-edition book
from a store, with the probabilities of successfully purchasing being 0.8 and 0.7,
respectively. Given that at least one customer succeeds in purchasing the book,
find the probability that customer A is the one who gets it.

March 19, 2024 48

Exercises

5. Consider a family with two children. We are interested in the genders of the children.
The sample space is:

where G denotes a girl and B denotes a boy. Assume that all four outcomes are equally
likely.
a. What is the probability that both children are girls, given that the first child is a
girl?
b. Suppose we instead ask the father: “Do you have at least one daughter?” and he
answers “Yes”. Given this additional information, what is the probability that both
children are girls?

March 19, 2024 49

References
[1] Michael Baron. (2014). Probability and Statistics for Computer Scientists, Second
Edition (2nd. ed.). Chapman & Hall/CRC.
[2] Molina Peralta, I. and García-Portugués, E. (2024). A First Course on Statistical
Inference. Version 2.4.1. ISBN 978-84-09-29680-4. Available
at [Link]
[3] Sheldon M. Ross. (2009). Introduction to probability and statistics for engineers and
scientists (4th. ed.). Elsevier.
[4] Marco Taboga. (2021). Lectures on probability theory and mathematical statistics.
Kindle Direct Publishing. Online appendix.
[5] Hossein Pishro-Nik. (2014). Introduction to probability, statistics, and random processes.
available at [Link] Kappa Research LLC.
March 19, 2024 50