Scribd
Upload
26 views3 pages

Understanding the Sum Rule in Probability

The document discusses three key concepts in probability: 1. The sum rule, which states that the marginal probability of an event is equal to the sum of its joint probabilities. This allows calculating individual probabilities from a table of joint probabilities. 2. Conditional probability, which is the probability of one event given that another event has occurred. It is calculated as the number of relevant outcomes divided by the total remaining outcomes. 3. The product rule, which relates joint, marginal, and conditional probabilities using the formula that conditional probability is equal to the joint probability divided by the probability of the conditioning event. The product rule provides an alternative definition of independent events.

Uploaded by

Ignacio Labaqui
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
26 views3 pages

Understanding the Sum Rule in Probability

The document discusses three key concepts in probability: 1. The sum rule, which states that the marginal probability of an event is equal to the sum of its joint probabilities. This allows calculating individual probabilities from a table of joint probabilities. 2. Conditional probability, which is the probability of one event given that another event has occurred. It is calculated as the number of relevant outcomes divided by the total remaining outcomes. 3. The product rule, which relates joint, marginal, and conditional probabilities using the formula that conditional probability is equal to the joint probability divided by the probability of the conditioning event. The product rule provides an alternative definition of independent events.

Uploaded by

Ignacio Labaqui
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Data Science Math Skills

Paul Bendich and Daniel Egger


Duke University

Problem Solving Methods:


The Sum Rule, Conditional Probability,
and the Product Rule
Video companion

1 Marginal probabilities and the sum rule


Often know the joint probabilities, but don’t know individual probabilities.

Table of known joint probabilities:


(X, Y ) X
x1 x2 x3
y1 P (x1 , y1 ) P (x2 , y1 ) P (x3 , y1 )
0.01 0.02 0.03
Y y2 P (x1 , y2 ) P (x2 , y2 ) P (x3 , y2 )
0.10 0.20 0.49
y3 P (x1 , y3 ) P (x2 , y3 ) P (x3 , y3 )
0.04 0.05 0.06

Can refer to P (x1 ) as the “marginal probability of x1 ” because it is in the margins of the
matrix.

Sum rule: The marginal probability is equal to the sum of the joint probabilities.

For x1 , this means:

P (x1 ) = P (x1 , y1 ) + P (x1 , y2 ) + P (x1 , y3 )


= 0.01 + 0.10 + 0.04 = 0.15

and for y2 :

P (y2 ) = P (x1 , y2 ) + P (x2 , y2 ) + P (x3 , y2 )


= 0.10 + 0.20 + 0.49 = 0.79

1
Data Science Math Skills
Paul Bendich and Daniel Egger
Duke University

Sum rule for binary probability distribution:

P (A) = P (A, B) + P (A, ∼B)

Sum rule for series of n probabilities:

P (A) = P (A, B1 ) + P (A, B2 ) + ... + P (A, Bn )

2 Conditional probability
Definition:
conditional probability—the probability that a statement is true given that some other state-
ment is true with certainty.

Symbol P (A | B) means the “probability of A given that B is true with certainty.”

Example: What is the probability of rolling a 3 on a 6-sided die, given that the roll is odd?
Of the outcomes when the roll is odd (3), one is the relevant outcome, so the probability is
1/3.

Example: What is the probability of rolling an odd, given that the roll is a 3? Of the
outcomes when the roll is a 3 (1), one is the relevant outcome, so the probability is 1.

Formula for conditional probability:

(relevant outcomes)
P (A | B) =
(total outcomes remaining in universe, when B is true)

3 Product rule
Want to relate concepts of joint probability, marginal probability, and conditional probability.

Product rule:
P (A, B)
P (A | B) =
P (B)

Conditional probability of A given that B is true is equal to the joint probability that A and
B are true, divided by the probability that B is true.

2
Data Science Math Skills
Paul Bendich and Daniel Egger
Duke University

Old definition of independence:

P (A, B) = P (A)P (B)

Dividing by P (B) gives

P (A, B)
P (A) =
P (B)

Using the product rule gives another definition of independence.

New definition of independence:

P (A | B) = P (A)

Intuitively, this means knowing that B is true tells us nothing about the probabilities of A.
The outcome B has no effect on A; therefore, they are independent.

Conversely, if P (A | B) 6= P (A), then the events are dependent.

You might also like