Chapter 4
Basic Probability
Learning Objectives
• In this chapter, you learn:
- Basic probability concepts
- Conditional probability
- To use Bayes’ Theorem to revise probabilities
Basic Probability Concepts
• Probability – the chance that an uncertain
event will occur (always between 0 and 1)
• Impossible Event – an event that has no
chance of occurring (probability = 0)
• Certain Event – an event that is sure to occur
(probability = 1)
Assessing Probability
• There are three approaches to assessing the
probability of an uncertain event:
Example of a priori probability
Example of empirical probability
Events
• Each possible outcome of a variable is an event.
• Simple event
- An event described by a single characteristic e.g., A red
card from a deck of cards
• Joint event
- An event described by two or more characteristics
- e.g., An ace that is also red from a deck of cards
• Complement of an event A (denoted A’)
- All events that are not part of event A
- e.g., All cards that are not diamonds
Sample Space
• The Sample Space is the collection of all
possible events e.g. All 6 faces of a die:
Visualizing Events
Visualizing Events
Definitions Simple vs. Joint Probability
• Simple Probability refers to the probability of
a simple event.
- ex. P(King) ex. P(Spade)
• Joint Probability refers to the probability of an
occurrence of two or more events (joint
event).
- ex. P(King and Spade)
Mutually Exclusive Events
• Mutually exclusive events
- Events that cannot occur simultaneously
Example: Drawing one card from a deck of
cards A = queen of diamonds; B = queen of
clubs
• Events A and B are mutually exclusive
Collectively Exhaustive Events
• Collectively exhaustive events
- One of the events must occur
- The set of events covers the entire sample space
example: A = aces; B = black cards; C = diamonds; D =
hearts
• Events A, B, C and D are collectively exhaustive (but
not mutually exclusive – an ace may also be a heart)
• Events B, C and D are collectively exhaustive and
also mutually exclusive
Computing Joint and Marginal Probabilities
Joint Probability Example
Marginal Probability Example
Marginal & Joint Probabilities In A
Contingency Table
Probability Summary So Far
• Probability is the numerical measure of the likelihood
that an event will occur
• The probability of any event must be between 0 and
1, inclusively
• The sum of the probabilities of all mutually exclusive
and collectively exhaustive events is 1
General Addition Rule
General Addition Rule Example
Computing Conditional Probabilities
• A conditional probability is the probability of one event, given that
another event has occurred:
• Where P(A and B) = joint probability of A and B P(A) = marginal or simple
probability of A P(B) = marginal or simple probability of B
Conditional Probability Example
• What is the probability that a car has a CD
player, given that it has AC ?
i.e., we want to find P(CD | AC)
Conditional Probability Example
Conditional Probability Example
Using Decision Trees
Using Decision Trees
Multiplication Rules
Independence
• Two events are independent if and only if:
• Events A and B are independent when the
probability of one event is not affected by the
fact that the other event has occurred
Multiplication Rules
Marginal Probability
• Marginal probability for event A:
• Where B1, B2, …, Bk are k mutually exclusive
and collectively exhaustive events
Bayes’ Theorem
• Bayes’ Theorem is used to revise previously
calculated probabilities based on new
information.
• Developed by Thomas Bayes in the 18th
Century.
• It is an extension of conditional probability.
Bayes’ Theorem
• where: Bi = ith event of k mutually exclusive
and collectively exhaustive events A = new
event that might impact P(Bi )
Bayes’ Theorem Example
• A drilling company has estimated a 40% chance of
striking oil for their new well.
• A detailed test has been scheduled for more
information. Historically, 60% of successful wells have
had detailed tests, and 20% of unsuccessful wells
have had detailed tests.
• Given that this well has been scheduled for a
detailed test, what is the probability that the well will
be successful?
Bayes’ Theorem Example
• Let S = successful well U = unsuccessful well
• P(S) = 0.4 , P(U) = 0.6 (prior probabilities)
• Define the detailed test event as D
• Conditional probabilities:
P(D|S) = 0.6 P(D|U) = 0.2
• Goal is to find P(S|D)
Bayes’ Theorem Example
• Apply Bayes’ Theorem:
• So the revised probability of success, given that
this well has been scheduled for a detailed test, is
0.667
Bayes’ Theorem Example
•
Chapter Summary
• Discussed basic probability concepts
- Sample spaces and events, contingency tables, Venn
diagrams, simple probability, and joint probability
• Examined basic probability rules
- General addition rule, addition rule for mutually
exclusive events, rule for collectively exhaustive events
• Defined conditional probability
- Statistical independence, marginal probability, decision
trees, and the multiplication rule
• Discussed Bayes’ theorem