Probability
Welcome: An Introduction to the Mathematics of Uncertainty
What is Probability?
• Probability is the branch of mathematics that quantifies uncertainty by assigning a number between 0
and 1 to the likelihood of an event occurring.
• This powerful mathematical tool helps us make sense of uncertain situations and guide rational
decision-making across countless fields.
• From predicting weather patterns to analyzing financial markets, from medical diagnoses to quality
control in manufacturing, probability theory provides the foundation for understanding and working with
uncertainty in our complex world.
Fundamental Probability Concepts
Understanding probability begins with grasping several key concepts that form the foundation of all probability calculations. These
principles govern how we assign, calculate, and interpret probability values in any uncertain situation.
Sample Space Event Probability Definition
The set of all possible outcomes in a Any subset of the sample space. An The ratio of favorable outcomes to
random experiment. For a coin flip, event can be simple (one outcome) or total possible outcomes within the
the sample space is {Heads, Tails}. compound (multiple outcomes). sample space, expressed as P(E) =
For a die roll, it's {1, 2, 3, 4, 5, 6}. Rolling an even number on a die is a Number of favorable outcomes / Total
compound event. outcomes.
Essential Rule: The sum of probabilities of all mutually exclusive outcomes in a sample space always equals 1. This is
because one of the possible outcomes must occur.
Three Approaches to Assigning Probability
Probability can be determined through three distinct approaches, each suited to different types of situations and available
information. Understanding when and how to apply each method is crucial for accurate probability analysis.
1 Classical Probability 2 Empirical Probability 3 Axiomatic Probability
Used when all outcomes are Based on observed relative Based on Kolmogorov's three
equally likely. This theoretical frequencies from actual fundamental axioms that provide
approach assumes perfect experiments or historical data. This the mathematical foundation for all
conditions and symmetry in the approach becomes more accurate probability theory:
random process. with larger sample sizes. • P(E) ≥ 0 for any event E
Example: If a coin lands heads 45 • P(S) = 1, where S is the sample
times in 100 flips, the empirical space
probability of heads is P(Heads) =
• For mutually exclusive events:
Example: Rolling a 4 on a fair six- 45/100 = 0.45.
P(∪Eᵢ) = ΣP(Eᵢ)
sided die has probability 1/6, since
there's 1 favorable outcome out of
6 equally likely possibilities.
The Addition Theorem
Understanding Union of Events
The Addition Theorem calculates the probability of either event A or event B (or both) occurring. This is
fundamental when dealing with multiple possible outcomes.
For any two events A and B:
Key Insight: We subtract P(A ∩ B) to avoid double-counting the overlap when both events occur simultaneously.
Special Case: Mutually Exclusive Events
When events cannot happen together, P(A ∩ B) = 0, so:
Example: Die Rolling
What's the probability of rolling a 2 or 3 on a fair die?
• P(2) = 1/6
• P(3) = 1/6
The Multiplication Theorem
The Multiplication Theorem determines the probability that both events A and B occur together. The calculation depends on whether the events are independent or dependent.
Independent Events Dependent Events
When one event doesn't affect the other: When one event influences the other:
Example: Rolling two dice - the first roll doesn't influence the second. Example: Drawing cards without replacement - first draw affects second.
Detailed Example: Drawing Without Replacement
From a box containing 5 red balls and 3 blue balls, what's the probability that both balls drawn (without replacement) are red?
First draw: P(red) = 5/8
Second draw: P(red|first red) = 4/7
Both red: P = (5/8) × (4/7) = 20/56 = 5/14
Conditional Probability and Bayes' Theorem
Conditional Probability 1
The probability of event A occurring given that event B has
already occurred:
2 Bayes' Theorem
Updates probability based on new evidence:
Example: In a class of 100 students with 40 girls, if 25 girls like
math, then P(Math|Girl) = 25/40 = 0.625
This theorem is revolutionary in statistics, allowing us to revise our
beliefs when new information becomes available.
Bayes' Theorem in Action
Consider two bags: Bag 1 has 4 white and 6 black balls; Bag 2 has 4 white and 3 black balls. A black ball is drawn randomly. What's the probability it
came from Bag 1?
01 02
Step 1: Calculate P(Black) = 0.5 × 0.6 + 0.5 × (3/7) = 0.5145 Step 2: Apply Bayes: P(Bag1|Black) = (0.5 × 0.6)/0.5145 ≈ 0.584
Essential Probability Rules
Complement Rule Law of Total Probability
The probability that an event does NOT occur: When events B₁, B₂, ... Bₙ form a partition:
Example: If the probability of rain is 0.3, then P(no rain) Example: In a school with 60% female students (80%
= 1 - 0.3 = 0.7 pass rate) and 40% male students (60% pass rate):
This rule is particularly useful when it's easier to P(Pass) = 0.6 × 0.8 + 0.4 × 0.6 = 0.72
calculate the probability of the opposite event.
Memory Tip: The complement rule is your friend when dealing with "at least one" problems. Instead of calculating P(at
least one), calculate P(none) and subtract from 1!
Random Variables and Probability Distributions
A random variable is a function that assigns numerical values to each outcome of a random experiment. This concept bridges the gap between abstract probability spaces and practical numerical analysis.
Types of Random Variables
Discrete: Countable values (number of heads in coin tosses, number of defective items)
Continuous: Any value in a range (height, weight, temperature)
Probability Distribution Properties
A probability distribution shows how probabilities are distributed over all possible values of a random variable. For discrete random
variables:
• Each probability is between 0 and 1
• Sum of all probabilities equals 1
• Provides complete description of the variable's behavior
Expected Value: The Long-Run Average
Expected value represents the average outcome you'd expect if you repeated a random experiment many times. It's a crucial measure for decision-making under uncertainty.
For a discrete random variable X:
Central Tendency Decision Making Prediction Tool
Expected value provides the central measure that Expected value is fundamental in business, economics, Helps predict long-term outcomes and establish
describes the "typical" value of a random variable, even and statistics for comparing alternatives and making benchmarks for performance evaluation in uncertain
though the actual value may never equal the expected optimal choices under uncertainty. situations.
value.
Calculating Expected Value: Fair Die Example
For a fair six-sided die, each outcome (1, 2, 3, 4, 5, 6) has probability 1/6:
Notice that 3.5 is never actually rolled, but it represents the average value over many rolls.
Key Takeaway: Probability theory provides the mathematical framework for quantifying uncertainty and making informed decisions. From basic concepts to advanced theorems, these
tools are essential for understanding our uncertain world.