9/8/25, 1:03 PM Probability

Probability
AUTHOR
Nalini Ravishanker

A study of probability deals with a numerical description of how likely an event is to occur, or a proposition is
true. The higher the probability of an event, the more likely it is that the event will occur.

We can talk about objective probability versus subjective probability.

Objective probability
Objective probability often uses a relative frequency interpretation. A probability is assigned to a possible
outcome on the basis of what is likely to happen over the long run when we repeatedly observe a random
circumstance. We then define the probability of a specific outcome as the proportion of times it would occur
over the long run (i.e., the relative frequency of that particular outcome).

Example 1: Male and Female births. The U.S. Centers for Disease Control and Prevention says that the long-
run relative frequency of male births in the U.S. is about 0.512. That is, in the long run, 512 male babies and 488
female babies are born per 1000 births. See [Link]

Example 2: Winning a lottery. Consider a lottery in which you would choose a three-digit number between
000 and 999, and you win if your number is picked. You reckon that since there are 1000 possible three-digit
numbers, the probability that the state picks your number is , so in the long run, a player should win about
1

1000

1 out of 1000 times.

Subjective probability
Subjective probability can be regarded as a degree of belief, and incorporates expert knowledge as well as
experimental data to produce probabilities. For instance, as an expert on your own life, you probably would
find it easy to give a probability for an event such as whether you will watch a movie sometime in the next
week, or buy a house in the next 3 years.

Example 3: Sports Season Victory. Suppose we ask NY Giants fans before the football season starts, about the
chances that their team will win that season. When fans give us numbers ranging from 25% to 68%, they are
giving their subjective probabilities as percentages.

The notion of probability is to specify a model for experiments and observational studies which admit random
outcomes. Probability theory is based on:

an underlying collection S of all possible outcomes of an experiment, and

a rule P that assigns a number between zero and one to each subset of the sample space satisfying
certain rules.

We first define some basic ideas about a sample space and events.

Sample Space and Events

Definition 1: Sample space. The set of all possible outcomes is called the sample space of an experiment,
denoted by S .

[Link] 1/4
9/8/25, 1:03 PM Probability

Example 4: Toss a single coin. Outcomes are Heads (H ) and Tails (T ). The sample space S has two elements,
i.e., S = {H , T }.

Example 5: If there are multiple tosses of the coin (or one toss of many coins), the outcomes are sequences of
Heads and Tails. For example, if we toss two coins, S = {H H , H T , T H , T T }.

Definition 2: Simple events. The elements of the sample space or population are the outcomes or simple
events or sample points.

In Example 4 (tossing a single coin), H and T are the two possible outcomes or simple events.

In Example 5 (two coin tosses), the four possible outcomes (simple events) are H H , H T , T H , and T T .

Definition 3: Events. An event is defined as a subset of the sample space S . We denote events by A, B, C , etc.

In Example 5, an event “occurrence of at least one Tail” is A = {T T , H T , T H } . An event defined as

“occurrence of two Tails” is B = {T T } . An event defined as the “occurrence of two Heads” is C .
= {H H }

An event B is defined to be a subset of an event A if B is contained in A, i.e., every outcome in B is also in A.

We denote by B ⊂ A B ( is a subset of A) or A ⊃ B A ( is a superset of B).

In Example 5, since A = {T T , H T , T H } , then B = {T T } is a subset of A.

Two events are said to be equal if each is a subset of the other.

Definition 4: Complement of an Event. The set that contains all the outcomes in S which are not contained in
A is called the complement of A and is denoted by A . c

In Example 4, the complement of H is T .

In Example 5, the complement of A = {T T , H T , T H } is C = {H H } = A

c
.

Definition 5: Union of Events. The set containing outcomes belonging either to A or B or to both is called the
union of and is denoted by A ∪ B.

In Example 5, since A = {T T , H T , T H } and B = {T T } , then A ∪ B = {T T , H T , T H } . Since

C = {H H } , then A ∪ C = {T T , H T , T H , H H } = S .

Definition 6: Intersection of Events. The set containing outcomes belonging to both A and B is called the
intersection of A and B, and is denoted by A ∩ B.

In Example 5, A ∩ B = {T T } . Also, let D denotes the set with one Tail, i.e., D = {H T , T H }, then
A ∩ D = {H T , T H } , while C ∩ D has no outcomes, i.e, C ∩ D = ∅ (the empty set).

Definition 7: Disjoint or Mutually Exclusive Events. If C ∩ D = ∅ , the events C and D are said to be disjoint,
or mutually exclusive, and the occurrence of one of the events precludes the occurrence of the other.

Probability Measure
The probability measure is a function defined on the class of all events satisfying the following conditions
(axioms):

Axiom 1. The probability of any event A is non-negative and is at most 1:

[Link] 2/4
9/8/25, 1:03 PM Probability
0 ≤ P (A) ≤ 1 (1)

Axiom 2. The probability of the entire sample space is 1.

P (S) = 1. (2)

Axiom 3. If the probability of the occurrence of an event A is P (A), the probability that A will not occur is
c
P (A ) = 1 − P (A).

Hence, P (∅) = 1 − P (S) = 0 .

Axiom 4. Special rule of addition (Countable Additivity): If A and B are disjoint (or, mutually exclusive) events,
the probability that at least one of them will occur is

P (A ∪ B) = P (A) + P (B). (3)

In Example 4, if we toss a fair coin, the sample space is S = {H , T } . The probabilities of the two outcomes
are

1 1
P (H ) = and P (T ) = . (4)
2 2

P (S) = P ({H , T }) =
1
2
+
1
2
= 1 .

In Example 5, if we toss two fair coins, with S = {H H , H T , T H , T T } . Probabilities of the outcomes are

1
P (H H ) = P (H T ) = P (T H ) = P (T T ) = . (5)
4

P (S) = P ({H H , H T , T H , T T }) =
1
4
+
1
2
+
1
4
= 1 .

We defined these events: A ,

= {T T , H T , T H } B = {T T } C = {H H } , , and D .
= {H T , T H }

P (A) =
3

4
, P (B) =
1

4
, P (C) =
1

4
, and P (D) =
1

2
.

If C = {H H } , and D = {H T , T H } , they are disjoint events, so

P (C ∪ D) = P (C) + P (D) =
1

4
+
1

2
=
3

4
.

Note: Axiom 4 can be extended to n events:

If A 1, A2 , … , An are mutually exclusive events, the probability that at least one of them will occur is

P (A 1 ∪ A 2 ∪ … ∪ A n ) = P (A 1 ) + P (A 2 ) + … + P (A n ). (6)

Axiom 4 also holds for an infinite number of events. If A 1, A2 , … are mutually exclusive events, the
probability that at least one of them will occur is
∞

∞
P (∪ i=1 A i ) = ∑ P (A i ).

i=1

More general form of additivity

Let A and B be two events (we do not assume they are disjoint). Then,

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

[Link] 3/4
9/8/25, 1:03 PM Probability

In Example 5, A ∩ B = {T T } , and P (A ∩ B) =
1
4
. Then,

3 1 1 3
P (A ∪ B) = P (A) + P (B) − P (A ∩ B) = + − = .
4 4 4 4

Example 6: Customer Insurance. Suppose that 60% of all customers of a large insurance agency have
automobile (A) policies with the agency, 40% have homeowner’s (H) policies, and 25% have both types of
policies.

Let E be the event that Customer has Insurance A, and F be the event that Customer Insurance H. Suppose
P (E) = 0.60, P (F ) = 0.40, P (E ∩ F ) = 0.25 . Find the following probabilities.

a. Probability that a customer has at least one policy:

P (E ∪ F ) = P (E) + P (F ) − P (E ∩ F ) = 0.75 .

b. Probability that a customer has exactly one policy:

P(Exactly one) = P (E ∪ F ) − P (E ∩ F ) = 0.50

c. Probability that a customer has neither policy:

c
= P ((E ∪ F ) ) = 1 − P (E ∪ F ) = 0.25

Note: Another useful type of probability is conditional probability, which we will look at in a later unit.

[Link] 4/4