Bayesian

Notation
A set is a collection of elements, which hold certain values. Additionally, every event has a
set of outcomes that satisfy it.
The null-set (or empty set), denoted ∅, is an set which contain no values.

𝑥
∈𝐴 Set (upper-
case)
Element (lower-case)
𝑥∈𝐴 2 ∈ 𝐴𝑙𝑙 𝑒𝑣𝑒𝑛
Notation: Interpretation: Example:

𝑛𝑢𝑚𝑏𝑒𝑟𝑠
“Element x is a part of set A.”

A∋𝑥 𝐴𝑙𝑙 𝑒𝑣𝑒𝑛 𝑛𝑢𝑚𝑏𝑒𝑟𝑠 ∋

2
“Set A contains element x.”

𝑥∉𝐴 1 ∉ 𝐴𝑙𝑙 𝑒𝑣𝑒𝑛

𝑛𝑢𝑚𝑏𝑒𝑟𝑠
“Element x is NOT a part of set

∀𝑥: ∀𝑥: 𝑥 ∈ 𝐴𝑙𝑙 𝑒𝑣𝑒𝑛

A.”

𝑛𝑢𝑚𝑏𝑒𝑟𝑠
“For all/any x such that…”

𝐴⊆𝐵 𝐸𝑣𝑒𝑛 𝑛𝑢𝑚𝑏𝑒𝑟𝑠 ⊆

Remember! Every set has at least 2

• 𝐴⊆𝐴 𝐼𝑛𝑡𝑒𝑔𝑒𝑟𝑠
subsets. “A is a subset of B”

• ∅⊆𝐴
 Multiple
Events
The sets of outcomes that satisfy two events A and B can interact in one of the
following 3 ways.

One completely
Not touch at Intersect (Partially
overlaps the
all. Overlap)
other.

Examples:
A -> Diamon Red
Diamonds B ds Cards
-> Hearts Queens Diamon
d
 Intersection
The intersection of two or more events expresses the set of outcomes that
satisfy all the events simultaneously. Graphically, this is the area where the sets
intersect.

We denote the
intersection of two
sets with the
“intersect” sign,
which resembles
an upside-down
capital letter U:

𝑨∩𝑩
 Union
The union of two or more events expresses the set of outcomes that satisfy at least
one of the events. Graphically, this is the area that includes both sets.

𝑨
𝖴𝑩

𝑨𝖴𝑩=𝑨+𝑩
−𝑨∩𝑩
 Mutually Exclusive
Sets
Sets with no overlapping elements are called mutually exclusive. Graphically, their
circles never touch.

If 𝑨 ∩ 𝑩 = ∅,
then the two sets
are mutually
exclusive.

Remember:
All complements are mutually exclusive, but not all mutually exclusive sets are complements.

Example:
Dogs and Cats are mutually exclusive sets, since no species is simultaneously a feline and a canine, but the two
are not complements, since there exist other types of animals as well.
 Independent and Dependent
Events
If the likelihood of event A occurring (P(A)) is affected event B occurring, then we say that A and B are
dependent events. Alternatively, if it isn’t – the two events are independent.

We express the probability of event A occurring, given event B has occurred the following way 𝑷 𝑨 𝑩
.
We call this the conditional probability.

Independe Depende
nt: nt
• All the probabilities we have examined so far. • New concept.
• The outcome of A does not depend on the • The outcome of A depends on the
outcome of B. outcome of B.
• 𝑃 𝐴 𝐵 = 𝑃(𝐴) • 𝑃 𝐴 𝐵 ≠ 𝑃(𝐴)

Example
Examp
• A -> le
• A ->
Hearts
Hearts
• B -> Jacks
• B -> Red
 Conditional
Probability
For any two events A and B, such that the likelihood of B occurring is greater > 0), the conditional
than 0 (𝑃 𝐵 probability

𝑃 𝐴
formula states the following.

𝑃 ∩𝐵
Probability of

= 𝑃(𝐵
the

𝐴𝐵
Probability of intersection.

)
A, given B
has occurred
Probability
of event B
Intuition behind the formula: Rememb
er:
• Only interested in the outcomes where B is • Unlike the union or the intersection, changing
satisfied. the order of A and B in the conditional
probability alters its meaning.
• 𝑃 𝐴 𝐵 is not the same as 𝑃 𝐵 𝐴 , even
• Only the elements in the intersection would satisfy

if 𝑃 𝐴 𝐵 =
A as well.
• Parallel to the “favoured over all” formula: 𝑃(𝐵|𝐴) numerically.
• Intersection = “preferred outcomes” • The two conditional probabilities possess
• B = “sample space” different meanings even if they have equal
values.
 Law of total
probability
The law of total probability dictates that for any set A, which is a union of many mutually exclusive sets 𝐵1, 𝐵2, … , 𝐵𝑛,
its probability equals the
following sum.

𝑃 𝐴 = 𝑃𝐴 𝐵1 × 𝑃 𝐵1 + 𝑃𝐴 𝐵2 × 𝑃 𝐵2 + ⋯ + 𝑃
𝐴 𝐵𝑛
Conditional ×𝑃 𝐵𝑛
Conditional
A, given 𝐵1 of 𝐵2
Probability of Probability
of 𝐵1
Probability
𝐵
Probability of
has
A,
given 2
occurring. occurring.
occurre
has
occurred.
d.
Intuition behind the formula:
• Since P(A) is the union of mutually exclusive sets, so it equals the sum of the
individual sets.

to get 𝑷 𝑨 ∩ 𝑩 = 𝑷 𝑨 𝑩
• The intersection of a union and one of its subsets is the entire subset.
𝑷
= 𝑷
formula 𝑷 𝑨|𝑩 ×𝑷 𝑩 .
• We can rewrite the conditional probability
𝑨∩𝑩𝑩

• Another way to express the law of total

probability is:
 Additive
Law
The additive law calculates the probability of the union based on the probability of the individual

𝑃𝐴 𝖴 𝐵 + 𝑃 𝐵 − 𝑃(𝐴
sets it accounts for.

=𝑃 𝐴 ∩ 𝐵)

Probability of Probability of
the union the
intersection
Intuition behind the formula
• Recall the formula for finding the size of the Union using the size of the
Intersection:
• 𝐴𝖴𝐵=𝐴+𝐵−𝐴∩𝐵
• The probability of each one is simply its size over the size of the sample
space.
• This holds true for any events A and B.
 The Multiplication
Rule
The multiplication rule calculates the probability of the intersection based on the
conditional probability.

𝑃 𝐴∩𝐵 = 𝑃𝐴
𝐵 × 𝑃(𝐵)
Probability of Condition Probability
the al of event B
Intersection Probabilit
y
Intuition behind the formula
𝑃
= 𝑃
formula 𝑃 𝐴|𝐵 𝑃 𝐵
• We can multiply both sides of the conditional probability by to
𝐴∩𝐵𝐵
𝑃 𝐴∩𝐵 =𝑃 𝐴 𝐵 ×𝑃 𝐵 .
get
event B occurs in 40% of the time (𝑃 𝐵 = 0.4) and event A occurs in 50% of the
time B occurs (𝑃 𝐴|𝐵 = 0.5), then they would simultaneously occur in 20% of the time
• If

(P A 𝐵 × 𝑃 𝐵 = 0.5 × 0.4 = 0.2).

 Bayes’
Law
Bayes’ Law helps us understand the relationship between two events by computing the different
conditional probabilities. We also call it Bayes’ Rule or Bayes’ Theorem.

Conditional
probability of B,

𝑃 𝐵|𝐴 × 𝑃
given A.

𝑃 𝐴|𝐵 𝐴
Conditional

=
probability

𝑃𝐵
of A, given
B.

Intuition behind the formula

• According to the multiplication rule 𝑃 𝐴 ∩ 𝐵 =𝑃 𝐴 𝐵 × 𝑃 𝐵 , so 𝑃 𝐵 ∩ 𝐴=
𝑃 𝐵 𝐴 ×𝑃 𝐴 .
𝑃
= 𝑃(𝐵
• Since 𝑃 𝐴 ∩ 𝐵 = 𝑃(𝐵 ∩ 𝐴), we plug in 𝑃 𝐵 𝐴 × 𝑃 𝐴 for 𝑃 𝐴 ∩ 𝐵 𝐴∩𝐵)
formula 𝑃 𝐴 𝐵
in the
• conditional
Bayes’ Law probability
is often used in medical or business analysis to determine which of two symptoms affects
the other one more.