CHAPTER 5

Elementary Probability
“Life is a school of probability.”
Walter Bagehot
Definitions of Some concepts of Probability Terms
• Probability
• The chance of an event occurring.
• Probability is the basis of inferential statistics.
• Probability experiment: It is a process that leads to well-defined results called
outcomes.
• For example, flipping a coin once, rolling one die once, or the like.
• Outcomes: It is the result of a single trial of probability experiment.
• It is sometimes called sample point.
• Sample Space: It is the set of all possible outcomes of a probability
experiment and denoted by𝑆 𝑜𝑟 Ω.
• Event: It is a subset of sample space (contains one or more outcomes which
are in the sample space) and is defined for a particular purpose.
• An event can be one outcome or more than one outcome.
• Simple event is an event having only single outcome.
• Compound event consisting of one or more outcomes or simple events.
• Event is denoted by capital letters such as A, B, F etc.
Definitions of Some concepts of Probability Terms
• Mutually exclusive events: Suppose you have two events, say A and
B. if these events have no common sample point(s) or do not occur
simultaneously, then the two events are called mutually exclusive
events.
• Exhaustive events: It is a satiation where the events contain all
elements based on the definition of the events.
• Union of events: The union of two events A and B, denoted by𝐴 ∪ 𝐵,
consists of all outcomes that are in A or in B or both A and B.
• Intersection of events: The intersection of event A and B, denoted
by𝐴 ∩ 𝐵, consists of all outcomes that are in both A and B.
• Compliment of an event: The compliment of event A, denoted
by𝐴𝑐 𝑜𝑟 𝐴′ , consists of all outcomes that are not in A.
Definitions of Some concepts of Probability Terms
• Null event: The event containing no outcomes. It is the compliment
of the sample space.
• Probability of an event: The probability of event A, denoted by𝑃 𝐴 ,
is the probability the outcome of the experiment is contained in A.
• Equally-likely events: It is a situation where the probability of the
occurrence of one event as likely as the other event.
• That is, they must have equal probability of occurrence.
• Independent events: Two events said to be independent if knowing
whether a specific one has occurred does not change the probability
that the other occurs.
Counting Rules
• In order to probabilities, we have to know
• The elements of an event.
• The number of elements of the sample space.
• i.e. In order to judge what is probable, we have to
know what is possible.
• In order to determine the number of outcomes one
can use several rules of counting.
• Addition rule
• Multiplication rule
• Permutation rule
• Combination rule
Counting Rules
• Addition Rule
• If the choices can’t be performed together then the
number of ways in which you can make a choice
in 𝑛1 +𝑛2 + 𝑛3 + ⋯ + 𝑛𝑘 different ways.
• E.g: If there are two way of bus to voyage Debre
Derhan from Addis Ababa and three railways, then
collectively we have 2 + 3 = 5 different way to
arrive Debre Berhan.
Counting Rules

• Multiplication (Fundamental) Rule

• In sequence of 𝑛 events in which the first one has 𝑘1 possibilities and
the second event has 𝑘2 and the third has 𝑘3 , and so forth, the total
number of possibilities of the sequence will be 𝑘1 . 𝑘2 . 𝑘3 … 𝑘𝑛 .
• E.g1: How many different 7-place license plates are possible if the
first 3 places are to be occupied by letters and the final 4 by
numbers?
• E.g2: In the above example, how many license plates would be
possible if repetition among letters or numbers were prohibited?
Counting Rules

• Permutation Rule
• How many different ordered arrangements of letters 𝑎, 𝑏, 𝑐 are
possible?
• By direct enumeration we see that there are 6: namely,
𝑎𝑏𝑐, 𝑎𝑐𝑏, 𝑏𝑎𝑐, 𝑏𝑐𝑎, 𝑐𝑎𝑏 and 𝑐𝑏𝑎.
• Each arrangement is known as a permutation.
• i.e. a permutation is an arrangement of 𝑛 objects in a specific order.
• Thus, there are six possible permutations of a set of 3 objects.
• This result could also have been obtained from the basic principle,
• the first object can be in any of the 3,
• the second object be chosen any of the remaining 2,
• and the third object chosen the remaining one.
• Thus there are 3.2.1 = 6 possible permutations.
Counting Rules

• Permutation Rule 1: Suppose now that we have 𝑛 objects.

Reasoning, similar to that we have just used for the 3 letter shows
that there are
𝑛. 𝑛 − 1 . 𝑛 − 2 … 3.2.1 = 𝑛!
• Different permutations of the 𝑛 objects
• E.g: A class of basic stat consists of 6 men and 4 women. An
examination is given, and the students are ranked according to their
performance. Assume that no two students obtain the same score.
a. How many different rankings are possible?
b. If the men are ranked just among themselves and women among
themselves, how many different rankings are possible?
Counting Rules

• Permutation Rule 2: We shall now determine the number of

permutations of a set of 𝑛 objects when certain of the objects are
indistinguishable from each other. Then the formula is:
𝑛!
𝑛1 !. 𝑛2 ! … 𝑛𝑟 !
• Different permutations of 𝑛 objects, of which 𝑛1 are alike 𝑛2 are
alike, …, 𝑛𝑟 are alike.
• Permutation Rule 3: Generally, if we are asked to arrange 𝑟 objects
among 𝑛 objects, then we will have the following total arrangements
𝑛!
𝑛𝑃𝑟 =
𝑛−𝑟 !
Counting Rules

• E.g1: How many different letter arrangements can be formed using

the letter PEPPER?
• E.g2: Suppose a business man has a choice of five locations in which
to establish his business. He wishes to arrange only the top three
locations. How many different ways can he arrange them?
Counting Rules

• Combination Rule
• We are often interested in determining the number of different
groups of 𝑟 objects that could be formed from a total of 𝑛 objects.
• A selection of objects without regard to order is called a
combination.
• That is, combinations are used when the order or arrangement is not
important.
• The number of combinations of 𝑟 objects selected from 𝑛 objects is
𝑛! 𝑛
denoted by 𝑛𝐶𝑟 and is given by the formula 𝑛𝐶𝑟 = =
𝑟! 𝑛−𝑟 ! 𝑟
• E.g: From a group of 5 women and 7 men, how many different
committees consisting of 2 women and 3 men can be performed?
What if 2 of the men are feuding and refuse to serve on the
committee together?
Approaches in Probability Definition
• The probability of an event is denoted by 𝑃 . where 𝑃
stands for probability and the dot stands for any event, say
A, B, G etc.
• Generally approaches to probability can be divided into two:
• Subjective approach: A probability derived from an
individual's personal judgment about whether a specific
outcome is likely to occur.
• Probabilities contain no formal calculations and only reflect the
subject's opinions and past experience.
• Probabilities differ from person to person.
• Because the probability is subjective, it contains a high degree of
personal bias.
Approaches in Probability Definition

• Objective approach: The probability of an event in

a certain experiment based on an experimental
evidence or random process.
• In this approach to study probability theory there are
three sub [Link] are
1. The classical approach
2. The frequentist approach
3. The axiomatic approach
Approaches in Probability Definition

1. The Classical Approach : If a procedure has 𝑛 different simple

events, each with an equal chance of occurring, and event A can occur
in 𝑠 of these ways, then
𝑛(𝐴) 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠 𝑖𝑛 𝐴
•𝑃 𝐴 = =
𝑛(𝑆) 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠 𝑖𝑛 𝑎 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑝𝑎𝑐𝑒
• Assumptions in classical approach
• The outcomes must be equally-likely
• The experiment should never be repeated more than once
• The sample space should be finite
• E.g1: Toss a fair coin once and find the probability of the occurrence
of head.
• E.g2: For a card drawn from an ordinary deck, find the probability of
getting a queen.
Approaches in Probability Definition
2. Frequentist (empirical) Approach: If after 𝑛 repetition of an
experiment, where 𝑛 is very large, an event is observed to occur in ℎ of
ℎ
these, then the probability of an event is or conduct an experiment a
𝑛
large number of times, and count the number of times event A actually
occurs, then an estimate of 𝑃 𝐴 is
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑖𝑚𝑒𝑠 𝐴 𝑜𝑐𝑢𝑢𝑟𝑒𝑑
𝑃 𝐴 ≈
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑖𝑚𝑒𝑠 𝑡𝑟𝑖𝑎𝑙 𝑤𝑎𝑠 𝑟𝑒𝑝𝑒𝑎𝑡𝑒𝑑
• E.g: Suppose a coin was tossed 1000 times and the result was 587
tails.
587
• The relative frequency of tails is 1000.
• Another 1000 tosses lead to 511 tails.
587+511 1098
• Then the relative frequency of tails is = 2000 .
1000+1000
𝑛(𝐴)
• Therefore, 𝑃 𝐴 = lim
𝑛→∞ 𝑛
Approaches in Probability Definition

3. Axiomatic Approach: Both the classical and frequentist approaches

have serious drawbacks.
• The first because the words “equally likely” are vague and
• The second because the “large number” involved is vague.
• Because of these difficulties, statisticians have been led to an
axiomatic approach of probability.
• Axiom 1: For every event A, 𝑃 𝐴 ≥ 0
• Axiom 2: For the sure or certain event, 𝑃 𝑆 = 1
• Axiom 3: For any number of mutually exclusive events 𝐴1 , 𝐴2 , 𝐴3 …
• 𝑃 𝐴1 ∪ 𝐴2 ∪ 𝐴3 ∪ ⋯ = 𝑃 𝐴1 + 𝑃 𝐴2 + 𝑃 𝐴3 + ⋯
• In particular, for two mutually exclusive events 𝐴1 𝑎𝑛𝑑 𝐴2
• 𝑃 𝐴1 ∪ 𝐴2 = 𝑃 𝐴1 + 𝑃 𝐴2
Some Probability Rules
Approaches in Probability Definition

• E.g: Suppose we toss two coins and suppose that each of the four
points in the sample space is 𝑆 = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇} equally likely and
1
hence has probability .
4
• Let E is the event that the first coin falls head, and F is the event that
the second coin falls heads.
• Then the probability of either the first or the second coin falls head
is?
Conditional Probability and Independence
Conditional Probability and Independence

• E.g1: A jar contains black and white marbles. Two marbles are
chosen without replacement. The probability of selecting a black
marble and then a white marble is 0.34, and the probability of
selecting a black marble on the first draw is 0.47. What is the
probability of selecting white marble on the second draw, given that
the first marble drawn was black?

• Eg2: The probability that it is Friday and that a student is absent is

0.03. Since there are 5 schooldays in a week, the probability that it is
Friday is 0.2. What is the probability that a student is absent given
that today is Friday?
Conditional Probability and Independence

E.g: Suppose that we roll a pair of fail dice, so each of the 36 possible
out come is equally likely. Let A denotes the event that the first die
lands on 3, let C be the event that the sum of the dice is 7
• Are A and B independent?
• Are A and C independent