Manju, Associate professor, CSE, IIUC

CHAPTER FIVE

FUNDAMENTALS OF PROBABILITY

Mr. Mohammad Manjur Alam (Manju)

Associate professor
Department of Computer Science and Engineering
International Islamic University Chittagong.
FB and Email: manjuralam44@[Link]

Concept related to probability

Define the followings with examples:

Experiment: Experiment is an act that can be repeated under given conditions.

Tossing of a coin or throwing of a dice or the drawing of a cards etc, are the example of experiment.

Outcomes: The results of an experiment are called outcomes.

Random experiment: Experiments are called random experiments if the outcomes depend on
chance and cannot be predicted with certainty.

Example: Tossing of a fair coin, throwing of dice etc are the examples of random experiments.

1
 Manju, Associate professor, CSE, IIUC

Sample space: The collection or totality of all possible outcomes of a random experiment is called
sample space. It is usually denoted by S or Ω.

If we toss a coin, the sample space is S = {H, T } where H and T denote the head and tail of the coin
respectively.

If we toss a coin two times, than the sample space is S = {HH , HT , TH , TT }

Sample point: Each element of a sample space is called sample point.

Event: Any subset of a sample space is called event. There are two types of event:

Simple event and Compound event

Simple event: An event is called simple event if it contains only one sample point.

Compound event: An event is called simple event if it contains more than one sample point.

If we toss a coin two times, than the sample space is S = {HH , HT , TH , TT }

where H and T denote the head and tail of the coin respectively.

Here, A = {HH } is a simple event and B = {HH, HT } is a compound event.

Sure event: An event is called sure event when it always happens. The probability of a sure event is
one.

Impossible event: An event is called impossible event when it never happens. The probability of an
impossible event is always zero.

2
 Manju, Associate professor, CSE, IIUC

Mutually exclusive events: Two events are said to be mutually exclusive if they have no common
points. If A and B are two mutually exclusive events, then AB  ø.

Non mutually exclusive events: Two events are said to be not mutually exclusive event if they have
common points. If A and B are two not mutually exclusive events, then AB  ø.

Complementary event: Let A be any event defined on a sample space S or Ω then the
complementary of A, denoted by A is the event consisting of all sample points in S but not in A.

Equally likely outcomes: Outcomes are called equally likely if one does not occur more often than
the other. In this case the sample points of a sample space are all equal probable.

Exhaustive outcomes: Outcomes of an experiment are said to be exhaustive if they include all
possible outcomes.

In throwing a die exhaustive numbers of outcomes are 6.

Conditional Probability: If A and B are two events in S. Then the conditional probability of A for
given value of B, denoted by P [A | B] is defined by

P[ AB ]
P[B ]
P [A | B] = ; P [B]>0

P[ AB ]
P[ A]
Similarly, P [B | A ] = ; P[A]>0

3
 Manju, Associate professor, CSE, IIUC

Independent Event: Two events A and B are said to be independent if and only if one of the
following conditions holds:

(i) P[AB]=P[A]P[B]
(ii) P [A | B ] = P[A]
(iii) P [B | A ]= P[B]

Dependent Event: Two events A and B are said to be dependent if and only if one of the following
conditions holds:

(i) P[AB] ≠ P[A]P[B]

(ii) P [A | B ] ≠ P[A]
(iii) P [B | A ] ≠ P[B]

Definition of probability:

There are four approaches of defining probability. They are

(i) Classical or mathematical or priori definition of probability.

(ii) Empirical or statistical or posterior or frequency probability.
(iii) Subjective probability.
(iv) Axiomatic probability.

Classical or mathematical or priori definition of probability: If there are n mutually exclusive,

equally likely and exhaustive outcomes of a random experiment and if m of these outcomes are
favorable to an event A, then the probability of the event A, denoted by P[A] is defined as

4
 Manju, Associate professor, CSE, IIUC

N [ A] m
; 0 ≤ P[ A] ≤ 1
N [S ]
P[A] = =
n

This definition of probability is given by Laplace.

Axiomatic probability: Suppose S is a sample space and A is an event of this sample space. Then
the probability of the event A, denoted by P[A] must satisfy the following four axioms:

(i) P[A]  0
(ii) P[S] = 1
(iii) If A and B are mutually exclusive events, then P[AUB]= P[A]+P[B]
(iv) Let A1, A2,………………Ak be a sequence of K mutually exclusive events, then
P[A1UA2U…………UAK]= P[A1]+ P[A2]+… ………+p[Ak]

Laws of Probability

There are two important rules or laws of probability;

(i) Addition laws of probability
(ii) Multiplication laws of probability.

Addition laws of probability are two types

(i) Mutually exclusive events (ii) Non Mutually exclusive events

Theorem: State and prove additive laws of probability for two non mutually exclusive events.

Statement: If A and B are two events, then P[AUB]= P[A]+P[B] – P[AB]

Proof:

A B

AB AB AB

AUB  AB UABUA B

It is obvious from the Venn-diagram A= A B ∪ A B

Therefore, P [A] = P [ A B ∪ A B ]

5
 Manju, Associate professor, CSE, IIUC

P [A] = P [ A B ] + P [ A B ] ……………..(i);

Since A B and A B are mutually exclusive.

Similarly, B = AB ∪ A B

Therefore, P [B] = P [ A B ∪ A B ]

P [B] = P [ A B ] + P [ A B ] ……………..(ii) ;

Since A B and A B are mutually exclusive.

Now, A∪ B = A B ∪ A B ∪ A B

Therefore, P [ A∪ B ] = P [ A B ∪ A B ∪ A B ]……………..(iii);

Since A B , A B and A B are mutually exclusive.

Now adding (i) and (ii) we get,

P [A] + P [B] = P [ A B ] + P [ A B ] + P [ A B ] + P [ A B ]

=P[A B]+P[A B]+P[A B]+P[A B]

= P [ A∪ B ] + P [ A B ]

Therefore, P [ A∪ B ] = P [A] + P [B] - P [ A B ]

(This completes the proof of the theorem)

But if A and B are mutually exclusive, then P [ A B ] = 0

In that case, P [ A∪ B ] = P [A] + P [B] .

Theorem: State and prove additive laws of probability for three non mutually exclusive events.

Statement: If A, B, C are three non mutually exclusive events then,

P(A U B U C) = P(A) + P(B)+ P(C)- P(AB)- P(AC)- P(BC)+ P(ABC)

Proof: Let, B U C = D; then

P(A U D) = P(A)+ P(D)- P(AD)

6
 Manju, Associate professor, CSE, IIUC

Or, P(A U B U C) = P(A)+P(B U C)-P[A(B U C)]

= P (A) + P(B)+ P(C)- P(BC)-P(AB U AC)

= P(A) + P(B)+ P(C)- P(BC)-[ P(AB)+ P(AC)-P(ABAC)]

= P(A) + P(B)+ P(C)- P(AB)- P(AC)- P(BC)+ P(ABC);

[Since, ABAC =ABC]

(This completes the proof of the theorem)

But if A and B are mutually exclusive, then P(AB) = 0, P(BC)= 0, P(AC)=0 and P(ABC) = 0. In that
case, P (A U B U C) = P (A) + P (B) + P(C).

Multiplication law of probability: If A and B are two events, then

P [AB] = P [A] P [B | A] = P [B] P [A | B]

Proof: From the definition of conditional probability, we have

P[ AB ]
P[B ]
P [A | B ] = ; P[B]>0

It follows that P [AB] = P [B] P [A | B]………………….(i)

P[ AB ]
P[ A]
Similarly, P [B | A ] = ; P[A]>0

It follows that P [AB] = P [A] P [B | A]………………….(ii)

From (i) and (ii) we have, P [AB] = P [A] P [B | A] = P [B] P [A | B]

But if A and B are independent events, then P [A] = P [A | B] and P [B] = P [B | A]

Hence, P [AB] = P [A] P [B].

Theorem: Show that 0 ≤ P( A) ≤ 1 or Show that the value of the probability lies between 0 to 1.

Proof: Let A be the event in S (sample space)

Also let, N(S) = n

And N(A) = m

N ( A) m
N (S )
By the definition of probability, P [A] = =
n

7
 Manju, Associate professor, CSE, IIUC

Clearly, 0≤m≤n

0 m n
Or,   [Dividing bots side by n]
n n n

Or, 0 ≤ P( A) ≤ 1 (Proved)

Theorem: If A and A are mutually and exhaustive, then show that P( A)+ P A = 1 ()
Proof: From the complementary laws of events, we have A∪ A = S

Therefore, P ( A∪ A) = P(S )

) (
Or, P A∪ A = 1 [By axiomatic definition P(S) = 1]

Or, P( A)+ P(A)= 1 [since A and A are mutually exclusive]

Therefore, P( A)+ P(A)= 1 (Proved)

Theorem: State and Prove Bayes’ theorem.

Statement of Bayes theorem: Let B1, B2,………………,Bn be n mutually exclusive and exhaustive
events in a random experiment and A be any event in S., then Bayes theorem state that

P[Bi ].P[A Bi ]
P[Bi|A] = ; i = 1,2,............, n B1 B2 .. .. ... B
∑ P[B ]P[A B ]
n
n
i i
i=1

Proof: From the definition of conditional probability,

we have

A = AB1 U AB2 U ………………U ABn

P[ABi ]
…………… (i)
P[ A]
P [Bi | A] =

P[Bi ].P[A Bi ]
……… (ii) (Since, P [AB] = P [B] P [A | B])
P[A]
=

It is obvious from the venn diagram

8
 Manju, Associate professor, CSE, IIUC

A = AB1 U AB2 U ………………U ABn

Therefore, P(A) = P(AB1 U AB2 U ………………U ABn)

= P(AB1)+ P(AB2) + ………………+P( ABn)

Since, AB1 , AB2 …..and ABn are mutually exclusive.

= P[B1]P[A|B1] + P[B2]P[A|B2]+………………..+ P[Bn]P[A|Bn]

(Since, P [AB] = P [B] P [A | B])

∑ P[B ]P[A B ]…………………..(iii)

n
P(A) = i i
i=1

Putting the value of P(A) in equation (ii), we have

P[Bi ].P[A Bi ]
P[Bi|A] = ; i = 1,2,............, n
∑ P[B ]P[A B ]
n

i i
i=1

(This completes the proof)

Mathematical problem of Bayes’ theorem: In a factory, machines A, B and C manufacture

respectively 25%, 35% and 40% of the product. Of the total of their output 5, 4 and 2 percent
are defective bolts respectively. A bolt is drawn at random from the product and is found to be
defective. What are the probabilities that it was manufactured by machine (i) A and (ii) C.

Solution: Let

B1 : An item produced by machine A

B2: An item produced by machine B

B3: An item produced by machine C

A: Defective item produced by the machines.

We have, P [B1]= 25%=0.25, P[B2]= 35%=0.35, P[B3]= 40%=0.40,

And P(A|B1)= 5%=0.05, P(A|B2)= 4%=0.04, P(A|B3)= 2%=0.02,

We have to find, P(B1|A) and P(B3|A)

P[Bi ].P[A Bi ]
According to Bayes’ theorem, P[Bi|A] = ; i = 1,2,............, n
∑ P[B ]P[A B ]
n

i i
i=1

9
 Manju, Associate professor, CSE, IIUC

P(B1 )P( A B1 )
P(B1 )P( A B1 )+ P(B2 )P(A B2 )+ P(B3 )P(A B3 )
P(B1|A) =

=
(0.25)(0.05)
(0.25)(0.05)+ (0.35)(0.04)+ (0.40)(0.02)
= 0.3623

P(B3 )P(A B3 )
P(B1 )P( A B1 )+ P(B2 )P(A B2 )+ P(B3 )P(A B3 )
Similarly, P(B3|A) =

=
(0.40)(0.02)
(0.25)(0.05)+ (0.35)(0.04)+ (0.40)(0.02)
= 0.2319

Problem: If P[A] = 0.6, P[B]= 0.8 and P[AB]= 0.50 Find (i) P[ A ] ; (ii) P[A U B] ;(iii) P[A |B] ;
iv) P[B |A] (v) P[ A B ]; (vi) P[ A B ]; (vii) P[ A B ] ; (viii) P[ A∪ B] (xi) Are the events A and
B independent? (x) Are A and B mutually exclusive?

Ans: Solution: (i) We know: P( A)+ P A = 1 ()

Here P[A] = 0.6. Hence, P[ A ] =1-0.6=0.40

(ii)By additive law of probability, we know P[AUB]= P[A]+P[B] – P[AB]

= 0.60+0.80-.50=0.90

(iii)From the definition of conditional probability, we have

P[ AB ]
P [A | B ] = ; P[B]>0
P[B ]
0.50
=  0.625
0.80

P[ AB ]
P[ A]
(iv)Similarly, P [B | A ] = ; P[A]>0

0.50
=  0.833
0.60

[ ]
(v)P[ A B ]= P A ∩ B =P[A]-P[AB]

10
 Manju, Associate professor, CSE, IIUC

= 0.60-0.50=0.10

[ ]
(vi)P[ A B ]= P A ∩ B == P[B]-P[AB]

= 0.8-0.50=0.30

(vii) P[ A B ]

According to De Morgan’s law, P[ A B ] = P[ A ∩ B] ] = P[ A∪ B] =1- P[A U B]

= 1-0.90=0.10

(viii)P[ A∪ B] =1- P[A U B]= 1-0.90=0.10

(xi) Are the events A and B independent?

Ans: The events A and B are independent if P [AB] = P [A] P [B].

Here, P [A] P [B]=(0.60) (0.80)=0.48≠ P [AB]=0.50

Hence the events A and B are not independent.

(x) Are A and B mutually exclusive?

Ans: A and B mutually exclusive if P [AB]=0

Here, P[AB]= 0.50≠0

Hence A and B are not mutually exclusive.

. Find (i) P( A | B ) (iii) P A | B  .

1 3 11
Assignment: If P(A)= , P(B)= and P(A U B)=
3 4 12

Application Problem: Suppose A and B are two mutually exclusive events with P[A]=.35 and
P[B]=.15. Find (i) P[A  B] (ii) P[ A ] (iii) P[A  B] (iv) P[ A  B ]

Solution:

(i) According to axiom 3.

P[AUB]= P[A]+P[B]= 0.35+0.15=0.50

(ii) P[ A ] =1-0.35=0.65
(iii) P[A ∩ B]=0; Since A and B are mutually exclusive.

(iv) According to De Morgan’s law, A∪ B = A ∩ B( )

11
 Manju, Associate professor, CSE, IIUC

( )
P ( A∪ B) = P A ∩ B =1- P[A ∩ B]=1-0=1

Application problem: The distribution of number of stores according to size in 3 areas is given
in following table:

Area Store size

Large(L) Medium(M) Small(S)
A 30 45 75
B 150 125 275
C 20 130 150
Find the probabilities (i) P(M); (ii) P[BM] (iii) P(B U L) and (iv) P(A | M) (v) Are the events A and
L independent?

Solution:

Area Store size

Large(L) Medium(M) Small(S) Total
A 30 45 75 150
B 150 125 275 550
C 20 130 150 300
Total 200 300 500 1000
Here total number of stores is [Link] is N(S)=1000.

(i) Here M is the event of medium size store. Then N(M)= 300
N ( M ) 300
Therefore P[M] = = = 0.300
N ( S ) 1000
(ii) B ∩ M = BM.
N ( BM ) 125
P[BM]= = = 0.125
N (S ) 1000
(iii) P(B U L)=P[B]+P[L]-P[BL]

N ( B ) N (L ) N (BL)
= +
N ( S ) N (S ) N (S )

550 200 150

= +
1000 1000 1000
=0.60
N ( AM )
p( AM ) N (S ) N ( AM ) 45
= 0.15
N (M )
(iv) P(A | M)= = =
N (S ) 300
=
P(M )
N (S )
(v) Are the events A and L independent?

12
 Manju, Associate professor, CSE, IIUC

Solution: The events A and L are independent if P [AL] = P [A] P [L].

N ( AL ) 30
Here, P [AL] = =
N ( S ) 1000

N ( A) N ( L) 150 200 30
And P [A] P [L]= × = × = = P[ AL]
N (S ) N (S ) 1000 1000 1000

Hence the events A and L are independent.

13