INTRODUCTION TO

PROBABILITY

Prof. Ayush Gupta

ayushg@[Link]
Chapter 1

The Concept of Probability

Most of us already have a notion of probability. Consider the following questions:

1. What is the probability of heads if I toss a fair coin? The answer is, of course, 50% or 0.5. What
do we mean by this, though? It simply means that if I keep tossing the coin a large number of
times, then approximately 50% of the time I would get heads. If I do it long enough, i.e., infinitely
many times, I will get exactly 50% heads. This is the "Relative Frequency" notion of probability.
It tacitly assumes that whatever is the experiment that led to the outcome can be repeated
infinitely many times under identical conditions.
2. What is the probability that AAP will lose the next Delhi assembly election? The answer to
this is less straightforward and may depend on each individual's viewpoint. Suppose I say there
is an 80% chance. What does this mean? Clearly, this does not have the relative frequency
interpretation, for we cannot repeat the elections under identical conditions infinitely many
times! Instead, it can be interpreted as a "degree of belief".

Both the above interpretations are fine for our understanding whenever they make sense. Before formally
defining probability, let us understand some associated terminology.

 Random Experiment: It is that which leads to a random outcome. Example 1: Tossing a coin
twice, Example 2: Occurrence of an insurance claim

Sample Space is a collection of all possible outcomes of a random experiment. This is usually denoted
by S. In Example 1: S = {HH, HT, TH, TT}. In Example 2: S = the interval (0, policy limit), i.e., the claim
value can be any value between zero and the policy limit. The former is a "discrete" sample space, and
the latter is a "continuous" sample space.

 Event is a subset of the sample space.

o If the event consists of a single outcome, it is a "Simple Event". Example: In Example 1,
A = {HH} is a simple event.
o If an event consists of multiple outcomes, it is a "Compound Event". Example: In Example
1, A = {HH, HT, TH} is the compound event that at least one toss is a head. In Example
2, B = {10000, 50000} is the compound event that a claim amount is at least 10000 but
not more than 50000.
 We will often work with combinations of various events. Towards that end, let's recall a few
notations and facts from set theory:

1. A ⊆ B means the set A is contained in set B. This means every outcome in A is also in B.
Example: In Example 1, take A = first toss is a head, B = at least one toss is a head. Then
A ⊆ B.
2. A ∪ B is the union set of A and B, which contains all those elements that belong to either
A or B.
3. A ∩ B is the intersection set of A and B, which contains all those elements that belong
to both A and B.
4. We can also talk about union and intersection of multiple (possibly infinitely many)
events, denoted by ⋃ 𝐴 and ⋂ 𝐴 respectively. Note that N can potentially be
infinity (∞).
5. Aᶜ is the complement of A, which contains all those elements of S that do not belong to
A.
 6. A and B are disjoint if A ∩ B = ∅. Similarly, events A₁, A₂, ..., Aₙ, ... are pairwise-
disjoint or mutually exclusive if any two events Aᵢ and Aⱼ are disjoint.
7. A₁, A₂, ..., Aₙ, ... are exhaustive if ⋃ 𝐴 = 𝑆
8. As a matter of convention, the null set (denoted by { } or ∅) is considered an event.

Exercises

1. Two 6-faced dice are rolled sequentially.

a. Write down the sample space.

b. List the elements that make up the following events:

o A = sum of the two values is 5

o B = value of the first die is higher than the second
o C = the first value is 4

c. What are the elements of A∩C, 𝐵 ∩ 𝐶, 𝐴 ∩ 𝐵 ∩ 𝐶?

2. Show that (𝐴 ∪ 𝐵) =𝐴 ∩ 𝐵 and in general that (⋃ 𝐴) =⋂ 𝐴 .

Axioms of Probability

Probability is a real-valued function (P) on the sample space S (or, more precisely, on the set of events
based on S) with the following properties:

 P(S) = 1, P(∅)=0
 For any A⊆S, 0≤P(A)≤1
 For mutually exclusive events 𝐴 , 𝐴 , 𝐴 , 𝐴 , … 𝐴 , … . . : 𝑃(⋃ 𝐴)=∑ 𝑃(𝐴 )

Exercises

3. Show that P(𝐴 ) = 1 − 𝑃(𝐴)

4. Show that (𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵)

5. Prove Boole's Inequality: P(A∪B)≤P(A)+P(B)

6. Use Exercise 4 to show that: 𝑃(𝐴 ∪ 𝐵 ∪ 𝐶) = 𝑃(𝐴) + 𝑃(𝐵) + 𝑃(𝐶) − 𝑃(𝐴 ∩ 𝐵) − 𝑃(𝐴 ∩ 𝐶) −
𝑃(𝐵 ∩ 𝐶) + 𝑃(𝐴 ∩ 𝐵 ∩ 𝐶).

Can you guess and write down the general formula for 𝑃(⋃ 𝐴 )?

7. Three distinguishable balls are randomly placed in three cells.

(a) What is the sample space associated with the experiment? Let B1, B2, B3 respectively denote the
balls. Examples of elements of S would be:

 {(B1, B2, B3), { }, { }}

  {(B1, B2), {B3}, { }}
 {(B1), {B2}, {B3}}, etc.

(b) Let, S1: event that cell 1 is singly occupied and S2: event that cell 2 is singly occupied

Find:

 P(S1)
 P(S2)
 P(S1∪S2)

∗ ∗
Answer: P(S1)=P(S2)= , P(S1∩S2)=

8.A readership survey among adults showed the following statistics:

 35% read Times

 15% read Express
 25% read Herald
 10% read both Times and Express
 8% read both Express and Herald
 5% read both Times and Herald
 4% read all three publications

If an adult is chosen at random, find the probability that: (i) They do not read any newspaper (ii) They
read only one newspaper (iii) They read exactly two newspapers (iv) They read all three newspapers

Answers:

 (i) 44%
 (ii) 41%
 (iii) 11%
 (iv) 4%

9. Someone throws a pair of fair dice. True or False: The chance of getting at least one six is

1/6+1/6

Answer: False
Chapter 2

Enumeration Principles
Much of computing probability has to do with counting the number of outcomes that satisfy some criteria.
Here are some basic counting principles.

If r experiments are performed such that:

 Experiment 1 may result in 𝑛 outcomes,

 Experiment 2 in 𝑛 outcomes,
 … up to experiment r, which can result in 𝑛 outcomes,

Then the total number of potential outcomes is: 𝑛 ∗ 𝑛 ∗ 𝑛 ∗ … 𝑛

Combinations

!
𝑛 =Number of ways of choosing r objects out of n= )!
!(

Exercises

1. Show that for any positive integer n and real numbers x,y: (𝑥 + 𝑦) = ∑ 𝑛 𝑥 𝑦

(This result is known as the Binomial Theorem.)

2. As a special case of the above problem and also by a direct argument, show: 2 =∑ 𝑛

Permutations

!
𝑛 =Number of ordered ways of choosing r objects out of n= ( )!

Exercises

3. There are 10 brands (B1,B2,…,B10) in the market which have been ranked by a market research
firm. If you believe that this ranking is random, what is the probability that the first three brands
are B1,B2,B3 in that order? Answer:
4. The numbers 1,2,3,…,n are ordered randomly. What is the probability that: (a) 1 and 2 are
neighbours in that order? (b) 1, 2 and 3 are neighbours in that order?
5. You own Brand A of a certain product and there are 9 other competing brands. What is the
probability that your brand is among the top 3 in terms of consumer preference?

𝟑∗𝟗!
Answer:
𝟏𝟎!

[Link] how many ways can we put r distinguishable balls in n cells? Answer:𝑛
 ( )!
7. In how many ways can we put r indistinguishable balls in n cells? Answer: (𝑛 − 1 + 𝑟) = ( )! !

8. How many non-negative integer solutions are there to the equation 𝒙𝟏 + 𝒙𝟐 + 𝒙𝟑 + ⋯ + 𝒙𝒏 = 𝒓

[Link] how many ways can we put r indistinguishable objects in n cells so that each cell contains at least
one object?

10. There are n objects such that n1 are of type 1, n2 of type 2, ..., and nr of type r. Objects of each
type are indistinguishable among themselves. How many distinguishable arrangements of these n objects
!
are possible? Answer:
!∗ !∗ !… !

11. In a city, 7 accidents happened in a week. Compute the probability that all 7 happened on different
days of the week if: (a) The accidents are distinguishable and each distinguishable distribution of
accidents across 7 days has equal probability.

In a city, 7 accidents happened in a week. Compute the probability that all 7 happened on different days
of the week if:

(a) The accidents are distinguishable and each distinguishable distribution of accidents across 7 days has
!
equal probability. Answer:

(b) The accidents are indistinguishable and each distinguishable distribution of accidents across 7 days
has equal probability. Answer:

Note:

 The assumption in part (a) corresponds to the Maxwell-Boltzmann statistics in Physics, used for
the distribution of distinguishable atomic particles into energy states.
 The assumption in part (b) corresponds to the Bose-Einstein statistics, used for indistinguishable
particles.

Problem 12

There are n flag poles and r flags, of which r1 are red and r2 are blue, with r1+r2=r. Each flag pole can
hoist at most one flag. The poles are distinguishable, but the flags are not. All flags need to be hoisted.
Assume 𝑛 ≥ 𝑟.

Question: How many distinguishable ways are there to hoist the flags?

𝒓!
Answer:𝒏𝒄 𝒓 ∗
𝒓𝟏 𝒓𝟐
Chapter 3: Conditional Probability and Bayes Theorem
Conditional Probability

Consider the experiment where we throw a fair 6-faced die. The set of possible outcomes or the sample
space is:

S={1,2,3,4,5,6}

Let A={3}. Then P(A)=1/6

Suppose we are given additional information that the die throw resulted in an outcome divisible by 3.
What is the probability that the outcome was 3?

With this additional information, the sample space is restricted to a subset of the original sample space:
B={3,6}

Among these two outcomes, there is no reason to believe one is more likely than the other. Therefore,
the new probability for event {3} is 1/2.

This is the conditional probability of observing A={3}given or conditioned on event B={3,6}.

Formal Definition

Let B be an event such that P(B) > 0. The conditional probability of an event A given B is defined as:

( ∩ )
P(A∣B)=
( )

Exercises

1. Conditional probability is also a probability, i.e., it is a function satisfying the axioms of

probability. For a fixed event B with P(B)>0, let Q(A)=P(A∣B). What is Q(𝐵 )?
2. Consider families with 2 children. If one child of a family is a boy, what is the probability that
the other child is a girl?
3. An insurance company sells a number of different policies. Among these:
o 60% are for autos
o 40% are for homeowners
o 20% are for both

Suppose a person is picked at random from the population. Let:

o A1: person has only an auto policy

o A2: person has only a homeowner's policy
o A3: person has both auto and homeowner's policies
o A4: person has neither auto nor homeowner's policy

a. Find P(A1), P(A2), P(A3), and P(A4)

b. Let B be the event that the person renews at least one of the auto or homeowner's policies.
From past experience:
 o P(B∣A1)=0.6
o P(B∣A2)=0.7
o P(B∣A3)=0.8

Given that a person selected at random has an auto or a homeowner's policy, what is the
probability that they will renew at least one of them

4. A common test for AIDS is called ELISA (Enzyme-Linked Immunosorbent Assay). Among
1,000,000 people who were given the test, we can expect results similar to those in Table 3.1.
If one of the 1,000,000 people is selected at random, what is:
o (a) P(B1)
o (b) P(A1)
o (c) P(A1∣B2)
o (d) P(B1∣A1)

What is the interpretation of each of these probabilities? (Answers: (a) 0.005, (b) 0.0785, (c)
0.074, (d) 0.062)

5. Simpson's Paradox (Yule-Simpson Effect): A paradox in which a trend that appears in different
groups of data disappears when these groups are combined, and the reverse trend appears for
the aggregate data.

Example: A newspaper conducted a survey on the popularity of soap operas among men and
concluded that working men were more addicted to soap operas than house husbands. They
surveyed 60 working men and 50 house-husbands, and found that 70% of the working men were
addicted to soap operas, whereas Only 62% of the house husbands were addicted to soap operas.
Table 3.2 gives the age-wise breakup of the responses to the survey.

ELISA Test Example

Table 3.1: ELISA Test Results

B₁: Has AIDS B₂: No AIDS Total

A₁: Test +ve 4,885 73,630 78,515
A₂: Test –ve 115 921,370 921,485
Total 5,000 995,000 1,000,000

Table 3.2: Addiction to Soap Operas by Age Group

Age Group Working men House-husband

Surveyed Addicted Surveyed Addicted
Below 30 10 4 32 14
30 to 55 20 14 10 9
55 and Above 30 24 8 8

Questions:

(i) Do you consider the conclusion of the newspaper valid? Give reason for your answer.

(ii) What would be your conclusion if you were to summarise the main findings of the survey?
Bayes Theorem

Let A and B be two events. Bayes' Theorem provides a way to relate P(A∣B) and P(B∣A):

𝑃(𝐴 ∩ 𝐵) 𝑃(𝐴|𝐵) ∗ 𝑃(𝐵) 𝑃(𝐴|𝐵)𝑃(𝐵) 𝑃(𝐴|𝐵)𝑃(𝐵)

𝑃(𝐵|𝐴) = = = =
𝑃(𝐴) 𝑃(𝐴) 𝑃(𝐴 ∩ 𝐵) + 𝑃(𝐴 ∩ 𝐵 ) 𝑃(𝐴|𝐵) ∗ 𝑃(𝐵) + 𝑃(𝐴|𝐵 )𝑃(𝐵 )

Problems

1. A bolt factory has three machines A, B, and C producing 25%, 35%, and 40% of the total output
respectively. The defect rates are 5%, 4%, and 2% respectively. A bolt is drawn at random and found to
be defective. What is the probability it was manufactured by machine A?

(Answer: 0.3623)

Given:

 P(A)=0.25, P(B)=0.35, P(C)=0.40

 P(D∣A)=0.05, P(D∣B)=0.04, P(D∣C)=0.02
 Let D be the event that the bolt is defective.

𝑷(𝑫|𝑨)𝑷(𝑨)
To Find: 𝑷(𝑨|𝑫) = . 𝑷(𝑫) = 𝑷(𝑫|𝑨)𝑷(𝑨) + 𝑷(𝑫|𝑩)𝑷(𝑩) + 𝑷(𝑫|𝑪)𝑷(𝑪)
𝑷(𝑫)

2. According to a research study, the incidence rate of HIV in India is 0.4% for a certain section of the
population. A clinical test in India is 95% accurate in detecting HIV, i.e.:

 If a person has HIV, the test correctly detects it 95% of the time.
 If a person does not have HIV, the test correctly returns negative 95% of the time.

A person from this section undergoes a test and the result is positive.

(a) What is the probability that he really has the disease?

(b) A second independent test with similar accuracy also comes out positive. What is the updated
probability that he has the disease?

Note: This is an example of Bayesian updating.

[Link] industrial raw material is graded into three categories:

 Category 1: Super quality

 Category 2: Medium quality
 Category 3: Low quality

Grading is done visually, and misclassification is possible only between neighboring categories:

 Category 1 may be misclassified as Category 2

 Category 2 may be misclassified as Category 1 or 3
 Category 3 may be misclassified as Category 2
Let p be the common probability of misclassification. Let Bi be the event that a lot is actually of Category
i. Let Cj be the event that a lot is classified into Category j. answer following in terms of ‘p’

 P(C1∣B1)=
 P(C2∣B2)=
 P(C3∣B3)=

4.A production process involves three machines A, B, and C, which produce 50%, 30%, and 20% of the
total output respectively. The quality control failure rates are:

 Machine A: 10% fail

 Machine B: 20% fail
 Machine C: 30% fail

Items that fail the quality control test are further processed, and of those:

 40% from A become marginally acceptable

 50% from B become marginally acceptable
 60% from C become marginally acceptable

(a) Find the unconditional probability that a randomly chosen item is directly acceptable.

(b) Find the probability that a randomly chosen item is marginally acceptable.

(c) Given that an item failed the quality control test, what is the probability it is marginally acceptable?

(d) Given that an item is marginally acceptable, what is the probability it was produced by machine A?

(e) Given that an item was not produced by machine B, what is the probability it is marginally acceptable?

Answers:

 (a) 0.83
 (b) 0.086
 (c) 0.506
 (d) 0.232
 (e) 0.08

[Link] morning, a woman walking in an expensive part of London had her bag snatched. It contained a
large amount of cash. She and an eyewitness described the thief as:

 Very tall (over 2 metres)

 Black man
 Aged between 20 and 30
 Black hair and beard
 Pronounced limp

Later that day, a policeman spotted a man matching this description carrying a large plasma-screen TV.
He was arrested but denied involvement and could not provide an alibi. A man was arrested after being
spotted with a plasma-screen TV. He matched the eyewitness description of the thief: very tall (over 2
metres), black, aged 20–30, with black hair, a beard, and a pronounced limp. He claimed to have been
home alone during the crime.

In court, the prosecutor presented the following demographic probabilities for London:

 Being male: 0.51

 Being over 2 metres tall: 0.025
 Being between 20 and 30 years old: 0.25
 Being black: 0.2
 Having black hair: 0.5
 Having a black beard: 0.37
 Having a pronounced limp: 0.02

Assuming independence, the prosecutor multiplied these probabilities:

0.51×0.025×0.25×0.2×0.5×0.37×0.02=0.0000020.

He concluded that the chance of any random person having all these characteristics is 0.000002, and
since the accused matches all of them, he must be the thief.

The jury was persuaded, and the man was convicted.

Question

(a) What is wrong with the prosecutor's argument?

6. Scott Myers is a security analyst for a telecommunications firm called Webtalk. Although he is
optimistic about the firm's future, he is concerned that its stock price will be hugely affected by the
condition of credit flow in the economy. He believes that the probability is 0.2 that credit flow will
improve significantly, 0.5 that it will improve only marginally and 0.3 that it would not improve at all.
He also estimates that the probability that the stock price of Webtalk will go up by at least 20% is 0.9
with significant improvement in credit flow, 0.4 with marginal improvement in credit flow and 0.1 with
no improvement in credit flow.

(a) What is the unconditional probability that the stock price of Webtalk goes up by
at least 20%?

(b) Given that the stock price of Webtalk has not gone up by more than 20%, what
is the probability that there was no improvement in credit flow?

7. Mektek, a new company, is thinking about developing a smartphone. However, before they start
developing the smartphone, they want to make sure that it will make sense to invest on this. They are
planning to employ a market survey farm to sense the market. The following are the considerations: The
market survey can predict 3 scenarios: high, medium or low demand. In a high demand situation, Mektek
makes a profit of Rs. 10,00,00,000 (10 Crores). In case of medium demand, they make a profit of Rs.
1,00,00,000 (1 Crore), and in case of a low demand, they lose Rs. 2,00,00,000 (2 Crores). The probabilities
with which the market survey will predict high, medium or low demand are 40%, 20% and 40%
respectively. The problem is that historically when such a survey had predicted high demands, 40% of
the time it were wrong (half of such cases in fact being that of low demand,) and rest of medium demand.
Similarly, when such a survey predicted medium demand, it was correct only 70% of the times, with rest
of the cases being split equally between high and low demand. When the survey predicted low demand,
it was correct 90% of the times in with the rest of the cases all being of medium demand.
(a) What is the probability that Mektek will make a profit given that the market survey has predicted a
low demand scenario?

(b) What is the (unconditional) probability that Mektek will make a loss?

(c) What is the probability that Mektek will make a loss given that the market survey has not predicted
a low demand scenario

Independence of Events

Pairwise independence: Two events A and B are said to be independent if P(A ∩ B) = P(A) × P(B).

If P(B) > 0, this is equivalent to saying P(A|B) = P(A), i.e. knowing that event B has occurred does not
alter the likelihood of event A and vice-versa.

Exercises:

1. A 6-faced fair die is tossed and the outcome is noted. Are the events A = outcome is divisible by
2 and B = outcome is divisible by 3 independent?
2. Show that if A and B are independent then so are (a) A and Bᶜ (b) Aᶜ and B (c) Aᶜ and Bᶜ.

Independence of Three Events: Events A, B and C are said to be (mutually) independent if all of the
following hold:

(a) P(A ∩ B) = P(A) · P(B), P(A ∩ C) = P(A) · P(C), P(B ∩ C) = P(B) · P(C)

(b) P(A ∩ B ∩ C) = P(A) · P(B) · P(C).

3. Suppose A, B and C are mutually independent events and that P(A) = 0.5, P(B) = 0.8 and P(C) = 0.9.
Find the probabilities that

(a) All three events occur.

(b) Exactly two of three events occur.

(c) None of the events occur.

Ans. a) 0.36 b) 0.49 c) 0.01)

Note: It is important to note that both the above (a) and (b) in the definition need to be verified for
independence of three events. i.e., Just (a) does not ensure (b) holds and also just (b) does not ensure
(a) holds. Examples for both these situations are covered in the problems.

Below is an example to show that pairwise independence of events does not imply mutual independence
of many events

4. Two six faced fair dice are thrown.

Let Event A = Die 1 shows six, Event B = Die 2 shows six, and Event C = Both dice show the same face.

Show that A, B, C are pairwise independent but not mutually independent.

Below is an example to show that P(A ∩ B ∩ C) = P(A) · P(B) · P(C) does not imply Pairwise independence.

5. Consider families with 3 children.

Let B denote boy and G denote girl. Let sample space be S = {BBB, BBG, BGB, BGG, GBB, GBG, GGB,
GGG}. Assume that all 8 possibilities for the three children in order have equal probabilities.

Define events A, B and C as follows: A = at least 2 boys, B = 1st child is a girl, C = 2nd child is a boy.

Show that P(A ∩ B ∩ C) = P(A) · P(B) · P(C) but A, B, C are not pairwise independent.

(Solution: Show that P(A) = P(B) = P(C) = 1/2 P(ABC) = P(GBB) = 1/8 = P(A) · P(B) · P(C) but now note that
P(AB) is also equal = P(GBB) = 1/8 which is not equal to P(A) · P(B)).

Independence of many events: Events A₁, A₂, ⋯ , Aₙ are said to be independent if P(Aᵢ₁ ⋯ Aᵢₖ) = P(Aᵢ₁)
· P(Aᵢ₂) ⋯ P(Aᵢₖ) for all 1 < k ≤ N and all k-tuples (i₁, i₂, ⋯, iₖ) of {1, 2, ⋯, N}.

6. A machine has n components. It functions only if at least one component functions. All components
function independently and probability of any one component not functioning is p. What is the probability
that all components are functioning given that the machine is functioning?

What if events are not independent?

In general, if A₁, A₂, ⋯ , Aₙ are not necessarily independent, we can write P(A₁ ∩ A₂ ∩ A₃ ⋯ Aₙ) = P(A₁)
· P(A₂|A₁) · P(A₃|A₁A₂) ⋯ P(Aₙ|A₁A₂ ⋯ Aₙ₋₁)

[Notation: Sometimes for convenience P(A₁ ∩ A₂ ∩ A₃ ⋯ Aₙ) may just be written as P(A₁A₂ ⋯ Aₙ) (without
explicitly writing the intersection (∩) sign)].

[Link] urn has 3 black balls and 5 white balls. Each time I draw a ball. If it is black, I add 1 black ball and
2 white balls. If it is white, I add 2 black balls and 1 white ball. Assume that the ball that is originally
drawn is also replaced along with the additional balls. In four successive draws, what is the probability
of getting a Black-White-Black-White in four successive draws?

Ans: 0.06016

8. Explain whether anything is wrong in this argument: 20% of the Indians are senior citizens, 45% of the
Indians are women. So the proportion of women senior citizens is 45% of 20% = 9%.