Lecture Note on Basic Probability

Concepts
Why Study Statistics?
• Data are everywhere.
• Statistical techniques help make informed decisions in business, healthcare, engineering, social sciences,
and more.
• Regardless of your career, understanding data and using statistical methods leads to better decisions.

Applications of Statistical Concepts in the Business World • Finance:

Correlation, regression, index numbers, time series analysis.
• Marketing: Hypothesis testing, chi-square tests, nonparametric statistics.
• Personnel: Hypothesis testing, chi-square tests, nonparametric methods.
• Operations Management: Estimation, analysis of variance, time series forecasting.

What is Statistics?
Statistics is the science of collecting, organizing, presenting, analyzing, and interpreting data to assist in
making more effective decisions.

Branches of Statistics
• Descriptive Statistics: Organizes and summarizes data using charts, graphs, and numerical mea sures.
• Inferential Statistics: Uses sample data to make generalizations or predictions about a population, relying
heavily on Probability Theory.

How is Probability Related to Statistics?

• Probability provides the mathematical foundation for measuring uncertainty in Inferential Statistics. •
Statistical inference uses probability to make decisions about populations based on samples.

• Without probability, we cannot assess risks, perform hypothesis testing, or calculate confidence inter vals.

Interdisciplinary Applications of Statistics and Probability 1. Computer

Science:
• Machine Learning and AI models rely on probability and statistics for prediction and decision making.
• Algorithms like Na¨ıve Bayes, Hidden Markov Models, and Random Forests are based on proba
bilistic reasoning.

1
• Big data analytics, cryptography, and image processing require statistical modeling. 2.
Environmental Science and Engineering:

• Statistical models predict climate change patterns, pollution levels, and disaster risk assessment.
• Probability is used in environmental simulations (e.g., Monte Carlo methods) to model uncertain
systems.
 • Engineering designs use statistical quality control and reliability analysis.

3. Economics and Political Science:

• Econometric models use regression analysis and probability to predict economic trends. •
Voting behavior, policy impact studies, and public opinion surveys use statistical inference. •
Risk analysis and game theory apply probabilistic models for strategic decision-making.

4. Bioinformatics and Biology:

• Probability is used to analyze genetic sequences and evolutionary models.

• Statistical methods are fundamental in gene expression studies, drug development, and epidemi
ological research.
• Hypothesis testing is critical in validating biological experiments.

5. Public Health:

• Probability models are used to predict disease outbreaks and control epidemics.
• Clinical trials rely on hypothesis testing and confidence intervals to determine the effectiveness of
treatments.

1. Introduction to Probability
Probability quantifies the uncertainty or likelihood of events occurring in random experiments. Definition: A
numerical measure of the uncertainty of an event or an experiment is called probability

Experiment
Experiment is an act that can be repeated under given conditions.
Example: Tossing of a coin is a trial, and getting head and tail are outcomes.

Outcome
The result of an experiment is known as an outcome.
Example: In throwing a die, the possible outcomes are

{1, 2, 3, 4, 5, 6}.

Random Experiment
If in each trial of an experiment counted under identical conditions, the outcome is not unique but may be any
one of the possible outcomes, then such an experiment is called a random experiment. Example: Tossing a
coin, throwing a die.

2
Sample Space
The collection of all possible outcomes of a random experiment is called the sample space. It is usually
denoted by S.

• Note: Sample space and exhaustive case are the same.

Sample Space and Sample Point

Example: If we toss a coin, the sample space is

S = {H, T}

where H and T denote head and tail respectively.

Sample Point: An element of the sample space is called a sample point. It is denoted by a small Greek
letter ω.
Example: If we toss a coin, the sample space is S = {H, T} = {ω1, ω2} where H = ω1 and T = ω2.

Finite Sample Space

A sample space is called a finite sample space if it contains a finite number of sample points.
Example: If we toss a coin three times, the sample space will contain 8 sample points:

S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }

This is a finite sample space.

Infinite Sample Space

A sample space is called an infinite sample space if it contains an infinite number of sample points.
Example: Tossing a coin until we get a head:

A = {H, T H, T T H, T T T H, . . .}

Discrete and Continuous Sample Space (not included in exam)

Discrete Sample Space: A sample space is called discrete if it contains a finite or countably infinite number of
sample points.
Continuous Sample Space: A sample space is called continuous if it contains an uncountable (non
denumerable) number of sample points.
Example: Select a number at random from the interval [0, 1] of real numbers. The sample space is S =

{x | x ∈ [0, 1]}

Events
An event is a collection of outcomes and is a subset of the sample space. It is usually denoted by capital letters
A, B, C, etc.
There are two types of events:

• Simple Event

• Compound Event

3
Simple Event
An event is called a simple event if it contains only one sample point.
Example: Tossing a fair coin twice. Sample space:

S = {HH, HT, T H, T T}

Simple events:
ω1 = {HH}, ω2 = {HT}, ω3 = {T H}, ω4 = {T T}

Compound Event
An event is called a compound event if it contains more than one sample point or is the union of simple events.
 Example: Tossing a fair coin twice:

S = {HH, HT, T H, T T}

A compound event could be {HT, T H}.

Event Space
The class of all events associated with a given experiment is defined as the event space.

Sure Event
A sure event is an event that always happens. The probability of a sure event is 1.

Impossible Event
An impossible event is an event that never happens. The probability of an impossible event is 0.

Mutually Exclusive Events

Events A and B are said to be mutually exclusive if

A∩B=∅

That is, they have no common outcomes.

Complementary Event
¯
If A is an event in sample space S, then the complementary event of A, denoted by A , is the event consisting
of all sample points in S that are not in A.

Probability Space
The triplet (S, A, P) is called a probability space, where:
• S is the sample space,
• A is an event in S,
• P(A) is the probability of event A.

4
Definition of a Probability Tree
A Probability Tree Diagram is a graphical representation used to map out all possible outcomes of a sequence
of events. It helps visualize and calculate the combined probabilities of different outcomes by following the
branches of the tree.

• Each branch represents a possible outcome with its associated probability.

• The probability of a combined event is obtained by multiplying along the branches. • This
diagram is particularly useful when dealing with compound or sequential events.

Example: Tossing a Coin Twice

Consider tossing a fair coin twice. The sample space is:
 S = {HH, HT, TH, T T}

Each branch has a probability of 12.

P(TT)
=14

T P(TH) 2

P H)
1 P(T)= ( =
1
1
2

Start P(H) =14 P(HT)

P(T)= =
21
2 1
=14
P(T)=
H HH PH
1 ( )=
4 21
2

Sample Space for Throwing Two Dice The

sample space S for throwing two fair six-sided dice is:
S = {(1, 1),(1, 2),(1, 3),(1, 4),(1, 5),(1, 6),
(2, 1),(2, 2),(2, 3),(2, 4),(2, 5),(2, 6),
(3, 1),(3, 2),(3, 3),(3, 4),(3, 5),(3, 6),
(4, 1),(4, 2),(4, 3),(4, 4),(4, 5),(4, 6),
(5, 1),(5, 2),(5, 3),(5, 4),(5, 5),(5, 6),
(6, 1),(6, 2),(6, 3),(6, 4),(6, 5),(6, 6)}
Total number of outcomes: |S| = 36

5
Sample Questions and Solutions
What is the probability that the sum of the two dice is 7?
Solution: The outcomes that give a sum of 7 are:
(1, 6),(2, 5),(3, 4),(4, 3),(5, 2),(6, 1)
There are 6 such outcomes. 6 1
P(sum = 7) = 36= 6
What is the probability that both dice show the same number? Solution:
The outcomes where both dice show the same number are:
(1, 1),(2, 2),(3, 3),(4, 4),(5, 5),(6, 6)
There are 6 such outcomes. 6 1
P(same number) = 36= 6
What is the probability that at least one die shows a 5?
Solution: From die 1:
 (5, 1),(5, 2),(5, 3),(5, 4),(5, 5),(5, 6) (6 outcomes)
From die 2:
(1, 5),(2, 5),(3, 5),(4, 5),(6, 5) (5 more outcomes)
Total (excluding duplicate (5,5)): 6 + 5 = 11
11
P(at least one 5) = 36
What is the probability that the sum is less than or equal to 4?
Solution: The valid outcomes are:
(1, 1),(1, 2),(1, 3),(2, 1),(2, 2),(3, 1)
Total = 6 outcomes. 6 1
P(sum ≤ 4) = 36= 6
Some problems that are related to the probability tree or diagram.
• If we toss a coin twice, then what is the probability of getting two heads?
• If we throw two dice at a time, then what will be the probability of getting the sum of outcomes greater
than 10?
• If a single card is drawn from a well-shuffled standard deck of 52 cards, what is the probability of getting a
red king?
• If we toss three coins at the same time, what is the probability of getting at least two tails?
• If a number is selected at random from 1 to 20, what is the probability that the number is a multiple of 3 or
5?
The axioms provide the formal basis of probability theory:
1. 0 ≤ P(E) ≤ 1 for any event E.
2. P(S) = 1, where S is the sample space.

3. For mutually exclusive events E1, E2, . . . , En,

P(E1 ∪ E2 ∪ · · · ∪ En) = P(E1) + P(E2) + · · · + P(En)
Example: Probability of rolling a ”1” or ”2”:
1 1 2 1
P({1, 2}) = 6+ 6= 6= 3

6
General Additive Rule of Probability
Theorem: For any two events A and B defined on a probability space, the following identity holds: P(A ∪
B) = P(A) + P(B) − P(A ∩ B).

Proof: Let A and B be two events in a probability space with sample space S. The union A ∪ B can be
expressed as the union of three mutually exclusive events:
A ∪ B = (A \ (A ∩ B)) ∪ (B \ (A ∩ B)) ∪ (A ∩ B).
By the axiom of probability for mutually exclusive events, we have
P(A ∪ B) = P(A \ (A ∩ B)) + P(B \ (A ∩ B)) + P(A ∩ B).
Now, observe that
A = (A \ (A ∩ B)) ∪ (A ∩ B),
so
P(A) = P(A \ (A ∩ B)) + P(A ∩ B),
which implies
P(A \ (A ∩ B)) = P(A) − P(A ∩ B).
Similarly,
B = (B \ (A ∩ B)) ∪ (A ∩ B),
so
 P(B \ (A ∩ B)) = P(B) − P(A ∩ B).
Substituting these results back, we get
P(A ∪ B) = [P(A) − P(A ∩ B)] + [P(B) − P(A ∩ B)] + P(A ∩ B)
= P(A) + P(B) − P(A ∩ B).
Thus, the identity is proved.

Conditional Probability
Definition: The conditional probability of an event A given that another event B has occurred (with P(B) > 0) is
defined as:

P(A | B) = P(A ∩ B)
P(B)
This represents the probability that event A occurs under the condition that event B has already occurred.
Example: A fair six-sided die is rolled once. Let:
• A: the event that the number rolled is even = {2, 4, 6}
• B: the event that the number rolled is greater than 3 = {4, 5, 6}
Then:
A ∩ B = {4, 6}
3 3 2
P(A) = 6, P(B) = 6, P(A ∩ B) = 6
Using the definition:
P(B)=
2 3
6
P(A | B) = P(A ∩ B) 2
6 = 3
2
Interpretation: Given that the number rolled is greater than 3, the probability that it is an even num ber is 3.

7
Multiplication Law (Conditional Probability)
For two events A and B,
P(A ∩ B) = P(A) · P(B|A) if P(A) > 0
P(A ∩ B) = P(B) · P(A|B) if P(B) > 0
Where P(B|A) is the conditional probability of B given that A has occurred, and P(A|B) is the conditional
probability of A given that B has occurred.

Proof
Let n be the total number of occurrences in sample space S, and let n(A), n(B), and n(A∩B) be the number of
outcomes in events A, B, and A ∩ B respectively.

P(A) = n(A)
n(S), P(B) = n(B)
n(S), P(A ∩ B) = n(A ∩ B)
n(S)

P(A|B) = P(A ∩ B)
P(B)=n(A ∩ B)
n(S)·n(S)
 n(B)=n(A ∩ B)
n(B)

P(B|A) = n(A ∩ B)
n(A)

P(A ∩ B) = n(A)

n(S)·n(A ∩ B)
n(A)= P(A) · P(B|A)
Special Case (Independent Events): If events A and B are independent, then: P(A ∩
B) = P(A) · P(B)
because P(B | A) = P(B) when A and B are independent.

Example: Suppose we toss a fair coin and roll a fair six-sided die.
Let:
• A: the event that the coin shows heads. So, P(A) = 12
• B: the event that the die shows a 4. So, P(B) = 16
Since the outcome of the coin toss does not affect the outcome of the die roll, events A and B are
independent.
Therefore, 1 1 1
P(A ∩ B) = P(A) · P(B) = 2· 6= 12
1
Interpretation: The probability of getting a head on the coin and a 4 on the die is 12 .

Problem 1: With Replacement

A box contains 3 red balls and 2 green balls. A ball is drawn at random, its colour noted, and it is replaced.
Then another ball is drawn.
Find the probability that both balls are red.

Solution:
Since the ball is replaced, the draws are independent.
3
P(Red on one draw) = 5
3 3 9
P(Red and Red) = 5× 5= 25

8
Problem 2: Without Replacement
A bag has 4 blue balls and 3 yellow balls. Two balls are drawn without replacement. What
is the probability that the first is blue and the second is yellow?

Solution:
4 3 1
P(Blue first) = 7, P(Yellow second) = 6= 2
4 1 4 2
P(Blue then Yellow) = 7× 2= 14= 7

Problem 3: With Replacement

From a deck of 52 playing cards, one card is drawn at random, noted, and replaced. Then a second card is
drawn.
What is the probability that both cards are aces?

Solution:
 4 1
P(Ace) = 52= 13
1 1 1
P(Ace and Ace) = 13× 13= 169

Problem 4: Without Replacement

From the same deck of 52 cards, two cards are drawn without replacement.
Find the probability that both cards are aces.

Solution:
4 1 3
P(First ace) = 52= 13, P(Second ace) = 51
4 3 1 1 1
P(Two aces) = 52× 51= 13× 17= 221

Problem 5: Comparison of With and Without Replacement

A jar has 5 white marbles and 5 black marbles. Two marbles are drawn.
Compare the probability of drawing 2 black marbles:
(a) With replacement
(b) Without replacement

Solution:
(a) With Replacement: (b) Without Replacement: 5 1
P(Black) = 10= 2
1 1 1
P(Black, Black) = 2× 2= 4

5 4
P(Black first) = 10, P(Black second) = 9 9
5 4 20 2
P(Black, Black) = 10× 9= 90= 9
Conclusion: The probability is higher with replacement, since:
2
4= 0.25 > 9≈ 0.222
1

Self-Study Problems (Practice)

1. A jar contains 6 red balls and 4 white balls. Two balls are drawn with replacement. Find the probability
that both are white.
2. A class has 12 boys and 8 girls. Two students are selected without replacement. Find the probability that
both are girls.

3. A coin is tossed twice. What is the probability of getting a head in both tosses?
4. In a bag of 5 green, 3 yellow, and 2 red balls, one ball is drawn, not replaced, and then another ball is
drawn. Find the probability that the first is green and the second is red.
5. A digit is chosen at random from 1 to 9, replaced, and another digit is chosen. What is the probability that
both digits are even?
 10
Overview of a Standard 52-Card Deck
A standard deck contains 52 playing cards, divided into:

Figure 1: A standard 52-card deck divided into four suits: Spades, Hearts, Clubs, and Diamonds, each
containing 13 cards including numbered cards, face cards, and an Ace.(Source: Adobe Stock)

• 4 Suits: Hearts (♡), Diamonds (♢), Clubs (♣), Spades (♠)

• 13 Ranks in each suit: Ace (A), Numbers 2–10, Jack (J), Queen (Q), King (K)
Hearts ♡ (Red) Diamonds ♢ Clubs ♣ (Black) Spades ♠
(Red) (Black)

A A A A
2 2 2 2
3 3 3 3
4 4 4 4
5 5 5 5
6 6 6 6
7 7 7 7
8 8 8 8
9 9 9 9
10 10 10 10
J J J J
Q Q Q Q
K K K K

Note: Each suit contains exactly 13 cards, making a total of 4 × 13 = 52 cards in the full deck. 11
Theorem (Bayes’ Theorem for Two Events): Let A and B be two events in a probability space with P(B) > 0.
Then the conditional probability of A given B is given by

P(A | B) = P(B | A) · P(A)

P(B).
Proof:
By the definition of conditional probability, we have

P(A | B) = P(A ∩ B)
P(B)provided P(B) > 0. (1)
Also, from the definition of conditional probability,

P(B | A) = P(A ∩ B)
P(A)provided P(A) > 0.
Multiplying both sides by P(A), we obtain

P(A ∩ B) = P(B | A) · P(A). (2)

Substituting equation (2) into equation (1), we get

P(A | B) = P(B | A) · P(A)

P(B).
This completes the proof.

Definition: Complement of an Event

Let S be the sample space and A ⊆ S be an event. The complement of event A, denoted by Ac or A′, is the set
of all outcomes in the sample space that are not in A. That is,

Ac = {x ∈ S | x /∈ A}

The probability of the complement of an event is given by:

P(Ac) = 1 − P(A)

Probability of an event and its complement: By the definition of the complement, the events A and Ac are
mutually exclusive and exhaustive. That is,

A ∩ Ac = ∅ and A ∪ Ac = S.

Using the axioms of probability, we have:

P(A ∪ Ac) = P(A) + P(Ac).

Since A ∪ Ac = S, we get
P(S) = P(A) + P(Ac).
But by the normalization axiom of probability,

P(S) = 1.

Therefore,
P(A) + P(Ac) = 1.
 12
Some Problems and Solutions
1. A card is drawn at random from a standard deck. What is the probability that the card is not a face card?
Solution: Face cards = 12. Total = 52. Non-face cards = 52 − 12 = 40.
40 10
P(Not a face card) = 52= 13
2. A fair die is rolled once. What is the probability of not getting a 6?

1 5
P(Not 6) = 1 − P(6) = 1 − 6= 6
3. A card is drawn. What is the probability that it is not a heart?
Hearts = 13, so:
P(Not a heart) = 52 − 13
39 3
52= 52= 4
4. Two coins are tossed. What is the probability of not getting two heads?
Sample space: HH, HT, TH, TT P(HH) = 14

1 3
P(Not HH) = 1 − 4= 4
5. Three coins are tossed. What is the probability of getting at least one
tail? Complement: All heads (HHH) → P(All heads) = 18

1 7
P(At least one tail) = 1 − 8= 8
6. A card is drawn. What is the probability that it is not a red card? Red
cards (hearts and diamonds) = 26

P(Not red) = 52 − 26
26 1
52= 52= 2

7. A die is rolled. What is the probability that the result is not a 5? P(Not

1 5
5) = 1 − 6= 6

8. A die is rolled. What is the probability of not getting an even

number? Even numbers = 2, 4, 6 → 3 outcomes; Odd = 1, 3, 5 → 3
outcomes

3 1
P(Not even) = 6= 2
9. A card is drawn. What is the probability that it is not an ace?
Aces = 4
P(Not ace) = 52 − 4
48 12
52= 52= 13

10. A card is drawn. What is the probability that it is not a red king?
Red kings = 2 50 25
52= 52= 26

P(Not red king) = 52 − 2 13

11. In a survey, 30% of people like tea, 40% like coffee, and 10% like both. What is the probability that a
person likes either tea or coffee?
 Solution: Let T be the event ”likes tea” and C be the event ”likes coffee”. We are asked to find P(T ∪ C).
P(T ∪ C) = P(T) + P(C) − P(T ∩ C)
= 0.3 + 0.4 − 0.1 = 0.6
Answer: 0.6
12. The probability that a student passes the Physics exam is 0.7 and the probability of passing the
Chemistry exam is 0.8. The probability of passing both is 0.56. What is the probability that the student
passes at least one of the exams?
Solution: Let P be passing Physics and C be passing Chemistry.

P(P ∪ C) = P(P) + P(C) − P(P ∩ C)

= 0.7 + 0.8 − 0.56 = 0.94

Answer: 0.94

13. Two dice are thrown. Find the probability that the sum is at least 10.
Solution: Total possible outcomes when two dice are rolled = 6 × 6 = 36
Favourable outcomes where the sum is at least 10:

(4, 6),(5, 5),(5, 6),(6, 4),(6, 5),(6, 6) ⇒ 6 outcomes

6 1
P(Sum ≥ 10) = 36= 6
Answer: 16
14. Events A and B are independent with P(A) = 0.5, P(B) = 0.4. Find P(A ∩ B).
Solution: Since A and B are independent:

P(A ∩ B) = P(A) · P(B) = 0.5 · 0.4 = 0.2

Answer: 0.2

15. Events A and B are such that P(A) = 0.6, P(B′) = 0.3, P(A ∩ B) = 0.4. Find P(A ∪ B).
Solution: First, find P(B) using complement:

P(B) = 1 − P(B′) = 1 − 0.3 = 0.7

Now use the union formula:

P(A ∪ B) = P(A) + P(B) − P(A ∩ B) = 0.6 + 0.7 − 0.4 = 0.9

Answer: 0.9

16. Let P(A) = 0.2, P(B) = 0.3, and A and B are mutually exclusive. Find P(A ∪ B).
Solution: For mutually exclusive events, P(A ∩ B) = 0, so:

P(A ∪ B) = P(A) + P(B) = 0.2 + 0.3 = 0.5

Answer: 0.5

14
17. If P(A) = 0.45, what is P(A′)?
Solution: Use the complement rule:

P(A′) = 1 − P(A) = 1 − 0.45 = 0.55

Answer: 0.55
18. Events A and B are such that P(A ∪ B) = 0.85, P(A) = 0.5, and P(B) = 0.6. Find P(A ∩ B).
Solution: Rearranging the union formula:

P(A ∩ B) = P(A) + P(B) − P(A ∪ B) = 0.5 + 0.6 − 0.85 = 0.25

Answer: 0.25

19. A and B are independent events. If P(A) = 0.3, P(B′) = 0.6, find P(A ∩ B).
Solution: First find P(B) = 1 − P(B′) = 1 − 0.6 = 0.4.
Since A and B are independent:

P(A ∩ B) = P(A) · P(B) = 0.3 · 0.4 = 0.12

Answer: 0.12

20. If P(A) = 0.7, P(B) = 0.5, and P(A ∩ B) = 0.3, find P(A ∪ B′).
Solution:
P(B′) = 1 − P(B) = 1 − 0.5 = 0.5
Now find P(A ∩ B′) = P(A) − P(A ∩ B) = 0.7 − 0.3 = 0.4
Use the union formula:

P(A ∪ B′) = P(A) + P(B′) − P(A ∩ B′) = 0.7 + 0.5 − 0.4 = 0.8

Answer: 0.8

15