Aksum University Aksum Institute of Technology Faculty of ECE

Chapter 1
1.1Introduction to probability theory
Probability deals with unpredictability and randomness, and probability theory is the branch of
mathematics that is concerned with the study of random phenomena. The theory of probability
and random processes is useful for modeling the uncertainty of various events (e.g., the arrival of
telephone calls and the failure of electronic components). This theory provides a tool to model
and analyze phenomena that occur in many diverse fields, such as communications, signal
processing, control, and computers. Perhaps the major reason for studying probability and
random processes is to be able to model complex systems and phenomena. One of the main goals
of the course is to prepare the students to study random signals and systems. This material is
fundamental to the study of digital signal processing (voice, image, video, etc.), communications
systems and networks, radar systems, power systems, and many other applications.

1.2 Sample space and events

Random experiments: In the study of probability, any process of observation is referred to as
an experiment. The results of an observation are called the outcomes of the experiment. An
experiment is called a random experiment if its outcome cannot be predicted. Typical examples
of a random experiment are the roll of a die, the toss of a coin, drawing a card from a deck, or
selecting a message signal for transmission from several messages.
Sample Space: The set of all possible outcomes of a random experiment is called the sample
space (or universal set), and it is denoted by S. An element in S is called a sample point. Each
outcome of a random experiment corresponds to a sample point.
Example 1.1: Find the sample space for the experiment of tossing a coin (a) once and (b) twice.
(a) There are two possible outcomes, heads or tails. Thus
S = {H, T} where H and T represent head and tail, respectively.
(b) There are four possible outcomes. They are pairs of heads and tails. Thus
S = {HH, HT, TH, TT}
Example 1.2: Find the sample space for the experiment of tossing a coin repeatedly and of
counting the number of tosses required until the first head appears.

Probability and Random Process By: Tesfay H. Chapter 1 Page 1

Aksum University Aksum Institute of Technology Faculty of ECE

Clearly all possible outcomes for this experiment are the terms of the sequence 1, 2, 3, … . Thus
S = {1, 2, 3, …}. Note that there are an infinite number of outcomes.
Example 1.3: Find the sample space for the experiment of measuring (in hours) the lifetime of a
transistor.
Clearly all possible outcomes are all nonnegative real numbers. That is,
S = {τ: 0 ≤ τ ≤ ∞} where τ represents the life of a transistor in hours.
Note that any particular experiment can often have many different sample spaces depending on
the observation of interest. A sample space S is said to be discrete if it consists of a finite number
of sample points (as in Example 1.1) or countably infinite sample points (as in Example 1.2). A
set is called countable if its elements can be placed in a one-to-one correspondence with the
positive integers. A sample space S is said to be continuous if the sample points constitute a
continuum (as in Example 1.3).
Events: Since we have identified a sample space S as the set of all possible outcomes of a
random experiment, we will review some set notations in the following.
If ξ is an element of S (or belongs to S), then we write ξ ∈ S.
If ξ is not an element of S (or does not belong to S), then we write ξ ∉ S.
A set A is called a subset of B, denoted by A ⊂ B if every element of A is also an element of B.
Any subset of the sample space S is called an event. A sample point of S is often referred to as an
elementary event. Note that the sample space S is the subset of itself, that is, S ⊂ S. Since S is the
set of all possible outcomes, it is often called the certain event.
Example 1.4: Consider the experiment of Example 1.2. Let A be the event that the number of
tosses required until the first head appears is even. Let B be the event that the number of tosses
required until the first head appears is odd. Let C be the event that the number of tosses required
until the first head appears is less than 5. Express events A, B, and C.
A = {2, 4, 6, …}
B = {1, 3, 5, …}
C = {1, 2, 3, 4}
Example 1.5: A cubical die with numbered faces is rolled and the result observed. The sample
space consists of six possible outcomes, ξ1 = 1, ξ2 = 2, ..., ξ6 = 6, indicating the possible faces of
the cubical die that may be observed.
S = {1, 2, 3, 4, 5, 6}

Probability and Random Process By: Tesfay H. Chapter 1 Page 2

Aksum University Aksum Institute of Technology Faculty of ECE

Example 1.6: Consider the experiment of rolling two dice and observing the results. The sample
space consists of 36 outcomes, which may be labeled by the ordered pairs ξ1 = (1, 1), ξ2 = (1, 2),
ξ3 = (1, 3), ..., ξ6 = (1, 6), ξ7 = (2, 1), ξ8 = (2, 2), ..., ξ36 = (6, 6); the first component in the ordered
pair indicates the result of the toss of the first die, and the second component indicates the result
of the toss of the second die. Several interesting events can be defined from this experiment, such
as:
A = {the sum of the outcomes of the two rolls = 4},
B = {the outcomes of the two rolls are identical},
C = {the first roll was bigger than the second}.

1.3 Review of set theory

A set is a collection of objects possessing some common properties. These objects are called
elements of the set and they can be of any kind with any specified properties. We may consider,
for example, a set of numbers, a set of mathematical functions, a set of persons, or a set of a
mixture of things.
Capital letters A, B, C, Φ, Ω, … shall be used to denote sets, and lower-case letters a, b, c, ϕ, ω,
… to denote their elements. A set is thus described by its elements. Notationally, we can write,
for example,
A = {1, 2, 3, 4, 5, 6},
which means that set A has as its elements integers 1 through 6. For a set consisting of all
nonnegative real numbers, a convenient description is
C = {x: x ≥ 0}.
A set containing no elements is called an empty or null set and is denoted by ϕ. We distinguish
between sets containing a finite number of elements and those having an infinite number. They
are called, respectively, finite sets and infinite sets. An infinite set is called enumerable or
countable if all of its elements can be arranged in such a way that there is a one-to-one
correspondence between them and all positive integers; thus, a set containing all positive integers
1, 2, ... is a simple example of an enumerable set.
A nonenumerable or uncountable set is one where the above-mentioned one-to-one
correspondence cannot be established. A simple example of a nonenumerable set is the set C
described above.

Probability and Random Process By: Tesfay H. Chapter 1 Page 3

Aksum University Aksum Institute of Technology Faculty of ECE

Set operations:
1. Equality: Two sets A and B are equal, denoted A = B, if and only if
A ⊂ B and B ⊂ A.
2. Complementation: Suppose A ⊂ S. The complement of set A, denoted 𝐴̅, is the set containing
all elements in S but not in A.
𝐴̅ = {ξ: ξ ∈ S and ξ ∉ A} (2.1)
3. Union: The union of sets A and B, denoted A ∪ B, is the set containing all elements in either
A or B or both.
A ∪ B = {ξ: ξ ∈ A or ξ ∈ B} (1.2)
4. Intersection: The intersection of sets A and B, denoted A ∩ B, is the set containing all
elements in both A and B.
A ∩ B = {ξ: ξ ∈ A and ξ ∈ B} (1.3)
5. Null set: The set containing no element is called the null set, denoted ϕ. Note that
ϕ = 𝑆̅ (1.4)
6. Disjoint Sets: Two sets A and B are called disjoint or mutually exclusive if they contain no
common element, that is, if
A∩B=ϕ (1.5)
The definitions of the union and intersection of two sets can be extended to any finite number of
sets as follows:
⋃𝑛𝑖=1 𝐴𝑖 = 𝐴1 ∪ 𝐴2 ∪ … ∪ 𝐴𝑛 = {𝜉: 𝜉 ∈ 𝐴1 𝑜𝑟 𝜉 ∈ 𝐴2 𝑜𝑟 … 𝑜𝑟 𝜉 ∈ 𝐴𝑛 } (2.6)
⋂𝑛𝑖=1 𝐴𝑖 = 𝐴1 ∩ 𝐴2 ∩ … ∩ 𝐴𝑛 = {𝜉: 𝜉 ∈ 𝐴1 𝑎𝑛𝑑 𝜉 ∈ 𝐴2 𝑎𝑛𝑑 … 𝑎𝑛𝑑 𝜉 ∈ 𝐴𝑛 } (1.7)
In our definition of event, we state that every subset of S is an event, including S and the null set
ϕ. Then
S = the certain event
ϕ = the impossible event
If A and B are events in S, then
𝐴̅ = the event that A did not occur
A ∪ B = the event that either A or B or both occurred
A ∩ B = the event that both A and B occurred
Similarly, if A1, A2, …, An are the sequence of events in S, then

Probability and Random Process By: Tesfay H. Chapter 1 Page 4

Aksum University Aksum Institute of Technology Faculty of ECE

⋃𝑛𝑖=1 𝐴𝑖 = the event that at least one of the Ai occurred;

⋂𝑛𝑖=1 𝐴𝑖 = the event that all of the Ai occurred.

Venn diagram:
A graphical representation that is very useful for illustrating set operation is the Venn diagram.
For instance, in the three Venn diagrams shown in Fig. 2.1, the shaded areas represent,
respectively, the events A ∪ B, A ∩ B, and 𝐴̅. The Venn diagram in Fig. 2.2 indicates that B ⊂ A
and the event A ∩ 𝐵̅ is shown as the shaded area.

Figure 2.1 Venn diagram-1

Figure 2.2 Venn diagram-2

Probability and Random Process By: Tesfay H. Chapter 1 Page 5

Aksum University Aksum Institute of Technology Faculty of ECE

Identities:
By the above set definitions or reference to Fig. 2.1, we obtain the following identities:
𝑆̅ = 𝜙 𝐴̅ = 𝐴 S∩A=A A ∩ 𝐴̅ = ϕ
𝜙̅ = 𝑆 S∪A=S A ∪ 𝐴̅ = S
The union and intersection operations also satisfy the following laws:
Commutative Laws:
A∪B=B∪A
A∩B=B∩A
Associative Laws:
A ∪ (B ∪ C) = (A ∪ B) ∪ C
A ∩ (B ∩ C) = (A ∩ B) ∩ C
Distributive Laws:
A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)
A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)
De Morgan's Laws:
̅̅̅̅̅̅̅
𝐴 ∪ 𝐵 = 𝐴̅ ∩ 𝐵̅ (2.8)
̅̅̅̅̅̅̅
𝐴 ∩ 𝐵 = 𝐴̅ ∪ 𝐵̅ (2.9)
These relations are verified by showing that any element that is contained in the set on the left
side of the equality sign is also contained in the set on the right side, and vice versa. One way of
showing this is by means of a Venn diagram. The distributive laws can be extended as follows:
𝐴 ∩ (⋃𝑛𝑖=1 𝐵𝑖 ) = ⋃𝑛𝑖=1(𝐴 ∩ 𝐵𝑖 ) (2.10)
𝐴 ∪ (⋂𝑛𝑖=1 𝐵𝑖 ) = ⋂𝑛𝑖=1(𝐴 ∪ 𝐵𝑖 ) (2.11)
Similarly, De Morgan's laws also can be extended as follows:
̅̅̅̅̅̅̅̅̅̅̅̅
(⋃𝑛𝑖=1 𝐴𝑖 ) = ⋂𝑛𝑖=1 𝐴̅𝑖 (2.12)
̅̅̅̅̅̅̅̅̅̅̅̅
(⋂𝑛𝑖=1 𝐴𝑖 ) = ⋃𝑛𝑖=1 𝐴̅𝑖 (2.13)

Probability and Random Process By: Tesfay H. Chapter 1 Page 6

Aksum University Aksum Institute of Technology Faculty of ECE

1.4 Axioms of probability

Probability is a function of an event that produces a numerical quantity that measures the
likelihood of that event. There are many ways to define such a function, which is called
probability.
An assignment of real numbers to the events defined in a sample space S is known as the
probability measure. Consider a random experiment with a sample space S, and let A be a
particular event defined in S.

Definitions of probability
There are several ways to define probability. In this section we consider three definitions: the
relative-frequency definition, the axiomatic definition, and the classical definition.
A. Relative Frequency Definition:
Suppose that the random experiment is repeated n times. If event A occurs N(A) times, then the
probability of event A, denoted P(A), is defined as
𝑁(𝐴)
𝑃(𝐴) = lim (2.14)
𝑛→∞ 𝑛

where N(A)/n is called the relative frequency of event A. Note that this limit may not exist, and
in addition, there are many situations in which the concepts of repeatability may not be valid. It
is clear that for any event A, the relative frequency of A will have the following properties:
1. 0 ≤ N(A)/n ≤ 1, where N(A)/n = 0 if A occurs in none of the n repeated trials and N(A)/n
= 1 if A occurs in all of the n repeated trials.
2. If A and B are mutually exclusive events, then
𝑁(𝐴 ∪𝐵) 𝑁(𝐴) 𝑁(𝐵)
𝑁(𝐴 ∪ 𝐵) = 𝑁(𝐴) + 𝑁(𝐵) and = + (2.15)
𝑛 𝑛 𝑛

B. Axiomatic Definition:
Let S be a finite sample space and A be an event in S. Then in the axiomatic definition, the
probability of the event A, P(A), is a real number assigned to A which satisfies the following
three axioms :
Axiom 1: P(A) ≥ 0
Axiom 2: P(S) = 1
Axiom 3: P(A ∪ B) = P(A) + P(B) if A ∩ B = ϕ
If the sample space S is not finite, then axiom 3 must be modified as follows:

Probability and Random Process By: Tesfay H. Chapter 1 Page 7

Aksum University Aksum Institute of Technology Faculty of ECE

Axiom 3': If A1, A2, … is an infinite sequence of mutually exclusive events in S (Ai ∩ Aj = ϕ for
i ≠ j), then
𝑃(⋃∞ ∞
𝑖=1 𝐴𝑖 ) = ∑𝑖=1 𝑃(𝐴𝑖 ) (2.16)
These axioms satisfy our intuitive notion of probability measure obtained from the notion of
relative frequency.
C. Classical Definition:
In the classical definition, the probability of an event A, P(A), is the ratio of the number of
outcomes M(A) of an experiment that are favorable to A to the total number of possible
outcomes of the experiment M. That is,
𝑀(𝐴)
𝑃(𝐴) = 𝑀
(2.17)

This probability is determined apriori without actually performing the experiment. For example,
in a coin toss experiment, there are two possible outcomes: heads or tails. Thus, M = 2, and if the
coin is fair, the probability of the event that the toss comes up heads is 1/2.

Elementary Properties of Probability

By using the above axioms, the following useful properties of probability can be obtained:
1. P(𝐴̅) = 1 – P(A)
2. P(ϕ) = 0
3. P(A) ≤ P(B) if A ⊂ B
4. P(A) ≤ 1
5. P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
6. If A1, A2, …, An are n arbitrary events in S, then
𝑃(⋃𝑛𝑖=1 𝐴𝑖 ) = ∑𝑛𝑖=1 𝑃(𝐴𝑖 ) − ∑𝑖≠𝑗 𝑃(𝐴𝑖 ∩ 𝐴𝑗 ) + ∑𝑖≠𝑗≠𝑘 𝑃(𝐴𝑖 ∩ 𝐴𝑗 ∩ 𝐴𝑘 ) − ⋯ + (−1)𝑛−1 𝑃(𝐴1 ∩ 𝐴2 ∩
… ∩ 𝐴𝑛 ) (2.18)
where the sum of the second term is over all distinct pairs of events, that of the third term is over all
distinct triples of events, and so forth.
7. If A1, A2, …, An is a finite sequence of mutually exclusive events in S (Ai ∩ Aj = ϕ for i≠ j), then
𝑃(⋃𝑛𝑖=1 𝐴𝑖 ) = ∑𝑛𝑖=1 𝑃(𝐴𝑖 ) (2.19)
and a similar equality holds for any subcollection of events.
Note that property 4 can be easily derived from axiom 2 and property 3. Since A ⊂ S, we have
P(A) ≤ P(S) = 1

Probability and Random Process By: Tesfay H. Chapter 1 Page 8

Aksum University Aksum Institute of Technology Faculty of ECE

Thus, combining with axiom 1, we obtain 0 ≤ P(A) ≤ 1. Property 5 implies that P(A ∪ B) ≤
P(A) + P(B) since P(A ∩ B) ≥ 0 by axiom 1.
Example 1.7: Two fair dice are tossed. Find the probability of each of the following events:
a. The sum of the outcomes of the two dice is equal to 7
b. The sum of the outcomes of the two dice is equal to 7 or 11
c. The outcome of the second die is greater than the outcome of the first die
d. Both dice come up with even numbers
Solution: The total number of sample points is 36. We evaluate the three probabilities using the
classical definition method.
(a) Let A1 denote the event that the sum of the outcomes of the two dice is equal to seven. Then
A1 = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}. Since the number of sample points in the event is
6, we have that P(A1) = 6/36 = 1/6.
(b) Let B denote the event that the sum of the outcomes of the two dice is either seven or eleven,
and let A2 denote the event that the sum of the outcomes of the two dice is eleven. Then, A2 =
{(5, 6), (6, 5)} with 2 sample points. Thus, P(A2) = 2/36 = 1/18. Since B is the union of A1 and
A2, which are mutually exclusive events, we obtain P(B) = P(A1 ∪ A2) = P(A1) + P(A2) = 1/6 +
1/18 = 2/9.
(c) Let C denote the event that the outcome of the second die is greater than the outcome of the
first die. Then C = {(1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (2, 3), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3,
6), (4, 5), (4, 6), (5, 6)} with 15 sample points. Thus, P(C) = 15/36 = 5/12.
(d) Let D denote the event that both dice come up with even numbers. Then D = {(2, 2), (2, 4),
(2, 6), (4, 2), (4, 4), (4, 6), (6, 2), (6, 4), (6, 6)} with 9 sample points. Thus, P(D) = 9/36 = 1/4.

Probability and Random Process By: Tesfay H. Chapter 1 Page 9

Aksum University Aksum Institute of Technology Faculty of ECE

1.5 Conditional Probability

Consider a finite sample space S with n finite elements S = {ξ1, ξ2, …, ξn} where ξi's are
elementary events. Let P(ξi) = pi. Then
1. 0 ≤ pi ≤ 1, i = 1, 2, …, n
2. ∑𝑛𝑖=1 𝑝𝑖 = 𝑝1 + 𝑝2 + ⋯ + 𝑝𝑛 = 1
3. If A = ⋃𝑖∈𝐼 𝜉𝑖 , where I is a collection of subscripts, then
𝑃(𝐴) = ∑𝜉𝑖 ∈𝐴 𝑃(𝜉𝑖 ) = ∑𝑖∈𝐼 𝑝𝑖
When all elementary events ξi (i = 1, 2, …, n) are equally likely, that is, p1 = p2 = … = pn then
1 𝑀(𝐴)
𝑝𝑖 = 𝑛 𝑖 = 1, 2, … , 𝑛 and P(A) = 𝑀

where M(A) is the number of outcomes belonging to event A and M is the number of sample
points in S. The conditional probability of an event A given event B, denoted by P(A|B), is
defined as
𝑃(𝐴∩𝐵)
𝑃(𝐴|𝐵) = 𝑃(𝐵) > 0 (2.20)
𝑃(𝐵)

where P(A ∩ B) is the joint probability of A and B. Similarly,

𝑃(𝐴∩𝐵)
𝑃(𝐵|𝐴) = 𝑃(𝐴) > 0 (2.21)
𝑃(𝐴)

is the conditional probability of an event B given event A. From Equations (2.20) and (2.21), we
have:
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴|𝐵)𝑃(𝐵) = 𝑃(𝐵|𝐴)𝑃(𝐴) (2.22)
Equation (2.22) is often quite useful in computing the joint probability of events.
Bayes' Rule
From Equation (2.22) we can obtain the following Bayes' rule:
𝑃(𝐵 |𝐴)𝑃(𝐴)
𝑃(𝐴|𝐵) = (2.23)
𝑃(𝐵)

The events A1, A2, …, An are called mutually exclusive and exhaustive if
𝑛

⋃ 𝐴𝑖 = 𝐴1 ∪ 𝐴2 ∪ … ∪ 𝐴𝑛 𝑎𝑛𝑑 𝐴𝑖 ∩ 𝐴𝑗 = 𝜙 𝑖 ≠ 𝑗
𝑖=1

Let B be any event in S. Then

𝑃(𝐵) = ∑𝑛𝑖=1 𝑃(𝐵 ∩ 𝐴𝑖 ) = ∑𝑛𝑖=1 𝑃(𝐵|𝐴𝑖 )𝑃(𝐴𝑖 ) (2.24)

Probability and Random Process By: Tesfay H. Chapter 1 Page 10

Aksum University Aksum Institute of Technology Faculty of ECE

which is known as the total probability of event B. Let A = Ai in Equation (2.23); then, using
Equation (2.24), we obtain
𝑃(𝐵 |𝐴𝑖 )𝑃(𝐴𝑖 )
𝑃(𝐴𝑖 |𝐵) = ∑𝑛 (2.25)
𝑖=1 𝑃( 𝐵|𝐴𝑖 )𝑃(𝐴𝑖 )
Note that the terms on the right-hand side are all conditioned on events Ai, while the term on the
left is conditioned on B. Equation (2.25) is sometimes referred to as Bayes' theorem.
Example 1.8: A student buys 1000 integrated circuits (ICs) from supplier A, 2000 ICs from
supplier B, and 3000 ICs from supplier C. He tested the ICs and found that the conditional
probability of an IC being defective depends on the supplier from whom it was bought.
Specifically, given that an IC came from supplier A, the probability that it is defective is 0.05;
given that an IC came from supplier B, the probability that it is defective is 0.10; and given that
an IC came from supplier C, the probability that it is defective is 0.10. If the ICs from the three
suppliers are mixed together and one is selected at random, what is the probability that it is
defective?
Solution: Let P(A), P(B), and P(C) denote the probability that a randomly selected IC came from
supplier A, B, and C, respectively. Also, let P(D|A) denote the conditional probability that an IC
is defective, given that it came from supplier A; P(D|B) denote the conditional probability that an
IC is defective, given that it came from supplier B; and P(D|C) denote the conditional probability
that an IC is defective, given that it came from supplier C. Then the following are true:
P(D|A) = 0.05, P(D|B) = 0.10, P(D|C) = 0.10, P(A) = 1/6, P(B) = 1/3, and P(C) = ½.
Let P(D) denote the unconditional probability that a randomly selected IC is defective. Then,
from the principles of total probability,
P(D) = P(D|A)P(A) + P(D|B)P(B) + P(D|C)P(C)
= (0.05)(1/6) + (0.10)(1/3) + (0.10)(1/2)
= 0.09167
Example 1.9: In Example 2.8, given that a randomly selected IC is defective, what is the
probability that it came from supplier A?
Solution: Using the same notation as in Example 2.8, the probability that the randomly selected
IC came from supplier A, given that it is defective, is given by:
1
𝑃(𝐷|𝐴)𝑃(𝐴) 0.05 ∗ 6
𝑃(𝐴|𝐷) = = = 0.0909
𝑃(𝐷) 0.09167

Probability and Random Process By: Tesfay H. Chapter 1 Page 11

Aksum University Aksum Institute of Technology Faculty of ECE

1.6 Independent events

Two events A and B are said to be (statistically) independent if and only if
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴)𝑃(𝐵) (2.26)
It follows immediately that if A and B are independent, then by Equations (2.20) and (2.21),
𝑃(𝐴|𝐵) = 𝑃(𝐴) 𝑎𝑛𝑑 𝑃(𝐵|𝐴) = 𝑃(𝐵) (2.27)
If two events A and B are independent, then it can be shown that A and 𝐵̅ are also independent;
that is,
𝑃(𝐴 ∩ 𝐵̅ ) = 𝑃(𝐴)𝑃(𝐵̅ )
𝑃(𝐴∩𝐵) ̅
Then 𝑃(𝐴|𝐵̅ ) = 𝑃(𝐵̅) = 𝑃(𝐴)

Thus, if A is independent of B, then the probability of A's occurrence is unchanged by

information as to whether or not B has occurred. Three events A, B, C are said to be independent
if and only if
𝑃(𝐴 ∩ 𝐵 ∩ 𝐶) = 𝑃(𝐴)𝑃(𝐵)𝑃(𝐶) (2.28)
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴)𝑃(𝐵), 𝑃(𝐴 ∩ 𝐶) = 𝑃(𝐴)𝑃(𝐶), and 𝑃(𝐵 ∩ 𝐶) = 𝑃(𝐵)𝑃(𝐶)
We may also extend the definition of independence to more than three events. The events A1, A2,
…, An are independent if and only if for every subset { Ai1, Ai2, …, Aik} (2 ≤ k ≤ n) of these
events,
𝑃(𝐴𝑖1 ∩ 𝐴𝑖2 ∩ … ∩ 𝐴𝑖𝑘 ) = 𝑃(𝐴𝑖1 )𝑃(𝐴𝑖2 ) … 𝑃(𝐴𝑖𝑘 ) (2.29)
Finally, we define an infinite set of events to be independent if and only if every finite subset of
these events is independent. To distinguish between the mutual exclusiveness (or disjointness)
and independence of a collection of events we summarize as follows:
1. If {Ai, i = 1, 2, …, n} is a sequence of mutually exclusive events, then
𝑃(⋃𝑛𝑖=1 𝐴𝑖 ) = ∑𝑛𝑖=1 𝑃(𝐴𝑖 ) (2.30)
2. If {Ai, i = 1, 2, …, n} is a sequence of independent events, then
𝑃(⋂𝑛𝑖=1 𝐴𝑖 ) = ∏𝑛𝑖=1 𝑃(𝐴𝑖 ) (2.31)
and a similar equality holds for any subcollection of the events.

Probability and Random Process By: Tesfay H. Chapter 1 Page 12

Aksum University Aksum Institute of Technology Faculty of ECE

Example 2.0: (The binary symmetric channel) A discrete channel is characterized by an input
alphabet X ={x1, x2, …, xn}; an output alphabet Y ={y1, y2, …, ym}; and a set of conditional
probabilities (called transition probabilities), Pij, which are defined as follows: Pij = P(yj|xi) =
P(receiving symbol yj|symbol xi was transmitted), i = 1, 2,..., n; j = 1, 2,..., m. The binary channel
is a special case of the discrete channel, where n = m = 2. It can be represented as shown in
Figure 2.3. In the binary channel, an error occurs if y2 is received when x1 is transmitted or y1 is
received when x2 is transmitted. Thus, the probability of error, Pe, is given by
Pe = P(x1 ∩ y2) + P(x2 ∩ y1)
= P(x1)P(y2|x1) + P(x2)P(y1|x2)
= P(x1)P12 + P(x2)P21

Figure 2.3 The binary channel

If P12 = P21, we say that the channel is a binary symmetrical channel (BSC). Also, if in the BSC
P(x1) = p, then P(x2) = 1 − p = q. Consider the BSC shown in Figure 2.4, with P(x1) = 0.6 and
P(x2) = 0.4. Evaluate the following:
a. The probability that x1 was transmitted, given that y2 was received
b. The probability that x2 was transmitted, given that y1 was received
c. The probability that x1 was transmitted, given that y1 was received
d. The probability that x2 was transmitted, given that y2 was received
e. The unconditional probability of error

Probability and Random Process By: Tesfay H. Chapter 1 Page 13

Aksum University Aksum Institute of Technology Faculty of ECE

Solution:

Figure 2.4 The Binary Symmetric Channel for Example 2.10

Probability and Random Process By: Tesfay H. Chapter 1 Page 14

Aksum University Aksum Institute of Technology Faculty of ECE

Probability and Random Process By: Tesfay H. Chapter 1 Page 15

Aksum University Aksum Institute of Technology Faculty of ECE

Example 2.1: Two events A and B are such that P(A∩B) = 0.15, P(A∪B) = 0.65, and P(A|B) =
0.5. Find P(B|A).
Solution:
P(A∪B) = P(A) + P(B) – P(A∩B) ⇒ 0.65 = P(A) + P(B) − 0.15.
This means that:
P(A) + P(B) = 0.65 + 0.15 = 0.80. Also, P(A∩B) = P(A|B) P(B).
This then means that:
P(B) = P(A∩B)/P(A|B) = 0.15/0.50 = 0.30
Thus,
P(A) = 0.80 − 0.30 = 0.50.
Since
P(A∩B) = P(B|A)P(A),
we have that
P(B|A) = P(A∩B)/P(A) = 0.15/0.50 = 0.30.

Probability and Random Process By: Tesfay H. Chapter 1 Page 16