Principles of Communication System Module-1

Module-1
Random Variables and Processes
Introduction:
 Random signals are encountered in every practical communication system.
 A signal is ‘‘random’’ if it is not possible to predict its precise value in advance.
 In communication received signal consists of information signal and noise. Thus the
received signal is completely random in nature.
 A major source of receiver noise is thermal noise, which is caused by the random
motion of electrons.
 It is not possible to predict the precise value of a random signal in advance, but it may
be described in terms of its statistical properties such as the average power in the
random signal, or the average spectral distribution of this power.
 The mathematical discipline that deals with the statistical characterization of random
signals is probability theory.
Probability:
 Probability can be modeled by an experiment with an outcome that is subject to chance.
if the experiment is repeated, the outcome can differ because of the influence of an
underlying random phenomenon or chance mechanism.
 For example, the experiment may be the observation of the result of tossing a fair coin.
the possible outcomes of a trial are ‘‘heads’’ or ‘‘tails.’’

Properties (axioms):
1) The probability of any event will be between 0 and 1.

2) The total probability of the sample space is 1

3) If A and B are two mutually exclusive events, then

Fig: Venn diagram presenting a

geometric interpretation

of the three
axioms of probability
The following properties of probability measure P may be derived from the above
axioms:
where 𝐴̅ is the complement of the event 𝐴.

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 1

Principles of Communication System Module-1

2. When events A and B are not mutually exclusive, then the probability of the union
event ‘‘A or B’’ satisfies
Where P[A ∩ B] is the probability of the joint event ‘‘A and B.’’
3. If A1, A2,....., Am are mutually exclusive events that include all possible outcomes
of the random experiment, then

Relationship between sample space, events, and probability.

Fig: Illustration of the relationship between sample space, events, and probability.
 Probability can be modeled by an experiment with an outcome that is subject to chance.
if the experiment is repeated, the outcome can differ because of the influence of an
underlying random phenomenon or chance mechanism.
 An outcome (sample point) is a single possible result of a random experiment.
 The set of all possible outcomes of the experiment is called the sample space, which
we denote by S.
 An event corresponds to either a single sample point or a set of sample points in the
space S.
 The probability function assigns a value between 0 and 1 to each of these events.
 Two events are mutually exclusive if the occurrence of one event precludes the
occurrence of the other event.

What is conditional probability? Prove that

 Suppose in an experiment that involves a pair of events A and B. Let P[B|A] denote the
probability of event B, given that event A has occurred. The probability P[B|A] is called
the conditional probability of B given A.
 Assuming that A has nonzero probability, the conditional probability P[B|A] is defined
by

where is the joint probability of A and B

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 2

Principles of Communication System Module-1

or

 The joint probability of two events may be expressed as the product of the conditional
probability of one event given the other, and the elementary probability of the other.

The above equation is called as Baye’s rule.

Random variables
 Random variable describes the process of assigning a number or a range of values to
the outcomes of a random experiment.
Example: in tossing a coin head corresponds to 1 and tail to 0.
OR
 A function whose domain is a sample space and whose range is a set of real numbers
is called a random variable of the experiment.
 If the outcome of the experiment is s, we denote the random variable as X(s) or X.
 A particular outcome of a random experiment is 𝑋(𝑠𝑘 ) = 𝑥.

Fig: Illustration of the relationship between sample space, random variable, and probability

 Random variables may be discrete and take only a finite number of values, such as in
the coin-tossing experiment. Alternatively, random variables may be continuous and
take a range of real values.
 Consider the random variable X and the probability of the event X ≤ x is P[X ≤ x].

 The function FX(x) is called the cumulative distribution function (cdf ) or simply the
distribution function of the random variable X.

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 3

Principles of Communication System Module-1

The distribution function FX(x) has the following properties:

1. The distribution function FX(x) is bounded between zero and one.
2. The distribution function FX(x) is a monotone-nondecreasing function of x; that is,

The derivative of the distribution function is called the probability density function (pdf )

Several Random Variables

 Few experiments requires several random variables for its description.
 Consider two random variables X and Y. the joint distribution function

 If joint distribution function FX,Y (x,y) is continuous then the partial derivative

the function fX,Y (x,y) the joint probability density function of the random variables X
and Y.
 The joint distribution function FX,Y(x,y) is a monotone-nondecreasing function of both
x and y.
 The total volume under the graph of a joint probability density function must be unity.
 The conditional probability density function of Y given that X = x is defined by

Statistical Averages:
 The expected value or mean of a random variable X is defined by

where E denotes the statistical expectation operator.

Function of a random variable

 Let X denote a random variable, and let g(X) denote a real-valued function

 Expected value of the random variable Y

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 4

Principles of Communication System Module-1

Example: Cosinusoidal Random Variable

where X is a random variable uniformly distributed in the interval (-π,π);

the expected value of Y is

Moments
 For the special case of g(X) = Xn , nth moment of the probability distribution of the
random variable X

 Most important moments of X are the first two moments.

For n=1, mean of the random variable

For n=2, mean-square value of X

Central moments
 Moments of the difference between a random variable X and its mean µX.
 nth central moment is

For n = 1, the central moment is zero.

for n = 2 the second central moment is referred to as the variance of the random
variable X

 The variance of a random variable X is commonly denoted as σ𝑋2 .

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 5

Principles of Communication System Module-1

 The square root of the variance, is called the standard deviation of the random
variable X.

Random Processes
 Statistical analysis of communication systems is the characterization of random
signals such as voice signals, television signals, computer data, and electrical noise.
 These random signals have two properties:
1. the signals are functions of time
2. the signals are random; it is not possible to describe exactly the waveforms
that will be observed.
 The sample space or ensemble comprised of functions of time is called a random or
stochastic process.
 Consider then a random experiment specified by the outcomes s from some sample
space S.

Fig: An ensemble of sample functions.

Definition of a random process X(t)

 An ensemble of time functions together with a probability rule that assigns a
probability to any meaningful event associated with an observation of one of the
sample functions of the random process.
 Difference between a random variable and a random process as follows:

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 6

Principles of Communication System Module-1

 For a random variable, the outcome of a random experiment is mapped into a

number.
 For a random process, the outcome of a random experiment is mapped into a
waveform that is a function of time.
Mean
 Consider a random process X(t). We define the mean of the process X(t) as the
expectation of the random variable obtained by observing the process at some time t.

Where is the probability density function of the process at time t

 The mean of the random process is a constant

Autocorrelation Function (ACF)

 The autocorrelation function of the process X(t) as the expectation of the product of
two random variables X(t1) and X(t2), obtained by observing X(t) at times t1 and t2,
respectively.

Where is the joint probability density function of the random variables

X(t1) and X(t2).
 the autocorrelation function of a stationary random process depends only on the
difference between the observation times t1 and t2.

Properties of the autocorrelation function:

Redefining the autocorrelation function of a stationary process X(t) as

1) The mean-square value of the process may be obtained from 𝑅𝑋 (𝜏) simply by
putting 𝜏 = 0 in the above equation

2) The autocorrelation function 𝑅𝑋 (𝜏) is an even function of 𝜏

autocorrelation function 𝑅𝑋 (𝜏) can be re written as

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 7

Principles of Communication System Module-1

3)The autocorrelation function 𝑅𝑋 (𝜏) has its maximum magnitude at 𝜏 = 0

Proof: Consider the nonnegative quantity

Expanding terms

From property 1

Autocovariance function
 Autocovariance is the correlation between deviations of random process from its
mean at two different times.
 The autocovariance function of a stationary random process X(t) is written as

Cross-Correlation Function
 Cross-correlation measures similarity between two signals or random processes as a
function of the time shift between them.
i.e., how much one signal resembles another when one is shifted in time.
 Consider two random processes X(t) and Y(t) with autocorrelation functions RX(t,u)
and RY(t,u).
 The cross-correlation function of X(t) and Y(t) is defined by

 If the random processes X(t) and Y(t) are stationary, then

Properties of cross-correlation
 The cross-correlation function is not generally an even function of 𝜏.
 It does not have a maximum at the origin.
 It obeys a certain symmetry.

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 8

Principles of Communication System Module-1

Problem:

The random variable its plot is given as 𝒇𝑿(𝒙) = 𝒆−𝟐𝒙 for x≥0. Find the probability that
it will take value between 1 and 3.
Solution:

Given

Evaluate the integral

Apply limits:

Gaussian process
 A Gaussian process is a type of random process used in probability, statistics, and
communication systems.
 A Gaussian process is a random process X(t) such that for any set of time instants
t1,t2,...,tn the random variables have a joint Gaussian (normal)
distribution.
 Instead of a single random variable, a Gaussian process deals with a collection of
random variables indexed by time (or space).

Gaussian distribution function

 A Gaussian distribution function is a probability density function that describes a
normal (bell-shaped) distribution of a random variable around its mean.

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 9

Principles of Communication System Module-1

 It is a probability distribution where:

 Most values are close to the average (mean)
 Fewer values occur as you move away from the mean
 The graph forms a bell-shaped curve
 Symmetrical about the mean

Properties:
Property 1:
The Gaussian PDF is symmetric about its mean value μ.
𝑓X (μ − σ) = 𝑓X (μ + σ)

Property 2:
The Gaussian PDF reaches its peak at the mean value x=μ.

Property 3:
The area under the Gaussian PDF curve below and above the mean value is ½.

The total area under the Gaussian PDF curve is 1

Property 4:

For a Gaussian (normal) distribution:

 The first derivative is positive for x<μ

 The first derivative is zero at x=μ
 The first derivative is negative for x>μ

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 10

Principles of Communication System Module-1

Develop program to generate the probability density function of gaussian distribution

function.
% Parameters
mu = 0; % Mean
sigma = 1; % Standard deviation

% Range of x values
x = -10:0.01:10;

% Gaussian PDF formula

pdf = (1/(sigma*sqrt(2*pi))) * exp(-((x - mu).^2) / (2*sigma^2));

% Plot
plot(x, pdf, 'b', 'LineWidth', 2);
grid on;
xlabel('x');
ylabel('Probability Density Function');
title('Gaussian (Normal) Distribution PDF');

Explain central limit theorem as applied to Gaussian random process

 Adding a large number of independent random variables, the results a sum which tends
to follow a Gaussian (normal) distribution, regardless of the original distributions
(provided they have finite mean and variance).
 let Xi, i = 1, 2, . . . , N, be a set of random variables that satisfies the following
requirements:

1. The Xi are statistically independent.

2. The Xi have the same probability distribution with mean μX and variance σ𝑋2 .
 Let these random variables be normalized as follows;

 The central limit theorem states that the probability distribution of VN approaches a
normalized Gaussian distribution N(0,1) in the limit as N approaches infinity. That is,
regardless of the distribution of the Yi, the sum VN approaches a Gaussian distribution.

SHIVA KUMAR S/Dept. of ECE/AIET/Devanahalli 11