Introductory Lecture – Random Process

2/10/2024 Dept. of ECE, ASE, Bengaluru 1

 Introduction

• A random process is the natural extension of random variables when

dealing with signals.
• In many situations, the deterministic assumption on time-varying signals
is not a valid assumption, and it is more appropriate to model signals as
random rather than deterministic functions.
• One such example is the case of thermal noise in electronic circuits.
• Thermal noise is due to the random movement of electrons as a result
of thermal agitation, therefore, the resulting current and voltage can
only be described statistically.
• A random process, or a random signal, can be viewed as a set of
possible realizations of signal waveforms.
• The realization of one from the set of possible signals is governed by
some probabilistic law.

2/10/2024 Dept. of ECE, ASE, Bengaluru 2

 Example of random process

• Assume that we have a signal generator that can generate one of

the six possible sinusoids. The amplitude of all sinusoids is one,
and the phase for all of them is zero, but the frequencies can be
100, 200, . . . , 600 Hz.
• We throw a die, and depending on its outcome, which we denote
by F, we generate a sinusoid whose frequency is 100 times what
the die shows (100*F).
• This means that each of the six possible signals will be realized
with equal probability.
• This random process can be defined as
• 𝑋𝑋 𝑡𝑡 = 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴(2𝜋𝜋100𝐹𝐹𝐹𝐹)

2/10/2024 Dept. of ECE, ASE, Bengaluru 3

 Example of random process
Assume that we uniformly choose a phase between 0 and 2π and generate a
sinusoid with a fixed amplitude and frequency but with a random phase φ. In
this case, the random process is

𝑋𝑋 𝑡𝑡 = 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴(2𝜋𝜋𝜋𝜋ot+ φ),where A and fo denote the fixed amplitude and

frequency respectively and φ denotes the random phase.

we see that corresponding to each outcome ω;in a samples pace Ω, there

exists a signal x(t; ωi).

This description is similar to the description of random variables in

which a real number is assigned to each outcome ωi.

2/10/2024 Dept. of ECE, ASE, Bengaluru 4

 Sample functions of the random Process
𝑋𝑋 𝑡𝑡 = 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴(2𝜋𝜋𝜋𝜋0t+ φ),

2/10/2024 Dept. of ECE, ASE, Bengaluru 5

 Sample functions of the random process
• For each ωi; there exists a deterministic time function x(t;
ωi;) , which is called a sample function or a realization of
the random process.

• At each time instant to and for each, we have the

number

• For the different outcomes (ωi 's) at a fixed time t0, the numbers
x(t0; ωi) constitute a random variable denoted by X (t0)

• As, a random variable is nothing but an assignment of real

numbers to the outcomes of a random experiment, any time
instant, the value of a random process is a random variable.
2/10/2024 Dept. of ECE, ASE, Bengaluru 6
 Sample functions of a random process

2/10/2024 Dept. of ECE, ASE, Bengaluru 7

Connectivity between random variable and random
process

2/10/2024 Dept. of ECE, ASE, Bengaluru 8

 Statistical averages
• At any given time ,the RP defines a RV
• For instance, at any time instance t0,the RP at that time X(t0) is an
ordinary RV, which has a density function ,mean and variance at that
time.
• Even though mean and variance are deterministic numbers ,they
depend on the time t0.
• That is ,at time t1,the density function ,mean and the variance of
X(t1)will be different from that of X(t0)

2/10/2024 Dept. of ECE, ASE, Bengaluru 9

 Mean or expectation of a random process

2/10/2024 Dept. of ECE, ASE, Bengaluru 10

 Pictorial representation of the mean of a
random process

2/10/2024 Dept. of ECE, ASE, Bengaluru 11

 Autocorrelation Function

2/10/2024 Dept. of ECE, ASE, Bengaluru 12

 Wide sense stationary process

• Again, a random process observed at any given time is just a random

variable and the properties of this random variable depend on the time
at which the random process is observed.
• It can happen that some of the properties of this random variable are
independent of time.
• Depending on what properties are independent of time, different
notions of stationarity can be defined.
• One of the most useful notions of stationarity is the notion of wide-
sense stationary (WSS) random processes.
• A process is WSS if its mean and autocorrelation do not depend
on the choice of the time origin.

2/10/2024 Dept. of ECE, ASE, Bengaluru 13

 Wide sense stationary process

2/10/2024 Dept. of ECE, ASE, Bengaluru 14

 Example

• Let 𝑋𝑋 𝑡𝑡 = 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴(2𝜋𝜋𝜋𝜋0t+ φ) be a random process, where A

and fo denote the fixed amplitude and frequency
respectively. Also, assume that the phase φ is uniformly
distributed between 0 and 2π.Check whether the process is
WSS or not.

2/10/2024 Dept. of ECE, ASE, Bengaluru 15

 Solution

2/10/2024 Dept. of ECE, ASE, Bengaluru 16

 Therefore, the given process is WSS.

2/10/2024 Dept. of ECE, ASE, Bengaluru 17

 Example

2/10/2024 Dept. of ECE, ASE, Bengaluru 18

2/10/2024 Dept. of ECE, ASE, Bengaluru 19
 Multiple random Process

• Multiple random processes arise naturally when dealing with two or more
random processes.
• For example, take the case where we are dealing with a random process
X (t) and we pass it through a linear time-invariant (LTI) system.
• For each sample function input , we have a sample function
output where h(t) denotes the
impulse response of the system.
• We can see that for each , we have the two
signals and . Therefore, we are dealing with two random
processes, X (t) and Y (t).

2/10/2024 Dept. of ECE, ASE, Bengaluru 20

 Dependency between the
random processes

we know that the independence of random processes implies that they are
uncorrelated, whereas uncorrelatedness generally does not imply
independence, except for the important class of Gaussian processes for which
the two properties are equivalent.

2/10/2024 Dept. of ECE, ASE, Bengaluru 21

 Cross correlation between two
random Processes

2/10/2024 Dept. of ECE, ASE, Bengaluru 22

 Jointly Stationary

2/10/2024 Dept. of ECE, ASE, Bengaluru 23

 Example

2/10/2024 Dept. of ECE, ASE, Bengaluru 24

 Solution

2/10/2024 Dept. of ECE, ASE, Bengaluru 25

 Random Processes and linear systems
We know that, when a random process passes through an LTI
system, the output is also a random process .
This section focusses on the properties of the output process based on
the knowledge of the input process.
Assuming that a stationary process X (t) is the input to an LTI system
with the impulse response h(t) and the output process is denoted by
Y(t),we would try the find solutions for the following questions.

Under what conditions will the output process be stationary?

Under what conditions will the input and output processes be jointly
stationary?
How can we obtain the mean and the autocorrelation of the output
process, as well as the cross correlation between the input and output
processes?

2/10/2024 Dept. of ECE, ASE, Bengaluru 26

2/10/2024 Dept. of ECE, ASE, Bengaluru 27
 Mean of the output random Process in terms of
the input random process

2/10/2024 Dept. of ECE, ASE, Bengaluru 28

 Cross correlation between the output and the input

2/10/2024 Dept. of ECE, ASE, Bengaluru 29

 Autocorrelation of output process in
terms of cross correlation between the
input and output process

2/10/2024 Dept. of ECE, ASE, Bengaluru 30

 Power Spectral Density of the Stationary
Processes

• A random process is a collection of signals

• Spectral characteristics of these signals determine the spectral
characteristics of the random process
• If the signals of the random process are slowly varying, then the
random process will mainly contain low frequencies and its power will
be mostly concentrated at low frequencies
• On the other hand, if the signals change very fast, then most of the
power in the random process will be at the high-frequency components

2/10/2024 Dept. of ECE, ASE, Bengaluru 31

• power spectral density or power spectrum of the random process is a
function that determines the distribution of the power of the random
process at different frequencies
• The power spectral density of a random process X (t) is denoted by
SX (f)
• The unit for power spectral density is W /Hz
• For a stationary random process X (t), the power spectral density is the
Fourier transform of the autocorrelation function
SX(f)=FT{RX (τ)} [Wiener-Khinchin Theorem]
• For a cyclostationary process, the power spectral density is the Fourier
transform of the average autocorrelation function
SX(f)=FT{Avg(RX (τ))}

2/10/2024 Dept. of ECE, ASE, Bengaluru 32

 Power of a Random Process
• Power of a random process is the sum of the powers at all
frequencies in that random process
• Total power can be obtained by integrating power spectral density
over all frequencies
• Power in the random process which is denoted by PX is given by

• Since SX ( f ) is the Fourier transform of RX (τ), then RX (τ), will be the

inverse Fourier transform of SX ( f ) Therefore

2/10/2024 Dept. of ECE, ASE, Bengaluru 33

• Substituting τ = 0 into this relation yields

• Which is same as PX Therefore

PX = RX (0)

• Power in a stationary random process can be found either by

integrating its power spectral density (adding all power components) or
substituting τ = 0 in the autocorrelation function of the process.

2/10/2024 Dept. of ECE, ASE, Bengaluru 34

 Example
• Assume that we uniformly choose a phase Ѳ between 0 and 2π and
generate a sinusoid with a fixed amplitude and frequency but with a
random phase Ѳ. In this case, the random process is X (t) = A cos(2πf0t +
Ѳ), where A and f0 denote the fixed amplitude and frequency and Ѳ
denotes the random phase. Find the power spectral density of X(t) and
the power in the process.

2/10/2024 Dept. of ECE, ASE, Bengaluru 35

Solution: The autocorrelation of X(t) is found to be

Hence power spectral density is

All the power content of the process is located at f0 and –f0. This is
expected because the sample functions of this process are sinusoidals with
their power at those frequencies.

2/10/2024 Dept. of ECE, ASE, Bengaluru 36

Solution:
Using power spectral density Using autocorrelation
function

2/10/2024 Dept. of ECE, ASE, Bengaluru 37

 Example
The process X (t) is defined by X (t) = X, where X is a random variable
uniformly distributed on [- 1 , 1]. Find the power spectral density of X(t).

2/10/2024 Dept. of ECE, ASE, Bengaluru 38

 Solution

Here X(t) is a stationary process and autocorrelation

function is given by

Hence power spectral density

Sx(f) = FT{Rx(τ)}
= FT{⅓}
Sx(f) = ⅓δ(f)
In this case, for each realization of the process, we have a different
power spectrum.

2/10/2024 Dept. of ECE, ASE, Bengaluru 39

 Power spectra in LTI system
When a stationary random process with mean mx and autocorrelation function RX (τ)
passes through an LTI system with the impulse response h(t), the output process will
be also stationary with
Mean of the output process
∞
We know that the Fourier Transform of h(t) is H ( f ) = ∫ h(t ) e − j 2π ft dt
−∞
∞
When f =0 H (0) = ∫ h(t ) dt
−∞

Hence mY = mX H(0)

The mean value of the response of the system only depends on the value of
H ( f ) at f = 0 (DC response), Since the mean of a random process is basically its
DC value.

2/10/2024 Dept. of ECE, ASE, Bengaluru 41

Power Spectral density of the output process
Autocorrelation of the output process is

Taking FT on both the sides, we get

FT{RY(τ)}=FT{RX(τ)} FT{h(τ)} FT{h(-τ)}
SY( f ) = SX( f ) H( f ) H*( f )
SY( f ) = SX( f ) |H( f )|2

In power spectrum, the phase of H (f) is irrelevant; only the magnitude

of H (f) affects the output power spectrum. This is also intuitive since
power depends on the amplitude and not on the phase of the signal.

2/10/2024 Dept. of ECE, ASE, Bengaluru 42

Cross-spectral density SXY ( f )
X(t) and Y(t) are jointly stationary with cross-correlation function
RXY(τ) = RX (τ) * h(- τ). The cross spectral density is
SX Y ( f ) = FT{RXY(τ) }
FT{RXY (τ)}=FT{RX(τ)} . FT{h(-τ)}
SX Y ( f ) = SX ( f ) H*( f )
and since RYX (τ) = RXY (-τ) , we have
SY X ( f ) = S*X Y ( f )= SX ( f ) H( f )
Although SX ( f ) and SY ( f ) are real nonnegative functions, SX Y ( f ) and
SY X ( f ) can generally be complex functions.

Dept. of ECE, ASE, Bengaluru

2/10/2024 43
Consider a random process X (t) = A cos(2πf0t + Θ), where A and f0 denote
the fixed amplitude and frequency, Θ denotes the random phase which is
assumed to be uniformly distributed between 0 and 2π. X(t) is passed
through a differentiator with frequency response H( f ) = j2πf. Find the
power spectral density of the output process Y(t) and cross spectral
density.

2/10/2024 Dept. of ECE, ASE, Bengaluru 44

 Example
Consider a random process X (t) = A cos(2πf0t + Θ), where A and f0 denote the
fixed amplitude and frequency, Θ denotes the random phase which is assumed to be
uniformly distributed between 0 and 2π. X(t) is passed through a differentiator with
frequency response H( f ) = j2πf. Find the power spectral density of the output
process Y(t) and cross spectral density.
The power spectral density f X(t) is found to be

Output power spectral density is Cross spectral density is

SY( f ) = SX( f ) |H( f )|2 or SX Y ( f ) = SX ( f ) H*( f )

2/10/2024 Dept. of ECE, ASE, Bengaluru 45

 The process X (t) is defined by X (t) = X, where X is a random variable
uniformly distributed on [- 1 , 1]. Fid the power spectral density of the process.
X(t) is passed through a differentiator with frequency response H( f ) = j2πf. Find
the Power spectral density of the output process Y(t) and cross spectral density.

2/10/2024 Dept. of ECE, ASE, Bengaluru 46

 Example
The process X (t) is defined by X (t) = X, where X is a random variable
uniformly distributed on [- 1 , 1]. Fid the power spectral density of the process. X(t)
is passed through a differentiator with frequency response H( f ) = j2πf. Find the
Power spectral density of the output process Y(t) and cross spectral density.
Solution: The autocorrelation of X(t) is
or

Hence power spectral density of X(t) is

Output power spectral density is Cross spectral density is

SY( f ) = SX( f ) |H( f )|2 or SX Y ( f ) = SX ( f ) H*( f )

2/10/2024 Dept. of ECE, ASE, Bengaluru 47

 Thank You

2/10/2024 Dept. of ECE, ASE, Bengaluru 48