AU

Department of Electrical & Computore Engineering

Probability and Random Process (EEEg-2114)

Chapter 4: Random Processes

 Random Processes
Outline
§ Introduction
§ Definition of a Random Process
§ Characterization of Random Processes
§ Mean, Correlation, and Covariance Functions
§ Classification of Random Processes
§ Power Spectral Densities of Random Processes

2
 Introduction

§ The theory of random processes was first developed in

connection with the study of fluctuations and noise in physical
systems.

§ A random process is the mathematical model of an empirical

process whose development is governed by probability laws.

§ Random processes provides useful models for the studies of

such diverse fields as statistical physics, communication and
control, time series analysis, population growth, and
management sciences.
3
 Definition of a Random Process

§ A random process is a family of random variables {X(t), tϵT}

defined on a given probability space, indexed by the parameter t,
where t varies over an index set T.

§ In a random process {X(t), tϵT}, the index set T is called the

parameter set of the random process.

§ The values assumed by X(t) are called states, and the set of all
possible values forms the state space E of the random process.

§ If the index set T of a random process is discrete, then the

process is called a discrete-time random process.
4
 Definition of a Random Process Cont’d…..

§ A discrete-time random process is also called a random

sequence and is denoted by {Xn , n = 1, 2, 3, . . .).

§ If T is continuous, then we have a continuous-time random

process.

§ In fact, a random process {X(t), tϵT} is a function of two

arguments {X(t, ω), tϵT, ωϵΩ}.

§ For a fixed time t=tk, X(tk, ω) = Xk(ω) is a random variable

denoted by X(tk), as ω varies over the sample space Ω.

5
 Definition of a Random Process Cont’d…..

§ On the other hand, for a fixed sample point ωi ϵΩ, X(t, ωi) =
Xi(t) is a single function of time t, called a sample function or a
realization of the process.

§ The totality of all sample functions is called an ensemble.

§ Of course if both ω and t are fixed, X(tk , ωi) is simply a real

number.

§ In the following discussion, we use the notation X(t) to

represent X(t, ω).

6
 Characterization of Random Processes

§ If X(t) is a random process, then for fixed t=t1, X1=X(t1)

represents a random variable.
§ Its distribution function is given by:
FX ( x1 , t1 )  P{ X (t )  x1 }
§ Notice that FX(x, t) depends on t, since for a different t, we
obtain a different random variable.
§ The first-order probability density function of the process X(t)
is defined as:
dF ( x1 , t1 )
f ( x1 , t1 )  X

dx1
X

7
 Characterization of Random Processes Cont’d…..

§ For t = t1 and t = t2, X(t) represents two different random

variables X1 = X(t1) and X2 = X(t2) respectively.
§ Their joint distribution is given by:
FX ( x1 , x 2 , t1 , t 2 )  P{ X (t1 )  x1 , X (t 2 )  x 2 }
§ The second-order probability density function of the random
process X(t) is:
 2 FX ( x1 , x2 , t1 , t2 )
f X ( x1 , x2 , t1 , t2 ) 
x1 x2

§ Similarly f X ( x1 , x2 ,  xn , t1 , t 2  , t n ) represents the nth order

density function of the process X(t).
8
 Mean, Correlation, and Covariance Functions

§ As in the case of random variables, random processes are often

described by using statistical averages.

§ The mean of X(t) is defined by:

 X (t )  E X (t )
where X(t) is treated as a random variable for a fixed value of t.

§ In general, μX(t) is a function of time, and it is often called the

ensemble average of X(t).

9
 Mean, Correlation, and Covariance Functions …...

§ A measure of dependence among the random variables of X(t)

is provided by its autocorrelation function, defined by:

R XX (t1 , t 2 )  E  X (t1 ) X ( 2 )

§ Note that:

R XX (t1 , t 2 )  R XX (t 2 , t1 ) and R XX (t , t )  E X 2 (t ) 
§ The autocovariance function of X(t) is defined by:

C XX (t1 , t 2 )  Cov X (t1 ) , X (t 2 )  E X (t1 )   X (t1 ) X (t 2 )   X (t 2 )

 R XX (t1 , t 2 )   X (t1 )  X (t 2 )

10
 Mean, Correlation, and Covariance Functions …...

§ It is clear that if the mean of X(t) is zero, then:

C XX (t1 , t 2 )  R XX (t1 , t 2 )

§ Note that the variance of X(t) is given by:


 X 2 (t )  Var X (t )  E X (t )   X (t )2 
§ If X(t) is a complex random process, then its autocorrelation
function RXX(t1, t2) and autocovariance function CXX(t1, t2) are
defined, respectively, by:
 
R XX (t1 , t 2 )  E X (t1 ) X * (t 2 ) and


C XX (t1 , t 2 )  E  X (t1 )   X (t1 ) X (t 2 )   X (t 2 )
*

11
 Classification of Random Processes

i. Stationary Processes
§ A random process {X(t), tϵT} is said to be stationary or strict-
sense stationary (SSS) if, for all n and for every set of time
instants (ti ϵT, i = 1,2, . . . , n),
FX ( x1 ,........, x n , t1 , ....., t n )  FX ( x1 , ........, x n , t1   , ......, t n   )

§ Hence, the distribution of a stationary process will be

unaffected by a shift in the time origin, and X(t) and X(t+τ)
will have the same distributions for any τ.
§ Nonstationary processes are characterized by distributions
depending on the points t1, t2, . . . , tn.

12
 Classification of Random Processes Cont’d……

ii. Wide-Sense Stationary Processes

§ A random process X(t) is wide-sense stationary (WSS) if:

1 . E  X (t )    X (constant )

2. R XX (t1 , t 2 )  E  X (t1 ) X (t 2 )   R XX  t 2  t1 

§ Note that a strict-sense stationary process is also a WSS

process, but, in general, the converse is not true.

13
 Classification of Random Processes Cont’d……

§ Other types of random processes include:

ü Independent Processes
ü Markov Processes
ü Normal Processes
ü Ergodic Processes
ü Poisson Processes

14
 Power Spectral Densities of Random Processes

§ The autocorrelation function of a continuous-time random

process X(t) is defined as:

R XX ( )  E X (t ) X (t   )

§ Properties of RXX(τ):

1. R XX ( )  R XX ( )

2. R XX ( )  R XX (0)


3. R XX (0)  E X 2 (t )  0 
15
 Power Spectral Densities of Random Processes……

§ In case of a discrete-time random process X(n), the

autocorrelation function of X(n) is defined by:

R XX (k )  E X (n) X (n  k )

§ Properties of RX(k):

1. R XX ( k )  R XX (k )

2. R XX (k )  R XX (0)


3. R XX (0)  E X 2 (n)  0 
16
 Power Spectral Densities of Random Processes……

§ Two processes X(t) and Y(t) are called (mutually) orthogonal if:

R XY ( )  0 , for all 

§ Similarly, the cross-correlation function of two discrete-time

jointly WSS random processes X(n) and Y(n) is defined by:

R XY (k )  E X (n)Y (n  k )

§ The various properties of RXY(k) similar to those of RXY(τ) can

be obtained by replacing τ by k in the above equations.

17
 Power Spectral Densities of Random Processes……

§ The power spectral density (or power spectrum) SXX(ω) of a

continuous-time random process X(t) is defined as the Fourier
transform of RXX(τ), i.e. ,

S XX     R XX ( )e  j d


§ Thus, taking the inverse Fourier transform of SX(ω), we obtain:

1 
R XX     S XX ( )e j d
2 

§ The above equations are known as the Wiener-Khinchin

relations.
18
 Power Spectral Densities of Random Processes……

§ Properties of SXX(ω):

1. S XX ( ) is real and S XX ( )  0

2. S XX ( )  S XX ( )

 2

3. E X (t )  R XX (0) 
1
2 



S XX ( )d

§ Similarly, the power spectral density SXX(Ω) of a discrete-time

random process X(n) is defined as the Fourier transform of
RXX(k): 
S XX     XX
R ( k ) e  jk

k  
19
 Power Spectral Densities of Random Processes……

§ Thus, taking the inverse Fourier transform of SXX(Ω), we obtain:

1 
R XX (k ) 
2 

S XX ()e jk d

§ Properties of SXX(Ω):
1. S XX (  2 )  S XX ()

2. S XX () is real and S XX ()  0

3. S XX ()  S XX ()

 2

3. E X (n)  R XX (0) 
1
2

S

XX ()d

20
 Power Spectral Densities of Random Processes……

§ The cross power spectral density (or cross power spectrum)

SXY(ω) of two continuous-time random processes X(t) and Y(t)
is defined as the Fourier transform of RXY(τ):

S XY     R XY ( )e  j d


§ Thus, taking the inverse Fourier transform of SXY(ω), we get:

1 
R XY     S XY ( )e j d
2 

21
 Power Spectral Densities of Random Processes……

§ Properties of SXY(ω):
§ Unlike SXX(ω), which is a real-valued function of ω, SXY(ω), in
general, is a complex-valued function.

1. S XY ( )  S YX ( )
*
2. S XY ( )  S XY ( )

§ Similarly, the cross power spectral density SXY(Ω) of two

discrete-time random processes X(n) and Y(n) is defined:

S XY ()   XY
R (
k  
k ) e  jk

22
 Power Spectral Densities of Random Processes……

§ Taking the inverse Fourier transform of SXY(Ω), we get:

1 
R XY (k ) 
2 

S XY ()e jk d

§ Properties of SXY(ω):
§ Unlike SXX(Ω), which is a real-valued function of Ω, SXY(Ω), in
general, is a complex-valued function.

1. S XY (  2 )  S XY ()

2. S XY ()  S YX ()
*
3. S XY ()  S XY ()
23
Example on Random Processes
Example:
Consider a random process X(t) defined by
X (t )  A cos( 0 t   )

where  0 and A are constants and  is a uniform

random variable over the interval (0, 2 )

a. Find the mean  X (t ).

b. Find the autocorrelation function R XX (t1 , t 2 ).

c. Find the autocovariance function C XX (t1 , t 2 ).

d . Determine whether X (t ) is WSS random process or not.

e. Find the power spectral density of X (t ).

24
Example on Random Processes Cont’d……
Solution:
a .  X (t )  E  X (t )   E  A cos(  0 t   )   AE cos(  0 t   ) 

But , cos(  0 t   )  cos(  0 t)cos  - sin(  0 t)sin 

  X (t )  E  X (t )   AE cos(  0 t)cos  - sin(  0 t)sin  

 A cos(  0 t) E cos    A sin(  0 t) E sin  

1 2
E cos     cos d   0
2 0

1 2
Similarly , E sin     sin d   0
2 0

  X (t )  E  X (t )   0

25
Example on Random Processes Cont’d……
Solution:
b. R XX (t1 , t 2 )  E  X (t1 ) X (t 2 ) 

 E  A cos(  0 t1   ) A cos(  0 t 2   ) 

 A 2 E cos(  0 t1   ) cos(  0 t 2   ) 

A2
 E cos  0 (t 2  t1 )  cos(  0 (t1  t 2 )  2 ) 
2
But , E cos  0 (t 2  t1 )   cos  0 (t 2  t1 ) and

E cos(  0 (t1  t 2 )  2 )   0

A2
 R XX (t1 , t 2 )  cos  0 (t 2  t1 )
2
26
Example on Random Processes Cont’d……
Solution:
c. C XX (t1 , t 2 )  R XX (t1 , t 2 )   X (t1 )  X (t 2 )

A2
 cos  0 (t 2  t1 )  0
2
A2
 C XX (t1 , t 2 )  cos  0 (t 2  t1 )
2
d . Since the mean is constant and the autocorrelation function
depends on time difference only, X (t ) is a WSS random process.

27
Example on Random Processes Cont’d……
Solution:
e. Since X (t ) is a WSS random process, the autocorrel ation
function can be simply wri tten as :

A2
R XX ( )  cos( 0 )
2
The power spectral density of X (t ) is given by :

S XX ( )  

R XX ( )e  j d

But from Fourier tr ansform pair table , we have :

FT cos( 0 t )   (   0 )   (   0 )

A 2 A 2
 S XX ( )   (   0 )   (   0 )
2 2
28