RANDOM PROCESSES

1
 Random Process
Introduction: (Physical phenomenon)
Deterministic model : No uncertainty about its time-
dependent behavior at any instant of time .
Random model :The future value is subject to
“chance”(probability)
Example: Thermal noise , Random data stream
Mathematical Definition of a Random Process (RP)
The properties of RP
a. Function of time.
b. Random in the sense that before conducting an
experiment, not possible to define the waveform.
Sample space S function of time, X(t,s)
mapping 2
An ensemble of sample functions:
{x j (t ) | j = 1,2,, n}
3
Sample space S function of time, X(t,s)
mapping
S → X (t,s ) -T  t  T
2T:The total observation interval
s j → X ( t , s j ) = x j (t )
x j (t ) = sample function
At t = tk, xj (tk) is a random variable (RV).

To simplify the notation, let X(t,s) = X(t)

X(t): Random process, an ensemble of time function together
with a probability rule.

Difference between RV and RP

RV: The outcome is mapped into a number
RP: The outcome is mapped into a function of time 4
5
 Stationary Process
Stationary Process :
The statistical characterization of a process is independent of
the time at which observation of the process is initiated.
Nonstationary Process:
Not a stationary process (unstable phenomenon )
Consider X(t) which is initiated at t = −,
X(t1),X(t2)…,X(tk) denote the RV obtained at t1,t2…,tk
For the RP to be stationary in the strict sense (strictly stationary)
The joint distribution function
FX ( t1 +τ ),..., X ( tk +τ ) ( x ,.., x ) = FX ( t1 ) ,...,X ( tk ) ( x ,... x )
1 k 1 k

For all time shift t, all k, and all possible choice of t1,t2…,tk
6
 Mean, Correlation, and Covariance Function
Let X(t) be a strictly stationary RP. The mean of X(t) (Statistical
 X (t ) = E X (t )
Average) is


=  xf X (t ) ( x) d x
−

= X =constant for all t

The autocorrelation function of X(t) is

R X (t1,t2 ) = E X (t1 ) X (t2 )

 
=  x1 x2 f X ( t1 ) X ( t2 ) ( x1 , x2 )dx1dx2
- -
 
=  x1 x2 f X (0) X ( t2 −t1 ) ( x1 , x2 )dx1dx2
- -

= RX (t2 − t1 ) for all t1 and t2 7

The autocovariance function
C X (t1,t 2) = E ( X (t1 ) −  X )( X (t 2 ) −  X )
= RX (t 2 − t1 ) −  X2
Which is of function of time difference (t2-t1).
We can determine CX(t1,t2) if X and RX(t2-t1) are known.

Note that:

1. X and RX(t2-t1) only provide a partial description.

2. If X(t) = X and RX(t1,t2)=RX(t2-t1),

then X(t) is wide-sense stationary (stationary process).
8
Properties of the autocorrelation function
For convenience of notation , we redefine

RX (t ) = E X (t − τ ) X (t ) , for all t
1. The mean-square value
 
RX (0) = E X 2 (t ) , τ = 0

2. RX (t ) = R(− τ)

3. RX (t )  RX (0)

9
The RX(t) provides the interdependence information
of two random variables obtained from X(t) at times
t seconds apart

10
Example X (t ) = Acos( 2πfct + Θ)
 1
, −π θ π
f  ( ) =  2π
 0, elsewhere
f

θ
−π π

A2
RX (t ) = E  X (t + τ ) X (t )  = cos(2πf ct )
2
Prove!

11
12
Summary of Ergodic Processes
An ergodic random process is one where time averages are
equal to ensemble averages. Hence, for all g(X) and X

For a random process to be ergodic, it must be strictly stationary.

However, not all strictly stationary random processes are ergodic.
A random process is ergodic in the mean if

and ergodic in the autocorrelation if

RX ( τ ) 13
 Example 1
Recall the random process

where A and fc are constants, and  is assumed to be a uniformly

distributed random phase having the pdf:

The time average mean of X(t) is:

Thus,

so the random process X(t) is ergodic in the mean.

14
The time average autocorrelation of X(t) is

Thus, RX ( τ ) =
so the random process X(t) is ergodic in the autocorrelation.

It follows that the random process X(t) in this example is ergodic

in the mean and autocorrelation.

15
 Power Spectral Density (PSD)

Define: Power Spectral Density ( Fourier Transform of R(τ ) )


S X ( f ) =  RX (t ) exp(− j 2πfτ ) dt
-

It can be shown that:


E Y (t )  =  H ( f ) S X ( f ) df
2 2
-

=  SY ( f ) df
-

Thus,

SY ( f ) = H ( f ) S X ( f )
2

16
 Properties of The PSD


S X ( f ) =  RX ( τ ) exp( − j 2 f t ) dt
−

RX ( τ ) =  S X ( f ) exp( j 2 f t ) df
−

Einstein-Wiener-Khintahine relations:
S X ( f )  RX ( τ )

S X ( f ) is more useful than RX (τ ) !

17
 Properties of The PSD


a. S X (0) =  RX ( τ ) dt
−

b. E  X (t )  =  S X ( f ) df
2
−

c. If X (t ) is stationary,
SX ( f )  0 for all f
d. S X (− f ) = S X ( f )
e. The PSD can be associated with a pdf:
SX ( f )
pX ( f ) = 

−
S X ( f ) df
18
Example: Sinusoidal Wave with Random Phase
X (t ) = A cos(2 fct + ),  ~ U (− , −  )
A2
RX (t ) = cos(2 fct )
2

S X ( f ) =  RX (t ) exp( − j 2 f t ) dt
−

A2 
= −
exp( j 2 fct )dt + exp(− j 2 f ct ) exp(− j 2 f t ) dt
4
A2
=  ( f − fc ) +  ( f + fc )
4

19
 Gaussian Process
Define : Y as a linear functional of X(t)
T
Y =  g (t ) X (t )dt ( g(t): some function)
0

The process X(t) is a Gaussian process if every linear

functional of X(t) is a Gaussian random variable
1  ( y − Y ) 2 
fY ( y ) = exp  − 
2 Y  2 Y
2

1 y2
Normalized fY (y ) = exp(− ) , as N (0,1)
2 2

Fig. Normalized Gaussian distribution

20
Central Limit Theorem
Let Xi , i =1,2,3,….N be (a) statistically independent R.V.
and (b) have mean μ X and variance σ X2 .
Since they are independently and identically distributed (i.i.d.)
Normalized Xi
1
 Yi = ( X i − X ) i = 1,2,...., N
X
Hence, E Yi  = 0,
Var Yi  = 1.
N
1
Define VN =
N
Y i =1
i

The Central Limit Theorem

The probability distribution of VN approaches N(0,1)
as N approaches infinity.
21
 Properties of A Gaussian Process
1. X(t) Y(t)
h(t)
Gaussian Gaussian

T
Y (t ) =  h(t − t )X (t )dt
0

By definition X is a Gaussian random variable

T
 Y (t ) =  h(t − t )X (t )dt , 0  t   is also Gaussian
0

2. If X(t) is Gaussian
Then X(t1) , X(t2) , X(t3) , …., X(tn) are jointly Gaussian.

22