ME591

Random Data
Analysis and Processing

Lecture 2

Terminologies,
Averages
ME591 Random Data  Prof. Yong-Hwa Park, KAIST
 Terminology
Time history record: data source for amplitude(ensemble), time and frequency domain analyses

Deterministic: accurately predicted in detail; repeatable; “trends”

Random: not accurately predicted in detail; not repeatable; “errors”

Ensemble: collection of all time history records (sample functions), xi (t ), i = 1,2,3, defines a random process, {x(t )}

Transient data
Steady state data

Continuous system = distributed parameter system = Infinite degree-of-freedom system; described by PDE
Discrete system = lumped parameter system = Multi-degree-of-freedom system; described by ODE

Continuous time system: analog

Discrete time system: sampling; digital

Linear system: additive and homogeneous (superposition) properties hold; L(ax + by) = aL( x) + bL( y)
Non-linear system : L(ax + by )  aL( x) + bL( y )

ME591 Random Data 2  Prof. Yong-Hwa Park, KAIST

 Terminology (cont.)
Stationary process: Probability distributions are invariant under a shift of the time scale p ( x(t1 )) = p ( x(t2 )) = p ( x(t3 )) = ...
p ( x(t1 ), x(t2 )) = p ( x(t2 ), x(t3 )) = ... when t2 − t1 = t3 − t2 = ...
Strongly stationary
p ( x(t1 ), x(t2 ), x(t3 )) = p( x(t3 ), x(t4 ), x(t5 )) = ... when t2 − t1 = t4 − t3 =  1 ,
Weakly stationary: first and second order probability distributions are invariant. t3 − t2 = t5 − t4 =  2 ,...

p ( x(t1 )) p( x(t2 )) p( x(t3 ))

Nonstationary process
x1 (t )

Locally stationary: R (t , ) = R1 (t ) R2 ( ) t

Ergodic process: Ensemble averages = Temporal averages x2 (t )

t
Almost all stationary processes are ergodic
x3 (t )
Strongly ergodic t

Weakly ergodic

…
p ( x(t1 ), x(t2 )) p ( x(t2 ), x(t3 ))

p ( x(t1 ), x(t3 ))
xk (t )
Non-ergodic process
t

t1 t2 t3
p ( x(t1 )) :Probability density function of random variable x(t ) at t = t1
p ( x(t1 ), x(t2 )) : Joint probability density function of random variable x(t ) at t = t1 and t = t 2

ME591 Random Data 3  Prof. Yong-Hwa Park, KAIST

 Averages
To describe a random process (phenomenon),  x(t ), we need to compute the average properties:
px (t1 ), px (t1 ), x(t 2 ), px (t1 ), x(t 2 ), x(t 3 ), 

 Stationarity
p  x(t ) , p  x(t ), x(t +  ) , p  x(t ), x(t +  1 ), x(t +  2 ) ,  t : arbitraty
p ( x(t1 )) p( x(t2 )) p( x(t3 ))
( x −  x )2
x1 (t ) −
 Normal Distribution p ( x) = 1
e 2 x 2

t 2 x
p  x(t ) , p x(t ), x(t +  )

x2 (t )
t
 Ensemble Averages
N
 1
 x (t ) =  x = E[ x(t )] =  xp( x)dx = lim
− N → N
 x (t )
i =1
i t : arbitraty

x3 (t )   1 N

t Rxx ( )  E[ x(t ) x(t +  )] = 

−  −
x(t ) x(t +  ) p  x(t ), x(t +  )dx(t )dx(t +  ) = lim
N → N
 x (t ) x (t +  )
i =1
i i

 Ergodic
…

p ( x(t1 ), x(t2 )) p ( x(t2 ), x(t3 )) 1 T 1 T

p ( x(t1 ), x(t3 ))
 x = lim 
T → T 0
x(t )dt , Rxx ( ) = lim  x(t ) x(t +  )dt Temporal average
T → T 0

xk (t )
 Finite record length T
t 1 T 1 T −
ˆ x =  x(t )dt , Rˆ xx ( ) =
T −  0
x(t ) x(t +  )dt Estimates (continuous)
t3
T 0
t1 t2
 Sampling h = t ; T = ( N − 1)h
1 N
1 N −r
ˆ x =
N
 x(nh), Rˆ xx (rh) =
n =1 (N − r) 
n =1
x(nh) x (n + r )h, r = 0,1, 2,3, , m  N Estimates (discrete)

ME591 Random Data 4  Prof. Yong-Hwa Park, KAIST

 Example of Nonstationary Process
p ( x (ti ))
A random process (total 2 samples): {x(t )} = {x1 (t ), x2 (t )} 1 1
2 2 i = 1, 2,..., 5
x1 (t )
−1 1
x (ti )
1
1
3 4 t p ( x(t1 )) = p ( x(t2 )) = p ( x(t3 )) = p ( x(t 4 )) = p ( x(t5 )) = = p ( x(t ))
2
1 2 5 The first order probability is time invariant.
-1

x2 (t ) p ( x(t1 ), x(t2 )) 1 p( x(t2 ), x(t3 ))

2 1
1 1 x(t2 ) 2 x(t3 )
1 2 5 2 0 1
t 0
2
3 4 0 x(t1 ) x(t2 )
-1 0

Here, p( x(t1 ), x(t2 )) = p( x(t3 ), x(t4 )) , p ( x(t2 ), x(t3 )) = p ( x(t4 ), x(t5 ))

but p ( x(t1 ), x(t2 ))  p ( x(t2 ), x(t3 )) .

Thus the second order probability is time variant.

ME591 Random Data 5  Prof. Yong-Hwa Park, KAIST

 Ergodic process Non-ergodic process
{xk (t )} = { X sin(2 ft +  k )} {xk (t )} = { X k sin(2 ft +  k )}

for uniformly distributed Θk

ME591 Random Data 6  Prof. Yong-Hwa Park, KAIST
 Proof: ergodicity and autocorrelation function for a stationary process: {xk (t )} = { X sin(2 ft +  k )}, k = 1, 2,...

where X, f are constants and  k is a uniformly distributed random variable.

Rxx ( ) = E[ x(t ) x(t +  )]
Ensemble average of function g ( )
= E[ xk (t ) xk (t +  )] ; ensemble average over sample functions 

= E[ X sin(2 ft +  k ) X sin{2 f (t +  ) +  k }] E[ g ( )] =  g ( ) p( )d

−
; a single random variable 
2
= X sin(2 ft +  ) sin{2 f (t +  ) +  }p( ) d ; a single random variable 
2

0
2
1
= X sin(2 ft +  ) sin{2 f (t +  ) +  }d ; p( ) = 1
2

2 2
0

2
T
1
=  X 2 sin(2 ft +  ) sin{2 f (t +  ) +  }dt ;  = 2 ft +  0  d = 2 fdt = dt
T 0 T
T
1
=  x(t ) x(t +  )dt
T 0
T
1
= lim  x(t ) x(t +  )dt periodic function
T → T 0

: function of known constants (X, f ) and 

X2
= cos(2 f  )
2

It concludes that the process is ergodic.

ME591 Random Data 7  Prof. Yong-Hwa Park, KAIST

 3 Domains for Random Data
Ensemble average of function g ( x)

E[ g ( x)] =  g ( x) p( x)dx ; a random variable x
Amplitude −


when g ( x) = x : E[ x] =  xp( x)dx =  x (mean)

(Ensemble) Domain
−

g ( x) = x : E[ x ] = x p ( x)dx =  x2
2 2 2

(Probability)
− (mean square)
T
1
 f ( x) = lim  f ( x(t ))dt ; a random variable x(t )
T → T
0

Time average of function f ( x)

Time Frequency
when f ( x) = x(t ) x(t +  ) :


1
T
Rxx ( ) = lim  x(t ) x(t +  )dt
Domain Domain S xx ( f ) = R xx ( )e − j 2 f  d
T → T −

(Correlation) (Spectrum)
0

Wiener-
Khinchine
Relation

ME591 Random Data 8  Prof. Yong-Hwa Park, KAIST

 Background Knowledge
(Probability)
uniform, normal,
chi-square, sine
distributions
PSD: Power Spectral Density

1
T
(Correlation) (Spectrum) 
Rxx ( ) = lim  x(t ) x(t +  )dt
T → T 0
Auto & cross PSD,
S xx ( f ) = R xx ( )e − j 2 f  d
Auto & cross F.T.
−

coherence,


R
T
1 S xy ( f ) = ( )e − j 2 f  d
Rxy ( ) = lim  x(t ) y (t +  )dt
correlations
xy
T → T 0 conditioned −
2

spectra
S xy ( f )
 xy2 ( f ) =
S xx ( f ) S yy ( f )

…
ME591 Random Data 9  Prof. Yong-Hwa Park, KAIST
 Correlation and Spectral Analysis of Random Data requires

Fourier Transform in theory X ( f ) =  x(t )e− j 2 ft dt
−

Fourier Series in practice 1 T

Ak =  x(t )e − j 2 fk t dt
T 0

Continuous Spectrum in theory

Discrete Spectrum in practice

Discrete Spectrum is the result of 1

f =
finite record length, not of sampling T

Infinite frequency bandwidth in theory 1

f max =
Finite frequency bandwidth due to sampling 2t

ME591 Random Data 10  Prof. Yong-Hwa Park, KAIST

 u (t ) : true input
v(t ) : true output
System and Source
n(t )
ID
x(t ) : input measurement w/ noise m(t)
u (t ) v(t )
y (t ) : output measurement w/ noise n(t) H( f ) + y(t ) SISO/SIMO
H ( f ) : Frequency Response Function(FRF)
m(t ) + x(t ) Grand questions in this
course:
u1 (t ) H1 ( f )
n(t ) How much random variables
x(t) and y(t) are linearly
u2 (t ) H2 ( f ) related? : Correlation analysis,
MISO/MIMO y(t ) Coherence

How can we identify H(f) by

un (t ) Hn ( f ) measurements of random
variables x(t) and y(t)? :
System identification
SISO w/ zero- H( f ) n(t )
memory NL How can we identify x(t) (or
u(t)) by measurement of y(t)
u (t ) y(t ) (or v(t)) with known H(f)? :
Source identification (inverse
Non- A( f ) problem)
linear
ME591 Random Data 11  Prof. Yong-Hwa Park, KAIST
 Techniques for non-stationary process

Short time F.T

Time-frequency map (i.e., Spectrogram)

Wavelets

Techniques for complex data

Cepstrum Analysis

Hilbert Transform

ME591 Random Data 12  Prof. Yong-Hwa Park, KAIST