l

CHAPTER THREE INTRODUCTION 111

techniques for deriving or building random process models by collecting and

analyzing data are discussed in Chapters 8 and 9. We assume that the reader
bas a background in deterministic systems and signal analysis, including analysis
in the frequency domain.

Random Processes
and Sequences 3.1 INTRODUCTION

In many engineering problems, we deal with time-varying waveforms that have

some element of chance or randomness associated with them. As an example,
consider the waveforms that occur in a typical data communication system such
as the one shown in Figure 3.1 in which a number of terminals are sending
information in binary format over noisy transmission links to a central computer.
A transmitter in each link converts the binary data to an electrical waveform in
which binary digits are converted to pulses of duration T and amplitudes ± 1.
The received waveform in each link is a distorted and noisy version of the
transmitted waveform where noise represents interfering electrical disturbances.
From the received waveform, the receiver attempts to extract the transmitted
binary digits. As shown in Figure 3.1, distortion and noise cause the receiver to
make occasional errors in recovering the transmitted binary digit sequence.
In electrical systems we use voltage or current waveforms as signals for collecting, As we examine the collection or "ensemble" of waveforms shown in Figure
transmitting and processing information, as well as for controlling and providing 3.1, randomness is evident in all of these waveforms. By observing one wave-
power to a variety of devices. Signals, whether they are voltage or current form, say x;(t), over the time interval {!1, 12 ] we cannot. with certainty, predict
waveforms, are functions of time and belong to one of two important classes: the value of x;(t) for any other value [t 11 t 2]. Furthermore, knowledge of
deterministic and random. Deterministic signals can be described by functions one member function x;(t) will not enable us to know the value of another
in the usual mathematical sense with time t as the independent variable. In member function x/t). We will use a probabilistic model to describe or char-
contrast with a deterministic signal, a random signal always has some element acterize the ensemble of waveforms so that we can answer questions such as:
of uncertainty associated with it and hence it is not possible to determine exactly
its value at any given point in time. 1. What are the spectral properties of the ensemble of waveforms shown
Examples of random signals include the audio waveform that is transmitted in Figure 3.1?
over a telephone channel, the data waveform transmitted from a space probe, 2. How does the noise affect system performance as measured by the re-
the navigational information received from a submarine, and the instantaneous ceiver's ability to recover the transmitted data correctly?
load in a power system. In all of these cases, we cannot precisely specify the 3. What is the optimum processing algorithm that the receiver should use?
value of the random signal in advance. However, we may be able to describe
the random signal in terms of its average properties such as the average power By extending the concept of a random variable to include time, we can build
in the random signal, its spectral distribution on the average, and the probability a random process model for characterizing an ensemble of time functions. For
that the signal amplitude exceeds a given value. The probabilistic model used the waveforms shown in Figure 3.1, consider a random experiment that consists
for characterizing a random signal is called a random process (also referred to of tossing N coins simultaneously and repeating the N tossings once every T
as a stochastic process or time series). In this and the following four chapters, lf we label the outcomes of the experiment by "1" when a coin flip
we will study random process models and their applications. results in a head and "0" when the toss results in a tail, then we have a prob-
Basic properties of random processes and analysis of linear systems driven abilistic model for the bit sequences transmitted by the terminals. Now, by
by random signals are dealt with in this chapter and in Chapter 4. Several classes representing 1s and Os by pulses of amplitude ±1 and duration T, we can model
of random process models that are commonly used in various applications are the transmitted waveform X;(t). If the channel is linear, its impulse response
presented in Chapter 5. The use of random process models in the design of. h(t) is known, and the noise is additive, then we can express y;(t) as x;(t) * h;(t)
communication and control systems is introduced in Chapters 6 and 7. Finally, + nJt), where n;(t) is the additive channel "noise," and * indicates convolution.
112 RANDOM PROCESSES AND SEQUENCES DEFINITION OF RANDOM PROCESSES 113 I'
•·
l,
By processing y;(t) the receiver can generate the output sequence b;(k). Thus,
by extending the concept of random variables to include time and using the
results from deterministic systems analysis, we can model random signals and
ti
analyze the response of systems to random inputs.
The validity of the random-process model suggested in the previous para- !I
graph for the signals shown in Figure 3.1 can be decided only by collecting and
analyzing sample waveforms. Model building and validation fall into the realm "
ll
j of statistics and will be the subject of coverage in Chapters 8 and 9. For the ll
m-
:g
time being, we will assume that appropriate probabilistic models are given and
II
-o - proceed with the analysis.
8
.f
0

0:
-
0 We start our study of random process models with an introduction to the
notation, terminology, and definitions. Then, we -present a number of examples
ll

and develop the idea of using certain averages to characterize random processes. li
Basic signal-processing operations such as differentiation, integration, and lim- !i
iting will be discussed next. Both time-domain and frequency-domain techniques
will be used in the analysis, and the concepts of power spectral distribution and ll
bandwidth will be discussed in detail. Finally, we develop series approximations II
to random processes that are analogous to Fourier and other series represen-
E vi
<U tations for deterministic signals. II
u

..,
c
0

.B
c<U
:;:3
0"
<U
!I

3.2 DEFINITION OF RANDOM PROCESSES ll

i1 "0"'
E
8 c
I!
"'"'
<U 3.2.1 Concept of Random Processes
:s
0 "
"'"'<U H
u A random variable maps the outcomes of a random experiment to a set of real
0
...
0.. numbers. In a similar vein, a random process can be viewed as a mapping of il
s0 the outcomes of a random experiment to a set of waveforms or functions of
"0 H
c time. While in some applications it may not be possible to explicitly define the
..."' underlying random experiment and the associated mapping to waveforms, we 11
""'0
can still use the random process as a model for characterizing a collection of
"'
..!:! !i
waveforms. For example, the waveforms in the data communication system
s
0..
shown in Figure 3.1 were the result of programmers pounding away on terminals. Ji
"'
><
U-l Although the underlying random experiment (what goes through the minds of
programmers) that generates the waveforms is not defined, we can use a hy-
'""
..-i pothetical experiment such as tossing N coins and define the waveforms based il
-.
<:;•
i<•
...
=
lOll
on the outcomes of the experiment.
By way of another example of a random process, consider a random exper-
iment that consists of tossing a die at t = 0 and observing the number of dots
showing on.1he top face. [Link] sample space .of .the consists of the
outcomes 1, 2, 3, 4, 5, and 6. For each outcome of the experiment, let us
arbitrarily assign the following functions of time, t, 0 ,;; t < co. F

Outcome Waveform

1 x 1 (t) -4 .:F

2 x 2 (t) -2 ·L
 < •••

I
114 RANDOM PROCESSES AND SEQUENCES DEFINITION OF RANDOM PROCESSES 115

Outcome Waveform For a specific value of time J = 111 , X(t0 , A) represents a collection of
3 x 3 (t) = +2 numerical values of the various member functions at t = t 0 • The actual value
depends on the outcome of the random experiment and the member function
4 x 4 (t) = +4
associated with that outcome. Hence, X(t 0 , A) is a random variable and the
5 x 5 (t) = - t/2 probability distribution of the random variable, X(t 0 , A), is derived from the
6 x 6 (t) = t/2 probabilities of the various outcomes of the random experiment E.
When t and A are fixed at say t = t 0 , and A = A.;, then X(t 0 , A.;) represents
The set of waveforms {x 1 (t), x 2 (t), ... , x 6 (t)}, which are shown in Figure a single numerical value of the ith member function of the process at t = t 0 •
3.2, represents this random process and are called the ensemble. That is X(t 0 , A;) = x;(t 0). Thus, X(t, A) can denote the following quantities:

1. X(t, A) = {X(t, [Link]; E S} = {x 1 (t), x 2 (t), · · ·}, a collection of

3.2.2 Notation functions of time.
2. X(t, A.;) = x;(t), a specific member function or deterministic function
A random process, which is a collection or ensemble of waveforms, can be of time.
denoted by X(t, A), where t represents time and A is a variable that represents
3. X(to, A) = {X(t 0 , A.;)IA.; E S} = {x 1 (t 0 ), x 2 (t 0 ), •• • }, a collection of
an outcome in the sample space S of some underlying random experiment E.
the numerical values of the member functions at t == t 0 , that is, a random
Associated with each specific outcome', say A.;, we have a specific member
variable.
function x;(t) of the ensemble. Each member function, also referred to as a
sample function or a realization of the process, is a deterministic function of 4. X(t 0 , A.J = x;(t 0 ), numerical value of the ith member function at
time even though we may not always be able to express it in closed form. t = to.

While the notation given in the preceding paragraphs is well defined, con-
vention adds an element of confusion for the sake of conformity with the notation
for deterministic signals by using X(t) rather than X(t, A) to denote a random
lf2t process. X(t) may represent a family of time functions, a single time function,
a random variable, or a single number. Fortunately, the specific interpretation
of X(t) usually can be understood from the context.

EXAMPLE 3.1 (NOTATION)

For the random process shown in Figure 3.2, the random experiment E consists
of tossing a die and observing the number of dots on the up face.

A= {1, 2, 3, 4, 5, 6} = {A 1, A-z, l\3, A4, As, l\6}

X(t, l\ 1) = X(t, A= 1) = x 1 (t) -4, 0 ::; t

X(t, l\ 5 ) = X(t, A = 5) = Xs(t) - 12t, 0 ::; t

-'12t
X(6, A) = X(6) is a random variable that has values from the set

Figure 3.2 Example of a random process. { -4, -3, -2, 2, 3, 4}

X(t = 6, A = 5) = -3, a constant
*If the number of outcomes is countable, then we will use the subscripted notation A, and x,(t) to
denote a particular outcome and the corresponding member function. Otherwise, we will use A and
x(t) to denote a specific outcome and the corresponding member function.
116 RANDOM PROCESSES AND SEQUENCES DEFINITION OF RANDOM PROCESSES 117

3.2.3 Probabilistic Structure functions are equal to -2. Then,

The probabilistic structure of a random process comes from the underlying
random experiment E. Knowing the probability of each outcome of E and the k
-2] = 1Im-
0

time function it maps to, we can derive probability distribution functions for P[X(4)
P[X(t 1) :5 ad, P[X(t 1) :5 a1 and X(t 2 ) :5 a2 ], and so on. If A 1 is a subset of
the sample space S of E and it contains all the outcomes A. for which A.)
:5 a 1 , then We can use a similar interpretation for joint and conditional probabilities.

P[X(t 1):::;; ad = P(A 1 )

3.2.4 Classification of Random Processes

Note that A 1 is an event associated with E and its probability is derived from Random processes are classified according to the characteristics of t and the
the probability structure of the random experiment E. In a similar fashion, we random variable X(t) at time t. If t has a continuum of values in one or more
can define joint and conditional probabilities also by first identifying the event intervals on the real line then X(t) is called a continuous-time random
that is the inverse image of a given set of values of X(t) and then calculating process, examples of which are shown in Figures 3.1 and 3.2. If t can take on
the probability of this event. a finite, or countably infinite, number of values, say{· · · , L 2 , L 1 , t 0 , t 1 , t2 ,
· · ·}then X(t) is called a discrete-time random process or a random sequence,
an example of which is the ensemble of random binary digits shown in Figure
3.1. We often denote a random sequence by X(n) where n represents tn.
EXAMPLE 3.2 (PROBABILISTIC STRUCTURE) X(t) [or X(n)] is a discrete-state or discrete-valued process (or sequence) if
its values are countable. Otherwise, it is a continuous-state or continuous-valued
For the random process shown in Figure 3.2, find (a) P[X(4) = - 2]; (b) P[X(4) random process (or sequence). The ensemble of binary waveforms X(t) shown
:::;; 0]; (c) P[X(O) = 0, X(4) = -2]; and (d) P[X(4) = -2IX(O) = OJ. in Figure 3.1 is a discrete-state, continuous-time, random process. From here
on, we will use a somewhat abbreviated terminology shown in Table 3.1 to refer
SOLUTION: to these four classes of random processes. Note that "continuous" or "discrete"
will be used to refer to the nature of the amplitude distribution of X(t), and
(a) Let A be the set of outcomes such that for every Ai E A, X(4, A.;) "process" or "sequence" is used to distinguish between continuous time or
-2. It is clear from Figure 3.2 that A = {2, 5}. Hence, P[X(4) discrete time, respectively. Additional classification of random processes given
-2J = P(A) = i = !. in the following sections apply to both random processes and random sequences.
(b) P[X(4) :::;; OJ = P[set of outcomes such that X(4) :::;; 0] = = !. Another attribute that is used to classify random processes is the dependence
(c) Let B be the set of outcomes that maps to X(O) = 0 and X(4) = -2. of the probabilistic structure of X(t) on t. If certain probability distributions or
Then B = {5}, and hence P[X(O) = 0, X(4) = -2] = P(B) = k. averages do not depend on t, then the process is called stationary. Otherwise it
(d) P[X(4) = -2IX(O) = 0] = P[X(4) = -2, X(O) = 0] is called nonstationary. The random process shown in Figure 3.1 is stationary if
P[X(O) = 0]
(1/6) 1
= (2/6) = 2
TABLE 3.1 CLASSIFICATION OF RANDOM PROCESSES

Continuous Discrete

We can attach a relative frequency interpretation to the probabilities as follows. Continuous I Continuous random Continuous; random
process sequence
In the case of the previous example, we toss the die n times and observe a time
function at each trial. We note the values of these functions at, say, time t = Discrete I Discrete random
process
Discrete random
sequence
4. Let k be the total number of trials such that at time t = 4 the values of the
I
- - - - - - - - - - - - - - ----l
118 RANDOM PROCESSES AND SEQUENCES METHODS OF DESCRIPTION 119

the noise is stationary, whereas the process shown in Figure 3.2 is nonstationary, 3.2.5 Formal Definition of Random Processes
that is, X(O) has a different distribution than X(4). More concrete definitions
of stationarity and several examples will be presented in Section 3.5 of this Let S be the sample space of a random experiment and let t be a variable that
chapter. can have values in the set r C RI> the real line. A real-valued random process
A random process may be either real-valued or complex-valued. In many X(t), t E f, is then a measurable function' on f X S that maps f X S onto
applications in communication systems, we deal with real-valued bandpass ran- R 1 • If the set r is a union of one or more intervals on the real line, then X(t)
dom processes of the form is a random process, and if r is a subset of integers, then X(t) is a random
sequence.
A real-valued random process X(t) is described by its nth order distribution
Z(t) = A(t)cos[21Tfct + 8(t)] functions.

where fc is the carrier or center frequency, and A(t) and 8(t) are real-valued Fx(t 1), X(t 2), ••• , X(<.) (x!, Xz, ... , Xn)
random processes. Z(t) can also be written as = P[X(t!) ::s: xl, 0 0 0 'X(tn) ::s: Xn]
for all n and t I ' 0 0 0 'tn E r (3.1)
Z(t) Real part of {A (t)exp[j8(t)] exp(j21Tfct)}
Real part of {W(t)exp(j21Tfct)} These functions satisfy all the requirements of joint probability distribution
functions.
Note that if r consists of a finite number of points, say ti> t 2 , • • • , tn, then
where the complex envelope W(t) is given by the random sequence is completely described by the joint distribution function
of the n-dimensional random vector, [X(t 1), X(t 2 ), • • • , X(tn)JY, where T
denotes the transpose of a vector.
W(t) = A(t)cos 8(t) + jA(t)sin 8(t)
= X(t) + jY(t)
3.3 METHODS OF DESCRIPTION
W(t) is a complex-valued random process whereas X(t), Y(t), and Z(t) are
real-valued random processes. A random process can be described in terms of a random experiment and the
Finally, a random process can be either predictable or unpredictable based associated mapping. While such a description is a natural extension of the concept
on observations of its past values. In the case of the ensemble of binary wave- of random variables, there are alternate methods of characterizing random proc-
forms X(t) shown in Figure 3.1, randomness is evident in each member function, esses that will be of use in analyzing random signals and in the design of systems
and future values of a member function cannot be determined in terms of past that process random signals for various applications.
values taken during the preceding T seconds, or earlier. Hence, the process is
unpredictable. On the other hand, all member functions of the random process
X(t) shown in Figure 3.2 are completely predictable if past values are known.
For example, future values of a member function can be determined completely 3.3.1 Joint Distribution
fort> t 0 > 0 if past values are known for 0 ::s: t ::s: t 0 • We know the six member Since we defined a random process as an indexed set of random variables, we
functions, and the uncertainty results from not knowing which outcome (and
can obviously use joint probability distribution functions to describe a random
hence the corresponding member function) is being observed. The member process. For a random process X(t), we have many joint distribution functions
function as well as the outcome can be determined from two past values. Note
that we cannot uniquely determine the member function from one observed
value, say at t = 4, since X(4) = 2 could result from either x 3 (t) or x 6 (t). If
*It is necessary only to assume that X(l) is measurable on S for every 1 E r. A random process is
we observe X(t) at two values oft, then we can determine the member function sometimes also defined as a family of indexed random variables, denoted by [X(I, ·);IE f], where
uniquely. the index set r represents the set of observation times.
 '··

120 RANDOM PROCESSES AND SEQUENCES METHODS OF DESCRIPTION 121

of the form given in Equation 3.1. This leads to a formidable description of the 100a 1 cos noat+ o1J
process because at least one n-variate distribution function is required for each
value of n. However, the first-order distribution function(s)P(X(t 1) :5 ad and
the second-order distribution function(s) P[X(t1 ) :5 a 1 , X(t 2 ) :5 a 2 ] are primarily
used. The first-order distribution function describes the instantaneous amplitude
distribution of the process and the second-order distribution function tells us
something about the structure of the signal in the time-domain and thus the
spectral content of the signal. The higher-order distribution functions describe
the process in much finer detail.
While the joint distribution functions of a process can be derived from a 100 cos (lOSt)
description of the random experiment and the mapping, there is no technique
for constructing member functions from joint distribution functions. Two dif-
ferent processes may have the same nth order distribution but the member
functions need not have a one-to-one correspondence. Figure 3.3 Example of a broadcasting system.

3.3.2 Analytical Description Using Random Variables

EXAMPLE 3.3.
We are used to expressing deterministic signals in simple analytical forms such
For the random process shown in Figure 3.2, obtain the joint probabilities as x(t) = 20 sin(lOt) or y(t) = exp( -t 2). It is sometimes possible to express a
P[X(O) and X(6)] and the marginal probabilities P(X(O)] and P[X(6)]. random process in an analytical form using one or more random variables.
Consider for example an FM station that is broadcasting a "tone," x(t) = 100
cos(l08t), to a large number of receivers distributed randomly in a metropolitan
SOLUTION: We know that X(O) and X(6) are discrete random variables and
area (see Figure 3.3). The amplitude and phase of the waveform received by
hence we can obtain the distribution functions from probability mass functions,
the ith receiver will depend on the distance between the transmitter and the
which can be obtained by inspection from Table 3.2.
receiver. Since we have a large number of receivers distributed randomly over
an area, we can model the distance as a continuous random variable. Since the
attenuation and the phase are functions of distance, they are also random vari-
ables, and we can represent the ensemble of received waveforms by a random
process Y(t) of the form

TABLE 3.2 JOINT AND MARGINAL PROBABILITIES OF X(t) AT t = 0 AND t 6

Y(t) = A cos(l08t + 8)
Values of Values of X(6).
-4 -3 -2 3 4
where A and e are random variables representing the amplitude and phase of
X(O) 2
-4 1/6 0 0 0 0 0 1/6 the received waveforms. It might be reasonable to assume uniform distributions
-2 0 0 1/6 0 0 0 1/6 Marginal for A and -6.
0 0 1/6 0 0 1/6 0 2/6 probabilities Representation of a random process in terms of one or more random variables
of X(O) whose probability law is known is used in a variety of applications in commu-
2 0 0 1/6 0 0 1/6
nication systems.
4 0 jo 0 0 0 1/6 1/6
1/6 /116 1/6 1/6 1/6 1/6
I
3.3.3 Average Values
As in the case of random variables, random processes can be described in terms
Joint probabilities of X(O) and X(6) of averages or expected values. In many applications, only certain averages
 122 RANDOM PROCESSES AND SEQUENCES
METHODS OF DESCRIPTION 123

derived from the first- and second-order distributions of X(t) are of interest. Note that because X is real, complex conjugates are omitted.
For real- or complex-valued random processes, these averages are defined as
follows:
Cxx(t!, tz) = Rxx(t!> tz)
Mean. The mean of X(t) is the expected value of the random variable X(t)
and
1-Lx(t) E{X(t)} (3.2)
1
Autocorrelation. The autocorrelation of X(t), denoted by Rxx(t 1 , t 2 ), is 40 + ;:;-t!t2
the expected value of the product X*(t 1 ) X(t 2 )
Rxx(tl> tz) E{X*(t 1 ) X(t 2 )} (3.3)
'=('·· ,,) 40 + H"'(•o H +
where * denotes conjugate.
Autocovariance. The autocovariance of X(t) is defined as
Cxx(t 1, tz) Rxx(tl> tz) - 1-1Ht J)I-LxCtz) (3.4)
Correlation Coefficient. The autocorrelation coefficient of X(t) is defined EXAMPLE 3.5.
as
Cxx(t 1> tz)
(3.5) A random process X(t) has the functional form
rxx(tJ, tz) = YCxxCtu t 1 ) Cxx(tz, tz)

The mean of the random process is the "ensemble" average of the values of all
the member functions at timet, and the' autocovariance function t 1 ) is
X(t) = A cos(lOOt + e) il
"
the variance of the random variable X(t 1 ). For t 1 ¥ t 2, the second moments
Rxx(tl, t 2 ), Cxx(t 1 , t 2 ), and rxx(tl> t2 ) partially describe the time domain where A is a normal random variable with a mean of 0 and variance of 1, and
il
structure of the random process. We will see later that we can use these functions e is uniformly distributed in the interval [ -71", 'Tr]. Assuming A and e are ll:!
'l
to derive the spectral properties of X(t). independent random variables, find [Link](t) and Rxx(t, t + T). '!
•I
'·I
For random sequences the argument n is substituted fort, and n 1 and n 2 are
substituted for t 1 and t 2, respectively. In this case the four functions defined SOLUTION:
:!
l
above are also discrete time functions.
l
1-Lx(t) = E{A} E{cos(lOOt + e)} = 0 l
EXAMPLE 3.4.
l
Rxx(t, t + T) = E{X(t 1 ) X(t 2 )} with t1=t+= t and t2 T

= E{A cos(lOOt + e) A cos(lOOt + lOOT + 8)}

Find 1-Lx(t), Rxx(t 1 , t 2 ), t2), and rxx(ti> t 2) for the random process
shown in Figure 3.2.
= E{ [cos(lOOT) + cos(200t + lOOT + 28)]}
lJ

SOLUTION: We compute these expected values by averaging the appropriate Az Az

+
ensemble values. = 2cos(l00T)
2 E{cos(200t + lOOT + 26)}
1
A2
1 6 = cos(l00T),
[Link](t) = E{X(t)} = 6 X;(t) = 0 2
1 6 since E{cos(200t + lOOT + 28)} = 0
Rxx(tl> t 2 ) = E{X(t 1 ) X(tz)} = 6 X;(t 1 ) X;(t 2 )
Note that Rxx (t, t + T) is a function only ofT and is periodic in T. In general,
= { 16 + 4 + 4 + 16 + 4
1 t!t2 + 4
1 t!t2 }
if a process has a periodic component, its autocorrelation function will also have
a periodic component with the same period.
= { 40 + t!t2}
 ;(f.

124 RANDOM PROCESSES AND SEQUENCES SPECIAL CLASSES OF RANDOM PROCESSES 125

3.3.4 Two or More Random Processes EXAMPLE 3.6.

When we deal with two or more random processes, we can use joint distribution
functions, analytical descriptions, or averages to describe the relationship be- Let E 1 be a random experiment that consists of tossing a die at t = 0 and
tween the random processes. Consider two random processes X(t) and Y(t) observing the number of dots on the up face, and let E 2 be a random experiment
whose joint distribution function is denoted by that consists of tossing a coin and observing the up face. Define random processes
X(t), Y(t), and Z(t) as follows:
P[X(t1) s; x1, ... , X(tn) s; Xm Y(ti) s; Y1, ... , s; Ym] Outcome of Outcome of
Experiment X(t) Y(t) Experiment Z(t)

Three averages or expected values that are used to describe the relationship O<t<oo O<t<oo O<t<oo
E1 Ez
between X(t) and Y(t) are
A; X;(t) y;(t) qi zi(t)
Cross-correlation Function. 1 -4 2 1 (head) cost
2 -2 -4 2 (tail) sin t
RXY(t 1, t 2 ) E{X*(t 1)Y(t2 )} (3.6)
3 2 4
Cross-covariance Function. 4 4 -2 !tif
CXY(tJ, lz) RXY(tJ> lz) - [Link](tz) (3.7) 5 -t/2 0
Correlation Coefficient (Also called cross-correlation coefficient). 6 t/2 0

ll. CXY(tl> lz) Random processes X(t) and Y(t) are defined on the same random experiment
(3.8)
rXY(tJ, lz) = YCxx(t 11 t 1) Cyy(tz, lz) E 1. However, X(t)"' Y(t) since x;(t) "'y;(t) for every outcome, A.;. These two
processes are orthogonal to each other since int
Using the joint and marginal distribution functions as well as the expected values,
we can determine the degree of dependence between two random processes. As 6
above, the same definitions are used for random sequences with n 1 and n 2 E{X(t 1)Y(t 2 )} = 2:: X;(tJ) y;(t 2) P[A.;)
replacing the arguments tl and lz. i=l
lut
=0
Equality. Equality of two random processes will mean that their respective
member functions are identical for each outcome A E S. Note that equality also They are also uncorrelated because CXY(tl> t 2 ) = 0. However, X(t) and Y(t)
implies that the processes are defined on the same random experiment. are clearly not independent. On the other hand, X(t) and Z(t) are independent
Uncorrelated. Two processes X(t) and Y(t) are uncorrelated when processes since these processes are defined on two unrelated random experiments
E 1 and £ 2 , and hence for any pair of outcomes A; E S1 and qi E S2 ,
C XY(t I> t 2 ) = 0, tl> lz E f (3.9)
Orthogonal. X(t) and Y(t) are said to be orthogonal if P(A.; and qi) = P(A;) P(qi)
RXY(tl, lz) = 0, tl> tz E r (3.10)
Independent. Random processes X(t) and Y(t) are independent if
P[X(tJ) s; x1, ... , X(tn) s; Xm Y(ti) s; Yt, ... , s; Ym]
=P[X(tt) s; x1, ... , X(tn) s; xn] P[Y(ti) s; Yt, ... , s; Ym] 3.4 SPECIAL CLASSES OF RANDOM PROCESSES
for all n, m and tl> t 2 , ••• , tn, ti, t2, ... , E f. (3.11)
In deterministic signal analysis, we use elementary signals such as sinusoidal,
As in the case of random variables, "independent" implies uncorrelated but not exponential, and step signals as building blocks from which other more com-
conversely. plicated signals can be constructed. A number of random processes with special

: i-t"'
1
 . ···Yc"-< ,,. . ),,' "·;;.,.·

126 RANDOM PROCESSES AND SEQUENCES SPECIAL CLASSES OF RANDOM PROCESSES 127

properties are also used in a similar fashion in random signal analysis_ In this [Link]. .A random [Link] X(t), .t. E r is called a Gaussian process if all
section, we introduce examples of a few specific processes. These processes and its nth order distributions Fx 1• x 2 • ... , Xn (x 1 , x 2 , • • • , xn) are n-variate Gaussian
their applications will be studied in detail in Chapter 5, and they are presented distributions [t 1 , t2 , • •• , tn E r, and X; = X(t;)].
here only as examples to illustrate some of the important and general properties
of random processes. Gaussian random processes are widely used to model signals that result from
the sum of a large number of independent sources, for example, the noise in a
low-frequency communication channel caused by a large number of independent
sources such as automobiles, power lines, lightning, and other atmospheric phe-
3.4.1 More Definitions
nomena. Since a k-variate Gaussian density is specified by a set of means and
a covariance matrix, knowledge of the mean f.l-x (t), t E r, and the correlation
Markov. A random process X(t), t E r, is called a first-order Markov (or function Rxx(t 1 , t 2 ), t 1 , t 2 E r, are sufficient to completely specify the probability
Markoff) process if for all sequences of times t 1 < t 2 < · · · < tk E rand k = distribution of a Gaussian process.
1, 2, ... we have
If a Gaussian process is also a Markov process, then it is called a Gauss-
Markov process.
P[X(tk) :s xkiX(tk- 1 ), ••• , X(t 1 )]
= P[X(td ::s xkiX(tk_J)] (3.12) 3.4.2 Random Walk and Wiener Process
Equation 3.12 says that the conditional probability distribution of X(tk) given In the theory and applications of random processes, the Wiener process, which
all past values X(t 1 ) = x 1 , • • • , X(tk_ 1) = xk-J depends only upon the most provides a model for Brownian motion and thermal noise in electrical circuits,
recent value X(tk_ 1) = xk-J· plays a fundamental role. In 1905, Einstein showed that a small particle (of say
diameter I0- 4 em) immersed in a medium moves randomly due to the continual
Independent Increments. A random process X(t), t E f is said to have bombardment of the molecules of the medium, and in 1923, Wiener derived a
independent increments if for all times t 1 < t 2 • • • < tk E r, and k = 3, 4, random process model for this random Brownian motion. The Wiener process
... , the random variables X(t 2 ) - X(t 1 ), X(t 3 ) - X(t 2 ), • •• , and X(tk) - can be derived easily as a limiting operation on a related random process called
X(tk_ 1 ) are mutually independent. a random walk.

The probability distribution of a process with independent increments is Random Walk. A discrete version of the Wiener process used to model the
completely specified by the distribution of an increment, X(t) - X(t'), for all random motion of a particle can be constructed as follows: Assume that a particle
t' < t and by the first-order distribution P[ X(t 0 ) :s x 0 ] at some single time is moving along a horizontal line until it collides with another molecule, and
instant, t 0 E f, since there is a simple linear relationship between X(tJ), ... , that each collision causes the particle to move "up" or "down" from its previous
X(tk) and the increments X(t 2 ) - X(t 1 ), • • • , X(tk) - X(tk_ 1 ), and since the path by a distance "d." Furthermore, assume that the collision takes place once
joint distribution of the increments is equal to the product of the marginal every T seconds and that the movement after a collision is independent of all
distributions. previous jumps and hence independent of its position. This model, which is
Two processes with independent increments play a central role in the theory analogous to tossing a coin once every T seconds and taking a step "up" if heads
of random processes. One is the Poisson process that has a Poisson distribution show and "down" if tails show, is called a random walk. The position of the
for the increments, and the second one is the Wiener process with a Gaussian particle at t = nTis a random sequence X(n) where in this notation for a
distribution for the increments. We will study these two processes in detail later. sequence, X(n) corresponds with the process X(nT), and one member function
of the sequence is shown in Figure 3.4. We will .assume that we start observing
Martingale. A random process X(t), t E f, is called a Martingale if the particle at t = 0, its initial location X(O) = 0 and that the jump of ± d
E{IX(t)l} < oo for all t E r, and appears instantly after each toss.
If k heads show up in the first n tosses, then the position of the particle at
E{X(t 2 )IX(t 1 ), t1 :s t 2} = X(t 1 ) for all t1 :s t2 (3.13) t = nTis given by
Martingales have several interesting properties such as having a constant
mean, and they play an important role in the theory of prediction of future X(n) kd + (n - k) (-d)
values of random processes based on past observations. (2k - n) d (3.14)
 •.1 11 .
I

128 RANDOM PROCESSES AND SEQUENCES SPECIAL CLASSES OF RANDOM PROCESSES 129

X(n) Since the number of heads in n tosses has a binomial distribution, we have

3d
P[X(n) = md] = (:)(ir, k = 0, 1, 2, ... , n; m= 2k - n
2d r-1
hI I tail
d _ _. L-1 and

It 5
t!T=n E{X(n)} = 0
-d L-'1 E{X(n) 2} = E{[ll + lz + · · · + ln]Z}
It
= nd 2
-2d L-1
It hi
-3d L-J We can obtain the autocorrelation function of the random walk sequence as

Figure 3.4 Sample function of the random walk process. Values of X(n) are shown Rxx(n 1 , n 2 ) = E{X(n 1 ) X(n 2 )}
as "e". = E{X(n 1 ) [X(n 1 ) + X(n 2 ) - X(n 1 )]}
= E{X(n 1 ) 2 } + E{X(n 1 ) [X(n:) - X(n 1 )]}

Now, if we assume n 2 > n 1 , then X(n 1) and [X(n 2 ) - X(nJ)] are independent
and X(n) is a discrete random variable having values md, where m equals - n, random variables since the number of heads from the first to n 1th tossing is
- n + 2, ... , n - 2, n. If we denote the sequence of jumps by a sequence of independent of the number of heads from (n 1 + l)th tossing to the n 2 th tossing.
random variables {J;}, then we can express X(n) as
Hence,

n 2 ) = E{X(ni) 2} + E{X(n1)} E{[X(n 2 ) - X(n 1)]}

X(n) = 1 1 + 1 2 + · · · + 1. 2
= E{X(n 2
1) } = n 1d

The random variables 1;, i = 1, 2, ... , n, are independent and have identical If n 1 > n 2 , then Rxx(n 1 , n 2 ) = n 2 d 2 and in general we can express Rxx(n 1 ,
distributions with n 2 ) as

Rxx(n 1 , n 2 ) = min(n 1 , n 2 ) d 2 (3.15)

1 1
P(J.l = d) = -2' P(l; = -d) -
2 It is left as an exercise for the reader to show that X(n) is a Markov sequence
E{J;} = 0 E{JT} = d 2 and a Martingale.

Wiener Process. Suppose we define a continuous-time random process Y(t),

From Equation 3.14 it follows that t E f = [0, oo) from the random sequence X(n) as

0, t = 0
P[X(n) = md] = P[k heads inn tosses), k = m +n Y(t) = { X(n), (n - 1)T < t:::; nT, n = 1, 2, ...
2
 --
130 RANDOM PROCESSES AND SEQUENCES
SPECIAL CLASSES OF RANDOM PROCESSES 131
A sample function of Y(t) is shown as a broken line in Figure 3.4. The mean
and variance of Y(t) at t = nT are given by

7
td 2
E{Y(t)} = 0 and E{Y2(t)} = T = nd 2 (3.16) 6

The Wiener process is obtained from Y(t) by letting both the time (T) between 4

jumps and the step size (d) approach zero with the constraint d 2 = aT to assure 3
that the variance will remain finite and nonzero for finite values oft. As a result
2
of the limiting, we have the Wiener process W(t) with the following properties:

1. W(t) is a continuous-amplitude, continuous-time, independent-incre-

ment process.
2 3
2. E{W(t)} = 0, and E{W 2 (t)} = at.
Random times at which
3. W(t) will have a Gaussian distribution since the total displacement or events occur
position can be regarded as the sum of a large number of small inde- Figure 3.6 Sample function of the Poisson random process.
pendent displacements and hence the central limit theorem applies. The
probability density function of W is given by

fw(w) = 1
;;:;----. exp - (-wz) 3.4.3 Poisson Process
v2Trat 2at
4. For any value of t', 0 ::s t' < t, the increment W(t) - W(t') has a The Poisson process is a continuous time, discrete-amplitude random process
Gaussian pdf with zero mean and a variance of a(t - t'). that is used to model phenomena such as the emission of photons from a light-
S. The autocorrelation of W(t) is emitting diode, the arrival of telephone calls at a central exchange, the occur-
rence of component failures, and other events. We can describe these events by
Rww(t 1 , t 2 ) = a min(ti> t 2 ) (3.17) a counting function Q(t), defined for t E f = [0, oo), which represents the
number of "events" that have occurred during the time period 0 to t. A typical
A sample function of the Wiener process, which is also referred to as the Wiener- realization Q(t) is shown in Figure 3.6. The initial value Q(O) of the process is
Levy process, is shown in Figure 3.5. The reader can verify that the Wiener assumed to be equal to zero.
process is a (nonstationary) Markov process and a Martingale. Q(t) is an integer-valued random process and is said to be a Poisson process
if the following assumptions hold:

1. For any times ti> t 2 E r and t 2 > the number of events Q (t 2 )

W(t) Q(t 1) that occur in the interval t 1 to t2 is Poisson distributed according
to the probability law

P[Q(t 2 ) - Q(tJ) = k] [A.(tz _- t 1)jk exp[- A.(tz - t1)]

k = 0, 1, 2, . . . (3.18)

2. The number of events that occur in any interval of time is independent

Figure 3.5 Sample function of the Wiener-Levy process. of the number of events that occur in other nonoverlapping time inter-
vals.

I
132 RANDOM PROCESSES AND SEQUENCES SPECIAL CLASSES OF RANDOM PROCESSES 133

From Equation 3.18 we obtain X(t)

P[Q(t) = k] = k! exp( k = 0, 1, 2, ... T

Ol ID

and hence the mean and variance of Q(t) are -1

E{Q(t)} = var{Q(t)} = (3.19) Figure 3.7 Random binary waveform.

Using the property of independent increments, we find the autocorrelation of

Q(t) as
The random sequence of pulses shown in Figure 3. 7 is called a random binary
waveform, and it can be expressed as
RQQ(t 1 , t 2 ) = E{Q(tJ) Q(t 2 )}
E{Q(tJ) [Q(t 1) + Q(t 2 ) - Q(t 1)]} for t2 t1
= E{Q (t 1)} + E{Q(t 1 )} E{Q(t 2 ) - Q(t1)}
2
X(t) = 2.: Akp(t- kT- D)
k= -oo

= + + - t1)]
= + At 2 ] for t 2 t 1 where p (t) is a unit amplitude pulse of duration T, Ak is a binary random variable
= t 1 t 2 + · min(t 1, t 2 ) for all t 1, t 2 E f (3.20) that represents the amplitude of the kth pulse, and D is the random start time
with a uniform distribution in the interval [0, T]. The sample function of X(t)
shown in Figure 3.7 is defined by a specific amplitude sequence{· · · 1, -1, 1,
-1, -1, 1, 1, -1, · · ·}and a specific value of delay D = T/4.
The reader can verify that the Poisson process is a Markov process and is For any value oft, X(t) has one of two values, ±1, with equal probability,
nonstationary. Unlike the Wiener-Levy process, the Poisson process is not a and hence the mean and variance of X(t) are
Martingale since its mean is time varying. Additional properties of the Poisson
process and its applications are discussed in Chapter 5.
E{X(t)} = 0 and E{X 2 (t)} = 1 (3.21)

To calculate the autocorrelation function of X(t), let us choose two values of

3.4.4 Random Binary Waveform
time t 1 and t 2 such that 0 < t 1 < t 2 < T. After finding t 2 ) with 0 <
Waveforms used in data communication systems are modeled by a random t 1 < t 2 < T, we will generalize the result for arbitrary values of t 1 and t 2 •
sequence of pulses with the following properties: From Figure 3.8 we see that when 0 < D < t 1 or t 2 < D < T, t 1 and t 2 lie
in the same pulse interval and hence X(t 1 ) = X(t 2 ) and the product X(t 1 )
1. Each pulse has a rectangular shape with a fixed duration of T and a X(t 2 ) = 1. On the other hand, when t 1 < D < t 2 , t 1 and t 2 lie in different pulse
random amplitude of ±1. intervals and the product of pulse amplitudes X(t 1) X(t 2 ) has the value + 1 or
- 1 with equal probability. Hence we have
2. Pulse amplitudes are equally likely to be ±1.

:1
3. All pulse amplitudes are statistically independent.
4. The start times of the pulse sequences are arbitrary; that is, the starting if 0 < D < t 1 or t 2 < D < T
time of the first pulse following t = 0 is equally likely to be any value X(t,) X(t,) {
if t1 < D < fz
between 0 and T.
 ,_. • -

134 RANDOM PROCESSES AND SEQUENCES

STATIONARITY 135

11- thermore, Rxx(t 1 + kT, t 2 + kT) = Rxx(t 1 , t 2 ), and hence

fT
I I OsDs1 1
ltz - t1l
{
11 and 12 belong to the
t
01 Dl
{:
same pulse interval and
t1 12 T
X(1 1) X(1 2) =1 T ltz-tii<T
-1 Rxx(ti> lz) (3.22)
elsewhere

The reader can verify that the random binary waveform is not an independent
If-.------.----- increment process and is not a Martingale.
A general version of the random binary waveform with multiple and cor-
12sD:s;T
related amplitude levels is widely used as a model for digitized speech and other
0

-I
I , : ·: I
I I
D '1 11 and 12 belong to the
same pulse interval and
X(ll)X(12) =1
signals. We will discuss this generalized model and its application in Chapters
5 and 6.

3.5 STATIONARITY
1r---
I
I l1sDst2
Time-invariant systems and steady-state analysis are familiar terms to electrical
I I 11 and 12 belong to different engineers. These terms portray certain time-invariant properties of systems and
1d Dl 12 T { pulse intervals and
X(tl)X(12) = ±1 signals. Stationarity plays a similar role in the description of random processes,
-II
I and it describes the time invariance of certain properties of a random process.
I
Whereas individual member functions of a random function may fluctuate rapidly
as a function of time, the ensemble averaged values such as the mean of the
Figure 3.8 Calculation of t2 ).
process might remain constant with respect to time. Loosely speaking, a process
is called stationary if its distribution functions or certain expected values are
invariant with respect to a translation of the time axis.
There are several degrees of stationarity ranging from stationarity in a strict
The random variable D has a uniform distribution in the interval [0, T] and
sense to a less restrictive form of stationarity called wide-sense stationarity. We
hence P[O < D < t 1 or t 2 < D < T] = 1 - (t 2 - t 1 )/T, and P(t 1 < D < ! 2 )
define different forms of stationarity and present a number of examples in this
= (t 2 - t 1 )1T. Using these probabilities and conditional expectations, we obtain section.

Rxx(t 1 , t 2 ) = E{X(t 1 ) X(t 2 )}

= E{X(t 1 ) X(t 2 )IO < D < t 1 or t2 < D < T}
3.5.1 Strict-sense Stationarity
· P[O < D < t1 or t 2 < D < T]
A random process X(t) is called time stationary or stationary in the strict sense
+ E{X(t 1 ) X(t 2 )lt 1 :s D :s t 2 } • P(t 1 :s D :s t 2 )
(abbreviated as SSS) if all of the distribution functions describing the process
1 . [ 1 _ (tz - t J)] + 1 . ! . (t 2 - t J) _ 1 . ! . (t 2 - t 1) are in¥[Link] -unden11ral}slation.·of ·time. ·That is, for all t 2 , • • • , tk> t 1 + T,
T 2 T 2 T tz + T, . • • 'tk + T E rand all k = 1, 2, ... '
_ (tz - t1)
1
T P[X(t 1) :s xi> X(t 2) :s x 2 , ••• , X(tk) :s xk]
= P[X(t 1 + T) :s x 11 X(t 2 + T) :s x 2, ... , X(tk + T) :s xk] (3.23)
To generalize this result to arbitrary values of t 1 and t 2 , we note that
t 2) = Rxx(t 2 , tJ), and that t 2 ) = 0 when lt2 - t 1l > T. Fur- If the foregoing definition holds for all kth order distribution functions k =
 """··

136 RANDOM PROCESSES AND SEQUENCES STATIONARITY 137

1, ... , N but not necessarily for k > N, then the process is said to be Nth Two processes X(t) and Y(t) are jointly WSS if each process satisfies Equation
order stationary. 3.28 and for all t E f.
From Equation 3.23 it follows that for a SSS process
E[X*(t)Y(t + T)] = [Link](T) (3.29)
P[X(t) =s x] = P[X(t + T) =s x] (3.24)
For random sequences, the conditions for WSS are
for any T. Hence, the first-order distribution is independent oft. Similarly
E{X(k)} = [Link] (3.30.a)
P[X(t 1) =s XI> X(t 2 ) =s x 2] = P[X(t 1 + T) =s XI> X(t 2 + T) =s x 2] (3.25)
and

for any T implies that the second-order distribution is strictly a function of the E{X*(n)X(n + k)} = Rxx(k) (3.30.b)
time difference t2 - t 1 • As a consequence of Equations 3.24 and 3.25, we
conclude that for a SSS process ---;·It is easy to show that SSS implies WSS; however, the converse is not true in
general.
E{X(t)} = [Link] = constant (3.26)

3.5.3 Examples
and the autocorrelation function will be a function of the time difference t 2 -
t1• We denote the autocorrelation of a SSS process by RxxCtz - t 1), defined as
EXAMPLE 3.7.
E{X*(t 1)X(tz)} = Rxx(tz - tt) (3.27)
Two random processes X(t) and Y(t) are shown in Figures 3.9 and 3.10. Find
the mean and autocorrelation functions of X(t) and Y(t) and discuss their sta-
- ···. It should be noted here that a random process with a constant mean and an tionarity properties.
autocorrelation function that depends only on the time difference t 2 - t 1 need
not even be first-order stationary.
Two real-valued processes X(t) and Y(t) are jointly stationary in the strict
sense if the joint distributions of X(t) and Y(t) are invariant under a translation
of time, and a complex process Z(t) = X(t) + jY(t) is SSS if the processes
5 Xj (t) = 5
X(t) and Y(t) are jointly stationary in the strict sense.
3 x 2 (t) = 3

3.5.2 Wide-sense Stationarity

_E _______ ;3(1)=1

-------l - X4(t)= -1
A less restrictive form of stationarity is based on the mean and the autocorre-
lation function. A process X(t) is said to be stationary in the wide sense (WSS
3 -3
or weakly stationary) if its mean is a constant and the autocorrelation function xs(t) =

depends only on the time difference:

-5 X6(t) = -5

E{X(t)} = [Link] (3.28.a)

Figure 3.9 Example of a stationary random process. (Assume equal probabilities of
E{X*(t)X(t + T)} = Rxx(T) (3.28.b) occurrence for the six outcomes in sample space.)
 ....,...-- -----

138 RANDOM PROCESSES AND SEQUENCES STATIONARITY 139

Since the mean of the random process Y(t) is constant and the autocorrelation
s function depends only on the time difference t 2 - t 1, Y(t) is stationary in the
wide sense. However, Y(t) is not strict-sense stationary since the values that
Y(t) can have at t = 0 and t = 11"14 are different and hence even the first-order
distribution is not time invariant.
3

EXAMPLE 3.8.

A binary-valued Markov sequence X(n), n E I= { ... , -2, -1, 0, 1, 2, ... }

has the following joint probabilities:

3 cos (t) = -3 cos (t)

Y4 (t) = Y5(t)
P[X(n) = 0, X(n + 1) = OJ = 0.2,
P[X(n) = 0, X(n + 1) = 1J = 0.2
Y&(t)
-6 P[X(n) = 1, X(n + 1) = OJ = 0.2,
Figure 3.10 Example of a nonstationary random process. (Member function are
P[X(n) = 1, X(n + 1) = 1] = 0.4
assumed to have equal probabilities.)
Find 1-lxCn), Rxx(n, n + 1), Rxx(n, n + 2), ... , and show that the sequence :j
I
is wide-sense stationary.
SOLUTION: SOLUTION:
I
_,
I
l'
E{X(t)} = 0 for all t E f = ( -oo, oo) P[X(n) = OJ = P[X(n) = 0, X(n + 1) = OJ !:
l'

Rxx(t 1 , t 2) =
1
6 (25 + 9 + 1 + 1 + 9 + 25) =
70
6 =
+ P[X(n) = 0, X(n + 1) = 1]
0.4
I
!
i:
P[X(n) = 1] = 0.6 !

Furthermore, a translation of the time axis does not result in any change in any
member function, and hence, Equation 3.23 is satisfied and X(t) is stationary Hence, E{X(n)} = 0.6, and E{[X(n)p} == 0.6.
in the strict sense.
For the random process Y(t), E{Y(t)} = 0, and E{X(n)X(n + 1)} == 1 · P[X(n) = 1, X(n + 1) == 1] == 0.4
E{X(n)X(n + 2)} = 1 · P[X(n) == 1, X(n + 2) = 1]
= l·P[X(n) = 1,X(n + 1) == 1,X(n + 2) = 1]
Ryy(tr, t 2) = {36 + 9 sin t 1 sin t2 + 9 sin t 1 sin t2
+ 1 · P[X(n) = 1, X(n + 1) = 0, X(n + 2) == 1J
+ 9 cos t 1 cos t2 + 9 cos t 1 cos t2 + 36} = 1 · P[X(n) = 1JP[X(n + 1) = 1[X(n) = 1J

1 x P[X(n + 2) = 1[X(n) = 1, X(n + 1) == 1J

= 6{72 + 18 cos(t 2 - t 1)} + 1 · P[X(n) = 1]P[X(n + 1) == O[X(n) = 1J
= Ryy(t 2 - t1) x P[X(n + 2) = 1[X(n) = 1, X(n + 1) == OJ
 ,,

140 RANDOM PROCESSES AND SEQUENCES STATIONARITY 141

The Markov property implies that SOLUTION:

n
P[X(n + 2) = 1IX(n) = 1, X(n + 1) = 1] E{X(t)} = .2: [E{A;} cos w;t + E{B;} sin w;t] = 0
= P[X(n + 2) = 1IX(n + 1) = 1] i=l

n n
P[X(n + 2) = 1IX(n) = 1, X(n + 1) = 0]
E{X(t)X(t + ,-)} = E {
[A; cos w;t + B; sin w;t]
= P[X(n + 2) = 1IX(n + 1) = 0]

x [Ai cos wi(t + 7) + Bi sin wi(t + ,-)]}

and hence

Since E{A;AJ, E{A;B;}, E{A;BJ, and E(B;Bi}, i =? j are all zero, we have

E{X(n)X(n + 2)} = (0.6) ( 0.4)

0.6 (0.4)
0.6 + (0.6) (0.2)
0.6 (0.2)
0.4
n

= 0.367 E{X(t)X(t + ,-)} = .2: [E{AT} cos w;t cos w;(t + ,-)

+ E{B1} sin w;t sin w;(t + T)]

Thus we have n

= u2 .2: cos W;T = Rxx(7)

i=l
E{X(n)} = 0.6
Rxx(n, n) = 0.6 all independent of n Since E{X(t)} and E{X(t)X(t + T)} do not depend on t, the process X(t) is
WSS.
Rxx(n, n + 1) = 0.4
This process X(t) for any values of tb t 2 , • • • , tk is a weighted sum of 2n
Rxx(n, n + 2) = 0.367 Gaussian random variables, A; and B;, i = 1, 2, ... , n. Since A;'s and B;'s
have a joint Gaussian distribution, any linear combinations of these variables
will also have a Gaussian distribution. That is, the joint distribution of X(t 1),
Proceeding in a similar fashion, we can show that Rxx(n, n + k) will be in- X(t 2), • • • , X(tk) will be Gaussian and hence X(t) is a Gaussian process.
dependent of n, and hence this Markov sequence is wide-sense stationary. The kth-order joint distribution of X(t 1), X(t 2 ), • • • , X(tk) will involve the
parameters E{X(t;)} = 0, and E{X(t;)X(ti)} = Rxx(lt; - til), which depends
only on the time difference t; - ti. Hence, the joint distribution of X(t 1),
X(t 2 ), • • • , X(tk), and the joint distribution of X(t 1 + T), X(t 2 + 7), ... ,
X(tk + ,-)will be the same for all values of,. and t; E r, which proves that X(t)
is SSS.
EXAMPLE 3.9.

A; and B;, i = 1, 2, 3, ... , n, is a set of 2n random variables that are uncor-

related and have a joint Gaussian distribution with E{A;} = E{B;} = 0, and
E{AT} = E{Bf} = u 2 • Let ----'';A Gaussian random process provides one of the few examples where WSS
implies SSS.

X(t) = .2: (A; cos w;t + B; sin w;t)

i= 1 3.5.4 Other Forms of Stationarity
A process X(t) is asymptotically stationary if the distribution of X(t 1 + ,-),
Show that X(t) is a SSS Gaussian random process. X(t 1 + ,-), ... , X(tn + ,-) does not depend on 7 when ,. is large.
 r -'"--·-.. . .
r,,
142 RANDOM PROCESSES AND SEQUENCES
AUTOCORRELATION AND POWER SPECTRAL DENSITY 143
r,,
A process X(t) is stationary in an interval if Equation 3.23 holds for all T for
autocorrelation function and the frequency content of a random process is the
Which tl + T, tz + T, . . . , tk + T lie in an interval that is a SUbset of r. main topic of .discussion .in this .[Link].
A process X(t) is said to have stationary increments if its increments Y(t) =
Throughout this section we will assume the process to be real-valued. The
X(t + T) - X(t) form a stationary process for every T. The Poisson and Wiener
concepts developed in this section can be extended to complex-valued random
processes are examples of processes with stationary increments.
processes. These concepts rely heavily on the theory of Fourier transforms.
Finally, a process is cyclostationary or periodically stationary if it is stationary
under a shift of the time origin by integer multiples of a constant T0 (which is
the period of the process).
3.6.1 Autocorrelation Function of a Real WSS Random Process and
Its Properties
3.5.5 Tests for Stationarity The autocorrelation function of a real-valued WSS random process is defined
as
If a fairly detailed description of a random process is available, then it is easy
to verify the stationarity of the process as illustrated by the examples given in il
Section 3.5.3. When a complete description is not available, then the stationarity 'i
Rxx(T) = E{X(t)X(t + T)}
of the process has to be established by collecting and analyzing a few sample
functions of the process. The general approach is to divide the interval of ob-
servation into N nonoverlapping subintervals where the data in each interval There are some general properties that are common to all autocorrelation func-
may be considered independent; estimate the parameters of the process using tions of stationary random processes, and we discuss these properties briefly
the data from nonoverlapping intervals; and test these values for time depend- before proceeding to the development of power spectral densities.
ency. If the process is stationary, then we would not expect these estimates from
the different intervals to be significantly different. Excessive variation in the 1. If we assume that X(t) is a voltage waveform across a 1-il resistance,
estimated values from different time intervals would indicate that the process is then the ensemble average value of X 2(t) is the average value of power
nonsta tionary. delivered to the 1-Q resistance by X(t):
Details of the estimation and testing procedures are presented in Chapters E{X 2(t)} = Average power
8 and 9.
Rxx(O) :=:: 0 (3.31)
2. Rxx(T) is an even function ofT
Rxx(T) = Rxx( -T) (3.32)
3.6 AUTOCORRELATION AND POWER SPECTRAL DENSITY 3. Rxx(T) is bounded by Rxx(O)
FUNCTIONS OF REAL WSS RANDOM PROCESSES
/Rxx(T)/ :S Rxx(O)
Frequency domain descriptions of deterministic signals are obtained via their This can be verified by starting from the inequalities
Fourier transforms, and this technique plays an important role in the charac-
terization of random waveforms. However, direct transformation usually is not E{[X(t + T) - X(t)]Z} :::: 0
applicable for random waveforms since a transform of each member function
E{[X(t + T) + X(t)] 2}:::: 0
of the ensemble is often impossible. Thus, spectral analysis of random processes
differs from that of deterministic signals. which yield
For stationary random processes, the autocorrelation function Rxx(T) tells
E{X 2(t + T)} + E{X 2(t)} - 2Rxx(T) :=:: 0
us something about how rapidly we can expect the random signal to change as
a function of time. If the autocorrelation function decays rapidly to zero it E{X 2(t + T)} + E{X 2(t)} + 2Rxx(T) :=:: 0
indicates that the process can be expected to change rapidly with time. And a
slowly changing process will have an autocorrelation function that decays slowly. Since E{X 2(t + T)} = E{X 2(t)} = Rxx(O), we have
Furthermore if the autocorrelation function has periodic components, then the 2Rxx(O) - 2Rxx(T) :=:: 0
underlying process will also have periodic components. Hence we conclude,
correctly, that the autocorrelation function contains information about the ex- 2Rxx(O) + 2Rxx(T) :=:: 0
pected frequency content of the random process. The relationship between the Hence, - Rxx(O) :S Rxx(T) :S Rxx(O) or /Rxx(T)/ :S Rxx(O)

I
 •,

144 RANDOM PROCESSES AND SEQUENCES AUTOCORRELATION AND POWER SPECTRAL DENSITY 145

4. If X(t) contains a periodic component, then Rxx(r) will also contain a 3.6.3 Power Spectral Density Function of a WSS Random Process and
periodic component. Its Properties
5. ,.....oo Rxx(T) = C, then C = fLi-.
If lim
For a deterministic power signal, x(t), the average power in the signal is defined
6. If Rxx(T0 ) = Rxx(O) for some T0 ¥ 0, then Rxx is periodic with a period
as
T0 • Proof of this follows from the cosine inequality (Problem 2.22a)
[E{[X(t + T + T0) - X(t + T)]X(t)}F Px = lim 1T IT x 2 (t) dt (3.36)
T--.:,.oo 2 -T
:S E{[X(t + T + T0) - X(t + T)] 2}E{X2(t)}

Hence
If the deterministic signal is periodic with period T0 , then we can define a time-
[Rxx(T + To) - Rxx(T)]Z :S 2[Rxx(O) - Rxx(T;J)]Rxx(O) averaged autocorrelation function {Rxx(-r)}r. as*
for every,. and T0 • If Rxx(T0 ) = Rxx(O), then Rxx(T + T0) = Rxx(T)
for every,. and Rxx(T) is periodic with period T0 • 1 (To
7. If Rxx(O) < oo and Rxx(-r) is continuous at,. = 0, then it is continuous (Rxx(-r))T0 = To Jo x(t)x(t + -r) dt (3.37)
for every T.

and show that the Fourier transform SxxCf) of {Rxx( T)h. yields
Properties 2 through 7 say that any arbitrary function cannot be an autocorre-
lation function.
Px = roo SxxCf) df (3.38)

3.6.2 Cross-correlation Function and Its Properties In Equation 3.38, the left-hand side represents the total average power in the
signal, f is the frequency variable expressed usually in Hertz (Hz), and Sxx(f)
The cross-correlation function of two real random processes X(t) and Y(t) that has the units of power (watts) per Hertz. The function SxxCf) thus describes the
are jointly WSS will be independent of t, and we can write it as power distribution in the frequency domain, and it is called the power spectral
density function of the deterministic signal x(t).
The concept of power spectral density function also applies to stationary
Rxy(T) E{X(t) Y(t + T)} random processes and the power spectral density function of a WSS random
process X(t) is defined as the Fourier transform of the autocorrelation function

The cross-correlation function has the following properties:

SxxU) = F{Rxx(T)} = roo Rxx(T)exp(- j2TrjT) dT (3.39)
1. = Rrx( --r) (3.33)
2. IRXY(-r)l :S YRxx(O)Rrr(O) (3.34) Equation 3.39 is called the Wiener-Khinchine relation. Given the power spectral
density function, the autocorrelation function is obtained as
1
2 [Rxx(O) +
r,
3. IRXY(-r)l :S Ryy(O)] (3.35)

4. RXY(T) = 0 if the processes are orthogonal, and Rxx(T) = p-l{SxxCf)} = SxxCf)exp(j2TrjT) df (3.40)

RXY( T) = fLxfLY if the processes are independent.

*The notation ( )r0 denotes integration or averaging in the time domain for a duration of T0 seconds
Proofs of these properties are left as exercises for the reader. whereas E{} denotes ensemble averaging.
 146 RANDOM PROCESSES AND SEQUENCES
t·l •..

AUTOCORRELATION AND POWER SPECTRAL DENSITY 147

Properties of the Power Spectral Density Function. The power spectral density
(psd) function, which is also called the spectrum of X(t), possesses a number
of important properties: I
I
1. Sxx(f) is real and nonnegative.
2. The average power in X(t) is given by

2
E{X (t)} == Rxx(O) == f" Sxx(f) df (3.41)
-B 0

(a) Lowpass spectrum

B
f

Note that if X(t) is a current or voltage waveform then E{XZ(t)} is the average Sxx(f)

fill fTl
I I 1
power delivered to a one-ohm load. Thus, the left-hand side of the equation
represents power and the integrand SxxU) on the right-hand side has the units
of power per Hertz. That is, S xx(i) gives the distribution of power as a function
of frequency and hence is called the power spectral density function of the
stationary random process X(t).
f
-fc-B/2 -{, -fc+B/2 0 fc-B/2 {, fc+B/2

3. For X(t) real, Rxx(-r) is an even function and hence SxxU) is also even. (b) Bandpass spectrum
That is
Sxx<fJ
S xx( -f) == S xx(f) (3.42) I
4. If X(t) has periodic components, then Sxx(f) will have impulses.

Lowpass and Bandpass Processes. A random process is said to be lowpass if

its psd is zero for /f/ > B, and B is called the bandwidth of the process. On the -B -rz o f1 fz B
other hand, a process is said to be bandpass if its psd is zero outside the band
(c) Power calculations

B B Total average power in the Average power in the

fc - - ::5 /J/ ::5 fc +- I:L:L2 signal X(t) frequency range { to h
2 2 1
Figure 3.11 Examples of power spectral densities.

r. is usually referred to as the center frequency and B is the bandwidth of the

process. Examples of lowpass and bandpass spectra are shown in Figure 3.11.
Notice that we are using positive and negative values of frequencies and the The proof of this equation is given in the next chapter. Figure 3.1l.c makes it
seem reasonable. The factor 2 appears in Equation 3.43 since we are using a
psd is shown on both sides of f == 0. Such a spectral characterization is called
a two-sided psd. two-sided psd and SxxU) is an even function (see Figure 3.1l.c and Equation
3.42).
Some processes may have psd functions with nonzero values for all finite
Power and Bandwidth Calculations. As stated in Equation 3.41, the area under values of f. io.r example, .Sxx(f) == exp(- :f/2). For such processes, several
the psd function gives the total power in X(t). The power in a finite band of indicators are used as measures of the spread of the psd in the frequency domain.
frequencies, fr to j 2 , 0 < j 1 < f 2 is the area under the psd from - f to - f One popular measure is the effective (or equivalent) bandwidth Beff· For zero
2 1
plus the area between fr to j 2 , and for real X(t)
mean random processes with continuous psd, Beff is defined as

Px[Jr, fz] == 2 J:' Sxx(f) df (3.43)

1 foo Sxx(f) df
Z max[SxxU)J
[,
Bert== (3.44)
 . '!

148 RANDOM PROCESSES AND SEQUENCES AUTOCORRELATION AND POWER SPECTRAL DENSITY 149 ,I

Sxx(fl II and
,,:
I,

RXY(-r) = roo SXY(f)exp(j2nfr) df (3.48)

:q

I
Equal areas Unlike the psd, which is a real-valued function off, the cpsd will, in general,
be a complex-valued function. Some of the properties of cpsd are as follows: 'I
q:
I 1. S xy(f) = S 'Yx(f)
-Bet! 0 Ben 2. The real part of SXY(f) is an even function off, and the imaginary part
Figure 3.U Definition of effective bandwidth for a lowpass signal. off is an odd function of f. 'I
3. SXY(f) = 0 if X(t) and Y(t) are orthogonal and SXY(f) = !J.x!J.y8(f) if
X(t) and Y(t) are independent. 1\
1\
In many applications involving the cpsd, a real-valued function
(See Figure 3.12.) The effective bandwidth is related to a measure of the spread ·I'
of the autocorrelation function called the correla-tion time T 0 where
IsXYUW ::S 1 1
(3.49)
p);y(f) = Sxx<f)Syy(f)
roo Rxx(-r) d-r
Tc = (3.45) 1.
Rxx(O) called tire coherence function is used as an indicator of the dependence between
two random processes X(t) and Y(t). When p);y(f0) = 0 at a particular fre- qi
quency, f 0 , then X(t) and Y(t) are said to be incoherent at that frequency, and I
If SxxU) is continuous and has a maximum at f = 0, then it can be shown that the two processes are said to be fully coherent at a particular frequency,
f 0 , when ph(f0) = 1. If X(t) and Y(t) are statistically independent, then !
1 = 0 at all frequencies except at f = 0.
(3.46) 4
Bell = 2Tc

Other measures of spectral spread include the rms bandwidth defined as the 3.6.5 Power Spectral Density Function of Random Sequences
standard deviation of the psd and the half-power bandwidth (see Problems 3.23 J
and 3.24). The psd of a random sequence X(nT,) with a uniform sampling time of one
second (T, = 1) is defined by the Fourier Transform of the sequence as l,
]

3.6.4 Cross-power Spectral Density Function and Its Properties Sxx(f) = L exp( -j2nfn)Rxx(n),
1 1
-- < f <-
2 2
(3.50.a)
11·=·-.Xl
The relationship between two real-valued random processes X(t) and Y(t) is
expressed in the frequency domain via the cross-power spectral density (cpsd)
function SXY(f), which is defined as the Fourier transform of the cross-correlation The definition implies that SxxCf) is periodic in f with period 1. We will only
function R XY( T), consider the principal part, -1/2 < f < 1/2. Then it follo\vs that

SXY(f) = roo RXY(-r)exp(- j2nf-r) dT (3.47) Rxx(n) =

J
l/2

-112
Sxx(f) exp([Link]) df (3.50.b)

·:i
 -r
;_!
150 RANDOM PROCESSES AND SEQUENCES
AUTOCORRELATION AND POWER SPECTRAL DENSITY 151
It is important to observe that if the uniform sampling time ( T.) is not one second
(i.e., if nT. is the time index instead of n) then the actual frequency range is X(n)le
not 1, but is 1/ T•.
If X(n) is real, then Rxx(n) will be even and
Xp(t)

Sxx(f) = L cos 2Trfn Rxx(n), lfl <

1
2 I I) I \'
i
(3.50.c) t
n= -oo
t1=nT nT+D (n+k) T t2 = (n+k) T+T'
(n+k)T+D
which implies that Sxx(f) is real and It is also nonnegative. In fact, SxxU) Figure 3.14 Details of calculations for Rx,x, (kT + -r').
of a sequence has the same properties assxx(f) of a continuous process except
of course, as defined, Sxx(f) of a sequence is periodic.
Although the psd of a random sequence can be defined as the Fourier trans-
form of the autocorrelation function Rxx(n) as in Equation 3.50.a, we present where p(t) is a pulse of height liE and duration E << T, and D is a random
a slightly modified version here that will prove quite useful later on. To simplify delay that has a uniform probability density function in the interval [- T/2,
the derivation, let us assume that E{X(n)} = 0. T/2] (see Figure 3.13). Except for its width and varying height, Xp(t) is similar
We start with the assumption that the observation times of the random in structure to the random binary waveform discussed earlier. It is fairly easy
sequence are uniformly spaced in the time domain and that the index n denotes to verify that Xp(t) will be WSS if X(n) is WSS.
t = nT. From the random sequence X(n), we create a random process Xp(t) To find the autocorrelation function of XP(t), let us arbitrarily choose t 1
of the form nT, and t2 = nT + kT + -r', 0 < -r' < E (see Figure 3.14). Following the line
of reasoning used in the derivation of the autocorrelation function of the random
binary waveform, we start with
Xp(t) = L
n= -oo
X(n)p(t - nT - D)
E{Xp(t 1)Xp(t2 )} = E{XP(nT)Xp(nT + kT + T')}
= Rx,x,(kT + -r')

From Figure 3.14, we see that the value of the product Xp(t 1)Xp(t2) will depend
on the value of D according to
X(O)

X(-1) X(3)

Xp(tt)Xp(tz) = { X(n)X(n + k) - - -r') :5 D :5 0< T' < E

€.2 '

I ' olI '

1 I I 0
otherwise
I (a) 3 n

X(2)
and Rx,xp(kT + T') is given by

Rxpxp(kT + -r') = E { Xp(t 1)Xp(tz)l- - T') :s D :s n

. p [- - -r') :s D :s
(b)
€.- -r'
Figure 3.13a Random sequence X(n). Figure 3.13b Random process XP(t). = E{X(n)X(n + k)} Til' 0< T' < E
 I

I
152 RANDOM PROCESSES AND SEQUENCES AUTOCORRELATION AND POWER SPECTRAL DENSITY 153

When T' > e, then irrespective of the value of D, t2 will fall outside of the pulse Rx,x,.l d
at t = kT and hence X(t2) and the product X(t1)X(t2) will be zero. Since XP(t)
is stationary, we can generalize the result to arbitrary values ofT' and k and I I Rxx(O)iET

write Rx,x, as
ll
Rxx(k) e - IT'! 1
Rx,x,(kT + T') IT'I < e
Te 2 '
j{
{
0 e < IT'I < T- e i
Rxx<2li<T Rxx(2)1•T

or Figure 3.15 Autocorrelation function of XP(t).

Rxx(k) e - i(T - kT)i

ikT- Ti < e
Rx,xJr) Te 2 '
{ If the random sequence X(n) has a nonzero mean, then Sxx(f) will have
0 elsewhere
discrete frequency components at multiples of 1/T (see Problem 3.35). Other-
1
= TL Rxx(k)q(T - kT) (3.51) wise, Sxx(f) will be continuous in f.
k The derivation leading to Equation 3.53 seems to be a convoluted way of
obtaining the psd of a random sequence. The advantage of this formulation will
be explained in the next chapter.
where q(t) is a triangular pulse of width 2e and height 1/e. An example of
Rx,xp(T) is shown in Figure 3.15. Now if we let e ----'> 0, then both p(t) and
q(t)----'> o(t) and we have
EXAMPLE 3.10.

Xp(t) = 2:
n=
X(n)o(t - nT- D) (3.52.a) Find the power spectral density function of the random process X(t) =
-x
10 cos(2000r.t + e) where e is a random variable with a uniform pdf in the
interval [- r., r.].
and
SOLUTION:
1
Rx,xp(T) = T 2: Rxx(k)o(T - kT) (3.52.b) Rxx('T) = 50 cos(2000m)
k= - x

The psd of the random sequence X(n) is defined as the Fourier transform of and hence
Rx,xp(T), and we have
Sxx(f) = 25[o(f - 1000) + o(f + 1000)]
Sx,x,(f) = F{Rxpxp(T)}
The psd of Sxx(fi) shown in .Figure 3.16 has two discrete components in the
= [ Rxx(O) + 2 Rxx(k) cos [Link] J (3.53) frequency domain at f = ± 1000 Hz. Note that

Rxx(O) = average power in the signal

Note that if T = 1, this is the Fourier transform of an even sequence as defined
in Equation 3.50.a, except the spectral density given in Equation 3.53 is valid (1Q)2 Joo
for -co < f < ro. =- -
2
= _, Sxx(f) df

:,';
 ·I
;.t
154 RANDOM PROCESSES AND SEQUENCES AUTOCORRELATION AND POWER SPECTRAL DENSITY 155

Sxx<fl Y{ n) are given by

I
25 0 (f+ 1000) 25 0 (f-1000)

I
1
T .2:
00

Rz,zp(T) = 6 exp( -0.5lkl)o(T - kT)

k= -oo

- 1000 Hz
i
0
t
1000 Hz
, Rrpr,(T) =
1
T .2:
k=
00

-oo
4o(T - kT)
Figure 3.16 Psd of 10 cos(Z0001rt + e) and 10 sin(Z0001rt + e).
andRx x (T) = R 2 2 (T) + Rr r (T)(see Figure 3.17). Taking the Fourier transform,
PP PP PP
we obtain the psd's as

Also, the reader can verify that Y(t) = 10 sin(20007rt + 8) has the same psd
as X(t), which illustrates that the psd does not contain any phase information. Szpz,(f) = .j. [6 + kt! 12 exp( -0.5k) cos 21rk[T J
= T6 [ -1 +
00

exp{ -(.5 + j27rfT)k} + 00

exp{ -(.5 - j27rfT)k} J
EXAMPLE 3.11. = [ -1 + 1/(1 - exp{- (.5 + j27rfT)}) + 1/(1 - exp{- (.5 - j27rfT)})]
6
A WSS random sequence X(n) has the following autocorrelation function: = - [(1-e- 1)/(1 - 2e-· 5 cos 21rjT + e- 1)]
T

Rxx(k) = 4 + 6 exp( -0.5lkl) 1

Sr,rP(f) = T2
"'
48
(
f - T
k)
Find the psd of Xp(t) and of X(n).
and
SOLUTION: We assume that as k---'; oo, the sequence is uncorrelated. Thus
Rxx(k) = [E{X(n)}]2 = 4. Hence E{X(n)} = ±2. If we define X(n) = sXPXP (f) = szpz.(f) + sY, Yp (f)
Z(n) + Y(n), with Y(n) = ± 2, then Z(n) is a zero mean stationary sequence
with Rzz(k) = Rxx(k) - 4 = 6 exp( -0.5lkl), and Rrr(k) = 4. The psd of XP(t) has a continuous part S2 p 2 .(f) and a discrete sequence of
The autocorrelation functions of the continuous-time versions of Z(n) and impulses at multiples of 11 T.
The psd of X(n) is the Fourier transform of R 22 (k) plus the Fourier transform
of Ryy(k) where

Rxpxp<Tl
1
I s yy(f) = 4o(f), III <2
(10/T)o(T)
/ '-..
/ '-.. and

S zz(f) = 6 exp(- .5lkl)exp(- j27rfk) J

-5T -4T -3T -2T -T 0 T 2T 3T 4T 5T 1
= 6[(1 - e- 1)/(1 - 2e-· 5 cos 21rf + e- 1)], If!< 2
Figure 3.17 Autocorrelation function of the random sequence X(n).
156 RANDOM PROCESSES AND SEQUENCES AUTOCORRELATION AND POWER SPECTRAL DENSITY 157

Thus

1
SxxU) = 4&(f) + 6[(1 - e- 1)/(1 - ze-.5 cos 271'f + e- 1)], Iii <2

Note the similarities and the differences between and Sxx· Essentially
SxxlfJ is the principal part of Sx;cp (i.e. the value of Sx;P(f) for <f <
and it assumes that Tis 1.
I
-3/T -2/T -1/T 0 liT 2/T 3/T
Figure 3.18b Power spectral density function of the random binary waveform.

EXAMPLE 3.12.
lobe. For many applications, the "bandwidth" of the random binary waveform
Find the psd of the random binary waveform discussed in Section 3.4.4. is defined to be 11 T.
SOLUTION: The autocorrelation function of X(t) is
EXAMPLE 3.13.
1- 1-rl ITI < T
Rxx(-r) = T' The autocorrelation function Rxx(T) of a WSS random process is given by
{
0 elsewhere
Rxx(T) = A exp( -aiTI); A, a> 0
The psd of X(t) is obtained (see the table of Fourier transform pairs in Appendix .!
A) as Find the psd and the effective bandwidth of X(t).

SOLUTION:
Sxx(f) = T [sin 7rfT]z
7rfT

A sketch of SxxU) is shown in Figure 3.18b. The main "lobe" of the psd extends
SxxU) = foo A exp( -al-rl)exp(- j271'jT) dT
from -liT to liT Hz, and 90% of the signal power is contained in the main 2Aa
az + (271'!)2

The effective bandwidth of X(t) is calculated from Equation 3.44 as

Rxx(r)
I
I
roo
SxxU) df 1 Rxx(O)

6.
1
B -- =---
eff -2 max[ S xx(f)] 2 S xx(O)
= A a
2 2Aia =4Hz
-T 0 T
Figure 3.18a Autocorrelation function of the random binary waveform.
 r

158 RANDOM PROCESSES AND SEQUENCES

AUTOCORRELATION AND POWER SPECTRAL DENSITY 159

EXAMPLE 3.14. 1000

1
The power spectral density function of a zero mean Gaussian random process
is given by (Figure 3.19)

1,
Sxx(f) = { 0 lfl <500Hz
elsewhere
7
v 7T I \1
X JL r (ms)

Find Rxx(T) and show that X(t) and X(t + 1 ms) are uncorrelated and, hence, Figure 3.19b Autocorrelation function of X(t).
independent.

SOLUTION:

EXAMPLE 3.15.
!
500
Rxx(T) = exp(j2TijT) df = exp(j2TijT) 15oo
-500 j21T'T -500
X(t) is a stationary random process with a psd
= (ZB) sin 2TI s,.
2TIBT ' B = 500Hz
1,
SxxU) = { 0
lfl< B
elsewhere
To show that X(t) and X(t + 1 ms) are uncorrelated we need to show that
E{X(t)X(t + 1 ms)} = 0.
X(t) is multiplied by a random process Y(t) of the form Y(t) = A cos
(2TifJ + 8), fc >> B, where 8 is a random variable with a uniform distribution
E{X(t)X(t + 1 ms)} = Rxx(l ms) in the interval [ -TI, 1r]. Assume that X(t) and Y(t) are independent and find
sin 1r the psd of Z(t) = X(t) Y(t).
= 28-- = 0
1T
SOLUTION:
Hence, X(t) and X(t + 1 ms) are uncorrelated. Since X(t) and X(t + 1 ms)
have a joint Gaussian distribution, being uncorrelated implies their indepen- A2
dence. Ryy(T) = 2 cos(27rfcT)

and

Sxx<fl
Rzz(T) = E{X(t)Y(t)X(t + T)Y(t + T)}
1 = E{X\t)X(t + -r)}£{Y(t)Y(t + T)}
= Rxx(T)Ryy(T)

f !
-500
Figure 3.19a
0 500I """
Psd of a lowpass random process X(t).
=

=
Rxx(T) ·

Rxx(T)
Az
4
A2
2 cos(27rfcT)

[exp(21TjfcT) + exp( -2TijfcT)]

I
 160 RANDOM PROCESSES AND SEQUENCES CONTINUITY, DIFFERENTIATION, AND INTEGRATION 161

Sxx<fl
Szz(fl
equations. In analyzing the response of these systems to deterministic input

ill,
I A 2/4 I I signals, we make use of rules of calculus as they apply to continuity, differen-
tiation, and integration. These concepts can be applied to random signals also,
either on a sample-function-by-sample-function basis or to the ensemble as a
-B 0 B
lowpass
signal X(t) l signal Z(t) -{, 0 {,
whole. When we discuss any of these concepts or properties as applying to the
whole ensemble, this will be done in terms of probabilities.
Consider, for example, the continuity property. A real (deterministic) func-
Carrier Y(t) =A (2T {,t+ 9)
tion x(t) is said to be continuous at t = t0 if
Syy(f)

(A2f4>6(f+f,)t t
(A2f4)5f(/-{,)
lim x(t) = x(to)
t-t 0

-{, "v 0 "v----','---

We can define continuity of a random process X(t) at t0 by requiring every
Figure 3.20 Psd of X(t), Y(t), and X(t) Y(t).
member function of the process to be continuous at t0 (sample continuity) or by
requiring continuity in probability,

Szz(f) = F{Rzz(r)} P[ X(t) is continuous at t 0] 1 (3.54)

= A24 [J"_, Rxx(r)exp(j2TrfcT)exp(- j2TrfT) dT or in a mean square (MS) sense by requiring

+ f, Rxx(T)exp(- j2Tr/cT)exp(- j2TrfT) dT J [Link]. X(t) = X(t 0 )

r-to

4
A2 [J"'_, Rxx(T)exp[-j2Tr(/- fc)T] dT
where l.i.m. denotes mean square (MS) convergence, which stands for
+ f, Rxx(T)exp[- j2Tr(f + fc)T] dT J lim E{[X(t) - X(t 0)F} = 0 (3.55)
A2 r-to
= 4 [Sxx<J - fc) + Sxx(f + fc)]
While sample continuity is the strongest requirement, MS continuity is most
useful since it involves only the first two moments of the process and much of
The preceding equations shows that the spectrum of Z(t) is a translated version the analysis in electrical engineering is based on the first two moments.
of the spectrum of X(t) (Figure 3.20). The operation of multiplying a "message" In the following sections we will define continuity, differentiation, and in-
signal X(t) by a "carrier" Y(t) is called "modulation" and it is a fundamental tegration operations in a MS sense as they apply to real stationary random
operation in communication systems. Modulation is used primarily to alter the processes, and derive conditions for the existence of derivatives and integrals
frequency content of a message signal so that it is suitable for transmission over of random processes.
a given communication channel.

3.7.1 Continuity
A stationary, finite variance real random process X(t), t E r, is said to be ,,

3.7 CONTINUITY, DIFFERENTIATION, AND INTEGRATION continuous in a mean square sense at t 0 E r if

Many dynamic electrical systems can be considered linear as a first approximation lim E{[X(t) - X(to)f} = 0
and their dynamic behavior can be described by linear differential or difference t-t0
 -
162 RANDOM PROCESSES AND SEQUENCES CONTINUITY, DIFFERENTIATION, AND INTEGRATION 163

Continuity of the autocorrelation function Rxx(T) at T = 0 is a sufficient condition Note that the definition does not explicitly define the derivative random process
for the MS continuity of the process. X'(t). To establish a sufficient condition for the existence of the MS derivative,
The sufficient condition for MS continuity can be shown by writing we make use of the Cauchy criteria (see Equation 2.97) for MS convergence
E{[X(t) - X(t 0 )]2} as which when applied to Equation 3.56 requires that

2
E{[X(t) - X(t0)]2} = E{XZ(t)} + E{X2(t 0)} - 2E{X(t)X(t0 )} lim E { [X(t + e 1) - X(t) _ X(t + e 2) - X(t)] } = O (3 .S 7)
-o
., 1 ,E 2 E1 Ez
= Rxx(O) + Rxx(O) - 2Rxx(t - to)
Completing the square and taking expected values, we have for the first term
and taking the ordinary limit

lim E{[X(t) - X(t 0 ))2} = Rxx(O) + Rxx(O) - 2lim Rxx(t- t0) E { [ X(t + - X(t) T} = 2[Rxx(O) - Rxx(e 1)]

Now, since Rxx(O) < oo, and if we assume Rxx(T) to be continuous at T = 0, Now, suppose that the first two derivatives of Rxx(T) exist at T = 0. Then, since
then Rxx(T) is even in T, we must have

lim Rxx(t - to) = Rxx(lo - to) = Rxx(O) = 0

r-r 0

and
and hence
Rxx(O) = lim 2[Rxx(E) - Rxx(O)]
E.-0 E2
lim E{[X(t) - X(t0)]2} = 0
t-r 0

Hence
Thus, continuity of the autocorrelation function at T = 0 is a sufficient condition
2
for MS continuity of the process.
MS continuity and finite variance guarantee that we can interchange limiting lim E { [X(t + e 1) - X(t)] } -RXx(O)
and expected value operations, for example "1-o Et

Proceeding along similar lines, we can show that the cross-product term in
lim E {g(X(t))} = E {g(X(t 0))}
r-ro Equation 3.57 is equal to 2Rxx(O), and the last term is equal to -Rxx(O).
Thus,

when g(·) is any ordinary, continuous function. 2

lim E {[X(t + e,) - X(t) _ X(t + e2) - X(t)] }

-o
E ,E 2
1
E1 E2

3.7.2 Differentiation
= 2[- Rxx(O) + Rxx(O)] = o
The derivative of a finite variance stationary process X(t) is said to exist in a
if the first two derivatives of Rxx(T) exist at T = 0, which guarantees the existence
mean square sense if there exists a random process X'(t) such that
of the MS derivative of X(t). This development is summarized by:

X(t + e) - X(t) = X'(t) A finite variance stationary real random process X(t) has a MS derivative, X'(t),
l.i.m. e (3.56)
•-o if Rxx(T) has derivatives of order up to two at T = 0.
 ft

164 RANDOM PROCESSES AND SEQUENCES CONTINUITY, DIFFERENTIATION, AND INTEGRATION 165 i!l

!It
The mean and autocorrelation function of X'(t) can be obtained easily as t 1) = Rxx(T), and we have
follows. The mean of X'(t) is given by i tf!

E{X'(t)} = O (3.58) !t(

E{X'(t)} = E{ [ X(t + - X(t)]} Rxx•(T) = dRxx(T) (3.59)
it(
dT
= lim E{X(t + e)} - E{X(t)} · !If;
2
•-o E Rx'X'(T) = d Rxx(T) (3.60) ill'
dT 2
= [Link](t) q
:i
For a stationary process, [Link](t) is constant and hence
3.7.3 Integration !il

E{X'(t)} = 0 The Riemann integral of an ordinary function is defined as the limit of a summing u;
operation
;!(
To find the autocorrelation function of X'(t), let us start with
n-1 r1r
{ X(T) dT = X(T;) flt; q
E{X(ti)X'(tz)} = tz) = E {x(t1) lim X(tz+ e) - X(tz)}
•-o E 11
where t 0 < t 1 < t2 < · · · < tn = tis an equally spaced partition of the interval,
which yields [t0, t], tlt; ""' t 1 +1 - t1, and 1"; is a 1JOint in the ith interval, [t;, t; +d. For a random 1{'
process X(t), the MS integral is defined as the process Y(t)
II
Rxx•(tJ. tJ = lim {RxxCtt, lz + E) - Rxx(tJ, tz)J
•-0 E n-1 il
Y(t) ={ X(T) dT = X(T;) tlt; (3.61)
The functions on the right-hand side of the preceding equation are deterministic
and the limiting operation yields the partial derivative of Rxx(t 1, t2) with respect
ii
to t2• Thus, It can be shown that a sufficient condition for the existence of the MS integral lf
Y(t) of a stationary finite variance process X(t) is the existence of the integral ::If
t ) = aRxx(tt. t2)
2
atz

Proceeding along the same lines, we can show that

J rr Rxx(t
r
to )to
1 - t2) dt 1 dt2 !{
l:
'11t
Note that finite variance implies that Rxx(O) < oo and MS continuity implies
Rx•x•(t1 , t 2) = t 2) continuity of Rxx(T) at T "'- 0, which alsojmplies continuity for all values ofT.
at 1 These two conditions guarantee the existence of the preceding integral and,
azRxx(tt, tz) hence, the existence of the MS
at 1Bt2 When the MS integral exists, we c'\'n show that l
l;
For a stationary process X(t), [Link](t) = constant, and Rxx(th t 2) = RxxCtz - E{Y(t)} = (t - to)[Link] (3.62)

.•.
 r--

166 RANDOM PROCESSES AND SEQUENCES TIME AVERAGING AND ERGODICITY 167

and - - - x(t)
'M xtfl +[Link]
Ryy(tb t 2 )
f, f''
to to
Rxx(T 1 - T2) dT 1 dT2 (3.63)

EXAMPLE 3.16.
: j

Discuss whether the random binary waveform is MS continuous, and whether

the MS derivative and integral exist. Figure 3.21 A member function of signal + noise.

SOLUTION: For the random binary waveform X(t), the autocorrelation func-
tion is
errors." If the value of the variable being measured is constant, and errors are
due to "noise" or due to the instability of the measuring instrument, then av-
Rxx(T) =
1-l:l ITI < T eraging is indeed a valid and useful technique. Time averaging is an extension
{ 0 T' elsewhere
of this concept and is used to reduce the variance associated with the estimation
of the value of a random signal or the parameters of a random process.
(a) Since Rxx(T) is continuous at T = 0, X(t) is MS continuous for all t. As an example, let us consider the problem of estimating the amplitudes of
(b) The derivative of X(t) does not exist on a sample-function-by-sample- the pulses in a random binary waveform that is corrupted by additive noise.
function basis and and R:¥x(O) do not exist. However, since their That is, we observe Y(t) = X(t) + N(t) where X(t) is a random binary wave-
existence is only a sufficient condition for the existence of the MS de- fonn, N(t) is the independent noise, and we want to estimate the pulse ampli-
rivative of X(t), we cannot conclude whether or not X(t) has a MS tudes by processing Y(t). A sample function of Y(t) is shown in Figure 3.21.
derivative. Suppose we observe a sample function y(t) with D = 0 over the time interval
(c) Finite variance plus MS continuity guarantees the existence of the MS (0, T), or from (k - 1) T to kT in general, and estimate the amplitude of x(t)
integral over any finite interval [t 0 , t]. in the interval (0, T). A simple way to estimate the amplitude of the pulse is
to take one sample of y(t) at some point in time, say t 1 E (0, T), and estimate
the value of x(t) as

The MS integral of a random process is used to define the moving average for 0< t< T if y(t 1) > 0; t 1 E (0, T)
x(t) = { +1
of a random process X(t) as -1 for 0 < t< T if y(t 1) :s 0; t 1 E (0, T)

f'
(X(t))r=T1 t-TX(T)dT The ' on x(t) denotes that i(t) is an estimate of x(t).
Because of noise, y(t) has positive and negative values in the interval (0, T)
even though the pulse amplitude x(t) is positive, and whether we estimate the
(X(t))r is also referred to as the time average of X(t) and has many important pulse amplitude correctly will depend on the instantaneous value of the noise.
applications. Properties of (X(t))r and its applications are discussed in the fol- Instead of basing our decision on a single sample of y(t), we can take m
lowing section. sample:; -of:r(t)in the 'interval fO, T); average the values, and decide

{+I
1 m

3.8 TIME AVERAGING AND ERGODICITY fm

0 < t < T if - 2.: y(t;) > 0;
m;=J
t; E (0, T)
i(t) = 1 m
When taking laboratory measurements, it is a common practice to obtain mul- -1 for 0 < t < T if - 2.: y(t;) :s 0; f; E (0, T)
tiple measurements of a variable and "average" them to "reduce measurement mi=l
168 RANDOM PROCESSES AND SEQUENCES TIME AVERAGING AND ERGODICITY 169

If the distribution of noise is assumed to be symmetrical about 0, then y(t) is or

more likely to have positive values than negative values when x(t) = 1, and
hence, the average value is more likely to be >0 than a single sample of y(t).
And we can conclude correctly that a decision based on averaging a large number
2.: g(x;)P(X = x;) (discrete case) (3.65.b)
of samples is more likely to be correct than a decision based on a single sample.
We can extend this concept one step further and use continuous time aver-
aging to estimate the value of x(t) as Some time averages that are of interest include the following.

Time-averaged Mean.
+1 if
1 rr
T Jo y(t) dt > 0 Ll. 1 IT/2
Ll.
(X(t))y = ([Link] = -T X(t) dt (3.66)
1 rr
x(t) = -T/2
{ -1
if T Jo y(t) dt :::; 0 Time-averaged Autocorrelation Function.
1 IT/2
The decision rule given above, which is based on time averaging, is extensively
(X(t)X(t + -r))r
Ll.
= (Rxx('r))y =
Ll.
T -T/ X(t)X(t + -r) dt (3.67)
2
used in communication systems. The relationship between the duration of the Time-averaged Power Spectral Density Function or Periodogram.
integration and the variance of the estimator is a fundamental one in the design
of communication and control systems. Derivation of this relationship is one of
the topics covered in this section.
I(X(t)exp( -2Tijft)hl
2
! (Sxx(f)h
We have used ensemble averages such as the mean and autocorrelation 2

function for characterizing random processes. To estimate ensemble averages T I IT/2

=Ll. 1 -r X(t)exp(- j2Tift) dt
12
1 (3.68)
one has to perform a weighted average over all the member functions of the
random process. An alternate practical approach, which is often misused, in- Interpretation of Time Averages. Although the ensemble average has a unique
volves estimation via time averaging over a single member function of the proc- numerical value, the time average of a function of a random process is, in general,
ess. Laboratory instruments such as spectrum analyzers and integrating volt- a random variable. For any one sample function of the random process, time
meters routinely use time-averaging techniques. The relationship between averaging produces a number. However, when all sample functions of a random
integration time and estimation accuracy, and whether time averages will con- process are considered, time averaging produces a random variable. For ex-
verge to ensemble averages (i.e., the concept of ergodicity) are important issues ample, the time averaged mean of the random process shown in Figure 3.9
addressed in this section. produces a discrete random variable.

5 when X(t) = x 1(t)

3.8.1 Time Averages 3 when X(t) = X 2 (t)
Definitions. The time average of a function of a random process is defined as ([Link] = -1
T
IT/2
-T!2
X(t) dt = i _ 11 when
when X(t) = x 3 (t)
X(t) =
-3 when X(t) = x 5 (t)
1 IT/2 -5 when X(t) = x 6 (t)
(g[X(t)]h = -T g[X(t)] dt (continuous case) (3.64.a)
-T/2
1 m Notice that in this example, none of the values of ([Link] equals the true ensemble
= - 2.: g[X(i)] (discrete case) (3.64.b)
mean of X(t), which is zero.
m i=t
The determination of the probability distribution function of the random
variable (g[ X(t)])r is in general very complicated. For this reason, we will focus
The corresponding ensemble average is given by our attention only on the mean and variance of (g[X(t)])r and use them to
analyze the asymptotic distribution of (g(X(t)])r as T--" 00 . In the following
E{g[X(t)]} = foo g(a)fx(a) da (continuous case) (3.65.a) derivation, we will assume the process to be stationary so that the ensemble
170 RANDOM PROCESSES AND SEQUENCES TIME AVERAGING AND ERGODICITY 171

averages do not depend on time. Finite variance and MS continuity will also be Then
assumed so that the existence of the time averages is guaranteed.
12
E{Y} = E{l_ fr Z(t) dt}
Mean and Variance of Time Averages. If we define a random variable Y as T -T/2
the average of m values of a real-valued stationary random process X(t) 1 fT/2
= -T E{Z(t)} dt
-T/2
1
L
m
(3.69) 1 fT/2
y =- = - 1-lz dt = 1-lz (3.73)
m i=l T -n2

is the time between samples, then we can calculate E{Y} and as To calculate the variance, we need to find £{Y 2}. By writing Y 2 as a double
integral and taking the expected value, we have
1 m } 1 m
E{Y} = E { -;;; =m
(3.70)
E{Y
2
} = E{ T1 fT/2-T/ Z(t1) dt1 T1 fT/2
-TiZ Z(t 2) dt 2
}

= !Lx 2
T/2

and = ; 2 JJ E{Z(tl)Z(t 2 )} dt 1 dt 2
-Tt2

;2 JJ
T/2
<T} = E{(Y - !Lx) 2
}
= Rzz(tl - t2) dt 1 dt 2
= E{ - - 1-lxl} -T/2

1
= ----:;
m·
2: Li
i
Cxx(li - (3.71) and

T/2
If the samples of X(t), taken seconds apart, are uncorrelated, then <T} = ; 2 JJCzz(t 1 - t 2) dt 1 dt 2 (3. 74)
-T/2
<Tk
E{Y} = !Lx and <Ty- m
0--
(3.72)
With reference to Figure 3.22, if we evaluate the integral over the shaded strip
centered on the line t 1 - t 2 = -r, the integrand C 22 (t 1 - t 2 ) is constant and
which shows that averaging of m uncorrelated samples of a stationary random equal to C22 (T), and the area of the shaded strip is [T- ITI] d-r. Hence, we
process leads to a reduction in the variance by a factor .of m. can write the double integral in Equation 3.74 as
We can extend this development to continuous time averages as follows. To
simplify the notation, let us define
:2 JJ Czz(tl- :2 rT
T/2

tz) dtl dt2 = [T- ITIJCzz(T) d7

Z(t) = g[X(t)] -T/2

and or

Y = -1
T
JT/2
-T/2
Z(t) dt 1
<T} = T JT [ 1 - T1-rl]
-T Czz(-r) dT (3.75.a)
I ·:-, i

172 RANDOM PROCESSES AND SEQUENCES TIME AVERAGING AND ERGODICITY 173 ii

Sxx<fl H
//' 12

/\'1- 'If
/ \"> T/2
10 -6 II
/
/ 'tt
/
< xO.' H·
// /
/\"'/ /vt i!'
-T/21 0 I T/21 \">/ / / \ . :
Jl_ A t1
-500 0 500 {(KHz)

Figure 3.23 Psd of X(t) for Example 3.17. . ! H·

, L::::¥..-1 dT iiP
:II
SOLUTION:
I
-T/2
t r---7 1/ /\'1.
/1 \">
l!i
r---- T- T---...,
t....,__
!l;
1
E{Y} = E{X(6.) + X(26.) + · · · + X(106.)}
Figure 3.22 Evaluation of the double integral given in Equation 3.74. 10 lf
1 n
= 10 [E{X(6.)} + E{X(2t1)} + · · · + E{X(106.)}]
ll
= 0
It is left as an exercise for the reader to show that can be expressed as the tf
1
following integral in the frequency domain: E{Y 2} = E{[_!_
10 z=l
X(it1)] [
10 J=l
X(j6.)]}
lli

= E{X(it1)X(j6.)} !·

= (si:;:Tr df (3.75.b) I I

1
= 100 Rxx(li - jjA) •In
I I

if
where
Since Rxx(k) = 0 fork ¥ 0 (why?), and Rxx(O) = ai = 1, we obtain ;I;

1 10 1 It
Sh(f) = F{C 22 (-r)} = exp(-j2'ITf-r)Czz(-r) d-r 2
E{Y } =- Rxx(O) = - tj
100 i=l 10
:l
or
The advantages of time averaging and the use of Equations 3.71, 3.75.a, and j·£,

3.75.b to compute the variances of time averages are illustrated in the following
examples. 1 ai
a}= 10 = 10
'li

EXAMPLE 3.17. \.I

X(t) is a stationary, zero-mean, Gaussian random process whose power spectral EXAMPLE 3.18. a
density is shown in Figure 3.23. Let Y = 1110{X(6.) + X(26.) + · · · + X(106.)},
6. = 1 fLS. Find the mean and variance of Y. A lowpass, zero-mean, stationary Gaussian random process X(t) has a power

....
 r
l

174 RANDOM PROCESSES AND SEQUENCES TIME AVERAGING AND ERGODICITY 175

spectral density of or

for lfl < B 1 A

Sxx(f)
for lfl 2 B a}= Sxx(O) ·y. = T
Let
and

y = T1 IT/2 X(t) dt
-T/2 ai 2AB
a}= (AIT) = ZBT, BT >> 1
Assuming that T >> 11 B, calculate a} and compare it with ai.

SOLUTION:

2 The result derived in this example is important and states that time averaging
ai = E{X } = Rxx(O) = Sxx(f) df of a lowpass random process over a long interval results in a reduction in variance
= 2AB
by a factor of 2BT (when BT >> 1). Since this is equivalent to a reduction in
variance that results from averaging 2BT uncorrelated samples of a random
E{Y} = -1 IT/2 E{X(t)} dt = 0 sequence, it is often stated that there are 2BT uncorrelated samples in a T
T -T/2 second interval or there are 2B uncorrelated samples per second in a lowpass
random process with a bandwith B.
a}= E{Y2} = SxxU) (sin nfT)2 df
nfT

From Figure 3.24, we see that the bandwidth or the duration of (sin nfT!nfT) 2 EXAMPLE 3.19.
is very small compared to the bandwidth of SxxU) and hence the integral of
the product can be approximated as Consider the problem of estimating the pulse amplitudes in a random binary
waveform X(t), which is corrupted by additive Gaussian noise N(t) with 1-LN =

I x
_zSxxU)
(sinnfT
nfT)
2

df=Sxx(O)
[
areaunder (sinnfT n/T) 2
]
0 and RNN( T) = exp( -ITI!a.). Assume that the unknown amplitude of X(t) in
the interval (0, T) is 1, T = 1 ms, a = 1 fLS, and compare the accuracy of the
following two estimators for the unknown amplitude:

(a) S1 = Y(t 1 ), t 1 E (0, T)

(b) S' 2 = -1 IT Y(t) dt

T o

[sin [ 1r fl')/1r (FJ2

where Y(t) = X(t) + N(t).

SOLUTION:
Sxx(fl

S1 = X(t1) + N(t1) = 1 + N(t1)

Figure 3.24 Variance calculations in the frequency domain. E{S 1} = 1
 176 RANDOM PROCESSES AND SEQUENCES TIME AVERAGING AND ERGODICITY 177

and a central problem in the theory of random processes is the estimation of the
parameters of random processes (see Chapter 9). If the theory of random proc-
var(S 1) = RNN(O) = 1 esses is to be useful, then we have to be able to estimate such quantities as the
mean and autocorrelation from data. From a practical point of view, it would

T1 Jorr £{1 +
' be very attractive if we can do this estimation from an actual recording of one
E{S2} = N(t)} dt = 1 sample function of the random process.
Suppose we want to estimate the mean [Link](t) of the random process X(t).
The mean is defined as an ensemble average, and if we observe the values of
and X(t) over several member functions, then we can use their average as an en-
semble estimate of [Link](t). On the other hand, if we have access to only a single

var{S 2} = r N(t) dt}

member function of X(t), say x(t), then we can form a time average (x(t))r

= 1 T IT-T [1 - T
ITIJ CNN(-r) d-r
1 JT/2
(x(t))r = -T x(t) dt

r [1- i]
-T/2

= d-r
and attempt to use the time average as an estimate of the ensemble average,
[Link](t).
= T- T2 [1- Whereas the time average (x(t))r is a constant for a particular member
function, the set of values taken over all member functions is a random variable.
That is, (X(t))r is a random variable and (x(t))r is a particular value of this
Since 1, the second term in the preceding equation can be neglected
random variable. Now, if [Link](t) is a constant (i.e., independent of t), then the
and we have
"quality.,., of the time-averaged estimator will depend on whether E{(X(t))r}--'>
[Link] and the variance of {(X(t))r}--'> 0 as T--'> oo. If
1
var{Sz} =y = 500
lim E{(X(t))r} = [Link]

and standard deviation of S2 = 1/VsOO = 0.0447.

Comparing the standard deviation (a) of the estimators, we find that a of and
sl is 1, which is the same order of magnitude of the unknown signal amplitude
being estimated. On the other hand, a of S2 is 0.0447, which is quite small
lim var{(X(t))r} = 0
compared to the signal amplitude. Hence, the fluctuations in the estimated value
due to noise will be very small for S2 and quite large for S1• Thus, we can expect
S2 to be a much more accurate estimator.
then we can conclude that the time-averaged mean converges to the ensemble
mean and that they are equal. In general, ensemble averages and time averages
are not equal except for a very special class of random processes called ergodic
processes- The concept of ergodicity deals with the equality of time averages
and ensemble averages.
The problem of determining the properties of a random process by time
3.8.2 Ergodicity
averaging over a single member function of finite duration belongs to statistics
In the analysis and design of systems that process random signals, we often and is covered in detail in Chapter 9. In the following sections, we will derive
assume that we have prior knowledge of such quantities as the means, auto- the conditions for time averages to be equal to ensemble averages. We will focus
correlation functions, and power spectral densities of the random processes our attention on the mean, autocorrelation, and power spectral density functions
involved. In many applications, such prior knowledge will not be available, and of stationary random processes.
178 RANDOM PROCESSES AND SEQUENCES TIME AVERAGING AND ERGODICITY 179

General Definition of Ergodicity. A stationary random process X(t) is called and the variance of (!-lxh can be obtained from Equation 3.75.a as
ergodic if its ensemble averages equal (in a mean-square sense) appropriate time
averages. This definition implies that, with probability one, any ensemble av-
erage of X(t) can be determined from a single member function of X(t). In .
var{(!-lxh} = T
1 JT [ 1 - T1-rl]
-T Cxx(-r) d-r
most applications we are usually interested in only certain ensemble averages
such as the mean and autocorrelation function, and we can define ergodicity
with respect to these averages. In presenting these definitions, we will focus our If the variance given in the preceding equation approaches zero, then X(t) is
attention on time averages over a finite interval (- T/2, T/2) and the conditions ergodic in the mean. Note that E{(!-lx)r} is always equal to 1-lx for a stationary
under which the variances of the time averages tend to zero as T oo. random process. Thus, a stationary process X(t) is ergodic in the mean if
It must be pointed out here that ergodicity is a stronger condition than
stationarity and that not all processes that are stationary are ergodic. Further-
more, ergodicity is usually defined with respect to one or more specific ensemble . T1 JT_T (1 - T
1-rl) Cxx( T) d-r = 0 (3.77)
averages, and a process may be ergodic with respect to some ensemble averages
but not others.
Although Equation 3.77 states the condition for ergodicity of the mean of
X(t), it does not have much use in applications involving testing for ergodicity
Ergodicity of the Mean. A stationary random process X(t) is said to be ergodic of the mean. In order to use Equation 3.77 to justify time averaging, we need
in the mean if 'pr:ior knowledge of Cxx(-r). However, Equation 3.77 might be of use in some
situations if only partial knowledge of C xx( -r) is available. For example, if we
know that ICxx(-r)l decreases exponentially for large values of 1-rl, then we can
l.i.m. (!-lxh = 1-lx show that Equation 3.77 is satisfied and hence the process is ergodic in the mean.

Ergodicity of the Autocorrelation Function. A stationary random process X(t)

is said to be ergodic in the autocorrelation function if
where l.i.m. stands for equality in the mean square sense, which requires

l.i.m. (Rxx(a))y = Rxx(a) (3.78)

r-oo
lim £{(1-lxh} 1-lx
T-x

The reader can show using Equations 3.73 and 3.75.a that
and
E{(Rxx(a))y} = Rxx(a) (3.79)
lim var{(!-lxh} = 0
T-x
and

Now, the expected value of (1-lxh for a finite value ofT is given by
var{(Rxx(a))r} = T
1 JT (1 -
-T
1-rl)
T Czz(-r) d-r (3.80)

1E
E{(!-lxh} = T {JT/2
-T/ X(t) dt } where Z(t) = X(t)X(t + a).
2
As in the case of the time-averaged mean, the expected value of the time-
1 JT/2 1 JT/2 averaged autocorrelation function is equal to Rxx(-r) irrespective of the length
= -T E{X(t)} dt = - 1-lx dt
- n2 T -n2 of averaging (T). If the right-hand side of Equation 3.80 approaches zero as
= 1-lx (3.76) T oo, then the time-averaged autocorrelation function equals the true auto-
,- TIME AVERAGING AND ERGODICITY 181
180 RANDOM PROCESSES AND SEQUENCES

correlation function. Hence, for any given a EXAMPLE 3.20. i

l.i.m. (Rxx(a))r = Rxx(a) For the stationary random process shown in Figure 3.9, find E{(!-Lxh} and lm
I
r-oo var{(!-Lxh}. Is the process ergodic in the mean?
:lm
if SOLUTION: (!-Lxh has six values: 5, 3, 1, -1, -3, -5, and

!lit
. 1
hm-
r-ooT
Jr (1 -
-r
1-rl)
T [E{Z(t)Z(t + -r)} - Rh(a)] d-r = 0 (3.81) E{(!-Lxh} =
1
6 {5 + 3 + 1 - 1 - 3 - 5} = 0 jill
'f
Variance of (!-Lxh can be obtained as
where Z(t) = X(t)X(t + a).
Note that to verify ergodicity of the autocorrelation function we need to
have knowledge of the fourth-order moments of the process. 1
var{(!-Lxh} = + 3 2 + 12 + (-1) 2 + (-3) 2 + (-5)2}
6 {5
2

Ergodicity of the Power Spectral Density Function. The psd of a stationary = 70/6
random process plays a very important role in the frequency domain analysis
and design of signal-processing systems, and the determination of the spectral lnl
characteristics of a random process from experimental data is a common engi- Note that the variance of (!-Lxh does not depend on T and it does not decrease
as we increase T Thus, the condition stated in Equation 3.77 is not met and
:Ill
neering problem. The psd may be estimated by taking the Fourier transform of
the time-averaged autocorrelation function. the process is not ergodic in the mean. This is to be expected since a single
A faster method of estimating the psd function involves the use of the time member function of this process has only one amplitude, and it does not con-
tain any of the other five amplitudes that X(tj am have.
. !II!
average

2
(Sxx(f)h = T
1 I JT/2
-r X(t)exp( -j21Tjt) dt (3.82)
12 1

which is also called the periodogram of the process. Note that the integral EXAMPLE 3.21.
represents the finite Fourier transform; the magnitude of the Fourier transform
squared is the energy spectral density function (Parseval's theorem); and liT is Consider the stationary random process
the conversion factor for going from energy spectrum to power spectrum.
Unfortunately, the time average (Sxx(f))r does not converge to the ensemble X(t) = 10 cos(lOOt + e)
average Sxx(f) as T--'> oo. We will show in Chapter 9 that while

where e is a random variable with a uniform probability distribution in the

lim E{(Sxx(f))r} = Sxx(f) interval [ -1T, 1r]. Show that X(t) is ergodic in the autocorrelation function.
r-'"

SOLUTION: iif
the variance of (Sxx(f)h does not go to zero as T--'> oo. Further averaging of
the estimator (S xx(f))r in the frequency domain is a technique that is commonly Iii!
used to reduce the variance of (Sxx(f))r. Although we will deal with the problem Rxx(-r) = E{lOO cos(lOOt + e) cos(lOOt + 100-r + 6)}
ill
of estimating psd functions in some detail in Chapter 9, we want to point out = 50 cos(lOO-r)
here that estimation of psd is one important application in which a direct sub- Ill!
1 JT/2
stitution of the time-averaged estimate (Sxx (f))r for the ensemble average S xx (f) (Rxx(-r))r = -T X(t)X(t + -r) dt
-r12
is incorrect.
 .r-
182 RANDOM PROCESSES AND SEQUENCES
t; '
TIME AVERAGING AND ERGODICITY 183
T
expect the process to be at least weakly ergodic. On the other hand, each of
1 JT/2
= -T 100 cos(100t + 8) cos(100t + lOOT + 8) dt tl:le functions of .the nmdom process shown in Figure 3.9 is a constant
-T/2
and by observing one member function we learn nothing about other member
1 JT/2 functions of the process. Hence, for this process, time averaging will tell us
= -T 50 cos(100T) dt
-T/2 nothing about the ensemble averages. Thus, intuitive justification of ergodicity
boils down to deciding whether a single member function is a "truly random
+ 1
-T
JT/2 50 cos(200t + lOOT + 28) dt signal" whose variations along the time axis can be assumed to represent typical
-T/2 variations over the ensemble.
The comments given in the previous paragraph may seem somewhat circular,
Irrespective of which member function we choose to form the time-averaged and the reader may feel that the concept of ergodicity is on shaky ground.
correlation function (i.e., irrespective of the value of 8), as T--"> oo, we have However, we would like to point out that in many practical situations we are
forced to use models that are often hard to justify under rigorous examination.
Fortunately, for Gaussian random processes, which are extensively used in
(Rxx(T)h = 50 cos(lOOT)
a variety of applications, the test for ergodicity is very simple and is given below.
= Rxx(T)

Hence, E{(Rxx(T))r} = Rxx(T) and var{(Rxx(T))r} = 0. Thus, the process is EXAMPLE 3.22.
ergodic in autocorrelation function.
Show that a stationary. zero-mean, finite variance Gaussian random process is
ergodic in the general sense if
Other Forms of Ergodicity. There are several other forms of ergodicity and
some of the important ones include the following: ]Rxx(T)j dT < oo

Wide Sense Ergodic Processes. A random process is said to be wide-sense

SOLUTION: Since a stationary Gaussian random process is completely speci-
ergodic (WSE) if it is ergodic in the mean and the autocorrelation function.
fied by its mean and autocorrelation function, we need to be concerned only
WSE processes are also called weakly ergodic.
with the mean and autocorrelation function (i.e., weakly ergodic implies ergod-
Distribution Ergodic Processes. A random process is said to be distribution icity in the, general sense for a stationary Gaussian random process). For the
ergodic if time-averaged estimates of distribution functions are equal to the process to be ergodic in mean, we need to show that
appropriate (ensemble) distribution functions.
Jointly Ergodic Processes. Two random processes are jointly (wide-sense)
ergodic if they are ergodic in their means and autocorrelation functions and
also have a time-averaged cross-correlation function that equals the ensemble
T1 JT ( 1 - T]Ti) Cxx('r) dT
-T = 0
averaged cross-correlation functions.
The preceding integral can be written as
Tests for Ergodicity. Conditions for ergodicity derived in the preceding sections
are in general of limited use in practical applications since they require prior
knowledge of parameters that are often not available. Except for certain simple
cases, it is usually very difficult to establish whether a random process meets 0 :5 H)
I1 JT-r (1 - T Cxx(T) dT :5 T1 IT-T ]Cxx(T)] dT
the conditions for the ergodicity of a particular parameter. In practice, we are
usually forced to consider the physical origin of the random process to make an
intuitive judgment of ergodicity. Hence,
For a process to be ergodic, each member function should "look" random,
even though we view each member function to be an ordinary time signal. For
example, if we consider the member functions of a random binary waveform, 1
lim -T JT ( 1 - ITI) Cxx(T) dT = 0
-
-T T
randomness is evident in each member function and it might be reasonable to
184 RANDOM PROCESSES AND SEQUENCES SPECTRAL DECOMPOSITION AND SERIES EXPANSION 185

since Since \Rxx(T)\ dT < cc, the upper bound approaches 0 as T ______,.co, and hence
the variance (V) of the time-averaged autocorrelation function______,. 0 as,.______,. co.

roo \Rxx(T)\ dT < cc

Thus, if the autocorrelation function is absolutely integrable, then the stationary
Gaussian process is ergodic. Note that this is a sufficient (but not a necessary)
condition for ergodicity. Also note that dT < co requires that
[Link] = 0.
To prove ergodicity of the autocorrelation function, we need to show that, for
every a, the integral

V = T1 IT (1 - T,,.,)
-T Czz(T) dT; Z(t) = X(t)X(t + a)

3.9 SPECTRAL DECOMPOSITION AND SERIES EXPANSION OF

approaches zero as T co. RANDOM PROCESSES
The integral V can be bounded as
We have seen that a stationary random process can be described in the frequency
domain by its power spectral density function which is defined as the Fourier
0 ::5 V 1
::5- IT \Czz(T)\ dT transform of the autocorrelation function of the process. In the case of deter-
T -T ministic signals, the expansion of a signal as a superposition of complex expo-
nentials plays an important role in the study of linear systems. In the following
where discussion, we will examine the possibility of expressing a random process X(t)
by a sum of exponentials or other orthogonal functions. Before we start our
discussion, we would like to point out that each member function of a stationary
Czz(T) = E{X(t)X(t + a)X(t + T)X(t + a + T)} - Rh(a) random process has infinite energy and hence its ordinary Fourier transform
does not exist.
Now, making use of the following relationship for a four-dimensional Gaussian We present three forms for expressing random processes in a series form,
distribution (Equation 2.69) starting with the simple Fourier series expansion.

E{X1X2X3X4} = E{X1X2}E{X3X4} + E{XtX3}E{X2X 4}

+ E{X1X4}E{X2X3}
3.9.1 Ordinary Fourier Series Expansion
we have A stationary random process that is MS periodic and MS continuous can be
expanded in a Fourier series of the form
Czz(T) = R_h(a) + Rh(T) + Rxx(T + a)Rxx(T - a) - R_h(a)
N

X(t) = 2.: Cx(nfo)exp(j2-rrnf0 t); (3.83.a)

and n=-N

0::5 V 1
::5- IT \Rh(T)\ dT 1
+ -T IT \Rxx(T + a)Rxx(T -a)\ dT
where
T -T -T

Rxx(O) {.!IT \Rxx(T)\ dT \Rxx(T + a)\ dT} 1

Cx(nfo) = -T IT/2 X(t)exp(- j2-rrnf t) dt (3.83.b)
::5
-T/2
0
T -T -T
 ,r
186 RANDOM PROCESSES AND SEQUENCES SPECTRAL DECOMPOSITION AND SERIES EXPANSION 187

and Tis the period of the process and / 0 = liT. X(t) converges to X(t) in a 3.9.2 Modified Fourier Series for Aperiodic Random Signals
MS sense, that is,
A stationary MS continuous aperiodic random process X(r) can be expanded in
a series form as
lim E{IX(t) - X(t)j 2 } = 0
N

for all values oft E ( -ro, ro). X(t) = L

n= -N
C x(nf0 )exp(j2-rrnfot), ltl <<-
1
fo
(3.85)
Note that the coefficients Cx(nf0 ) of the Fourier series are complex-valued
random variables. For each member function of the random process these ran-
dom variables have a particular set of values. where
The reader can easily verify the following:

sin(-rrfot)
J
oo
1. E{ Cx(nfo) Ck(mf0 )} = 0, n ""m; Cx(nf0 ) = X(t) exp( -j2rmf0 t) dt (3.86)
-oo Tif
that is, the coefficients are orthogonal

2. Rxx(T) = L
n= -oo
cxnexp(j2-rrnfoT), where cxn = E{ICx(nfo)l 2 }
The constants N and fo are chosen to yield an acceptable of normalized
MS error defined in Equation 3.84. As N--? w and fo--? 0, X(t) converges in
MS sense to X(t) for all values of ltl << llf0 • It can be shown that this series
3. E{IX(t)l 2 } = 2: E{ICx(nf0 )j2} (Parseval'stheorem) representation has the following properties:
n= -:Jo

1. E{ Cx(nf 0)C}(mfu)} = 0, m "" n

The rate of convergence or how many terms should be included in the series
expansion in order to provide an "accurate" representation of X(t) cpn be 2. E{IX(t)l 2 } = E{IX(t)l 2 } as n--? oo
determined as follows: The MS difference between X(t) and the series X(t) is (n+l/2)/0
given by 3. E{ICx(nfo)l 2 } =
J (n-112)fo
SxxU) df

N
2
E{IX(t) - X(t)l
2
} = I
E{ X(t) - Cx(nf0 )exp(j2-rrnf0 t)l }
4. [Link](f) = L E{ICx(nfo)l 2}8(f - nfo)
n= -N

= E{IX(t)IZ}- L
n= -N
E{iCx(nfo)iZ}

and the normalized MS error, which is defined as Sxx<fl cos (2 ·df- 2fo)t)

/
2 - E{IX(t) - X(t)IZ}
EN - E{IX(t)l 2} (3.84)

can be used to measure the rate of convergence and the accuracy of the series
representation as a function of N. As a rule of thumb, is chosen to have a
value less than 0.05, which implies that the series representation accounts for -2fo -fo 0 fo 2{0 f
95% of the normalized MS variation of X(t). Figure 3.25 Error in the Fourier series approximation.
 188 RANDOM PROCESSES AND SEQUENCES SAMPLING AND QUANTIZATION OF RANDOM SIGNALS 189

5. lim E{IX(t) - X(t)l 2 } The K-L series expansion has the following properties: ln
hl
"' (<• + 1/2)!0
= 2 Sxx(f)[l - 27rt(f - nfo)] df 1. l.i.m. X(t) = X(t) It!

r,
J(n-ll )fo COS
2
2 m = n I!!
:s 4 E{X2 (t)}sin 2 ( 'IT;ot) for ltl <
2.
-T/2
<!>n(t)<j>;;'.(t) dt = {
m
lU
m=n
6. For any finite value of N, the MS error is the shaded area shown in 3. E{A.A;;'.} =
Figure 3.25.
lu
4. E{X 2 (t)} = Rxx(O) = L"' An
The proofs of some of these statements are rather lengthy and the reader is n=l

referred to Section 13-2 of the first edition of [9] for details. "'
L An
5. Normalized MSE =

3.9.3 Karhunen-Loeve (K·L) Series Expansion

L An Itt

The normalized mean_ squared error E{IX(t) - X(t)]Z} between X(t) and its
series representation X(t) depends on the number of terms in the series and the The main difficulty in the use of Karhunen-Loeve expansion lies in finding the
(basis) functions used in the series expansion. A series expansion is said to be eigenfunctions of the random process. While much progress has been made in Iii
optimum in a MS sense if it yields the smallest MS error for a given number of developing computational algorithms for solving integral equations of the type
It!
terms. The K-L expansion is optimum in a MS sense for expanding a stationary given in Equation 3.87, the computational burden is still a limiting factor in the
random process X(t) over any finite time interval [- T/2, T/2]. application of the K-L series expansion. Ill
The orthonormal basis function, <!>;(t), used in the K-L expansion are ob-
tained from the solutions of the integral equation Ill
ll!
3.10 SAMPLING AND QUANTIZATION OF RANDOM SIGNALS
T/2
. It!
J
-T/2
Rxx(t- T)<j>(T) dT = A<j>(t), ltl < !_
2
(3.87) Information-bearing random signals such as the output of a microphone, a TV
camera, or a pressure or temperature sensor are predominantly analog (contin-
fit
uous-time, continuous-amplitude) in nature. These signals are often transmitted
The solution yields a set of eigenvalues A1 > A2 > A3 , • • • , and eigenfunctions over digital transmission facilities and are also processed digitally. To make these Ill
<l> 1(t), <l> 2(t), <l> 3(t), ... , and the K-L expansion is written in terms of the analog signals suitable for digital transmission and processing, we make use of
eigenfunctions as two operations: sampling and quantization. The sampling operation is used to !JI
convert a continuous-time signal to a discrete-time sequence. The quantizing
li!
operation converts a continuous-amplitude signal to a discrete-amplitude signal.
N In this section, we will discuss techniques for sampling and quantizing a
X(t) = L A.<!>.(t), ltl <
T
2 (3.88) continuous-amplitude, continuous-time signalX(t). We will first show that, given
n=l
the values of X(t) at t = kT., k = · · · -3, -2, -1, 0, 1, 2, 3, ···,we can
reconstruct the signal X(t) for a11 values of t if X(t) is a stationary random %i
where process with a bandwidth of B and Ts is chosen to be smaller than 1/2B. Then
we will develop procedures for representing the analog amplitude of X(kTs) by
a finite set of precomputed values. This operation amounts to approximating a
An =
IT/2
X(t)<l>:(t) dt, n = 1, 2, · · · N (3.89) continuous random variable X by a discrete random variable Xq, which can take
-T/2 on one of Q possible values such that E{(X -Xq) 2 } . _ 0 as Q ._co. J

.;t
I -.
r
190 RANDOM PROCESSES AND SEQUENCES
SAMPLING AND QUANTIZATION OF RANDOM SIGNALS 191
Sxx(fl

2L JB' -B'
1 JB
280 _: 0 SxxUz)exp(- }21TnizTs) diz
X exp[j21Tj 1(T + nTs)] di1
£ Rxx( -nTs) 2Ba
n= -x
_1_ [sin 21TB'(T + nTs)]
1r(T + nTs)
0 f
B' + f,/2
1!(2T8) l/(2T.) If we choose the limits of integration B' to be equal to B, we have
Figure 3.26 Power spectral density of the signal being sampled.
2:
X

Rxx(T) = 2BTs Rxx(nTs) sin 21TB(T - nTs) {3.92)

n= -X 21T B( T - nTs)
3.10.1 Sampling of Lowpass Random Signals
It is convenient to state two other versions of Equation 3.92 for use in deriving
Let X(t) be a real-valued stationary random process with a continuous power the sampling theorem for lowpass random signals. With a an arbitrary constant,
spectral density function, SxxU), that is zero for Iii > B (Figure 3.26). Since the transform of Rxx(T - a) is equal to SxxU)exp(- j2rria). This function is
SxxU) is a real function of i, we can use an ordinary Fourier series to represent also lowpass, and hence Equation 3.92 can be applied to Rxx(T - a) as
Sxx(f) as

Rxx(T - a) = 2BTs L Rxx(nTs - a)sinc 2B(T - nTs) (3.93)

Sxx(f) 2:
n= -:>:J
Cx(nTs)exp(j21TniTs), Iii < Ba (3.90)
n= -cc

where
where

sine x = sin 1TX

1 1TX
Ba = 2T/ Bo > B
Changing (T - a) toT in Equation 3.93, we have
and

Rxx(T) = 2BTs L Rxx(nTs - a)sinc 2B(T + a - nTs) (3.94)

Cx(nTs) = 28
1 JBo Sxx(f)exp(- j2rrniTs) df
n= -x

0 -8 0 (3.91)
We now state and prove the sampling theorem for band-limited random pro-
Taking the inverse Fourier transform of Sxx(f) as given in Equation 3.90, we cesses.
have
The Uniform Sampling Theorem for Band-limited Random Signals. If a real
random process X(t) is band-limited to B Hz, then X(t) can be represented
Rxx(-r) = p-l Cx(nTs)exp(j2rrniTs)} .using the instantaneous values X(kTs) as

B' oo

= J-B' Cx(nTs)exp(j21Tni1T5 )exp(j2rri1T) B::::; B'::::; Bo XN(t) = 2BTs

N
L
n= -N
X(nTs)sinc[2B(t - nT5 )], Ts < 1/(2B) (3.95)

J
192 RANDOM PROCESSES AND SEQUENCES SAMPLING AND QUANTIZATION OF RANDOM SIGNALS 193

and XN(t) converges to X(t) in a MS sense. That is E{[X(t) - XN(t)]Z} = 0, Now

as N- oo.
To prove MS convergence of XN(t) to X(t), we need to show that
E{[X(t) - X(t)]X(mTs)}

E{[X(t) - XN(t)]Z} = 0 as N- oo (3.96) = Rxx(t - mTs) - 2:

n=-oc
2BTsRxx(nTs - mTs)sinc[2B(t - nTs)]

Let N- oo, then

and from Equation 3.93 with T = t and a = mT., we have

X(t) = 2BTs 2:
n=
X(nTs)sinc[2B(t - nTs)], Ts < 2B
1

2:
-co
Rxx(t- mTs) = 2BTs Rxx(nTs - mTs)sinc[2B(t- nTs)]
n= - x
Now

Hence
E{[X(t) - X(t)]Z} = E{[X(t) - X(t)]X(t)}
- E{[X(t) - X(t)]X(t)} (3.97) E{[X(t) - X(t)]X(t)} = 0 (3.99)

The first term on the right-hand side of the previous equation may be written
as Substitution of Equations 3.98 and 3.99 in Equation 3.97 completes the proof
of the uniform sampling theorem.
E{[X(t) - X(t)]X(t)} The sampling theorem permits us to store, transmit, and process the sequence
®
X(nTs) rather than the continuous time signal X(t), as long as the samples are
= Rxx(O) - 2BTs 2:
n= - x
Rxx(nTs - t)sinc[2B(t - nTs)] taken at intervals Jess than 1/(28). The minimum sampling rate is 28 and is
called the Nyquist rate. If X(t) is sampled at rates lower than 2B samples/second,
then we cannot reconstruct X(t) from X(nTs) due to "aliasing," which is ex-
plained next.
From Equation 3.94 with T = 0 and a = t, we have

Aliasing Effect. To examine the aliasing effect, let us define the sampling
2BTs 2:
n= -oo
Rxx(nTs - t)sinc[2B(t- nTs)] = Rxx(O) operation as

Xs(t) = X(t) · S(t)

and hence
where Xs(t) is the sampled version of a band-limited process X(t) and S(t) is
E{[X(t) - X(t)]X(t)} = 0 (3.98) the sampling waveform. Assume that the sampling waveform S(t) is an impulse
sequence (see Figure 3.27) of the form

The second term in Equation 3.97 can be written as

S(t) = 2: ll(t - kT, - D)
k=-00
E{[X(t) - X(t)]X(t)}
00

= 2: E{[X(t) - X(t)]X(mTs)}2BTs sinc[2B(t - mTs)] where Dis a random variable with a uniform distribution in the interval [0, Ts],
m=-::o and D is independent of X(t). The product Xs(t) = X(t) · S(t) as shown in
 r

194 RANDOM PROCESSES AND SEQUENCES SAMPLING AND QUANTIZATION OF RANDOM SIGNALS 195

(a)
Following the derivation in Section 3.6.5, the reader can show that the auto-
.rorrclation function of X,(t) is [Link] by

"' 1
Rx,x.('r) = 'rs Rxx(k'rs)'&(T - k'rs)

1
= T Rxx(T)
s
L"'
k=-00
8(T - kT,)

& ... '

:·:TfTfTion:i:fTfl The last step results from one of the properties of delta functions. Taking the
Fourier transform of Rx,x,(T) we obtain

(c) Sx,x,(f) = SxxU) * F 'O(T - k'rs)}

X,(t) X(t)S(t)

The reader can show that

/

F 'O{f - k'rs)} r1 2: au- kf,)

s k= -oo

(d)

where J, = liT, is the sampling rate, and hence

-B 0 B
f
Sx,x,(f) = ; ; SxxU - kf,)}
(e)

1
= T 2 {SxxU) + SxxU - f,) + SxxU + f,)
s

- {, f + SxxU - 2f,) + Sxx(f + 2f,) + ... } (3.100)

(/)
The preceding equation shows that the psd of the sampled version X,(t) of X(t)
consists of replicates of the original spectrum SxxU) with a replication rate of
f,. For a band-limited process X(t), the psd of X(t) and X,(t) are shown in
c J n" c '> t / '\ /l '\ I / '\ I / '\ f
Figure 3.27 for two sampling rates f, > 2B and f, < 2B.
Aliasing When f, > 2B or T, < 11(2B), Sx,x,(f) contains the original spectrum of
Figure 3.27 Sampling operation. X(t) .intact .and recovery of X(t) from X,(t) is possible. But when f, < 2B,
replicates of SxxU) overlap and the psd of X,(t) does not bear much resemblance
to·the psd of X(t). This is called the aliasing effect, and it often prevents us from
reconstructing X(t) from X,(t) with the required accuracy.
Figure 3.27c can be written as When f, > (2B), we have shown that X(t) can be reconstructed in the time
domain from samples of X(t) according to Equation 3.95. Examination of Figure
3.27e shows that if we select only that portion of Sx,x,U) that lies in the interval
X,(t) = L X(t - k'rs - D)'O(t - k'rs - D) [- B, B], we can recover the psd of X(t). This selection can be accomplished
k= -oo
in the frequency domain by an operation known as "lowpass filtering," which
196 RANDOM PROCESSES AND SEQUENCES SAMPLING AND QUANTIZATION OF RANDOM SIGNALS 197

will be discussed in Chapter 4. Indeed, Equation 3.95 is the time domain equiv- m7
Actual value
of the signal
alent of lowpass filtering in the frequency domain.
%6
----------\- Quantized value
m6 --of the signal-
3.10.2 Quantization
The instantaneous value of a continuous amplitude (analog) random process xs
X(t) is a continuous random variable. If the instantaneous values are to be
processed digitally, then the continuous random variable X, which can have an ms
uncountably infinite number of possible values, has to be represented by a
discrete random variable with a finite number of values. For example, if the "'•
instantaneous value is sampled by a 4-bit analog-to-digital converter, then X is
approximated at the output by a discrete random variable with one of 24 possible m4

values. We now develop procedures for quantizing or approximating a contin- X3 .._--'

uous random variable X by a discrete random variable Xq. The device that
performs this operation is referred to as a quantizer or analog-to-digital con- m3/:, > <6 I
verter.
An example of the quantizing operation is shown in Figure 3.28. The input xz
to the quantizer is a random process X(t), and we will assume that the random
signal X(t) is sampled at an appropriate rate and the sample values X(kTs) are mz
converted to one of Q allowable levels, m 2 , • • • , mQ, according to some
predetermined rule: x,
Xq(kT,J = rn; if Xi-1 < X(kTs) :S X;
m,
x0 = -oo, XQ = +co (3.101) Figure 3.28 Quantizing operation; m, ... , m1 are the seven output levels of the
quantizer.
The output of the quantizer is a sequence of levels, shown in Figure 3.28 as a
waveform Xq(t), where
mean, stationary random process with a pdf fx(x). We will use the abbreviated
Xq(t) = Xq(kTs), kT. :S t < (k + 1) T. notation, X to denote X(kTs) and Xq to denote Xq(kTs). The problem of quan-
tizing consists of approximating the continuous random variable X by a discrete
random variable Xq such that E{(X - Xq) 2} is minimized.
We see from Figure 3.28 that the quantized signal is an approximation to the
original signal. The quality of the approximation can be improved by increasing
the number of quantizing levels Q and for fixed Q by a careful choice of x;'s
and m/s such that some measure of performance is optimized. The measure of 3.10.3 Uniform Quantizing r' '·
performance that is most commonly used for evaluating the performance of a In this method of quantizing, the range of the continuous random variable X is
quantizing scheme is the normalized MS error - divided into Q intervals ()f equal length, say A. If the value of X falls in the ith
:,
quantizing interval, then the quantized value of X is taken to be the midpoint
_ E{[X(k'F.) - Xq(kT.)F} of the interval (see Figure 3.29). If a and b are the minimum and maximum
2
EQ - values of X, respectively, then the step size or interval length A is given by

We will now consider several methods of quantizing the sampled values of A = (b - a) (3.102.a)
a random process X(t). For convenience, we will assume X(t) to be a zero- Q
 rI

198 RANDOM PROCESSES AND SEQUENCES

SAMPLING AND QUANTIZATION OF RANDOM S(GNALS 199

The ratio NQ/SQ is and it gives us a measure of the MS error of the uniform
quantizer. This ratio can be computed if the pdf of X is known.

EXAMPLE 3.23.
a b
xo m1 Xj m2 X2 m3 x3 m• x. The input to a Q-step uniform quantizer has a(uniform pdfover the interval
[-a, a]. Calculate the normalized MS error as a function of the number of
Figure 3.29 Example of uniform quantizing. Step size = A, Q = 4. quantizer levels.
';·
SOLUTION: From Equation 3.103.a we have
The quantized output Xq is generated according to

Xq = m; if Xi- I <X:::; X;, l = 1, 2, ... , Q (3.102.b)

NQ = LQ JX·'
t=I
(x - mY ( 2 1)
a
dx !
d) --1
x 1_ 1

LQ J-a+iA (x
2

where = + a - id +- dx
i=l -a+(i-l)A 2 2a ,,

X;= a + id (3.102.c) =
Qd3 d2
and = (2a)12 = 12 since Qd = 2a

m; =X;_, + x; Now, the output signal power S0 can be obtained using Equation 3.103.b as
2 (3.102.d)

The quantizing "noise power" NQ for the uniform quantizer is given by sQ = f

i-=1
(mY
1'

Qz - 1 (A)z
N 0 = E{(X - Xq) 2} 12

= f (x - xq) 2fx(x) dx
and hence the normalized MS error is given by

0 Jx (x - m;ffx(x) dx (3.103.a) !!s;__ 1 -1

SQ - Qz- 1- Qz when Q >> 1 (3.104)

where x 1 = a + id and m 1 = a + id - d/2. The "signal power" SQ at the

output of the quantizer can be obtained from

Equation 3.104 can be used to determine the number of quantizer levels needed
SQ = E{(Xq)2}
for a given application. In quantizing audio and video signals the ratio N ! S
0 0
is kept lower than 10-4, which requires that Q be greater than 100. It is a common
=
Q
(mY JX·x,·. fx(x) dx (3.103.b) practice to use 7-bit AID converters (128 levels) to quantize voice and video

•
I
 r
!
j,

!It(
;

200 RANDOM PROCESSES AND SEQUENCES SAMPLING AND QUANTIZA TJON OF RANDOM SIGNALS 201
111
equal to zero: Ill
l!!'r'
aNQ Ill
(xi - mi) 2fx(x) - (xi - mi+1?fx(xi) = 0
axi
j = 1, 2, ... ' Q - 1 (3.106.a) Ill
3 I• t.2=--.........,...j....,_ ""IE t.-8-
!If
aNQ = _ 2 J'x; (x - m)fx(x) dx 0, j = 1, 2, ... , Q (3.106.b)
ami
x7 m 8
1u
'
·
xj-1
ml xl m2 x2 m 3 x3 m 4 x 4 ms xs m6 X6 m7

Figure 3.30 A nonuniform quantizer for a Gaussian variable. X 0 = -oo, XQ = oo,

Q = 8, and 6., = D.Q+l-i• (i = 1, 2, 3, 4). From Equation 3.106.a we obtain
111
1
lu
xi = Z(mi + mi+ 1)
3.10.4 Nonuniform Quantizing
The uniform quantizer is optimum (yields the lowest NQ/SQ for a given value
of Q) if the random process X(t) has uniform amplitude distribution. If the pdf or )H
is nonuniform, then the quantizer step size should be variable, with smaller step
sizes near the mode of the pdf and larger step sizes near the tails of the pdf. mi = 2xi- 1 j = 2, 3, ... , Q (3.107.a)
An example of nonuniform quantizing is shown in Figure 3.30. The input to the
- mi-1•
)n
quantizer is a Gaussian random variable and the quantizer output is determined
according to Equation 3.106.b reduces to

Xq = m;
x 0 = -oo,
if X;-l

xQ = oo
<X ::s X;, l = 1, 2, ... , Q
(3.105)
J"x,_ I
(x - mi)fx(x) dx
,
0, j = 1, 2, ... ' Q (3.107.b)

The step size 41; = X; - X;_ 1 is variable. The quantizer end points x;'s and the which implies that mi is the centroid (or mean) of the jth quantizer interval.
output levels m;'s are chosen to minimize NQISQ. The foregoing set of simultaneous equations cannot be solved in closed form
The design of an optimum nonuniform quantizer can be approached as fol- for an arbitrary fx(x). For a specific fx(x), a method of solving Equations 3.107.a
lows. We are given a continuous random variable X with a pdf f x(x). We want and 3.107.b is to pick m 1 and calculate the succeeding x;'s and m;'s using Equa-
to approximate X by a discrete random variable Xq according to Equation 3.105. tions 3.107.a and 3.107.b. If m 1 is chosen correctly, then at the end of the
The quantizing intervals and the levels are to be chosen such that NQ is mini- iteration, mQ will be the mean of the interval [xQ_ 1 , oo].lf mQ is not the centroid
mized. This minimizing can be done as follows. We start with or the mean of the Qth interval, then a different choice of m 1 is made and the
procedure is repeated until a suitable set of :r;'s and m;'s is reached. A computer
program to solve for the quantizing intervals and the means by this iterative
NQ LQ Jx ' (x - mYfx(x) dx, Xo -oo and xQ = oo method can be written.
1= 1 x1_ 1
Quantizer for a Gaussian Random Variable. The end points of the quantizer
intervals and the output levels for a Gaussian random variable have been com-
Since we wish to minimize NQ for a fixed Q, we get the necessary* conditions puted by J. Max [ 15]. Attempts have also been made to determine the functional
by differentiating NQ with respect to the x/s and m/s and setting the derivatives dependence of NQ on the number of levels Q. For a Gaussian random variable
with a variance of 1, Max has found that N Q is related to Q by
*After finding all the x,'s and m;'s that satisfy the necessary conditions, we may evaluate NQ at these
points to find a set of x,'s and m!s that yield the absolute minimum value of NQ. In most practical
cases we will get a unique solution for Equations 3.106.a and 3.106.b. NQ = (2.2)Q- 196 , when Q >> 1
 r
202 RANDOM PROCESSES AND SEQUENCES
REFERENCES 203

If the variance is <Ti-, then the preceding expression becomes width calculations, which are patterned after deterministic signal definitions,
were introduced.
NQ = (2.2)<Ti-Q- L96 (3.108)
The concepts of continuity, differentiation, and integration were introduced
for random processes. If all member functions of the ensemble have one of
Now if we assume X to have zero mean, then SQ = E{X2} = <Ti, and hence these three properties, then the random process has that property. In addi-
tion, these properties were defined in the mean-square sense as they apply to
stationary (WSS) processes. It was shown that this extends these important
E(>-
N
' - _g
SQ = 2. 2Q-1.96 (3.109) operations to a wider class of random signals.
The time average of a random process or a function, for example (X(t)
Equation 3.109 can be used to determine the number of quantizer levels needed f.L)Z, of a random process is a random variable. This time average will have a
to achieve a given normalized mean-squared error for a zero-mean Gaussian mean and a variance. For stationary processes, it was shown that the mean of
random process. the time average equals the ensemble mean. In order for the time average to
equal the ensemble average, it was shown that it is necessary for the variance
of the time average to be zero. When this is the case, the stationary process is
called ergodic. Various definitions of ergodicity were given.
Series expansions of random processes were introduced. Fourier series and a
3.11 SUMMARY modified Fourier series were presented, and the Karhunen-Loeve series ex-
In this chapter, we introduced the concept of random processes, which may pansion, which is optimum in the MS sense for a specified number of terms,
be viewed as an extension of the concept of random variables. A random was introduced.
process maps outcomes of a random experiment to functions of time and is a
The sampling theorem for a random process band-limited to B Hz was
useful model for both signals and noise. For many engineering applications, a
proved. It shows that if the sampling rate j, is greater than 28, then samples
random process can be characterized by first-order and second-order proba-
X(nTJ can be used to reproduce, in the MS sense, the original process. Such
bility distribution functions, or perhaps just the mean and variance and auto-
sampling often requires quantization, which was introduced and analyzed in
correlation function. For stationary random processes, the mean and
Section 3.10. The mean-square error and normalized mean-square error were
autocorrelation functions are often used to describe the time domain structure
suggested as measures of performance for quantizers.
of the process in an average or ensemble sense. The Fourier transform of the
autocorrelation function, called the power spectral density function, provides
a frequency domain description of the random processes.
3.U REFERENCES
Markov, independent increments, Martingale, and Gaussian random pro-
cesses were defined. The random walk; its limiting version, the Wiener pro-
A number of texts are available to the interested reader who needs additional material
cess; the Poisson process; and the random binary waveform were introduced
on the topics discussed in this chapter. Background material on deterministic signal
as important examples of random processes, and their mean and autocorrela- processing may be found in Referen.:es (7] and (10]. Introductory treatment of the material
tion functions were found. of this chapter may be found in Cooper and McGillem (1], Gardner (4], Helstrom [5],
Peebles (11], O'Fiynn (12], and Schwartz and Shaw [ 13], and a slightly higher level
Different types of stationarity were defined and wide-sense stationarity (weak treatment i'> contained in Papoulis ..[9J. Davenport and .Root .[3] is the classical book in
stationarity) was emphasized because of its importance in applications. The this area, whereas Doob [2] is a primary reference in this field from the mathematical
properties of the autocorrelation and the cross-correlation functions of real perspective. Advanced material on random processes may be found in texts by Larson
wide-sense stationary (WSS) processes were presented. The Fourier trans- and Shubert [6], and Wong and Hajek (14], and Mohanty (8].
forms of these functions are called the power spectral density function and [1] G. R. Cooper and C. D. McGillem, Probabilistic Methods of Signal and System
cross-power density function, respectively. The Fourier transform was used to Analysis, 2nd ed., Holt, Rinehart, and Winston, New York, 1986.
define the spectral density function of random sequences. Power and band- (2] J. L. Doob, Stochastic Processes, John Wiley & Sons, New York, 1953.
204 RANDOM PROCESSES AND SEQUENCES PROBLEMS 205

Y(t)
[3] W. B. Davenport, Jr. and W. L. Root, Introduction to Random Signals and Noise, X(t)

McGraw-Hill, New York, 1958.

Y5(t) =t
[4] W. A. Gardner, Introduction to Random Processes: With Applications to Signals I
and Systems, Macmillan, New York, 1986. x3'(t)

0
[5] C. W. Helstrom, Probability and Stochastic Processes for Engineers, Macmillan, X2(1)

New York, 1984. -1

x1(t)

[6] H. J. Larson and B. 0. Shubert, Probabilistic Models in Engineering and Science,

Yl(t) = -t
Vols. I and II, John Wiley & Sons, New York, 1979.
[7] C. D. McGillem and G. R. Cooper, Continuous and Discrete Signal and System Figure 3.31 Member functions of X(t) and Y(t).
Analysis, 2nd ed., Holt, Rinehart, and Winston, New York, 1984.
[8] N. Mohanty, Random Signals, Estimation and Identification, Van Nostrand, New
York, 1986.
3.2 The member functions of two random processes X(t) and Y(t) are shown
[9] A. Papoulis, Probability, Random Variables and Stochastic Processes, McGraw- in Figure 3.31. Assume that the member functions have equal probabilities
Hill, New York, 1965, 1984.
of occurrence.
[10] A. Papoulis, Signal Analysis, McGraw-Hill, New York, 1977.
a. Find J-Lx(t), and Rxx(t, t + T). Is X(t) WSS?
[11] P. J. Peebles, Probability, Random Variables and Random Signal Principles, 2nd
ed., McGraw-Hill, New York, 1986. b. Find J-Ly(t), and Ryy(t, t + T). Is Y(t) WSS?
[12] M. O'Flynn, Probabilities, Random Variables and Random Processes, Harper and c. Find RXY(O, 1) assuming that the underlying random experiments
Row, New York, 1982 .. are independent.
[13] M. Schwartz and L. Shaw, Signal Processing: Discrete Spectral Analysis, Detection
and Estimation, McGraw-Hill, New York, 1975. 3.3 X(t) is a Gaussian random process with mean J-Lx(t) and autocorrelation
function Rxx(t 1, t2). Find E{X(t2)1X(tl)}, t1 < t2.
[14] E. Wong and B. Hajek, Stochastic Processes in Engineering Systems, Springer-
Verlag, New York, 1971, 1985.
3.4 Using the Markov property, show that if X(n), n = 1, 2, 3, ... , is
[15] Max, J., "Quantizing for Minimum Distortion," IRE Transaction on Information Markov, then
Theory, Vol. IT-6, 1960, pp. 7-12.
E{X(n + l)IX(l), X(2), ... , X(n)} = E{X(n + 1)IX(n)}

3.5 For a Markov process X(t) show that, fort> t 1 > t0 , ,:j
3.14 PROBLEMS
fx(r)IX(t 0 JCxlxo) = fx(r)IX<dxlxt)fx(r 1)1X(t0 )(xdxo) dxl .:1
3.1 Define a random process X(t) based on the outcome k of tossing a die as
(The preceding equation is called the Chapman-Kolmogoroff equation.)
-2 k =1
-1 k = 2, 3.6 Show that the Wiener process is a Martingale.
X(t) = 1 k =3
2 k =4 3.7 Consider the random walk discussed in Section 3.4.2. Assuming d = 1,
k =5 and T = 1, find
-t k =6
a. P[X(2) = 0]
a. Find the joint probability mass function of X(O) and X(2). b. P[X(8) = OIX(6) = 2]
b. Find the marginal probability mass functions of X(O) and X(2). c. E{X(lO)}
c. Find E{X(O)}, E{X(2)}, and E{X(O)X(2)}. d. E{X(10)IX(4) = 4}
,-
206 RANDOM PROCESSES AND SEQUENCES
T PROBLEMS 207
'
I
I

i
3.8 A symmetric Bernoulli random walk is defined by the sequence S(n) as 3.13 X(t) and Y(t) are independent WSS random processes with zero means. if

S(n) =
n
L X(k),
k=l
X(O) = 0, n = 1, 2, 3, ... I Find the autocorrelation function of Z(t) when
a. Z(t) = a + bX(t) + cY(t)
!I,,
I!
!I
II
b. Z(t) = aX(t)Y(t) I j
where X(n), n = 1, 2, 3, ... is a sequence of independent and identically
11
distributed (i.i.d) Bernoulli random variables with 3.14 X(t) is a WSS process and let Y(t) = X(t + a) - X(t - a). Show that I
!i

P[X(n) = 1] = P[X(n) =
1
-1] = -
2
a. Ryy(T) = 2Rxx(T) - Rxx(T + 2a) - Rxx(T - 2a)
2
I!d
b. Syy(f) = 4Sxx(f)sin (21Taf) ,.,J
a. Show that S(n) is a Martingale sequence.
i
b. Show that Z(n) = S 2(n) - n is also a Martingale. 3.15 Determine whether the following functions can be the autocorrelation
functions of real-valued WSS random processes:
I[ I
3.9 LetX(1),X(2), ... ,X(n), .. . [Link] a. (1 + 2T2)-l
variables with a pdf fx(x). Define Y(n) as
n
b. 2 sin 21T(1000)T 1·1
Y(n) L X(k), n = 1, 2, 3, ... sin 27TfoT
l1l

Il!jn
=
k=l c. fo > 0 l'
foT
a. Show that Y(n) is a Markov sequence and a Martingale. I'
d. O(T) + COS 21Tj0T
b. Show that rl
3.16 Determine whether the following functions can be power spectral density
il
h.Y,, ... ,Y/Yt. J2, · · · , Yn) = fx(Yt)ix(Y2 - Yt) · · · fx(Yn - Yn-t) functions of real-valued WSS random processes.
II
II
c. Find the conditional pdf [Link]._,. a. (1 + lOf)-112 1lj
3.10 Let N(t), t 2: 0 be the Poisson process with parameter A., and define b sin lOOOJ.
lr'i'•iS
i!
if N(t) is odd . 1000/ l!
X(t) = { 1 if N(t) is even c. 50 + 208(! - 1000) IlIa;i
X(t) is called a random telegraph signal. d. 10o(f) + 5o(f + 500) + so(f - 500)
'11,:1
a. Show that X(t) has the Markov property.
'

e. exp( -2001Tj2)
b. Find [Link](t) and RxxCtt, lz). f . .·
'I'
f. CP + 100) li
3.11 X(t) is a real WSS random process with an autocorrelation function Rxx(T). II'4'
Prove the following: 3.17 For each of the autocorrelation functions below, find the power spectral P'
111,
density function. 'iii
a. If X(t) has periodic components, then Rxx(T) will also have pe- I····
riodic components. a. exp( - ajTj), a>O !i!/i
b. If Rxx(O) < oo, and if Rxx(T) is continuous at T = 0, then it is !i
b. sin 1000 T
continuous for every T. 'i"'
1000 T
3.U X(t) and Y(t) are real random processes that are jointly WSS. Prove the
l il ·
following: c. exp( -IT I) [cos T + sin IT I]
'
'iii
a. IRXY(T)J :s .JRxx(O)Ryy(O) d. exp( -10- 2f6T 2) ' 't
H!
b. RXY(T) :s HRxx(O) + Ryy(O)] e. cos(l000T) i:l
':I:
.,1,.
,).1
!ri
I
208 RANDOM PROCESSES AND SEQUENCES PROBLEMS 209

S xx(fl 100/[ l + (2,.. f/100) 2]2

3.18 For each of the power spectral density functions given below, find the SxxU) 10 exp
autocorrelation function.
a. (40TI2j2 + 35)/[(41T2j2 +
9)(41T2j2 +
4)]
b. 11(1+ 41T j2) 2 2

c. lOOo(f) + 2a/(a2 + 4TI 2j2)

------'0---- f 0 f
3.1'1 X(n), n = ···, -1, 0, 1, ... is a real discrete-time, zero-mean, WSS Figure 3.33 Psd functions for Problem 3.23 and 3.44.
sequence. Find the power spectral density function for each of the following
cases.
a. X(n) is a sequence of i.i.d. random variables with unit variance.
3.22 For a wide-sense stationary random process, show that
b. X(n) is a discrete time Markov sequence with Rxx(m) =
exp(- almJ). a. Rxx(O) = area under Sxx(f).

c. X(n) is a sequence with Rxx(O) = 1, Rxx(± 1) = -1/2 and b. Sxx(O) = area under Rxx(T).
Rxx(k) = 0 for Jkl > 1.
3.23 For the random process X(t) with the psd's shown in Figure 3.33, deter-
3.20 The psd of a WSS random process X(t) is shown in Figure 3.32. mine
a. Find the power in the DC term. a. The effective bandwidth, and

b. Find £{X 2(t)}. b. The rms bandwidth which is defined as

c. Find the power in the frequency range [0, 100Hz]. r'" fSxxU) df
3.21 Let X and Y be independent Gaussian random variables with zero-mean s;m, = SxxU) df
and unit variance. Define
Z(t) = X cos 21T(lOOO)t + Y sin 2TI(1000)t [Note: The rms bandwidth exists only if S xxU) decays faster than 11 f]
a. Show that Z(t) is a Gaussian random process.
3.24 For bandpass processes, the rms bandwidth is defined as
b. Find the joint pdf of Z(t 1) and Z(t 2 ).
4 ("' (f - fo) 2Sxx(f) df
c.
d.
e.
Is the process WSS?
Is the process SSS?
Find E{Z(t 2 )IZ(t 1)}, t2 > t 1•
mms =
Jo
rs xx(f) df

where the mean or center frequency fu is defined as

J: f S xxU) df

L
fo=-'"---

&,
100 b ({)
Sxx(f) df

Find the rms bandwidth of

1+e [1 + e
A A
A, B, fo > 0
-1000 0 1000
SxxU) = [
Figure 3.32 Psd of X(t) for Problem 3.20.
210 RANDOM PROCESSES AND SEQUENCES
r PROBLEMS 211

Sxx<fl Syy({) 3.27 A WSS random process X(t) has a mean of 2 volts, a periodic component
A
XP(t), and a random component X,(t); that is, X(t) = 2 + Xp(t) + X,(t).
[Link] function of X(t) is given in Figure 3.35.
'
ar2 a. What is the average power in the periodic component?
By<< Bx
b. What is the average power in the random component?
f
-Bx 0 Bx
Figure 3.34 Psd functions for Problem 3.26. 3.28 A stationary zero-mean random process X(t) has an autocorrelation func-
tion
RxxCr) = 10 exp( -0.1T 2)
a. Find the autocorrelation function of X'(t) if X'(t) exists.
3.25 X(t) is a complex-valued WSS random process defined as b. Find the mean and variance of
X(t) A exp(27l'jYt + j8)
=
Y = -1 15 X(t) dt
where A, Y and e are independent random variables with the following 5 0

pdfs:
3.29 Show that if a finite variance process is MS differentiable, then it is nec-
fA(a) = a exp( -a 212), a>O
essarily MS continuous.
= 0 elsewhere
1/1000 for 10,000 < y < 11,000 3.30 Show that for a lowpass process with a bandwidth B, the amount of change
fv(Y) { 0 . from t to t + T is bounded by
elsewhere
a. E{!X(t + T) - X(!Jf} ::s + [Link]
for - .. < e< 71'
fo(O) b. R 11(0) - R.11 (T) ::S
2
C2nBT) R 11 (0)/2
elsewhere
3.31 X(t) and Y(t) are two independent WSS processes that are MS continuous.
Find the psd of X(t).
a. Show that the sum X(t) + Y(t) is MS continuous.
3.26 X(t) and Y(t) are two independent WSS random processes with the power
b. Show that the product X(t) Y(t) is also MS continuous.
spectral density functions shown in Figure 3.34. Let Z(t) = X(t)Y(t).
Sketch the psd of Z(t), and find 5 22 (0).
3.32 Show that both MS differentiation and integration obey the following rules
of calculus:
a. Differentiating and integrating linear combinations.
b. Differentiating and integrating products of independent random
processes.

3.33 Show that the sufficient condition for the existence of the MS integral of
a stationary finite variance process X(t) is the existence of the integral

J' f' Rxx(t

to to
1 - lz) dt 1dtz

-1 3
3.34 X(t) is WSS with E{X(t)} = 2 and Rxx(r) = 4 + exp( -ITI/10)
Milliseconds
Figure 3.35 Autocorrelation function for Problem 3.27. a. Find the mean and variance of s= n X(T) dT
I
r
····-----

212 RANDOM PROCESSES AND SEQUENCES PROBLEMS 213

b. How large should T be chosen so that b. Show that

- 21 < 0.1} > 0.95 E{(Sxx(f))r} = SxxU), as T oo

3.35 Let Z(t) = x(t) + Y(t) x(t) is a deterministic, periodic power signal and
with a period T and Y(t) is a zero mean ergodic random process. Find the Var{(Sxx(f))r} 2: [E{(Sxx(f))r}y
autocorrelation function and also the psd function of Z(t) using time av-
erages. 3.41 Define the time-averaged mean and autocorrelation function of a real-
valued stationary random sequence as
3.36 X(t) is a random binary waveform with a bit rate of liT, and let 1 N
Y(t) = X(t)X(t - T/2) = N X(i) LI

a. Show that Y(t) can be written as Y(t) = v(t) + W(t) where v(t) and
is a periodic deterministic signal and W(t) is a random binary waveform

n
of the form 1 N
(Rxx(k))N =N X(i)X(i + k)
for ltl < T/2
L Akp(t -
k
kT - D); p(t) = elsewhere a. Find the mean and variance of (!J.x)N and (Rxx(k))N

b. Find the psd of Y(t) and show that it has discrete frequency spectral b. Derive the condition for the ergodicity of the mean.
components.
3.42 Prove the properties of the Fourier series expansion given in section 3; 9.1
3.37 Consider the problem of estimating the unknown value of a constant signal and 3.9.2. :!
by observing and processing a noisy version of the signal for T seconds.
3.43 Let X = [X 11 X 2 , • • • , XnJT be a random vector with a covariance matrix !
Let X(t) = c + N(t) where cis the unknown signal value (which is assumed
to remain constant), and N(t) is a zero-mean stationary Gaussian random Ix. Let X1 > A2 > · · · > An be the eigenvalues of Ix. Suppose we want
process with a psd SNN(f) = N 0 for lfl < B and zero elsewhere (B >> to approximate X as
11 T). The estimate of c is the time-averaged value - X = A1V1 + AzVz + · · · + Amvm, m <n
I' j

- -
c = -1fT X(t) dt such that E{[X - XJT(X - X]} is minimized.
T o
a. Show that the basis vectors v11 v2 , ••• , Vm are the eigenvectors of
a. Show that E{c} = c. Ix corresponding to A11 Az, ... , Am, respectively.
·Jl

b. Find the value ofT such that P{lc - cl < 0.1c} 2: 0.999. (Express b. Show that the coefficients A; are random variables and that A; =
i
Tin terms of c, B, and N 0 .) xrvi. ·t,

3.38 Give an example of a random process that is WSS but not ergodic in mean. c. Find the mean squared error.

3.39 A stationary zero-mean Gaussian random process X(t) has an autocor- 3.4-l Suppose we want to sample the random processes whose spectral
relation function densities arc shown in Figure 3.33. Find a suitable sampling rate using
the constraint that the ratio of S.u(O) to the aliased spectral component
Rxx(r) = 10 exp( -1-rl) atf = 0 has to be greater than 100.
Show that X(t) is ergodic in the mean and autocorrelation function.
3.45 Show that a WSS bandpass random process can also be represented by
3.40 X(t) is a stationary zero-mean Gaussian random process. sampled values. Establish a relationship between the bandwidth Band the
minimum sampling rate. 'i :
a. Show that
3.46 The probability density function of a random variable X is shown in Figure
Var{(Rxx(-r))r} :s T4 f"' Rh{r) d-r 3.36.
0