2.

5 RANDOM PROCESSEs
To deternmine the probabilities of the various possible outcomes of an experiment, it is necessary to
repeat the experiment many times. Suppose that we are interested in establishing the statistics
associated with the tossing of a die. We might proceed in either of two ways. On one hand, we
might use a single die and toss it repeatedly. Alternatively, we might toss simultaneously a very
large number of dice. Intuitively, we would expect that both methods would give the same results.
Thus, we would expect that a single die would yield a particular outcome, on the average, of
Itime out of 6. Similarly, with many dice we would expect that 1/6 of the dice tossed would yield
a particular outcome.
Analogously, let us consider a random process such as a noise waveform n(t) mentioned at the
beginning of this chapter. To determine the statistics of the noise, we might make repeated mea
surements of the noise voltage output of a single noise source, or we might, at least conceptually,
make simultaneous measurements of the output of a very large collection of statistically identical
noise sources. Such acollection of sources is called an ensemble, and the individual noise wave
forms are called sample functions. A statistical average may be determined from measurements
made
2
at some fixed time t= t on all the sample functions of the ensemble. Thus to determine, say,
n*), we would, at t= t, measure the voltages n(t) of each noise source, square and add the
voltages, and divide by the (large) number of sources in the ensemble. The average so determined
is the ensemble average of n(u).
Now n(t;) is a random variable and will have associated with it a probability density function.
he ensemble averages will be identical with the statistical averages computed earlier in Secs. 2.2.6
and 2.2.7 and may be represented by the same symbols. Thus the statistical or ensemble average of
 determined by measurements on a sinol.
n(4) may be written E<n*r,)] = n(1,). The averages represent as (n't).
Sample function at successive times will vield a time average, which we
same. Suppose, for exarmple, th¡
In general, ensemble averages and time averages are not the ensemble were changing with time
the statistical characteristics of the sample functions in the
COr

measurements made at
a fixed time, and the ensemble trai
Such a variation could not be reflected in the sam
When the statistical characteristics of
times
averages would be different at diferent
ran

with time, the random process is described as being sStationary. HoweVe

functions do not change are th
property of being stationary does not ensure that ensemble and time averages
even the individual samn
may happen that while each sample function is stationary the
same. For it
case, the time average will depend o It
functions may differ statistically from one another. In this
average. When the nature of a random sig
the particular sample function which is used to form the
process is referred to as ergodic
process is such that ensemble and time averages are identical, the
necessarily ergodic.
An ergodic process is stationary, but, of course, a stationary process 1S not
se

We shall have a broader classification of random processes in Sec. 2.5.2 after we define th
autocorrelation of a random process. th
sp
2.5.2 Classification of Random Processes
random processes
We have already discussed stationarity and ergodicity in Sec. 2.5 while defining
subsection.
We look into subdivision of that and other classification in this
stationary(SSS) if its
Strict-sense stationary A random process X() is called strict sense
written as
statistics does not change with shift of origin, i.e. its n-th order density function can be
(2.142)

where, *,ni lj, l2, .., I,) =

dx, dx,. dx,
while n-th order distribution Fx, x, ..., xy; t, tz, ..., )= P[XU)Sx, X() S X, ..., XU,) Sx]
Thus, for SSS, first order densityfx; t) =fx) (By putting t = tand T = t]
Second order densityfx, X; tË, t) =fx, X, ) [putting T = -]
Wide sense stationary A random process X(1) is called wide sense stationary (WSS) if
Mean, E[X()] = m (constant) and Autocorrelation, E[X()X(t + )] = Ry)
utocorrelation depends only on time difference. Putting t = 0, we get
(2.143)
E;X()]= Ry0)
i.e. average power of WSS is constant and independent of time.
Multiple random process Now, two processes are called jointly WSS, if each is WSS and
their cross correlation depends only on time difference. (2.144)
Ryt, tt )= E[X() Yt + )] = Ryt)
Also, Gxyf)and Ryt) are Fourier transform pair such that
Ryylt) = Ryy-t) and Gyyf)= Gyr)
in mean)
Ergodic random process Astationary process X() is called ergodic (or ergodic
if time average is same as ensemble average, i.e. <x()> = E[X()] = m. It is called ergodie
autocorrelation if <x()x(1 + )> EX()X(1 + )]= Ryxt).
be
Band pass random process Aband pass (or band limited) random prOcess x() Can

expressed in terms of in-phase and in-quadrature components as shown in Eq. (1.169)

X() = X)cos(2 nf.) + X{() sin(2f ) (2.145)

If x(t) is a bandpass signal and is input to the BPF of Fig. 1.27c of Chapter 1, and theit
bandwidths are same then the output will be unaltered x(t). Since, multiplication by cos(21J )
 Random Variables and Processes 123

side shifts the bandpass signal by tf and J, along frequency axis and LPF1 filters out
the input components, we can write PSD for in-phase component as
the low frequency
only
Gy S) = Gyf+s)t Gr-f for |f|S Band zero elsewhere.
component,
Similarly, for quadrature
Gx)= Gy(f+s) + Grf-S) for |f|sB and zero elsewhere.

It can
be shown that, <X() X,(0)> = Ryx (0) = 0
also

x (t) = 0
and if Gyl) issymmetrical about f, Ry
called Gaussian process if X = [X(t;)
Caussian random process A random process X0) is
Y).. XU)] has a jointly multivariate Gaussian density function given
by
(2.146)
Sx(X) =
(2ryl2 |det Cq/2
Cin
: such that C; is covariance of X(t)
where, x = (xx ... x]', 4
E[X] and C=
Cml Cn

Rx;, 4,)- 4 ;
and X(t). Note that, C;=completely autocorrelations. If the input
specified by its set of means and process is WSS
A Gaussian process is output is also Gaussian. Finally, if a Gaussian
Gaussian, the
to a linear system is
then it is also SSS. = n/2, i.e.
process X() is called white noise when its PSD, Gf)
White noise A random mean is assumed to be zero.
frequency spectrum and
constant over entire
autocorrelation R(t) = &)
Fourier transform, 2
From inverse band.
has similar flat spectrum but only over the pass
The band-limited white noise

of Random Pulses
2.5.3 PSD of Seguence power spectral density of a
sequence of
information about the random
We shall occasionally need
to have pulses are of the same form but have
indicated in Fig. 2.19.
The waveform (the random
andom pulses such as is independent random times of occurrence. The invariant. Corre
alnplitudeS and statistically of the waveforms are time
features assume
that the statistical pulses. We further
Process) 1S stationary so time of separation 1, between
Spondingly, there is an invariant average
Parseval's theorem
that there is no overlap between pulses. sample pulse P.() is P,) then
of a single
the Fourier transform normalized energy of the pulse is
(E4. (1.136)]states that the (2.147)
APTER
oCESS

2.1 INTRODUCTION
asSociated with random
extensionsof the concepts
Random processes
are
parameter is brought into
the problem. In this
variables when the time issues:
discuss the following
chapter, we will
random process.
The representation of stationary.
random process to be
Required condition for a
average of a random process.
Time average and ensemble its mean.
description of a random process in terms of
The partial
correlation, and covariance
function.
ergodic.
stationary random process to be 2.2.1 Cla
Required condition for a
for a random process to be
at least WSS
Minimum required condition
(wide sense stationary).
of
description of a random process in terms
The frequency-domain
power spectral density.
Random process in a linear system.

RANDOM PROCESS
2.2
functions
process) is an indexed set of real
A random process (or stochastic 2.2.2 P:
Figure 2.1 shows the sample
of time that has certain statistical properties.
outcome of a sample
functions of a random process X(t, s) where s is the
simply use X(t)to
space S. To simplify the notation, we suppress s and
denote the random process.
Sample
space
S

X(t,s)

X(1,s,)
2.2.3
t+

X(t,s,)

X,=X()=(X(4,S), al } X,= X(4) ={X(1,S,), all

Figure 2.1 :Illustration of the Sample Functions of a Random Process
 Following are some key points related to the random process X(t, s) (see Page 83
figure 2.1): Chap 2
Random Process
PoINTS TO REMEMBER

1. One possible waveform is X(t, s). Another is X(t, s,). In general,

X(t, s;) denotes the waveform that is obtained when the event 8, of the
sample space S occurs.
2. For a fixed sample point 8,, the graph of the function X(t, s:) versus
time t is called a realization or sample function of the random proces8.
The set of sample functions(X(t, s)} is called the ensemble and defines
the random process X(t).
3. From the figure, we note that for a fixed time t, inside the observation
interval, the set of numbers {X(te, 1), X(th, %a),...., X
(h, 5a)}
constitute a [Link].
4. Fora random variable, the outcome of a random experiment is mapped
into a number; where as for a random process, the outcome of a random
experiment is napped into a waveform that is a function of
time.

2.2.1 Classifcation of Random Process

discrete.
Random processes may be classified as continuous or
random process consists
1. Continuous Random Process: A continuous
continuously distributed random
of a random process with associated
example of the
variables X(t, s;). The Gaussian random process is an
continuous random prOCeSs.
process consists of the
2 Discrete Random Process: A discrete random
example, the output of an
random variables with discrete distributions. For
levels) random process.
ideal (hard) limiter is a binary (discrete with two

Process
2.2.2 Probability Density Function of Random
process {X(t, s)} is given by the N-fold
A complete description of a random
describes the possible values assumed by
a
joint pdf that probabilistically
t - 1 . > t, where N is arbitrary.
typical sample function at time tN >
joint pdf as
For N=1, we can interpret this
at time t)
fs (2i, t) da, = P( - da < XËs pdf aS
where X = X(t, s). Similarly, for N=2, we can interpret the joint
< XË` Ë and - da, < X¡ s )
fxx,(1, t: 2, ta)da, da, = P(a - da, denote the N-dimensional PDF of a
general, we
where X,= X(t,s). In
random procesS as
fa(z) = fay({4), z(ta), ...tw)
2.2.3 Stationary Random Process
the order N if, for any
random process X(t) is said to be stationary to
A
ti, ta...., tN
fxo(z(t+ to), z(t t to),..a(ty t to))
fo((4), z(ta),..(tN)real= constant. Furthermore, the process is said to
where to is any arbitrary to the order N ’ 0,
strictly stationary if it is stationary
be
 2.3 AVERAGES OF RANDOM PROCESS
We may define the time average
and ensenble average of a random
in the following ways: nroos

2.3.1 Time Average of a Random Process

Consider a stationary random prOcess X(
The time average of this random t) with the sample function aA
process is defined as
< X(t) > =im+ t)dt
-T/2
2.3.2 Ensemble Average of a Random Process
For a given random process
Xt), the
mean value of X(t) at arbitrary time ensemble average is defined as the
t, i.e.
E(X(1)]=X(t) = #x(t) =/ ze(z) dr
An important property of ensemble
average is given below.
PROPERTY OF ENSEMBLE AVERAGE
The mean (ensemble average) of a
stationary process is constant, i.e.
Px(t) = Mx for all t

2.3.3 Autocorrelation function

The autocorrelation function of a random process X(t)
expectation of the product of two random variables is defined as the
X(t) and X(t), i.e.
Rx(th, ta) = E|X(4)X(4)] = X(H)X(a)
where Ë= at) and T) =a(t). Some
important properties of the
autocorrelation function of a stationary random
process are given below.
PROPERTIES OF AUTOcORRELATION FUNCTION
1. The autocorrelation function
of strictly stationary random
a function only of the time process is
difference T - t, i.e.
Rx(r)
= E|X(t) X(t+ r)]= X(t)X(t+ )
2. The mean square value of the process may be obtained from
simply by putting T=0 in above equation, i.e. Rx(T)

3. If the Rx(0) = E[x*()]=

autocorrelation function X(t)
of a random process has no periodic
component then, we may define
X(t) = Rx(t or X(t) =/Rx(t o)
4. The autocorrelation function is theoo)even function of T, i.e.
5. The autocorrelation
Rx(r)=
has
Rx(-r)
its maximum magnitude at T = 0,
i.e.
Rx(0) > Rx()
 Cross-Correlation Function Page 85
2.3.4
Chap 2
The cross-correlation function for two random processes X( t) and Y(t) is Random Process
defined as
Rxr(ts ta) = E;X(;) Y(.)] = X(4) Y\4)
where t and t denote the two values of time at which the processes are
observed. Following are some important properties of the crosS-correlation
function of two jointly stationary random processes X(t) and Y(t):
PROPERTIES OF CROSS-CORRELATION FUNCTION
1. The cross-correlation function of jointly random processes X(t) and
Yt) is a function only of the time differencë T = th ti, i.e.
Rxy(t, ta) = Rxr(r)
2. The cross-correlation function is the even function of T, i.e.
Rxr(-r) = Ryx(r)
uncorrelated if,
3. The random processes X(t) and Y(t) are said to be
for all t and te,
Rxr (ts ta) = X(&) xY()
4. Rxr(7) s /Rx(0)Rr (0)
5. |Rerr)| s R(0) +Ry (0))

NOTE :
are usually statistically
Physically independent random processes
uncorrelated processes are
independent and, hence, uncorrelated. However,
not necessarily independent.

Autocovariance Function
2.3.5
The autocovariance function of a stationary process is defined as

Cx(t, to) =E|(X(t) uxXX(4)-ux)

= Rx(te- ti)
Thus, the autocovariance function of a stationary process depends on the
time difference t - t1, only.

2.4 ERGODIC PROCESS

if all time averages of any sample
A random process is said to be ergodic
ensemble averages (expectations).
function are equal to the corresponding equal to its time average, we
average
As the ergodic process has its ensemble
ergodic process:
may deduce the following propèrties for the
PROPERTIES OF ERGODIC PROCESS
ensemble
process can be defined in terms of
1. The de value of an ergodic
average as

= E|X()]
X= (X(t )) ergodic = X(t)
process can be defined as
2. The dc power of an

Pae - (X(1)°={E[X(1) -{X(0)}

 3. The power in the ac (time-varying) component is given by
P = oi =(X'())-(X(1)' X()-X()
4. The rms power (total power) of an crgodic proceSs is defined as 2.6

Pan =(N())= E|r'()]= X(1) - o t X(L÷

5. The rms value of an ergodic process can be defined as
Ams =(X(1)) =/E x*(1)) = /X(0)

2.6
2.5 WIDE SENSE STATIONARY PROCESS

A random process is said to be wide-sense stationary (WSS) if its mean

is independent of time and autocorrelation function depends on the time
difference. i.e.
E(X(t)] = Mx where ux is a constant
and Rx(h,ta) = Rx(), where T= t - tË

POINT TO REMEMBER
A process that is stationary to order 2 or greater is wide-sense stationary.
However, the converse is not necessarily true.

METHODOLOGY: STATIONARITY OF A RANDOM PROCESs

2.6
To check the stationarity of a random process X{t), follow the steps
given below.
Step 1: Obtain the mean (ensemble average) of the given random process
using the expression,
X() =zh() de
Step 2: If the obtained value of mean depends on time, then the random
process is said to be not stationary in any sense. If the mean is
indepemdent of time, then proceed to next step.
Step 3: Obtain the autocorrelation function of random process using the
expression

Rx(6, ta) = E[X(4) X(t)}

Step 4: If the obtained autocorrelation is the fnction of time difference
only, i.e.
Rx (ti, tu) = Rx(4-- t) = Rx (t)
Then, the random process is said to be at least WSS (wide
stationary).
Step 5: If the random process is at least WSS, then
higher stationarity conditions (second order check it for the
th order stationarity, ergodicity, etc). stationarity, 1
process is said to be not stationary in Otherwise, the rando
any semse.
 2.6 POWER SPECTRAL DENSITY
Page 87
The power spectral density (PSD) for a random process X(t) is given by Chap 2
Random Process
S:(f) =lim 7
where XT() is the Fourier transform of the truncated sample function of
the random process X(t), given by
cT/2
Xr(f) =JT/2 a(t)e dt
2.6.1 Wiener-Khintchine Theorem
When X(t) is a wide-sense stationary process, the power spectral density
can be obtained from the Fourier transform of the autocorrelation function,
i.e.

Sx(f)= FRx(r)] = Rx(r)e t ar

Conversely, we have
Rx(r) =F[Sx()]= | S:(f)etaf
provided that Rx(T) becomes sufficiently small for large values of T, So that
|TRx(r) dr < oo
NOTE:
The Wiener-Khintchine theorem is also valid for a nonstationary process,
provided that we replace Rx(T) by (Rx(t,t+)
2.6.2 Properties of Power Spectral Density
Following are some important properties of the power spectral density of a
stationary process.
PROPERTIES OF POWER SPECTRAL DENSITY
1. The power spectral density Sxf) is always real and nonnegative, i.e.
Sx(f) >0
2. The power spectral density of a real-valued randomn process is an even
function of frequency, i.e.
Sx(-f)= Sx()
3. The total normalized power of a random process is defined in terms of
power spectral density as
Poal Sx(f) df
Or
B|x()] =3:()af
Or
Ra(0) =S:(/)4f
4. The zero-frequency value of the power spectral density of a stationary
process equals the total area under the graph of the autocorrelation
function, i.e.

Sx(0) =Rx(r) ar
 2.6.3
Cross Spectral Density
stationary processes
with their o
two jointly
X(t) and Y(t) be We then define
Let denoted Rxy(7) and Ryx(T).
by
correlation functions
ess
processes as
cross-spectral densities for the random
-2nf d
Sxr()= Rxr(r)e
Syx(f)= Ryx(r)e nt dr
and
Fourier transformation. w°e
formula for inverse
Accordingly, using the
also write

Rxy(r) = |Sxr()eh af
and Ryx(T) = Sxf)enh df
2.8
DENSITY
PROPERTY OF CROSS SPECTRAL
Syx(f) are not necessarily the real
The cross-spectral densities Sxyf)and
property of autocorrelation function.
function of f. However, using the
we may deduce that
Sxr(f) = Srx(-f) = Six (f)

SIGNAL
METHODOLOGY: TO EVALUATE AC POWER OF A RANDOM

Consider a randon signal Xt) with autocorrelation function Rx (T) and

power spectral density Sx()containing no periodic component. The
steps involved in evaluating the ac power component of the random signal
are as follows:
Step 1: Obtain the de power inrandom signal using the expression given
below.

Pae =X(tj' =| Sx)df

Or
Pac = X(t = Rx(+ o)
Step 2: Obtain the total power in the random signal using the expression
given below.

Paan =X(t) = |S)df

or Pota = X{t)= Rx(0)
Step 3: Evaluate the ac power in the random signal by subtracting the
dc power from the total power obtained in above steps, i.e.
Pac = Potal Pdc

2.7 SUPERPOSITION AND MODULATION

Some random signals may be viewed as a combination of other random
[Link] a random signal be defined as
Zlt) = X(t)± Y(t)
where X(t) and Yt) are jointly stationary process. Then, the autocorrelatio
function of Z(t) may be represented as
 Rz(T) = Rx (r)+ Ry (r) ± (Rxr (7) + Ryx()]
Also, we have the power spectral density of the process Z(t) as Chap 2
Sz() = S, () + S() +[Sxr(f) + Syx()) Randmn Process

Special Case :
When X(t) and Y(t)are uncorrelated and ux uy = 0, then the cross-spectral
density vanishes, i.e.
Rxy (7) = Ryx(t) =0
Under this condition, the above relation becomes
Rz(r) = Rx(r)+ Ry()
Sz(f) = Sx(f)+ Sy()
Also,

2.8 LINEAR SYSTEM

Consider a linear system shown in Figure 2.2. The input-output relationship

for the system is
v(t) = h(t)+ (t)
The corresponding Fourier transform relationship is
Y(ð) = H)X)
Linear Network
Input Output
z(t) h() +y(t)
H)
X) Y)
R(r) RA)

Figure 2.2: Input-Output Relationship of

a Linear System

process, then the output

If dt) is a wide-sense stationary random
autocorrelation is defined as

Ry(r) = h-r)* h(r)* Rx(r) density is given by

Correspondingly, the output power spectral
Sr() =|H() Sx(f)
of the network is
Thus, the power transfer function
Sr(5)
Gð) = Sx() =|H()
 CHAPTER 1 RANDOM VARIABLE

1.1 INTRODUCTION
The need for probability theory arises in every scientifie discipline, since it
Is impOssible to be eXactly sure of values that are obtained by measurernent.
Ihis chapter is intended to be a short course in probability and random
variables. Thc topics included in this chapter are as follows:
Probability theory that establishes a mathematical framework for the
study of randomn phenonenon.
Random variable that defines a rule or relationship for an event.
Cumulative distributionfunction (CDF) and probability density funetion
(PDF for continuous random variable.
Statistical averages (mean, variance, standard deviation, moments) that
play an important role in the characterization of random variables.
Some inportant probability distributions: Binomial distribution.
Gaussian distribution, Poisson distribution, and Rayleigh distribution.

1.2 PROBABILITY
Probability theory establishes a mathematical framework for the study of
random phenomenon. We may define the probability of an event in following
two ways:
DEFINITION I
The probability of an event A, denoted by P(A), nay bà defined as
n(A)
PA) =
where
n(S)
n(S) = Total number of sample space S (possible outcomes)
nA) = Number of favourable event A
In the case of the die, we have the sample space
S ={1,2,3. 4. 5,6}
Assume that the event A is defined s
A =(2.3}
ie.. the event Aconsists of two mutually exchusive outcomes. So, we get the
probability of event A aS
n4)
P(A) =
DEFINITION II
The probability of an event A, denoted by P(A), may be defined in ternas of
 the relative frequency of A oceurring in n trials.

P(A) - lim)
where n, is the number of times that A occurs inn trials.

1.2.1 Joint Probability

The probability of a joint event AB is defined as
PAB) -lim")
where nsg is the number of times event AB occurs in n trials.

MUTUALLY EXCLUSIVE
The events A and B are said to be mutually exclusive if
P(AB) = 0

1.2.2 Conditional Probability

The probabilitv that an event A occurs, given that an event
B has also
occurred, is denoted by PA|B). which is defined as
P(A|B) = limAR
Bayes Theorem
Consider the twO events A and B with probabilities
respectively. Let the probability of a joint event AB beP(A) and P(B)
conditional probability is given by P(AB), so the

P(B| A) = P(AB)
P(A)
P(AB)
PA| B) = P(B)
This is known as Bayes' theoren.

1.2.3 Statistical Independence

The events A and B are said to be
statistically independent if
or
PAB) = P(A)P(B)
P(B| A) = P(B)
or
P(A| B) = P(A)
1.3 RANDOM VARIABLE
A randon variable is a rule or
a real number X(S) to every poit relationship, denoted by X, that
variables cali be distinguished as in the sample space S. assigns
The random
1. Discete Random Variabie
2. Continuous Randou Variable
1.3.1 Discrete Random Variable
When the random variable
called a discrete random takes only a discrete
variable. For example, we setfip ofa values, then it s
coin, the possible
 outcomes nre hend (H). and tail ( ) S ontains twn potnt laheled H and Pagr 1
1 Supr we defineA functkon VSy ench that Chap 1
for S H Band Vriahl

for S I
Ths we have mnpped the tao outeomes into the two pointa on the real
hne S this is Calledn diecrete random variahir
Probability Density Fanction of Discrete Random Variable
Tet a diseTete radom variable N having the poesible ottcotnes

So the probability density funtion (PDE) of the disc rete random variable
is defied as
1L2.
h(z) - P(N- )
Cumulative Distribution Function of Discrete Random Variable
For the randon variable Wr define the Cumulative distribution function

(CDE) As
Fx(z) = P(XS )-f)+f )+...+fr)

1.3.2 Continuous Random Variable

ion interval, X
If the andom variable N takes aIY value in a whole observat
example, if we define a function
0s called a cont inuous random variable. For
X(e) such that
Ae) =tan'0
Then, every value in the range 0S < o is apossible outeome of this
experitnent. Thus, we can saY that N(e)
is a continuous randotn variable.
Random Variable
Cumulative Distribut ion Function of Continuous
(CDF) of the continuous random
The cunulative distribution function
variable is given by
E() =PXSr)
continuos random variable are
Some important properties of CDF of
given belo

PROPERTIES OF CDF OF cONTINUOUS RANDOM VARIABLE

Fr(-o) =0
E() = 1
2
3 Pa<zs b) = Fr(b)- Ex(a)

Coutinuous Random Variable

Probability Density Functiou of
of a continuous randou varlable is
The probabilaty density function (PDE)
defined a
dFs()
vriable are
PDF of continuous raidom
Sote inportait properties of
given e l o w
 PROPERTIES OF PDF OF CONTINUOUS RANDOM VARIABLE

2 JA)dr =1
3 PXS) = ()-[ajdA,
1. Pa<rs b)-[)dr

1.4 TRANSFORMATION OF RANDOM VARIABLES

variable X by the
Consider a random variable Z related to another random
transformation function,
Z = gX)
random variable Z is defined as
So, the probability density function of the
dr ...(1.1)
b(:) = de r(r)
inverse transformation
Since, I transforms to z= qlz), we insert the
I=g'(2) to obtain
(:)|
(:) =A9'z) dz

METHODOLOGY: TRANSFORMATION OF RANDOM VARIABLES

Let a linear transformation given by 2 = aX+8. Following are the steps

involved in evaluating the pdf of random variable Z:
Step 1: Rewrite the transformation equation as
X= 2-3
a

Step 2: Differentiate the obtained expression with respect to z, i.e.

Step 3: Express the probability density function of random variable X

in terms of z, i.e.

ke) =A()
Step 4: Substitute the results of step-2 and step-3 in equation (1.1) to,
get pdf of random variable z, i.e.

) =k()a)
1.5 MULTIPLE RANDOM VARIABLES
Multiple random variables are basically multidimensional function defined
on a sample space of a combined experiment.
Cumulative Distribution Function of Joint Random Variable
Let the two randon variables X and Y,each of which may be contiuous,
discrete. or mixed. The joint CDF for the two randon variables is defined as
 Fxy (z, y) = P(X s 1,Y S y)
Some important properties of CDF of joint random variable are given below.
Ran
PROPERTIES OF CDF OF JOINT RANDOM VARIABLE
1.
2.
Fry(-o,- co) = Fxr(3, - ) = Fxr(-0,u) =0
3.
Fxy(o0, oo) =1
Fx (z) = Fxy(,0)
4
F(v) = Fxy (0,y)
Probability Density Function of Joint Random Variable
The PDF of joint random variables X andY can be defined as

fxr(z.y) Fxr(r. y)
drôy
Some important properties of PDF of joint random variable are given below.

PROPERTIES OF PDF OF JOINT RANDOM VARIABLE

1. fr(z,y) 0
2.
C nty)dz dy =1
3. k(r) = ho(zy)dy
4.

5. P(XSz,Ys y) = Fr( y)= J r(u, v) dudu

Conditional Probability Density Function

Y on X is
For the two random variables X and Y, the dependence of
expressed by the conditional PDF
fkr (z, y)
x(yl) = R()
Similarly, the dependence of X on Y is expressed by
kr(I, y)
fsx(z|9) B(9)
VARIABLE
6 STATISTICAL AVERAGE OF RANDOM
characterization of outcomes
Statistical averages play an important role in the
sample space of the
of experiments and the random variables defined on theaverages.
experiments. Let us obtain some important statistical

.6.1 Mean or Expected Value

x(r). The nean or
Let a random variable X characterized by its PDF
expected value of X is defined as
1.6 STATISTICAL AVERAGE OF RANDOM VARIABLE

Statistical averages play an important role in the characterization of outcomes

of experiments and the random variables defined on the sample space of the
experiments. Let us obtain some imnportant statistical averages.

1.6.1 Mean or Expected Value

Let a random variable X characterized by its PDF x(r). The mean or
expected value of X is defined as

E(X)=X- h(2)dr
Similarly, we obtain the expected value of a function g(X) as

If N is a discretely distributed random variable, then the expected value of

X is given by

1.6.2 Moments
In general, the nth moment of the random variable X is defined as

The nth central moment of the random variable X is defined as

A(X-ul =(X- px)" = |(-)" k(z)dr

1.6.3 Variance
The variance o of a random variable X is the second moment taken about
its mean. i.e.

Var(X] =o- 4(X-ux]

-Je-u} k(=)ds
Expanding the above equation, we can write
o = EX-{EX|P = X-*
1.6.4 Standard Deviation
The standard deviation y of a random variable is the square root of its
variance. i.e.

1.6.5 Characteristic Function

The characteristic function ¢x(v) of the probability
distribution of a random
variable X is defined as the expectation of the complex
en, i.e. exponential function
ox(") =E<ek]= f"ek()dz
Also, we may deduce the inverse relation as

Hence, the characteristic function and probability

random variable constitute the Fourier transform density function of a
pair.

Layout 4 Save
 Ox =/Var[X] = /X
1.6.5 Characteristic Function
The characteristic function ox(v) of the probability
distribution of a random
variable X is defined as the expectation of the complex
r . i.e. exponential function

ox(") =E|e=[e"s(2)dr
Also, we may deduce the inverse relation as
1
k() =| ox(v)e dv
Hence, the characteristic function and
random variable constitute the Fourier probability density function of a
transform pair.
1.6.6 Joint Moments
For a pair of rarndom variables X and
Y, we define the joint moment as
E|X' Y*] =
where i and k nay assume any rk(r.y)dr dy
positive integer values.
1.6.7 Covariance
The covariance of the random
variables X and Y is defined as
cov[XY] = Oxy = E(X- px)(Y- ))
=(X- ux)X(Y- y)

where l and ly are the mean of random variablesX and Y. respectively.

We may expand the albove result as
R

1.6.8 Correlation Coefficient

The correlation coefficient of random variables X and Y can be defined as
Pry
cov[X)|
where covXY] is the covariance of X and Y, and dy, oy are the standard
deviations of random variables. Following are some important points related
to random variables:

POINTS TO REMEMBER
1. The random variables X and Y are uncorrelated if and only if their
covariance is zero, i.e
cov[XY] = 0
2. The random variables X and Y are orthogonal if and only if their
correlation is zero, i.e.
E[XY] = 0

METHODOLOGY: TO EVALUATE VARIANCE OFA RANDOM VARIABLE

Following are the steps involved in evaluating the variance of a random
variable X:
Step 1: Obtain the mean of given random variable by using the
expressions given below.
when X is continuous RV
X=
 be defined as
The correlation coefficient of random variables X and Y can
Pxy = cov[XY]
standard
where covXY] is the covariance of X and Y, and x. y are the
deviations of random variables. Following are some important points related
to random variables:

POINTS TO REMEMBER
1. The random variables X and Y are uncorrelated if and only if their
covariance is zero, i.e
cov[XY] =0
2. The random variables X and Y are orthogonal if and only if their
correlation is zero, i.e.
E[XY] = 0

METHODOLOGY: TO EVALUATE VARIANCE OF ARANDOM VARIABLE

Following are the steps involved in evaluating the variance of a random
variable X:
Step 1: Obtain the mean of given random variable by using the
expressions given below.
Coo
when X is continuous RV
X=
2+A(1)
=1
when X is discrete RV

Step 2: Obtain the second moment (mean square value ) of given random
variable by using the expressions given below.
when X is continuous RV

2k() when X is discrete RV

Step 3: Evaluate the variance of random variable X by substituting the

results obtained in step- and step-9 in the expression

1.7 SOME IMPORTANT PROBABILITY DISTRIBUTIONS

Many problems has been devised and studied as models for various random
phenomena. In this section, we will discuss the properties of two discrete
 continuous

Poisson) and two

(Binomial and
functions

and Rayleigh)

discrete
randor
Binomial Distribution
an
integer-valued
experiment witl
1.7.1 distribution
describes
a chance
Binomial Consider probabilitio
The repeated trials. the
variable
associated with outcomes A and A with
exhaustive
exchusive,
mutually
two
respectively as
P(A) = P
P(A) =q=1-p numerically equal to
and variable K to be
the discrete random chance experiment.
If we assign
event A occurs in n trials of our probability of
the number of
times Distribution. The
Binomial
distribution is called
the resulting
exactly k heads in n trials is given by
P(K= ) ="Cup q given by
random variable K
The mean of the binomial
MK = EK]= np
randon variable is given by
and the variance of the Binomial
aK = npg
notations and write
For simplicity, we omit the subscript K from the
u = np
and G' = npq

1.7.2 Pois3on Distribution

The Poisson random variable also describes the integer valued random
variable associated with repeated trials. Consider a chance experiment in
which the probability of occurrence of an event in a very small interval 4T
is
p=aAT

where a is a constant of proportionality. If successive occurrences are

statisticaly independent, then the probability of occurrence of k events in
time T is given by

PA) =e k!
This is called the Poisson distribution. The mean and variance of
Poisson
random variable is given by
u=oT
and o = =a1
The Poison model also approximates the
very large, p is very small, and the product npqBinomial model when n is
distribution is given by np. The approximated

P(k) =eu (u)

k!

.7.3 Gaussian Distribution

Gaussian distribution describes a
normal distribution encountered continuous random variable having the
Gaussian random variable X, the in many different applications. For a
probability density function is given by
 Pag
V2na' Cha
Random Vari
where and o' are respectively the mean and varinne of ranlon variable
This function defines the bell shaped curve nheown in figure 11

42)
2

Iigme LI Gaussian Distribution

[Link] Gaussian QFunction

The Gaussian probability density function of a random variable Z with unit
variance and zero mean is given by

So, we have the corresponding CDF of the random variable as

The Gaussian Q-function is defined by

Q(:) =1- P(*)
1

[Link] Gaussian CDF

In general, the cumulative distribution function for the Gaussian distribution
is given by
Fx(z) = )=q )
Also, we may define the Gaussian CDF as
P(2) =erfe()
where the complementary error function (erfc) is defined as
erfc(z) =r
Following are some important properties of Gaussian CDF:
PROPERTIES OF GAUSSIAN CDF
1. P(XS #) = P(X> )=
2. P(X> + z0) = ()
3.
erfc(z) = 1- erf(z)
where the error function defined as
ert(2) - a
1.7.4 Rayleigh Distribution
randon variable obtained
The Ravleigh distribution describes a continuous
independent Gaussian
from two Ganssian random variables. If X and Yare
variance o, then the
random variables with zero mean nnd the same
defined by
corresponding Ravleigh rAndom variable is
R-'+

The probability density function of the Rayleigh randomn variable is given by

is
The corresponding CDF of Ravleigh random variable
SR(T) = 1-eo'

The resulting mean of R is

R
The resulting second moment of R is
R = 26?
k**k k*k