BEC402
Module 1
Random Process and Probability
Preethi A,Sridevi S
ECE, CMRIT
Syllabus: Module 5
• Random Variables and Processes: Introduction, Probability,
Conditional Probability, Random [Link] Averages:
Function of a random variable, Moments, Random Processes, Mean,
Correlation and Covariance function: Properties of autocorrelation
function, Cross–correlation functions
• Gaussian Process:Gaussian Distribution Function.
[Text 2: 5.1, 5.2,5.3,5.4,5.5,5.6,5.9]
Communication signals and Randomness
• Signals in communication: information-bearing signal component, a random interference
component, and receiver noise.
• Received signal is completely random in nature:
• The information-bearing signal component may consist of randomly spaced bursts of energy of
random duration.
• The interference component may represent spurious electromagnetic waves produced by other
communication systems operating near the radio receiver.
• A major source of receiver noise is thermal noise, caused by the random motion of electrons in
conductors and devices at the receiver’s front end.
• It is not possible to predict the precise value of a random signal in advance, it may be
described in terms of its statistical properties such as
• the average power in the random signal
• the average spectral distribution of this power.
• The statistical characterization of random signals is probability theory
Probability
• Relative frequency of occurrence: in n trials of a random experiment,
if we expect an event A to occur m times, then we assign the
probability m/n to the event A.
• Based on set theory and axioms(mostly applicable if experiments are
not repeatable)
Probability: basic definitions
• The set of all possible outcomes of the experiment is called the
sample space, which we denote by S.
• An event corresponds to either a single sample point or a set of
sample points in the space S.
• A single sample point is called an elementary event.
• The entire sample space S is called the sure event, and the null set f is
called the null or impossible event.
• Two events are mutually exclusive if the occurrence of one event
precludes the occurrence of the other event.
Contd..
A probability measure P is a function that assigns a non-negative number to an event A in the
sample space S and satisfies the following three properties (axioms):
The probability value is not unique to the event; mutually exclusive events may be assigned the same
probability. However, probability of the union of all events — ie the sure event — is always unity.
Contd..
Conditional Probability
• conditional probability of B given A : P[B|A] denotes the probability of
event B, given that event A has occurred.
The joint probability of two events
• Bayes’ rule:
statistically independent events
• then
Random Variable
Joint Moments
• Consider a pair of random variables X and Y
• Correlation defined by E[XY]
• Covariance of X and Y :The correlation of the centered random
variables X- E[X| and Y- E[Y], that is, the joint moment
• cov[XY] = E[(X-E[X])(Y-E[Y] )
• Correlation coefficient:
• X and Y are uncorrelated if and only if their covariance is zero, that is,
if and only if cov[XY]=0
• Orthogonal if and only if their correlation is zero, that is, if and only if
E[XT] = 0
• If one of the random variables X and Y or both have zero means, and
if they are orthogonal, then they are uncorrelated, and vice versa.
• Note also that if X and Y are statistically independent, they are
uncorrelated; however, the converse of this statement is not
necessarily true.
Random Process
• In describing random signals, we find that each sample point in our
sample space is a function of time.
• The sample space or ensemble comprised of time functions is called a
random or stochastic process.
• Sample function
• Thus we have an indexed ensemble (family) of random variables
{X(t,s)}, which is called a random process.
• To simplify the notation, the customary practice is to suppress the s
and simply use X(t) to denote a random process.
• We may formally define a random process X(t) as an ensemble of time
functions together with a probability rule that assigns a probability to
any meaningful event associated with an observation of one of the
sample functions of the random process.
• For a random process, the outcome of a random experiment is
mapped into a waveform that is a function of time.
Ensemble of sample functions
MEAN,CORRELATION,AND COVARIANCE
FUNCTIONS
• Mean
• A random process is said to be stationary to first order if the distribution function (and
therefore density function) of X(t) does not vary with time. That is, the density
functions for the random variables satisfy
• Autocorrelation
• Stationary to second order
PROPERTIES OFTHE AUTOCORRELATION
FUNCTION
• The mean-square value of the process may be obtained from Rx(T)
simply by putting x = 0
• The autocorrelation function Rx(T) is an even function of T
• Has max magnitude at T=0
Autocovariance
wide-sense stationary: all strictly stationary processes are wide-sense
stationary, but not all wide-sense stationary processes are strictly
stationary
Cross Correlation
Ergodic Process
• It is difficult or impossible to observe all sample functions of a random
process at a given time. More convenient to observe a single sample function
for a long period. For a single sample function, we may compute the time
average of a particular function.
• For many stochastic processes of interest in communications, the time
averages and ensemble averages are equal, a property known as ergodicity.
• Implies that whenever an ensemble average is required, we may estimate it
by using a time average