CMR COLLEGE OF ENGINEERING & TECHNOLOGY

Kandlakoya (V), Medchal Road, HYDERABAD -501401

(An Autonomous Institution under UGC & JNTUH , Approved by AICTE,
Permanently Affiliated to JNTUH, Accredited by NAAC with 'A' Grade.)

DEPARTMENT OF ECE

COURSE FILE

SUBJECT: Probability Theory and

Random Process

PREPARED BY
[Link] Ram

DEPARTMENT OF ECE
 Unit-I: PROBABILITY AND RANDOM VARIABLE

Introduction
It is remarkable that a science which began with the consideration of games of
chance should have become the most important object of human knowledge.

A brief history

Probability has an amazing history. A practical gambling problem faced by the French
nobleman Chevalier de Méré sparked the idea of probability in the mind of Blaise Pascal
(1623-1662), the famous French mathematician. Pascal's correspondence with Pierre de
Fermat (1601-1665), another French Mathematician in the form of seven letters in 1654 is
regarded as the genesis of probability. Early mathematicians like Jacob Bernoulli (1654-
1705), Abraham de Moivre (1667-1754), Thomas Bayes (1702-1761) and Pierre Simon De
Laplace (1749-1827) contributed to the development of probability. Laplace's Theory
Analytique des Probabilities gave comprehensive tools to calculate probabilities based on
the principles of permutations and combinations. Laplace also said, "Probability theory is
nothing but common sense reduced to calculation."

Later mathematicians like Chebyshev (1821-1894), Markov (1856-1922), von Mises

(1883-1953), Norbert Wiener (1894-1964) and Kolmogorov (1903-1987) contributed to
new developments. Over the last four centuries and a half, probability has grown to be one
of the most essential mathematical tools applied in diverse fields like economics,
commerce, physical sciences, biological sciences and engineering. It is particularly
important for solving practical electrical-engineering problems in communication, signal
processing and computers.

Notwithstanding the above developments, a precise definition of probability eluded the

mathematicians for centuries. Kolmogorov in 1933 gave the axiomatic definition of
probability and resolved the problem.

Randomness arises because of

o random nature of the generation mechanism

o Limited understanding of the signal dynamics inherent imprecision in
measurement, observation, etc.

For example, thermal noise appearing in an electronic device is generated due to random
motion of electrons. We have deterministic model for weather prediction; it takes into
account of the factors affecting weather. We can locally predict the temperature or the
rainfall of a place on the basis of previous data. Probabilistic models are established from
observation of a random phenomenon. While probability is concerned with analysis of a
random phenomenon, statistics help in building such models from data.

Deterministic versus probabilistic models

A deterministic model can be used for a physical quantity and the process generating it
provided sufficient information is available about the initial state and the dynamics of the
process generating the physical quantity. For example,

 We can determine the position of a particle moving under a constant force if we

know the initial position of the particle and the magnitude and the direction of the
force.

 We can determine the current in a circuit consisting of resistance, inductance and

capacitance for a known voltage source applying Kirchoff's laws.

Many of the physical quantities are random in the sense that these quantities cannot be
predicted with certainty and can be described in terms of probabilistic models only. For
example,

 The outcome of the tossing of a coin cannot be predicted with certainty. Thus the
outcome of tossing a coin is random.

 The number of ones and zeros in a packet of binary data arriving through a
communication channel cannot be precisely predicted is random.

 The ubiquitous noise corrupting the signal during acquisition, storage and
transmission can be modelled only through statistical analysis.

Probability in Electrical Engineering

A signal is a physical quantity that varyies with time. The physical
quantity is converted into the electrical form by means of some transducers . For
example, the time-varying electrical voltage that is generated when one speaks
through a telephone is a signal. More generally, a signal is a stream of information
representing anything from stock prices to the weather data from a remote-sensing
satellite.
 
Figure 1 A sample of a speech signal

An analog signal is defined for a continuum of values of domain parameter

and it can take a continuous range of values.

A digital signal is defined at discrete points and also takes a discrete set of
values.

As an example, consider the case of an analog-to-digital (AD) converter. The input to the
AD converter is an analog signal while the output is a digital signal obtained by taking the
samples of the analog signal at periodic intervals of time and approximating the sampled
values by a discrete set of values.
 Figure 3 Analog-to-digital (AD) converters

Random Signal

Many of the signals encountered in practice behave randomly in part or as a whole

in the sense that they cannot be explicitly described by deterministic mathematical
functions such as a sinusoid or an exponential function. Randomness arises because of the
random nature of the generation mechanism. Sometimes, limited understanding of the
signal dynamics also necessitates the randomness assumption. In electrical engineering we
encounter many signals that are random in nature. Some examples of random signals are:

i. Radar signal: Signals are sent out and get reflected by targets. The reflected
signals are received and used to locate the target and target distance from the
receiver. The received signals are highly noisy and demand statistical techniques
for processing.

ii. Sonar signal: Sound signals are sent out and then the echoes generated by some
targets are received back. The goal of processing the signal is to estimate the
location of the target.

iii. Speech signal: A time-varying voltage waveform is produced by the speaker

speaking over a microphone of a telephone. This signal can be modeled as a
random signal. A sample of the speech signal is shown in Figure 1.

iv. Biomedical signals: Signals produced by biomedical measuring devices like

ECG, EEG, etc., can display specific behavior of vital organs like heart and brain.
Statistical signal processing can predict changes in the waveform patterns of these
signals to detect abnormality. A sample of ECG signal is shown in Figure 2.
v. Communication signals: The signal received by a communication receiver is
generally corrupted by noise. The signal transmitted may the digital data like video
or speech and the channel may be electric conductors, optical fiber or the space
itself. The signal is modified by the channel and corrupted by unwanted
disturbances in different stages, collectively referred to as noise.

These signals can be described with the help of probability and other concepts in
statistics. Particularly the signal under observation is considered as a realization of a
random process or a stochastic process. The terms random processes, stochastic processes
and random signals are used synonymously.
A deterministic signal is analyzed in the frequency-domain through Fourier series
and Fourier transforms. We have to know how random signals can be analyzed in the
frequency domain.

Random Signal Processing

Processing refers to performing any operations on the signal. The signal can be
amplified, integrated, differentiated and rectified. Any noise that corrupts the signal can
also be reduced by performing some operations. Signal processing thus involves
 o Amplification

o Filtering

o Integration and differentiation

o Nonlinear operations like rectification, squaring, modulation, demodulation

etc.

These operations are performed by passing the input signal to a system that
performs the processing. For example, filtering involves selectively emphasising certain
frequency components and attenuating others. In low-pass filtering illustrated in Fig.4,
high-frequency components are attenuated

Figure 4 Low-pass filtering

Signal estimation and detection

A problem frequently come across in signal processing is the estimation of the true
value of the signal from the received noisy data. Consider the received noisy signal
given by
 where is the desired transmitted signal buried in the noise .

Simple frequency selective filters cannot be applied here, because random noise
cannot be localized to any spectral band and does not have a specific spectral pattern. We
have to do this by dissociating the noise from the signal in the probabilistic sense. Optimal
filters like the Wiener filter, adaptive filters and Kalman filter deals with this problem.

In estimation, we try to find a value that is close enough to the transmitted signal.
The process is explained in Figure 6. Detection is a related process that decides the best
choice out of a finite number of possible values of the transmitted signal with minimum
error probability. In binary communication, for example, the receiver has to decide about
'zero' and 'one' on the basis of the received waveform. Signal detection theory, also known
as decision theory, is based on hypothesis testing and other related techniques and widely
applied in pattern classification, target detection etc.

Figure 6 Signal estimation problem

Source and Channel Coding

One of the major areas of application of probability theory is Information theory

and coding. In 1948 Claude Shannon published the paper "A mathematical theory of
communication" which lays the foundation of modern digital communication. Following
are two remarkable results stated in simple languages :

 Digital data is efficiently represented with number of bits for a symbol decided by
its probability of occurrence.

 The data at a rate smaller than the channel capacity can be transmitted over a noisy
channel with arbitrarily small probability of error. The channel capacity again is
determined from the probabilistic descriptions of the signal and the noise.

Basic Concepts of Set Theory

The modern approach to probability based on axiomatically defining probability as

function of a set. A background on the set theory is essential for understanding probability.

Some of the basic concepts of set theory are:

Set

A set is a well defined collection of objects. These objects are called elements or
members of the set. Usually uppercase letters are used to denote sets.

Probability Concepts

Before we give a definition of probability, let us examine the following concepts:

1. Random Experiment: An experiment is a random experiment if its outcome

cannot be predicted precisely. One out of a number of outcomes is possible in a
random experiment. A single performance of the random experiment is called a
trial.

2. Sample Space: The sample space is the collection of all possible outcomes of a
random experiment. The elements of are called sample points.

 A sample space may be finite, countably infinite or uncountable.

 A finite or countably infinite sample space is called a discrete sample

space.

 An uncountable sample space is called a continuous sample space

3. Event: An event A is a subset of the sample space such that probability can be
assigned to it. Thus

 For a discrete sample space, all subsets are events.

 is the certain event (sure to occur) and is the impossible event.

Figure 1

Consider the following examples.

Example 1: tossing a fair coin

The possible outcomes are H (head) and T (tail). The associated sample space is
It is a finite sample space. The events associated with the sample space are:
and .

Example 2: Throwing a fair die:

The possible 6 outcomes are:

. . .
The associated finite sample space is .Some events are

And so on.

Example 3: Tossing a fair coin until a head is obtained

We may have to toss the coin any number of times before a head is obtained. Thus the
possible outcomes are:
H, TH, TTH, TTTH,
How many outcomes are there? The outcomes are countable but infinite in number. The
countably infinite sample space is .

Example 4 : Picking a real number at random between -1 and +1

The associated sample space is

Clearly is a continuous sample space.

Definition of probability

Consider a random experiment with a finite number of outcomes If all the outcomes of
the experiment are equally likely , the probability of an event is defined by

where

Example 6 A fair die is rolled once. What is the probability of getting a ‘6’ ?

Here and
Example 7 A fair coin is tossed twice. What is the probability of getting two ‘heads'?

Here and .
Total number of outcomes is 4 and all four outcomes are equally likely.

Only outcome favourable to is {HH}

Discussion

 The classical definition is limited to a random experiment which has only a finite
number of outcomes. In many experiments like that in the above examples, the
sample space is finite and each outcome may be assumed ‘equally likely.' In such
cases, the counting method can be used to compute probabilities of events.

 Consider the experiment of tossing a fair coin until a ‘head' [Link] we have
discussed earlier, there are countably infinite outcomes. Can you believe that all
these outcomes are equally likely?

 The notion of equally likely is important here. Equally likely means equally
probable. Thus this definition presupposes that all events occur with equal
probability . Thus the definition includes a concept to be defined

Relative-frequency based definition of probability

If an experiment is repeated times under similar conditions and the event occurs in

times, then

Example 8 Suppose a die is rolled 500 times. The following table shows the frequency
each face.

We see that the relative frequencies are close to . How do we ascertain that these

relative frequencies will approach to as we repeat the experiments infinite no of times?

Discussion This definition is also inadequate from the theoretical point of view.

 We cannot repeat an experiment infinite number of times.

 How do we ascertain that the above ratio will converge for all possible
sequences of outcomes of the experiment?

Axiomatic definition of probability

We have earlier defined an event as a subset of the sample space. Does each subset of the
sample space forms an event?
The answer is yes for a finite sample space. However, we may not be able to assign
probability meaningfully to all the subsets of a continuous sample space. We have to
eliminate those subsets. The concept of the sigma algebra is meaningful now.

Definition Let be a sample space and a sigma field defined over it. Let be
a mapping from the sigma-algebra into the real line such that for each , there
exists a unique . Clearly is a set function and is called probability, if it satisfies
the following three axioms.

Figure 2

Discussion

 The triplet is called the probability space.

 Any assignment of probability assignment must satisfy the above three axioms

 If ,
 This is a special case of axiom 3 and for a discrete sample space , this simpler
version may be considered as the axiom 3. We shall give a proof of this result
below.

 The events A and B are called mutually exclusive .

Basic results of probability

From the above axioms we established the following basic results:

1.
Suppose,
Then

Therefore

Thus which is possible only if

2. If
We have ,

3. where where

We have,
4. If
 We have,

We can similarly show that ,

5. If
We have ,

6. We can apply the properties of sets to establish the following result for
,

The following generalization is known as the principle inclusion-exclusion.

Probability assignment in a discrete sample space

Consider a finite sample space . Then the sigma algebra is defined by the
power set of S. For any elementary event , we can assign a probability P( si ) such
that,

For any event , we can define the probability

In a special case, when the outcomes are equi-probable, we can assign equal
probability p to each elementary event.
Example 9 Consider the experiment of rolling a fair die considered in example 2.

Suppose represent the elementary events. Thus is the event of getting

‘1', is the event of getting '2' and so on.

Since all six disjoint events are equiprobable and we get ,

Suppose is the event of getting an odd face. Then

Example 10 Consider the experiment of tossing a fair coin until a head is obtained
discussed in Example 3. Here . Let us call

and so on. If we assign, then Let is the

event of obtaining the head before the 4 th toss. Then

Probability assignment in a continuous space

Suppose the sample space S is continuous and un-countable. Such a sample space
arises when the outcomes of an experiment are numbers. For example, such sample space
occurs when the experiment consists in measuring the voltage, the current or the
resistance. In such a case, the sigma algebra consists of the Borel sets on the real line.

Suppose and is a non-negative integrable function such that,

For any Borel set ,

defines the probability on the Borel sigma-algebra B .

We can similarly define probability on the continuous space of etc.

Example 11 Suppose

Then for

Probability Using Counting Method

In many applications we have to deal with a finite sample space and the
elementary events formed by single elements of the set may be assumed equiprobable. In
this case, we can define the probability of the event A according to the classical definition
discussed earlier:

where = number of elements favorable to A and n is the total number of elements

in the sample space .

Thus calculation of probability involves finding the number of elements in the

sample space and the event A. Combinatorial rules give us quick algebraic formulae to
find the elements in .We briefly outline some of these rules:

1. Product rule Suppose we have a set A with m distinct elements and the set B with

n distinct elements and . Then contains mn

ordered pair of elements. This is illustrated in Fig for m=5 and n=4 n other words
 if we can choose element a in m possible ways and the element b in n possible
ways then the ordered pair (a, b) can be chosen in mn possible ways.

Figure 1 Illustration of the product rule

The above result can be generalized as follows:

The number of distinct k -tupples in

is where
represents the number of distinct elements in .

Example 1 A fair die is thrown twice. What is the probability that a 3 will appear at least
once.

Solution: The sample space corresponding to two throws of the die is illustrated in the
following table. Clearly, the sample space has elements by the product rule.
The event corresponding to getting at least one 3 is highlighted and contains 11 elements.

Therefore, the required probability is .

Example 2 Birthday problem - Given a class of students, what is the probability of two
students in the class having the same birthday? Plot this probability vs. number of
students and be surprised!.
 Let be the number of students in the class.

The plot of probability vs number of students is shown in above table. Observe

the steep rise in the probability in the beginning. In fact this probability for a group of 25
students is greater than 0.5 and that for 60 students onward is closed to 1. This probability
for 366 or more number of students is exactly one.

Example 3 An urn contains 6 red balls, 5 green balls and 4 blue balls. 9 balls were picked
at random from the urn without replacement. What is the probability that out of the balls 4
are red, 3 are green and 2 are blue?

Solution :

9 balls can be picked from a population of 15 balls in .

Therefore the required probability is

Example 4 What is the probability that in a throw of 12 dice each face occurs twice.

Solution: The total number of elements in the sample space of the outcomes of a
single throw of 12 dice is

The number of favourable outcomes is the number of ways in which 12 dice can be
arranged in six groups of size 2 each – group 1 consisting of two dice each showing 1,
group 2 consisting of two dice each showing 2 and so on.
Therefore, the total number distinct groups

Hence the required probability is

Conditional probability

Consider the probability space . Let A and B two events in . We ask the
following question –
Given that A has occurred, what is the probability of B?

The answer is the conditional probability of B given A denoted by . We

shall develop the concept of the conditional probability and explain under what condition
this conditional probability is same as .

Let us consider the case of equiprobable events discussed earlier. Let sample
points be favourable for the joint event .
 Figure 1

Clearly ,

This concept suggests us to define conditional probability. The probability of an event B

under the condition that another event A has occurred is called the conditional probability
of B given A and defined by

We can similarly define the conditional probability of A given B , denoted by

.

From the definition of conditional probability, we have the joint probability

of two events A and B as follows
Example 1 Consider the example tossing the fair die. Suppose

Example 2 A family has two children. It is known that at least one of the children is a
girl. What is the
probability that both the children are girls?

A = event of at least one girl

B = event of two girls

Clearly,

Conditional probability and the axioms of probability

In the following we show that the conditional probability satisfies the axioms of
probability.

By definition

Axiom 1:

Axiom 2 :

We have ,
 Axiom 3 :

Consider a sequence of disjoint events .

We have ,

Figure 2

Note that the sequence is also sequence of disjoint events.

Properties of Conditional Probabilities

If , then

We have ,
 Chain Rule of Probability

We have ,

We can generalize the above to get the chain rule of probability

Joint Probability
Joint probability is defined as the probability of both A and B taking place, and is denoted
by P(AB).

Joint probability is not the same as conditional probability, though the two concepts are
often confused. Conditional probability assumes that one event has taken place or will take place,
and then asks for the probability of the other (A, given B). Joint probability does not have such
conditions; it simply asks for the chances of both happening (A and B). In a problem, to help
distinguish between the two, look for qualifiers that one event is conditional on the other
(conditional) or whether they will happen concurrently (joint).

Probability definitions can find their way into CFA exam questions. Naturally, there may also
be questions that test the ability to calculate joint probabilities. Such computations require use of
the multiplication rule, which states that the joint probability of A and B is the product of the
conditional probability of A given B, times the probability of B. In probability notation:

P(AB) = P(A | B) * P(B)

Given a conditional probability P(A | B) = 40%, and a probability of B = 60%, the joint probability
P(AB) = 0.6*0.4 or 24%, found by applying the multiplication rule.

P(AUB)=P(A)+P(B)-P(A‫ח‬B)

For independent events: P(AB) = P(A) * P(B)

Moreover, the rule generalizes for more than two events provided they are all independent of one
another, so the joint probability of three events P(ABC) = P(A) * (P(B) * P(C), again assuming
independence.

Total Probability

Let be n events such that

Then for any event B,

Proof : We have and the sequence is disjoint.

 Figure 3

Remark

(1) A decomposition of a set S into 2 or more disjoint nonempty subsets is called a

partition of [Link] subsets form a partition of S if

(2) The theorem of total probability can be used to determine the probability of a
complex event in terms of related simpler events. This result will be used in Bays'
theorem to be discussed to the end of the lecture.

Example 3 Suppose a box contains 2 white and 3 black balls. Two balls are picked at
random without replacement.

Let = event that the first ball is white and

Let = event that the first ball is black.

Clearly and form a partition of the sample space corresponding to picking

two balls from the box.

Let B = the event that the second ball is white. Then .

Bayes' Theorem
This result is known as the Baye's theorem. The probability is called the a priori
probability and is called the a posteriori probability. Thus the Bays' theorem
enables us to determine the a posteriori probability from the observation that B
has occurred. This result is of practical importance and is the heart of Baysean
classification, Baysean estimation etc.

Example 6

In a binary communication system a zero and a one is transmitted with probability

0.6 and 0.4 respectively. Due to error in the communication system a zero becomes a one
with a probability 0.1 and a one becomes a zero with a probability 0.08. Determine the
probability (i) of receiving a one and (ii) that a one was transmitted when the received
message is one.

Let S be the sample space corresponding to binary communication. Suppose be

event of transmitting 0 and be the event of transmitting 1 and and be
corresponding events of receiving 0 and 1 respectively.

Given and
Example 7: In an electronics laboratory, there are identically looking capacitors of three
makes in the ratio [Link]. It is known that 1% of , 1.5% of
are defective. What percentages of capacitors in the laboratory are defective? If a capacitor
picked at defective is found to be defective, what is the probability it is of make ?

Let D be the event that the item is defective. Here we have to

find .

Here

The conditional probabilities are

Independent events
 Two events are called independent if the probability of occurrence of one event
does not affect the probability of occurrence of the other. Thus the events A and B are
independent if

and

where and are assumed to be non-zero.

Equivalently if A and B are independent, we have

or --------------------

Two events A and B are called statistically dependent if they are not independent.
Similarly, we can define the independence of n events. The events are
called independent if and only if

Example 4 Consider the example of tossing a fair coin twice. The resulting sample space
is given by and all the outcomes are equiprobable.

Let be the event of getting ‘tail' in the first toss and be

the event of getting ‘head' in the second toss. Then

and

Again, so that

Hence the events A and B are independent.

 Example 5 Consider the experiment of picking two balls at random discussed in above
example

In this case, and .

Therefore, and and B are dependent.
Random variable

A random variable associates the points in the sample space with real numbers.

Consider the probability space and function mapping the sample

space

into the real line. Let us define the probability of a subset by

Such a definition will be valid if is a valid event. If is a discrete sample

space, is always a valid event, but the same may not be true if is infinite. The
concept of sigma algebra is again necessary to overcome this difficulty. We also need the
Borel sigma algebra -the sigma algebra defined on the real line.

The function is called a random variable if the inverse image of all

Borel sets under is an event. Thus, if is a random variable, then

Figure: Random Variable

Observations:

 is the domain of .
 The range of denoted by ,is given by
 Clearly .

• The above definition of the random variable requires that the mapping is such
that is a valid event in . If is a discrete sample space, this
requirement is met by any mapping . Thus any mapping defined on the
discrete sample space is a random variable.

Example 2 Consider the example of tossing a fair coin twice. The sample space is S={
HH,HT,TH,TT} and all four outcomes are equally likely. Then we can define a random
variable as follows

Here .

Example 3 Consider the sample space associated with the single toss of a fair die.
The sample space is given by .

If we define the random variable that associates a real number equal to the
number on the face of the die, then .

.
Discrete, Continuous and Mixed-type Random Variables

• A random variable is called a discrete random variable if is piece-wise

constant. Thus is flat except at the points of jump discontinuity. If the sample space
is discrete the random variable defined on it is always discrete.

• X is called a continuous random variable if is an absolutely continuous

function of x . Thus is continuous everywhere on and exists everywhere

except at finite or countably infinite points .
 • X is called a mixed random variable if has jump discontinuity at countable
number of points and increases continuously at least in one interval of X. For a such type
RV X,

where is the distribution function of a discrete RV, is the distribution

function of a continuous RV and o< p <1.
Typical plots of for discrete, continuous and mixed-random variables are shown
in Figure 1, Figure 2 and Figure 3 respectively.

The interpretation of and will be given later.

Figure 1 Plot of vs. for a discrete random variable

Probability Distribution Function
 UNIT-II: Distribution function &Density
function and One Random Variable-
Expectation

Probability Distribution Function

We have seen that the event and are equivalent and

.The underlying sample space is omitted in notation and we
simply write and instead of and
respectively.

Consider the Borel set , where represents any real number. The equivalent
event is denoted as .The event can be
taken as a representative event in studying the probability description of a random variable
. Any other event can be represented in terms of this event. For example,

and so on.

The probability is called the probability distribution

function ( also called the cumulative distribution function , abbreviated as CDF ) of and
denoted by . Thus
 Figure 4

Example 4: Consider the random variable in the above example. We have

Figure 5 shows the plot of FX(x)

 Figure 5

Properties of the Distribution Function


This follows from the fact that is a probability and its value should lie
between 0 and 1.

 is a non-decreasing function of . Thus, if

  Is right continuous.


.

We have ,

We can further establish the following results on probability of events on the real line:

Thus we have seen that given , we can determine the probability of

any event involving values of the random variable .Thus is a complete
description of the random variable .

Example 5 Consider the random variable defined by

Find a) .

b) .

c) .

d) .

Solution:
Figure 6 shows the plot of FX(x).

Figure 6

Discrete Random Variables and Probability DENSITY functions

A random variable is said to be discrete if the number of elements in the range is

finite or countably infinite.

First assume to be countably finite. Let be the elements of . Here

the mapping partitions into subsets .

The discrete random variable in this case is completely specified by the probability
mass function (pmf) .

Clearly,

• Suppose .Then

Figure 6 illustrates a discrete random variable.

 Figure 6 Discrete Random Variable

Example 1

Consider the random variable with the distribution function

The plot of the is shown in Figure 7 on next page.

 The probability mass function of the random variable is given by

Value of the random

pX(x)
variable X =x

Continuous Random Variables and Probability Density Functions

For a continuous random variable , is continuous everywhere. Therefore,

This implies that for

Therefore, the probability mass function of a continuous RV is zero for all .A

continuous random variable cannot be characterized by the probability mass function. A
continuous random variable has a very important chacterisation in terms of a function
called the probability density function.

If is differentiable, the probability density function ( pdf) of denoted by

is defined as

Interpretation of

so that

Thus the probability of lying in some interval is determined by .

In that sense, represents the concentration of probability just as the density
represents the concentration of mass.

Properties of the Probability Density Function

.
This follows from the fact that is a non-decreasing function
Figure 8 below illustrates the probability of an elementary interval in terms of the pdf.

Example 2 Consider the random variable with the distribution function

The pdf of the RV is given by

Remark: Using the Dirac delta function we can define the density function for a
discrete random variables.

Consider the random variable defined by the probability mass function (pmf)

The distribution function can be written as

where is the shifted unit-step function given by

 Then the density function can be written in terms of the Dirac delta function
as

Example 3
Consider the random variable defined with the distribution function given by,

Probability Density Function of Mixed Random Variable

Suppose is a mixed random variable with having jump discontinuity at

. As already stated, the CDF of a mixed random variable is given by

where is a discrete distribution function of and is a continuous distribution

function of .
The corresponding pdf is given by

where

and is a continuous pdf. We can establish the above relations as follows.

Suppose denotes the countable subset of points on such that the

random variable is characterized by the probability mass function .
Similarly, let be a continuous subset of points on such that RV is
characterized by the probability density function .
 Clearly the subsets and partition the set If , then
.
Thus the probability of the event can be expressed as

Taking the derivative with respect to x , we get

Example 4 Consider the random variable with the distribution function

The plot of is shown in Figure 9 on next page

where
 Figure 10

The pdf is given by

where

and

Example 5

X is the random variable representing the life time of a device with the PDF for
. Define the following random variable

Find FY(y).
Solution:

OTHER DISTRIBUTION AND DENSITY RVS

In the following, we shall discuss a few commonly-used discrete random variabes. The
importance of these random variables will be highlighted.

Bernoulli random variable

Suppose X is a random variable that takes two values 0 and 1, with probability mass
functions

And

Such a random variable X is called a Bernoulli random variable, because it describes the
outcomes of a Bernoulli trial.

The typical CDF of the Bernoulli RV is as shown in Figure 2

 Figure 2

Remark
We can define the pdf of with the help of Dirac delta function. Thus

Example 2 Consider the experiment of tossing a biased coin. Suppose

and .
If we define the random variable and then is a Bernoulli random
variable.

Mean and variance of the Bernoulli random variable

Remark

 The Bernoulli RV is the simplest discrete RV. It can be used as the building block
for many discrete RVs.
 For the Bernoulli RV,

Thus all the moments of the Bernoulli RV have the same value of

Binomial random variable

Suppose X is a discrete random variable taking values from the set . is

called a binomial random variable with parameters n and if

where

As we have seen, the probability of k successes in n independent repetitions of the

Bernoulli trial is given by the binomial law. If X is a discrete random variable representing
the number of successes in this case, then X is a binomial random variable. For example,
 the number of heads in ‘n ' independent tossing of a fair coin is a binomial random
variable.

 The notation is used to represent a binomial RV with the parameters

and .


 The sum of n independent identically distributed Bernoulli random variables is a
binomial random variable.
 The binomial distribution is useful when there are two types of objects - good, bad;
correct, erroneous; healthy, diseased etc.

Example 3 In a binary communication system, the probability of bit error is 0.01. If a

block of 8 bits are transmitted, find the probability that

(a) Exactly 2 bit errors will occur

(b) At least 2 bit errors will occur

(c) More than 2 bit errors will occur

(d) All the bits will be erroneous

Suppose is the random variable representing the number of bit errors in a block of
8 bits. Then

Therefore,
 The probability mass function for a binomial random variable with n = 6 and p =0.8 is
shown in the Figure 3 below.

Figure 3

Mean and Variance of the Binomial Random Variable

Where

Poisson Random Variable

A discrete random variable X is called a Poisson random variable with the parameter if
and

The plot of the pmf of the Poisson RV is shown in Figure 2

 Figure 2

Mean and Variance of the Poisson RV

The mean of the Poisson RV X is given by

 Example 3 The number of calls received in a telephone exchange follows a Poisson
distribution with an average of 10 calls per minute. What is the probability that in one-
minute duration?

i. no call is received
ii. exactly 5 calls are received
iii. More than 3 calls are received.

Solution: Let X be the random variable representing the number of calls received. Given

Where Therefore,

i. probability that no call is received 0.000095

ii. probability that exactly 5 calls are received 0.0378

iii. probability that more the 3 calls are received

0.9897

Poisson Approximation of the Binomial Random Variable

The Poisson distribution is also used to approximate the binomial distribution

when n is very large and p is small.

Consider binomial RV with with

Then
Thus the Poisson approximation can be used to compute binomial probabilities for large n.
It also makes the analysis of such probabilities easier. Typical examples are:

 number of bit errors in a received binary data file

 number of typographical errors in a printed page

Example 4 Suppose there is an error probability of 0.01 per word in typing. What is the
probability that there will be more than 1 error in a page of 120 words?

Solution: Suppose X is the RV representing the number of errors per page of 120 words.

Where Therefore,
In the following we shall discuss some important continuous random variables.

Uniform Random Variable

A continuous random variable X is called uniformly distributed over the interval [a,
b], , if its probability density function is given by

Figure 1

We use the notation to denote a random variable X uniformly distributed over

the interval

[a,b]. Also note that

Distribution function
Figure 2 illustrates the CDF of a uniform random variable.

Figure 2

Mean and Variance of a Uniform Random Variable

The characteristic function of the random variable is given by

Example 1

Suppose a random noise voltage X across an electronic circuit is uniformly distributed

between -4 V and 5 V. What is the probability that the noise voltage will lie between 2 V
and 3 V? What is the variance of the voltage?

Normal or Gaussian Random Variable

The normal distribution is the most important distribution used to model natural and
man made phenomena. Particularly, when the random variable is the result of the addition
of large number of independent random variables, it can be modelled as a normal random
variable.
 A continuous random variable X is called a normal or a Gaussian random variable with
parameters and if its probability density function is given by,

Where and are real numbers.

We write that X is distributed.

If and ,

and the random variable X is called the standard normal variable.

Figure 3 illustrates two normal variables with the same mean but different variances.

Figure 3

 Is a bell-shaped function, symmetrical about .

 Determines the spread of the random variable X . If is small X is more
concentrated around the mean .
 Distribution function of a Gaussian random variable
 Substituting , we get

where is the distribution function of the standard normal variable.

Thus can be computed from tabulated values of . The table was very
useful in the pre-computer days.

In communication engineering, it is customary to work with the Q function defined

by,

Note that and

These results follow from the symmetry of the Gaussian pdf. The function is
tabulated and the tabulated results are used to compute probability involving the Gaussian
random variable.

Using the Error Function to compute Probabilities for Gaussian Random Variables

The function is closely related to the error function and the complementary
error function .

Note that,

And the complementary error function is given by

Mean and Variance of a Gaussian Random Variable

If X is distributed, then

Proof:
Exponential Random Variable

A continuous random variable is called exponentially distributed with the

parameter if the probability density function is of the form
Figure 1 shows the typical pdf of an exponential RV.

Figure 1

Example 1

Suppose the waiting time of packets in in a computer network is an exponential RV

with

Rayleigh Random Variable

A Rayleigh random variable X is characterized by the PDF

where is the parameter of the random variable.

The probability density functions for the Rayleigh RVs are illustrated in Figure 6.

Figure 6

Mean and Variance of the Rayleigh Distribution

Similarly,
Relation between the Rayleigh Distribution and the Gaussian Distribution

A Rayleigh RV is related to Gaussian RVs as follow: If and

are independent, then the envelope has the Rayleigh

distribution with the parameter .
We shall prove this result in a later lecture. This important result also suggests the
cases where the Rayleigh RV can be used.

Application of the Rayleigh RV

 Modeling the root mean square error-

 Modeling the envelope of a signal with two orthogonal components as in the case
of a signal of the following form:

Conditional Distribution and Density functions

We discussed conditional probability in an earlier lecture. For two events A and B
with , the conditional probability was defined as

Clearly, the conditional probability can be defined on events involving a random

variable X .

Conditional distribution function

Consider the event and any event B involving the random variable X . The
conditional distribution function of X given B is defined as

We can verify that satisfies all the properties of the distribution function.
Particularly.

 And .
 .
 Is a non-decreasing function of .

Conditional Probability Density Function

In a similar manner, we can define the conditional density function of the

random variable X given the event B as

All the properties of the pdf applies to the conditional pdf and we can easily show that

Example 1 Suppose X is a random variable with the distribution function . Define

 Case 1:

Then

And

Case 2:

and
 and are plotted in the following figures.

Figure 1

Example 2 Suppose is a random variable with the distribution function and

.
 Then

For , .Therefore,

For , .Therefore,

Thus,

the corresponding pdf is given by

Example 3 Suppose X is a random variable with the probability density function

and . Then
 where and
Remark

is the standard Gaussian distribution.

is called the truncated Gaussian and plotted in Figure 3 on next
page.
Expected Value of a Random Variable
 The expectation operation extracts a few parameters of a random variable and
provides a summary description of the random variable in terms of these
parameters.
 It is far easier to estimate these parameters from data than to estimate the
distribution or density function of the random variable.
 Moments are some important parameters obtained through the expection operation.

Expected value or mean of a random variable

The expected value of a random variable is defined by

Provided exists.

Is also called the mean or statistical average of the random variable and is
denoted by
Note that, for a discrete RV with the probability mass function (pmf)
the pdf is given by

Thus for a discrete random variable with

Figure1 Mean of a random variable

Example 1

Suppose is a random variable defined by the pdf

Then
Example 2

Consider the random variable with the pmf as tabulated below

Value of the random

0 1 2 3
variable x

pX(x)

Then

Example 3 Let X be a continuous random variable with

Then

=
Hence EX does not exist. This density function is known as the Cauchy density function.

Expected value of a function of a random variable

Suppose is a real-valued function of a random variable as discussed in the

last class.
Then,

We shall illustrate the above result in the special case when is one-to-
one and monotonically increasing function of x In this case,

Figure 2

The following important properties of the expectation operation can be immediately

derived:

(a) If is a constant,
 Clearly

(b) If are two functions of the random variable and

are constants,

The above property means that is a linear operator.

MOMENTS ABOUT THE ORIGIN:

Mean-square value

MOMENTS ABOUT THE MEAN

Variance

Second central moment is called as variance

For a random variable with the pdf and mean the variance of is denoted
by and

defined as

Thus for a discrete random variable with

The standard deviation of is defined as

Example 4
 Find the variance of the random variable in the above example

Example 5

Find the variance of the random variable discussed in above example. As already
computed

For example, consider two random variables with pmf as shown below. Note

that each of has zero [Link] variances are given by and

implying that has more spread about the mean.

Properties of variance

(1)
 (2) If then

(3) If is a constant,

nth moment of a random variable

We can define the nth moment and the nth central- moment of a random variable X by
the following relations

Note that

 The mean is the first moment and the mean-square value is the
second moment
 The first central moment is 0 and the variance is the second
central moment

SKEWNESS

 The third central moment measures lack of symmetry of the pdf of a random

variable is called the coefficient of skewness and if the pdf is

symmetric this coefficient will be zero.
 The fourth central moment measures flatness or peakedness of the pdf of a random

variable. Is called kurtosis. If the peak of the pdf is sharper, then the
random variable has a higher kurtosis.

Inequalities based on expectations

 The mean and variance also give some quantitative information about the bounds of
RVs. Following inequalities are extremely useful in many practical problems.

Chebychev Inequality

Suppose a parameter of a manufactured item with known mean

The quality control department rejects the item if the absolute
deviation of from is greater than

The standard deviation gives us an intuitive idea how the random variable is
distributed about the mean. This idea is more precisely expressed in the remarkable
Chebysev Inequality stated below. For a random variable with mean

Proof:

Characteristic function

Consider a random variable with probability density function The characteristic

function of denoted by is defined as

Note the following:

  is a complex quantity, representing the Fourier transform of and
traditionally using instead of This implies that the properties of the
Fourier transform applies to the characteristic function.
 The interpretation that is the expectation of helps in calculating
moments with the help of the characteristics function. In a simple case ,

 As always non-negative and , always exists. We can

get from by the inverse transform

Example 1

Consider the random variable X with pdf given by

= 0 otherwise. The characteristics function is given by

Solution:

Example 2

The characteristic function of the random variable with

Characteristic function of a discrete random variable

Suppose X is a random variable taking values from the discrete set with
corresponding probability mass function for the value

Then,

If Rx is the set of integers, we can write

In this case can be interpreted as the discrete-time Fourier transform with

substituting in the original discrete-time Fourier transform. The inverse relation
is

Moments and the characteristic function

Given the characteristics function the nth moment is given by

To prove this consider the power series expansion of

Taking expectation of both sides and assuming to exist, we get

Taking the first derivative of with respect to at we get

Similarly, taking the derivative of with respect to at we get

Thus ,

TRANSFORMATION OF A RANDOM VARIABLE

Description:

Suppose we are given a random variable X with density fX(x). We apply a function g
to produce a random variable Y = g(X). We can think of X as the input to a black
box,and Y the output.
 UNIT-3
MULTIPLE RANDOM VARIABLES

Multiple Random Variables

In many applications we have to deal with more than two random variables. For
example, in the navigation problem, the position of a space craft is represented by three
random variables denoting the x, y and z coordinates. The noise affecting the R, G, B
channels of colour video may be represented by three random variables. In such situations,
it is convenient to define the vector-valued random variables where each component of the
vector is a random variable.

In this lecture, we extend the concepts of joint random variables to the case of
multiple random variables. A generalized analysis will be presented for random
variables defined on the same sample space.

Jointly Distributed Random Variables

We may define two or more random variables on the same sample space. Let and
be two real random variables defined on the same probability space The
mapping such that for is called a joint random variable.

Figure 1

Joint Probability Distribution Function

Recall the definition of the distribution of a single random variable. The event
was used to define the probability distribution function . Given , we
can find the probability of any event involving the random variable. Similarly, for two
random variables and , the event is considered as
the representative event.
 The probability is called the joint distribution function or
the joint cumulative distribution function (CDF) of the random variables and and
denoted by .

Figure 2

Properties of JPDF

satisfies the following properties:

1)

2)

3)

Note that

4)

5) is right continuous in both the variables.

6)

Given ,we have a complete

description of the random variables and .

7)
 To prove this

Similarly .

Given , each of is called a

marginal

Distribution function or marginal cumulative distribution function (CDF).

Jointly Distributed Discrete Random Variables

If and are two discrete random variables defined on the same probability space
such that takes values from the countable subset and takes values from
the countable subset .Then the joint random variable can take values from the
countable subset in . The joint random variable is completely specified by
their joint probability mass function

Given , we can determine other probabilities involving the random variables

and

Remark

This is because

• Marginal Probability Mass Functions: The probability mass functions and

are obtained from the joint probability mass function as follows

and similarly
 These probability mass functions and obtained from the joint probability
mass functions are called marginal probability mass functions .

Example 4 Consider the random variables and with the joint probability mass
function as tabulated in Table 1. The marginal probabilities and are as shown
in the last column and the last row respectively.

Table 1

Joint Probability Density Function

If and are two continuous random variables and their joint distribution
function is continuous in both and , then we can define joint probability density
function by

provided it exists.

Clearly

Properties of Joint Probability Density Function

• is always a non-negative quantity. That is,

• The probability of any Borel set can be obtained by

Marginal density functions

 The marginal density functions and of two joint RVs and are given
by the derivatives of the corresponding marginal distribution functions. Thus

Example 5 The joint density function of the random variables in Example 3 is

Example 6 The joint pdf of two random variables and are given by

• Find .
• Find .
• Find and .
• What is the probability ?
Conditional Distributions

We discussed the conditional CDF and conditional PDF of a random variable

conditioned on some events defined in terms of the same random variable. We observed
that

and

We can define these quantities for two random variables. We start with the conditional
probability mass functions for two random variables.
 Conditional Probability Density Functions

Suppose and are two discrete jointly random variable with the joint PMF
The conditional PMF of given is denoted by and defined as

Similarly we can define the conditional probability mass function

Conditional Probability Distribution Function

Consider two continuous jointly random variables and with the joint probability
distribution function We are interested to find the conditional distribution
function of one of the random variables on the condition of a particular value of the other
random variable.

We cannot define the conditional distribution function of the random variable on the
condition of the event by the relation

as in the above expression. The conditional distribution function is defined

in the limiting sense as follows:
Conditional Probability Density Function

is called the conditional probability density function of

given

Let us define the conditional distribution function .

The conditional density is defined in the limiting sense as follows

Because,

The right hand side of the highlighted equation is

Similarly we have

Two random variables are statistically independent if for all

 •

Example 2 X and Y are two jointly random variables with the joint pdf given by

find,

(a)
(b)
(c)

Solution:

Since

We get

Independent Random Variables (or) Statistical Independence

 Let and be two random variables characterized by the joint distribution function

and the corresponding joint density function

Then and are independent if and are independent
events. Thus,

and equivalently

Sum of Two Random Variables

We are often interested in finding out the probability density function of a function of
two or more RVs. Following are a few examples.

• The received signal by a communication receiver is given by

where is received signal which is the superposition of the message signal and the
noise .

• The frequently applied operations on communication signals like modulation,

demodulation, correlation etc. involve multiplication of two signals in the form Z = XY.

We have to know about the probability distribution of in any analysis of . More

formally, given two random variables X and Y with joint probability density function
 and a function we have to find .

In this lecture, we shall address this problem.

Probability Density of the Function of Two Random Variables

We consider the transformation

Consider the event corresponding to each z. We can find a variable subset

such that .

Figure 1

Probability density function of Z = X + Y .

Consider Figure 2
Figure 2

We have

Therefore, is the colored region in the Figure 2.

If X and Y are independent

 Where * is the convolution operation.

Example 1

Suppose X and Y are independent random variables and each uniformly distributed over
(a, b). And are as shown in the figure below.

The PDF of is a triangular probability density function as shown in the figure.

Central Limit Theorem

Consider independent random variables .The mean and
variance of each of the random variables are assumed to be known. Suppose
and . Form a random variable

The mean and variance of are given by

 and
Thus we can determine the mean and the variance of .
Can we guess about the probability distribution of ?
The central limit theorem (CLT) provides an answer to this question.

The CLT states that under very general conditions converges in distribution
to as . The conditions are:

1. The random variables are independent and identically distributed.

2. The random variables are independent with same mean and
variance, but not identically distributed.
3. The random variables are independent with different means and
same variance and not identically distributed.
4. The random variables are independent with different means and
each variance being neither too small nor too large.

We shall consider the first condition only. In this case, the central-limit theorem can be
stated as follows:

Proof of the Central Limit Theorem:

We give a less rigorous proof of the theorem with the help of the characteristic
function. Further we consider each of to have zero mean. Thus,

Clearly,
The characteristic function of is given by
We will show that as the characteristic function is of the form of the
characteristic function of a Gaussian random variable.
Expanding in power series

Assume all the moments of to be finite. Then

Substituting

where is the average of terms involving and higher powers of .

Note also that each term in involves a ratio of a higher moment and a power of
and therefore,

which is the characteristic function of a Gaussian random variable with 0 mean and
variance .
Expected Values of Functions of Random Variables

If is a function of a continuous random variable then

If is a function of a discrete random variable then

Suppose is a function of continuous random variables then

the expected value of is given by

Thus can be computed without explicitly determining .

We can establish the above result as follows.

Suppose has roots at . Then

Where

Is the differential region containing The mapping is illustrated in Figure 1

for .
 Figure 1

Note that

As is varied over the entire axis, the corresponding (non-overlapping) differential

regions in plane cover the entire plane.

Thus,

If is a function of discrete random variables , we can similarly show

that

Example 1 The joint pdf of two random variables is given by

 Find the joint expectation of

Example 2 If

Proof:

Thus, expectation is a linear operator.

Example 3

Consider the discrete random variables discussed in Example 4 in lecture

[Link] joint probability mass function of the random variables are tabulated in Table .
Find the joint expectation of .
Remark

(1) We have earlier shown that expectation is a linear operator. We can generally
write

Thus
(2) If are independent random variables and ,then

Joint Moments of Random Variables

Just like the moments of a random variable provide a summary description of the random
variable, so also the joint moments provide summary description of two random variables.
For two continuous random variables , the joint moment of order is defined
as

And the joint central moment of order is defined as

 where and

Remark
(1) If are discrete random variables, the joint expectation of order and
is defined as

(2) If and , we have the second-order moment of the random variables

given by

(3) If are independent,

Covariance of two random variables

The covariance of two random variables is defined as

Cov(X, Y) is also denoted as .

Expanding the right-hand side, we get

The ratio is called the correlation coefficient.

If then are called positively correlated.

If then are called negatively correlated

 If then are uncorrelated.

We will also show that To establish the relation, we prove the following
result:

For two random variables

Proof:

Consider the random variable

Non-negativity of the left-hand side implies that its minimum also must be nonnegative.

For the minimum value,

so the corresponding minimum is

Since the minimum is nonnegative,

Now
 Thus

Uncorrelated random variables

Two random variables are called uncorrelated if

Recall that if are independent random variables, then

then

Thus two independent random variables are always uncorrelated.

Note that independence implies uncorrelated. But uncorrelated generally does not
imply independence (except for jointly Gaussian random variables).

Joint Characteristic Functions of Two Random Variables

The joint characteristic function of two random variables X and Y is defined by

If and are jointly continuous random variables, then

 Note that is same as the two-dimensional Fourier transform with the basis

function instead of

is related to the joint characteristic function by the Fourier inversion formula

If and are discrete random variables, we can define the joint characteristic function in
terms of the joint probability mass function as follows:

Properties of the Joint Characteristic Function

The joint characteristic function has properties similar to the properties of the chacteristic
function of a single random variable. We can easily establish the following properties:

1.
2.
3. If and are independent random variables, then

4. We have,
 Hence,

In general, the order joint moment is given by

Example 2 The joint characteristic function of the jointly Gaussian random variables
and with the joint pdf

Let us recall the characteristic function of a Gaussian random variable

If and is jointly Gaussian,

we can similarly show that

We can use the joint characteristic functions to simplify the probabilistic analysis as
illustrated on next page:

Jointly Gaussian Random Variables

Many practically occurring random variables are modeled as jointly Gaussian random
variables. For example, noise samples at different instants in the communication system
are modeled as jointly Gaussian random variables.

Two random variables are called jointly Gaussian if their joint probability

density
The joint pdf is determined by 5 parameters

 means
 variances
 correlation coefficient

We denote the jointly Gaussian random variables and with these parameters as

The joint pdf has a bell shape centered at as shown in the Figure 1 below. The
variances determine the spread of the pdf surface and determines the
orientation of the surface in the plane.

Figure 1 Jointly Gaussian PDF surface

Properties of jointly Gaussian random variables

(1) If and are jointly Gaussian, then and are both Gaussian.

We have

Similarly
 (2) The converse of the above result is not true. If each of and is Gaussian, and
are not necessarily jointly Gaussian. Suppose

in this example is non-Gaussian and qualifies to be a joint pdf. Because,

And

The marginal density is given by

Similarly,

Thus and are both Gaussian, but not jointly Gaussian.

(3) If and are jointly Gaussian, then for any constants and ,the random
variable given by is Gaussian with mean and variance
 (4) Two jointly Gaussian RVs and are independent if and only if and are
uncorrelated .Observe that if and are uncorrelated, then

Example 1 Suppose X and Y are two jointly-Gaussian 0-mean random variables with
variances of 1 and 4 respectively and a covariance of 1. Find the joint PDF

We have

Example 2 Linear transformation of two random variables

Suppose then

If and are jointly Gaussian, then

Which is the characteristic function of a Gaussian random variable with

mean and variance

thus the linear transformation of two Gaussian random variables is a Gaussian

random variable.
 Example 3 If Z = X + Y and X and Y are independent, then

Using the property of the Fourier transform, we get

Hence proved.

Univariate transformations
When working on the probability density function (pdf) of a random variable X, one
is often led to create a new variable Y defined as a function f(X) of the original variable
X. For example, if X~N(µ, ²), then the new variable:

Y = f(X) = (X - µ)/
Is N (0, 1).

It is also often the case that the quantity of interest is a function of another (random)
quantity whose distribution is known. Here are a few examples:
*Scaling: from degrees to radians, miles to kilometers, light-years to parsecs, degrees
Celsius to degrees Fahrenheit, linear to logarithmic scale, to the distribution of the
variance
* Laws of physics: what is the distribution of the kinetic energy of the molecules of a
gas if the distribution of the speed of the molecules is known ?

So the general question is:

* If Y = h(X),
* And if f(x) is the pdf of X,

Then what is the pdf g(y) of Y?

TRANSFORMATION OF A MULTIPLE RANDOM VARIABLES

Multivariate transformations

The problem extends naturally to the case when several variables Yj are defined from
several variables Xi through a transformation y = h(x).
Here are some examples:

Rotation of the reference frame

Let f(x, y) be the probability density function of the pair of r.v. {X, Y}. Let's
rotate the reference frame {x, y} by an angle . The new axes {x', y'} define two new
r. v. {X', Y'}. What is the joint probability density function of {X', Y'}?
 Polar coordinates
Let f(x, y) be the joint probability density function of the pair of r. v. {X, Y},
expressed in the Cartesian reference frame {x, y}. Any point (x, y) in the plane can
also be identified by its polar coordinates (r, ). So any realization of the pair {X, Y}
produces a pair of values of r and , therefore defining two new r. v. R and .
What is the joint probability density function of R and? What are the (marginal)
distributions of R and of ?

Sampling distributions
Let f(x) is the pdf of the r. v. X. Let also Z1 = z1(x1, x2... xn) be a statistic, e.g.
the sample mean. What is the pdf of Z1?
Z1 is a function of the n r. v. Xi (with n the sample size), that are lid with pdf f(x). If it
is possible to identify n - 1 other independent statistics Zi, i = 2... n, then a
transformation Z = h(X) is defined, and g(z), the joint distribution of Z = {Z1, Z2, ...,
Zn} can be calculated. The pdf of Z1 is then calculated as one of the marginal
distributions of Z by integrating g(z) over zi , i = 2, .., n.

Integration limits
Calculations on joint distributions often involve multiple integrals whose
integration limits are themselves variables. An appropriate change of variables
sometimes allows changing all these variables but one into fixed integration limits,
thus making the calculation of the integrals much simpler.
Linear Transformations of Random Variables

A linear transformation is a change to a variable characterized by one or more of

the following operations: adding a constant to the variable, subtracting a constant from the
variable, multiplying the variable by a constant, and/or dividing the variable by a constant.

When a linear transformation is applied to a random variable, a new random

variable is created. To illustrate, let X be a random variable, and let m and b be constants.
Each of the following examples show how a linear transformation of X defines a new
random variable Y.

 Adding a constant: Y = X + b
 Subtracting a constant: Y = X - b
 Multiplying by a constant: Y = mX
 Dividing by a constant: Y = X/m
 Multiplying by a constant and adding a constant: Y = mX + b
 Dividing by a constant and subtracting a constant: Y = X/m - b
Suppose the vector of random variables has the joint
distribution . Set for some square matrix and vector . If
then has the joint distribution

Indeed, suppose (this is the notation for "the is the distribution density of ")
and . For any domain of the space we can

write

We make the change of variables

in the last integral.

(Linear
transformation of
random variables)

The linear transformation is distributed as . The was defined in the

section ( Definition of normal variable).

For two independent standard normal variables (s.n.v.) and the combination

is distributed as .

A product of normal variables is not a normal variable. See the section on the chi-squared
distribution.
 UNIT 4
STOCHASTIC PROCESSES-TEMPORAL
CHARACTERISTICS
Random Processes

In practical problems, we deal with time varying waveforms whose value at a time
is random in nature. For example, the speech waveform recorded by a microphone, the
signal received by communication receiver or the daily record of stock-market data
represents random variables that change with time. How do we characterize such data?
Such data are characterized as random or stochastic processes. This lecture covers the
fundamentals of random processes.

Recall that a random variable maps each sample point in the sample space to a point
in the real line. A random process maps each sample point to a waveform.

Consider a probability space . A random process can be defined on

as an indexed family of random variables where is an index
set, which may be discrete or continuous, usually denoting time. Thus a random process is
a function of the sample point and index variable and may be written as .

Example 1 Consider a sinusoidal signal where is a binary random

variable with probability mass functions and
 Clearly, is a random process with two possible realizations
and At a particular time is a random variable
with two values and .

Classification of a Random Process

a) Continuous-time vs. Discrete-time process

If the index set is continuous, is called a continuous-time

process.

If the index set is a countable set, is called a discrete-time process. Such

a random process can be represented as and called a random sequence.
Sometimes the notation is used to describe a random sequence indexed by the
set of positive integers.

We can define a discrete-time random process on discrete points of time.

Particularly, we can get a discrete-time random process by sampling a
continuous-time process \ at a uniform interval such that

The discrete-time random process is more important in practical implementations.

Advanced statistical signal processing techniques have been developed to process this type
of signals.

b) Continuous-state vs. Discrete-state process

The value of a random process is at any time can be described from its
probabilistic model.

The state is the value taken by at a time , and the set of all such states is called
the state space. A random process is discrete-state if the state-space is finite or countable.
It also means that the corresponding sample space is also finite or countable. Otherwise ,
the random process is called continuous state.

Firtst order and nth order Probability density function and Distribution functions

As we have observed above that at a specific time is a random variable and

can be described by its probability distribution function This
distribution function is called the first-order probability distribution function.
We can similarly define the first-order probability density function
To describe , we have to use joint distribution function of the random
variables at all possible values of . For any positive integer ,
represents jointly distributed random variables. Thus a random
process can thus be described by specifying the joint
distribution function .

or th the joint probability density function

If is a discrete-state random process, then it can be also specified by the

collection of joint probability mass function

Moments of a random process

We defined the moments of a random variable and joint moments of random variables. We
can define all the possible moments and joint moments of a random process .
Particularly, following moments are important.

• Mean of the random process at

Note that

• The autocovariance function of the random process at time is defined

by

These moments give partial information about the process.

The ratio is called the correlation coefficient.

The autocorrelation function and the autocovariance functions are widely used to
characterize a class of random process called the wide-sense stationary process.

We can also define higher-order moments like

The above definitions are easily extended to a random sequence .

On the basis of the above definitions, we can study the degree of dependence between two
random processes

This also implies that for such two processes

Orthogonal processes: Two random processes and

are called orthogonal if

Stationary Random Process

The concept of stationarity plays an important role in solving practical problems involving
random processes. Just like time-invariance is an important characteristics of many
deterministic systems, stationarity describes certain time-invariant property of a class of
random processes. Stationarity also leads to frequency-domain description of a random
process.

Strict-sense Stationary Process

A random process is called strict-sense stationary (SSS) if its probability structure
is invariant with time. In terms of the joint distribution function, is called SSS if

Thus, the joint distribution functions of any set of random variables

does not depend on the placement of the origin of the time axis. This
requirement is a very strict. Less strict form of stationarity may be defined.

Particularly,

If then
is called order stationary.

Is called order stationary does not depend on the placement of the origin of the
time axis. This requirement is a very strict. Less strict form of stationary may be defined.

 If is stationary up to order 1

Let us assume Then

As a consequence

 If is stationary up to order 2

Put

As a consequence, for such a process

Similarly,

Therefore, the autocorrelation function of a SSS process depends only on the time lag

We can also define the joint stationary of two random processes. Two processes

And are called jointly strict-sense stationary if their joint probability

distributions of any order is invariant under the translation of time. A complex random
process is called SSS if and are jointly SSS.

Example 1 A random process is SSS.

This is because

Wide-sense stationary process

It is very difficult to test whether a process is SSS or not. A subclass of the SSS process
called the wide sense stationary process is extremely important from practical point of
view.

A random process is called wide sense stationary process (WSS) if

Remark

(1) For a WSS process

(2) An SSS process is always WSS, but the converse is not always true.

Example 3 Sinusoid with random phase

Consider the random process given by

where are constants and are unifirmly distributed

between

This is the model of the carrier wave (sinusoid of fixed frequency) used to analyse
the noise performance of many receivers.

Note that

By applying the rule for the transformation of a random variable, we get

Which is independent of Hence is first-order stationary.

Note that
and

Hence is wide-sense stationary

Properties of Autocorrelation Function of a Real WSS Random Process

Autocorrelation of a deterministic signal

Consider a deterministic signal such that

Such signals are called power signals. For a power signal the autocorrelation function
is defined as

Measures the similarity between a signal and its time-shifted version.

Particularly, is the mean-square value. If is a voltage

waveform across a 1 ohm resistance, then is the average power delivered to the
resistance. In this sense, represents the average power of the signal.

Example 1 Suppose The autocorrelation function of at lag is given

by
We see that of the above periodic signal is also periodic and its maximum occurs

when The power of the signal is

The autocorrelation of the deterministic signal gives us insight into the properties of the
autocorrelation function of a WSS process. We shall discuss these properties next.

Properties of the autocorrelation function of a real WSS process

Consider a real WSS process Since the autocorrelation function of such

a process is a function of the lag we can redefine a one-parameter
autocorrelation function as

If is a complex WSS process, then

Where is the complex conjugate of For a discrete random sequence, we can

define the autocorrelation sequence similarly.

The autocorrelation function is an important function charactering a WSS random process.

It possesses some general properties. We briefly describe them below.

1. Is the mean-square value of the process? Thus,

Remark If is a voltage signal applied across a 1 ohm resistance, and then is

the ensemble average power delivered to the resistance.

2. For a real WSS process is an even function of the time Thus,

Because,
Remark For a complex process

3. This follows from the Schwartz inequality

We have

4. is a positive semi-definite function in the sense that for any positive integer

and real ,

Proof

Define the random variable

It can be shown that the sufficient condition for a function to be the autocorrelation
function of a real WSS process is that be real, even and positive semidefinite.

If is MS periodic, then . Is also periodic with the same period.

Proof: Note that a real WSS random process is called mean-square periodic ( MS
periodic ) with a period if for every
Again

Cross correlation function of jointly WSS processes

If and are two real jointly WSS random processes, their cross-correlation
functions are independent of and depends on the time-lag. We can write the cross-
correlation function

The cross correlation function satisfies the following properties:

We Have

Further,
 iii. If and Y (t) are uncorrelated,
iv. If X ( t ) and Y (t) are orthogonal processes,

Example 2

Consider a random process which is sum of two real jointly WSS random
processes.

We have

If and are orthogonal processes,then

Example 3

Suppose

Where X (t) is a WSS process and

Time averages and Ergodicity

Often we are interested in finding the various ensemble averages of a random

process by means of the corresponding time averages determined from single
realization of the random process. For example we can compute the time-mean of a single
realization of the random process by the formula

which is constant for the selected realization. Note that represents the dc value of
.

Another important average used in electrical engineering is the rms value

given by

Time averages of a random process

The time-average of a function of a continuous random process is defined

by

where the integral is defined in the mean-square sense.

Similarly, the time-average of a function of a continuous random process is

defined by

The above definitions are in contrast to the corresponding ensemble average defined by

The following time averages are of particular interest

(a) Time-averaged mean

(b) Time-averaged autocorrelation function

 Note that, and are functions of random variables and are governed
by respective probability distributions. However, determination of these distribution
functions is difficult and we shall discuss the behaviour of these averages in terms of their
mean and variances. We shall further assume that the random processes and
are WSS.

Mean and Variance of the Time Averages

Let us consider the simplest case of the time averaged mean of a discrete-time WSS
random process given by

The mean of

and the variance

If the samples are uncorrelated,

We also observe that

From the above result, we conclude that

Let us consider the time-averaged mean for the continuous case. We have

and the variance

The above double integral is evaluated on the square area bounded by and
We divide this square region into sum of trapezoidal strips parallel to
(See Figure 1)Putting and noting that the differential area between
and is , the above double integral is converted to a
single integral as follows:
 Figure 1

Ergodicity Principle

If the time averages converge to the corresponding ensemble averages in the probabilistic
sense, then a time-average computed from a large realization can be used as the value for
the corresponding ensemble average. Such a principle is the ergodicity principle to be
discussed below:

Mean ergodic process

A WSS process is said to be ergodic in mean, if as . Thus

for a mean ergodic process ,

We have earlier shown that

and

therefore, the condition for ergodicity in mean is

Further,

Therefore, a sufficient condition for mean ergodicity is

Example 1 Consider the random binary waveform discussed in Example 5 of

lecture [Link] process has the auto-covariance function given by

Here

hence is mean ergodic.

Autocorrelation ergodicity
 We consider so that,
Then will be autocorrelation ergodic if is mean ergodic.
Thus will be autocorrelation ergodic if

where

involves fourth order moment.

Simpler condition for autocorrelation ergodicity of a jointly Gaussian process can be

found.

Example 2

Consider the random–phased sinusoid given by

where are constants and is a random variable.

We have earlier proved that this process is WSS with and

For any particular realization

and

We see that as and

For each realization, both the time-averaged mean and the time-averaged autocorrelation
function converge to the corresponding ensemble averages. Thus the random-phased
sinusoid is ergodic in both mean and autocorrelation.

UNIT 5
STOCHASTIC PROCESSES—
SPECTRAL CHARACTERISTICS
Definition of Power Spectral Density of a WSS Process

Let us define the truncated random process as follows

where is the unity-amplitude rectangular pulse of width centering the

origin. As will represent the random process define the mean-
square integral

Applying the Pareseval's theorem we find the energy of the signal

Therefore, the power associated with is

And
 The average power is given by

Where the contribution to the average is power at frequency w and

represents the power spectral
density of . As , the left-hand side in the above expression represents the
average power of
Therefore, the PSD of the process is defined in the limiting sense by

Relation between the autocorrelation function and PSD: Wiener-Khinchin-Einstein

theorem

We have

Figure 1

Note that the above integral is to be performed on a square region bounded by

 and as illustrated in Figure [Link] so that is a
family of straight lines parallel to The differential area in terms of is given by
the shaded area and equal to The double integral is now replaced by a single
integral in

Therefore,

If is integral then the right hand integral converges to as

As we have noted earlier, the power spectral density is the

contribution to the average
power at frequency and is called the power spectral density of . Thus ,

and using the inverse Fourier transform

Example 1 The autocorrelation function of a WSS process is given by

Find the power spectral density of the process.

The autocorrelation function and the PSD are shown in Figure 2

Figure 2

Example 3 Find the PSD of the amplitude-modulated random-phase sinusoid

Where M(t) is a WSS process independent of

Figure 4 illustrates the above result.

Figure 4
Properties of the PSD

being the Fourier transform of it shares the properties of the Fourier

transform. Here we discuss
important properties of

1) the average power of a random process is

2) If is real, is a real and even function of .Therefore,

Thus for a real WSS process, the PSD is always real.

3) Thus is a real and even function of .

4) From the definition is always non-negative. Thus

5) If has a periodic component, is periodic and so will have impulses.

Cross Power Spectral Density

Consider a random process which is sum of two real jointly WSS random
processes As we have seen earlier,

If we take the Fourier transform of both sides,

 Where stands for the Fourier transform.

Thus we see that includes contribution from the Fourier transform of the cross-
correlation functions
These Fourier transforms represent cross power spectral densities.

Definition of Cross Power Spectral Density

Given two real jointly WSS random processes the cross power spectral
density (CPSD) is defined as

Where are the Fourier transform of the truncated processes

respectively and denotes the complex

conjugate operation.

We can similarly define by

Proceeding in the same way as the derivation of the Wiener-Khinchin-Einstein theorem for
the WSS process, it
can be shown that

and

The cross-correlation function and the cross-power spectral density form a Fourier
transform pair and we can
write

and
Properties of the CPSD

The CPSD is a complex function of the frequency ’w’. Some properties of the CPSD of
two jointly WSS processes
are listed below:

(1)

Note that

(2) is an even function of and is an odd function of .

We have

(3) If are uncorrelated and have constant means, then

Where is the Dirac delta function?

Observe that
(4) If are orthogonal, then

If are orthogonal, we have

(5) the cross power between is defined by

Applying Parseval's theorem, we get

Similarly,

Example 1 Consider the random process given by discussed in the

beginning of the lecture. Here is the sum of two jointly WSS orthogonal random
processes

We have,

Taking the Fourier transform of both sides,

Wiener-Khinchin-Einstein theorem

The Wiener-Khinchin-Einstein theorem is also valid for discrete-time random processes.

The power spectral density of the WSS process is the discrete-time Fourier
transform of autocorrelation sequence.

Is related to by the inverse discrete-time Fourier transform and given by

Thus and forms a discrete-time Fourier transform pair. A generalized PSD

can be defined in terms of as follows
clearly,

PREVIOUS UNIVERSITY QUESTION

PAPERS
 SET 1

1. (a) State and prove Bayes Theorem of probability.

(b) In a single throw of two dice, what is the probability of obtaining a sum of at
least 10.

2. (a) Define the joint distribution function. Explain how marginal density
functions are computed given their joint distribution functions.

(b) A product is classified according to the number of defects it contains (X1) and
the factory that produces it (X2). The joint probability distribution is given by :

X2 → 1 2
X1 ↓

0 1/8 1/16
1 1/16 1/16
2 3/16 1/8
3 1/8

i) Find the marginal distribution of X1.

ii) Find the conditional distribution of X1 when X2 is equal to 1.
iii) Are the variables X1 and X2 independent.

3. (a) Show that mean of the binomial distribution is the product of the parameters
p & the number of times n.

(b) Sketch the probability density function & Probability distribution function of

i) Exponential distribution
ii) Uniform Distribution
iii) Gaussian Distribution

4. (a) State the condition for wide sense stationary Random process.
 (b) Find the Auto Correlation function for white noise shown in the figure 4b
below :

-t0/2 t0/2

Figure 4b

5. (a) Derive the relation between PSDs of input and output random process of an
LTI system.

(b) X(t) is a stationary random process with zero mean and auto correlation R XX(
t ) e-2| t | is applied to a system of function H(w) = 1/(jw+2). Find the mean
and PSD of its output.

6. (a) What are the causes of thermal noise?

(b) What are the causes of short noise?

7. (a) What are the important parameters that determine the overall noise figure of a
multistage filtering?
(b) Bring out the importance of Frii’s formula.

8. (a) A code is composed of dots and dashes . Assume that a dash is three times as
long as the dot and has one-third the probability of occurrence. Find :

i) The information in a dot and that in a dash , and

ii) The entropy in the dot-dash code.

(b)Suppose 100 voltage levels are employed to transmit 100 equally likely
messages .Assume the system to be Gaussian channel with λ =3.5 and
bandwidth B= 104 Hz. Find S/N.
 SET 2

1 . (a) If A and B are any events, not necessarily mutually exclusive events, derive an
expression for probability of A Union B. When A and B are mutually
exclusive, ehat happens to the above expression derived.

(b) Define the term Independent events .State the conditions for independence of :

i. any two events A and B

ii. any three events A, B and C

(c) A coin is tossed .If it turns up heads, two balls will be drawn from Box A,
otherwise, two
balls will be drawn from Box B. Box A contains three black and five white balls. Box B
contains seven black and one white ball. In both cases,selctions are to be made with
replacement. What is the probability that Box A is used ,given that both balls are black?

2. (a) Two discrete random variables X and Y have joint p.m.f. given by the following
table

X↓ 1 2 3 Y
←

1 1/12 1/6 1/12

2 1/6 1/4 1/12
3 1/12 1/12 0

(b) Compute the probability of each of the following events :

i. X≤ 11/2
ii .XY is even
iii .Y is even given that X is even.

4. (a) State and prove any four properties of mean of a random variable X.

(b) Prove that the density function of sum of two statistically independent random
variables is the convolution of their individual density functions.

5. A class of modulation signal is modulated by

Xc(t) = AX(t)Cos(wct+θ)
 Where X(t) is the message signal and Cos(Wct+θ) is the carrier. The message Signal
X(t) is modulated as a zero mean stationary random process with the auto
correlation function Rxx(Τ) and the PSD Gx(f).The carrier amplitude A and
frequency Wc are assumed to be constants and the initial carrier phase θ is
assumed to be a random variable uniformly distributed in the interval (-
∏,∏).Further more X(t) and θ are assumed to be independent.

(a) Show that Xc(t)is stationary

(b) Find the PSD of Xc(t)

5. (a) Derive the relation between PSDs of input and output random process of an LTI
system.

(b) X(t) is a stationary random process with zero mean and auto correlation R xx(T)e-2|T|

.
is applied to a system of function Find mean and PSD of its
output.

6. (a) Explain the external noise sources of random noise.

(b) Calculate the rms noise voltage generated in a bandwidth if 15 kHz by a resistor of
2kΩ operating at [Link] the noise power over this bandwidth. Find the noise
PSD.

7. In TV receivers, the antenna is often mounted on a tall mask and a long lossy cable is
used to connect the antenna and receiver. To overcome the effect of noisy cable, a
preamplifier is mounted on the antenna. The parameters of the different stages are:
Preamplifier gain = 20dB
Preamplifier Noise Figure = 6dB
Lossy cable Noise figure = 3dB
Cable Loss = -20dB
Receiver front end gain = 60dB
Receiver Noise cable = 16dB
Determine the overall noise figure of the system.
8. (a) A code is composed of dots and [Link] that a dash is three times as long as
the dot and has one-third the probability of occurrence.
Find:

i. The information in a dot and that in a dash,and

ii. The entropy in the dot-dash code.

(b) Suppose 100 voltage levels are employed to transmit 100 equally likely messages.
Assume the system to be a Gaussian channel with λ=3.5 and bandwidth B=
[Link] S/N.
 SET 3
1. (a) Define Probability density function and obtain the relationship between
probability and probability density.
(b) Consider the probability density f(x) = ae -b|x| where x is a random variable
Whose allowable values range from .Find:
i. the CDF
ii. The relationship between a and b and
iii. The probability that the outcome X lies between 1 and 2.
2. (a) Derive an expression for the average value and variance associated with the
Gaussian probability density function.
(b) The average life of a certain type of electric bulb is Rs.1200 hours. What
percentage of this type of bulbs is expected to fail in the first 800 hours of
working? What percentage is expected to fail between 800 is 1000 hours? Assume
a normal distribution with

3. (a) State and prove any four properties of characteristic function.

(b) Find the density function of the distribution for which the characteristic
function is
e-(t2 2) /2.

4. X(t) is zero mean stationary Random process with an Auto correction function Rxx
( ). By integration X(t) form a Random variable Y as

Show that x = 0 and ) Rxx

( )dr

5. (a)Derive the relation between PSDs of input and output random process of an LTI
system.

(b) X(t) is a stationary random process with zero means and auto correlation
Rxx ( )e-2|T| is appied to a system of function .Find mean and
PSD of its output.

6. Write short notes on

(a)Flicker Noise
(b) Partition Noise
 (c) Johnson’s Noise

7. (a) What are the precautions to be taken in cascading stages of a network in the
point of view of noise reduction?

(b) What is the need for band limiting the signal towards the direction increasing
SNR.

8. (a) Obtain the Shannon –Hartley law giving the relation amongst channel capacity,
bandwidth and signal to noise ratio of a continuous system.

(b) Consider a message sequence having alphabets Q1, Q2, Q3 and Q4 with
probabilities 1/2, 1/4, 1/8 and 1/8 respectively.

i. Calculate the entropy of the message sequence.

ii. Find the information rate of the message rate is 1message/second
iii. What is the rate at which binary signals are transmitted if the signal is
sent
after encoding Q1, Q2, Q3 and Q4 is 00, 01, 10, 11.
SET 4

1. (a)Distinguish between mutually exclusive events and independent events.

(b)A letter is known to have come either from LONDON or [Link] the
postmark only the two consecutive letters ‘ON’ are [Link] is the Chance that
it came from London? Give step-by-step answer.

(c) Show that the chances ofthrowing six with 4.3 or 2 dice respectively are as
[Link].

2. (a) Derive an expression for the average value and variance associated with the
Gaussian probability density function.

(b) The average life of a certain type of electric bulb is Rs.1200 hours. What
percentage of this type of bulbs is expected to fail in the first 800 hours of
working? What percentage is expected to fail between 800 is 1000 hours? Assume
a normal distribution with

3. (a) State and Prove any four properties of mean of a random variable X.

(b) Prove that the density function of sum of two statistically independent random
variables is the convolution of their individual density functions.

4. (a) If the auto correlation function of a wss process is R ( )=k.e( -kт) ,Show that its

spectral density is given by Note:{ =(

(b)Find the PSD of a random process X(t) if E[X(t)]=1 and R xx ( )=1+e-α|T|.

 5. A Random process n(t) has power spectral density g(f)= for . Random
process is passed through a low pass filter which has transfer function H(f)=2 for
-fm f fm and H(f) =0 otherwise. Find the PSD of the waveform at the o/p of the
filter.

6. (a) Calculate the equivalent noise bandwidth of an RC Low pass filter. How it is
related to its 3dB bandwidth.

(b) Find the PSD of the noise voltage across the terminals 1 & 2 for the following
ckt figure 6b:

1
1Ω 1F
1Ω 1H

2
Figure 6b

7. (a)Evaluate the equivalent noise temperature of a two porr device with a matched
source and a matched load.

(b) Bring out the significance of noise figure in the determining the performance of
communication system.

8. (a) Describe the channel capacity of a discrete channel.

(b)Explain Shannon-Fano algorithm to develop a code to increase average

formation per bit.

SET 1

1. (a) Discuss Joint and conditional probability.

(b) When are two events said to be mutually exclusive? Explain with an example?
(c) Determine the probability of the card being either red or a king when one card
is drawn from a regular deck of 52 cards. [6+6+4]

2. (a) Define rayleigh density and distribution function and explain them with their
plots.
(b) Define and explain the guassian random variable in brief?
(c) Determine whether the following is a valid distrbution function. F(x) = 1-
exp(-x/2) for x ) 0 and 0 elsewhere. [5+5+6]
3. (a) State and prove properties of characteristic function of a random variable X
(b) Let X be a random variable defined by the density function
fX(x) = _ 5
4 (1 − x4) 0 < x _ 1
0 elsewhere
. Find E[X] ,E[X2] and variance. [8+8]

4. The joint space for two random variables X and Y and corresponding probabilities
are shown in table
Find and Plot
(a) FXY (x, y)
(b) marginal distribution functions of X and Y.
(c) Find P(0.5 < X < 1.5),
(d) Find P(X _ 1,Y _ 2) and
(e) Find P(1 < X _ 2,Y _ 3).
X, Y 1,1 2,2 3,3 4,4
P 0.05 0.35 0.45 0.15
[3+4+3+3+3]
5. (a) Show that the variance of a weighted sum of uncorrected random variables
equals the weighted sum of the variances of the random variables.
(b) Two random variables X and Y have joint characteristic function
φX, Y(ω1,ω2) = exp(−2ω21−8ω22).
i. Show that X and Y are zero mean random variables.
ii. are X and Y are correlated. [8+8]

6. Let X(t) be a stationary continuous random process that is differentiable. Denote

its time derivative by X˙ (t).
(a) Show that E h •× (t)i = 0.
(b) Find R× ˙× (τ ) in terms of R×× (τ )sss [8+8]

7. (a) Derive the expression for PSD and ACF of band pass white noise and plot
them
(b) Define various types of noise and explain. [8+8]

8. (a) Define the following random processes

i. Band Pass
ii. Band limited
iii. Narrow band. [3×2 = 6]
(b) A Random process X(t) is applied to a network with impulse response h(t) =
u(t) exp (-bt) where b > 0 is ω constant. The Cross correlation of X(t) with the
output Y(t) is known to have the same form:
RXY (τ ) = u(τ )τ exp (-bY)
i. Find the Auto correlation of Y(t)
ii. What is the average power in Y(t). [6+4]

SET 2
1. (a) With an example define and explain the following:
i. Equality likely events
 ii. Exhaustive events.
iii. Mutually exclusive events.
(b) In an experiment of picking up a resistor with same likelihood of being picked
up for the events; A as “draw a 47 resistor”, B as “draw a resistor with 5%
Tolerance” and C as “draw a 100 resistor” from a box containing 100. Resistors
having resistance and tolerance as shown below. Determine joint Probabilities and
conditional probabilities. [6+10]
Resistance() Tolerance
5% 10% Total
22 10 14 24
47 28 16 44
100 24 8 32
Total 62 38 100
2. (a) What is binomial density function? Find the equation for binomial distribution
function.
(b) What do you mean by continuous and discrete random variable? Discuss the
condition for a function to be a random variable. [6+10]

3. (a) Define moment generating function.

(b) State properties of moment generating function.
(c) Find the moment generating function about origin of the Poisson distribution.
[3+4+9]
4. (a) Define conditional distribution and density function of two random variables
X and Y
(b) The joint probability density function of two random variables X and Y is
given by f(x, y) = _ a(2x + y2) 0 _ x _ 2 , 2 _ y _ 4 0 elsewhere
Find
i. value of ‘a’
ii. P(X _ 1,Y > 3). [8+8]

5. (a) let Xi, i = 1,2,3,4 be four zero mean Gaussian random variables. Use the joint
characteristic function to show that E {X1 X2 X3 X4} = E[X1 X2] E[X3 X4]
+ E[X1X3]E[X2X4] + E[X2X3] E[X1X4]
(b) Show that two random variables X1 and X2 with joint pdf.
fX1X2(X1, X2) = 1/16 |X1|< 4, 2 < X2< 4 are independent and orthogonal.[8+8]

6. A random process Y (t) = X (t) - X (t +τ) is defined in terms of a process X(t) that
is at least wide sense stationary.
(a) Show that mean value of Y (t) is 0 even if X(t) has a non Zero mean value.
(b) Show that σY2= 2[RXX (0) − RXX (τ )]
(c) If Y (t) = X (t) +X (t + τ) find E[Y (t)] and σY 2. [5+5+6]

7. (a) If the PSD of X(t) is Sxx(ω ). Find the PSD of dx(t)dt

(b) Prove that Sxx (ω) = Sxx (-ω)
(c) If R (τ) = ae|by|. Find the spectral density function, where a and b are constants.
[5+5+6]

8. (a) A Signal x(t) = u(t) exp (-αt ) is applied to a network having an impulse
response h(t)= ω u(t) exp (-ω t). Here α & ω are real positive constants.
Find the network response? (6M)
 (b) Two systems have transfer functions H1 ( ω) & H2( ω). Show the transfer
Function H (ω) of the cascade of the two is H(ω ) =H1( ω) H2 (ω ).
(c) For cascade of N systems with transfer functions Hn(ω) , n=1,2,.... .N show
that H( ω) = πHn(ω). [6+6+4]

SET 3

1. (a) Define probability based on set theory and fundamental axioms.

(b) When two dice are thrown, find the probability of getting the sums of 10 or
11. [8+8]

2. (a) Define cumulative probability distribution function. And discuss distribution

function specific properties.
(b) The random variable X has the discrete variable in the set {−1,−0.5, 0.7, 1.5, 3}
the corresponding probabilities are assumed to be {0.1, 0.2, 0.1, 0.4, and 0.2}. Plot
its distribution function and state is it a discrete or continuous distribution
function. [8+8]

3. (a) Explain the concept of a transformation of a random variable X

(b) A Gaussian random variable X having a mean value of zero and variance one is
transformed to an another random variable Y by a square law transformation.
Find the density function of Y. [8+8]

4. Discrete random variables X and Y have a joint distribution function

FXY (x, y) = 0.1u(x + 4) u(y − 1) + 0.15u(x + 3) u(y + 5) + 0.17u(x + 1) u(y − 3) +
0.05u(x) u(y − 1) + 0.18u(x − 2) u(y + 2) + 0.23u(x − 3) u(y − 4) +0.12u(x − 4) u(y + 3)
Find
(a) Sketch FXY (x, y)
(b) Marginal distribution functions of X and Y.
(c) P (−1 < X _ 4, −3 < Y _ 3) and
(d) Find P(X < 1, Y _ 2). [4+6+3+3]

5. (a) let Y = X1 + X2 + ............+XN be the sum of N statistically independent

random variables Xi, i=1,2.............. N. If Xi are identically distributed then
Find density of Y, FY(y).
(b) Consider random variables Y1 and Y2 related to arbitrary random variables X
and Y by the coordinate rotation. Y1=X Cos θ + Y Sin θ Y2 = -X Sin θ + YCos θ
i. Find the covariance of Y1 and Y2, CY1Y2
ii. For what value of θ, the random variables Y1 and Y2 uncorrelated. [8+8]

6. (a) Define cross correlation function of two random processes X(t) and Y(t) and
State the properties of cross correlation function.
(b) Let two random processes X (t) and Y (t) be defined by
a) X (t) = A cos ω0t + B sin ω0t
Y (t) = B cos ω0t - A sin ω0t
Where A and B are random variables and ω0 is a constant. Assume A and
 B are uncorrelated, zero mean random variables with same variance. Find the cross
correlation function RXY (t, t+τ ) and show that X(t) and Y(t) are
jointly wide sense stationary. [6+10]

7. (a) If the PSD of X(t) is Sxx(ω ). Find the PSD of dx (t) dt

(b) Prove that Sxx (ω) = Sxx (-ω)
(c) If R (τ) = ae|by|. Find the spectral density function, where a and b are constants.
[5+5+6]

8. (a) A Stationary random process X(t) having an Auto Correlation function

RXX τ = 2e−4|_| is applied to the network shown in figure 8a find
i. SXX (ω)
ii. IH (ω) I2
iii. SY Y (ω). [4+4+2]
(b) Write short notes on different types of noises. [6]

SET 4

1. (a) Define and explain the following with an example:

i. Equally likely events
ii. Exhaustive events
iii. Mutually exclusive events
(b) Give the classical definition of probability.
(c) Find the probability of three half-rupee coins falling all heads up when tossed
simultaneously. [6+4+6]

2. (a) What is poisson random variable? Explain in brief.

(b) What is binomial density and distrbution function?
(c) Assume automobile arrives at a gasoline station are poisson and occur at an
Average rate of 50/hr. The station has only one gasoline pump. If all cars are
Assumed to require one minute obtaining fuel. What is the probability that a
Waiting line will occur at the pump? [5+5+6]

3. (a) Define moment generating function.

(b) State properties of moment generating function.
(c) Find the moment generating function about origin of the Poisson distribution.
[3+4+9]

4. Given the function f(x, y) = _ (x2 + y2)/8π x2 + y2 < b

0 elsewhere
(a) Find the constant ‘b’ so that this is a valid joint density function.
(b) Find P(0.5b < X2 + Y2 < 0.8b). [7+9]

5. Three statistically independent random variables X1, X2 and X3 have mean values
¯X1= 3, ¯X2= 6 and ¯X3= −2. Find the mean values of the following functions.
(a) g(X1,X2,X3) = X1 + 3X2 + 4X3
(b) g(X1,X2,X3) = X1 X2 X3
(c) g(X1,X2,X3) = −2X1,X2 −3X1 X3 + 4X2 X3
(d) g (X1,X2,X3) = X1+X2+X3. [16]
6. Statistically independent zero means random processes X (t) and Y (t) have auto
Correlations functions RXY (τ ) = e - |_| and RYY(τ ) = cos (2_τ ) respectively.
(a) Find the auto correlation function of the sum W1 (t) = X (t) + Y (t)
(b) Find the auto correlation function of difference W2 (t) = X (t) - Y(t)
(c) Find the cross correlation function of W1 (t) and W2 (t). [5+5+6]

7. (a) Find the ACF of the following PSD’s

i. S__ (ω) = 157+12! 216+! 2) (9+! 2)
ii. S__ (ω) = 8(9+! 2)2
(b) State and Prove wiener-Khinchin relations. [8+8]

8. A random noise X (t) having power spectrum SXX (ω) = 349+! 2 is applied to a to a
Network for which h (t) = u (t) t2 exp (−7t). The network response is denoted by Y (t)
(a) What is the average power is X (t)
(b) Find the power spectrum of Y (t)
(c) Find average power of Y (t). [5+6+5]
SOLVED EXAMPLES
 INTERNAL PAPERS
(MID, ASSSIGNMENT)
Hall Ticket No.
SET-1

CMR COLLEGE OF ENGINEERING & TECHNOLOGY

(AUTONOMOUS)
[Link] III Semester- I mid Examinations August – 2016
(Regulation: CMRCET-R15)
Subject Name: Probability Theory & Random Process
Department: ECE
Date: 09 .08.2016 Time: 10AM-11:20AM [Link]
PART-A
Answer all FIVE questions (Compulsory)
Each question carries ONE mark.
5x1=5M
1. Let A be the event defined on the null set, then P(A) is ________________
2. Define independent events?
3. What is the significance of skewness coefficient?
4. The signal received by RADAR system from target follows _____________
distribution
5. How many subsets are possible for a set consists of N elements?
PART-B
Answer any THREE questions. Each question carries FIVE Marks.
3x5=15M
6. a) State and prove the total probability theorem and baye’s theorem.
b) A binary symmetric channel is used for communication between a transmitter and
receiver. A transmitter transmits two possible inputs ‘0’ and ‘1’. The probability of
transmitting a ‘0’ is 0.6. Due to noise ‘0’ may be received as ‘1’ and vice versa. The
probability of transmitting ‘0’ and receiving ‘1’ is 0.5, The probability of transmitting ‘1’
and receiving ‘1’ is 0.4, find i) the probability of receiving ‘0’ ii) the probability of
receiving ‘1’ iii) the probability of receiving ‘0’ given that ‘0’ was transmitted.
7. a) Define the probability and explain the axioms of the probability
b) Explain in detail about the types of random variables with examples
8. a) In a binary noisy communication channel the bit error probability is 0.02. If 10 bits
are transmitted through the channel find the probability that i) Exactly 3 bits are in error ii)
No error bits in the received bits iii) More than 2 bits are in error iv) At most one error bit
b) If a coin is tossed 3-times. If the random variable X denotes “the number of heads are
shows up”. Find CDF, pdf and sketch them
9. a) State and prove properties of cumulative distribution function (CDF)
b) A random variable X is defined by distribution table given below
x -2 -1 0 1 2
P(x) 1/5 2/5 1/10 1/10 1/5
 Find i) E[X] ii) E[X2] iii) E[(2X+1)2] iv) X2 v) 3 vi) Skewness
10. a) Find the variance and skew of the uniform random variable.
b) State and prove properties of joint density function

*****
Hall Ticket No.

SET-2
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
(AUTONOMOUS)
[Link] III Semester- I mid Examinations August – 2016
(Regulation: CMRCET-R15)
Subject Name: Probability Theory & Random Process
Department: ECE
Date: 09 .08.2016 Time: 10AM-11:20AM [Link]
PART-A
Answer all TEN questions (Compulsory)
Each question carries ONE mark.
5x1=5M
1. Associative law is given as______________
2. Three events A,B and C are independents events then P(A∩B∩C) is _____________
3. List out the types of distribution funcions
4. What is the significance of standard deviation ?
5. What is the relation between density and distribution function?
PART-B
Answer any THREE questions. Each question carries FIVE Marks.
3x5=15M
6. a) State and prove the total probability theorem and baye’s theorem.
b) In a pen manufacture factory, machines A, B, C manufacture 20%, 30%, 50% of total
output respectively. From their outputs 4%, 5%, 3% are defective pens. A pen is drawn at
random and found to be defective. i) What is the probability of getting defective pen? ii)
What is the probability that it was manufactured by machine A? iii) What is the probability
that it was manufactured by machine C
7. a) What is conditional probability? Verify does it satisfy the axioms?
b) In an experiment of drawing a card from a pack of 52 cards, the event of getting a
spade is denoted by A, and getting a pictored card is denoted by B. Find the probabilities
of A,B, A∩B and AUB
8. a) A Gaussian random variable X has a X=0, and X=2. Then find i) P{2<X 3} ii)
P{2<X 3/X>2).
b) Two dice are rolled simultaneously, The random variable X denotes the “sum of two
faces shows up”. Find the mean and variance of random variable X.
9. a) State and prove the properties of Characteristic function.
b) The amplitude of output signal of a radar system is Rayleigh random variable with
a=0 and b=4 volts. The system gets false detection if signal exceeds V volts. What is the
value of V, if the probability of false detection is 0.001.
10. a) Show that mean and variance of binomial distribution is NP, NPq respectively
b) State and prove properties of joint distribution function.
Hall Ticket No.

SET-2
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
(AUTONOMOUS)
[Link] III Semester- I mid Examinations August – 2016
(Regulation: CMRCET-R15)
Subject Name: Probability Theory & Random Process
Department: ECE
Date: 09 .08.2016 Time: 10AM-11:20AM [Link]
PART-A
Answer all TEN questions (Compulsory)
Each question carries ONE mark.
5x1=5M
1. Associative law is given as______________
2. Three events A,B and C are independents events then P(A∩B∩C) is _____________
3. List out the types of distribution funcions
4. What is the significance of standard deviation ?
5. What is the relation between density and distribution function?
PART-B
Answer any THREE questions. Each question carries FIVE Marks.
3x5=15M
6. a) State and prove the total probability theorem and baye’s theorem.
b) In a pen manufacture factory, machines A, B, C manufacture 20%, 30%, 50% of total
output respectively. From their outputs 4%, 5%, 3% are defective pens. A pen is drawn at
random and found to be defective. i) What is the probability of getting defective pen? ii)
What is the probability that it was manufactured by machine A? iii) What is the probability
that it was manufactured by machine C
7. a) What is conditional probability? Verify does it satisfy the axioms?
b) In an experiment of drawing a card from a pack of 52 cards, the event of getting a
spade is denoted by A, and getting a pictored card is denoted by B. Find the probabilities
of A,B, A∩B and AUB
8. a) A Gaussian random variable X has a X=0, and X=2. Then find i) P{2<X 3} ii)
P{2<X 3/X>2).
b) Two dice are rolled simultaneously, The random variable X denotes the “sum of two
faces shows up”. Find the mean and variance of random variable X.
9. a) State and prove the properties of Characteristic function.
b) The amplitude of output signal of a radar system is Rayleigh random variable with
a=0 and b=4 volts. The system gets false detection if signal exceeds V volts. What is the
value of V, if the probability of false detection is 0.001.
10. a) Show that mean and variance of binomial distribution is NP, NPq respectively
b) State and prove properties of joint distribution function.
 CMR COLLEGE OF ENGINEERING AND TECHNOLOGY
KANDLAKOYA (VILLAGE), MEDCHAL
DEPARTMENT OF ECE
PTRPASSIGNMENT-I QUESTIONS
1. a) State and prove the total probability theorem and baye’s theorem.
b) Write short notes on independent events
2. a) Define Probability? State and prove its axioms?
b) A box contains 4 point contact diodes and 6 alloy junction diodes. What is the
probability that 3 diodes picked at random contain at least 2 point contact diodes.
3. b) It is known that a particular random voltage can be represented by Rayleigh
random variable with a=0 and b=5. The voltage is applied to a device that
generates power g(V)=v2. Find the average power.
b) State and prove properties of probability density function (PDF)
4. a) Explain in detail about Gaussian distribution
b) State and prove the properties of variance
5 a) Explain about moment generating function and find the mean value of
exponential random variable X by using moment generating function
b) State and prove the properties of joint density function
 RESULT ANALYSIS

Academic No of No of No of No of Pass
Year students students students students Percentage(%)
registered Attended Failed Passed

2016-17 272 271 51 220 81.8

2015-16 272 272 53 219 80.51

2014-15 201 201 08 93 96.02

PPTs OF EACH UNIT
 CO & PO MAPPING AND
EVALUATION

Program Outcomes - ECE

Ability to apply the knowledge of mathematics, science, engineering fundamentals for solution of
PO 1
complex engineering problems.
Ability to identify, formulate, research literature, and analyze complex engineering problems with
PO 2
appropriate considerations
Ability to design solution for complex engineering problems with appropriate consideration for
PO 3
society
Ability to use research –based knowledge and research methods including design of experiments
PO 4
to provide valid conclusions.
PO 5 Ability to learn and apply appropriate modern tools for engineering solutions
Ability to assess societal, health, safety, legal and cultural issues and the consequent
PO 6
responsibilities and follow them in professional practice
Ability to understand the impact of the professional practices on environment, society and its
PO 7
sustainable development
PO 8 Ability to apply ethical principles and commit to professional ethics in engineering practice
PO 9 Ability to function on an inter- disciplinary team
PO 10 Ability to communicate effectively
Ability to understand engineering and management principles and apply them to one’s own work,
PO 11
as a member and leader in a team, to manage projects
PO 12 Ability to engage in lifelong learning.

COURSE CLASS COs PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12
1 √ √ √
III
2 √ √ √
LDICA [Link]
3 √ √
I SEM
4 √ √ √
 CONTENT BEYOND THE SYLLABUS
Topics covered:

1. Sets, Venn diagrams

2. Random signals.
3. Impulse signal, Unit step signal
Definitions.
[Link] of signals.