294 Chapter 5 • A Brief Review of Probability Theory

examples is Ash (1992). Simon (2002) provides a compendium of relations involving the Gaussian
distribution.

Problems

Section 5.1
G)
5.1. A circle is divided into 21 equal parts. A pointer is A <:<-----------o
spun until it stops on one of the parts, which are numbered
from l through 21. Describe the sample space, and assum-
ing equally likely outcomes, find
a. P(an even number)
CD
b. P(the number 21)
c. P(the numbers 4, 5, or 9) o---------4'.>B
d. P(a number greater than IO) ©
5.2. What equations must be satisfied in order for three Figure 5.20
events A, B, and C to be independent?
(Hint: They must be independent by pairs, but this is not
sufficient.) a. What is the probability that at least one working
path is available between the nodes labeled A and
5.3. If five cards are drawn without replacement from B?
an ordinary deck of cards, what is the probability that
b. Remove link 4. Now what is the probability that at
a. Three kings and two aces result. least one working path is available between nodes
b. Four of a kind result.
A and B?
c. Remove link 2. What is the probability that at least
c. All are of the same suit.
one working path is available between nodes A and
d. An ace, king, queen, jack, and IO of the same suit B?
result.
d. Which is the more serious situation, the removal of
e. Given that an ace, king, jack, and IO have been link 4 or link 2? Why?
drawn, what is the probability that the next card drawn will
be a queen (not all of the same suit)? S.6. Given a binary communication channel where
A= input and B =output, let P(A) = 0.45, P(B/A) =
S.4. Two events, A and B, have marginal probabilities 0.95, and P(B/A) = 0.65. Find P(A/B) and P(A/B).
P(A) = 0.2andP(B) = 0.5,respectively. Their joint prob- S.7. Given the table of joint probabilities of Table 5.5.
ability P(A n B) = 0.4.
a. Find the probabilities omitted from Table 5.5 .
. a. Are they statistically independent? Why or why not?
b. Find the probabilities P(A 3/B 3),P(B2/A 1), and
b. What is the probability of A or B or both
P(B3/A2).
occurring?
c. In general, what must be true for two events be
both statistically independent and mutually exclusive? Table 5.5 Probabilities for Problem 5.7
S.S. Figure 5.20 is a graph that represents a commu-
Bi B2 B3 P(;4;)
nication network, where the nodes are receiver-repeater
boxes and the edges (or links) represent communication Ai 0.05 0.45 0.55
channels which, if connected, convey the message per- Az 0.15 [Link]
fectly. However, there is the probability p that a link will be A3 0.05 0.05 0.15
broken and the probability q = 1 - p that it will be whole. P(Bj) 1.0
Hint: Use a tree diagram like Figure 5.2.
 Problems 295

Section 5.2 c. fxr(x, 3).

5.8. Three fair coins are tossed simultaneously such that d. fx1r(xjl).
they don't interact. Define a random variable X = 1 if an
even number of heads is up and X = 0 otherwise. Plot the 5.14.
cumulative distribution function and the probability den-
a. For what value of a > 0 is the function
sity function corresponding to this random variable.
5.9. Two dice are tossed.
f(x) = ax- 2 u(x-a)
a. Let X1 be a random variable that is numerically a probability density function? Use a sketch to illustrate
equal to the total number of spots on the up faces of the your reasoning and recall that a pdf has to integrate to I.
dice. Construct a table that defines this random variable. [u(x) is the unit step function.]
b. Let X2 be a random variable that has the value of 1 b. Find the corresponding cumulative distribution
if the sum of the number of spots up on both dice is even function.
and the value zero ifit is odd. Repeat part (a) for this case.
c. Compute P(X 2 10).
5.10. A certain continuous random variable has the
5.15. The joint pdf of the random variables X and Y is
cumulative distribution function
x 2 0 and y 2 0
. {O,
Fx(x) = Ax4,
x<O
0:::; x:::; 12
- fxr(x,y) = Axye-<·'+Yi,

a. Find the constant A.

B, x > 12
b. Find the marginal pdfs of X and Y, fx(x) and
fy(y).
a. Find the proper values for A and B.
c. Are X and Y statistically independent? Justify
b. Obtain and plot the pdf/x(x). your answer.
c. Compute P(X > 5). 5.16. Given the Gaussian random variable with the pdf
d. Compute P(4:::; X < 6).
5.11. The following functions can be pdfs if constants are
chosen properly. Find the proper conditions on the con-
fx(x) = =
e-x2 /2a2

v 27T<T
stants [A, B, C, D, a, {3, y, and rare positive constants, and where <T > 0 is the standard deviation. If Y = X 2 find the
u(x) is the unit step function.] pdf of Y. Hint: Note that Y = X 2 is symmetrical about
X = 0 and that it is impossible for Y to be less than zero.
a. f(x) = Ae-"xu(x), where u(x) is the unit step.
5.17. A nonlinear system has input X and output Y.
b. f(x) = Bef3xu( - x).
The pdf for the input is Gaussian as given in Problem
c. f(x) = Ce-Y"u(x- I). 5.16. Determine the pdf of the output, assuming that the
d. f(x) = D[u(x) - u(x- r)]. nonlinear system has the following input-output
relationship:
5.12. Test X and Y for independence if

a. fxr(x,y) =Ae-lxl-2IYI. a.
Y-
-
{ax, x<2 o
0, X 0

b. fxr(x,y) = C(l-x-y),O:::; x:::; 1-yand Hint: When X < 0, what is Y? How is this manifested in
0:::; y:::; 1. the pdf for Y?

Prove your answers. b. Y= jXj.

5.13. The joint pdf of two random variables is c. Y=X-X 3 /3.

-{C(I+xy), 0 :::;x:::;4,0:::;y:::;2
f XY (X,y ) - O, otherwise
Section 5.3
5.18. Letfx(x) =Aexp(-bx)u(x-2)forallxwhercA
Find the following: and b are positive constants.
a. The constant C.
a. Find the relationship between A and b such that
b. fxr(l, 1.5). this function is a pdf.
296 Chapter 5 • A Brief Review of Probability Theory

b. Calculate E(X) for this random variable. 5.24. Two Gaussian random variables X and Y, with zero
c. Calculate E(X 2
) for this random variable. means and variances CT2, between which there is a correla-
tion coefficient p, have a joint probability density function
d. What is the variance of this random variable?
given by
5.19.
a. Consider a random variable uniformly distribu- l ( x2-2pxy+i)
ted between 0 and 2. Show that E(X 2 ) > E2 (X). f(x,y)= exp 2CT2(1-p2)
b. Consider a random variable uniformly distribu-
ted between 0 and 4. Show that E(X2 ) > E2 (X). The marginal pdf of Y can be shown to be
2 2
c. Can you show in general that for any random f: ( ) exp( -y /2CT )

variable it is tme that E(X 2 ) > E2 (X) unless the random r y v27fCT2
variable is zero almost always? Find the conditional pdffx1r(x / y). Simplify.
2
(Hint: Expand E[(X - E[X]) ] 2: 0, and note that it is 0 5.25. Two random variables X and Y have means and
only if X = 0 with probability 1.) variances given below:
5.20. A random variable X has the pdf
mx = 1, CT; = 4, my = 3, a; = 7
fx(x) = Ae-hx[u(x) -u(x-B)]
A new random variable Z is defined as
where u(x) is the unit step function and A, B, and bare
Z = 3X-4Y
positive constants.
a. Find the proper relationship between the con- Determine the mean and variance of Z for each of the
stants A, b, and B. Express b in terms of A and B. following cases of correlation between the random vari-
b. Determine and plot the cdf. ables X and Y:
c. Compute E[X]. a. Pxr = 0.
d. Determine E(X2]. b. Pxr = 0.2.
e. What is the variance of X? c. Pxr = 0.7.
5.21. Verify the entries in Table 5.5 for the mean and d. Pxr = 1.0.
variance of the following probability distributions:
5.26. Using the definition of a conditional pdf given by
a. Rayleigh (5.62) and the expressions for the marginal and joint
b. One-sided exponential Gaussian pdfs, show that for two jointly Gaussian random
variables X and Y, the conditional density function of X
c. Hyperbolic
given Yhas the form of a Gaussian density with conditional
d. Poisson mean and the conditional variance given by
e. Geometric
E[X/Y] = mx + PO"x (Y - my)
5.22. The random variable has pdf O'y

I 1 and
. fx(x) = 5) + S[u(x-4) - u(x - 8)]
2a(x - var(X/Y) = CT;(1 - p 2 )
where u(x) is the unit step. Determine the mean and the respectively.
variance of the random variable thus defined.
5.27. A random variable Xis defined by
5.23. If
fx(x) = 4e-Slxl
2
fx(x) = (2m:,
2 )-1;2 ( x )
exp - 2CT2 The random variable Y is related to X by Y 4 + 5X.

show that
a. Determine E(XJ,E[X2 ], and u;.
b. Determine fr (y).
a. E[X 2n] = I x 3 x 5 x ... (2n - 1)CT2n, for
n = 1,2, ...
c. DetermineE[Y], E[Y2 ], and a;.
(Hint: The result of
part (b) is not necessary to do this part, althongh it may be
b. E[X 2n - 1J = 0 for n = 1, 2, ... used)
 Problems 297

5.28. The random variable X has a probability density b. Letters can be repeated if desired, so that each X;
s
function uniform in the range 0 S x 2 and zero else- is completely arbitrary.
where. The independent variable Yhas a density uniform in c. If selection of letters for a given password is
s s
the range 1 y 5 and zero elsewhere. Find and plot the completely random, what is the probability that your
density of Z = X + Y. competitor could access, on a single try, your computer in
5.29. A random variable X has the probability density part (a)? and part (b)?
function 5.33. A digital data transmission system has an error
probability of 10- 5 per digit.
fx(x) = { ae-ax, x ?: 0
0, x<O a. Find the probability of exactly one error in 105
where a is an arbitrary positive constant. digits.
b. Find the probability of exactly two errors errors
a. Determine the characteristic function Mx(Jv). in 105 digits.
b. Use the characteristic function to determine E[X] c. Find the probability of more than five errors in
and E[X2 ]. 105 digits.
c. Check your results by computing 5.34. Assume that 20 honest coins are tossed.
a. By applying the binomial distribution, find the
X'fx(x) dx
probability that there will be fewer than three heads.
for n = 1 and 2. b. Do the same computation using the Laplace
approximation.
d. Compute
c. Compare the results of parts (a) and (b) by com-
puting the percent error of the Laplace approximation.
Section 5.4 5.35. Assume that two random variables X and Y are
5.30. An honest coin is flipped 10 times. jointly Gaussian with mx =my= 1, u; = u; = 4, and
correlation coeficient p = 0.5.
a. Determine the probability of the occurrence of
either five or six heads. a. Making use of (5.194), write down an expression
b. Determine the probability of the first head oc- for the margininal pdfs of X and of Y.
curring at toss number 5. b. Write down an expression for the conditional pdf
c. [Link] (a) and(b)forflipping IOOtimes and fx1r(x/y) by using the result of (a) and an expression for
the probability of the occurrence of 50 to 60 heads fxr(x, y) written down from (5.189). Deduce that
inclusive and the probability of the first head occurring fr1x(y/x) has the same form with y replacing x.
at toss number 50. c. Put fx1r(xly) into the form of a marginal
5.31. Compare the binomial, Laplace, and Poisson dis- Gaussian pdf. What is its mean and variance? (The mean
tributions for will be a function of y.)
5.36. Consider the Cauchy density. function
a •. n=3andp=!·
K
b. n = 3 and p = To. fx(x) = 1 +xz, -oo SxS oo

c.
a. Find K.
d. n = 10 and p = To .
b. Show that var[X] is not finite.
5.32. Passwords in a computer installation take the form ·c. Show that the characteristic function of a Cauchy .
X1X2X3X4 , where each character X; is one of the 26 letters random variable is Mx(Jv) = ?TKe-lvl.
of the alphabet. Determine the maximum possible number
of different passwords available for assignment for each of d. Now consider Z = X1 + ... + XN where the X/s
the two following conditions: are independent Cauchy random variables. Thus their
characteristic function is
a. A given letter of the alphabet can be used only
once in a password. Mz(jv) = (?TKtexp( -Nlvl)
298 Chapter 5 • A Brief Review of Probability Theory

Show thatfz(z) is Cauchy. (Comment:fz(z) is not Gaus- Q-function. Plot the resulting cdf for m = 0 and
sian as N-> oo because var[X;] is not finite and the con- cr = 0.5, I, and 2.
ditions of the central-limit theorem are therefore violated.)
5.40. A random variable X has the
5.37. (Chi-squared pdt) Consider the random variable
Y=
2
XT, where the X;', are independent Gaussian e-(x-10) /50
random variables with pdfs n(O, cr). fx(x) = J5Q;

a. Show that the characteristic function of XT is Express the following probabilities in terms of the Q
.) function and calculate numerical answers for each:
Mxf Uv = (I - 2'JVCT
. 2)-1/2
a. (P(IXI :S 15)
b. Show that the pdf of Y is
b. P(IO < X :S 20)
c. P(5 < X :S 25)
y2':0
d. P(20 < X :S 30)
y<O
5.41. Prove that the Q function may also be represented
where r(x) is. the gamma function, which, for x =nan as
integer, is r( n) = (n - 1) !. This pdf is known as the x2
(chi-squared) pdf with N degrees of freedom. Hint: Use I 7r/2 ( x2 )
the Fourier transform pair
Q(x) =;;: fo exp - 2sin2q:, d<f>.

yN/2-le-yfa . -N/ 5.42.

2
aN/2r(N/2) <--> (1-pv) a. Prove Chebyshev's inequality. Hint: Let
Y = (X - mx)/cr..., and find a bound for P(/YI < k) in
c. Show that for N large, the x2 pdf can be approxi- terms of k.
mated as
b. Let X be uniformly distributed over /xi :S 1. Plot
2 P(!XI :S kcrx) versus k and the corresponding bound
exp { - 2I [(y-ND'')]
../4ii04 }
given by Chebyshev's inequality.
fr(Y) N » 1 5.43. Speech is sometimes idealized as having a Lapla-
cian-amplitude pdf. That is, the amplitude is distributed
Hint: Use the central-limit theorem. Since the x;'s are according to ( ) a ( / I)
independent, fx x =2exp -ax
N a. Express the variance of X, cr2, in terms of a. Show
f= LXf =Ncr2 your derivation; don'tjust simply copy the result given in
i=l
Table 5.4.
and b. Compute the following probabilities: P(JXI >
N cr); P(IXI > 2cr); P(IXI > 3cr).
var[Y] = 2=var[X1] = Nvar[X;j -
5.44. If the random variable X is Gaussian with zero
i=l
mean and variance cr2, obtain numerical values for the
d. Compare the approximation obtained in part following probabilities:
(c) withfy(y) for N = 2,4,8.
e. Let R2 = Y. Show that the pdf of R for N = 2 is
a. P(IXJ_> cr)
Rayleigh. P(JXI > 2cr)
5.38. Compare the Q-function and the approximation to c. P(!XI > 3cr)
it for large arguments given by (5.202) by plotting both 5.45. Two jointly Gaussian zero-mean random variables,
expressions on a log-log graph. (Note: MATLAB is handy X and Y, have respective variances of 3 and 4 and correla-
for this problem.) tion coefficient Pxr = - 0.4. A new random variable is
5.39. Determine the cdffor a Gaussian random variable defined as Z = X + 2Y. Write down an expression for the
of mean m and variance cr2. Express in terms of the pdf of Z.
 Computer Exercises 299

5.46. Two Gaussian random variables, X and Y, are a. Write down expressions for their marginal pdfs.
independent. Their respective means are 5 and 3, and their b. Write down an expression for their joint pdf.
respective variances are I and 2.
c. What is the mean of Z 1 = 3X + Y? Z2 = 3X - Y?
a. Write down expressions for their marginal pdfs. d. What is the variance of Z1 = 3X + Y? Zz =
b. Write down an expression for their joint pdf. 3X-Y?
c. WhatisthemeanofZ1 =X+Y? Z2 =X-Y? e. Write down an expression for the pdf of Z 1
= 3X+Y.
d. What is the variance of Z 1 = X + Y?
Z2 =X-Y? f. Write down an expression for the pdf of Zz
= 3X-Y.
e. Write down an expression for the pdf of
Z1 -=X+Y. 5.49. Find the probabilities of the following random
f. Write down an expression for the pdf of variables, with pdfs as given in Table 5.4, exceeding their
Z2 =X-Y. means. That is, in each case, find the probability that
X:::: mx, where Xis the respective random variable and
5.47. Two jointly Gaussian random variables, X and Y, mx is its mean.
have means of 1 and 2, and variances of 3 and 2, respec-
a. Uniform
tively. Their correliition coefficient is PXY = 0.2. A new
random variable is defined as Z = 3X + Y. Write down an b. Rayleigh
expression for the pdf of Z. c. One-sided exponential
5.48. Two Gaussian random variables, X and Y, are
independent. Their respective means are 4 and 2, and their
respective variances are 3 and 5.

Computer Exercises

5.1. In this exercise we examine a useful technique for the following transformation generates two statistically
generating a set of samples having a given pdf. independent Gaussian random numbers with unit variance
a. First, prove the following theorem: If X is a and zero mean:
continuous random variable with cdf Fx(x), the random X R cos(27TU)
variable · Y R sin(21TU)
Y = Fx(X)
is a uniformly distributed random variable in the interval where
[0,1).
R=V-2lnV
b. Using this theorem, design a random number
generator to generate a sequence of exponentially
uted random variables having the pdf Hint: First show that R is Rayleigh.·
fx(x) = ae-axu(x) b. Generate 1000 random variable pairs according to
the above algorithm. Plot histograms for each set (i.e., X
where u(x) is the unit step. Plot histograms of the random and Y), and compare with Gaussian pdfs after properly
numbers generated to check the validity of the random scaling the histograms (i.e., divide each cell by the total
number generator you designed. number of counts times the cell width so that the histogram
approximates a probability density function).
5.2. An algorithm for generating a Gaussian random Hint: Use the hist function of MATLAB.
variable from two independent uniform random variables 5.3. Using the results of Problem 5.26 and the Gaussian
is easily derived. random number generator designed in Computer Exercise
a. Let U and V be two statistically independent 5.2, design a Gaussian random number generator that will
random numbers uniformly distributed in [O, l]. Show that provide a specified correlation between adjacent samples.
300 Chapter 5 • A Brief Review of Probability Theory

Let random variables in the interval ( - 0.5, 0.5), forming the

sum given by (5.187), and plotting the histogram. Do this
for N = 5, 10, and 20. Can you say anything qualitatively
and quantitatively about the approach of the sums to
and plot sequences of Gaussian random numbers for
Gaussian random numbers? Repeat for exponentially dis-
various choices of a. Show how stronger correlation
tributed component random variables (do Computer Ex-
between adjacent samples affects the variation from sam-
ercise 5.1 first). Can you think of a drawback to the
ple to sample. (Note: To get memory over more than
approach of summing uniformly distributed random vari-
adjacent samples, a digital filter should be used with
ables to generating Gaussian random variables? (Hint:
independent Gaussian samples at the input.)
Consider the probability of the sum of uniform random
5.4. Check the validity of the central-limit theorem by variables being greater than 0.5N or less than -0.5N. What
repeatedly generating n independent uniformly distributed are the same probabilities for a Gaussian random variable?)