Chapter 4.

One Random Variable

In this chapter we consider the general case where the random
variable may be discrete, continuous, or of mixed type

 Cumulative distribution function (cdf)

 Probability density function (pdf) for continuous random
variable

 Def) A random experiment with sample space S and event

class F. A random variable X is a function from the sample
space S to R with the property that the set Ab = {  : X ()  b}
is in F for every b in R

 Why does the definition use sets of the form {  : X ()  b} and
not {  : X () = b} ?
Prob. & Random Proc. 1
4.1 The Cumulative Distribution Function (cdf)

The cdf of a random variable X : the probability of the event

{X  x}, where x is a real number

FX(x) = P[X  x] for - < x < 

a random variable a numerical variable (not random)

 The probability that X takes on a value in the set (-, x]

 The probability of the event { : X( )  x}

Note : “The event {X  x} and P[X  x] vary with x” or FX(x) :

a function of the variable x.

Prob. & Random Proc. 2

4.1 The Cumulative Distribution Function (cdf)

Events of interest when dealing with numbers

 Intervals of the real line, their complements, unions,
intersections
 The probability of all of these events can be expressed in
terms of the cdf

Ex 4.2) Uniform RV in the Unit Interval

 Uniform random variable X in a ≤ x ≤ b  pdf, cdf ?

 Fig 4.2

Prob. & Random Proc. 3

 Ex 4.1 (fig. 4.1) : three coin tosses
 The cdf is continuous from the right and equal to ½ at the point
x = 1.
 P[X = 1] = 3/8, the magnitude of the jump at the point x = 1.
 cf) For a small positive number ,

 cf) In terms of unit step function

Prob. & Random Proc. 4

4.1 The Cumulative Distribution Function (cdf)

Properties of the CDF

1. 0  FX(x)  1
cf) Axiom I : 0  P[A] and Corollary 2: P[A]  1

2.

cf) The event {x < } is the entire sample space, then

Axiom II : P[S] = 1

Prob. & Random Proc. 5

4.1 The Cumulative Distribution Function (cdf)

3.

cf) The event {x  －} is the empty set, then

Corollary 3: P[Φ]=0

4. FX(x) is a nondecreasing function of x,

i.e., if a < b, then FX (a)  FX(b)  단조증가함수
cf) Corollary 7: the event {x  a}  the event {x  b}, then
P[x  a]  P[x  b]

5. FX(x) is continuous from the right 

for h > 0,

Prob. & Random Proc. 6

4.1 The Cumulative Distribution Function (cdf)

6. P[a < X  b]= FX(b)－ FX(a)

cf) {X  a} ∪ {a < X  b} = {X  b}, then by Axiom III
P[X  a] + P[a < X  b] = P[X  b]  FX(a) + P[a < X  b] = FX(b)

7. P[X = b] = FX(b)－ FX(b－ )

cf) let a = b－  and  > 0
P[b－ < X  b] = FX(b)－ FX(b－ )

 The probability of P(X=b) is given by the magnitude of the jump of

the cdf at the point b

Note : If the CDF is continuous at a point b, then the event {X = b} has

probability zero,
i.e., P[X = b] = 0 if CDF is continuous at a point b.

Prob. & Random Proc. 7

4.1 The Cumulative Distribution Function (cdf)

Note : {a  X  b} = {X = a} ∪ {a < X  b}
 P[a  X  b] = FX(a)－ FX(a－ ) + FX(b)－ FX(a)
= FX(b)－ FX(a－ )

Note : If the CDF is continuous at endpoints,

i.e. at the points X = a and X = b, then
P[a < X < b] = P[a  X < b] = P[a < X  b] = P[a  X  b]

8. P[X > x] = 1－ FX(x)

Prob. & Random Proc. 8

4.1.1 The Three Types of Random Variables

Discrete Random Variable

 cdf is a right-continuous, staircase function of x with jumps at

a countable set of points x0, x1, x2,…
 Discrete RVs take on values from a finite or SX = {x0, x1, x2,…}
 probability mass function (pmf) of X : the set of probabilities
pX(xk) = P[X = xk] of the elements in SX
 cdf of a discrete random variable

where pX(xk) = P[X = xk] : the magnitude of jumps in the cdf

Prob. & Random Proc. 9

4.1.1 The Three Types of Random Variables

Continuous Random Variable

 cdf FX (x) is continuous everywhere and sufficiently smooth.

 It can be written as an integral of some nonnegative
function f(x)

where f(t) is a nonnegative function

 P[X = x] = 0

Prob. & Random Proc. 10

4.1.1 The Three Types of Random Variables

Random Variable of Mixed Type

 cdf has jumps on a countable set of points x0, x1, x2,…

 cdf increases continuously over at least one interval of

values of x.

 FX(x) = pF1(x) + (1－ p)F2(x)

where 0<p<1, and F1(x) is the cdf of a discrete random
variable and F2(x) is the cdf of a continuous random
variable

Prob. & Random Proc. 11

4.2 The Probability Density Function (pdf)
The probability density function of X (pdf) : fX(x)

 The pdf is an alternative, and more useful way of specifying the

information contained in the cdf
Note : The pdf represents the density of probability at the
point x.

 fX(x) represents the “density” of prob. at the point x in the sense

that the prob. that X is in a small interval in the vicinity of x is
approximately
Prob. & Random Proc. fX(x)h 12
4.2 The Probability Density Function (pdf)

1. fX(x)  0
cf) and FX(x) is a nondecreasing function

 Provide an alternative approach to specifying the probabilities

involving the RV X.

PDF specifies the probabilities

of events of the form
“X falls in a small
Interval of width dx about the
point x”

Prob. & Random Proc. 13

4.2 The Probability Density Function (pdf)

2.

 cf) The probability of an interval [a, b] is the area under fX(x) in

that interval

The probability of the union of

disjoint Intervals can be found by
adding the Integrals of the pdf over
each of theintervals

Prob. & Random Proc. 14

4.2 The Probability Density Function (pdf)

3.
 Note : The pdf completely specifies the behavior of continuous
random variables

4. : normalization condition for pdf’s

 Note : a valid pdf can be formed from any nonnegative,

piecewise continuous function g(x) has a finite integral

Prob. & Random Proc. 15

4.2 The Probability Density Function (pdf)

Ex. 4.6 Uniform RV, [a, b]

Ex. 4.8
 The pdf of the samples of the amplitude of speech

waveforms.

 c = ? (using the normalization condition)

Prob. & Random Proc. 16

4.2.1 Unit Step and Delta functions

cf) The derivative of the cdf does not exist at the

discontinuities.
 The pdf does not apply to discrete random variable at the
points where the cdf is discontinuous

cf) The relation between u(x) and (x)

 Unit step function

 Delta function

(definition of the delta function)

Prob. & Random Proc. 17
4.2.1 pdf of Discrete Random Variables

 The cdf of a discrete random variable

 Generalized definition of the pdf fX(x)

The pdf for a discrete random variable

Prob. & Random Proc. 18

4.2.1 pdf of Discrete Random Variables

Ex.4.9
 X : the number of heads in three coin tosses.

SX = {0, 1, 2, 3}
 Find the pdf of X

Find P[1 < X  2] and P[2  X  3] by integrating the pdf.

sol)

Prob. & Random Proc. 19

4.2.2 Conditional cdf’s and pdf’s

Definition
 conditional cdf of x given A concerning X

 conditional pdf of x given A

Ex. 4.10
 life-time of a machine : random variable X → cdf FX(x)
 Find the conditional cdf and pdf given the event A = {X > t}
(i.e., “machine is still working at time t” )

Prob. & Random Proc. 20

4.2.2 Conditional cdf’s and pdf’s

 sol) The conditional cdf

cf) continuous at x = t

Prob. & Random Proc. 21

4.2.2 Conditional cdf’s and pdf’s

 The conditional pdf

Prob. & Random Proc. 22

4.3 The Expected Value of X
The expected value (average, mean) of X

 cf) The convergence of the expected value

 Discrete random variable

Prob. & Random Proc. 23

 Ex. 4. 12 Uniform RV, [a,b], mean ?
When the pdf is symmetric about a point m, E[X] = m

fX(m－ x) = fX (m + x) for all x

(m－ t) : odd symmetric about t = m

fX(t) : symmetric about t = m
 (m－ t) fX(t) : odd symmetric about t = m

Prob. & Random Proc. 24

 Ex. 4.13 Mean of a Gaussian RV

When X is a nonnegative random variable

Table 4.1 : The expected values of important RV

Prob. & Random Proc. 25

 Proof)


Prob. & Random Proc. 26

 

Prob. & Random Proc. 27

4.3.1 Expected Value of Y=g(X)

Prob. & Random Proc. 28

 Let h → 0
then

cf)

Prob. & Random Proc. 29

 Ex. 4.15
 Y = a cos(t + ), a, , : constants, t : variable
 : r.v. in (0,2)

Prob. & Random Proc. 30

 Note

HW
 4.5, 4.7, 4.10, 4.13, 4.14, 4.24, 4.29, 4.44, 4.50,
4.52
Prob. & Random Proc. 31
4.3.2 Variance of X
The variance of the random variable X

The standard deviation of the RV X

The nth moment of the RV X

Ex. 4.18, Variance of Uniform RV, [a,b]

Ex. 4.19, Variance of Gaussian RV

Prob. & Random Proc. 32

4.4. Some Important Random Variables

Continuous Random Variable (Table 4.1)

 Uniform Random Variable

 Exponential Random Variable

 Gaussian Random Variable

 m-Erlang Random Variable

 Chi-Square Random Variable

 Rayleigh Random Variable

 Cauchy Random Variable

 Laplacian Random Variable

Prob. & Random Proc. 33

Exponential Random Variable

The time between occurrence of events

The lifetime of devices and systems
pdf and cdf

fX(x) FX(x)
e-x 1－ e-x

x x
Prob. & Random Proc. 34
   = the rate at which events occur
cf) the probability of an event occurring by time x increases as
the rate  increases.
 Limiting form of the geometric random variable

 An interval of duration T → Subintervals of length

cf)
 discrete model → continuous model

 The sequence of subintervals → a sequence of independent

Bernoulli trials with

where  = the average number of events per T seconds.

Prob. & Random Proc. 35

  The number of subintervals until the occurrence of an event
→ a geometric random variable M.
The time until the occurrence of the first event


Prob. & Random Proc. 36

 the exponential random variable
cdf with

 Note : Poisson random variable

P[N(t) = 0] = e-t → No calls for t seconds
→ t seconds is an interval between any two calls

Prob. & Random Proc. 37

 Meanwhile

In conclusion, for Poisson random variable.

the time between events is an exponentially distributed random
variable with

 Note : The exponential random variable is the only continuous

random variable that satisfies the memoryless property.

Prob. & Random Proc. 38

Gaussian (Normal) Random Variable

A random variable consisting of the sum of a large number of “

small” random variables approaches the Gaussian random
variable : the central limit theorem

The pdf for the Gaussian random variable X

 Where m and  are the mean and standard deviation of X.

=1
=1
m m=0

Prob. & Random Proc. 39

 cdf

cf) (x) is the cdf of a Gaussian random variable with

m = 0 and  = 1

Prob. & Random Proc. 40

 Ex. 4.21: Show that

Prob. & Random Proc. 41

 Q-function

 the probability of the “tail” of the pdf

Prob. & Random Proc. 42

  Note : Table 4.2
The value of x for which Q(x) = 10－ k
where k = 1, 2, …,10

 k = 1 : the probability of tale 0.1 or k = 2: the probability of tale 0.01

Prob. & Random Proc. 43

Gamma Random Variable

Pdf

 where two parameters  and  are positive numbers and 

(z) is the gamma function

 Many random variables are special cases of the gamma

random variable
 The exponential random variable
 Special case of the gamma r.v. with  = 1
Prob. & Random Proc. 44
 Chi-Square Random Variable


Prob. & Random Proc. 45

 m-Erlang Random Variable
 = m

 The time Sm that elapses until the occurrence of the

mth event. : m-Erlang r.v.
 Sm = X1 + X2 +…+ Xm
 X1, X2, , Xm : the times between events
→ exponential random variables
 Sm  t if and only if m or more events occur in t seconds
→ N(t)  m → mth event occurred by time t

Prob. & Random Proc. 46

  where N(t) is the Poisson random variable for the number of
events in t seconds.

cf)  = t

Sm : m-Erlang random variable

Prob. & Random Proc. 47

4.5 Functions of a Random Variable
Y = g(X) :

Ex. 4.29
 X : the number of active speakers in a group of N
independent speakers
 p : the probability that a speaker is active
 X has a binomial distribution with parameter N and p

 M : Transmittable voice signals at a time

 If X > M, X－ M randomly selected signals are discarded.
The number of signals discarded = r.v. Y

Prob. & Random Proc. 48

 sol)

cf) pj : the pmf of a binomial random variable X.

Prob. & Random Proc. 49

 Ex. 4.31 A Linear Function

 Let Y = aX + b, a  0
 FX(x) : the cdf of X
 Find the cdf of Y : FY(y)

sol)
 The event {Y  y} occurs when A = {aX + b  y} occurs
(Fig. 3.16)

Prob. & Random Proc. 50

  pdf

Prob. & Random Proc. 51

 Ex. 4.32
 X : a Gaussian random variable with mean m and

standard deviation 
 Y = aX + b : a linear function of X

 Mean b + am, standard deviation

 A linear function of a Gaussian random variable is also a
Gaussian random variable

Prob. & Random Proc. 52

 Ex. 4.33
 Y = X , X : a continuous random variable
2

 Find the cdf and pdf of Y

sol)

cf) the event is null for y < 0, (Fig. 3.17)

Prob. & Random Proc. 53

 Ex. 4.34
X : a Gaussian random variable with m = 0 and  = 1
→ A standard normal random variable
Y = X2
Find the pdf of Y

Prob. & Random Proc. 54

  The pdf of a Chi-Square random variable with one degree of
freedom:

Prob. & Random Proc. 55

 A nonlinear function Y = g(X)

① The event Cy = {y < Y < y + dy}

② Its equivalent event By
Prob. & Random Proc. 56
 ③ g(x) = y has three solutions x1, x2 , x3
④ By = {x1 < X < x1 + dx1}∪{ x2 + dx2 < X < x2 }∪{x3 < X < x3 + dx3}
⑤

where |dy| is the length of the interval y < Y  y + dy

⑥

Prob. & Random Proc. 57

 Ex. 4.36
 Y = cos(X)

 X is uniformly distributed in the interval (0,2]

 Y = cos(x) has two solutions for －1 < y < 1

→ x0 = cos-1(y), x1= 2－ x0

Prob. & Random Proc. 58

Prob. & Random Proc. 59
 cf)

But fY(y)  0 for －1  y  1

Prob. & Random Proc. 60

Prob. & Random Proc. 61
4.6 The Markov and Chebyshev Inequalities

Markov inequality

Bound worst case analysis

for X nonnegative

 Proof)

Prob. & Random Proc. 62

 Chebyshev inequality
 E[X] = m, VAR[X] = 
2

 Proof)

 Note {D2  a2} and {|X－ m|  a} are equivalent events.

Prob. & Random Proc. 63

  cf) VAR[X] = 0

P[X－ m  a]  0
→ P[X = m] = 1
⇒ the random variable is equal its mean with probability one.

 Note
Chebyshev inequality can give rather loose bounds.

Prob. & Random Proc. 64

 Chernoff bound

⇒ Markov inequality

⇒ Chernoff bound

Prob. & Random Proc. 65

4.7 Transform method

The characteristic function

 cf) “a reversal in the sign of the

exponent”

 Every pdf and its characteristic function form a unique Fourier

transform pair

Prob. & Random Proc. 66

 For a discrete random variable X

discrete random variable

integer valued random variable

 Fourier transform of the
sequence pX(k)

: periodic function of 
with period of 2.

Prob. & Random Proc. 67

 ∵ X () is periodic → Fourier series expansion.
i.e., pX(k) : the coefficients of the Fourier series of the periodic
function X()

Moment theorem
fX(x) and X() : a transform pair

Prob. & Random Proc. 68

  Proof)

Prob. & Random Proc. 69

The Probability Generating Function

 N = a nonnegative integer-valued r.v.

the z-transform of the pmf with a sign

change in the exponent

 X()=GN(ej)

Prob. & Random Proc. 70

  pmf by GN(z) : Probability
generating function

Prob. & Random Proc. 71

 

Prob. & Random Proc. 72

The Laplace Transform of the pdf

Nonnegative continuous r.v.’s

Laplace transform of the pdf

The moment theorem

 HW : 4.54, 4.62, 4.65, 4.79, 4.82, 4.87, 4.89, 4.94,

4.96, 4.97

Prob. & Random Proc. 73