IEEE TRANSACTIONS ON LATEX 1

Random Variables: An Overview

Monson Hayes, Fellow, IEEE
Chung-Ang University
Seoul, Korea
mhh3@[Link]
(Invited Paper)

Abstract—This paper introduces the concept of a random values of N . For example, the probability that the number of
variable, which is nothing more than a variable whose numeric α particles is less than some number, N0 , may be found as
value is determined by the outcome of an experiment. To describe follows. Since the event {N < N0 } is the union of the events
the probabilities that are associated with these numeric values
in a concise and conceptually useful manner, the probability {N = k} for k = 0, 1, . . . , N0 − 1,
distribution and probability density function are introduced. 0 −1
N[
Then, the moment generating function is defined, and several {N < N0 } = {N = n}
examples are given. Finally, the concept of a correlation function
n=0
and correlation matrices is introduced.
and since these events are mutually exclusive, then
0 −1
NX 0 −1
NX
I. I NTRODUCTION λn −λ
P{N < N0 } = P{N = n} = e
The concept of a random variable is a simple one, and n=0 n=0
n!
one that is important. Although perhaps sounding at first
This last sum may be evaluated using the following
like something difficult, random variables are conceptually
quite simple. Given a sample space Ω corresponding to some k
X λn −λ Γ(k + 1, λ)
random experiment, this sample space contains elementary e =
n! k!
events, ω ∈ Ω, and when an experiment is performed, a n=0

specific elementary event (experimental outcome) is observed. where Z ∞

When these experimental outcomes are numeric values, such Γ(k, λ) = xk−1 e−x dx
as the temperature of a fluid, the velocity of a particle, or the λ
value of a commodity or stock, we have a random variable. In
some experiments, the outcomes may be non-numeric, such as III. P ROBABILITY M ASS F UNCTION
the gender or the blood type of a person selected at random
from within a given population, or the outcome of the flip of a Discrete random variables have a discrete set of possible
coin. In these cases, a random variable may be formed simply outcomes, xk , and the probabilities of these outcomes are
by mapping these non-numeric outcomes to real numbers We denoted by pk ,
begin this paper by introducing an example of a random P{X = xk } = pk
variable, and how a probability assignment may be made on
the experimental outcomes. These probabilities are often referred to as probability masses
since they represent discrete (point) masses of probability at
II. P ROBABILITY A SSIGNMENTS specific locations along the real axis, just as point charges in
electrostatics are used to represent discrete charges distributed
Let N be a variable that represents the number of α particles
along a line. These probabilities, written or plotted as a
that are counted over a given period of time. The ensemble
function of x, is called the probability mass function (PMF).
for N is the set of non-negative integers
In order to express probability mass functions mathemat-
EN = {0, 1, 2, . . .} ically, we introduce the delta function,2 which is defined as
follows: 
Since the number of outcomes is unknown until we actually 1 ; n=0
make a count, then N is a random variable. In many cases, δ[n] =
0 ; n 6= 0
it is appropriate to model N as a Poisson random variable
Shifted delta functions may be used to represent functions that
where1
λn −λ have a value of one at other values of n. For example, δ[n−1]
P{N = n} = e n≥0 (1) is equal to one when n = 1 and equal to zero for all other
n!
for some λ > 0. values of n. Therefore, for an integer-valued discrete random
Given this probability assignment for N , it is then easy to variable X with
find the probability of any event that is defined in terms of P{X = n} = pX [n] ; −∞ < n < ∞
1 Note that with this probability assignment it is assumed that the number
of particles may be arbitrarily large and, in fact, approach infinity. 2 In digital signal processing, δ[n] is referred to as the unit sample function.
2 IEEE TRANSACTIONS ON LATEX

the PMF may be written as Name Density Function

∞
X
pX (n) = pX [k]δ[n − k] Exponential fX (x) = λe−λx
k=−∞ Laplace fX (x) = 12 αe−α|x−m|
2 2
For example, a Bernoulli Random Variable X has an ensemble Rayleigh fX (x) = α2 xe−α x /2 , x ≥ 0
of possible outcomes of zero and one Uniform fX (x) = 1/(b − a), b ≤ x ≤ a

EX = {0, 1}
TABLE I
where A TABLE OF COMMON AND IMPORTANT RANDOM VARIABLES .
P{X = 0} = p ; P{X = 1} = 1 − p
Therefore, the probability mass function for X may be ex-
pressed in terms of delta functions as follows:
Conversely, the distribution function is the integral of the
pX (n) = pδ[n] + (1 − p)δ[n − 1] density function,
Z x
Another example is the The Geometric Random Variable that FX (x) = fX (α)dα (2)
has an ensemble equal to the set of all positive integers −∞

EX = {1, 2, 3, . . .} Two properties of the density function are:

1) fX (x) ≥ 0 for all x.
with a probability law given by Z ∞
2) fX (x)dx = 1
P{N = k} = ( 12 )k ; k > 0 −∞
The first property follows from the monotonicity of the
The probability mass function for this random variable is
distribution function. Specifically, since FX (x) is a non-
∞
X decreasing function of x, then the derivative will always be
pN (n) = ( 12 )k δ[n − k]
non-negative for all x. The second property follows by letting
k=1
x → ∞, in Eq. (2),
Another random variable that occurs frequently in applica- Z ∞
tions is one that corresponds to the number of successes, N , fX (α)dα = FX (∞) = 1
in n Bernoulli trials, with the probability of a success being −∞

equal to p. In this case, N has a Binomial Distribution with It is important to understand that fX (x) is a probability
 
n k density and not a probability. Probabilities are found by
pN (k) = P{N = k} = p (1 − p)n−k ; 0 ≤ k ≤ n integrating fX (x). For example, the probability of the event
k
{X ≤ x} is given by the integral in Eq. (2), and the probability
where nk is the number of combinations of n objects that are


of the event {x1 ≤ X ≤ x2 } is found by evaluating the
taken k at a time, and is defined by integral Z x2
 
n n! P{x1 ≤ X ≤ x2 } = fX (α)dα (3)
= x1
k k!(n − k)!
Unlike the distribution function, which is constrained to have
Alternative notations include C(n, k), n Ck , n Ck , and Ckn .
values between zero and one, there is no such constraint on
Interesting problems that are sometimes challenging to the density function. In fact, fX (x) may be arbitrarily large
solve, are those such as as long as the integral of fX (x) over all x is equal to one (the
X
P{N is odd} = P{N = n} area under the curve is unity).
0≤n≤∞ A table of common random variables is given in Table I.
n odd

V. M OMENT G ENERATING F UNCTIONS

IV. P ROBABILITY D ENSITY F UNCTION The complete statistical characterization of a random vari-
able of the continuous type requires the specification of its
For a continuous random variable, the counterpart of the probability distribution or density function. Another way to
probability mass function is the probability density function. provide this statistical characterization is with the character-
The probability density function, fX (x), of a continuous ran- istic function, which is the expected value of ejωX
dom variable X is the derivative of the probability distribution Z ∞
function, MX (jω) = E{ejωX } = ejωx fX (x)dx
dFX (x) −∞
fX (x) =
dx where ω √is a variable taking on any real number between ±∞
where and j = −1. Since ejω = cos ω + j sin ω, the characteristic
FX (x) = P{X ≤ x} function is, in general, a complex-valued function of ω. It
HAYES 3

where the term under the braces is equal to one because this
is just the integral of a Gaussian density. Therefore, we have
2 2
MX (jω) = e−ω σx /2

The moments of a Gaussian may be derived easily from this

characteristic function.

VI. M ULTIPLE R ANDOM VARIABLES

In many applications, there will be two or more random

variables that are associated with a single experiment. In order
to work with these jointly distributed random variables, it is
necessary to specify the joint probability density function.
For a pair of continuous random variables, X and Y , the
Fig. 1. A Gaussian Density Function
joint density function is the derivative of the joint distribution
function,
may be recognized that, except for the sign in the exponent, ∂2
fX,Y (x, y) = FX,Y (x, y) (4)
MX (ejω ) is the Fourier transform of the probability density ∂x∂y
function. Just as Fourier transforms are useful in the analysis where
of signals and linear systems, the characteristic function is also FX,Y (x, y) = P{X ≤ x, Y ≤ y}
a useful tool in probability and random variables. Since
Z ∞ Z ∞ provided the derivatives exist. Conversely, the joint distribution
|MX (jω)| = ejωx fX (x)dx ≤ ejωx fX (x) dx function is the integral of the joint density function,
Z −∞ −∞ Z x Z y
∞ Z ∞
FX,Y (x, y) = fX,Y (α, β)dαdβ (5)
= ejωx |fX (x)| dx = fX (x)dx = 1 −∞ −∞
−∞ −∞
Given the joint density or distribution function, the probability
then the characteristic function is well-defined and will always
of any event that is defined in terms of values of X may be
exist for any probability density function.
found. For example, P{X ≤ x, Y ≤ y} = FX,Y (x, y) is
A. Exponential Random Variable found by integrating the joint density function as in Eq. (5).
Two important properties of the joint density function are
If X is an exponential random variable with parameter λ, given below.
fX (x) = λe−λx u(x) 1) fXY (x, y) ≥ 0 for all x, y.
Z ∞Z ∞
then the characteristic function is 2) fXY (x, y)dxdy = 1
Z ∞ Z ∞
jωx −λx −∞ −∞
MX (jω) = λe e dx = λ e(jω−λ)x dx The first property follows from the monotonicity of the
0 0
λ ∞ λ distribution function. Specifically, since FX (x) is a non-
= e(jω−λ)x = decreasing function of x, then the derivative will always be
jω − λ 0 λ − jω
non-negative for all x. The second property follows from
which is a complex-valued function of ω. Eq. (5) by letting x → ∞ and y → ∞,
Z ∞Z ∞
B. Gaussian Random Variable fXY (x, y)dxdy = 1
A zero-mean Gaussian random variable X has a density −∞ −∞
function of the form As was the case for a single random variable, it is important
1 2 2 to understand that the joint density function fXY (x, y) is
fX (x) = √ e−x /2σx
σx 2π a probability density and not a probability. Probabilities are
found by integrating the density function. For example, the
where σx2 is the variance of X. A plot of the density function
probability of the event
of a Gaussian for several different values of σx is shown in
Fig. 1. A = {x1 ≤ X ≤ x2 } ∪ {y1 ≤ X ≤ y2 }
The characteristic function is
Z ∞ is found by integrating the joint density function as follows
1 2 2
MX (jω) = √ ejωx e−x /2σx Z x2 Z y2
σx 2π −∞ P{x1 ≤ X ≤ x2 , y1 ≤ X ≤ y2 } = fXY (x, y)dxdy
Z ∞
2 2 1 2 2 x1 y1
= e−ω σx /2 √ e−(x−jωσx ) /2σx dx (6)
σ 2π −∞
|x {z } and to find the probability that (x, y) ∈ R where R is an
=1 arbitrarily defined region in the x-y plane, we integrate the
4 IEEE TRANSACTIONS ON LATEX

density function over R,

ZZ
P{(X, Y ) ∈ R} = fXY (x, y)dxdy (7)
R

Unlike the joint distribution function, which is constrained

to have values between zero and one, there is no such
constraint on the joint density function. In fact, fXY (x, y)
may be arbitrarily large as long as the integral of fXY (x, y)
over all x and y is equal to one (a) Density Function (b) Distribution Function

VII. C ORRELATION AND C ORRELATION M ATRICES Fig. 2. The Chi-square Random Variable.
For two or more random variables, an important ensemble
average is the correlation. Given two random variables X and
Y , the correlation is defined by IX. C ONCLUSION
There are many excellent textbooks where the reader may
rX1 X2 = E{X1 X2 } find advanced developments of the results presented in this
If we consider these two random variables to be a random paper. The classic work in the field is the text by Papoulis [1].
vector Another recommended text is [2]. An introduction to Monte
X = [X1 , X2 ]T Carlo simulations may be found in [3].

then the correlation matrix is the expected value of XXT , R EFERENCES

" #
2
E{X 1 } E{X 1 X2 } [1] A. Papoulis and S. Pillai, Probability, Random Variables, and Stochasic
RX = E{XXT } = Processes. New York: McGraw-Hill, 2002.
E{X2 X1 } E{X22 } [2] H. Larsen and B. Shubert, Probabilistic Models in Engineering Sciences,
Vol. 1. New York: John Wiley and Sons, 1979.
For n random variables, the correlation matrix has the form [3] S. Raychaudhuri, “Introduction to monte carlo simulation,” in Simulation
  Conference, 2008. WSC 2008. Winter, pp. 91 –100, Dec. 2008.
r11 r12 · · · r1n
 r21 r22 · · · r2n 
RX =  .
 
.. .. .. 
 .. . . . 
rn1 rn2 · · · rnn
where
rkl = E{Xk Xl }
These correlations are often estimated from a set of real-
izations (experimental outcomes) of the random variables as
follows Sample cross-correlation
n
X
(xi − x)(yi − y)
i=1
r̂xy = " n n
#
X X
(xi − x)2 (yi − y)2
i=1 i=1

where r̂xy is called a sample autocorrelation.

VIII. C HI -S QUARE R ANDOM VARIABLE

We conclude this paper with one last example of a random
variable, one that is found often in statistics. If Z1 , . . . , Zn
are independent, unit variance Gaussian random variables with
zero mean, then the sum of their squares,
n
X
Y = Zk2
k=1

is a Chi-square random variable with n degrees of freedom.

A plot of the probability density and cumulative distribution
functions are shown in Fig. 2.