McGill University

Civil Engineering and Applied Mechanics

Winter 2020

CIVE 302: Probabilistic Systems

Instructor: Dr. Mohammad H. Alobaidi

Chapter 4: Continuous Random Variables and Probability

Distributions

4-1 CONTINUOUS RANDOM VARIABLES

Suppose a dimensional length is measured on a manufactured part selected from a day’s

production. In practice, there can be small variations in the measurements due to many causes,
such as vibrations, temperature fluctuations, operator differences, calibrations, cutting tool wear,
bearing wear, and raw material changes. In an experiment such as this, the measurement is
naturally represented as a random variable and it is reasonable to model the range of possible
values of with an interval of real numbers. Recall from Chapter 2 that a continuous random
variable is a random variable with an interval (either finite or infinite) of real numbers for its range.
The model provides for any precision in length measurements.

Because the number of possible values of is uncountably infinite, has a distinctly different
distribution from the discrete random variables studied previously. But as in the discrete case, many
physical systems can be modeled by the same or similar continuous random variables. These
random variables are described, and example computations of probabilities, means, and variances
are provided in the remaining sections of this chapter.

1
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

4-2 PROBABILITY DISTRIBUTIONS AND PROBABILITY DENSITY

FUNCTIONS

Density functions are commonly used in engineering to describe physical systems. For example,
consider the density of a loading on a long, thin beam as shown in the figure below. For any point
along the beam, the density can be described by a function (in grams/cm). Intervals with large
loadings correspond to large values for the function. The total loading between points and is
determined as the integral of the density function from to . This integral is the area under the
density function over this interval, and it can be loosely interpreted as the sum of all the loadings
over this interval.

Probability Density Function

Similar to a density function of loading discussed above, a probability density function can
be used to describe the probability distribution of a continuous random variable . If an interval is
likely to contain a value for , its probability is large and it corresponds to large values for . The
probability that is between and is determined as the integral of from to .

Definition: For a continuous random variable , a probability density function is a function such
that

2
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

A probability density function provides a simple description of the probabilities associated with a
random variable. As long as is nonnegative and , so
that the probabilities are properly restricted. A probability density function is zero for values that
cannot occur and it is assumed to be zero wherever it is not specifically defined.

A histogram is an approximation to a probability density function. For each interval of the histogram,
the area of the bar equals the relative frequency (proportion) of the measurements in the interval.
The relative frequency is an estimate of the probability that a measurement falls in the interval.
Similarly, the area under over any interval equals the true probability that a measurement falls
in the interval.

The important point is that is used to calculate an area that represents the probability that
assumes a value in . For the current measurement example, the probability that results in
[14 mA, 15 mA] is the integral of the probability density function of X over this interval. The
probability that results in [14.5 mA, 14.6 mA] is the integral of the same function, , over the
smaller interval. By appropriate choice of the shape of , we can represent the probabilities
associated with any continuous random variable . The shape of determines how the
probability that assumes a value in [14.5 mA, 14.6 mA] compares to the probability of any other
interval of equal or different length.

For the density function of a loading on a long, thin beam, because every point has zero width, the
loading at any point is zero. Similarly, for a continuous random variable and any value ,

3
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Based on this result, it might appear that our model of a continuous random variable is useless.
However, in practice, when a particular current measurement is observed, such as 14.47
milliamperes, this result can be interpreted as the rounded value of a current measurement that is
actually in a range such as 14.465 ≤ ≤ 14.475 Therefore, the probability that the rounded value
14.47 is observed as the value for is the probability that assumes a value in the interval
[14.465, 14.475], which is not zero. Similarly, because each point has zero probability, one need not
distinguish between inequalities such as < or ≤ for continuous random variables.

Property: if is a continuous random variable, for any and ,

Example 4-1

Let the continuous random variable denote the current measured in a thin copper wire in
milliamperes. Assume that the range of is [0, 20 mA], and assume that the probability density
function of is = 0.05 for 0 ≤ ≤ 20. What is the probability that a current measurement is
less than 10 milliamperes?

The probability density function is shown in a figure below. It is assumed that = 0 wherever it is
not specifically defined. The probability requested is indicated by the shaded area in the figure.

As another example,

4
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

4-3 CUMULATIVE DISTRIBUTION FUNCTIONS

An alternative method to describe the distribution of a discrete random variable can also be used for
continuous random variables.

Cumulative Distribution Function

Definition: the cumulative distribution function of a continuous random variable is

for

Extending the definition of to the entire real line enables us to define the cumulative distribution
function for all real numbers.

Example 4-2

For the copper current measurement in Example 4-1, the cumulative distribution function of the
random variable consists of three expressions. If < 0, = 0. Therefore,

, for

and

, for

Finally,

, for

Therefore,

5
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Notice that in the definition of , any "<" can be changed to "≤" and vice versa. That is,
can be defined as either 0.05x or 0 at the end-point and = 0, and can be defined as either
0.05 or 1 at the end-point = 20. In other words, is a continuous function. For a discrete
random variable, is not a continuous function. Sometimes, a continuous random variable is
defined as one that has a continuous cumulative distribution function.

Properties

The following properties exist for a CDF (cumulative distribution function).

• As goes to ,
• As goes to ,
• is monotonically increasing ( is positive)
•

The probability density function of a continuous random variable can be determined from the
cumulative distribution function by differentiating.

4-4 MEAN AND VARIANCE OF A CONTINUOUS RANDOM

VARIABLE

The mean and variance can also be defined for a continuous random variable. Integration replaces
summation in the discrete definitions. If a probability density function is viewed as a loading on a
beam, the mean is the balance point.

6
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

Mean and Variance

Definition: Suppose is a continuous random variable with probability density function . The
mean or expected value of , denoted as or , is

The standard deviation of is .

The equivalence of the two formulas for variance can be derived from the same approach used for
discrete random variables.

Example 4-3

For the copper current measurement in Example 4-1, the mean of is

The variance of is

Expected Value of a Function of a Continuous Random Variable

Definition: if is a continuous random variable with probability density function ,

7
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Example 4-4

In Example 4-1, is the current measured in milliamperes. What is the expected value of the
squared current? Now, . Therefore,

In the previous example, the expected value of does not equal squared. However, in the
special case that for any constants and , . This can be
shown from the properties of integrals.

4-5 CONTINUOUS UNIFORM DISTRIBUTION

The simplest continuous distribution is analogous to its discrete counterpart.

Continuous Uniform Distribution

Definition: a continuous random variable with probability density function

is a continuous Uniform random variable.

8
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

Mean and Variance

Definition: if is a continuous Uniform random variable over ,

The mean of the continuous Uniform random variable is

The variance of is

Cumulative Distribution Function

The cumulative distribution function of a continuous Uniform random variable is obtained by

integration. If ,

Therefore, the complete description of the cumulative distribution function of a continuous Uniform
random variable is

Below, we show an example of a MATLAB code for visualising the distributions.

9
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

% Plot PDFs of continuous Uniform distributions

% Create distribution objects with different parameters

lower_1 = -3;
upper_1 = 2; % upper must be greater than lower
lower_2 = -1;
upper_2 = 2; % upper must be greater than lower

pd1 = makedist('Uniform','lower',lower_1,'upper',upper_1);
pd2 = makedist('Uniform','lower',lower_2,'upper',upper_2);

% Compute the pdfs

x = -3:.01:3;
pdf1 = pdf(pd1,x);
pdf2 = pdf(pd2,x);

% Plot the pdfs

figure;
stairs(x,pdf1,'r','LineWidth',2);
hold on;
stairs(x,pdf2,'b-.','LineWidth',2);
ylim([0 1.1]);
title('Uniform PDFs')
legend({'lower =' + string(lower_1) + ', ' + 'upper =' + string(upper_1),'lower
=' + string(lower_2) + ', ' + 'upper =' + string(upper_2)},'Location','NW');
hold off;

10
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

% Compute the cdfs

cdf1 = cdf(pd1,x);
cdf2 = cdf(pd2,x);

% Plot the cdfs

figure;
plot(x,cdf1,'r','LineWidth',2);
hold on;
plot(x,cdf2,'b-.','LineWidth',2);
ylim([0 1.1]);
title('Uniform CDFs')
legend({'lower =' + string(lower_1) + ', ' + 'upper =' + string(upper_1),'lower
=' + string(lower_2) + ', ' + 'upper =' + string(upper_2)},'Location','NW');
hold off;

11
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Example 4-5

Let the continuous random variable denote the current measured in a thin copper wire in
milliamperes. Assume that the range of is [0, 20 mA], and assume that the probability density
function of is = 0.05, 0 ≤ ≤ 20.

What is the probability that a measurement of current is between 5 and 10 milliamperes?

The mean and variance formulas can be applied with = 0 and = 20. Therefore,

= 10 mA and = 202/12 = 33.33 mA^2

Consequently, the standard deviation of is 5.77 mA.

12
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

4-6 NORMAL DISTRIBUTION

Undoubtedly, the most widely used model for the distribution of a random variable is a Normal
distribution. Whenever a random experiment is replicated, the random variable that equals the
average (or total) result over the replicates tends to have a Normal distribution as the number of
replicates becomes large. De Moivre presented this fundamental result, known as the central limit
theorem, in 1733. Unfortunately, his work was lost for some time, and Gauss independently
developed a normal distribution nearly 100 years later. Although De Moivre was later credited with
the derivation, a normal distribution is also referred to as a Gaussian distribution.

When do we average (or total) results? Almost always. For example, an automotive engineer may
plan a study to average pull-off force measurements from several connectors. If we assume that
each measurement results from a replicate of a random experiment, the Normal distribution can be
used to make approximate conclusions about this average. These conclusions are the primary topics
in the subsequent chapters of this book.

Furthermore, sometimes the central limit theorem is less obvious. For example, assume that the
deviation (or error) in the length of a machined part is the sum of a large number of infinitesimal
effects, such as temperature and humidity drifts, vibrations, cutting angle variations, cutting tool
wear, bearing wear, rotational speed variations, mounting and fixture variations, variations in
numerous raw material characteristics, and variation in levels of contamination. If the component
errors are independent and equally likely to be positive or negative, the total error can be shown to
have an approximate Normal distribution. Furthermore, the Normal distribution arises in the study of
numerous basic physical phenomena. For example, the physicist Maxwell developed a Normal
distribution from simple assumptions regarding the velocities of molecules.

The theoretical basis of a Normal distribution is mentioned to justify the somewhat complex form of
the probability density function. Our objective now is to calculate probabilities for a Normal random
variable. The central limit theorem will be stated more carefully later.

Random variables with different means and variances can be modeled by Normal probability density
functions with appropriate choices of the center and width of the curve. The value of
determines the center of the probability density function and the value of determines the
width. Each of Normal distributions has the characteristic symmetric bell-shaped curve, but the
centers and dispersions differ. The following definition provides the formula for Normal probability
density functions.

13
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Normal Distribution

Definition: a random variable with probability density function

is a Normal random variable with parameters , where , and . Also,

and

and the notation is used to denote the distribution.

% Plot Normal distributions

% Create distribution objects with different parameters

mu_1 = -2;
mu_2 = 0;
sigma_1 = 1.5;
sigma_2 = 2;

% Compute the pdfs

x = -10:.01:10;
pdf1 = normpdf(x,mu_1,sigma_1);
pdf2 = normpdf(x,mu_2,sigma_2);

% Plot the pdfs

figure;
stairs(x,pdf1,'r','LineWidth',2);
hold on;
stairs(x,pdf2,'b-.','LineWidth',2);
title('Normal PDFs')
legend({'mu =' + string(mu_1) + ', ' + 'sigma =' + string(sigma_1),'mu =' +
string(mu_2) + ', ' + 'sigma =' + string(sigma_2)},'Location','NW');
hold off;

14
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

% Compute the cdfs

cdf1 = cdf(pd1,x);
cdf2 = cdf(pd2,x);

% Plot the cdfs

figure;
plot(x,cdf1,'r','LineWidth',2);
hold on;
plot(x,cdf2,'b-.','LineWidth',2);
title('Normal CDFs')
legend({'mu =' + string(mu_1) + ', ' + 'sigma =' + string(sigma_1),'mu =' +
string(mu_2) + ', ' + 'sigma =' + string(sigma_2)},'Location','NW');
hold off;

15
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Properties

1. The Normal distribution is always symmetrical (mean, median and mode are the same) and
mesokurtic.

2. Also, for any Normal random variable,

16
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

Also, from the symmetry of , = = 0.5. Because is positive for all , this
model assigns some probability to each interval of the real line. However, the probability density
function decreases as moves farther from . Consequently, the probability that a measurement
falls far from is small, and at some distance from the probability of an interval can be
approximated as zero.

The area under a Normal probability density function beyond 3 from the mean is quite small. This
fact is convenient for quick, rough sketches of a Normal probability density function. The sketches
help us determine probabilities. Because more than 0.9973 of the probability of a Normal distribution
is within the interval ( − 3 , + 3 ), 6 is often referred to as the width of a Normal distribution.
Advanced integration methods can be used to show that the area under the Normal probability
density function from is is 1.

3. The Normal distribution has reproductive property.

Let be the sum of normally distributed random variables:

where 's are constants, and

Then, will be Normal , where

Assume are independent.

17
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

4. The Normal distribution is preserved under a linear transformation:

i.e., if , then follows a Normal distribution .

Standard Normal Random Variable

Definition: a Normal random variable with

and

is called a standard Normal random variable and is denoted as . The cumulative distribution
function of a standard Normal random variable is denoted as

Note that .

Table below provides cumulative probabilities for a standard Normal random variable. Cumulative
distribution functions for Normal random variables are also widely available in computer packages.
They can be used in the same manner as the table to obtain probabilities for these random
variables.

18
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

19
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Standardizing a Normal Random Variable

Definition: If is a Normal random variable with and , the random variable

is a Normal random variable with and . That is, is a standard Normal random
variable.

20
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

Creating a new random variable by this transformation is referred to as standardizing. The random
variable represents the distance of from its mean in terms of standard deviations. It is the key
step to calculating a probability for an arbitrary Normal random variable.

Standardizing to Calculate a Probability

Suppose is a Normal random variable with mean and variance . Then,

where is a standard Normal random variable, and

is the -value obtained by standardizing . The probability is obtained by using the look-up table
with .

Example 4-6

The drainage from a community during a storm is a Normal random variable estimated to have a
mean of 1.2 million gallons per day (mgd) and a standard deviation of 0.4 mgd. If the storm drain
system is designed with a maximum drainage capacity of 1.5 mgd, what is the underlying probability
of flooding during a storm that is assumed in the design of the drainage system?

Flooding in the community will occur when the drainage load exceeds the capacity of the drainage
system; therefore, the probability of flooding is

In the above, we obtained =0.7734 from the look-up table. Also of related interest are the
following: (i) The probability that the drainage during a storm will be between 1.0 mgd and 1.6 mgd,
which is

21
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

(ii) The 90-percentile drainage load from the community during a storm. This is the value of the
random variable at which the cumulative probability is less than 0.90, which we would obtain as

Therefore,

From the look-up table, we obtain thus

mgd

4-7 NORMAL APPROXIMATION TO THE BINOMIAL AND POISSON

DISTRIBUTIONS

We began our section on the Normal distribution with the central limit theorem and the Normal
distribution as an approximation to a random variable with a large number of trials. Consequently, it
should not be a surprise to learn that the Normal distribution can be used to approximate Binomial
probabilities for cases in which is large. The following example illustrates that for many physical
systems the Binomial model is appropriate with an extremely large value for . In these cases, it is
difficult to calculate probabilities by using the Binomial distribution. Fortunately, the Normal
approximation is most effective in these cases. An illustration is provided in a figure below. The area
of each bar equals the Binomial probability of . Notice that the area of bars can be approximated by
areas under the Normal density function.

22
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

From the figure, it can be seen that a probability such as (3 ≤ ≤ 7) is better approximated by the
area under the Normal curve from 2.5 to 7.5. This observation provides a method to approximate
Binomial probabilities. Because a continuous Normal distribution is used to approximate a discrete
Binomial distribution, the modification is referred to as a continuity correction.

Normal Approximation to the Binomial Distribution

Definition: if is a Binomial random variable with parameters and ,

is approximately a standard Normal random variable. To approximate a Binomial probability with a

Normal distribution, a continuity correction is applied as follows:

and

The approximation is good for > 5 and > 5.

23
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Recall that for a Binomial variable , and . Consequently, the

expression above is nothing more than the formula for standardizing the random variable .
Probabilities involving can be approximated by using a standard Normal distribution. The
approximation is good when is large relative to .

A way to remember the approximation is to write the probability in terms of ≤ or ≥ and then add or
subtract the 0.5 correction factor to make the probability greater.

Example 4-7

In a digital communication channel, assume that the number of bits received in error can be modeled
by a Binomial random variable, and assume that the probability that a bit is received in error is
. If 16 million bits are transmitted, what is the probability that 150 or fewer errors occur?

Let the random variable denote the number of errors. Then is a Binomial random variable and

Clearly this probability is difficult to compute. Fortunately, the Normal distribution can be used to
provide an excellent approximation in this example.

Because = (16 × )(1 × ) = 106 and is much larger, the approximation is

expected to work well in this case.

Practical Interpretation: Binomial probabilities that are difficult to compute exactly can be
approximated with easy to compute probabilities based on the Normal distribution.

The correction factor is used to improve the approximation. However, if or is small, the
Binomial distribution is quite skewed and the symmetric Normal distribution is not a good
approximation.

Recall that the Binomial distribution is a satisfactory approximation to the hypergeometric distribution
when , the sample size, is small relative to , the size of the population from which the sample is
selected. A rule of thumb is that the Binomial approximation is effective if < 0.1. Recall that for a
hypergeometric distribution, is defined as . That is, is interpreted as the number of
successes in the population. Therefore, the Normal distribution can provide an effective
approximation of hypergeometric probabilities when < 0.1, > 5, and > 5.

24
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

% Normal distribution and Binomial distribution

n_bi = 30;
p_bi = 0.4;

x1=1:n_bi;
pd1 = binopdf(x1,n_bi,p_bi);
x2=1:.01:n_bi;
pd2 = normpdf(x2,n_bi*p_bi,sqrt(n_bi*p_bi*(1-p_bi)));

figure
bar(pd1,'blue')
hold on;
stairs(x2,pd2,'r','LineWidth',2);
xlabel('X')
ylabel('Probability (density)')
title('Normal approximation to Binomial distribution')
hold off;

25
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Normal Approximation to the Poisson Distribution

Recall that the Poisson distribution was developed as the limit of a Binomial distribution as the
number of trials increased to infinity. Consequently, it should not be surprising to find that the Normal
distribution can also be used to approximate probabilities of a Poisson random variable.

Definition: if is a Poisson random variable with and ,

is approximately a standard Normal random variable. The same continuity correction used for the
Binomial distribution can also be applied. The approximation is good for .

Example 4-8

Assume that the number of asbestos particles in a squared meter of dust on a surface follows a
Poisson distribution with a mean of 1000. If a squared meter of dust is analyzed, what is the
probability that 950 or fewer particles are found?

This probability can be expressed exactly as

The computational difficulty is clear. The probability can be approximated as

Practical Interpretation: Poisson probabilities that are difficult to compute exactly can be
approximated with easy to compute probabilities based on the Normal distribution.

26
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

4-8 LOGNORMAL DISTRIBUTION

Variables in a system sometimes follow an Exponential relationship as . If the exponent

is a random variable , then = is a random variable with a distribution of interest. An
important special case occurs when has a normal distribution. In that case, the distribution of
is called a Lognormal distribution. The name follows from the transformation = . That is,
the natural logarithm of is normally distributed.

Probabilities for are obtained from the transformation to , but the range of is (0, ).
Suppose that is normally distributed with mean and variance ; then the cumulative
distribution function for is

for > 0, where is a standard normal random variable. Therefore, the look-up table cab be used
to determine the probability. Also, for .

The probability density function of can be obtained from the derivative of . This derivative is
applied to the last term in the expression for , the integral of the standard normal density
function. Furthermore, from the probability density function, the mean and variance of can be
derived. The details are omitted, but a summary of results follows.

Lognormal Distribution

Let have a normal distribution with mean and variance ; then is a Lognormal
random variable with probability density function

The mean and variance of are

The parameters of a Lognormal distribution are and , but care is needed to interpret that these
are the mean and variance of the normal random variable . The mean and variance of are the
functions of these parameters.

27
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

The lifetime of a product that degrades over time is often modeled by a Lognormal random variable.
For example, this is a common distribution for the lifetime of a semiconductor laser. A Weibull
distribution can also be used in this type of application, and with an appropriate choice for
parameters, it can approximate a selected Lognormal distribution. However, a Lognormal distribution
is derived from a simple Exponential function of a normal random variable, so it is easy to
understand and easy to evaluate probabilities.

Example 4-9

The lifetime of a semiconductor laser has a Lognormal distribution with = 10 hours and = 1.5
hours. What is the probability that the lifetime exceeds 10,000 hours?

From the cumulative distribution function for ,

What lifetime is exceeded by 99% of lasers? The question is to determine such that =
0.99. Therefore,

From the look-up table, when = -2.33. Therefore,

Determine the mean and standard deviation of lifetime. Now,

so the standard deviation of is 197,661.5 hours.

Practical Interpretation: The standard deviation of a log-normal random variable can be large relative
to the mean.

28
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

% Plot the Lognormal distribution

theta_1 = 0.2;
theta_2 = 0.4;
omega_1 = 0.7;
omega_2 = 1.5;

pd1=makedist('Lognormal','mu',theta_1,'sigma',omega_1);
pd2=makedist('Lognormal','mu',theta_2,'sigma',omega_2);

x=0:0.01:10;
y1=pdf(pd1,x);
y2=pdf(pd2,x);

figure
plot(x,y1,'blue','Linewidth',2)
hold on;
plot(x,y2,'red','Linewidth',2)
xlabel('X')
ylabel('Probability (density)')
title('Lognormal distributions')
legend({'theta =' + string(theta_1) + ', ' + 'omega =' + string(omega_1),'theta
=' + string(theta_2) + ', ' + 'omega =' + string(omega_2)})
hold off;

29
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

4-9 EXPONENTIAL DISTRIBUTION

The discussion of the Poisson distribution defined a random variable to be the number of flaws along
a length of copper wire. The distance between flaws is another random variable that is often of
interest. Let the random variable denote the length from any starting point on the wire until a flaw
is detected. As you might expect, the distribution of can be obtained from knowledge of the
distribution of the number of flaws. The key to the relationship is the following concept. The distance
to the first flaw exceeds 3 millimeters if and only if there are no flaws within a length of 3
millimeters—simple, but sufficient for an analysis of the distribution of .

In general, let the random variable denote the number of flaws in millimeters of wire. If the
mean number of flaws is per millimeter, has a Poisson distribution with mean . We assume
that the wire is longer than the value of . Now,

Therefore,

30
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

is the cumulative distribution function of . By differentiating , the probability density function of

is calculated to be

The derivation of the distribution of depends only on the assumption that the flaws in the wire
follow a Poisson process. Also, the starting point for measuring doesn’t matter because the
probability of the number of flaws in an interval of a Poisson process depends only on the length of
the interval, not on the location. For any Poisson process, the following general result applies.

Exponential Distribution

Definition: the random variable that equals the distance between successive events of a Poisson
process with mean number of events per unit interval is an Exponential random variable with
parameter . The probability density function of is

The Exponential distribution obtains its name from the Exponential function in the probability density
function. For any value of , the Exponential distribution is quite skewed. The following results are
easily obtained and are left as an exercise.

Mean and Variance

Definition: if the random variable has an Exponential distribution with parameter ,

It is important to use consistent units in the calculation of probabilities, means, and variances
involving Exponential random variables. The following example illustrates unit conversions.

31
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Relation to Poisson Distribution

An interesting feature of the Poisson and exponential distributions is that, if the Poisson provides an
appropriate description of the number of occurrences per interval of time, then the exponential will
provide a description of the length of time between occurrences. To understand this, consider that, in
a Poisson process, if events occur on average at the rate of per unit of time, then there will be
average occurrences per units of time. The Poisson distribution describing this process is
therefore , from which is the probability of no occurrences in
units of time.

Another interpretation of is that this is the probability that the time, , to the first
occurrence is greater than , i.e.,

Conversely, the probability that an event does not occur during units of time is given by

Note that this is the cumulative exponential distribution which, when differentiated with respect to ,
produces the probability density function of the exponential distribution .

Example 4-10

In a large corporate computer network, user log-ons to the system can be modeled as a Poisson
process with a mean of 25 log-ons per hour. What is the probability that there are no log-ons in an
interval of 6 minutes?

Let denote the time in hours from the start of the interval until the first log-on. Then, has an
Exponential distribution with = 25 log-ons per hour. We are interested in the probability that
exceeds 6 minutes. Because is given in log-ons per hour, we express all time units in hours. That
is, 6 minutes = 0.1 hour. Therefore,

Also, the cumulative distribution function can be used to obtain the same result as follows:

32
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

An identical answer is obtained by expressing the mean number of log-ons as 0.417 log-ons per
minute and computing the probability that the time until the next log-on exceeds 6 minutes. Try it.

What is the probability that the time until the next log-on is between 2 and 3 minutes? Upon
converting all units to hours,

An alternative solution is

Determine the interval of time such that the probability that no log-on occurs in the interval is 0.90.
The question asks for the length of time such that = 0.90. Now,

Take the (natural) log of both sides to obtain . Therefore,

hour minutes

The standard deviation of the time until the next log-on is

hours = minutes

Practical Interpretation: Probabilities for Exponential random variables are widely used by
organization to evaluate resources and staffing levels to meet customer service needs.

In the previous example, the probability that there are no log-ons in a 6-minute interval is 0.082
regardless of the starting time of the interval. A Poisson process assumes that events occur
uniformly throughout the interval of observation; that is, there is no clustering of events. If the log-ons
are well modeled by a Poisson process, the probability that the first log-on after noon occurs after
12:06 P.M. is the same as the probability that the first log-on after 3:00 P.M. occurs after 3:06 P.M.
And if someone logs on at 2:22 P.M., the probability that the next log-on occurs after 2:28 P.M. is still
0.082.

Our starting point for observing the system does not matter. However, if there are high-use periods
during the day, such as right after 8:00 A.M., followed by a period of low use, a Poisson process is
not an appropriate model for log-ons and the distribution is not appropriate for computing
probabilities. It might be reasonable to model each of the high- and low-use periods by a separate
Poisson process, employing a larger value for during the high-use periods and a smaller value
otherwise. Then, an Exponential distribution with the corresponding value of can be used to
calculate log-on probabilities for the high- and low-use periods.

33
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Lack of Memory Property

An even more interesting property of an Exponential random variable is concerned with conditional
probabilities.

Example 4-11

Let denote the time between detections of a particle with a Geiger counter and assume that
has an Exponential distribution with minutes. The probability that we detect a particle
within 30 seconds of starting the counter is

In this calculation, all units are converted to minutes. Now, suppose we turn on the Geiger counter
and wait 3 minutes without detecting a particle. What is the probability that a particle is detected in
the next 30 seconds?

Because we have already been waiting for 3 minutes, we feel that we are “due.” That is, the
probability of a detection in the next 30 seconds should be greater than 0.3. However, for an
Exponential distribution, this is not true. The requested probability can be expressed as the
conditional probability that . From the definition of conditional probability,

where

and

Therefore,

Practical Interpretation: After waiting for 3 minutes without a detection, the probability of a detection
in the next 30 seconds is the same as the probability of a detection in the 30 seconds immediately
after starting the counter. The fact that you have waited 3 minutes without a detection does not
change the probability of a detection in the next 30 seconds.

34
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

Example above illustrates the lack of memory property of an Exponential random variable, and a
general statement of the property follows. In fact, the Exponential distribution is the only continuous
distribution with this property.

Property: for an Exponential random variable ,

Figure below graphically illustrates the lack of memory property. The area of region A divided by the
total area under the probability density function (A + B + C + D = 1) equals . The area of
region C divided by the area C + D equals . The lack of memory property
implies that the proportion of the total area that is in A equals the proportion of the area in C and D
that is in C. The mathematical verification of the lack of memory property is left as a mind-expanding
exercise.

The lack of memory property is not that surprising when you consider the development of a Poisson
process. In that development, we assumed that an interval could be partitioned into small intervals
that were independent. These subintervals are similar to independent Bernoulli trials that comprise a
Binomial experiment; knowledge of previous results does not affect the probabilities of events in
future subintervals. An Exponential random variable is the continuous analog of a geometric random
variable, and they share a similar lack of memory property.

The Exponential distribution is often used in reliability studies as the model for the time until failure of
a device. For example, the lifetime of a semiconductor chip might be modeled as an Exponential
random variable with a mean of 40,000 hours. The lack of memory property of the Exponential
distribution implies that the device does not wear out. That is, regardless of how long the device has
been operating, the probability of a failure in the next 1000 hours is the same as the probability of a
failure in the first 1000 hours of operation. The lifetime of a device with failures caused by random
shocks might be appropriately modeled as an Exponential random variable. However, the lifetime
of a device that suffers slow mechanical wear, such as bearing wear, is better modeled by a
distribution such that increases with . Distributions such as the Weibull
distribution are often used, in practice, to model the failure time of this type of device. The Weibull
distribution is presented in a later section.

35
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

% Plot Exponential distributions

lambda_1 = 2.9;
lambda_2 = 1.5;

x=0:0.1:10;
y1=exppdf(x,lambda_1);
y2=exppdf(x,lambda_2);

figure
plot(x,y1,'blue','Linewidth',2)
hold on;
plot(x,y2,'red','Linewidth',2);
xlabel('X')
ylabel('Probability density')
title('Exponential distributions')
legend({'lambda =' + string(lambda_1), 'lamnda =' + string(lambda_2)})
hold off;

36
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

4-10 ERLANG AND GAMMA DISTRIBUTIONS

An Exponential random variable describes the length until the first count is obtained in a Poisson
process. A generalization of the Exponential distribution is the length until counts occur in a
Poisson process. Consider the following example.

Example 4-12

The failures of the central processor units of large computer systems are often modeled as a
Poisson process. Typically, failures are not caused by components wearing out, but by more random
failures of the large number of semiconductor circuits in the units. Assume that the units that fail are
immediately repaired, and assume that the mean number of failures per hour is 0.0001. Let
denote the time until four failures occur in a system. Determine the probability that exceeds
40,000 hours.

Let the random variable denote the number of failures in 40,000 hours of operation. The time until
four failures occur exceeds 40,000 hours if and only if the number of failures in 40,000 hours is three
or less. Therefore,

The assumption that the failures follow a Poisson process implies that has a Poisson distribution
with

failures per 40,000 hours

Therefore,

The previous example can be generalized to show that if is the time until the th event in a
Poisson process, then

Because , the probability density function of equals the negative of the

derivative of the right-hand side of the previous equation. After extensive algebraic simplification, the
probability density function of can be shown to equal

for and

37
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

This probability density function defines an Erlang distribution. Clearly, an Erlang random variable
with = 1 is an Exponential random variable.

It is convenient to generalize the Erlang distribution to allow to assume any nonnegative value.
Then the Erlang and some other common distributions become special cases of this generalized
distribution. To accomplish this step, the factorial function has to be generalized to apply to
any nonnegative value of ; but the generalized function should still equal ()! when is a positive
integer.

Gamma Function

Definition: the Gamma function is

, for

It can be shown that the integral in the definition of is finite. Furthermore, by using integration by
parts it can be shown that

Therefore, if is a positive integer (as in the Erlang distribution),

Also, and it can be shown that . The Gamma function can be interpreted
as a generalization to non-integer values of of the term that is used in the Erlang probability
density function. Now the Erlang distribution can be generalized.

Gamma Distribution

Definition: the random variable with probability density function

, for

has a Gamma random variable with parameters and . If is an integer, has an

Erlang distribution.

38
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

The parameters and are often called the scale and shape parameters, respectively. However,
one should check the definitions used in software packages. For example, Minitab defines the scale
parameter as . Many different shapes can be generated from changes to the parameters. Also,
the change of variable and the definition of the Gamma function can be used to show that
the density integrates to one.

% Plot Gamma distributions

lambda_1 = 0.8;
lambda_2 = 1.2;
r_1 = 1;
r_2 = 3.5;

x=0:0.1:10;
y1=gampdf(x,r_1,1/lambda_1);
y2=gampdf(x,r_2,1/lambda_2);

figure
plot(x,y1,'blue','Linewidth',2)
hold on;
plot(x,y2,'red','Linewidth',2);
xlabel('X')
ylabel('Probability density')
title('Gamma distributions')
legend({'lambda =' + string(lambda_1) + ', ' + 'r=' + string(r_1), 'lamnda =' +
string(lambda_2) + ', ' + 'r=' + string(r_2)})
hold off;

39
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Recall that for an Exponential distribution with parameter , the mean and variance are and
, respectively. An Erlang random variable is the time until the th event in a Poisson process
and the time between events are independent. Therefore, it is plausible that the mean and variance
of a Gamma random variable multiply the Exponential results by . This motivates the following
conclusions. Repeated integration by parts can be used to derive these, but the details are lengthy
and omitted.

Mean and Variance

Definition: if is a Gamma random variable with parameters and ,

40
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

4-11 WEIBULL DISTRIBUTION

As mentioned previously, the Weibull distribution is often used to model the time until failure of many
different physical systems. The parameters in the distribution provide a great deal of flexibility to
model systems in which the number of failures increases with time (bearing wear), decreases with
time (some semiconductors), or remains constant with time (failures caused by external shocks to
the system).

Weibull Distribution

Definition: the random variable with probability density function

, for

is a Weibull random variable with scale parameter and shape parameter .

Cumulative Distribution Function

The cumulative distribution function is often used to compute probabilities.

Definition: if has a Weibull distribution with parameters and , then the cumulative distribution
function of is

Mean and Variance

Definition: if has a Weibull distribution with parameters and ,

41
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

Example 4-13

The time to failure (in hours) of a bearing in a mechanical shaft is satisfactorily modeled as a Weibull
random variable with and hours. Determine the mean time until failure.

From the expression for the mean,

Determine the probability that a bearing lasts at least 6000 hours. Now

Practical Interpretation: Consequently, only 23.7% of all bearings last at least 6000 hours.

% Plot Weibull distributions

beta_1 = 4;
delta_1 = 3;
beta_2 = 2;
delta_2 = 2.5;

x=0:0.1:10;
y1=wblpdf(x,delta_1,beta_1);
y2=wblpdf(x,delta_2,beta_2);

figure
plot(x,y1,'blue','Linewidth',2)
hold on;
plot(x,y2,'red','Linewidth',2)
xlabel('X')
ylabel('Probability density')
title('Weibull distributions')

42
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

legend({'beta =' + string(beta_1) + ', ' + 'delta=' + string(delta_1), 'beta ='

+ string(beta_2) + ', ' + 'delta=' + string(delta_2)})
hold off;

4-12 BETA DISTRIBUTION

A continuous distribution that is flexible, but bounded over a finite range, is useful for probability
models. The proportion of solar radiation absorbed by a material or the proportion (of the maximum
time) required to complete a task in a project are examples of continuous random variables over the
interval [0, 1].

Definition: the random variable with probability density function

, for

is a Beta random variable with parameters and .

43
Alobaidi, M. H. Chapter 4: Continuous RVs & Prob. Distributions

The shape parameters and allow the probability density function to assume many different
shapes. If the distribution is symmetric about , and if the Beta distribution
equals a continuous uniform distribution.

In general, there is not a closed-form expression for the cumulative distribution function, and
probabilities for Beta random variables need to be computed numerically.

Mean and Variance

Definition: if has a Beta distribution with parameters and ,

Example 4-14

Consider the completion time of a large commercial development. The proportion of the maximum
allowed time to complete a task is modeled as a Beta random variable with and .
What is the probability that the proportion of the maximum time exceeds 0.7?

Suppose denotes the proportion of the maximum time required to complete the task. The
probability is

% Plot Beta distributions

alpha_1 = 1;
alpha_2 = 2;
beta_1 = 0.5;
beta_2 = 2;

44
McGill University - Winter 2020 CIVE 302 - Probabilistic Systems

x=0:0.01:1;
y1=betapdf(x,alpha_1,beta_1);
y2=betapdf(x,alpha_2,beta_2);

figure
plot(x,y1,'blue','Linewidth',2)
hold on;
plot(x,y2,'red','Linewidth',2);
xlabel('X')
ylabel('Probability density')
title('Beta distributions')
legend({'alpha =' + string(alpha_1) + ', ' + 'beta=' + string(beta_1), 'alpha
=' + string(alpha_2) + ', ' + 'beta=' + string(beta_2)})
hold off;

45