-3-

Chapter 2: Description and Definition of Stochastic Processes

2.1 Description
Normally we are not interested in the value of a single random variable but rather a whole
collection of random variables. Let us indicate the variables we are interested in by X t
where t is some element of the set T . We refer to t as the indexing parameter for the
stochastic process. In many cases t denotes the time of the observation e.g. the size of a
queue for all times in a certain interval may be of interest. We usually use 0 (but could also
use 1) for the starting point for some process and X 0 is then the random variable that
denotes the starting value of the process – which may be some constant with probability 1.

For each value t ∈ T , the random variable X t is a function defined on a probability space
(Ω, ξ , P ) which is preferably the same for all X t . On this probability space P is a
probability measure function which gives the probabilities of all the subsets of the
sample space Ω which are elements of the σ - field ξ .

NOTE: If ω ∈ Ω , then X t (ω ) denotes the value of the random variable X t for the element
ω of the sample space. For different values of the indexing parameter t , say t1 and t 2 , we
have that X t1 and X t2 are different random variables.

Definition 2.1.1 A stochastic process is a collection of random variables

{X t :t ∈ T } .

All the possible values that can be assumed by X t , t ∈ T , are called the states of the
stochastic process and the set of all possible states is called the state space. Let S denote
the state space. Since the X t ’s are random variables the possible values are the real numbers
i.e. S is a subset of the real numbers.

The set of all indexing parameters T is called the parameter space. If the parameter t is
time the value of X t is called the state the process is in at time t . In many cases the
indexing parameter t is time, but other parameter spaces are possible. The parameter space
T is called discrete if it is finite or enumerable and is called continuous if it is not
enumerable. Similarly, the state space S can be either discrete or continuous.

To identify the nature of a stochastic process, a first step is to classify them on the basis of
the nature of the parameter space and the nature of the state space. For example:

Example 2.1.1
 -4-

Nature of State Nature of Parameter Space

Space Discrete Continuous
T: Number of items inspected T: Time of day
Discrete
S: Number of defective items observed S: Number of persons in a queue

T: Number of persons arriving at a bus stop T: Time

Continuous
S: Waiting time for the bus S: Content of a dam
♪
2.2 Sample Paths and Probability Descriptions
Suppose that {X t : t ∈ T } , where T is some ordered set, is a stochastic process defined on
the probability space (Ω, ξ , P ) i.e. X t : Ω → R is a random variable defined on the
probability space (Ω, ξ , P ) for all values of t ∈ T . A single performance of the ‘experiment’
corresponding to this sample space, will determine an element ω from Ω . For this element
ω ∈ Ω we can determine the value of X t (ω ) for each t ∈ T . The set of points
{(t , X t ) : t ∈ T } is called a sample path of the process. For instance, if X t is the number of
insurance claims received up to and including time t , then the sample path may be as
follows:

Example 2.2.1

Figure 2.2.1 Number of Claims

Recieved

5
4
Number

3
2
1
0
0 2 4 6 8 10
Time

This graph shows a particular outcome of the ‘experiment’ which determines the observed
values of the stochastic process for the element ω . The sample path above indicates that
the first claim is received at time 2 since the function is 0 for t < 2 and equal to 1 for t = 2
; the second claim is received at time 3 since the function is 1 for 2 ≤ t < 3 and is equal to
2 for t = 3 ; the third claim at time 6, the fourth at time 8, etc. If we repeat the experiment
 -5-

under the same conditions we will get a second value of ω , say ω 2 , and then get the
observed values of the stochastic process for all values of t i.e. we get another possible
sample path. Each repetition of the experiment determines an element ω of Ω and
therefore determines a sample path.
♪

Result 2.2.1 Given the probability space (Ω, ξ , P ) , we can determine the joint distribution
of any finite number of the X t ’s. Since each X t is a random variable the set
{ω : X t (ω ) ≤ x} is an element of ξ . Since ξ is a σ -field the intersection of any finite
number of such sets is an element of ξ and hence P is defined for such a set. For any value
n and any values t1 < t 2 < t 3 < ... < t n the joint distribution function of X t1 , X t2 , X t3 ,..., X tn
is given by
FX t
1
, X t 2 ,..., X t n (x1 , x2 ,..., x n ) [
= P X t1 ≤ x1 , X t2 ≤ x 2 ,..., X tn ≤ x n ]
= P[{ω : X t1 }]
(ω ) ≤ x1 , X t2 (ω ) ≤ x 2 ,..., X tn (ω ) ≤ x n .
Thus, if given the joint cumulative distribution function of X t1 , X t2 , X t3 ,..., X tn , the joint
density function or probability mass function of X t1 , X t2 , X t3 ,..., X tn can be determined.

Conversely, if we have a set of random variables {X t : t ∈ T } and know all the joint
distribution functions FX t , X t , X t ,..., X t for all values of n and t1 < t 2 < t 3 < ... < t n we can
1 2 3 n

find a probability space (Ω' , ξ ' , P ') and define functions {X t' : t ∈ T } on this space such that
the joint distribution of X t'1 , X t'2 , X t'3 ,..., X t'n is the same as the joint distribution of
X t1 , X t2 , X t3 ,..., X tn . The probabilities of a stochastic process can therefore be defined by
the joint distributions of all X t1 , X t2 , X t3 ,..., X tn .

Note: To determine, or even just to list, all the joint distributions would be very
cumbersome and hard to use in some practical problems unless there is some structure in
the nature of the process which simplifies the specification of all the joint distributions. In
many cases a set of random variables with a very simple structure for all the joint
distributions is defined and then a set of new random variables i.e. a stochastic process is
defined using the random variables with the simple structure.

Example 2.2.2

Consider a gambler that makes successive bets of R1 each. The outcomes of the different
bets are independent events. Let X n be the amount of money the gambler wins (or looses)
at the n th bet. Assume that the probability of winning a bet is p for all bets. Then the
probability mass function of X n is given by
 -6-

 p for x = 1

f X n ( x) = P[ X n = x] = 1 − p for x = −1
0 otherwise.

Since the X n ’s are independent random variables, the joint probability mass function of
X 1 , X 2 ,..., X n is given by

f X 1 , X 2 ,..., X n ( x1 , x 2 ,..., x n ) = f X 1 ( x1 ) f X 2 ( x 2 )... f X n ( x n ) for all x1 , x 2 ,..., x n .

Therefore {X n : n = 1,2,3,...} is a well-defined stochastic process. This is a stochastic

process called white noise i.e. it consists of independent and identically distributed random
variables.

Let Yn , n = 1,2,3,... be the total winnings (or losses) of the gambler up to and including the
n th bet i.e. Yn = X 1 + X 2 + ... + X n . Each Yn is therefore a linear transformation of the X n
’s and the joint distribution of each Yn can therefore be obtained from the joint distribution
of the X n ’s. Thus, we can use a stochastic process { X n : n = 1,2,3,...} for which the joint
distributions are easily determined to define the stochastic process { Yn : n = 1,2,3,...} .

2.3 Some General Properties of Stochastic Processes

2.3.1 Stationary A stochastic process {X t : t ∈ T } is a stationary process if for any

integer n , any values t1 < t 2 < ... < t n which are all elements of T and any value k such
that k + t1 , k + t 2 ,..., k + t n are all elements of T , it is true that the joint distribution of
X t1 , X t2 ,..., X tn is the same as the joint distribution of X t1 + k , X t2 + k ,..., X tn + k .

For n = 1 this means X t1 has the same distribution as X t1 + k for any k i.e. all the X t ’s
have the same distribution. For n = 2 it means that for any t1 and t 2 the joint distribution
of X t1 , X t2 is the same as the joint distribution of X t1 + k , X t2 + k . Therefore, a shift in time of
both t1 and t 2 by the same quantity k does not change the joint distribution of the random
variables. Put differently, if the distance between t1 and t 2 is the same as the distance
between t 3 and t 4 the joint distribution of X t1 , X t2 is the same as the joint distribution of
X t3 , X t4 . As a result, we have, for example, that cov( X t1 , X t2 ) = cov( X t3 , X t4 ) . Similarly
for the cases n = 3,4,... etc.

2.3.2 Covariance Stationary Process A stochastic process {X t : t ∈ T } is

called covariance stationary if cov( X s , X t ) depends only on t − s - in other words,
depends only on how far apart the two random variables are in time.
 -7-

A stationary process is covariance stationary, but the converse is not necessarily true.

2.3.3 Weakly Stationary Process A stochastic process {X t : t ∈ T } is weakly

stationary if all the X t ’s have the same expected value and if the process is covariance
stationary.

2.3.4 Independent Increments A stochastic process {X t : t ∈ T } is said to have

independent increments if the increment in the value of the process over any interval t to
t + s (i.e. X t + s − X t ) is independent of what happened before or at time t i.e. the increment
is independent of {X u : 0 ≤ u ≤ t}. This means that the process is such that the value of the
change in the process from time t to time t + s is independent of the state of the process
before time t and at time t .

2.3.5 The Markov Property A stochastic process {X t : t ∈ T } is said to have the

Markov Property if for any set A ⊆ S , for any value of n and for any values of
t1 < t 2 < ... < t n < t n +1 , it is true that

[ ] [
P X tn +1 ∈ A | X t1 = xt1 , X t2 = xt2 ,..., X tn = xtn = P X tn +1 ∈ A | X tn = xtn ]
i.e. probabilities of future events (what happens at time t n +1 ) only depend on the state of
the process at the latest point in time ( t n ) for which we have information. The information
about the state of the process at the previous points in time ( t 0 , t1 ,..., t n −1 ) does not have any
effect on the probabilities of future events and will therefore not help to make a better
prediction of what will happen at time t n +1 .
For a stochastic process with a discrete state space, the process will have the Markov
property if for any a ∈ S , for any value of n and for any values of t1 < t 2 < ... < t n < t n +1 ,
it is true that

[ ] [
P X tn +1 = a | X t1 = xt1 , X t2 = xt2 ,..., X tn = xtn = P X tn +1 = a | X tn = xtn . ]

Theorem 2.3.1 A stochastic process {X t : t ∈ T } with independent increments

has the Markov property.

1 (ST)
Proof For any a ∈ S , for any value of n and for any values of
t1 < t 2 < ... < t n < t n +1 , we have that
 -8-

[
P X tn +1 = a | X t1 = xt1 , X t2 = xt2 ,..., X tn = xtn ]
[
= P X tn +1 − X tn + xtn = a | X t1 = xt1 , X t2 = xt2 ,..., X tn = xtn ]
= P[X t n +1 − X t n + xt n = a ] (independent increments)
= P[X t n +1 − X t n + xt n = a | X t n = xt n ] (independent increments)
= P[X t n +1 = a | X t n = xt n ]
i.e. the Markov property holds.
■

NB: The converse is not necessarily true i.e. a Markov process does not necessarily have
independent increments. (Exercise: Find an example to show that this is true.)

A Markov process with discrete parameter space T = {0,1,2,...} and discrete state space
(finite or countably finite) S is called a Markov chain.
A Markov process with continuous parameter space T = [0, ∞ ) and discrete state space
(finite or countably finite) S is called a Markov jump process.

2.3.6 Martingale A discrete-time stochastic process {X t : t = 0,1,2,...} is a

martingale if
(i) E [ X t ] < ∞ for all t
(ii) E [X n X 1 , X 2 , X 3 ,..., X m ] = X m for all m < n.
The second point implies that if we know the values of the process up to time m , the best
(using the MSE criterion) guess or prediction of any future value of the process is the latest
value of the process that we know.

2.4 Some Specific Stochastic Processes

2.4.1 White Noise A stochastic process {X t : t = 0,1,2,...} is called white noise if

X 0 , X 1 , X 2 ,... is an independent sequence of random variables, all of them identically
distributed.
White noise is often used as an intermediary process to define other stochastic processes.
White noise is a stationary process and possesses the Markov property. (Prove this
statement as an exercise.) It does not have independent increments.

2.4.2 Random Walk Let {X t : t = 0,1,2,...} be a white noise process and define a
t
sequence of random variables Y0 , Y1 , Y2 ,... where Y0 = 0 and Yt = ∑ X i for all t ≥ 1. Then
i =1

{Yt : t = 0,1,2,...} is called a random walk process.

 -9-

A random walk is a process with independent increments and therefore has the Markov
property (prove this as an exercise). It is, however, not necessarily stationary (prove this
for the case where E [X t ] = µ ≠ 0 ).
A special case is when X t is either +1 or -1 with probabilities p and 1 − p respectively.
This may represent a person (maybe drunk?) who takes steps independent of each other
either to the right with probability p or steps to the left with probability 1 − p . Then Yt is
the position of the person after t steps. This is called a simple random walk.

2.4.3 Moving Average Process Suppose that X − p , X − p +1 ,..., X 0 , X 1 , X 2 ,... is a

sequence of independent identically distributed random variables. Let α 0 , α 1 ,..., α p be
p
p + 1 real numbers. Let Yt = ∑ α i X t −i for t = 0,1,2,... .Then Yt is a weighted average of
i =0

the last p + 1 X ’s up to and including X t . (The weight for the latest one ( X t ) is α 0 , the
weight for the previous one ( X t −1 ) is α 1 etc.) Then {Yt : t = 0,1,2,...} is a moving average
process of order p . A moving average process is a stationary process but in general is not
a Markov process.

2.4.4 Poisson Process A Poisson process with parameter λ > 0 is a continuous

time process {N t : t ∈ [0, ∞ )} and state space S = {0,1,2,...} with the following properties:
(i) N 0 = 0
(ii) {N t } has independent increments
(iii) {N t } has stationary increments which have a Poisson distribution with parameter
proportional to λ > 0 i.e.
e − λ (t − s ) (λ (t − s ) )
n
P[N t − N s = n] = for n = 0,1,2,...
n!
for all s < t .

The Poisson process is often used as a model where we count the number of events that
take place during some interval of time – for instance the number of accidents or the
number of insurance claims. The Poisson process is not stationary (why?).

2.4.5 Compound Poisson Process Suppose that {N t : t ∈ [0, ∞ )} is a Poisson

process with parameter λ > 0 and that X 1 , X 2 , X 3 ,... is a sequence of independent
identically distributed random variables. Let

0 if N t = 0

Yt =  N t
∑ X i if N t > 0.
 i =1
 - 10 -

Then {𝑌𝑌𝑡𝑡 : 𝑡𝑡 ∈ [0, ∞)} is called a compound Poisson process. Such a process can be used as
a model where N t is the number of insurance claims up to time t , for example, and X i is
the amount of the i th claim. Then Yt is the total amount of claims received up to time t .

2.4.6 Brownian Motion A stochastic process {Bt : t ∈ [0, ∞ )} with state space
S = R is called Brownian motion if:
(i) {Bt } has independent increments.
(ii) Each increment Bt − Bs , s < t , has a normal distribution with expected value µ (t − s )
and variance σ 2 (t − s ) , where µ and σ 2 > 0 are constants.
(iii) {Bt } has continuous sample paths.
Brownian motion is also referred to as a Wiener process.
The case when µ = 0 and σ 2 = 1 is referred to as standard Brownian motion.

2.5 Important Theoretical Results

Theorem 2.5.1 Suppose that N is a non-negative integer valued random variable,

Z 1 , Z 2 , Z 3 ,... are independent identically distributed random variables and that all Z i ’s are
independent of N . Let
N
∑ Z i if N > 0
T =  i =1 .
0 if N = 0

Then E[T ] = E[ N ].E[ Z ] and var(T ) = E[ N ]. var(Z ) + var( N ).E[ Z ] 2 .
Proof We know that for any random variables T and N
E[T ] = E N [ E[T | N ]] (Bain and Engelhardt).
So
E[T ] = E N [ E[T | N ]]
∞ N
= P[ N = 0]E[T | N = 0] + ∑ P[ N = n]E[∑ Z i | N = n]
n =1 i =1
∞ n
= P[ N = 0].0 + ∑ P[ N = n]E[∑ Z i ] (since all Z i are independent of N )
n =1 i =1
∞
= ∑ P[ N = n].n E[ Z ] (since n is a given constant)
n =0
∞
= E[ Z ]∑ P[ N = n].n
n =0

= E[ Z ]E[ N ].

We know that for any random variables T and N

var(T ) = E N [var(T | N )] + varN ( E[T | N ]). (Bain and Engelhardt).
 - 11 -

So
var(T ) = E N [var(T | N )] + varN ( E[T | N ])
∞
 N   N 
= ∑ P[ N = n] var ∑ Z i N = n  + varN  E[∑ Z i N ] 
n =0  i =1   i =1 
∞
 n
  N

= ∑ P[ N = n] var ∑ Z i  + varN  E[∑ Z i N ] .
n =0  i =1   i =1 
But
N n
E[∑ Z i N = n] = E[∑ Z i ] = n E[ Z ] (true for n > 0 and n = 0 ),
i =1 i =1
so
∞
var(T ) = ∑ P[ N = n] n var(Z ) + varN (N E[ Z ])
n =0
∞
= var(Z )∑ P[ N = n] n + E[ Z ] 2 var( N )
n =0

= var(Z ) E[ N ] + E[ Z ] 2 var( N ).

Definition 2.5.1 Let U be a random variable which can take on the values 0,1,2,3,...
and let pi = P[U = i ] . The probability generating function of U is defined by
∞
u ( z ) = p 0 + p1 z + p 2 z 2 + p3 z 3 + ... = ∑ pi z i .
i =0
This sum will converge for | z |< 1 since for | z |< 1
∞ ∞

∑ | pi z i | < ∑ pi = 1 ,
i =0 i =0
since an absolutely convergent series is convergent.

Note that if we know the function u (z ) and expand it as a power series the coefficient of
z i is equal to pi i.e. knowing u (z ) is equivalent to knowing all the probabilities.

Theorem 2.5.2 If u (z ) is the probability generating function of U then u ' (1) = E[U ] .
Proof
u ' ( z ) = p1 + 2 p 2 z + 3 p3 z 2 + 4 p 4 z 3 + ...
i.e.
 - 12 -

u ' (1) = p1 + 2 p 2 + 3 p3 + 4 p 4 + ...

= 0 p 0 + 1 p1 + 2 p 2 + 3 p3 + 4 p 4 + ...
= E[U ] (by the definition of the expected value of a discrete random variable)
■

Theorem 2.5.3 If u (z ) is the probability generating function of U then

u ' ' (1) = E[U (U − 1)] .
Proof
u ' ' ( z ) = 2 p 2 + 3 × 2 p3 z + 4 × 3 p 4 z 2 + ...
u ' ' (1) = 2 p 2 + 3 × 2 p3 + 4 × 3 p 4 + ...
i.e. = 0 × (−1) p 0 + 1 × 0 p1 + 2 × 1 p 2 + 3 × 2 p3 + 4 × 3 p 4 + ...
= E[U (U − 1)] (definition of discrete exp ected value)
■

From the above it follows that var(U ) = u ' ' (1) + u ' (1) − (u ' (1) ) .
2

NOTE: u ( z ) = E[ z U ] = E[e (ln z )U ] = M U (ln z ) where M U is the moment generating

function of U .

Theorem 2.5.4 Let a0 , a1 , a 2 ,... and b0 , b1 , b2 ,... be two non-negative sequences of

numbers. Then
 ∞  ∞  ∞  k  ∞ k 
∑ i  ∑ j  ∑ ∑ k −l l  = ∑ ∑ al bk −l  .
a b = a b
 i =0   j =0  k =0  l =0  k =0  l =0 

3 (ST) Proof
 ∞  ∞ 
∑ ai  ∑ b j  = [a 0 + a1 + a 2 + ...][b0 + b1 + b2 + ...]
 i =0   j =0 
= a 0 b0 + a 0 b1 + a 0 b2 + a 0 b3 + a 0 b4 + ...
+ a1b0 + a1b1 + a1b2 + a1b3 + a1b4 + ...
+ a 2 b0 + a 2 b1 + a 2 b2 + a 2 b3 + a 2 b4 + ...

 - 13 -


= a 0 b0
+ a1b0 + a 0 b1
+ a 2 b0 + a1b1 + a 0 b2
+ a3 b0 + a 2 b1 + a1b2 + a 0 b3

i.e. to get the sum of all the products ai b j first get the sum of the products on the various
diagonals and then get the sum of all the sums on the diagonals. Note that on the k th
diagonal the sum of the subscripts are equal to k . Therefore

 ∞  ∞  ∞  k  ∞ k 
 ∑ a i  ∑ b j  = ∑  ∑
 i =0   j =0  k =0  l =0
a k −l l  = ∑ ∑ a l bk −l  .
b
 k =0  l =0 
■
Note: This video will be interesting to watch:
[Link]
bigger-than-others

Theorem 2.5.5 Let U be a non-negative integer valued random variable with

∞
P[U = i ] = u i for i = 0,1,2,3,... and let u ( z ) = ∑ u i z i i.e. u (z ) is the generating function
i =0
of the probabilities of U . Let V be an independent non-negative integer valued random
∞
variable with P[V = j ] = v j , j = 0,1,2,3,... and let v( z ) = ∑ v j z j i.e. v(z ) is the
j =0

generating function of the probabilities of V . Then Let W = U + V and let w(z ) be the
probability generating function of W . Then w( z ) = u ( z )v( z ) .

Proof We have that for any integer k ≥ 0 that

wk = P[U + V = k ]
= P[U = 0 and V = k or U = 1 and V = k − 1 or U = 2 and V = k − 2 ... or U = k and V = 0 ]
= P[U = 0 and V = k ] + P[ U = 1 and V = k − 1] + ... + P[ U = k and V = 0 ]
(since the events are disjoint)
= P[U = 0]P[V = k ] + P[U = 1]P[V = k − 1] + ... + P[U = k ]P[V = 0]
(since U and V are independent)
k
= u 0 v k + u1v k −1 + ... + u k v0 = ∑ u i v k −i .
i =0
This is called the convolution of {u} and {v} .
Hence
 - 14 -

∞
w( z ) = ∑ wk z k
k =0
∞ k
= ∑∑ u i v k −i z k
k =0 i =0
∞ k
= ∑∑ u i z i v k −i z k −i
k =0 i =0
∞ ∞
= ∑ ui z i ∑ v j z j
i =0 j =0

= u ( z )v( z ).
■

NOTE: If U 1 and U 2 are non-negative integer valued, identically distributed and

independent random variables, i.e. both have the same probability generating function, say
u (z ) , then the probability generating function for U 1 + U 2 is given by [u (z )] .
2

NOTE: If U 1 , U 2 , U 3 ,..., U n are non-negative integer valued, identically and independent

random variables all with probability generating function u (z ) , then the probability
generating function of U 1 + U 2 + U 3 + ... + U n is given by [u ( z )] .
n

Theorem 2.5.6 Let N be a non-negative integer valued random variable with

probabilities p n = P[ N = n] and with probability generating function Φ N (z ) . Let
X 1 , X 2 , X 3 ,... be a sequence of independent identically distributed random variables with
moment generating function m X (u ) . Suppose that N is independent of all the X i ′s. Let
N
∑ X i if N > 0
T =  i =1
0 if N = 0

and let M T (u ) be the moment generating function of T . Then
M T (u ) = Φ N (m X (u ) ) .
Proof
M T (u ) = E[e uT ]
 u∑ Xi 
N

= E e i =1 
 
 
 - 15 -

  u∑ 
N
Xi

= EN E e  i =1
N 
  
  
 u∑ Xi   u∑ Xi 
N N
∞
= E e i =1 N = 0 p 0 + ∑ E e i =1 N = n  p n
  n =1  
   
 u∑ Xi 
n

[ ]
∞
= E e p 0 + ∑ E e i =1  p n
u0

n =1  
 
(since all X i ' s are independent of N )
∞
= 1 p 0 + ∑ [m X (u )] p n
n

n =1

(since the X i ' s are independent )

= Φ N (m X (u ) ). (since [Φ N (u )] = 1)
0

Theorem 2.5.7 For a compound Poisson process with parameter λ the moment
 Nt
generating function of Yt = ∑
 X i if N t > 0
i =1 is given by e λt ( M X (u ) −1)
0 if N t = 0

4 (ST) Proof The probability generating function of a Poisson random variable with
parameter λt is given by
∞
−𝜆𝜆𝜆𝜆
(𝜆𝜆𝜆𝜆)𝑛𝑛 𝑛𝑛
𝛷𝛷𝑁𝑁𝑡𝑡 (𝑢𝑢) = � 𝑒𝑒 𝑢𝑢 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑁𝑁𝑡𝑡 − 𝑁𝑁0 = 𝑁𝑁𝑡𝑡 ~𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃(𝜆𝜆𝜆𝜆)
𝑛𝑛!
𝑛𝑛=0
∞
(𝜆𝜆𝜆𝜆𝜆𝜆)𝑛𝑛
= 𝑒𝑒 −𝜆𝜆𝜆𝜆 �
𝑛𝑛!
𝑛𝑛=0
∞
−𝜆𝜆𝜆𝜆 𝜆𝜆𝜆𝜆𝜆𝜆 𝜆𝜆𝜆𝜆(𝑢𝑢−1) 𝑥𝑥
𝑥𝑥 𝑛𝑛
= 𝑒𝑒 𝑒𝑒 = 𝑒𝑒 (𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 𝑜𝑜𝑜𝑜 𝑒𝑒 = � )
𝑛𝑛!
𝑛𝑛=0
and therefore, by theorem 2.5.6 M Yt (u ) = e λt ( M X (u ) −1) . ■

Test Yourself:
Exercise 2.1
(a) If you were to model the maximum daily temperature in Pretoria, what state space and
parameter space would you use? Describe the nature of these spaces.
(b) Suppose you have available a continuous record of temperature instead and wanted to
model Pretoria’s temperature at all times. Describe the parameter space, state space and the
nature of these spaces.
Exercise 2.2
 - 16 -

Consider independent identically distributed random variables X t for t = 1,2,3,... where

X t = 1 with probability p and X t = −1 with probability q = 1 − p . Let Y0 = 0 and
t
Yt = ∑ X i i.e. {Yt : t = 0,1,2,...} is a random walk.
i =1

(a) Explain why the Yt ’s are not identically distributed (HINT: Determine the distributions
of Y1 and Y2 ).
(b) Explain why the Yt ’s are not independent (HINT: determine the conditional
distribution of Y2 given that Y1 = 1 ).
Exercise 2.3
For the process {Yt : t = 0,1,2,...} defined in exercise 2.2, determine
(a) P[Y2 = 1, Y4 = 3 Y0 = 0]
(b) P[Y2 = 0, Y4 = 2 Y0 = 0].
Exercise 2.4
To specify the probability structure of a stochastic process {X t : t ∈ T } it is required to
specify the distribution of X t for all values of t ∈ T - TRUE or FALSE?
Exercise 2.5
For the process {Yt : t = 0,1,2,...} defined in exercise 2.2
(a) Determine P[Y10 = 10] and P[Y2 = 10] .
(b) Is the random walk stationary?
Exercise 2.6
What can be concluded about the variance of a weakly stationary stochastic process
{X t : t ∈ T }?
Exercise 2.7
Prove that a random walk has the Markov property.
Exercise 2.8
Suppose that {Yt : t = 0,1,2,...} is a moving average process of order p .
(a) Prove that such a process is weakly stationary.
(b) Does the process have the Markov property? Explain.
(c) Does the process have independent increments? Explain.