EC451: Introductory Macroeconomics for MSc EME Fall 2024

Dynamic Methods
Dmitry Mukhin
[Link]@[Link]

1 Sequential Approach
Savings problem Consider a simple savings problem of a household living for T periods:

T
X
max β t u(Ct )
{Ct ,Bt+1 }
t=0

s.t. Bt+1 = RBt + Yt − Ct (1)

BT +1 ≥ 0, (2)

where Ct and Bt+1 are consumption and savings in period t, R ≡ 1 + r denotes the gross
returns on assets, and Yt is an exogenous income. Equation (1) is the flow budget constraint of
the household, while equation (2) restricts the agent to pay back any debt before her death.
We make the standard assumptions about preferences:

1. discount factor β ∈ [0, 1),

2. utility function u(·) is increasing and concave,

3. lim u0 (C) = ∞.
C→0

Optimality conditions Perhaps the simplest way to approach this problem is to apply the
same optimization methods that are used to solve the static problem. The dynamic prob-
lem is indeed not that different from a static one given that consumption in different periods
C0 , C1 , . . . , CT can be interpreted as consumption of different goods. To this end, write down
the Lagrangian

T
X T
X  
t
L= β u(Ct ) + λt RBt + Yt − Ct − Bt+1 + µBT +1 ,
t=0 t=0

1
where λt , µ ≥ 0, and take the first-order conditions (FOC):

wrt Ct : β t u0 (Ct ) = λt ,
wrt Bt+1 : λt = Rλt+1 , t < T,
wrt BT +1 : λT − µ = 0.

Combining the first two equations for t < T , we get the intertemporal optimality conditions,
which are often called the Euler equations (EE):

u0 (Ct ) = βR u0 (Ct+1 ), (3)

for ∀t < T . The last period requires special treatment because of the complementary slackness
condition
µBT +1 = 0,

i.e. either (i) constraint (2) binds and BT +1 = 0 or (ii) constraint (2) does not bind and µ = 0.
Combining this expression with the first-order conditions, we get the transversality condi-
tion (TVC)
β T u0 (CT )BT +1 = 0, (4)

which implies that the optimal bequest is BT +1 = 0. In sum, the optimal path {Ct , Bt+1 } with
2 × (T + 1) unknowns is determined by T + 1 budget constraints (1), T Euler equations (3)
and one transversality condition (4).

Example To solve the model in closed form, assume, for example, a constant relative risk
1−σ
aversion (CRRA) utility u(C) = C1−σ and follow these steps:
1. Iterate forward the budget constraint

B0 = R−1 C0 − Y0 + B1 = R−1 C0 − Y0 + R−2 C1 − Y1 + B2 = . . .

to obtain
T
X
R−t Ct − Yt + R−T BT +1 .


RB0 =
t=0

2. Impose the terminal condition BT +1 = 0 to obtain the intertemporal budget constraint:

T
X T
X
−t
R Ct = RB0 + R−t Yt .
t=0 t=0

3. Express the Euler equation as

1
Ct+1 = (βR) σ Ct

2
 and substitute into the budget constraint to solve for
PT −t
RB0 + t=0 R Yt
C0 = PT t .
t=0 R−t (βR) σ

Infinite horizon So far, we assumed a finite T . However, it is often more convenient to

focus on infinite horizon. Although the formal proofs are more complicated for infinite hori-
zon, most results can be derived heuristically by simply taking the limit T → ∞ in the ex-
pressions above. In particular, the household problem remains almost unchanged, except for
constraint (2):
X∞
max β t u(Ct )
{Ct ,Bt+1 }
t=0

s.t. Bt+1 = RBt + Yt − Ct (1’)

Bt ≥ −B̄, (2’)

where B̄ is a sufficiently high borrowing limit. The latter constraint is called the no-Ponzi
game condition (NPGC) and does not allow the agent to choose an infinitely high consump-
tion by borrowing more and more each period. Often, this restriction does not appear in the
optimization problem and is only implicitly assumed. Write down the Lagrangian assuming
that constraint (2’) does not bind
∞
X ∞
X  
t
L= β u(Ct ) + λt RBt + Yt − Ct − Bt+1 .
t=0 t=0

Taking the optimality conditions, it is straightforward to show that the Euler equations (3)
remain unchanged and hold for any period t. Remarkably, the transversality condition is also
similar to the one above:
lim β T u0 (CT )BT +1 = 0. (4’)
T →∞

Following the same steps as above, derive the intertemporal budget constraint
∞
X ∞
X
−j
R Ct+j = RBt + R−j Yt+j
j=0 j=0

and solve for the optimal consumption under the CRRA utility:
∞
1 1
−1

 X
−j

Ct = 1 − β R σ σ RBt + R Yt+j .
j=0

Shooting algorithm What numerical methods can be used to solve models that have no
closed-form solution? The simplest way to do this is to use the “shooting algorithm”. The

3
main advantage of this method is that it can solve even very complicated non-linear dynamic
models.1 The main limitation is that it only applies to deterministic models and cannot be
extended to solve stochastic problems.
The basic idea of the shooting algorithm can be shown in a context of the consumption-
saving problem from above. Suppose we knew the values of B0 and C0 . Then one can use
the Euler equation (3) to solve for C1 and the budget constraint (1) to solve for B1 . Given
B1 , C1 , continue solving for B2 , C2 and so on. Of course, the problem is that we do not know
the initial value of consumption. This is where we need to use the remaining NPGC (2’) and
TVC (4’). If C0 is too high, then the household debt −Bt will go to infinity violating the NPGC.
If on the other hand, C0 is too low, then the TVC will be violated. This suggests the following
algorithm:

1. Using the budget constraint, compute the maximum and minimum possible values of
consumption, Cmin and Cmax .

2. Start with some initial guess for C0 ∈ [Cmin , Cmax ].

3. Simulate {Bt , Ct } iteratively for a large number of periods T using

1
Ct+1 = (βR) σ Ct ,

Bt+1 = RBt + Yt − Ct .

4. If the NPGC is violated, then adjust Cmax downwards to C0 , if the TVC is violated then
adjust Cmin upwards to C0 .

5. Go back to step 2 and iterate until Cmax − Cmax < ε for a given tolerance ε.

Uncertainty Consider again the baseline saving model, but assume that the income process
Yt is stochastic and depends on a realization of the state of the world zt . Denote the history that
summarizes the realization of states up until period t with z t = {z1 , z2 , . . . zt } and let π(z t )
be the probability of such history from the ex-ante (period 0) perspective. All macroeconomic
variables are function of histories and the household problem can be written as
∞ X
X
max β t π(z t )u(Ct (z t ))
{Ct (z t ),Bt+1 (z t )}
t=0 zt

s.t. Bt+1 (z t ) = RBt (z t−1 ) + Yt (z t ) − Ct (z t )

1
This includes frontier models with heterogenous agents – for the recent treatment see Auclert, Bardoczy,
Ronglie and Straub (2021).

4
Note that Bt+1 is a function of z t , not z t+1 . This because savings are chosen in period t given
the information known by that moment. Also notice that zt π(z t ) = 1 for any t and the
P

second summation can be replaced with the expectation operator E[·]. It follows that the
Lagrangian can be written as

∞ X
" #
X  
t t t t t−1 t t t
L= π(z ) β u(Ct (z )) + λt (z ) RBt (z ) + Yt (z ) − Ct (z ) − Bt+1 (z ) ,
t=0 zt

and the optimality conditions are

β t u0 (Ct (z t )) = λt (z t ),
X
π(z t )λt (z t ) = R π(z t+1 )λt+1 (z t+1 ).
z t+1 |z t

Combine the two conditions and use the definition of conditional probability:
X
u0 (Ct (z t )) = βR π(z t+1 |z t )u0 (Ct+1 (z t+1 )).
z t+1 |z t

To economize on the notation, this Euler equation is often written without an explicit depen-
dence of the variables on histories:

u0 (Ct ) = βR Et u0 (Ct+1 ).

Despite only minor differences from the deterministic case, this system cannot be solved an-
alytically. Even under the CRRA utility, it is not possible to express Ct+1 from this condition

Ct−σ = βR Et Ct+1
−σ
.

and to substitute into the budget constraint. Neither a finite horizon nor a continuous-time
limit help to solve this problem. Instead, macroeconomists usually employ one of the following
approaches (or their combination):

1. recursive formulation of the problem and dynamic programming,

2. approximate solution based on the linearized equilibrium conditions.

The next two lectures discuss in detail these two methods.

Continuous time The basic idea is to rewrite all equations assuming an arbitrary length
of period ∆ (instead of normalizing it to one) and then to take the limit of ∆ → 0. To do
this correctly, recall that macroeconomic variables can be divided into (i) stocks measured at a

5
given point in time (e.g. capital, debt, prices) and (ii) flows measured over a period of time (e.g.
inflation, interest, consumption, GDP). It follows that the former variables are independent of
∆, while the latter ones are proportional to the length of period. Applying this principle to
the household budget constraint, we get

Bt+∆ = (1 + r∆)Bt + Yt ∆ − Ct ∆,

where r = R − 1. Rewrite the equation in terms of growth rates

Bt+∆ − Bt
= rBt + Yt − Ct
∆

and take the continuous-time limit ∆ → 0:

Ḃt = rBt + Yt − Ct .

The left hand side of the equation Ḃt ≡ lim Bt+∆∆−Bt corresponds the instantaneous change
∆→0
in wealth Bt in period t. To express the objective function in continuous time, it is convenient
to define the discount factor ρ > 0 such that β ≡ e−ρ∆ . The discounted utility is then equal to
∞
X Z ∞
−ρt
e u(Ct )∆ −→ e−ρt u(Ct )dt.
t=0 0

Hamiltonian Consider next a generalized optimal control problem:

Z ∞
max e−ρt u(Ct )dt
{Ct ,Bt } 0

s.t. Ḃt = g(Bt , Ct ),

where Ct is a flow variable and Bt is a stock variable. Define the Hamiltonian as

Ht ≡ e−ρt u(Ct ) + λt g(Bt , Ct ),

where λt is called a co-state variable. The Lagrangian of the discrete-time version of the
problem can then be written as
∞ h
X
i
L= Ht ∆ − λt Bt+∆ − Bt .
t=0

6
Taking the first-order conditions and the limit ∆ → 0, we get

∂Ht ∂Ht
∆=0 ⇒ =0
∂Ct ∂Ct
∂Ht+∆ ∂Ht
∆ − λt + λt+1 = 0 ⇒ = −λ̇t
∂Bt+∆ ∂Bt

The intuition for these conditions is virtually the same as in the discrete time. As before, the
transversality condition states
lim λt Bt = 0.
t→∞

The maximum principle due to Pontryagin says that these three equations are the necessary
conditions for optimality. In contrast to the Lagrangian in discrete time, the method only
works for deterministic problems and we will follow another approach to solve stochastic
models in continuous time.

Example Consider again the savings problem with CRRA preferences and assume constant
income Yt = Y . The household problem in continuous time is
∞
Ct1−σ
Z
−ρt
max e dt
{Ct ,Bt }∞
t=0 0 1−σ

s.t. Ḃt = rBt + Y − Ct .

The Hamiltonian is given by

Ct1−σ
Ht ≡ e−ρt


+ λt rBt + Y − Ct
1−σ

and therefore, the optimality conditions are

∂Ht
0= = e−ρt Ct−σ − λt
∂Ct
∂Ht
−λ̇t = = rλt
∂Bt

Combine these equation to get the Euler equation:

Ċt 1
= (r − ρ).
Ct σ

Solve this differential equation:

r−ρ
t
Ct = C0 e σ ,

7
where C0 is the (unknown) level of consumption in the initial period. This constant is deter-
mined by the intertemporal budget constraint. To obtain the latter, multiply both sides of the
(flow) budget constraint by e−rt and express it as

e−rt Ḃt − re−rt Bt = (Y − Ct )e−rt ,

or equivalently as
d(e−rt Bt ) = (Y − Ct )e−rt dt.

Integrate both sides from the initial period to period t:

Z t
−rt
e Bt = B0 + (Y − Cs )e−rs ds.
0

Next, use the transversality condition

lim λt Bt = 0,
t→∞

which combined with the expression for λt and Ct implies e−rt Bt → 0 as t → ∞. Applying
this limit to the integrated budget constraint, obtain
Z ∞
B0 + (Y − Cs )e−rs ds = 0.
0

Finally, substitute the process for consumption

Z ∞
r−ρ
B0 + (Y − C0 e σ
s
)e−rs ds = 0
0

and assuming r is not too high, solve

ρ − r(1 − σ)  1 
C0 = B0 + Y .
σ r

Notice that the same expression can be obtained from the solution of the discrete-time model
taking the limit ∆ → 0.

8
2 Dynamic Systems
Dynamic linear equations Consider a first-order (one lag) deterministic difference equa-
tion:
xt+1 = λxt ,

where parameter λ ∈ R and initial value x0 ∈ R are some constants. Solving this equation
means finding the values of xt in every period t. This can be easily done by iterating the
equation backward:
xt = λxt−1 = λ2 xt−2 = · · · = λt x0 .

This simple example shows that dynamics of xt depends crucially on the value of λ:

1. Convergence:

(a) if |λ| < 1, the system converges to zero xt → 0,

(b) if |λ| > 1, the system diverges from zero xt → ∞.

2. Monotonicity:

(a) if λ > 0, the dynamics is monotonic,

(b) if λ < 0, the dynamics is oscillating.

The steady state (SS) of the system is a point x̄ such that if we start from it, we stay there
forever, i.e. if xt = x̄, then xt+j = x̄ for ∀j > 0. It follows that the steady state can be found
by solving equation x̄ = λx̄ and depends again on the value of λ.

3. Steady state:

(a) if λ 6= 1, the system has a unique steady state x̄ = 0,

(b) if λ = 1, the system has a continuum of steady states x̄ ∈ R.

The steady state is stable if xt → x̄ as t → ∞. Thus, steady state x̄ = 0 is stable iff |λ| < 1.
How can on solve a higher-order linear equation? For example, consider a second-order
difference equation
xt+1 = λ1 xt + λ2 xt−1 .

It turns out, one can reduce the order of the system by increasing its dimensionality:
! !
xt λ1 λ2
zt+1 = Azt , where zt ≡ and A ≡ .
xt−1 1 0

Thus, solving a high-order equation is isomorphic to solving a system of first-order equations

— the problem we discuss next.

9
Linear systems Suppose next that there are two variables (e.g. consumption and savings)
and two linear dynamics equations:
!
x1t
xt+1 = Axt , where xt ≡
x2t

and A is a 2 × 2 matrix. Again, the steady state of the system can be found from x̄ = Ax̄. It
follows that there is a unique steady state equal x̄ = 0 if |A − I| =
6 0 and continuum of steady
states otherwise. Given initial values x0 , one can iterate the system backward and obtain

xt = A t x0 .

Thus, solving the dynamic system boils down to finding At . The latter can be easily calculated
when A is a diagonal matrix
!t !
a11 0 at11 0
=
0 a22 0 at22

and the system is uncoupled, i.e. the dynamics of each variable can be analyzed separately
from the other one. This is, however, rarely the case in macro models, and finding At is more
complicated in general case. To make progress, we use the results from linear algebra that
effectively transform an arbitrary A into an (almost) diagonal matrix.
We focus here on the simplest case when the eigenvalues of matrix A are real λ1 , λ2 ∈ R
and different λ1 6= λ2 .2 In this case, there exist a diagonal matrix Λ and a matrix Q such that
!
−1 λ1 0
A = QΛQ , Λ= and Q = (ν 1 ν 2 ),
0 λ2

where ν 1 and ν 2 are the eigenvectors corresponding to eigenvalues λ1 and λ2 . Using this
decomposition, it is easy to solve for At :
!
λt1 0
At = QΛQ−1 × QΛQ−1 × · · · × QΛQ−1 = QΛΛ . . . ΛQ−1 = QΛt Q−1 = Q Q−1 .
0 λt2
!
1 0.5
To illustrate this approach, consider an example with A = and follow these steps:
1 1.5
2
The complex and repeated eigenvalues appear routinely in macroeconomic problems and require special
treatment. This material, however, goes beyond the scope of this course and can be found in textbook “Discrete
Dynamical Systems” by [Link].

10
 1. Find eigenvalues:

1−λ 0.5
|A − λI| = = (1 − λ)(1.5 − λ) − 1 · 0.5 = λ2 − 2.5λ + 1 = 0
1 1.5 − λ

The roots of this equation are λ1 = 2 and λ2 = 0.5.

2. Find eigenvectors and matrix Q:

! !
−1 0.5 1
A − λ1 I = ⇒ v1 =
1 −0.5 2
! !
0.5 0.5 1
A − λ2 I = ⇒ v2 =
1 1 −1
!
1 1
Thus, the matrix is given by Q = .
2 −1

3. Find matrix Q−1 : applying the formula

!−1 !
a b 1 d −b
=
c d ad − bc −c a

we get !
1 1 1
Q−1 = .
3 2 −1

4. Solve for xt :
! ! ! !
1 1 1 2t 0 1 1 x10
xt = QΛt Q−1 x0 =
3 2 −1 0 0.5t 2 −1 x20
!
1 2t (x10 + x20 ) + 0.5t (2x10 − x20 )
=
3 2t (2x10 + 2x20 ) + 0.5t (−2x10 + x20 )

Of course, one can only guarantee analytical solution for a system of 2 equations. If the number
of variables and equations exceeds 2, numerical methods can be used to solve the system.

Phase diagram The dynamics of two-dimensional systems can be illustrated with phase
diagrams. Although this also applies to non-linear systems, we focus here on the case of
linear dynamic equations. Drawing a phase diagram includes several steps, each one making
it more informative about the dynamics of the system:

11
 1. Draw the iso-lines, i.e. the two lines that show the points where respectively x1t and x2t
do not change in time. To solve for the iso-lines, it is sufficient to impose x1t+1 = x1t
and x2t+1 = x2t . The intersection of the two lines corresponds to the steady state of the
model as both variables are stationary at that point.

2. Each iso-line also divides the space into an area where xit+1 > xit and an area where
xit+1 < xit . Putting arrows allows to see in which direction the system evolves starting
from any given point.

3. Using the fact that Q = (v 1 v 2 ), notice next that the dynamic system can be rewritten
in terms of eigenvectors:

xt = Axt−1 = QΛQ−1 xt−1 = v 1 λt1 y10 + v 2 λt2 y20 ,

where yt ≡ Q−1 xt . Here, y10 and y20 are some scalars and at any point in time, xt
is a weighted sum of eigenvectors v 1 and v 2 scaled by the corresponding eigenvalues.
Effectively, the idea is to go from the original basis to the one given by the eigenvectors
y0 = Q−1 x0 , then find the dynamics of yt = Λt y0 , and finally to go back to the original
basis xt = Qyt . Thus, drawing eigenvectors helps better understand the dynamics.

4. Similarly to the case of one difference equation, the dynamics of the system depends on
the eigenvalues λ1 and λ2 . Each of them determines the dynamics of yit in the decoupled
system. The different combinations of paths for y1t and y2t give rise to different dynamics
of xt around the steady state, which is called

(a) Sink: if |λ1 | < 1 and |λ2 | < 1, then xt → 0, the system is stable.
(b) Source: if |λ1 | > 1 and |λ2 | > 1, then xt → ∞, the system is unstable.
(c) Saddle point: if |λ1 | > 1 and |λ2 | < 1 (or vice versa), then the system is stable
along one dimension and unstable along the other.

Blanchard-Kahn method So far, we have assumed that the initial values of the variables x0
are known and therefore, the dynamic path xt can be found by iterating the system backward.
In contrast, the agents in most macroeconomic models are forward-looking, i.e. their actions
today depend on what their expectations about the future. This can be easily seen in the
saving problem discussed in the previous lecture, where according to the Euler equation, the
optimal consumption today depends on agent’s expectations about consumption in the next
period u0 (Ct ) = βR Et u0 (Ct+1 ). In contrast to the accumulated wealth Bt , the initial value of
consumption is not given and has to be determined endogenously. In particular, suppose we

12
can reduce the saving problem to a system
!
bt
xt+1 = Axt , where xt = ,
ct

where bt and ct are some transformation of savings and consumption. Decomposing matrix
A we get a phase digram with multiple transition paths, each corresponding to some initial
values of b0 (given) and c0 (not known). Each of the paths satisfies both the budget constraint
and the Euler equation and hence, to choose c0 and the equilibrium trajectory we need to
use another optimality condition — the tranversality condition. In “well-behaved models”, the
latter usually implies that the optimal trajectory should eventually lead to the steady state. If
there is only one such trajectory, then the equilibrium is unique. If the the number of such
trajectories is greater than one, then there is indeterminacy and multiplicity of equilibria.
Finally, if there are no such trajectories, then the model has no solution.
The Blanchard-Kahn principle generalizes this insight. Partition variables into two classes:

1. state variables are predetermined at the beginning of a period by the previous history,
2. control variables are not predetermined and can be freely adjusted by the agents.

E.g. household wealth Bt is a state variable and the level of consumption Ct is a control
variable. Similarly, the capital stock of a firm Kt is a state variable, while the level of investment
It is a control.3 Denote the number of control variables with m and the number of eigenvalues
greater than one (in absolute terms) |λi | > 1 with n. Then the Blanchard-Kahn principle says
that the system has (i) exactly one solution if n = m, (ii) multiple solutions if n < m, (iii) no
solution if n > m. ! !
bt 1 0.5
As an example consider again the case when xt = and A = . As
ct 1 1.5
shown above, the eigenvalues of the matrix are λ1 = 2 and λ2 = 0.5 and hence, n = m and
the model has a unique solution. To find it, consider the equation of the decoupled system that
corresponds to λ1 :
y1t+1 = λ1 y1t .

Iterating this equation forward we get

y1t = lim λ−j

1 y1t+j .
j→∞

Assume that the transversality condition requires convergence to the steady state. Then
limj→∞ y1t+j = 0 and y1t = 0. Another way to understand this condition is to notice that
if y1t 6= 0 in some period t, then y1t → ∞ and the system does not converge to zero. Given
3
However, the same macro variables can play the role of states or controls depending on a particular model.

13
yt = Q−1 xt , condition y1t = 0 implies bt + ct = 0. Thus, for any given initial value of b0 , we
can find the initial value of consumption c0 = −b0 and then iterate the system to solve for the
equilibrium path (bt , ct ), which corresponds to moving along saddle path y2t .

Stochastic shocks The important advantage of the perturbation methods is that they can
be used to solve stochastic models. Numerically, this approach allows to solve the dynamic
stochastic general equilibrium (DSGE) models with hundreds of variables and equations used
by the central banks around the world. To build intuition, consider the first-order stochastic
system of two linear equations:

xt+1 = Axt + Dεt+1 ,

where εt are exogenous shocks. If all eigenvalues are less than one in absolute terms, then
the system is completely backward-looking and the dynamics of the model is given by initial
conditions x0 and the history of shocks {εt }.
A more interesting case that arises often in macroeconomic applications is the following

Et xt+1 = Axt + Dεt .

Here, the optimal decisions of agents today xt depend on their expectations about the future
Et xt+1 . As above, we focus on the case when λ1 > 1, λ2 < 1 and solve for the saddle path. To
this end, decompose matrix A and rewrite the linear system as

Et yt+1 = Λyt + ωt ,

where yt ≡ Q−1 xt and ωt ≡ Q−1 Dεt . Consider the first equation of the system that corre-
sponds to the eigenvalue greater than one:

Et y1t+1 = λ1 y1t + ω1t .

As before, iterate forward this equation:

∞
1 1 1 X −j
y1t = − ω1t + Et y1t+1 = · · · = − λ1 Et ω1t+j + lim Et λ−j
1 y1t+j .
λ1 λ1 λ1 j=0 j→∞

The transversality condition implies that the last term is zero: in expectation, the system con-
verges back to the steady state. The first term, on the other hand, depends on the stochastic
properties of the shocks. For example, if shock εt+1 are i.i.d. (uncorrelated across time periods)

14
and have zero mean, then Et ωt+j = 0 for j > 0 and we get

1
y1t = − ω1t .
λ1

Using equation xt = Qyt , we get back to the original variables xt and solve for the saddle path,
i.e. express a control variables x2t in terms of the state variable x1t and shocks ω1t .

Linearization The equilibrium conditions in macroeconomic models are usually not linear
and there are no general mathematical tools that can be used to solve such systems. To make
progress, economists often focus on linear approximations to the equilibrium systems, which
can then be solved using the techniques discussed above. This approach includes:
1. Choosing a point of approximation. If the economy is stable, then it is reasonable to
assume that it spends most of the time in the neighbourhood of the steady state and
hence, the latter can be used as a point of approximation.
2. Linearizing the equilibrium conditions. Because the deviations in levels are sensitive to
denomination (pounds vs. pence) and hard to interpret, economists usually focus on
log-percent deviations from the steady state.
The following approach works in most cases:

Xt Xt
Xt = X̄ = X̄elog X̄ ≡ X̄ext ≈ X̄(1 + xt ),
X̄

where xt is the log-deviation of Xt from the steady-state value X̄. A few useful properties:

Zt = Xta ⇒ zt = axt ,
Zt = Xt Yt ⇒ zt = xt + yt ,
X̄ Ȳ
Zt = Xt + Yt ⇒ zt = axt + (1 − a)yt , a ≡ =1− .
Z̄ Z̄

Example Consider again the savings model discussed in the previous lecture.4 To simplify
the analysis, assume the CRRA preferences, βR = 1, and i.i.d. shocks to Yt . The equilibrium
system includes the household budget constraint

Bt+1 = RBt + Yt − Ct ,

and the Euler equation

Ct−σ = Et Ct+1
−σ
.

Solving the model consists of several steps:

4
As shown below, this model is not stationary and therefore, local approximations might be not very accurate.
We use this model only for pedagogical reasons to show how it can be solved using different methods.

15
1. Choose the point of approximation: this model does not a have a unique steady state,
which complicates the first step. Instead, we are going to use the initial level of wealth
B̄ = B0 > 0. Substitute this value into the budget constraint to find the steady-state
value of consumption C̄ = (R − 1)B̄ + Ȳ . Define the exogenous ratio α ≡ ȲB̄ and note
C̄
that B̄ = R − 1 + α.

2. Log-linearize the equilibrium system: apply the formulae from above to the EE

C̄ −σ (1 − σct ) = Et C̄ −σ (1 − σct+1 )

and simplify to
ct = Et ct+1 .

Similarly, the budget constraint can be expressed as

B̄(1 + bt+1 ) = RB̄(1 + bt ) + Ȳ (1 + yt ) − C̄(1 + ct ),

which after simplification becomes

Ȳ C̄
bt+1 = Rbt + y t − ct ,
B̄ B̄

or equivalently,
bt+1 = Rbt + αyt − (R − 1 + α)ct ,

Thus, the linearized system is

! ! !
bt R −(R − 1 + α) α
Et xt+1 = Axt + Dyt , xt = , A= and D =
ct 0 1 0

3. Diagonalize matrix A:
|A − λI| = (R − λ)(1 − λ) = 0

The roots of this equation are λ1 = R > 1 and λ2 = 1. The corresponding eigenvectors
are given by
! !
0 −(R − 1 + α) 1
A − λ1 I = ⇒ v1 =
0 1−R 0
! !
R − 1 −(R − 1 + α) R−1+α
A − λ2 I = ⇒ v2 =
0 0 R−1

16
 It follows that !
R 0
Λ= ,
0 1
!
1 R−1+α
Q= ,
0 R−1
!
1 R − 1 −(R − 1 + α)
Q−1 =
R−1 0 1
and !
α
Q−1 D = .
0

4. Find the saddle path: define zt ≡ Q−1 xt and consider the first equation of the system

Et zt+1 = Λzt + Q−1 Dyt

that corresponds to the eigenvalue greater than one:

∞
α 1 α X −j
z1t = − yt + Et z1t+1 = · · · = − R Et yt+j + lim R−j Et z1t+j
R R R j=0 j→∞

The last term is equal zero according to the transversality condition. The i.i.d. nature
of income shocks ensures that Et yt+j = 0 for j > 0. It follows that z1t = − Rα yt and
substituting in the definition of zt we get

α
(R − 1)bt − (R − 1 + α)ct = − (R − 1)yt .
R

Thus, the saddle path is given by

R−1  α 
ct = bt + y t .
R−1+α R

5. Find the law of motion for the state variable: substitute the optimal consumption func-
tion into the budget constraint:

α
bt+1 = bt + yt .
R

Thus, the wealth of households bt follows a random walk with innovations Rα yt .

Continuous time Similar methods can be used to solve dynamic models in continuous-
time. The difference equations are replaced with the differential equations in this case. Given

17
this analogy, we only scetch the key ideas. Start again with a one-dimension linear equation

ẋt = λxt ,

which can also be written as

dxt
= λdt.
xt
Integrating the latter expression, we get

xt
log = λt
x0

or equivalently,
xt = x0 eλt .

It is clear from this expression that xt → 0 if λ < 0 and xt → ∞ if λ > 0. The steady
state can be found from ẋt = 0 and is unique as long as λ 6= 0. Thus, the threshold for
λ of 0 plays the same role in continuous time as the threshold of ±1 in discrete time. This
analogy can be seen clearly if we rewrite the difference equation for an arbitrary period of
time xt+∆ = Λxt = (1 + λ∆)xt and transform it as xt+∆∆−xt = λxt . Taking the limit ∆ → 0,
we get the differential equation and the mapping λ = Λ − 1.
Consider next the system of linear differential equations

ẋt = Axt ,

where xt is a vector. If matrix A is diagonal, then the system is decoupled and each equation
can be solved independently. Otherwise, we can use the same decomposition A = QΛQ−1 to
transform the system into a decoupled system for y ≡ Q−1 xt :

ẏt = Λyt .

The Blanchard-Kahn principle still applies: to have a unique solution, the number of control
variables in a model has to be equal to the number of eigenvalues greater than one. The
continuous-time equations can also be linearized in the same way as the discrete-time analogs.

18
3 Recursive Approach
Backward induction Consider again a finite-period deterministic savings problem:

T
X
max β t u(Ct )
{Ct ,Bt+1 }
t=0

s.t. Bt+1 = RBt + Yt − Ct

BT +1 ≥ 0.

Following a standard sequential approach, we get the Euler equations

u0 (Ct ) = βR u0 (Ct+1 ), ∀t < T

and the transversality condition

BT +1 = 0.

These optimality conditions together with the budget constraint constitute a system of 2 ×
(T + 1) equations with 2 × (T + 1) unknowns. As the horizon T goes up, the dimensionality
of the problem and the computational burden increase as well.
The key idea of the recursive methods is to solve the problem sequentially, period by pe-
riod, starting with the very last one and then moving backwards to period zero. In this case,
there is no need to solve a large system of equations. Instead, the problem reduces to multiple
steps, each computationally very simple. To see this, we solve the model backwards assuming
for simplicity log utility u(C) = log C and zero income Yt = 0:

1. Period T : the optimal decision is trivial as the agent consumes all of her wealth leaving
zero bequest
BT +1 = 0, CT = RBT

It follows the utility the household gets in the last period is equal

VT = log(RBT )

2. Period T − 1: the agent starts the period with wealth RBT −1 and chooses saving to
maximize utility in the two remaining periods of life:
n o
max log(CT −1 ) + βVT (BT )
BT

s.t. BT = RBT −1 − CT −1

19
 Substitute VT and the budget constraint into the objective function:
n o
max log(RBT −1 − BT ) + β log(RBT )
BT

The first-order condition

1 β
=
RBT −1 − BT BT
implies
β 1
BT = RBT −1 , CT −1 = RBT −1 .
1+β 1+β
and the total utility from the last two periods is equal

1 βR
VT −1 = log CT −1 + βVT = log RBT −1 + β log RBT −1
1+β 1+β

RBT −1
= (1 + β) log + β log(βR).
1+β

3. Period T − 2: the agent starts the period with wealth RBT −2 and chooses saving to
maximize utility in the current and all remaining periods:
n o
max log(CT −2 ) + βVT −1 (BT −1 )
BT −1

s.t. BT −1 = RBT −2 − CT −2

Substitute VT −1 and the budget constraint into the objective function:

n RBT −1 2
o
max log(RBT −2 − BT −1 ) + β(1 + β) log + β log(βR)
BT −1 1+β

Take the first-order condition

1 β(1 + β)
=
RBT −2 − BT −1 BT −1

and solve it to obtain

β(1 + β) 1
BT −1 = RBT −2 , CT −2 = RBT −2 .
1 + β(1 + β) 1 + β(1 + β)

It follows that the total welfare from the last three periods is equal

RBT −2
VT −2 = log CT −2 + βVT −1 = (1 + β + β 2 ) log + β(1 + 2β) log(βR)
1 + β + β2

4. . . .

20
 5. Period t: although we can follow the same steps up until period zero, this problem is
simple enough to solve it using the “guess and verify” method. Indeed, the only thing
we need to know about Vt+1 to solve the optimization problem in period t is how it
depends on Bt+1 . The previous steps suggest that

Vt+1 = (1 + β + · · · + β T −t−1 ) log Bt+1 + constt+1 ,

where constt+1 is some constant that depends only on parameters β and R. Using this
“educated guess”, the optimization problem of period t can be written as
n o
T −t−1
max log(RBt − Bt+1 ) + β(1 + β + · · · + β ) log Bt+1 + βconstt+1 .
Bt+1

Take the first-order condition

1 β(1 + β + · · · + β T −t−1 )
=
RBt − Bt+1 Bt+1

and solve it to get

β + β 2 + · · · + β T −t 1
Bt+1 = RBt , Ct = RBt .
1 + β + · · · + β T −t 1 + β + · · · + β T −t

The resulting total welfare is equal

Vt = log Ct + βVt+1 = (1 + β + · · · + β T −t ) log Bt + constt ,

confirming the guess above.

Of course, not every problem can be solved using the guess-and-verify method. In some cases,
it is not even possible to solve analytically the optimization problem in a given period. How-
ever, implemented numerically, the method is extremely powerful and can be applied to a
large class of models. Moreover, increasing horizon T to infinity actually simplifies the prob-
lem. This surprising result can be easily seen from the expressions above: while Vt (B) depends
on t when T is finite, taking the limit T → ∞, results in

1
V (B) = log B + const,
1−β

where const is independent of t or Bt . Thus, the continuation value V (Bt ) depends only on
the level of wealth at the beginning of the corresponding period. The optimization problem
in arbitrary period of an infinitely-lived agent is given by
n 1 o
max log(RB − B 0 ) + β log B 0 + β · const ,
B0 1−β

21
where a prime denotes the next period’s variable. The optimality condition

1 β 1
0
=
RB − B 1 − β B0

implies
Bt+1 = βRBt , Ct = (1 − β)RBt

and

1 β
V (B) = log Ct + βV (Bt+1 ) = log Bt + log(1 − β)R + log βR + βconst.
1−β 1−β

Substituting the guess for V (B) on the left hand side of the equation, we can solve for the
constant term:
1 h β i
const = log(1 − β)R + log βR .
1−β 1−β
The theory of dynamic programming generalizes these insights. Instead of fousing on
the optimal sequences {Ct , Bt+1 }, it looks for time-invariant value function V (·) and policy
function Bt+1 = g(·) that solve the Bellman equation:
n o
0 0
V (B) = max
0
u(RB − B ) + βV (B ) .
B

Intuitively, the policy function allows to recover the optimal path as B1 = g(B0 ), B2 =
g(B1 ) = g(g(B0 )), etc. To find the policy function, we need to solve the optimization prob-
lem in some period t. The state variable summarizes the past, i.e. the wealth accumulated in
previous periods, while the value function summarizes the future, i.e. the maximum utility
attainable in the next periods. Knowing these two objects is sufficient to resolve the trade-off
between current and future consumption.

Optimality conditions Although this is hardly the main application of the Bellman equa-
tion, the recursive methods can be used to derive the same optimality conditions that we
previously obtained following the sequential approach. Indeed, the first order condition for
the Bellman equation is

u0 (RB − B 0 ) = βV 0 (B 0 ) ⇒ u0 (C) = βV 0 (B 0 ),

while the envelope condition for the value function, also called the Benveniste-Scheinkman
equation, implies

V 0 (B) = u0 (RB − B 0 )R ⇒ V 0 (B) = u0 (C)R.

22
Combining together these conditions, we get the Euler equation from the previous lecture

u0 (C) = βR u0 (C 0 ).

Computation How can we solve the Bellman equation when simple analytical expressions
for V (·) do not exist? Luckily, the contraction mapping theorem can be used to show that
under some regularity conditions, the Bellman equation has unique solution and iterating the
equation starting from an arbitrary guess for a value function will eventually lead to the right
solution. This formalizes the idea that VT (·) → V () as the horizon goes to infinity T → ∞.
This theoretical result suggests the following numerical algorithm:

1. Start with an arbitrary function V0 (·) defined on a grid B.

2. For every value of B, solve the Bellman equation to obtain V1 (·)

n o
0 0
V1 (B) = max
0
u(RB − B ) + βV0 (B ) .
B

3. Iterate this procedure updating the value function at each step

n o
0 0
Vj+1 (B) = max
0
u(RB − B ) + βV j (B ) .
B

4. Stop when the value function converges, i.e. the functions Vj+1 (·) and Vj (·) are “close”
for all values of B.

5. Solve for the policy function B 0 = g(B).

This algorithm works surprisingly well for basic macroeconomic models and can be improved
in many ways to accommodate more complex problems. The important advantage of the
dynamic programming over the sequential approach is that it can solve discrete problems. In-
deed, the computation does not involve taking the derivatives and does not require the func-
tions to be differentiable. Similarly, the method can easily accommodate occasionally binding
constraints, which give rise to additional complementary slackness conditions under the se-
quential approach. The same iterative method can also be applied to policy functions instead
of the value function.

Stochastic shocks Another big advantage of the recursive approach is that it can be easily
applied to models with uncertainty. To see this, consider the baseline savings problem with
random income shocks Yt . In particular, assume that shocks follows a first-order Markov
process, i.e. the value of Yt depends only on the realization of income in the previous period
Yt−1 , but not on Yt−2 , Yt−3 , etc. In this case, there are two state variables that determine the

23
continuation value in every state of the world — the endogenous wealth Bt and the exogenous
income shock Yt . Therefore, the Bellman equation is
n o
0 0 0
V (B, Y ) = max
0
u(RB + Y − B ) + βEY 0 |Y V (B , Y ) ,
B

where EY 0 |Y [·] denotes conditional expectation given the value of Y . The only difference from
the deterministic case is that there is uncertainty about the future values of shocks and when
making the optimal choice, the agent uses the average continuation value. The resulting policy
function is B 0 = g(B, Y ). Except for the expectation operator, the numerical algorithm and
the derivation of the optimality conditions are same as above.

Theory We briefly discuss the main theoretical results about dynamic programming. For
simplicity, the focus is exclusively on models without uncertainty. Define the value function
as a solution to a general dynamic problem
∞
X
V (Bt ) = max ∞ β j u(Bt+j , Bt+j+1 ) (5)
{Bt+j+1 }j=0
j=0

s.t. Bt+j+1 ∈ Γ(Bt+j ),

where B is a vector, Γ(·) is a non-empty correspondence, and Bt is given. The next result
shows that this problem can be restated in terms of a functional equation, i.e. an equation
with unknown function, rather than a variable.

Theorem 1 (Principle of optimality) Solving sequential problem is equivalent to solving the

Bellman equation n o
0 0
V (B) = max
0
u(B, B ) + βV (B ) . (6)
B ∈Γ(B)

Proof: Partition the sum into two terms:

n ∞
X o
V (Bt ) = max ∞ u(Bt , Bt+1 ) + β j u(Bt+j , Bt+j+1 )
{Bt+j+1 }j=0
j=1
n ∞
X o
= max u(Bt , Bt+1 ) + max ∞ β j u(Bt+j , Bt+j+1 )
Bt+1 {Bt+j+1 }j=1
j=1
n ∞
X o
= max u(Bt , Bt+1 ) + max β j+1 u(B(t+1)+j , B(t+1)+j+1 )
Bt+1 {B(t+1)+j+1 }∞
j=0
j=0
n o
= max u(Bt , Bt+1 ) + βV (Bt+1 )
Bt+1

where the last step follows from the definition of V (Bt+1 ) and the constraint Bt+j+1 ∈ Γ(Bt+j )
is suppressed to simplify notation. 

24
 Additional results from functional analysis are required to characterize solution to the
Bellman equation. Recall that a metric space is a set of elements with a distance function
(metric) ρ that satisfies three conditions:
1. ρ(x, y) ≥ 0 with equality if and only if x = y,
2. ρ(x, y) = ρ(y, x),
3. ρ(x, y) ≤ ρ(x, z) + ρ(z, y) (the triangle inequality).
A metric space S is complete if the limit of any converging sequence {xn } ∈ S belongs to S.
The mapping of a metric space into itself T : S → S is a contraction mapping if there exists
β ∈ (0, 1) such that ρ(T x, T y) ≤ βρ(x, y) for any x, y ∈ S. The fixed point of a mapping is
defined by T x = x. The next result plays central role in our analysis.

Contraction Mapping Theorem If T : S → S is contraction mapping in complete metric

space S, then (i) T has unique fixed point x ∈ S and (ii) T n x0 → x for any x0 ∈ S.

Proof: choose arbitrary x0 ∈ S and consider a sequence defined by xn = T xn−1 = T n x0 . From

the definition of the contraction mapping,

ρ(xn+1 , xn ) = ρ(T xn , T xn−1 ) ≤ βρ(xn , xn−1 ) ≤ · · · ≤ β n ρ(x1 , x0 ).

For any m > n, the triangular inequality implies that

ρ(xm , xn ) ≤ ρ(xm , xm−1 ) + · · · + ρ(xn+1 , xn ) ≤ (β m−1 + · · · + β n )ρ(x1 , x0 )

βn
= β n (1 + · · · + β m−n−1 )ρ(x1 , x0 ) ≤ ρ(x1 , x0 ).
1−β
It follows that ρ(xm , xn ) → 0 as n → ∞ and hence, there exists x ∈ S such that {xn } → x.
Using the triangular inequality and the definition of the contraction mapping, we get

ρ(T x, x) ≤ ρ(T x, T n x0 ) + ρ(T n x0 , x) ≤ βρ(x, T n−1 x0 ) + ρ(T n x0 , x).

Both terms on the right hand side converge to zero as n → ∞, which implies ρ(T x, x) = 0
and by the properties of the metric function, T x = x. To prove the uniqueness, suppose there
are two fixed points x and y. It follows that

ρ(x, y) = ρ(T x, T y) ≤ βρ(x, y),

which contradicts β < 1. 

Corollary Let T : S → S be a contraction mapping in complete metric space S with a fixed

point x. If S 0 is a closed subset of S and T (S 0 ) ⊆ S 0 , then x ∈ S 0 .

25
Proof: The assumption T (S 0 ) ⊆ S 0 implies that starting with arbitrary x0 ∈ S 0 , the sequence
T n x0 stays within S 0 . Because T is a contraction mapping, the limit of this sequence x is a
fixed point. Since S 0 is closed, x ∈ S 0 . 
To apply these results to the Bellman equation, we need to choose the appropriate metric
space and to show that the Bellman operator is a contraction mapping. In particular, consider
a set of bounded continuous functions f : X → R with ρ(f, g) ≡ max|f (x) − g(x)|. It can be
x∈X
shown that this is a complete metric space. Define mapping
n o
T f ≡ max
0
u(B, B 0 ) + βf (B 0 ) .
B ∈Γ(B)

and impose the following restrictions

Assumption 1 Function u(·, ·) is bounded and continuous, correspondence Γ(·) is compact-

valued and continuous.

While this assumption is not satisfied in most macroeconomic models because the objective
function is not bounded, it is usually possible to impose a non-binding constraint B ≤ B̄ such
that all functions are bounded in the admissible domain.

Theorem 2 Under Assumption 1, the Bellman operator T is a contraction mapping in a space

of bounded continuous functions, has a unique fixed point V with T n V0 → V for any V0 .

Proof: Assume that f belongs to a set of bounded continuous function. Then T f is also
bounded. The theorem of the maximum implies that T f is also continuous. Thus, the Bell-
man operator maps the space of bounded continuous functions into itself. It remains to show
that it is also a contraction mapping. To this end, consider functions f and g and denote the
corresponding policy functions with B 0 = hf (B) and B 0 = hg (B):

T f − T g = u(B, hf (B)) + βf (hf (B)) − u(B, hg (B)) − βg(hg (B))

≤ u(B, hf (B)) + βf (hf (B)) − u(B, hf (B)) − βg(hf (B))
h i
= β f (hf (B)) − g(hf (B))

≤ β max |f (B) − g(B)| = βρ(f, g).

B

By symmetry, T g − T f ≤ βρ(f, g). Because this holds for any value of B, we get

ρ(T f, T g) = max |T f − T g| ≤ ρ(f, g).

B

From the contraction mapping theorem, it follows that T has a unique fixed point V and any
sequence T n V0 converges to V . 

26
 This theorem guarantees that the Bellman equation has exactly one solution, which can be
found by iterating the equation starting with an arbitrary guess V0 . In addition, one can also
prove some properties of the value function.

Assumption 2 Function u(·, B 0 ) is increasing in the first argument and correspondence Γ(·) is
monotone, i.e. B1 ≤ B2 implies Γ(B1 ) ⊆ Γ(B2 ).

Theorem 3 Under Assumptions 1-2, the fixed point V (·) is an increasing function.

Proof: Let S’ be a set of bounded, continuous, increasing functions, which is a closed subset of
a set S of bounded continuous functions. Then for f ∈ S 0 and B2 ≥ B1

T f (B2 ) − T f (B1 ) = u(B2 , hf (B2 )) + βf (hf (B2 )) − u(B1 , hf (B1 )) − βf (hf (B1 ))
≥ u(B2 , hf (B1 )) + βf (hf (B1 )) − u(B1 , hf (B1 )) − βf (hf (B1 ))
= u(B2 , hf (B1 )) − u(B1 , hf (B1 )) ≥ 0,

where we used the fact that hf (B1 ) ∈ Γ(B2 ) according to the monotonicity assumption. Thus,
T f is an increasing function and T maps S 0 into itself. The corollary above implies then that
the fixed point belongs to S 0 . 

Assumption 3 Function u(·, ·) is concave and correspondence Γ(·) is convex.

Theorem 4 Under Assumptions 1 and 3, the fixed point V (·) is a concave function.

The proof of this result relies on the corollary from above and follows the same steps as the
previous theorem.

HJB equation Just like the sequential approach, the recursive methods can be also applied to
solve continuous-time problems. To see this, consider again the deterministic savings problem
with zero income Yt = 0 and assume an arbitrary period length ∆. Note that the value function
is a stock as it reflects the continuation value at a given point in time. Given that β ≡ e−ρ∆ ,
the Bellman equation (6) becomes
n o
V (Bt ) = max u(Ct )∆ + e−ρ∆ V (Bt+∆ ) .
Ct

Because V (Bt ) does not affect the optimal choice, we can subtract it from both sides of the
equation to obtain
n o
max u(Ct )∆ + e−ρ∆ V (Bt+∆ ) − V (Bt ) = 0.
Ct

27
Use the approximation e−ρ∆ ≈ 1 − ρ∆ for small values of ∆ and rewrite the problem in terms
of the control variable:
n o
max u(Ct )∆ − ρV (Bt+∆ )∆ + V (Bt+∆ ) − V (Bt ) = 0.
Ct

Divide the equation by ∆ and express it as follows:

n V (Bt+∆ ) − V (Bt ) Bt+∆ − Bt o
max u(Ct ) − ρV (Bt+∆ ) + = 0.
Ct Bt+∆ − Bt ∆

V (Bt+∆ )−V (Bt ) Bt+∆ −Bt

Note that as ∆ → 0, Bt+∆ → Bt and hence, Bt+∆ −Bt
→ V 0 (Bt ) and ∆
→ Ḃt :
n o
0
max u(Ct ) − ρV (Bt ) + V (Bt )Ḃt = 0.
Ct

Lastly, substitute in the continuous-time budget constraint from the previous lecture, drop
time subscripts, and rearrange terms to get the Hamilton-Jacobi-Bellman (HJB) equation:
n o
0
ρV (B) = max u(C) + V (B)(rB − C) . (7)
C

Although this might look like a standard differential equation, solving it does not really require
boundary conditions. Similarly to the Bellman equation, this is a contraction that converges to
unique solution. There are well-developed numerical tools to solve such differential equations.
As the Bellman equation in discrete time, the HJB equation can be used to derive the Euler
equation. The first order optimality condition requires

u0 (Ct ) = V 0 (Bt )

in every period of time. Therefore, differentiating this equation with respect to t, we get

u00 (Ct )Ċt = V 00 (Bt )Ḃt .

On the other hand, the envelope condition implies

ρV 0 (Bt ) = rV 0 (Bt ) + V 00 (Bt )Ḃt .

Combining these expressions, we get

ρu0 (Ct ) = ru0 (Ct ) + u00 (Ct )Ċt

28
which can be rewritten as the Euler equation from the previous lecture:

u00 (Ct )
− Ċt = r − ρ.
u0 (Ct )

Example Consider again the case of log utility u(C) = log C and no labor income Yt = 0,
so that the HJB equation is given by
n o
ρV (B) = max log C + V 0 (B)(rB − C) .
C

Following our results for the discrete-time problem, conjecture that V (B) = α + γ log B,
where α and γ are some unknown constants. Substitute this conjecture into the HJB, take the
FOC, and express consumption function as C = Bγ . Substitute this result together with the
conjectured value function into the HJB:
 
B γ B
ρα + ργ log B = log + rB − .
γ B γ

Combing constant terms and terms with log B, we get

1 log ρ + r/ρ − 1
γ= , α= .
ρ ρ

Finally, notice that this solution can also be obtained by writing the discrete-time value func-
tion from above for an arbitrary period length ∆ and taking the limit ∆ → 0.

Stopping-time problem A large number of models in economics belongs to the optimal

stopping time problems, in which the agent needs to decide when to make a discrete choice
of selling an asset, adjusting a price, closing a plant. Using continuous time and the HJB is
the easiest way to solve such models. To see this, consider a simple option problem: an agent
holds an asset that pays dividends Xt = X0 − µt every period and can sell it at any point
at price P . The optimal strategy for the agent is then to hold the asset as long as Xt is high
enough and to sell it once the dividends fall below some threshold X̄. Let V (X) denote the
value of holding the asset. The discrete-time version of the Bellman equation is
n o
V (Xt ) = max Xt ∆ + e−ρ∆ V (Xt+∆ ), P .

Consider first the inaction region Xt ≥ X̄ and use the first-order approximations for small
∆ to derive the HJB equation
ρV (X) = X − µV 0 (X). (8)

Notice this differential equation alone is not sufficient to solve model because it requires a

29
boundary condition and an additional condition to pin down X̄. These two additional condi-
tions describe the properties of value function at the threshold X̄. In particular, consider one
period before the stopping time when Xt = X̄ + µ∆:

V (X̄ + µ∆) = (X̄ + µ∆)∆ + (1 − ρ∆)P.

Taking the limit ∆ → 0, we get the value matching condition

lim V (X) = P, (9)

X↓X̄

which says that the value function converges to P from the right and is continuous at X = X̄.
This condition holds for any stopping threshold X̄, not just the optimally chosen one. On the
other hand, X̄ is also the point when the agent is indifferent between holding the asset and
selling it, which implies that for Xt = X̄

Xt ∆ + (1 − ρ∆)V (Xt+∆ ) = P.

Given that V (Xt+∆ ) = P , it follows

X̄ = ρP.

Going back to the expression one period before the stopping time, we get

V (X̄ + µ∆) − V (X̄) = X̄∆ + (1 − ρ∆)P − V (X̄) = (X̄ − ρP )∆.

It follows the right derivative of the value function at the optimal threshold is zero:

lim V 0 (X) = 0. (10)

X↓X̄

This is the smooth pasting condition, which combined with the value matching condition
says that both the value function and its derivative are continuous at X = X̄.
Combined together, the system (8)-(10) determines the optimal solution:
1. Get the homogeneous solution to the differential equation
ρ
µV 0 (X) = −ρV (X) ⇒ V (x) = eA− µ X ,

where A is a constant to be determined below.

2. Guess a particular solution V (X) = B + CX with unknown constants B, C and sub-

stitute into the differential equation:

ρ(B + CX) = X − µC.

30
 Using the method of undetermined coefficients, we get

1 µ
ρC = 1, ρB = −µC ⇒ C= , B=− .
ρ ρ2

3. Thus, the total solution to the differential equation is

ρ 1 µ
V (X) = eA− µ X + X − 2 .
ρ ρ

4. Write down the value matching and the smooth pasting conditions:

ρ 1 µ
eA− µ X̄ + X̄ = P + 2 ,
ρ ρ

ρ ρ 1
− eA− µ X̄ + = 0.
µ ρ
It follows that the optimal threshold is

X̄ = ρP,

the unknown constant is

ρ2
 
µ
A = P + log
µ ρ2
and thus, the value function for X ≥ X̄ is given by

µ µρ (ρP −X) 1 µ
V (X) = e + X − .
ρ2 ρ ρ2

The figure shows the threshold and the value function.

200

150

100

50

0
0 0.2 0.4 0.6 0.8 1

31
Diffusion process The HJB equation can be used to solve problems with uncertainty in
continuous time. However, before turning to the optimization, we need to introduce stochastic
processes in continuous time. To this end, rewrite a random walk process Wt+1 = Wt + εt for
an arbitrary length of period ∆:
√
Wt+∆ = Wt + εt ∆, εt ∼ N (0, 1), W0 = 0.

Intuitively, Wt should be interpreted as a stock and the innovation εt as a flow. Taking the
limit ∆ → 0, we get a standard Brownian motion (equivalently, Wiener process):
√
dWt = εt dt.

It can be shown that any continuous (i.e. without jumps) Markov process in continuous-time
can be expressed as a sum of rescaled by σ standard Brownian motion and a drift µ:

dxt = µ(x, t)dt + σ(x, t)dWt .

The mostly common examples include:

1. Brownian motion with drift (equivalent to a random walk with drift in discrete time):

dxt = µdt + σdWt .

2. Geometric Brownian motion (random walk in logs):

dxt = µxdt + σxdWt .

3. Ornstein-Uhlenbeck process (AR process in discrete time):

dxt = −θ(x − x̄)dt + σdWt .

From now on we will focus on processes with drift µ(x) and diffusion σ(x) independent of
time. For this reason, we will also suppress the time subscript. Two mathematical results about
stochastic processes are commonly used in economics:

Ito’s Lemma If dx = µ(x)dt + σ(x)dW , then the process for yt = f (xt ) is given by
 
1 00
dy = f (x)µ(x) + f (x)σ (x) dt + f 0 (x)σ(x)dW.
0 2
(11)
2

32
Proof: Take the second-order approximation

1
yt+∆ − yt = f (xt+∆ ) − f (xt ) = f 0 (xt )(xt+∆ − xt ) + f 00 (xt )(xt+∆ − xt )2
2

and the limit ∆ → 0

1
dyt = f 0 (xt )dxt + f 00 (xt )dx2t .
2
Suppress the time subscript and substitute in the process for dx:

 1 2
dy = f 0 (x) µ(x)dt + σ(x)dW + f 00 (x) µ(x)dt + σ(x)dW .
 
2

Because f (·) is continuous dy is a diffusion process just like dx. It follows that it is sufficient
to find its drift and diffusion – all other terms are of higher order and drop out. There is
only one term that determines the diffusion, f 0 (x)σ(x)dW . On the other hand, given that
√
2
E(dW )2 = E ε dt = dt, there are two terms that determine the drift. Combining them
together, we get equation (11). 
The lemma can be easily extended to a multivariate case. The surprising aspect of the Ito’s
lemma is that the volatility affects the drift of the resulting process. This term only drops out
for linear transformations f (·) and deterministic processes with σ = 0. The intuition comes
from the Jensen’s inequality: high volatility on average decreases concave transformations
and increases convex transformations. As an example, consider a geometric Brownian motion
dx and derive the process for y = log x. A naíve answer is

dx
dy = d log x = = µdt + σdW,
x

which is wrong as it ignores the additional term in the drift. Instead, given f 0 (x) = 1/x and
f 00 (x) = −1/x2 , the Ito’s lemma implies
 
1 2
dy = µ − σ dt + σdW.
2

Kolmogorov forward equation If dx = µ(x)dt + σ(x)dW , then the distribution of x in

period t is described by density p(x, t) that solves the second-order differential equation

1
pt (x, t) = − µ(x)p(x, t) x + σ 2 (x)p(x, t) xx ,
  
p(x, 0) = p0 (x). (12)
2

Proof: Consider an arbitrary function g(x) and apply the Ito’s lemma:
 
1 2
dg(x) = gx (x)µ(x) + gxx (x)σ (x) dt + gx σ(x)dW.
2

33
Focus on the drift of this process
 
dEg(x) 1 2
= E gx (x)µ(x) + gxx (x)σ (x) .
dt 2

Rewrite this expression using the probability density:

Z ∞ Z ∞  
d 1 2
g(x)p(x, t)dx = gx (x)µ(x) + gxx (x)σ (x) p(x, t)dx.
dt −∞ −∞ 2

The next step is to ensure that g(x) appears in each term. The derivative of the LHS wrt t is
R∞
equal −∞ g(x)pt (x, t)dx and already has g(x) in it. On the other hand, we need to integrate
the RHS by parts (twise) using the fact that p(x, t) → 0 when x → ±∞ (otherwise, the density
would not integrate to one):
 
∞
Z ∞ 
1 2
  1  2 
g(x)µ(x) + gx σ (x) p(x, t) − g(x) µ(x)px (x, t) x + gx (x) σ (x)px (x, t) x dx
2 −∞ −∞ 2
Z ∞ ∞
Z ∞
1 1
g(x) µ(x)px (x, t) x dx − g(x) σ 2 (x)px (x, t) x g(x) σ 2 (x)px (x, t) xx dx
     
=− +
2 −∞ 2 −∞
Z−∞∞ Z ∞
1
g(x) σ 2 (x)px (x, t) xx dx.
   
=− g(x) µ(x)px (x, t) x dx +
−∞ 2 −∞

Combining all these terms together we get

Z ∞  
  1 2 
g(x) pt (x, t) + µ(x)px (x, t) x − σ (x)px (x, t) xx dx = 0.
−∞ 2

Because this has to hold for arbitrary g(·), it follows that the term in brackets has to be zero,
which leads to equation (12). 
Thus, the Kolmogorov forward equation (KFE) – also called the Fokker-Planck equation
– describes the distribution of xt given the initial distribution and the law of motion. The
important corollary is that a stationary distribution p(x), if it exists, can be found from pt = 0:

1
µ(x)p(x) x = σ 2 (x)p(x) xx .
  
2

Stochastic HJB Consider again the consumption-savings problem when the length of the
period is ∆. The source of uncertainty is income Yt : denoting the output accumulated since
the starting period with Xt , we assume that the instantaneous output follows

dX = Y = µ(X)dt + σ(X)dW.

34
Substitute this into the expression for Ḃ derived above:

dB = (rB + µ(X) − C)dt + σ(X)dW.

Consider again the Bellman equation for period of length ∆ with uncertainty:
n o
V (Bt ) = max u(Ct )∆ + e−ρ∆ Et V (Bt+∆ ) .
Ct

Subtract V (Bt ) from both sides of the equation:

n

o
max u(Ct ) − ρV (Bt ) ∆ + (1 − ρ∆)Et V (Bt+∆ ) − V (Bt ) = 0.
Ct

Taking the limit ∆ → dt, we get

n
o
max u(C) − ρV (B) dt + EdV = 0.
C

Apply the Ito’s lemma to get the drift of dV to obtain stochastic HJB equation:
n 1 o
ρV (B) = max u(C) + V 0 (B)(rB + µ(X) − C) + V 00 (B)σ 2 (X) . (13)
C 2

Thus, we get another functional equation, which now also has a term with the second deriva-
tive and the volatility of shocks. The intuition remains the same as before: the value of the
asset depends on a flow utility, the price change due to fundamentals, and gains from uncer-
tainty if V (·) is convex.

Example Go back to the consumption-saving problem with u(C) = log C. To keep the
problem tractable, assume Y = 0 and the source of uncertainty is the stochastic return on
wealth (e.g. invested in equities). In particular, the wealth is assumed to evolve according to

dB = (rB − C)dt + σBdW.

The HJB is then given by

n
0 σ 2 00 2
o
ρV (B) = max log C + V (B)(rB − C) + V (B)B .
C 2

As before, we can apply the guess-and-verify method. Conjecture that V = α + γ log B and
substitute into the HJB:
n γ γσ 2 o
max log C + (rB − C) − .
C B 2

35
From the FOC we get C = B/γ. Substitute this back into the HJB and check

γσ 2
 
B 1
ρα + ργ log B = log + γ r − − .
γ γ 2

This confirms the conjecture and implies that

1 log ρ + r/ρ − σ 2 /(2ρ) − 1

γ= , α= .
ρ ρ

Thus, the volatility does not affect the policy function C(B), but lowers the value function:
because of concavity of the utility function, the uncertainty reduces the welfare.
Consider next an economy populated by a large number of agents, each solving this prob-
lem, but having idiosyncratic shocks to their income. How does the distribution of wealth
evolve in this economy? Substituting the optimal consumption in the budget constraint, we
get the individual wealth follows a geometric Brownian motion:

dB = (r − ρ)Bdt + σBdW.

To ensure that the stationary distribution exists, assume r < ρ and there is a “reflecting barrier”
Bmin such that the government does not allow private wealth to fall below this threshold (by
paying social security benefits). The KFE for a stationary distribution implies for B > Bmin

  σ2  2 
(r − ρ) Bp(B) B = B p(B) BB .
2

Conjecture that the solution to this ODE is p(B) = AB α and substitute it into the KFE:

σ2
(r − ρ)(1 + α)AB α = (2 + α)(1 + α)AB α .
2

This equality holds for any B if

2(r − ρ)
α= − 2.
σ2
To pin down the value of A, require that the density integrates to one:
Z ∞ ∞
A A
1= AB α dB = B 1+α =− B 1+α
Bmin 1+α Bmin 1 + α min

where we used the fact that α < −2. Denote ξ ≡ −(1 + α) and rewrite the solution as

ξ
p(B) = ξBmin B −ξ−1 .

This is the Pareto distribution: a higher reflection barrier shifts the wealth distribution to the

36
right, while higher returns r bring −ξ closer to zero increasing the fatness of the tail (i.e. more
rich people). The effect of higher volatility σ is similar.

Euler equation Finally, one can also derive the Euler equation for stochastic problems in
continuous time. This can be done either using the HJB or heuristically from the discrete-time
Euler equation. We will focus on the latter approach and use the example from above assuming
shocks to asset returns. The discrete-time EE is given by

u0 (Ct ) = Et e−ρ∆ Rt+∆ u0 (Ct+∆ ),

√
where Rt+∆ = 1 + r∆ + σεt+∆ ∆. Taking the approximation, we get
h √ i
Et u0 (Ct+∆ ) − u0 (Ct ) − u0 (Ct+∆ )(ρ − r)∆ + u0 (Ct+∆ )σεt+∆ ∆ = 0.

Take the limit ∆ → dt:

h i
Et du0 (Ct ) − (ρ − r)u0 (Ct )dt + σdu0 (Ct )dWt = 0,

where we used the fact that Et u0 (Ct )dWt = 0. Assuming that Ct is a continuous function of
Bt , we get that Ct follows a diffusion process and can be written as

dCt = µct dt + σct dWt

for some unknown µct and σct . Thus, we can apply the Ito’s lemma to express the first term in
the Euler equation as
 
0 00 1 000 2
Et du (Ct ) = u (Ct )µct + u (Ct )σct dt.
2

The last term in the Euler equation, on the other hand, is driven solely by the covariance
between consumption and the interest rate:

Et σdu0 (Ct )dWt = σσct u00 (Ct )dt.

Combine all these pieces together:

1
u00 (Ct )µct + u000 (Ct )σct
2
− u0 (Ct )(ρ − r) + u00 (Ct )σσct = 0,
2

and rewrite the resulting Euler equation in a standard form:

u00 (Ct ) 1 u000 (Ct ) 2 u00 (Ct )

− µ ct = r − ρ + σ + σσct .
u0 (Ct ) 2 u0 (Ct ) ct u0 (Ct )

37
Thus, relative to the deterministic case, there are two new terms: the precautionary savings
motive driven by u000 (·) > 0 increases savings and lowers consumption, while positive returns
on assets provide a natural hedge that boosts consumption.
It is easy to see that in the case of logarithmic preferences u(C) = log C, these two addi-
tional motives cancel each other and consumption function is the same as in the deterministic
case. Indeed, the Euler equation in this case becomes

µct σ2 σct
= r − ρ + ct2 − σ .
Ct Ct Ct

Conjecture that Ct = ρBt and therefore, dCt = (r − ρ)Ct dt + σCt dWt . Substituting these µct
and σct into the Euler equation confirms the optimality condition is satisfied.

38