Background Background

Dynamic Optimization in Discrete Time Dynamic Optimization in Discrete Time

Dynamic Optimization in Continuous Time Dynamic Optimization in Continuous Time
An EITM Example An EITM Example

Outline
1 Background
What is Optimization?
EITM: The Importance of Optimization
Dynamic Optimization 2 Dynamic Optimization in Discrete Time
An Introduction A Simple Two-period Consumption Model
The Bellman Equation
Cake Eating Problem
M. C. Sunny Wong
Profit Maximization
University of San Francisco 3 Dynamic Optimization in Continuous Time
The Method of Hamiltonian Multiplier
University of Houston, June 20, 2014 Cake Eating Problem Revisited
4 An EITM Example
Dynamics in a Money-in-the-Utility Model
TM: Theoretical Model
EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background Background
Dynamic Optimization in Discrete Time What is Optimization? Dynamic Optimization in Discrete Time What is Optimization?
Dynamic Optimization in Continuous Time EITM: The Importance of Optimization Dynamic Optimization in Continuous Time EITM: The Importance of Optimization
An EITM Example An EITM Example

Background Background
What is Optimization? What is Optimization?

Adam Smith (1776): “It is not from the benevolence of the

butcher, the brewer, or the baker that we expect our dinner,
but from their regard to their own self-interest. We address
In general, optimization is a technique which either maximizes ourselves not to their humanity but to their self-love, and
or minimizes the value of an objective function by never talk to them of our own necessities, but of their
advantages.” (The Wealth of Nations, Book I, Chapter II)
systematically choosing values of inputs (or choice variables)
from a feasible range. Key assumptions: (1) Rationality, and (2) Efficiency.
Optimization is one of the key ideas in the literature of If rationality is assumed, firms will maximize their profits
Economics. (supply) and households will maximize their utility (demand).
When their results are achieved, the market is in equilibrium
(efficiency).
This idea is first discovered by Wilfredo Pareto (Pareto
Optimality).

EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background Background
Dynamic Optimization in Discrete Time What is Optimization? Dynamic Optimization in Discrete Time What is Optimization?
Dynamic Optimization in Continuous Time EITM: The Importance of Optimization Dynamic Optimization in Continuous Time EITM: The Importance of Optimization
An EITM Example An EITM Example

Background Background
Types of Optimization Analytical Optimization

There are two general methods of optimization: There are three general types of analytical optimization:
Optimization without Constraints
Analytical optimization
First-order conditions (FOCs)
Solving the optimal solution(s) mathematically.
Optimization with Constraints
Numerical (or computational) optimization
The method of Lagrangian multiplier
Searching for the optimal solution(s) according to different
algorithms (using computers). Dynamic Optimization (with/without Constraints)
For example, simulations, calibrations, and maximum
Discrete time: The Bellman Equation
likelihood estimations.
Continuous time: The method of Hamiltonian multiplier

EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background Background
Dynamic Optimization in Discrete Time What is Optimization? Dynamic Optimization in Discrete Time What is Optimization?
Dynamic Optimization in Continuous Time EITM: The Importance of Optimization Dynamic Optimization in Continuous Time EITM: The Importance of Optimization
An EITM Example An EITM Example

Outline EITM: The Importance of Optimization

Causality, Assumptions and Models
1 Background
What is Optimization?
EITM: The Importance of Optimization Social scientists are interested in causal effects:
2 Dynamic Optimization in Discrete Time
A Simple Two-period Consumption Model How do x’s affect y ?
The Bellman Equation Empirical studies (e.g., regression analysis) can show us, at
Cake Eating Problem most, correlations among variables (not causality!) If the
Profit Maximization coefficient on x is significant, it could imply that:
3 Dynamic Optimization in Continuous Time x causes y ; or
The Method of Hamiltonian Multiplier y causes x ; or
Cake Eating Problem Revisited there is another unobservable variable, called z, which
4 An EITM Example contributes x and y to move simultaneously.
Dynamics in a Money-in-the-Utility Model But, how do we know if x’s really cause y ?
TM: Theoretical Model
EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background Background
Dynamic Optimization in Discrete Time What is Optimization? Dynamic Optimization in Discrete Time What is Optimization?
Dynamic Optimization in Continuous Time EITM: The Importance of Optimization Dynamic Optimization in Continuous Time EITM: The Importance of Optimization
An EITM Example An EITM Example

EITM: The Importance of Optimization EITM: The Importance of Optimization

Causality, Assumptions and Models Causality, Assumptions, and Models

But, how do we know if x’s really cause y ? A Famous Quote from Robert Solow (1956, page 65):
NOBODY TRULY KNOWS!! “All theory depends on assumptions which are not quite true.
We need to use our logical thinking and reasoning to describe That is what makes it theory. The art of successful theorizing
why x 0 s can cause y . is to make the inevitable simplifying assumptions in such a way
But, the world is just too complex! that the final results are not very sensitive.”
An easy way to do so is to build a theoretical model which “A "crucial" assumption is one on which the conclusions do
describes some aspect of the market (or the society) that depend sensitively, and it is important that crucial assumptions
includes only those features that are needed for the propose at be reasonably realistic. When the results of a theory seem to
hand. flow specifically from a special crucial assumption, then if the
It is necessary to impose assumptions to make a model simpler.
assumption is dubious, the results are suspect.”
But how do we impose “appropriate” assumptions in a model?

EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background Background
Dynamic Optimization in Discrete Time What is Optimization? Dynamic Optimization in Discrete Time What is Optimization?
Dynamic Optimization in Continuous Time EITM: The Importance of Optimization Dynamic Optimization in Continuous Time EITM: The Importance of Optimization
An EITM Example An EITM Example

EITM: The Importance of Optimization EITM: The Importance of Optimization

Causality, Assumptions, and Models Microfoundation of Macroeconomics

As NOBODY TRULY KNOWS how the world works, the In the literature of economics, we assume that people (or
theoretical model we build could be “wrong”. In other words, economic agents) are rational.
the predicted results in the model can be inconsistent with This assumption helps us formulate human behavior in order
what we observed in the real world. to predict outcomes in aggregate markets. This is called the
If this is the case, probably the assumptions we make are too microfoundation of macroeconomics.
sensitive (too strong) to the final results. Microfoundations refers to the microeconomic analysis of the
Removing those assumptions / imposing some more realistic behavior of individual agents such as households or firms that
assumptions would be necessary. underpins a macroeconomic theory. (Barro, 1993)
Therefore, both theoretical modeling (TM) and empirical In this lecture, we study how agents face a dynamic
testing (EI) enhance our understanding of the relationship optimization problem where actions taken in one period can
between x and y . affect the optimization decisions faced in future periods.

EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time What is Optimization? Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time EITM: The Importance of Optimization Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example An EITM Example Profit Maximization

This Presentation Outline

1 Background
In this lecture, we study two methods of dynamic optimization:
What is Optimization?
(1) Discrete-time Optimization - the Bellman equations; and
(2) Continuous-time Optimization - the method of Hamiltonian multiplier. EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete Time
Examples: A Simple Two-period Consumption Model
Discrete-time case: The Bellman Equation
Cake Eating Problem
1 Cake-eating problem
Profit Maximization
2 Profit maximization
3 Dynamic Optimization in Continuous Time
Continuous-time case: The Method of Hamiltonian Multiplier
1 Cake-eating problem Cake Eating Problem Revisited
2 Ramsey Growth Model (see lecture notes!) 4 An EITM Example
EITM: Dynamics in a Money-in-the-Utility (MIU) Model Dynamics in a Money-in-the-Utility Model
(Chari, Kehoe, and McGrattan, Econometrica 2000; Christiano, TM: Theoretical Model
Eichenbaum, and Evans, JPE 2005) EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Two-period Consumption Model Two-period Consumption Model

Two periods in the model

Two periods in the model Period 1: The present; and Period 2: The future
Period 1: The present; and Period 2: The future
The two-period utility function can be written as:
The two-period utility function can be written as:
1
1 U = u (c1 ) + u (c2 ) .
U = u (c1 ) + u (c2 ) . 1+r
1+r
Assuming that the agent has a first-period budget constraint:
We call 1/ (1 + r) as the discount factor, where r is called the Y1 + (1 + r ) A0 = c1 + A1 ,
discount rate (or the degree of impatience).
If an agent is more impatient (r ") 1/ (1 + r) #), then she where Yt = exogenous income at time t, At = assets / debts
would put less weight on the utility of future consumption. that the individual accumulates at time t, and r = an
exogenous interest rate.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Two-period Consumption Model Two-period Consumption Model

The complete model is: Lagrangian Multiplier

The two-period utility function: The system:

1
1 U = u (c1 ) + u (c2 ) .
U = u (c1 ) + u (c2 ) . 1+r
1+r
and
The first-period and second-period budget constraints: Y1 = c1 + A1 , and Y2 + (1 + r ) A1 = c2 .

Y1 = c1 + A1 (1st-period BC), and To maximize the system of equations, we can apply the method of
Lagrangian multiplier to solve the model:
Y2 + (1 + r ) A1 = c2 (2nd-period BC).
1
We assume that the individual does not have any inheritance/debt L = u (c1 )+ u (c2 )+l1 (Y1 c1 A1 )+l2 (Y2 + (1 + r ) A1 c2 ) ,
1+r
in period 1 (i.e., A0 = 0) and does not leave any bequest/debt after
period 2 (i.e., A2 = 0). where l1 and l2 are the Lagrangian multipliers.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Two-period Consumption Model Two-period Consumption Model

We have 5 choice variables: c1 , c2 , A1 , l1 , and l2 . We can solve

for those variables based on the 5 first-order conditions:
From equations (1)–(3), we have:
∂L
= 0 ) u 0 (c1 ) l1 = 0 (1)
∂ c1
∂L 1 l1 = u 0 (c1 ) , (6)
=0) u 0 (c2 ) l2 = 0 (2)
∂ c2 1+r 1
∂L l2 = u 0 (c2 ) , and (7)
= 0 ) l1 + (1 + r ) l2 = 0 (3) 1+r
∂ A1 l1 + (1 + r ) l2 = 0. (8)
∂L
= 0 ) Y 1 = c1 + A 1 (4)
∂ l1
∂L
= 0 ) Y2 + (1 + r ) A1 = c2 . (5)
∂ l2
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Two-period Consumption Model Two-period Consumption Model

Now we can plug (6) and (7) into (8), we have the following What does the Euler equation: u 0 (c1 ) = 1+r 0
tell us?
1+r u (c2 )
equation:

Suppose that an agent gives up $1 consumption today (the

1+r 0
u 0 (c1 ) = u (c2 ) . (9) present), the utility cost to her will be u 0 (c1 ) . In return, she
1+r will get $ (1 + r ) additional consumption tomorrow (the future)
Equation (9) is called the Euler equation. By combining equations so that her utility gain for the tomorrow will be (1 + r ) u 0 (c2 ) .
(4) and (5), we have the lifetime budget constraint: However, since we assume that agents are impatient, the
totally gain from giving up today’s consumption for tomorrow
Y2 c2 1
Y1 + = c1 + . (10) would be 1+r ⇥ (1 + r ) u 0 (c2 ).
1+r 1+r
In equilibrium, the agent will not give up more or less today’s
Finally, given a certain functional form of u (·), we can use consumption for tomorrow at the optimal level only if
equations (9) and (10) to obtain the optimal levels of c1 and c2, u 0 (c1 ) = [(1 + r ) / (1 + r)] u 0 (c2 ) .
(i.e., c1⇤ and c2⇤ ).
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Two-period Consumption Model: An Example Two-period Consumption Model

An Example

Plug eq.(12) into the lifetime budget constraint (eq.(10)), we have:

Let the utility function be the power function, u (c) = a1 c a , where
a 2 (0, 1), and a = 1/2, we have: Y2 1+r
Y1 + = c1 + c1
1+r (1 + r)2
1/2 !
u 0 (c) = c . (11)
Y2 1+r
Y1 + = c1 1 +
We can plug equation (11) into equation (9), we have: 1+r (1 + r)2
! 1✓ ◆
1/2 1 + r 1/2 1+r Y2
c1 = c ) c1⇤ = 1+ Y1 + . (13)
1+r 2 (1 + r)2 1+r
✓ ◆
1+r 2 Now we plug equation (13) into equation (12), we have:
) c2 = c1 . (12)
1+r ✓ ◆ ! 1✓ ◆
⇤ 1+r 1+r Y2
c2 = 1+ Y1 + . (14)
1+r (1 + r)2 1+r

EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Two-period Consumption Model Outline

An Example
1 Background
What is Optimization?
EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete Time
The optimized consumption levels in period 1 and period 2 are: A Simple Two-period Consumption Model
⇣ ⌘ The Bellman Equation
1⇣ ⌘
c1⇤ = 1 + 1+r Y2
Y1 + 1+r Cake Eating Problem
(1+r)2
Profit Maximization
⇣ ⌘⇣ ⌘ 1⇣ ⌘ 3 Dynamic Optimization in Continuous Time
1+r 1+r Y2
c2⇤ = 1+ Y1 + 1+r The Method of Hamiltonian Multiplier
1+r (1+r)2
Cake Eating Problem Revisited
4 An EITM Example
Dynamics in a Money-in-the-Utility Model
TM: Theoretical Model
EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

The Bellman Equation Consumption Dynamics

Let’s consider the following maximization problem:

In the previous section, we can use the method of Lagrangian
" #
multiplier for solving a simple dynamic optimization problem. • ✓ ◆t 1
1
However, such the method can be sometime tedious and max U = Â u (ct ) , (15)
t=1 1 + r
ct
inefficient.
This alternative technique is based on a recursive subject to the following budget constraint:
representation of a maximization problem, which is called the
Bellman equation. At = (1 + r ) At 1 + Yt ct . (16)
The Bellman equation represents a maximization decision
based on the forward (or backward) solution procedure with At is the state variable in each period t, which represents the
the property of time consistency. total amount of resources available to the consumer;
This time consistency property of the optimal solution is also ct is the control variable, where the consumer is choosing to
known as Bellman’s optimality principle. maximize her utility. Note that ct affects the amount of
resources available for the next period, that is, At .
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Consumption Dynamics Consumption Dynamics

Since the consumer would like to maximize her utility from time t
onwards, we define the following value function V1 (A0 ) which
represents the maximized value of the objective function from time
For this technique of dynamic optimization, the maximum t = 1 to the last period of t = T :
value of utility not only depends on the level of consumption T ✓ ◆t 1
1
at time t, but also the resource left for future consumption V1 (A0 ) = max  u (ct ) (17)
t=1 1 + r
c1
(i.e., At ). ✓ ◆ ✓ ◆T 1 !
In other word, given the existing level of asset At 1 , the level 1 1
V1 (A0 ) = max u (c1 ) + u (c2 ) + · · · + u (cT )
of consumption chosen at time t (that is , ct ) will affect the c1 1+r 1+r
level of assets available at time t + 1, (that is, At ). ( ✓ ◆" T ✓ ◆t 2 #)
1 1
V1 (A0 ) = max u (c1 ) +
1+r  u (ct ) (18)
t=2 1 + r
c1

subject to
At = (1 + r ) At 1 + Yt ct . (19)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Consumption Dynamics Consumption Dynamics

The Bellman Equation
From equation (17), we see that V1 (A0 ) is the maximized value of
the objective function at time t = 1 given an initial stock of assets
A0 .
After the maximization in the first period (t = 1) , the consumer Assuming the consumer is maximizing her utility every period, we
repeats the same procedure of utility maximization according to the rewrite the maximization problem recursively. Therefore, we can
objective function in period t = 2, given an initial stock of assets in present the well-known Bellman equation as follows:
period 1:
T ✓ ◆t 2  ✓ ◆
1 1
V2 (A1 ) = max  u (ct ) , (20) Vt (At 1 ) = max u (ct ) + Vt+1 (At ) ,
c2
t=2 1 + r
ct 1+r
subject to equation (19). where At = (1 + r ) At 1 + Yt ct .
By plugging equation (20) into equation (18), we have:
 ✓ ◆
1
V1 (A0 ) = max u (c1 ) + V2 (A1 ) .
c1 1+r

EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Outline Cake Eating Problem

1 Background
What is Optimization?
EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete Time
A Simple Two-period Consumption Model
The Bellman Equation Question
Cake Eating Problem
Profit Maximization Suppose the size of a cake at time t is ⇧t ; and the utility function
1/2
3 Dynamic Optimization in Continuous Time is presented as u (ct ) = 2ct . Given that ⇧0 = 1, ⇧T = 0, and the
The Method of Hamiltonian Multiplier discount rate is r, what is the optimal path of consumption?
Cake Eating Problem Revisited
4 An EITM Example
Dynamics in a Money-in-the-Utility Model
TM: Theoretical Model
EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Cake Eating Problem Cake Eating Problem

The Bellman Equation

Now we can formulate the Bellman equation:

Answer 
1
This optimization problem can be written as: V (⇧t 1 ) = max u (ct ) + V (⇧t ) , (22)
ct 1+r
T ✓ ◆t 1
1 subject to
max
c1 ,c2 ,...,cT
 1+r
u (ct ) , (21)
⇧t = ⇧ t 1 ct . (23)
t=1
1/2
subject to ⇧t = ⇧t 1 ct , for t = 1, 2, . . . , T , and ⇧0 = 1 and Since u (ct ) = 2ct , we can rewrite the Bellman equation as:
⇧T = 1. 
In this case, we see that the choice variable is ct and the state 1/2 1
V (⇧t 1 ) = max 2ct + V (⇧t ) . (24)
variable ⇧t . ct 1+r

EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Cake Eating Problem Outline

The Optimal Path of Cake Consumption
1 Background
What is Optimization?
EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete Time
A Simple Two-period Consumption Model
The Bellman Equation
Cake Eating Problem
Profit Maximization
3 Dynamic Optimization in Continuous Time
The Method of Hamiltonian Multiplier
Cake Eating Problem Revisited
4 An EITM Example
Dynamics in a Money-in-the-Utility Model
TM: Theoretical Model
EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background A Simple Two-period Consumption Model Background A Simple Two-period Consumption Model
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Bellman Equation
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem
An EITM Example Profit Maximization An EITM Example Profit Maximization

Profit Maximization Profit Maximization

Assuming that a representative firm maximizes the present value of
all future profits by choosing the levels of investment It and labor In this case,
Lt for t = 1, 2, . . . , T . Therefore, we have the following
the choice variable is: It and Lt , and
maximization problem:
⇣ ⌘t the state variable is Kt .
1
max ÂT
t=1 1+r pt According to the above system, we can formulate the following
I1 ,I2 ,...IT ,L1 ,L2 ,...LT
⇣ ⌘t Bellman equation:
1
max ÂT
t=1 1+r (F (Kt , Lt ) wt Lt I ) , 
I1 ,I2 ,...IT ,L1 ,L2 ,...LT
1
Vt (Kt ) = max F (Kt , Lt ) wt Lt It + Vt+1 (Kt+1 ) , (25)
subject to It ,Lt 1+r
Kt+1 = Kt d Kt + I t
where
for t = 1, 2, . . . , T , and K1 and KT are given, pt is the level of profit
Kt+1 = (1 d ) Kt + I t . (26)
at time t, Kt is the stock of capital at time t, F (Kt , Lt ) is a
production function, wt is the wage rate, r and d is the interest
rate and depreciate rate in the market, respectively.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background A Simple Two-period Consumption Model Background
Dynamic Optimization in Discrete Time The Bellman Equation Dynamic Optimization in Discrete Time The Method of Hamiltonian Multiplier
Dynamic Optimization in Continuous Time Cake Eating Problem Dynamic Optimization in Continuous Time Cake Eating Problem Revisited
An EITM Example Profit Maximization An EITM Example

Profit Maximization Outline

1 Background
What is Optimization?
In this optimization problem, we have two important conditions: EITM: The Importance of Optimization
∂ F (Kt ,Lt )
1
∂ Lt = wt . 2 Dynamic Optimization in Discrete Time
This result suggests that the optimal amount of labor satisfies
A Simple Two-period Consumption Model
the condition where marginal product of labor (MPL) equals The Bellman Equation
the real wage rate in each period, that is MPLt = wt . Cake Eating Problem
∂F Profit Maximization
2
∂ Kt = r +d.
3 Dynamic Optimization in Continuous Time
This result suggests that the firm must choose a level of The Method of Hamiltonian Multiplier
investment such that the marginal product of capital (MPK) Cake Eating Problem Revisited
equals the sum of interest rate and depreciation rate in each
4 An EITM Example
period, that is MPKt = r + d .
Dynamics in a Money-in-the-Utility Model
TM: Theoretical Model
EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background Background
Dynamic Optimization in Discrete Time The Method of Hamiltonian Multiplier Dynamic Optimization in Discrete Time The Method of Hamiltonian Multiplier
Dynamic Optimization in Continuous Time Cake Eating Problem Revisited Dynamic Optimization in Continuous Time Cake Eating Problem Revisited
An EITM Example An EITM Example

Dynamic Optimization in Continuous Time Dynamic Optimization in Continuous Time

A general continuous-time maximization problem can be written as
follows: Z
In the previous sections, we study the dynamic optimization T
rt
based the discrete-time dynamic model, where the change in max e f (xt , At ) dt, (27)
xt 0
time t is positive and finite (for example, we assume that
subject to the constraint:
t = 1 for all t 0, such that t follows the sequence of
{0, 1, 2, 3, . . . } . Ȧt = g (xt , At ) , (28)
In the section, we consider the method of dynamic
optimization in continuous time, where t ! 0. Therefore, we where Ȧt is a time derivative of At defined as dAt /dt, and r is the
assume that agents make optimizing choices at every instant discount rate in the model, which is equivalent to the r we use in the
in continuous time. discrete time models.
The method is called the technique of Hamiltonian If r = 0, then e rt = 1. It implies that the agent does not
multiplier. discount the activities in the future. On the other hand, if r ",
then e rt # . It implies that the agent becomes more impatient
and discounts the value (or utility) of future activities more.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background Background
Dynamic Optimization in Discrete Time The Method of Hamiltonian Multiplier Dynamic Optimization in Discrete Time The Method of Hamiltonian Multiplier
Dynamic Optimization in Continuous Time Cake Eating Problem Revisited Dynamic Optimization in Continuous Time Cake Eating Problem Revisited
An EITM Example An EITM Example

The Method of Hamiltonian Multiplier Outline

We set up the following Hamiltonian function: 1 Background
h i What is Optimization?
Ht = e rt f (xt , At ) + lt Ȧt , (29) EITM: The Importance of Optimization
where lt = µt e rt is called the Hamiltonian multiplier.
2 Dynamic Optimization in Discrete Time
The three conditions for a solution: A Simple Two-period Consumption Model
1 The FOC with respect to the control variable (x ):
The Bellman Equation
t
Cake Eating Problem
∂ Ht Profit Maximization
= 0; (30)
∂ xt 3 Dynamic Optimization in Continuous Time
2 The negative derivative of the Hamiltonian function w.r.t. At : The Method of Hamiltonian Multiplier
∂ Ht d (lt e rt ) Cake Eating Problem Revisited
= ; (31)
∂ At dt 4 An EITM Example
3 The transversality condition: Dynamics in a Money-in-the-Utility Model
rt
TM: Theoretical Model
lim lt e At = 0. (32) EI: Empirical Implications
t!•
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background Background
Dynamic Optimization in Discrete Time The Method of Hamiltonian Multiplier Dynamic Optimization in Discrete Time The Method of Hamiltonian Multiplier
Dynamic Optimization in Continuous Time Cake Eating Problem Revisited Dynamic Optimization in Continuous Time Cake Eating Problem Revisited
An EITM Example An EITM Example

Cake Eating Problem Cake Eating Problem

Let us consider the same cake eating problem in continuous time: We set up the Hamiltonian as follows:
Z T ⇣ ⌘
max e rt
u (ct ) dt, (33) Ht = e rt u (ct ) + lt ⇧˙ t , (35)
ct 0

subject to where lt is the co-state variable. Now, we can plug equation (34)
⇧˙ t = ct , (34) into equation (35), we have the final Hamiltonian equation:
rt
and ⇧0 and ⇧1 are given. Again, the choice variable is ct , and the Ht = e (u (ct ) lt ct ) (36)
state variable is ⇧t .

EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background Background
Dynamics in a Money-in-the-Utility Model
Dynamic Optimization in Discrete Time The Method of Hamiltonian Multiplier Dynamic Optimization in Discrete Time
TM: Theoretical Model
Dynamic Optimization in Continuous Time Cake Eating Problem Revisited Dynamic Optimization in Continuous Time
EI: Empirical Implications
An EITM Example An EITM Example

Cake Eating Problem Outline

We obtain the following conditions: 1 Background
1 The FOC w.r.t. (with respect to) c :
What is Optimization?
t
EITM: The Importance of Optimization
∂ Ht rt 2 Dynamic Optimization in Discrete Time
= e u 0 (ct ) lt = 0
∂ ct A Simple Two-period Consumption Model
) u 0 (ct ) = lt ; (37) The Bellman Equation
Cake Eating Problem
2 The negative derivative w.r.t. ⇧t equals the time derivative of
Profit Maximization
lt e rt :
∂ Ht d (lt e rt ) 3 Dynamic Optimization in Continuous Time
= ; (38) The Method of Hamiltonian Multiplier
∂ ⇧t dt
Cake Eating Problem Revisited
and
4 An EITM Example
3 The transversality condition:
Dynamics in a Money-in-the-Utility Model
rt TM: Theoretical Model
lim lt e ⇧t = 0. (39)
t!•
EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background Background
Dynamics in a Money-in-the-Utility Model Dynamics in a Money-in-the-Utility Model
Dynamic Optimization in Discrete Time Dynamic Optimization in Discrete Time
TM: Theoretical Model TM: Theoretical Model
Dynamic Optimization in Continuous Time Dynamic Optimization in Continuous Time
EI: Empirical Implications EI: Empirical Implications
An EITM Example An EITM Example

An MIU Model: An Introduction Outline

1 Background
Why do people want to hold money? What is Optimization?
Holding cash does not generate any returns! EITM: The Importance of Optimization
Two possible arguments: 2 Dynamic Optimization in Discrete Time
1 People feel good and safe if they hold some cash in their A Simple Two-period Consumption Model
pocket (Sidrauski, 1967) The Bellman Equation
2 Money can provide transaction services (transaction purpose!) Cake Eating Problem
(Baumol, 1952, Tobin, 1956) Profit Maximization
Here we study a basic neoclassical model where agents’ utility 3 Dynamic Optimization in Continuous Time
depends directly on their consumption of goods and their The Method of Hamiltonian Multiplier
holdings of money (money demand). Cake Eating Problem Revisited
Assumption: Money yields direct utility. 4 An EITM Example
This is called the Money-in-the-Utility model Dynamics in a Money-in-the-Utility Model
(Chari, Kehoe, and McGrattan, Econometrica 2000; Christiano, TM: Theoretical Model
Eichenbaum, and Evans, JPE 2005) EI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background Background
Dynamics in a Money-in-the-Utility Model Dynamics in a Money-in-the-Utility Model
Dynamic Optimization in Discrete Time Dynamic Optimization in Discrete Time
TM: Theoretical Model TM: Theoretical Model
Dynamic Optimization in Continuous Time Dynamic Optimization in Continuous Time
EI: Empirical Implications EI: Empirical Implications
An EITM Example An EITM Example

An MIU Model An MIU Model

Consider the following Money-in-the-Utility Model:
• ✓ ◆t
1
W=Â u (ct , mt ) ,
t=0 1 + r

where mt = Mt / (Pt Nt ) = real money holding per capita in an B Bt 1
Yt =Ct + [Kt (1 d ) Kt 1] tt Nt + (1 + it 1) +
economy. The budget constraint in the whole economy is: Pt Pt

Mt Mt 1
GDPt =Consumptiont + Investmentt + GovtSpendingt + .
Pt Pt
NewNationalDebtt + NewMoneyt .
We can translate as: where Yt = aggregate output (GDP), tt Nt = total lump-sum
 transfers (positive) or taxes (negative), i =interest rate, and Nt =
B Bt 1
Yt =Ct + [Kt (1 d ) Kt 1] tt Nt + (1 + it 1) + population.
Pt Pt

Mt Mt 1
.
Pt Pt
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background Background
Dynamics in a Money-in-the-Utility Model Dynamics in a Money-in-the-Utility Model
Dynamic Optimization in Discrete Time Dynamic Optimization in Discrete Time
TM: Theoretical Model TM: Theoretical Model
Dynamic Optimization in Continuous Time Dynamic Optimization in Continuous Time
EI: Empirical Implications EI: Empirical Implications
An EITM Example An EITM Example

An MIU Model An MIU Model

Finally, the complete system is:
• ✓ ◆t
1 Finally, the complete system is:
W=Â u (ct , mt ) ,
t=0 1 + r • ✓ ◆t
1
and the budget constraint can be presented as follows: W=Â u (ct , mt ) ,
t=0 1 + r
B t 1 Mt 1 Mt Bt
Yt +tt Nt +(1 d ) Kt 1 +(1 + it 1 ) + = Ct +Kt + + . and the budget constraint (per capita) is:
Pt Pt Pt Pt
✓ ◆
We also define the production function as Yt = F (Kt 1 , Nt ). kt 1 1 d (1 + it 1 ) bt 1 + mt
The per capita income is: f +tt + kt 1 + = ct +kt +mt +bt ,
1+n 1+n (1 + pt ) (1 + n)
✓ ◆ ✓ ◆
1 K t 1 Nt Kt 1
yt = F (Kt 1 , Nt ) = F , =F ,1 where pt is the inflation rate, such that, Pt = (1 + pt ) Pt 1,
Nt Nt Nt (1 + n) Nt 1
✓ ◆ bt = Bt / (Pt Nt ) , and mt = Mt / (Pt Nt ) .
kt 1
=f , where n = population growth rate.
1+n
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background Background
Dynamics in a Money-in-the-Utility Model Dynamics in a Money-in-the-Utility Model
Dynamic Optimization in Discrete Time Dynamic Optimization in Discrete Time
TM: Theoretical Model TM: Theoretical Model
Dynamic Optimization in Continuous Time Dynamic Optimization in Continuous Time
EI: Empirical Implications EI: Empirical Implications
An EITM Example An EITM Example

An MIU Model An MIU Model

Finally we have the following equilibrium:
um (ct , mt ) i
We can now formulate the model as the Bellman equation: = .
uc (ct , mt ) 1+i

1 Now assume that the utility function is of the constant elasticity of
V (Wt ) = max u (ct , mt ) + V (Wt+1 ) ,
1+r substitution (CES) form:
h i1/(1 b)
subject to u (ct , mt ) = act1 b + (1 a) mt1 b .
✓ ◆ ✓ ◆
kt 1 d (1 + it ) bt + mt The final solution based on the CES utility function is:
Wt+1 = f + tt+1 + kt + ✓ ◆ 1/b ✓ ◆ 1/b
1+n 1+n (1 + pt+1 ) (1 + n) a i
mt = ct , or
= kt+1 + ct+1 + mt+1 + bt+1 . 1 a 1+i
✓ ◆
1 1 a 1
ln mt = ln + ln ct ln g,
b a b
where g = i/ (1 + i) = the opportunity cost of holding money.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction

Background Background
Dynamics in a Money-in-the-Utility Model Dynamics in a Money-in-the-Utility Model
Dynamic Optimization in Discrete Time Dynamic Optimization in Discrete Time
TM: Theoretical Model TM: Theoretical Model
Dynamic Optimization in Continuous Time Dynamic Optimization in Continuous Time
EI: Empirical Implications EI: Empirical Implications
An EITM Example An EITM Example

Outline Empirical Estimation of Money Demand

1 Background According to the theoretical solution:
What is Optimization? ✓ ◆
EITM: The Importance of Optimization 1 1 a 1
ln mt = ln + ln ct ln g, (40)
2 Dynamic Optimization in Discrete Time b a b
A Simple Two-period Consumption Model where g = i/ (1 + i) which can be called the opportunity cost of
The Bellman Equation holding money.
Cake Eating Problem Therefore, we empirical model can be written as:
Profit Maximization
3 Dynamic Optimization in Continuous Time ln mt = a0 + a1 ln ct + a2 ln g + et . (41)
The Method of Hamiltonian Multiplier
Cake Eating Problem Revisited According to equation (41), we expect that he coefficient on
4 An EITM Example ln ct is a1 = 1. It implies that consumption (income) elasticity
Dynamics in a Money-in-the-Utility Model of money demand is equal to 1.
TM: Theoretical Model The coefficient on ln 1+i i
is a2 = 1/b. For simplicity, we call
EI: Empirical Implications it as the interest elasticity of money demand.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
 Background Background
Dynamics in a Money-in-the-Utility Model Dynamics in a Money-in-the-Utility Model
Dynamic Optimization in Discrete Time Dynamic Optimization in Discrete Time
TM: Theoretical Model TM: Theoretical Model
Dynamic Optimization in Continuous Time Dynamic Optimization in Continuous Time
EI: Empirical Implications EI: Empirical Implications
An EITM Example An EITM Example

Empirical Estimation of Money Demand

The empirical result of the money demand function for the United
States based on quarterly data from the period of 1984:1 – 2007:2.

Thank you!

Questions?

Source: Walsh (2010: 51)

ln m = the log of real money balance (M1); ct = real people

consumption expenditures; and ln g = ln 1+i
i
= opportunity cost of
holding money = return on M1 minus the 3-month T-Bill rate.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction