Practice Problems : Dynamic Programming (Macro II, 2024)

For questions (1)-(4): (A) State the control and state variables for the problem and write the
associated Bellman, (B) Please solve the problem using appropriate solution technique.

1. Assume that there is a cake whose size at time t is denoted by 𝑤𝑡 and a muncher who wants to
eat it in T periods (𝑤𝑇 = 0). The eater has a psychological discount factor 0 < 𝛽 < 1 and a
static logarithmic utility function. What is the optimal eating strategy?

2. Use an appropriate method to solve the following problem

𝑀𝑎𝑥{𝑐𝑡 }∞
0
∑∞ 𝑡
𝑡=0 𝛽 𝑙𝑛 (𝑐𝑡 )
Subject to
𝑘𝑡+1 = 𝐴𝑘𝑡 𝛼 − 𝑐𝑡 − 𝛿𝑘𝑡 , and given the initial capital stock 𝑘(0), 𝛿 > 0 , 0 < 𝛼 < 1

(𝑐𝑡 is consumption in period t, 𝑘𝑡 is the capital stock in period t, 𝛿 (a constant) is the depreciation
rate of the capital, 𝛽 is the discount rate)

3. Use an appropriate method to solve the following problem

𝑀𝑎𝑥{𝑐𝑡 }∞
0
∑∞ 𝑡
𝑡=0 𝛽 𝑢 (𝑐𝑡 )
Subject to 𝑘𝑡+1 = 𝑓(𝑘𝑡−1 ) − 𝑐𝑡 − 𝛿𝑘𝑡−1

That is, investment in period 𝑡 will become productive only in period 𝑡 +2.
(𝑐𝑡 is consumption in period t, 𝑘𝑡 is the capital stock in period t, 𝛿 (a constant) is the depreciation
rate of the capital, 𝛽 is the discount rate)

4. Consider a competitive firm producing a single output with “capital” (𝑘) according to the
production function 𝑓(𝑘). The firm’s net revenue in period t is 𝜋𝑡 = 𝑝𝑡 𝑓(𝑘𝑡 ) − 𝑐(𝐼𝑡 ) where 𝑝 is
the price of output and 𝑐(𝐼𝑡 ) is the cost of investment. So the firms’ objective is to maximize the
present value of its profits, its optimization problem is.

𝑀𝑎𝑥{𝐼𝑡 }∞
0
∑∞ 𝑡
𝑡=0 𝛽 [ 𝑝𝑡 𝑓(𝑘𝑡 ) − 𝑐(𝐼𝑡 )]

Subject to 𝑘𝑡+1 = 𝐼𝑡 − 𝛿𝑘𝑡 , and given the initial capital stock 𝑘(0), 𝑟, 𝛿 > 0

(𝐼𝑡 is the firm’s investment in period t, 𝑘𝑡 is the capital stock in period t, 𝛿 (a constant) is the
depreciation rate of the capital, 𝛽 is the discount rate)

1
5. Consider the following dynamic programming problem with uncertainty. The social planner
maximizes,
∞

max 𝐸0 ∑ 𝛽 𝑡 [log(𝑐𝑡 ) + 𝜓 𝑙𝑜𝑔(1 − 𝑙𝑡 )]

Subject to
𝑘𝑡+1 = (1 − 𝛿) 𝑘𝑡 + 𝑒 𝑧𝑡 𝑘𝑡 𝛼 𝑙𝑡 1−𝛼 − 𝑐𝑡
𝑧𝑡 = 𝜑𝑧 𝑧𝑡−1 + 𝜀𝑧,𝑡 , −1 < 𝜑𝑧 < 1

𝑐𝑡 ≥ 0, 0 < 𝑙𝑡 < 1

where 𝑙 is the total number of hours the household works. As in class, we normalize total time
household has available in a period to 1. 𝜓 determine how painful it is for the representative
household to work. Output is given by 𝑦𝑡 = 𝑒 𝑧𝑡 𝑘𝑡 𝛼 𝑙𝑡 1−𝛼 , where 𝑧𝑡 is the state of technology
which fluctuates following a mean zero AR(1) process.

𝑧𝑡 = 𝜑𝑧 𝑧𝑡−1 + 𝜀𝑧,𝑡 , −1 < 𝜑𝑧 < 1

(a) Write the Bellman equation and the first order conditions

(b) Write the optimality conditions.

(c) Provide intuitions for inter-temporal decision for consumption.

(d) Provide intuitions for intra-temporal decision between consumption and leisure.

6. Consider the following dynamic programming problem with uncertainty. The representative
individual seeks to maximize the utility function,
∞

max 𝐸0 ∑ 𝛽 𝑡 ln(𝐶𝑡 ) + b ln(1 − 𝐿𝑡 )

Subject to 𝐴𝑡+1 =𝑅𝑡 𝑧𝑡+1 , 𝑡 = 0, … … ..

𝐴𝑡+1 is the state of technology which depends on the investment decision taken last period, 𝑅𝑡
and a random variable 𝑧𝑡+1 . 𝑧𝑡+1 is assumed to follow a mean zero AR(1) process.

𝑙𝑛𝑧𝑡+1 =𝜑 ln 𝑧𝑡 + 𝜀𝑡+1 , -1<𝜑<1

Note that, 𝐴𝑡+1=(𝑌𝑡 −𝐶𝑡 )𝑧𝑡+1 = (𝐴𝛼𝑡 𝐿𝑡 − 𝐶𝑡 )𝑧𝑡+1

2
The representative individual is endowed with 1 unit of time out of which 𝐿𝑡 is spent working
and (1 − 𝐿𝑡 ) in leisure.

(a) Find the optimality conditions and provide their intuitions. [Hint: inter-temporal decision for
consumption and intra-temporal decision between consumption and leisure]
𝑅𝑡
(d) Define 𝑠𝑡 = . Using the optimality condition solve for 𝑠𝑡 in terms of parameters of the
𝑌𝑡
model. [Hint: impose 𝑠𝑡+1 = 𝑠𝑡 ]

7. Consider the following dynamic programming problem with uncertainty. The Social planner
maximizes,
∞ 1
𝑡 1−𝛾
max 𝐸0 ∑ 𝛽 [𝑏𝑐𝑡 + (1 − 𝑏)(1 − 𝑙𝑡 )1−𝛾 ]1−𝛾
𝑐𝑡 ,𝑙𝑡
𝑡=0

Subject to
𝑘𝑡+1 = (1 − 𝛿) 𝑘𝑡 + 𝑒 𝑧𝑡 𝑘𝑡 𝛼 𝑙𝑡 1−𝛼 − 𝑐𝑡
𝑧𝑡 = 𝜑𝑧 𝑧𝑡−1 + 𝜀𝑧,𝑡 , −1 < 𝜑𝑧 < 1

𝑐𝑡 ≥ 0, 0 < 𝑙𝑡 < 1

where 𝑙 is the total number of hours the household works. As in class, we normalize total time
household has available in a period to 1. (1 − 𝑏) determine how painful it is for the
representative household to work. Output is given by 𝑦𝑡 = 𝑒 𝑧𝑡 𝑘𝑡 𝛼 𝑙𝑡 1−𝛼 , where 𝑧𝑡 is the state of
technology which fluctuates following a mean zero AR(1) process.

𝑧𝑡 = 𝜑𝑧 𝑧𝑡−1 + 𝜀𝑧,𝑡 , −1 < 𝜑𝑧 < 1

(a) Set up the social planner problem. Write the relevant Bellman equation and write the first
order conditions

(b) Derive the optimality conditions.

(c) Provide intuitions for inter-temporal decision and for intra-temporal decisions.

(d) For a given value of 𝑐𝑡 , how does an increase in b affects the labor supply curves.

(e) Calibrate the values of the parameters 𝑏 and 𝛾.

3
8. Consider the following dynamic programming problem with uncertainty. The representative
household chooses consumption (𝑐) and leisure (𝑙),
∞
𝑡
𝑙𝑡 1+𝜂
max 𝐸0 ∑ 𝛽 (𝑙𝑛𝑐𝑡 − )
𝑐𝑡 ,𝑙𝑡 1+𝜂
𝑡=0

Subject to
𝑘𝑡+1 = (1 − 𝛿(𝑢𝑡 ))𝑘𝑡 + 𝑦𝑡 −𝑐𝑡

Where 𝑢𝑡 is the degree of capital utilization and it is chosen by the household.

Competitive firms produce output using capital services 𝑢𝑡 𝑘𝑡 and labor 𝑙𝑡 . Output is given by
𝑦𝑡 = 𝐴𝑡 (𝑢𝑡 𝑘𝑡 )𝛼 𝑙𝑡 1−𝛼 , where 𝐴𝑡 is the state of technology which fluctuates following a mean
zero AR(1) process.

ln 𝐴𝑡 = 𝐴̅ + 𝑔𝑡 + 𝐴̃𝑡 , 𝐴̃𝑡 captures exogenous shocks to technology where it is a stochastic

Households own the capital stock, invest and decide on the degree of utilization. Utilization is
costly and causes capital to depreciate more quickly, as indicated by the depreciation cost
function 𝛿(𝑢𝑡 ), where 𝛿(1) = 𝛿 in steady state. Firms choose capital to rent from the
households, understanding that they will be able to utilize it to a certain degree. You can think of
the firm choosing their demand for labor, 𝑙𝑡 and their demand for capital services 𝑢𝑡 𝑘𝑡 .

(a) Derive the optimality conditions. Briefly provide intuitions.

(b) Write down the firm’s problem and derive the optimality conditions.

(c) By linearizing the equilibrium condition governing the optimal degree of utilization, show
that
𝛼
𝑦̃𝑡 = 𝐴̃𝑡 + 𝛼 𝑘̃𝑡 + (1 − 𝛼) 𝑙̃𝑡 + ( 𝐴̃ − (1 − 𝛼) 𝑘̃𝑡 + (1 − 𝛼) 𝑙̃𝑡 )
(1 − 𝛼 + 𝜉) 𝑡

𝛿 ′′
Where 𝜉 = and 𝛿 ′ and 𝛿 ′′ refer to the first and second derivatives of the 𝛿 function with
𝛿′
respect to 𝑢𝑡 in steady state. Following the class notation, the tilde denotes percentage deviation
from steady state.

(d) Discuss how the addition of variable capital utilization helps the RBC model explain the
business cycle facts in the data? Please explain, why in the limit 𝜉 = ∞ and 𝜉 = 0 correspond to
the cases of no variable capital utilization and full variable capital utilization.

4
(e) Briefly explain how you would solve this model using a linearization-based method and how
would you produce impulse response functions for the effects of a temporary one percent shock
to 𝐴̃𝑡 .

9. Optimal Growth with keeping-up-with-the-Joneses utility.

Consider the following dynamic programming problem with uncertainty.

max 𝐸0 ∑ 𝛽 𝑡 𝑢(𝑐𝑡 , 𝑐̅𝑡 )

Subject to
𝑘𝑡+1 = 𝑘𝑡 𝛼 + (1 − 𝛿)𝑘𝑡 − 𝑐𝑡 , 

Preferences: the instantaneous household utility function over, individual period consumption,
𝑐𝑡 and average period consumption, 𝑐̅𝑡  is given by:

1−𝛾
𝑐
( 𝑡⁄ 𝜃 )
𝑐̅𝑡
𝑢(𝑐𝑡 , 𝑐̅𝑡 ) = 
1−𝛾

𝜃 ∈ [0,1) is the weight or importance an individual gives to average consumption of the society.
Also assume, 𝛾 ∈ (1, ∞). The discount factor is 𝛽2 (01) Individuals in this society are only
happy if they consume more than some average level of consumption. There is a constant returns
to scale technology over capital, 𝑦𝑡 = 𝑘𝑡 𝛼 . Capital depreciates at the rate 𝛿 < 1Household’s
initial capital stock is 𝑘0 is given.

(a) Write and solve the Planner’s problem for this economy. {Hint: As the social planner
cares for everyone, assume that he/she set 𝑐𝑡 = 𝑐̅𝑡 .}
(b) Now consider the decentralized economy. Write and solve the problems faced by the
representative household. {Hint: The representative household takes 𝑐̅𝑡 as given.}
(c) Compare the planner’s solution with the competitive equilibrium.

5
6