research article a general setting and solution of bellman...

Research ArticleA General Setting and Solution ofBellman Equation in Monetary Theory

Xiaoli Gan and Wanbo Lu

School of Statistics, Southwestern University of Finance and Economics, Chengdu 610074, China

Correspondence should be addressed to Wanbo Lu; [email protected]

Received 28 September 2014; Revised 22 November 2014; Accepted 22 November 2014; Published 21 December 2014

Academic Editor: Zhihua Zhang

Copyright © 2014 X. Gan and W. Lu. This is an open access article distributed under the Creative Commons Attribution License,which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

As an important tool in theoretical economics, Bellman equation is very powerful in solving optimization problems of discrete timeand is frequently used in monetary theory. Because there is not a general method to solve this problem in monetary theory, it ishard to grasp the setting and solution of Bellman equation and easy to reach wrong conclusions. In this paper, we discuss the rulesand problems that should be paid attention to when incorporating money into general equilibrium models. A general setting andsolution of Bellman equation in monetary theory are provided. The proposed method is clear, is easy to grasp, is generalized, andalways leads to the correct results.

1. Introduction

In recent years, many economists applied business cycleapproaches to macroeconomic modeling so that monetaryfactors could be modeled into dynamic general equilibriummodels. As an important method of monetary economicsmodeling, infinite horizon representative-agent models pro-vide a close link between theory and practice; its researchframework can guide practice behavior and be tested byactual data. This method can link monetary economics andother popularmodels for studying business cycle phenomenaclosely.There are three basicmonetary economics approachesintroducing money to economic general equilibrium modelsin the infinite horizon representative-agent framework. First,it is assumed that utility could be yielded by money directlyso that the money variable has been incorporated into utilityfunction of the representative-agent models [1]. Second,assuming asset exchanges are costly, transaction costs of someform give rise to amoney demand [2, 3]. Clower [4] considersthat money is used for some types of transactions. Brock [5],McCallum and Goodfriend [6], and Croushore [7] assumethat time and money can be combined to yield transactionservices. Third, money is treated as an asset that can be usedto transfer resources between generations [8].

Dynamic optimization is the main involved issue duringthemodeling process. It is represented and solved by Bellmanequation method, namely, the value function method. Themethod will obtain a forward-looking household’s path tomaximize lifetime utility through the optimal behavior andfurther relevant conclusions.The setting of Bellman equationis the first and crucial step to solve dynamic programmingproblems. It is hard to grasp the setting and solution ofBellman equation and easy to reach wrong conclusions sincethere is not a general method to set Bellman equation or thesettings of Bellman equation are excessively flexible. Walsh[9] used Bellman equation to set and solve dynamic generalequilibrium models of money. However he does not givea general law of Bellman equation setting. The setting andsolution of the equation in his book are ambiguous and notclear. To the best of the authors’ knowledge, there have beenno studies of uniform setting of Bellman equation.

In this paper, we provide a set of general setting andsolution methods for Bellman equation with multipliers. Itis very clear and easy to grasp. The most important thing isthat the proposed method can always lead to correct results.We apply our method to monetary general equilibriummodels that are in the framework of the first two basicmonetary economic approaches which incorporate money

Hindawi Publishing CorporationJournal of Applied MathematicsVolume 2014, Article ID 495089, 9 pageshttp://dx.doi.org/10.1155/2014/495089

2 Journal of Applied Mathematics

into economic general equilibrium model and make someextensions. At the same time, we compare our method withother current methods of setting and solution for Bellmanequation to display the clarity and correctness of the proposedmethod. It is assumed that all the relevant assumptions ofapplying Bellman equation are satisfied. This is to ensure thefeasibility of analysis and solution. For the convenience ofdiscussion and suitable length of the paper, wemainly discussthe certainty linear programming problems. The Bellmanequation’s setting and solution of uncertainty problems aresimilar to those with certainty problems essentially.

2. A New Method of Setting and Solution ofBellman Equation

First of all, we provide the theoretical details of the newgeneral method and steps when applying Bellman equationto solve problems. It hasmany advantages.The relationship ofthe results is very clear through the connection ofmultipliers.There are no tedious expansions during the derivations. Itdiffers from the expansion method because it does not needto consider which control variable should be replaced. Thetechnical details about the equivalence of Bellman equationsand dynamic programming problems and the solvability ofset problems can be found in [10]. Actually, it is easy toreach wrong conclusions using other settings and solutionsof Bellman equation with multipliers, or it has to find someparticular skills to get the right results. But our methoddoes not meet the problem. The proposed method can bedescribed as the following five steps.

Step 1. List the expression of target problem.Consider the dynamic programming problem as

𝐻 = max∞

∑

𝑡=0

𝛽𝑡

ℎ (𝑥1𝑡, 𝑥2𝑡, . . . , 𝑥

𝑝𝑡) (1)

s.t.{{{

{{{

{

𝑔1(𝑥1𝑡, 𝑥2𝑡, . . . , 𝑥

𝑝𝑡, 𝑥𝑝+1,𝑡

, . . . , 𝑥𝑞𝑡) = 0, 𝑝 ≤ 𝑞,

.

.

.

𝑔𝑑(𝑥1𝑡, 𝑥2𝑡, . . . , 𝑥


, . . . , 𝑥𝑞𝑡) = 0, 𝑝 ≤ 𝑞,

(2)

where the objective function (1) represents the maximumsum of present value of a forward-looking behavioral agent’severy period objective function ℎ(𝑥

1𝑡, 𝑥2𝑡, . . . , 𝑥

𝑝𝑡). In mon-

etary theory, ℎ(⋅) usually represents the utility function ofa family and 𝛽 is a subjective rate of discount. Equation(2) is constraints; 𝑔

1(⋅), . . . , 𝑔

𝑑(⋅) are dynamic constraint

functions and usually are intertemporal budget constraints.Let control variables {𝑥

(𝑐)

1𝑡, 𝑥(𝑐)

2𝑡, . . . , 𝑥

(𝑐)

𝑚𝑡} ∈ {𝑥

1𝑡, 𝑥2𝑡, . . . , 𝑥

𝑝𝑡,

𝑥𝑝+1,𝑡

, . . . , 𝑥𝑞𝑡}; the remaining variables {𝑥

1𝑡, 𝑥2𝑡, . . . , 𝑥

𝑝𝑡,

𝑥𝑝+1,𝑡

, . . . , 𝑥𝑞𝑡} − {𝑥

(𝑐)

1𝑡, 𝑥(𝑐)

2𝑡, . . . , 𝑥

(𝑐)

𝑚𝑡} = {𝑥

(𝑠)

1𝑡, 𝑥(𝑠)

2𝑡, . . . , 𝑥

(𝑠)

𝑛𝑡} are

state variables.

Step 2. Set up Bellman equation with multipliers to expressdynamic optimization problem in Step 1:

𝑉(𝑥(𝑠)

1𝑡, 𝑥(𝑠)

2𝑡, . . . , 𝑥

(𝑠)

𝑛𝑡)

= max {ℎ (𝑥1𝑡, 𝑥2𝑡, . . . , 𝑥

𝑝𝑡)

+ 𝛽𝑉 (𝑥(𝑠)

1,𝑡+1, 𝑥(𝑠)

2,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1)}

+ 𝜆1𝑡𝑔1(𝑥1𝑡, . . . , 𝑥


, . . . , 𝑥𝑞𝑡)

+ ⋅ ⋅ ⋅ + 𝜆𝑑𝑡

𝑔𝑑(𝑥1𝑡, . . . , 𝑥


, . . . , 𝑥𝑞𝑡) ,

(3)

where 𝑉(⋅) is the value function and 𝜆𝑖𝑡is the multiplier of

the 𝑖th constraint 𝑔𝑖(⋅), 𝑖 = 1, 2, . . . , 𝑑.

Step 3. Compute the partial derivatives of all control variableson the right side of the equation at Step 2 to derive first-orderconditions:

𝜕𝑉 (𝑥(𝑠)

1𝑡, 𝑥(𝑠)

2𝑡, . . . , 𝑥

(𝑠)

𝑛𝑡)

𝜕𝑥(𝑐)

𝑗𝑡

=

𝜕ℎ (𝑥1𝑡, 𝑥2𝑡, . . . , 𝑥

𝑝𝑡)

𝜕𝑥(𝑐)

𝑗𝑡

+ 𝛽[

[

𝜕𝑉 (𝑥(𝑠)

1,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1)

𝜕𝑥(𝑠)

1,𝑡+1

⋅

𝜕𝑥(𝑠)

1,𝑡+1

𝜕𝑥(𝑐)

𝑗𝑡

+ ⋅ ⋅ ⋅ +

𝜕𝑉 (𝑥(𝑠)

1,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1)

𝜕𝑥(𝑠)

𝑛,𝑡+1

⋅

𝜕𝑥(𝑠)

𝑛,𝑡+1

𝜕𝑥(𝑐)

𝑗𝑡

]

]

+ 𝜆1𝑡

𝜕𝑔1(𝑥1𝑡, . . . , 𝑥

𝑞𝑡)

𝜕𝑥(𝑐)

𝑗𝑡

+ 𝜆𝑑𝑡

𝜕𝑔𝑑(𝑥1𝑡, . . . , 𝑥

𝑞𝑡)

𝜕𝑥(𝑐)

𝑗𝑡

= 0,

𝑗 = 1, 2, . . . , 𝑚.

(4)

During the derivation, it should be taken into accountthat next period state variables can be represented by othercontrol variables according to the constraints, that is, toexpand 𝑥

(𝑠)

1,𝑡+1, 𝑥(𝑠)

2,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1. Because the constraints are

dynamic and are usually intertemporal budget constraints,those control variables 𝑥

(𝑠)

1,𝑡+1, 𝑥(𝑠)

2,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1at time 𝑡 +

1 can often be represented as expressions of the previ-ous period control variables. Attention should be paid tocompute the derivatives of control variables hiding in𝑉(𝑥(𝑠)

1,𝑡+1, 𝑥(𝑠)

2,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1). But there is no need to expand

𝑥(𝑠)

1,𝑡+1, 𝑥(𝑠)

2,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1after themultipliers during the deriva-

tion. Other than that, no particular attention should be paidto other current skills; for example, some control variablesshould be replaced after 𝑥

(𝑠)

1,𝑡+1, 𝑥(𝑠)

2,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1represented

by other control variables in𝑉(𝑥(𝑠)

1,𝑡+1, 𝑥(𝑠)

2,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1) and so

on.

Journal of Applied Mathematics 3

Step 4. By the envelope theorem, take the partial derivativesof control variables at time 𝑡 on both sides of Bellmanequation to derive the remaining first-order conditions:

𝜕𝑉 (𝑥(𝑠)

1𝑡, . . . , 𝑥

(𝑠)

𝑛𝑡)

𝜕𝑥(𝑠)

𝑗𝑡

= 𝜆1𝑡

𝜕𝑔𝑑(𝑥1𝑡, . . . , 𝑥

𝑞𝑡)

𝜕𝑥(𝑠)

𝑗𝑡

+ ⋅ ⋅ ⋅ + 𝜆𝑑𝑡

𝜕𝑔𝑑(𝑥1𝑡, . . . , 𝑥

𝑞𝑡)

𝜕𝑥(𝑠)

𝑗𝑡

,

𝑗 = 1, 2, . . . , 𝑚.

(5)

Still, it does not need to expand 𝑥(𝑠)

1,𝑡+1, 𝑥(𝑠)

2,𝑡+1, . . . , 𝑥

(𝑠)

𝑛,𝑡+1

after themultipliers during the derivations. Requirements arethe same as Step 3.

Step 5. Obtain the new relevant results about target problemthrough recursion and substitution according to the aboveresults.

We could combine several state variables to one asyou need according to specific economic significance andconstraints when there is no need to describe the relevanteconomic significance of state variables one by one. For exam-ple, 𝑥(𝑠)1𝑡, 𝑥(𝑠)2𝑡

could be combined as 𝑎(𝑥(𝑠)

1𝑡, 𝑥(𝑠)

2𝑡) = 𝐴

𝑡. 𝐴𝑡is a

state variable in value function 𝑉(𝑎(𝑥(𝑠)

1𝑡, 𝑥(𝑠)

2𝑡), 𝑥(𝑠)

3𝑡, . . . , 𝑥

(𝑠)

𝑛𝑡),

that is, 𝑉(𝐴𝑡, 𝑥(𝑠)

2𝑡, . . . , 𝑥

(𝑠)

𝑛𝑡). And then we follow the above

five steps to solve specific problem. The rest of state variablescould be operated similarly. But combinations should notbe overlapped over state or combined state variables. Forinstance, suppose that 𝑥

(𝑠)

1𝑡, 𝑥(𝑠)

2𝑡have been combined as

𝑎(𝑥(𝑠)

1𝑡, 𝑥(𝑠)

2𝑡) = 𝐴

𝑡. It is inflexible to take 𝑥

(𝑠)

2𝑡as an independent

state variable in value function𝑉(𝐴𝑡, 𝑥(𝑠)

2𝑡, . . . , 𝑥

(𝑠)

𝑛𝑡) or put the

combination of 𝑥(𝑠)

2𝑡, 𝑥(𝑠)3𝑡, 𝑏(𝑥(𝑠)1𝑡

, 𝑥(𝑠)

2𝑡) = 𝐵

𝑡in value function

𝑉(𝐴𝑡, 𝐵𝑡, . . . , 𝑥

(𝑠)

𝑛𝑡) as a state variable.

The steps above give a general setting and solution ofBellman equation. These can be summarized as follows: first,set Bellman equation with multipliers of target dynamicoptimization problem under the requirement of no overlapsof state variables; second, extend the late period state variablesin 𝑉(⋅) on the right side of Bellman equation and there isno need to expand these variables after the multipliers; third,let the derivatives of state variables of time 𝑡 equal zero andtake the partial derivatives of these variables on both sides ofBellman equation to derive first-order conditions; finally, getmore needed results for analysis from these conditions.

Different from some current settings which allow overlapof state variables in value function, our method does notpermit overlaps. In fact, overlap of state variable is easy toreach wrong conclusions or it has to find some particularskills to get the right results [9]. These methods are hard togeneralize. More details will be seen in the following sections.

Nowwewill take several discrete time dynamic optimiza-tion problems that are under framework of the basic twomethods of incorporating money into general equilibriummodels as examples to show the applications of our setting

and solution method and compare with current popularmethods.

3. Setting and Solution of Bellman Equation inBasic Money-In-Utility Model

3.1. Model and Solution Using Previous Approach. The basicMoney-In-Utility model has few features. The labor-leisureoptions of families are ignored temporarily in utility function.Only family consumption and real money balances areinvolved in utility function; that is, real money balances yieldutility directly. We ignore the uncertainty of currency impactand technological changes temporarily for convenience.

The total present utility value of family life cycle is𝑈 = ∑

∞

𝑡=0𝛽𝑡

𝑢(𝑐𝑡, 𝑚𝑡). Usually the per capita version of

intertemporal budget constraint is

𝜔𝑡≡ 𝑓(

𝑘𝑡−1

1 + 𝑛) + 𝜏𝑡+ (

1 − 𝛿

1 + 𝑛) 𝑘𝑡−1

+(1 + 𝑖𝑡−1

) 𝑏𝑡−1

+ 𝑚𝑡−1

(1 + 𝜋𝑡) (1 + 𝑛)

= 𝑐𝑡+ 𝑘𝑡+ 𝑚𝑡+ 𝑏𝑡,

(6)

where 𝑐𝑡, 𝑏𝑡, 𝑘𝑡, 𝑚𝑡, 𝜏𝑡, 𝑖𝑡−1

, and 𝜋𝑡are the per capita

consumption, bonds, stock of capital, money balances, netlump-sum transfer received from the government, nominalinterest rate, and inflation rate, respectively, at time 𝑡. 𝛿 is therate of depreciation of physical capital, 𝑛 is the populationgrowth rate (assumed to be constant), and 𝑓(𝑘

𝑡−1/(1 +

𝑛)) is the production function assumed to be continuouslydifferentiable and to satisfy the usual Inada conditions (see[11]). Note that control variables can be changed and affectthe total discounted value of utility through consumption 𝑐

𝑡

in period 𝑡; state variables cannot be changed. Determinationof state and control variables is crucial to derive the first-orderconditions and other results. The control variables for thisproblem are 𝑐

𝑡, 𝑏𝑡, 𝑘𝑡and, 𝑚

𝑡and the state variables are 𝑏

𝑡−1,

𝑘𝑡−1

,𝑚𝑡−1

, 𝜏𝑡, and𝜔

𝑡, the household’s initial level of resources.

This problem can be solved by the expansion methodbelow according to Walsh [9]. It is started with setting upvalue function

𝑉 (𝜔𝑡) = max {𝑢 (𝑐

𝑡, 𝑚𝑡) + 𝛽𝑉 (𝜔

𝑡+1)} . (7)

Using (6), value function can be expanded as

𝑉 (𝜔𝑡)

= max{𝑢 (𝑐𝑡, 𝑚𝑡)

+ 𝛽𝑉(𝑓(𝑘𝑡

1 + 𝑛)

+ 𝜏𝑡+1

+ (1 − 𝛿

1 + 𝑛) 𝑘𝑡

+(1 + 𝑖𝑡) 𝑏𝑡+ 𝑚𝑡

(1 + 𝜋𝑡+1

) (1 + 𝑛))} .

(8)


Replace 𝑘𝑡as 𝜔𝑡− 𝑐𝑡− 𝑚𝑡− 𝑏𝑡; according to (6), (8) can be

written as an expansion equation as follows:

𝑉 (𝜔𝑡) = max{𝑢 (𝑐

𝑡, 𝑚𝑡)

+ 𝛽𝑉(𝑓(𝜔𝑡− 𝑐𝑡− 𝑚𝑡− 𝑏𝑡

1 + 𝑛) + 𝜏𝑡+1

+ (1 − 𝛿

1 + 𝑛) (𝜔𝑡− 𝑐𝑡− 𝑚𝑡− 𝑏𝑡)

+(1 + 𝑖𝑡) 𝑏𝑡+ 𝑚𝑡

(1 + 𝜋𝑡+1

) (1 + 𝑛))} .

(9)

Compute the partial derivatives of control variables toderive first-order conditions:

𝜕𝑉 (𝜔𝑡)

𝜕𝑐𝑡

= 𝑢𝑐(𝑐𝑡, 𝑚𝑡) − 𝑉𝜔(𝜔𝑡+1

)

⋅𝛽

1 + 𝑛⋅ [𝑓 (𝑘

𝑡) + 1 − 𝛿] = 0,

𝜕𝑉 (𝜔𝑡)

𝜕𝑏𝑡

= 𝛽𝑉𝜔(𝜔𝑡+1

) ⋅1 + 𝑖𝑡

(1 + 𝜋𝑡+1

) (1 + 𝑛)

− 𝛽𝑉𝜔(𝜔𝑡+1

) [𝑓𝑘(𝑘𝑡) + 1 − 𝛿

1 + 𝑛] = 0,

𝜕𝑉 (𝜔𝑡)

𝜕𝑚𝑡

= 𝑢𝑚

(𝑐𝑡, 𝑚𝑡) +

𝛽𝑉𝜔(𝜔𝑡+1

)

(1 + 𝜋𝑡+1

) (1 + 𝑛)

− 𝛽𝑉𝜔(𝜔𝑡+1

) [𝑓𝑘(𝑘𝑡) + 1 − 𝛿

1 + 𝑛] = 0,

(10)

and the envelope theorem yields

𝑉𝜔(𝜔𝑡) = 𝑢𝑐(𝑐𝑡, 𝑚𝑡) . (11)

Another expansionmethod will replace 𝑐𝑡as𝜔𝑡−𝑘𝑡−𝑚𝑡−

𝑏𝑡; according to (6), (8) can be written as

𝑉 (𝜔𝑡) = max{𝑢 (𝜔

𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡, 𝑚𝑡)

+ 𝛽𝑉(𝑓(𝑘𝑡

1 + 𝑛) + 𝜏𝑡+1

+ (1 − 𝛿

1 + 𝑛) 𝑘𝑡

+(1 + 𝑖𝑡) 𝑏𝑡+ 𝑚𝑡

(1 + 𝜋𝑡+1

) (1 + 𝑛))} .

(12)

Compute the partial derivatives of control variables toderive first-order conditions as

𝜕𝑉 (𝜔𝑡)

𝜕𝑏𝑡

= −𝑢𝑐(𝜔𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡, 𝑚𝑡)

+ 𝛽𝑉𝜔(𝜔𝑡+1

) ⋅1 + 𝑖𝑡

(1 + 𝜋𝑡+1

) (1 + 𝑛)= 0,

𝜕𝑉 (𝜔𝑡)

𝜕𝑘𝑡

= −𝑢𝑐(𝜔𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡, 𝑚𝑡) + 𝑉𝜔(𝜔𝑡+1

)

⋅𝛽

1 + 𝑛⋅ [𝑓𝑘(𝑘𝑡) + 1 − 𝛿] = 0,

𝜕𝑉 (𝜔𝑡)

𝜕𝑚𝑡

= −𝑢𝑐(𝜔𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡, 𝑚𝑡)

+ 𝑢𝑚

(𝜔𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡, 𝑚𝑡)

+𝛽𝑉𝜔(𝜔𝑡+1

)

(1 + 𝜋𝑡+1

) (1 + 𝑛)= 0.

(13)

Using the envelope theoremand computing the derivativewith respect to state variable 𝜔

𝑡, we get

𝑉𝜔(𝜔𝑡) = 𝑢𝑐(𝜔𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡, 𝑚𝑡) . (14)

3.2. Our Solving Approach. Now, we use our proposed stepsof setting and solution of Bellman equation to solve theabove basic Money-In-Utility problem. First, let the Bellmanequation with multiplier 𝜆

𝑡be

𝑉 (𝜔𝑡) = max {𝑢 (𝑐

𝑡, 𝑚𝑡) + 𝛽𝑉 (𝜔

𝑡+1)}

+ 𝜆𝑡(𝜔𝑡− 𝑐𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡) .

(15)

Second, computing the partial derivatives for the controlvariables, we obtain the first-order conditions as

𝜕𝑉 (𝜔𝑡)

𝜕𝑐𝑡

= 𝑢𝑐(𝑐𝑡, 𝑚𝑡) = 𝜆𝑡,

𝜕𝑉 (𝜔𝑡)

𝜕𝑏𝑡


) ⋅1 + 𝑖𝑡

(1 + 𝜋𝑡+1

) (1 + 𝑛)= 𝜆𝑡,

𝜕𝑉 (𝜔𝑡)

𝜕𝑘𝑡


) ⋅1

1 + 𝑛⋅ [𝑓𝑘(𝑘𝑡) + 1 − 𝛿] = 𝜆

𝑡,

𝜕𝑉 (𝜔𝑡)

𝜕𝑚𝑡

= 𝑢𝑚

(𝑐𝑡, 𝑚𝑡) + 𝛽𝑉

𝜔(𝜔𝑡+1

)

⋅1

(1 + 𝜋𝑡+1

) (1 + 𝑛)= 𝜆𝑡.

(16)

Third, using the envelope theorem and computing thederivatives of both sides of Bellman equation with respect tostate variable 𝜔

𝑡, we get

𝑉𝜔(𝜔𝑡) = 𝜆𝑡. (17)


Finally, based on the above results, it is easy to get

𝑢𝑐(𝑐𝑡, 𝑚𝑡) = 𝜆𝑡= 𝑢𝑚

(𝑐𝑡, 𝑚𝑡)

+ 𝛽𝑉𝜔(𝜔𝑡+1

) ⋅1

(1 + 𝜋𝑡+1

) (1 + 𝑛).

(18)

This expression indicates that the marginal benefit ofincreased money holdings should be equal to the marginalutility of consumption on period 𝑡. Similarly, we can easilyobtain the allocation of initial level of resources 𝜔

𝑡among

consumption, capital, bonds, and money balances, and thesame marginal benefit must be yielded by each use at anoptimum allocation.

These results with multiplier are open-and-shut, and it iseasy to find the economic significance of marginal utility ofconsumption. Comparing with current approach mentionedabove, during the process of solving problem, there is no needto consider which variable should be replaced. There are notedious expanded expressions. Using expansion method, ifwe replace 𝑘

𝑡, the derived first-order conditions are seen to

be messy and it is not easy to find the relationships of results.If we replace 𝑐

𝑡, the derived results are a bit more clear, but we

might not think of replacing 𝑐𝑡at the beginning. So expansion

methods are not easy to operate, and it is not clear to replacevariable.

4. Setting and Solution of Bellman Equation inShopping-Time Model

4.1. Model and Solution Using Previous Approach. In Shop-ping-Time Model, shopping time is a function of consump-tion and money balances. Because consumption needs shop-ping time, leisure is reduced. Household utility is assumed todepend on consumption and leisure. Consumption can notonly yield utility directly but also decrease utility indirectly.In this section, 𝑛 is time spent in market employment and𝑛𝑠 is time spent shopping. 𝑛𝑠 is the function of consumption

and money balances; that is, 𝑛𝑠 = 𝑔(𝑐,𝑚), 𝑔𝑐

> 0, 𝑔𝑚

≤ 0.Total time available is normalized to equal 1. Growth rate ofpopulation is assumed to be 0 for convenience. Let 𝑙 be theleisure time. The utility function is 𝑢(𝑐,𝑚, 𝑙) = V[𝑐, 1 − 𝑛 −

𝑔(𝑐, 𝑚)]. It donates utility as a function of consumption, laborsupply, and money holdings.

The household’s intertemporal objective is maximumdiscounted utility subject to resource constraint as

max∞

∑

𝑖=0

𝛽𝑖V (𝑐𝑡+𝑖

, 1 − 𝑛𝑡+𝑖

− 𝑔 (𝑐𝑡+𝑖

, 𝑚𝑡+𝑖

))

= max∞

∑

𝑖=0

𝛽𝑖V (𝑐𝑡+𝑖

, 𝑙𝑡+𝑖

)

s.t. 𝑓 (𝑘𝑡−1

, 𝑛𝑡) + 𝜏𝑡+ (1 − 𝛿) 𝑘

𝑡−1

+(1 + 𝑖𝑡−1

) 𝑏𝑡−1

+ 𝑚𝑡−1

1 + 𝜋𝑡

= 𝑐𝑡+ 𝑘𝑡+ 𝑚𝑡+ 𝑏𝑡.

(19)

Because controllable factors, labor supply, and consump-tion affect labor supply, output is affected not only by state

variable 𝑘𝑡−1

but also by these controllable factors. The leftside of the budget constraint is unsuitable to be combined asa state variable 𝜔

𝑡. Combining the money holdings, bonds,

and transfers to 𝑎𝑡= 𝜏𝑡+ (((1 + 𝑖

𝑡−1)𝑏𝑡−1

+ 𝑚𝑡−1

)/(1 + 𝜋𝑡)), the

household’s financial assets, value function can be written as

𝑉 (𝑎𝑡, 𝑘𝑡−1

) = max {V (𝑐𝑡, 𝑙𝑡) + 𝛽𝑉 (𝑎

𝑡+1, 𝑘𝑡)} . (20)

According to Walsh [9], after replacing 𝑎𝑡+1

, 𝑘𝑡, value

function can be expanded as


)

= max{V [𝑐𝑡, 1 − 𝑛

𝑡− 𝑔 (𝑐

𝑡, 𝑚𝑡)]

+ 𝛽𝑉[𝜏𝑡+1

+(1 + 𝑖𝑡) 𝑏𝑡+ 𝑚𝑡

1 + 𝜋𝑡+1

, 𝑓 (𝑘𝑡−1

, 𝑛𝑡)

+ (1 − 𝛿) 𝑘𝑡−1

+ 𝑎𝑡− 𝑚𝑡− 𝑏𝑡− 𝑐𝑡]} .

(21)

Computing the partial derivatives with respect to controlvariables, we get

𝜕𝑉 (𝑎𝑡, 𝑘𝑡−1

)

𝜕𝑐𝑡

= V𝑐[𝑐𝑡, 1 − 𝑛

𝑡− 𝑔 (𝑐

𝑡, 𝑚𝑡)]

− V𝑙[𝑐𝑡, 1 − 𝑛

𝑡− 𝑔 (𝑐

𝑡, 𝑚𝑡)] 𝑔𝑐(𝑐𝑡, 𝑚𝑡)

− 𝛽𝑉𝑘(𝑎𝑡+1

, 𝑘𝑡) = 0,


)

𝜕𝑘𝑡

= −V𝑙[𝑐𝑡, 1 − 𝑛

𝑡− 𝑔 (𝑐

𝑡, 𝑚𝑡)]

+ 𝛽𝑉𝑘(𝑎𝑡+1

, 𝑘𝑡) ⋅ 𝑓𝑛(𝑘𝑡−1

, 𝑛𝑡) = 0,


)

𝜕𝑚𝑡

= −V𝑙[𝑐𝑡, 1 − 𝑛

𝑡− 𝑔 (𝑐

𝑡, 𝑚𝑡)] 𝑔𝑚


+ 𝛽𝑉𝑎(𝑎𝑡+1

, 𝑘𝑡) ⋅

1

1 + 𝜋𝑡+1

− 𝛽𝑉𝑘(𝑎𝑡+1

, 𝑘𝑡) = 0.

(22)

Computing the partial derivatives of both sides of thevalue equation with respect to the state variables 𝑎

𝑡, 𝑘𝑡−1

, weget

𝑉𝑎(𝑎𝑡, 𝑘𝑡−1

) = 𝛽𝑉𝑘(𝑎𝑡+1

, 𝑘𝑡)

𝑉𝑘(𝑎𝑡, 𝑘𝑡−1

) = 𝛽𝑉𝑘(𝑎𝑡+1

, 𝑘𝑡) [𝑓𝑘(𝑘𝑡−1

, 𝑛𝑡) + 1 − 𝛿] .

(23)

4.2. Our Solving Approach. Now, we use our proposedmethod to solve the above Shopping-Time Model problem.First, let the Bellman equation with multiplier 𝜆

𝑡be


) = max {V (𝑐𝑡, 𝑙𝑡) + 𝛽𝑉 (𝑎

𝑡+1, 𝑘𝑡)}

+ 𝜆𝑡[𝑎𝑡+ 𝑓 (𝑘

𝑡−1, 𝑛𝑡) + (1 − 𝛿) 𝑘

𝑡−1

− 𝑐𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡] ,

(24)


where state variable 𝑎𝑡+1

= 𝜏𝑡+1

+ (((1 + 𝑖𝑡)𝑏𝑡+𝑚𝑡)/(1 +𝜋

𝑡+1)).

Then, compute the partial derivatives with respect to controlvariables to derive first-order conditions,


)

𝜕𝑐𝑡

= V𝑐(𝑐𝑡, 𝑙𝑡) − V𝑙(𝑐𝑡, 𝑙𝑡) 𝑔𝑐(𝑐𝑡, 𝑚𝑡) = 𝜆𝑡,


)

𝜕𝑏𝑡

= 𝛽𝑉𝑎(𝑎𝑡+1

, 𝑘𝑡) ⋅

1 + 𝑖𝑡

1 + 𝜋𝑡+1

= 𝜆𝑡,


)

𝜕𝑘𝑡

= 𝛽𝑉𝑘(𝑎𝑡+1

, 𝑘𝑡) = 𝜆𝑡,


)

𝜕𝑚𝑡

= −V𝑙(𝑐𝑡, 𝑙𝑡) 𝑔𝑚

(𝑐𝑡, 𝑚𝑡) +

𝛽𝑉𝑎(𝑎𝑡+1

, 𝑘𝑡)

1 + 𝜋𝑡+1

= 𝜆𝑡.

(25)

Using the envelope theorem and computing the deriva-tives with respect to the state variables 𝑎

𝑡, 𝑘𝑡−1

, we get

𝑉𝑎(𝑎𝑡, 𝑘𝑡−1

) = 𝜆𝑡,


) = 𝜆𝑡[𝑓𝑘(𝑘𝑡−1

, 𝑛𝑡) + 1 − 𝛿] .

(26)

Now, we use the above results to compute the opportunitycost of holding money. Since the utility function is

𝑢 (𝑐, 𝑚, 𝑙) = V [𝑐, 𝑙 − 𝑛 − 𝑔 (𝑐,𝑚)] , (27)

and based on (6), (25), and (26), we obtain

𝑢𝑚

(𝑐𝑡, 𝑚𝑡, 𝑙𝑡)

𝑢𝑐(𝑐𝑡, 𝑚𝑡, 𝑙𝑡)

=−V𝑙(𝑐𝑡, 𝑙𝑡) 𝑔𝑚


V𝑐(𝑐𝑡, 𝑙𝑡) − V𝑙(𝑐𝑡, 𝑙𝑡) 𝑔𝑐(𝑐𝑡, 𝑚𝑡)

=𝜆𝑡− ((𝛽𝑉

𝑎(𝑎𝑡+1

, 𝑘𝑡)) / (1 + 𝜋

𝑡+1))

𝜆𝑡

= 1 −((𝛽𝑉𝑎(𝑎𝑡+1

, 𝑘𝑡)) / (1 + 𝜋

𝑡+1))

𝛽𝑉𝑎(𝑎𝑡+1

, 𝑘𝑡) ⋅ ((1 + 𝑖

𝑡) / (1 + 𝜋

𝑡+1))

= 1 −1

1 + 𝑖𝑡

=𝑖𝑡

1 + 𝑖𝑡

.

(28)

Obviously, the expressions of the derived first-orderconditions by previousmethod seem to be tedious andmessy,and it is not so easy to compute the relevant results such as theopportunity cost of holding money. Our proposed methodis comparatively neat and can easily obtain relevant resultscorrectly.

5. Setting and Solution of Bellman Equation inCash-In-Advance Model

5.1. Model and Solution Using Previous Approach. In basicCash-In-Advance model, money is used to purchase goods.Money cannot yield utility itself, but the consumption offuture can yield utility. Svensson [12] assumed that agentsare available for spending only the cash carried over fromthe previous period.This is essentially different fromMoney-In-Utility model. Consider a simple form of discounted

household utility value∑∞

𝑡=0𝛽𝑡

𝑢(𝑐𝑡). More complicated utility

function will be discussed in the following sections. TakeCash-In-Advance constraint form as 𝑐

𝑡≤ (𝑚𝑡−1

/(1 + 𝜋𝑡)) + 𝜏𝑡

due to Svensson [12]. This means that the agent enters theperiod with money holdings 𝑚

𝑡−1and receives a lump-sum

transfer 𝜏𝑡(in real currency terms) for consumption goods.

Bonds and capital may not be purchased by currency. Ifcapital is assumed to be purchased by money, the Cash-In-Advance constraint will become (𝑚

𝑡−1/(1 + 𝜋

𝑡)) + 𝜏𝑡≥ 𝑐𝑡+ 𝑘𝑡.

The budget constraint is rewritten in real terms as

𝑓 (𝑘𝑡−1

) + 𝜏𝑡+ (1 − 𝛿) 𝑘

𝑡−1

+(1 + 𝑖𝑡−1

) 𝑏𝑡−1

+ 𝑚𝑡−1

1 + 𝜋𝑡

≥ 𝑐𝑡+ 𝑘𝑡+ 𝑚𝑡+ 𝑏𝑡.

(29)

In monetary theory, constraints are expressed as inequal-ity frequently. This constraint describes that the represen-tative agent’s time 𝑡 real resources should be more thanor equal to the use of it, that is, purchasing consumption,capital, bonds, and money holdings that are then carried intoperiod 𝑡 + 1. Because of the assumption of rational agents,the certainty problems we are discussing, and the positiveopportunity cost of money holdings, the constraints becomeequations in equilibriums.

Output is only affected by state variable 𝑘𝑡−1

withoutconsidering the effect of labor supply. Following the solutionmethods ofWalsh [9], let the left side of the budget constraintbe a state variable𝜔

𝑡= 𝑓(𝑘

𝑡−1)+𝜏𝑡+(1−𝛿)𝑘

𝑡−1+(((1+𝑖

𝑡−1)𝑏𝑡−1

+

𝑚𝑡−1

)/(1 + 𝜋𝑡)). It will be very tedious adopting expansion

method with two constraints. Setting Bellman equation withmultipliers will be better. Let state variable 𝑚

𝑡−1in value

function get the economic significance of money in first-order conditions. However, there is an overlapped settingwith 𝜔

𝑡,

𝑉 (𝜔𝑡, 𝑚𝑡−1

) = max {𝑢 (𝑐𝑡) + 𝛽𝑉 (𝜔

𝑡+1, 𝑚𝑡)}

+ 𝜆𝑡(𝜔𝑡− 𝑐𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡)

+ 𝜇𝑡(

𝑚𝑡−1

1 + 𝜋𝑡

+ 𝜏𝑡− 𝑐𝑡) .

(30)

5.2. Our Solving Approach. Let 𝜉𝑡= 𝑓(𝑘

𝑡−1)+𝜏𝑡+(1−𝛿)𝑘

𝑡−1+

(((1 + 𝑖𝑡−1

)𝑏𝑡−1

)/(1 + 𝜋𝑡)). There is no overlap with 𝑚

𝑡in the

value function. The Bellman equation is set as

𝑉 (𝜉𝑡, 𝑚𝑡−1

) = max {𝑢 (𝑐𝑡) + 𝛽𝑉 (𝜉

𝑡+1, 𝑚𝑡)}

+ 𝜆𝑡(𝜉𝑡+

𝑚𝑡−1

1 + 𝜋𝑡

− 𝑐𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡)

+ 𝜇𝑡(

𝑚𝑡−1

1 + 𝜋𝑡

+ 𝜏𝑡− 𝑐𝑡) .

(31)

The solution of this problem is similar to the applicationof the proposed method in Sections 3 and 4. We will not giveunnecessary details here.

Note that Walsh [9] puts overlapping state variables 𝑚𝑡−1

and𝜔𝑡in value function; it is easy to get wrong results in those


problems with Cash-In-Advance constraints although it is noproblem setting 𝑉(𝜔

𝑡, 𝑚𝑡−1

) with no more results requirednow. This will be seen in the following example of Section 6.

6. Setting and Solution of Bellman Equationin Model considering Both Labor Time andCash-In-Advance Constraints

6.1. Model and Solution Using Previous Approach. Assum-ing that money is used to purchase consumption goodsand investments, the Cash-In-Advance constraint becomes(𝑚𝑡−1

/(1 + 𝜋𝑡)) + 𝜏

𝑡≥ 𝑐𝑡+ 𝑥𝑡, where 𝑥 is investment and

𝑥𝑡

= 𝑘𝑡− (1 − 𝛿)𝑘

𝑡−1. In considering of the effects of labor

supply on output and leisure on utility, we have 𝑙𝑡

= 1 − 𝑛𝑡.

The agent’s objective becomes

max∞

∑

𝑡=0

𝑢 (𝑐𝑡, 𝑙𝑡)

s.t. 𝑓 (𝑘𝑡−1

) + 𝜏𝑡+ (1 − 𝛿) 𝑘

𝑡−1

+(1 + 𝑖𝑡−1

) 𝑏𝑡−1

+ 𝑚𝑡−1

1 + 𝜋𝑡

≥ 𝑐𝑡+ 𝑘𝑡+ 𝑚𝑡+ 𝑏𝑡,

𝑚𝑡−1

1 + 𝜋𝑡

+ 𝜏𝑡≥ 𝑐𝑡+ 𝑥𝑡.

(32)

Output is affected not only by state variable 𝑘𝑡−1

but alsoby controllable factor 𝑛

𝑡; the left side of the budget constraint

is unsuitable to be combined as a state variable. Thinkingof the setting of the Shopping-Time Model and Cash-In-Advance model above, it is plausible to put overlapped statevariables 𝑎

𝑡= 𝜏𝑡+(((1+ 𝑖

𝑡−1)𝑏𝑡−1

+𝑚𝑡−1

)/(1+𝜋𝑡)) and𝑚

𝑡−1in

value function following the solution methods of Walsh [9].The Bellman equation will be


, 𝑚𝑡−1

)

= max {𝑢 (𝑐𝑡, 𝑙𝑡) + 𝛽𝑉 (𝑎

𝑡+1, 𝑘𝑡, 𝑚𝑡)}

+ 𝜆𝑡[𝑎𝑡+ 𝑓 (𝑘

𝑡−1, 𝑛𝑡) + (1 − 𝛿) 𝑘

𝑡−1

− 𝑐𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡]

+ 𝜇𝑡[𝑎𝑡− 𝑐𝑡− 𝑘𝑡+ (1 − 𝛿) 𝑘

𝑡−1] .

(33)

The first-order conditions are𝜕𝑉 (𝑎𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

)

𝜕𝑐𝑡

= 𝑢𝑐(𝑐𝑡, 𝑙𝑡) = 𝜆𝑡+ 𝜇𝑡,


, 𝑚𝑡−1

)

𝜕𝑏𝑡

= 𝛽𝑉𝑎(𝑎𝑡+1

, 𝑘𝑡, 𝑚𝑡) ⋅

1 + 𝑖𝑡

1 + 𝜋𝑡+1

= 𝜆𝑡,


, 𝑚𝑡−1

)

𝜕𝑘𝑡

= 𝛽𝑉𝑘(𝑎𝑡+1

, 𝑘𝑡, 𝑚𝑡) = 𝜆𝑡+ 𝜇𝑡,


, 𝑚𝑡−1

)

𝜕𝑚𝑡

= 𝛽𝑉𝑚

(𝑎𝑡+1

, 𝑘𝑡, 𝑚𝑡)

+ 𝛽𝑉𝑎(𝑎𝑡+1

, 𝑘𝑡, 𝑚𝑡) ⋅

1

1 + 𝜋𝑡+1

= 𝜆𝑡,


, 𝑚𝑡−1

)

𝜕𝑛𝑡

= −𝑢𝑙(𝑐𝑡, 𝑙𝑡) + 𝜆𝑡𝑓𝑛(𝑘𝑡−1

, 𝑛𝑡) = 0.

(34)

Using the envelope theorem and computing the deriva-tives with respect to state variables, we get


, 𝑚𝑡−1

)

𝜕𝑎𝑡

= 𝑉𝑎(𝑎𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

) = 𝜆𝑡+ 𝜇𝑡,


, 𝑚𝑡−1

)

𝜕𝑘𝑡−1

= 𝑉𝑘(𝑎𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

)

= 𝜆𝑡[𝑓𝑘(𝑘𝑡−1

, 𝑛𝑡) + (1 − 𝛿)]

+ 𝜇𝑡(1 − 𝛿) ,


, 𝑚𝑡−1

)

𝜕𝑚𝑡−1

= 𝑉𝑚

(𝑎𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

) = (𝜆𝑡+ 𝜇𝑡)

1

1 + 𝜋𝑡

.

(35)

When the question is to derive the effect of inflation rateon the steady-state capital-labor ratio, that is, the steady-state relationship of 𝑘𝑠𝑠/𝑛𝑠𝑠 and 𝜋

𝑠𝑠, assume that the aggregateproduction takes the form 𝑦

𝑡= 𝑓(𝑘

𝑡−1, 𝑛𝑡) = 𝐴𝑘

𝛼

𝑡−1𝑛1−𝛼

𝑡.

Using (34) and (35),

𝑢𝑙(𝑐𝑡, 𝑙𝑡)

𝑢𝑐(𝑐𝑡, 𝑙𝑡)

=𝜆𝑡𝑓𝑛(𝑘𝑡−1

, 𝑛𝑡)

𝜆𝑡+ 𝜇𝑡

= ([𝛽𝑉𝑚

(𝑎𝑡+1

, 𝑘𝑡, 𝑚𝑡) + 𝛽𝑉

𝑎(𝑎𝑡+1

, 𝑘𝑡, 𝑚𝑡) ⋅

1

1 + 𝜋𝑡+1

]

× 𝑓𝑛(𝑘𝑡−1

, 𝑛𝑡) ) (𝜆

𝑡+ 𝜇𝑡)−1

= ([𝛽 (𝜆𝑡+1

+ 𝜇𝑡+1

)1

1 + 𝜋𝑡+1

+ 𝛽 (𝜆𝑡+1

+ 𝜇𝑡+1

)1

1 + 𝜋𝑡+1

]

× 𝑓𝑛(𝑘𝑡−1

, 𝑛𝑡) ) (𝜆

𝑡+ 𝜇𝑡)−1

= 2𝛽1

1 + 𝜋𝑡+1

𝑓𝑛(𝑘𝑡−1

, 𝑛𝑡) ⋅

𝜆𝑡+1

+ 𝜇𝑡+1

𝜆𝑡+ 𝜇𝑡

.

(36)

Then𝜆𝑡

𝜆𝑡+ 𝜇𝑡

= 2𝛽1

1 + 𝜋𝑡+1

𝜆𝑡+1

+ 𝜇𝑡+1

𝜆𝑡+ 𝜇𝑡

. (37)

From (35),

𝑓𝑘(𝑘𝑡−1

, 𝑛𝑡) =


, 𝑚𝑡−1

) − (1 − 𝛿) (𝜆𝑡+ 𝜇𝑡)

𝜆𝑡

= [1

𝛽− (1 − 𝛿)]

𝜆𝑡+ 𝜇𝑡

𝜆𝑡

.

(38)

Rewriting the aggregate production as 𝑓𝑘

= 𝛼𝐴(𝑘𝑡−1

/

𝑛𝑡)𝛼−1, we have

𝜆𝑡

𝜆𝑡+ 𝜇𝑡

= (𝛼𝐴)−1

(𝑘𝑡−1

𝑛𝑡

)

1−𝛼

[1

𝛽− (1 − 𝛿)] . (39)


Using (37), we obtain

(𝛼𝐴)−1

(𝑘𝑡−1

𝑛𝑡

)

1−𝛼

[1

𝛽− (1 − 𝛿)]

= 2𝛽1

1 + 𝜋𝑡+1

𝜆𝑡+1

+ 𝜇𝑡+1

𝜆𝑡+ 𝜇𝑡

.

(40)

Rewriting this equation, we get

𝑘𝑡−1

𝑛𝑡

= [2𝛽𝛼𝐴

1 + 𝜋𝑡+1

(1

𝛽− 1 + 𝛿)

−1

𝜆𝑡+1

+ 𝜇𝑡+1

𝜆𝑡+ 𝜇𝑡

]

1/(1−𝛼)

. (41)

In steady-state, 𝜆𝑡= 𝜆𝑡+1

= 𝜆𝑠𝑠, 𝜇𝑡= 𝜇𝑡+1

= 𝜇𝑠𝑠, 𝜋𝑡+1

=

𝜋𝑠𝑠, 𝑘𝑡−1

= 𝑘𝑠𝑠, 𝑛𝑡= 𝑛𝑠𝑠, from (41), we have

𝑘𝑠𝑠

𝑛𝑠𝑠

= [1 + 𝜋𝑠𝑠

2𝛼𝛽(

1

𝛽− 1 + 𝛿)]

1/(𝛼−1)

. (42)

This steady-state capital-labor ratio is derived by usingcurrent prevailing methods. However, it is a wrong result. Wewill provide the correct result by using the proposed methodof this paper.

6.2. Our Solving Approach. Output is affected by state vari-able 𝑘

𝑡−1and control variable 𝑛

𝑡; it is unsuitable to let the left

side of the budget constraint be combined as a state variable.We should separate 𝑘

𝑡−1from it. Let state variable 𝑚

𝑡−1in

value function alone get the economic significance of moneyin first-order conditions. Because overlaps of state variablesare not allowed according to our proposed method, we put𝜁𝑡= 𝜏𝑡+ (((1 + 𝑖

𝑡−1)𝑏𝑡−1

)/(1 + 𝜋𝑡)) in value function as a state

variable.First, set up value function with multipliers:

𝑉 (𝜁𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

)

= max {𝑢 (𝑐𝑡, 𝑙𝑡) + 𝛽𝑉 (𝜁

𝑡+1, 𝑘𝑡, 𝑚𝑡)}

+ 𝜆𝑡[𝜁𝑡+

𝑚𝑡−1

1 + 𝜋𝑡

+ 𝑓 (𝑘𝑡−1

, 𝑛𝑡)

+ (1 − 𝛿) 𝑘𝑡−1

− 𝑐𝑡− 𝑘𝑡− 𝑚𝑡− 𝑏𝑡]

+ 𝜇𝑡[𝜁𝑡−

(1 + 𝑖𝑡−1

) 𝑏𝑡−1

1 + 𝜋𝑡

+𝑚𝑡−1

1 + 𝜋𝑡

− 𝑐𝑡− 𝑘𝑡+ (1 − 𝛿) 𝑘

𝑡−1] .

(43)

Second, compute the derivatives of state variables andderive first-order conditions:


, 𝑚𝑡−1

)

𝜕𝑐𝑡

= 𝑢𝑐(𝑐𝑡, 𝑙𝑡) = 𝜆𝑡+ 𝜇𝑡,


, 𝑚𝑡−1

)

𝜕𝑏𝑡

= 𝛽𝑉𝜁(𝜁𝑡+1

, 𝑘𝑡, 𝑚𝑡) ⋅

1 + 𝑖𝑡

1 + 𝜋𝑡+1

= 𝜆𝑡,


, 𝑚𝑡−1

)

𝜕𝑘𝑡

= 𝛽𝑉𝑘(𝜁𝑡+1

, 𝑘𝑡, 𝑚𝑡) = 𝜆𝑡+ 𝜇𝑡,


, 𝑚𝑡−1

)

𝜕𝑚𝑡

= 𝛽𝑉𝑚

(𝜁𝑡+1

, 𝑘𝑡, 𝑚𝑡) = 𝜆𝑡,


, 𝑚𝑡−1

)

𝜕𝑛𝑡

= −𝑢𝑙(𝑐𝑡, 𝑙𝑡) + 𝜆𝑡𝑓𝑛(𝑘𝑡−1

, 𝑛𝑡) = 0.

(44)

Third, compute the partial derivatives with respect to 𝜁𝑡,

𝑘𝑡−1

, and 𝑚𝑡−1

and the envelope theorem yields


, 𝑚𝑡−1

)

𝜕𝜁𝑡

= 𝑉𝜁(𝜁𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

) = 𝜆𝑡+ 𝜇𝑡,


, 𝑚𝑡−1

)

𝜕𝑘𝑡−1

= 𝑉𝑘(𝜁𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

)

= 𝜆𝑡[𝑓𝑘(𝑘𝑡−1

, 𝑛𝑡) + (1 − 𝛿)] + 𝜇

𝑡(1 − 𝛿) ,


, 𝑚𝑡−1

)

𝜕𝑚𝑡−1

= 𝑉𝑚

(𝜁𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

) = (𝜆𝑡+ 𝜇𝑡)

1

1 + 𝜋𝑡

.

(45)

Finally, derive the steady-state capital-labor ratio by theresults above:

𝑘𝑠𝑠

𝑛𝑠𝑠

= [1 + 𝜋𝑠𝑠

𝛼𝛽(

1

𝛽− 1 + 𝛿)]

1/(𝛼−1)

. (46)

The process of deriving this ratio is similar to thederiving process of (42). We will not give unnecessarydetails here. Compare (46) with (42); the result of (42) is(1/2)1/(𝛼−1) times that of (46). Equation (46) is the correct

result. It is the different partial derivatives with respectto 𝑚𝑡in (34) and (44) that cause this deviation. In (34),

𝛽𝑉𝑚(𝑎𝑡+1

, 𝑘𝑡, 𝑚𝑡) + 𝛽𝑉

𝑎(𝑎𝑡+1

, 𝑘𝑡, 𝑚𝑡) ⋅ (1/(1 + 𝜋

𝑡+1)) = 𝜆

𝑡. In

(44), 𝛽𝑉𝑚(𝜁𝑡+1

, 𝑘𝑡, 𝑚𝑡) = 𝜆

𝑡. Fundamentally speaking, the

reason for the deviation is the overlapped setting of statevariables 𝑎

𝑡and𝑚

𝑡−1versus the nonoverlapped setting of state

variables 𝜁𝑡, 𝑘𝑡−1

, and𝑚𝑡−1

. One has to use the particular skillsin [9] that is hard to think about and grasp to obtain the rightanswer or get a wrong result adopting the overlapped setting𝑉(𝑎𝑡, 𝑘𝑡−1

, 𝑚𝑡−1

). For this reason, the setting of value functionbyWalsh could not be generalized. In this model consideringboth Labor Time and Cash-In-Advance constraints, the basicmethods of incorporating money into general equilibriummodels are generalized, and our proposed method of setting


and solution of Bellman equation demonstrates its clarityand validity and corrects the defect that some other currentmethods of setting and solution are easy to get wrong results.

7. Conclusions

As an important tool in theory economics, Bellman equationis very powerful in solving optimization problems of discretetime and is frequently used in monetary theory. It is hard tograsp the setting and solution of Bellman equation and easy toreach wrong conclusions since there is no general method toset Bellman equation or the settings of Bellman equation areexcessively flexible. In this paper, we provide a set of generalsetting and solution methods for Bellman equation withmultipliers. In the processes of solving monetary problems,comparing with other current methods in classic reference,our proposed method demonstrates its features of clarity,validity, correct results, easy operation, and generalization.Bellman equation is used not only in monetary problemsbut also in almost every dynamic programming problemassociated with discrete time optimization. Our future workis to study the applicability of the proposed method in thispaper in other areas.

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper.

Acknowledgments

Wanbo Lu’s research is sponsored by the National ScienceFoundation of China (71101118) and the Program for NewCentury Excellent Talents in University (NCET-13-0961) inChina.

References

[1] M. Sidrauski, “Rational choice and patterns of growth in amon-etary economy,” American Economic Review, vol. 57, no. 2, pp.534–544, 1967.

[2] W. Baumol, “The transactions demand for cash,”Quarterly Jour-nal of Economics, vol. 67, no. 4, pp. 545–556, 1952.

[3] J. Tobin, “The interest elasticity of the transactions demand forcash,” Review of Economics and Statistics, vol. 38, no. 3, pp. 241–247, 1956.

[4] R. W. Clower, “A reconsideration of the microfoundations ofmonetary theory,” Western Economic Journal, vol. 6, no. 1, pp.1–9, 1967.

[5] W. A. Brock, “Money and growth: the case of long run perfectforesight,” International Economic Review, vol. 15, pp. 750–777,1974.

[6] B. T. McCallum and M. S. Goodfriend, “Demand for money:theoretical studies,” in The New Palgrave Dictionary of Eco-nomics, pp. 775–781, Palgrave MacMillan, Houndmills, UK,1987.

[7] D. Croushore, “Money in the utility function: functional equiv-alence to a shopping-time model,” Journal of Macroeconomics,vol. 15, no. 1, pp. 175–182, 1993.

[8] P. A. Samuelson, “An exact consumption-loan model of interestwith or without the social contrivance of money,” Journal ofPolitical Economy, vol. 66, no. 6, pp. 467–482, 1958.

[9] C. E. Walsh, Monetary Theory and Policy, The MIT Press,Cambridge, Mass, USA, 3rd edition, 2010.

[10] L. Gong,OptimizationMethods in Economics, PekingUniversityPress, Beijing, China, 2000.

[11] D. Romer,AdvancedMacroeconomics, McGraw-Hill, New York,NY, USA, 4th edition, 2011.

[12] L. E. O. Svensson, “Money and asset prices in a cash-in-advanceeconomy,” Journal of Political Economy, vol. 93, no. 5, pp. 919–944, 1985.

Submit your manuscripts athttp://www.hindawi.com

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

MathematicsJournal of


Mathematical Problems in Engineering

Hindawi Publishing Corporationhttp://www.hindawi.com

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of


Probability and StatisticsHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of


CombinatoricsHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

International Journal of


Operations ResearchAdvances in

Journal of


Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

International Journal of Mathematics and Mathematical Sciences


The Scientific World JournalHindawi Publishing Corporation http://www.hindawi.com Volume 2014


Algebra

Discrete Dynamics in Nature and Society



Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttp://www.hindawi.com

Volume 2014 Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Stochastic AnalysisInternational Journal of

research article a general setting and solution of bellman...

Documents