Vanguard Research Initiative Technical Report:Long-Term-Care Model

John Ameriks Joseph Briggs Andrew CaplinThe Vanguard Group, Inc. Federal Reserve Board of Governors New York University and NBER

Matthew D. Shapiro Christopher TonettiUniversity of Michigan and NBER Stanford GSB and NBER

June 2018

This document describes an algorithm to compute optimal saving and expenditure policies in a model featuredin papers associated with the Vanguard Research Initiative. This model appears in Ameriks, Briggs, Caplin,Shapiro, and Tonetti (2017) and Ameriks, Briggs, Caplin, Shapiro, and Tonetti (2018).

1 Model Environment

1.1 Consumers

Consumers are heterogeneous over wealth (a ∈ [0,∞)), income age-profile (y ∈ y1, y2, . . . , y5), age (t ∈55, 56, ..., 108), gender (g ∈ m, f), health status (s ∈ 0, 1, 2, 3), and health cost (h ∼ H(t, s) withsupport ΩH(t, s)). Time is discrete and the life-cycle horizon is finite. Consumers start at age t0 and liveto be at most T-1 years old, where in our parameterization t0 corresponds with age 55 and T correspondswith age 108. Each period individuals choose consumption (c ∈ [0,∞)), savings (a′ ≥ 0), and whether touse government care (G ∈ 0, 1). The model groups people into five income groups with deterministicage-income profiles.1 Each individual has a perfectly foreseen deterministic income sequence and receives arisk free rate of return of (1 + r) on savings. The only uncertainty an individual has is over health/death.

1.2 Government

The consumer always has the option to use a means-tested government provided care program. The cost ofusing government care is that a consumer’s wealth is set to zero, while the benefit is that the governmentprovides predetermined levels of expenditure, which depend on the health status of the individual as describedbelow. G = 1 if the consumer chooses to use government care and G = 0 if the consumer chooses not to usegovernment care.

1The model abstracts from labor supply decisions, including retirement. These labor market decisions are taken into accountthrough the exogenous income profiles.

1

1.3 Health and Death

There are four health states: s = 0 represents good health, s = 1 represents poor health, s = 2 represents theneed for long-term care (LTC), and s = 3 represents death. The health state evolves according to a Markovprocess, where the probability matrix, πg(s′|t, s) is gender, age, and health state dependent. h is a stochastichealth expenditure that must be paid—essentially a negative wealth shock. Each period the consumer hasto pay this health cost, h, where, h ∼ H(t, s) and H is the CDF of the health cost random variable withsupport ΩH(t, s).

If a consumer chooses to use government care when he does not need LTC (i.e., when s = 0, 1), then thegovernment provides a consumption floor, c = ωG, that is designed to represent welfare.

A consumer needs LTC if he needs help with the activities of daily living (ADLs), such as bathing, eating,dressing, walking across a room, or getting in or out of bed. Thus, state 2 is interchangeably referred to asthe LTC or ADL state. If a consumer needs LTC (s = 2), then he must either purchase private long-termcare or use government care. Capturing the fact that LTC provision is essential for those in need and privatelong-term care is expensive, there is a minimum level of expenditure needed to obtain private LTC, i.e.,c ≥ χ for those not using government care. In the model, government-provided care is loosely based onthe institutions of Medicaid. If a consumer needs LTC and uses government care, the government providesc = ψG. The value ψG parameterizes the consumer’s value of public care, since that parameter essentiallydetermines the utility of an individual who needs LTC and chooses to use government care.

In addition to affecting health costs and survival probabilities, health status affects preferences. Thereis a health-dependent utility function, such that spending when a consumer needs LTC (s = 2) is valueddifferently than spending when a consumer does not need LTC. Utility when in need of LTC associated withexpenditure level c is

(θADL)−γ(c+ κADL)1−γ

1− γ.

Upon death (s = 3), the agent receives no income and pays all mandatory health costs. Any remainingwealth is left as a bequest, b, which the consumer values with a warm glow utility function:

v(b) = (θbeq)−γ (b+ κbeq)

1−γ

1− γ.

2 The Model

The choice variables are:

c ∈ [0,∞)

a ∈ [0,∞)

G ∈ 0, 1

2

The exogenous state variables are:

g ∈ male, female

s ∈ 0, 1, 2, 3 evolves according to π(s′|t, s)

t ∈ 55, 56, ..., T

y ∈(

(yk(t))Tt=1

)5

k=1

h ∼ H(t, s) with support ΩH(t, s)

The consumer’s problem is

V (a, y, t, s, h, g) = maxa′, c, G

Is 6=3 (1−G)Us(c) + βE[V (a′, y, t+ 1, s′, h′)]

+ Is 6=3 G

Us(ωG, ψG) + βE[V (0, y, t+ 1, s′, h′)]

+ Is=3v(b)

s.t.

a′ = (1−G)[(1 + r)a+ y(t)− c− h] ≥ 0

c ≥ χADL if (G = 0 ∧ s = 2)

c = ψG if (G = 1 ∧ s = 2)

c = ωG if (G = 1 ∧ (s = 0 ∨ s = 1))

b = max(1 + r)a− h′ , 0

Us(c) = Is∈0,1c1−γ

1− γ+ Is=2 (θADL)−γ

(c+ κADL)1−γ

1− γ

v(b) = (θbeq)−γ (b+ κbeq)

1−γ

1− γ.

The G variable is a choice variable that is a function of the consumer’s states, i.e., G(a, y, s, j, h) ∈ 0, 1.Note that we can suppress the policy function G in favor of a max operator.

V (a, y, t, s, h, g) = Is 6=3 max

maxa′,c

Us(c) + βE[V (a′, y, t+ 1, s′, h′, g)] , Us(ωG, ψG) + βE[V (0, y, t+ 1, s′, h′, g)]

+ Is=3v(b)

3 Computing Optimal Policies

We solve this finite horizon model using backwards induction on a discretized state space. We first computeoptimal consumer policies at age T − 1, when the continuation value at age T is simple to compute, as itis just the value of leaving a bequest. This yields optimal consumer policies and the value function at eachpoint in the state space at time T − 1. Knowing these T − 1 values, we repeat the algorithm for date T − 2,T − 3,..., t0.

We adapt to our problem an extension of the endogenous grid method (EGM) for non-concave problems,

3

building heavily on the algorithm developed in Fella (2014).2

3.1 Endogenous grid method

The basic idea behind the EGM is that, rather than solve for the next period asset level given the initialperiod asset level, it is more efficient to solve for the initial period asset level given the next period assetlevel. Since at each time we know the expected continuation value (ECV) function from backwards induction,the first order condition (FOC) can be inverted analytically, avoiding the need for computationally costlynon-linear equation solving. This speeds up computation time significantly. The solution algorithm proceedsas follows:

3.2 T-1 Problem

• Define value functions

• Apply endogenous grid method

• Check constraints and boundary solutions

• Compare to the value of government care

• Store a matrix that reports the value of V (a, y, T − 1, s, h, g) for all s ∈ 0, 1, 2, 3, h ∈ ΩH(T − 1, s),and y = (y(t))Tt=t0 . Note that · denotes discretized values, e.g., ΩH(t, s) is the discretized space ofhealth costs for each age and health state.

We start out by solving the T − 1 problem under the assumption G = 0, i.e., the consumer does not usegovernment care. We will later compare this value to the value if the consumer does use government care ina final step. Given that the next period (time T ) continuation value is v(b), the T − 1 bellman equation is

J(a, y, T − 1, s, h, g) = maxc,e,a′

Us(c) + β

∫ΩH(T,s′=3)

v(max

(1 + r)a′ − h′, 0

)dH(h′|T, s′ = 3).

Define the expected continuation value, ECVT−1(a′), as

ECVT−1(a′) := β

∫ΩH(T,s′=3)

[v((1 + r)a′ − h′

)× I(1+r)a′−h′)>0

]dH(h′|T, s′ = 3).

Then the FOCs are

d

dcT−1Us(c) =

d

da′ECVT−1(a′).

2We have also solved the model by directly calculating the value function using a constrained optimizer. The EGM coderuns at least 10 times faster, so it is our chosen algorithm. We verified in a somewhat simpler model that the constrainedoptimization code and EGM code generate the same optimal policy functions, confirming the accuracy of our modified EGMalgorithm.

4

d

dcUs=0(c) =

d

dcUs=1(c) = c−γ =

((1 + r)a+ y(T − 1)− a′ − h

)−γd

dcUs=2(c) = (θADL)−γ (c+ κADL)−γ = (θADL)−γ

((1 + r)a+ y(T − 1)− a′ − h+ κADL

)−γ.

Thus, if we can solve the above equations for a′ for any a then we will have characterized the policyfunctions for the T − 1 problem (since given a′ and s, the budget constraint provides c).

To solve for the (a, a′) pairs that solve the above problem, we proceed with the Endogenous Grid method.

1. First, construct a grid over a′, named aFinal. For each a′,i ∈ aFinal, we solve the FOC for ai

((1 + r)ai + y(T − 1)− a′,i − h

)−γ=

d

da′ECVT−1(a

′,i)

ai =dda′ECVT−1(a

′,i)1−γ − y(T − 1) + a′,i + h

(1 + r)

(θADL)−γ((1 + r)ai + y(T − 1)− a′,i − h+ κADL

)−γ=

d

da′ECVT−1(a

′,i)

ai =

1θADL

(dda′ECVT−1(a

′,i)) 1−γ − y(T − 1) + a′,i + h− κADL(1 + r)

Thus, for each a′,i ∈ aFinal we have found a point ai which satisfies the FOC. Notice that ai is afunction of idiosyncratic states, including s, h, g, and y but we suppress this for notational convenience.However, ai must be evaluated at each point of the idiosyncratic state space grid (s, h, g, y). Thesepairs (ai, a′,i) define the policy function as the mapping from the endogenous a grid (hereafter referredto as aEndogenous) to a′,i.

2. To implement our solution technique, the policy function must be defined over a standardized a grid (callit aStart) to ensure both that the grid is constant in all time periods and that the policy function is de-fined over the range of the data. Thus, we would like to define a′(aj) ∀ aj ∈ aStart. To obtain it, we spec-ify the grid aStart, and for each point aj ∈ astart we find (ai, a′,i), (ai+1, a′,i+1) ∈ aEndogenous, aFinalsuch that aj ∈ [ai, ai+1]. We then calculate a′(aj) by interpolating a′(aj) over [a′,i, a′,i+1]. This gives agrid aStart, a′(aStart). After repeating the procedure for each grid point we have computed a set ofgrids that satisfies the T − 1 FOC: aStart, a′(aStart)s,h, s ∈ 0, 1, 2, h ∈ ΩH(T − 1, s).

3. Remembering that in state 2, there is an LTC expenditure constraint (eLTC > χ), we must check to see ifthis constraint is satisfied. For each pair in aStart, a′(aStart)s=2 such that the budget constraint can besatisfied (i.e, (1+r)aStart+y(T−1)−h(T−1, 2)−χ > 0) but a′(aStart) > (1+r)aStart−1−h(T−1, 2)−χ,we set a′(aStart) = (1 + r)aStart − 1− h(T − 1, 2)− χ (or equivalently, eLTC = χ) and proceed.

For each of the above policy functions and starting grid we calculate the value function. Define J asthe value of a consumer who does not go on government care evaluated at the optimal element of thea′ grid:

J(a, y, T − 1, s, h, g, a′) = Us(c) + β

∫ΩH (T,s′=3)

v(max

(1 + r)a′ − h′, 0

)dH(h′|T, s′ = 3)

≈ Us(c) + β∑

h′i∈ΩH (T,s′=3)

v(max

(1 + r)a′ − h′i, 0

)p(h′i|T, s′ = 3)

5

where now c is implied by a′ and we have approximated the integral numerically. This yields a gridof the value function J(a, y, T − 1, s, h, g, a′) for each non-age idiosyncratic state over all starting assetvalues aj ∈ aStart. We now need to check two things. First, we check the boundary condition of whetherJ(ajStart, y, T − 1, s, h, g, a′(ajStart)) > J(ajStart, y, T − 1, s, h, g, 0) (i.e., whether an agent is better offconsuming all wealth). We need to check this point because of the kink that can occur in the valuefunction due to b = max (1 + r)a′ − h(T, s′), 0 being bounded below by 0.

4. Finally, we check whether the value of government care,

GC(a, y, T − 1, s) = Us(ωG, ψG)] + βv(0),

yields a higher utility than the previously calculated value of utility, J(aj , y, T − 1, s, h, g, a′,i) for eachaj ∈ aStart.

The value function, V , is then the max over the optimal value when G = 0 and when G = 1:

V (ai, y, T − 1, s, h, g) = max(J(aj , y, T − 1, s, h, g, a′,i) , GC(aj , y, T − 1, s)

).

Note that if the budget constraint is unable to be satisfied (i.e. (1 + r)aj + y − h(T − 1, s) < 0), itmust be that V (aj , y, T − 1, s, h, g) = GC(aj , y, T − 1, s), since cj = ejLTC = 0 implies that V (aj , y, T −1, s, h, g, a′,j) = −∞. For state 3, we define a grid

V (aj , y, T − 1, 3, h, g) = v(max

(1 + r)aj + y − h, 0

)

and join it with the grids for states 0, 1, and 2.

Thus, we have calculated a grid of the value function value for an underlying asset grid for the T −1 problemfor each health state. This will be used to solve the T −2 problem as we continue to the backwards inductionprocedure.

3.3 T − j Problem, j ≥ 2

The solution to the T − 1 problem is taken as given and used to solve the T − 2 problem, which is then usedto solve the T − 3 problem recursively until reaching age t0. We use the previously calculated T − j+ 1 valuefunction V (a, y, T − j + 1, s, h, g) over a grid of (a, y, s, h, g) ∈ [amin, amax]× ((yk(T − j)))5

k=1×0, 1, 2, 3×ΩH(T − j+ 1, s)×male, female to define the expected continuation value (ECV) needed to solve the T − jproblem. The algorithm proceeds in a similar manner to that developed for the T − 1 period problem.

• Define value functions using continuation value from T − j + 1 problem

• Apply endogenous grid method to non-concave problems

• Check constraints and boundary solutions

• Compare to value of government care

• Store a matrix that reports the value of V (a, y, T − j, s, h, g) for all s ∈ 0, 1, 2, 3, h ∈ ΩH(T − j, s),(yk(T − j))5

k=1, g ∈ male, female

6

The T − j expectation of the T − j + 1 value function, i.e., the expected continuation value ECVT−j , is

ECVT−j(a′, s) = β

∑s′∈0,1,2,3

π(s′|T − j, s)∫

ΩH(T−j+1,s)V (a′, y, T − j + 1, s′, h′)dH(h′|T − j + 1, s′)

≈ β∑

s′∈0,1,2,3

π(s′|T − j, s)∑

h′i∈ΩH(T−j+1,s′)

V (a′, y, T − j + 1, s′, h′i)[H(h′i|T, s′)−H(h′i−1|T, s′)

]and thus the value function at time T − j if the consumer does not go on government care is

J(a, y, T − j, s, h, g) = maxcT−j,eT−j ,a′

Us(c) + ECVT−j(a′, s).

The FOC is

d

dc[Us(c)] =

d

da′[ECV (a′, s)

]=

d

da′[βE[V (a′, y, j + 1, s′, h′)]

]Since d

dc [Us(c)] = Is=0,1c−γ + Is=2θADL(c+ κADL)−γ , if we know dda′ [βE[V (a′, y, j + 1, s′, h′)]] then we

can substitute in c = (1+r)a+y−a′−h and invert the FOC to obtain initial period asset holdings. Doing thisover a grid of final assets aFinal will yield an endogenous grid of initial asset holdings, aEndogenous that implythe policy functions (aiEndogenous, a

iF inal). This process must be repeated for all s ∈ 0, 1, 2, 3 and h ∈ ΩH(T−

j, s) with a grid being calculated for each point. Analytical expressions of dda′ [βE[V (a′, y, j + 1, s′, h′)]] are

included in Section 4.If the expected continuation value βE[V (a′, T − j+ 1, s′, h′, g)] is globally concave, then the FOC will be

both necessary and sufficient, and thus the computed endogenous grid, specifically the pairs (aiEndogenous, aiF inal),

characterize optimal consumer behavior. If the value function V is not globally concave, then the FOC willonly be necessary but not sufficient. The model in this paper features a non-concave value function due tokinks induced by the government care options and expenditure constraints. Thus, the algorithm needs to beadapted to ensure that the optimal policies computed with the EGM correspond to the optimal policies ofthe model.

To illustrate the complication, suppose that the ECV is not concave. Then the derivative of the ECVmight look similar to the plot in Figure 1 taken from Fella (2014).

There is no longer a one-to-one mapping from dda′ [βE[V (a′, y, T − j + 1, s′, h′, g)]] to d

dc [Us(c)] (where weare adapting our notation) in the regions of a′ ∈ a′2, ...a′9. For any a′i in this set, there exists at least onea′k such that a′i 6= a′k but d

da′ [βE[V (a′i, y, T − j + 1, s′, h′, g)]] = dda′ [βE[V (a′k, y, T − j + 1, s′, h′, g)]]. Thus,

the inverted FOC pins down a non-singleton set of admissible next period wealth levels a′k or a′i associatedwith initial period asset level a.

To address this concern, we first partition the a′Final into concave and non-concave regions. In the concaveregion, the bijective mapping allows for straightforward calculation of optimal (a, a′) pairs as discussed above.In the non-concave region, we discard (a, a′Final) that are not a pair of initial assets and optimal savings.That is, we check to see for an initial period asset a, whether the policy a′k or a′i would optimize the valuefunction. We do this by finding the a′ that is the global arg max, saving that (a, a′) pair to be used ininterpolation, and discarding other pairs.

The algorithm is detailed below. Note that here we have suppressed the state dependence notation of

7

Figure 1: Non-concave Value Function FOC (Fella 2014)

the savings policy function and simply expressed it as a′. The below process must be repeated over theentire grid, (y, s, h, g) ∈ (yk(T − j))5

k=1 × 0, 1, 2, 3 × ΩH(T − j, s)× g ∈ male, female to obtain the pairsreferenced in step 3.

1. We calculate the lower and upper regions of our asset grid where there is a unique mapping fromdda′βE[V (a′, y, T − j + 1, s′, h′)] to d

dc [Us(c)] and thus to initial assets. These regions are determinedby the locally concave lower and upper tails of the value function and are labeled GC .

(a) In the above figure, the regionGC is characterized by all ai such that dda′ [βE[V (ai, y, T − j + 1, s′, h′, g)]] >

vmax or dda′βE[V (ai, y, T − j + 1, s′, h′, g)] < vmin, i.e., ai < a′2, ai > a′9

(b) In the GC region, the standard EGM can be applied without adjustment because the uniquemapping implies the FOC is both necessary and sufficient

We calculate this region by calculating the derivative of ECVT−j for every point a′i ∈ aFinal, yieldingdda′ [ECVT−j(a

′, s)]. We then find the regions such that

(a) dda′ [ECVT−j(a

′,i, s)] < dda′ [ECVT−j(a

′,j , s)] ∀j < i and

(b) dda′ [ECVT−j(a

′,i, s)] > dda′ [ECVT−j(a

′,j , s)] ∀j > i.

We then label the minimum i for which the first condition holds i, and label the maximum i for whichthe second condition holds as i. Finally, GC contains aj ∈ aFinal s.t. j > i or j < i. Since thegovernment care option generates a value function that is not concave for low asset values, we willgenerally only find a concave upper region.

To complete the partition of a′Final, we store all a′ in the non-concave region where there is not a unique

mapping between dda′βE[V (a′, y , j + 1 , s′, h′)] and d

dc [Us(c)] in GNC .

8

2. For each aiF inal ∈ GNC , we check pairs (aiEndogenous, aiF inal) to ensure that aiF inal are global arg max

for the corresponding aiEndogenous state. In GNC there may be multiple a′Final mapping to a singleaEndogenous. Thus, we must find the unique a′Final that is the optimal saving policy if an individualwere to start the period with aEndogenous assets. To find the optimal savings policy, we evaluate theobjective function for each a′i ∈ GNC to find

a′i = arg maxa′∈GNC

Us((1 + r)ai + y(T − j)− a′ − h(j, s)) + ECVT−j(a′, s).

We store the (ai, a′i) pair if this condition holds and discard the suboptimal pairs.

(a) In the above figureGNC would contain ai ∈ [a′2, a′9] or equivalently vmin < d

da′ [βE[V (ai, y, j + 1, s′)]] <

vmax.

3. The remaining optimal pairs are

(aiEndogenous, aiF inal)

for each element of the idiosyncratic state

space. Just as in period T − 1, we would like to have the policy functions defined over the same grid ineach period, denoted in the T − 1 problem as aStart. To obtain the policy function over each point inaStart, we use a combination of interpolation and grid search methods over the

(aiEndogenous, a

iF inal)

grids for each idiosyncratic state to obtain the optimal policy function pairs (aStart, a′(aStart)) asdescribed in the following discussion.

If policy functions are continuous, standard algorithms can interpolate over the the obtained policyfunctions to obtain approximations that are only subject to interpolation errors. In our model, standardinterpolation methods fail due to discontinuities in the policy functions a′(a, y, t, s, h, g). A discrete jump inthe policy function creates complications, as interpolation will assign an off-grid policy function value basedon a continuous functional approximation of the known points on the grid. In the presence of discontinuities,it may be that the optimal policy is better approximated only using points below or above the off-grid point,depending on which side of the discrete jump it falls. These discontinuities emerge because the value functionis bimodal in a′, as illustrated in Figure 1 of the main paper.

The two modes represent a case where the consumer has enough money to self insure against usinggovernment care in the healthy states, but not in the LTC state. For consumers with more wealth, the highervalue mode is the one associated with a higher level of savings because the consumer has enough wealth thathe is better off saving to self insure against future health and LTC shocks instead of relying on governmentprovided care. At lower levels of current wealth, the consumer is better off consuming more today and onlyinsuring against the potential use of state 0 and 1 government care, but expecting to use government carein a future LTC state. (A similar discrete jump in the policy function can occur where the consumer onlyself insures against government care in state 0 and plans to use government care should they transition tostate 1 or 2 next period). At the threshold where the value function jumps from the lower to the highermode (in main paper Figure 1, at approximately a = 68), there will be a discrete jump in the policy function(although the value function will remain continuous).

Such jumps are a potential problem for interpolation. Suppose in main paper Figure 1 we waned tofind the savings policy when a = 68, but we only observed the (approximate) saving policies (64, 38) and(72, 57). Naive interpolation would indicate a policy pair of (68, 47.5). However, an agent in this state wouldstrictly prefer a′ = 40 or a′ = 54 to a′ = 47.5. This example demonstrates both the presence and the

9

problems associated with this type discrete jump in the optimal policy function. Interpolation over regionsthat include jumps would lead to incorrect solutions.

The above illustrations also provide insight into why some points are only local maxima and are thusdiscarded in step 2 of the algorithm above. The points that we discard tend to be clumped together becauseof saving for lumpy costs. The local max in a value function correspond to self insuring for a relevant risk,or to forgo insurance and consume more. Thus, there exist asset levels at which the self insurance motivebecomes dominant, and the local maximum associated with higher immediate consumption becomes lowerthan the other local maximum associated with self insurance. At this point, the optimal savings decisionswitches, generating the discrete jumps in the savings policy. These saving motives tend to be monotonic: ifan agent with a1 initial assets should self insure against health state s, then an agent with a2 initial assetsshould also self insure against the same health state. Thus, when a local max is the highest local max, it isthe highest local max for a connected region. This is the key insight that allows us to propose the followingalternative to naive interpolation.

• Start from the grid (aiEndogenous, aiF inal) that has been restricted to globally optimal pairs. For each

element of the grid indexed by k, i.e., (akEndogenous, akF inal) ∈

(aiEndogenous, a

iF inal)

:

– If (ak−1Endogenous, a

k−1Final) ∈

(aiEndogenous, a

iF inal)

then we interpolate for all am ∈ aStart in

the region am ∈ [ak−1Endogenous, a

kEndogenous]. This yields a′(am) ∈ [ak−1

Final, akF inal]. Similarly, if

(ak+1Endogenous, a

k+1Final) ∈

(aiEndogenous, a

iF inal)

then we interpolate for all am ∈ aStart in the

region am ∈ [akEndogenous, ak+1Endogenous]. This yields a

′(am) ∈ [akF inal, ak+1Final].

– If (ak−1Endogenous, a

k−1Final) /∈

(aiEndogenous, a

iF inal)

, then for any am ∈ aStart with

am ∈ [ak−1Endogenous, a

kEndogenous], we do not interpolate. Instead, we run a grid search to find a′(am).

Similarly, if (ak+1Endogenous, a

k+1Final) /∈

(aiEndogenous, a

iF inal)

, then for any am ∈ [akEndogenous, a

k+1Endogenous]

we do not interpolate. Instead, we run a grid search to find a′(am).

The above steps ensure there is no interpolation over a point of discontinuity. If we haven’t discardedany grid points for a region on the aFinal grid that is connected, then the policy function is continuous forthe range defined by this connected region. Thus, we are able to interpolate over the corresponding domain.If we have discarded points, then it could imply a discontinuity of the policy function in this range. Thus,we do not interpolate over the corresponding domain. As always, this is repeated for each y, s, h, g.

Finally, notice that the above fix may be foregoing interpolation in favor of a grid search method (orconstrained optimization) when it doesn’t necessarily need to. If a region of the domain for which a policyfunction is continuous is small (i.e., has few endogenous grid points), we may be ignoring a large portionof this region that could be interpolated over, and instead are executing grid searches. The grid search willreturn an optimal policy (up to degree of tolerance specified in the grid). Technically the entire model couldbe solved with grid searches on a very fine grid and yield precise solutions. Such a grid search, however,would be (prohibitively) computationally costly. Favoring a grid search method over interpolation in regionswith potential discontinuities is a decision favoring a slower but more accurate algorithm in regions wherecomplications are likely to exist.

To present a brief illustrative example, suppose we started out with the following grid (aEndogenous,aFinal)

for a given y, s, h and g:

10

aEndogenous a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12

aFinal a′1 a′2 a′3 a′4 a′5 a′6 a′7 a′8 a′9 a′10 a′11 a′12

and suppose that after discarding non global arg max we are left with

aEndogenous a1 a3 a4 a6 a8 a9 a10 a11 a12

aFinal a′1 a′3 a′4 a′6 a′8 a′9 a′10 a′11 a′12.

If ai ∈ aStart such that ai ∈ [a3, a4] or ai ∈ [a8, a12], then we interpolate over the (aEndogenous,aFinal) grid. Ifai ∈ [0, a3), ai ∈ (a4, a6), or ai ∈ (a6, a8) then we do not interpolate, and instead perform a grid search.

Having defined the decision rule for when to use interpolation and when to use grid search methods, we canuse the appropriate method starting from aStart to obtain the policy function pairs (aStart, a

′(aStart)). Thiscompletes the description of how to implement point 3 of the algorithm.

4. Having defined the candidate optimal policy functions, we must check that they do not violate con-straints. Notice that by constructing the aFinal grid such that ai > 0 ∀ai ∈ aFinal the a′ > 0 constraintis satisfied by construction. For the policy function pair (a0

Endogenous, 0) if a0Endogenous > 0, then we

know that a0Endogenous is the point at which our non-negativity constraint binds. In the interpolation

procedure detailed above, the policy function a′(ai) = 0 for all ai < a0Endogenous. Thus, all that needs

to be checked is the LTC expenditure constraint: ciT−j ≥ χ if s = 2. If this allocation is not budgetfeasible, then we know that G = 1 and know the agent will use government care. If this constraint isbudget feasible but not satisfied by the previously calculated policy function a′(aistart), we run a gridsearch to find the optimal a′(aistart) ∈ [0 , (1 + r)aistart + y − h − χ]. It is likely that the grid searchwill return a value of a′(aistart) = (1 + r)aistart + y − h − χ or equivalently eLTC = χ, but because ofthe non-convexities (and inability to ensure monotonicity of our savings policy function, we err on theside of caution and execute a grid search over the entire budget feasible policy set.

This completes the computation of the optimal policy functions for the case of G = 0.

5. The optimal policy functions, a′,i(ai) for each point ai ∈ aStart for each element of the idiosyncraticstate space, are then used to compute the value function without government care (G = 0), J(ai, y, T −1, s, h, g, a′,i). This is done over the entire aStart grid (for each state (y, s, h, g)). The value of usinggovernment care (G = 1) is

GC(ai, y, T − j, s) = Us(ωG, ψG) + ECVT−j(0, s)

for each grid point. Finally, we calculate the final value function is

V (ai, y, T − j, s, h, g) = max(J(ai, y, T − j, s, h, g, a′,i) , GC(ai, y, T − j, s)

).

if the the budget constraint can be satisfied, and

V (ai, y, T − h, s, h) = GC(ai, y, T − j, s)

11

if positive consumption is not budget feasible due to health-care costs.

This algorithm computes the value function value for each s ∈ 0, 1, 2 over the fixed grid aStart. The valueof death when (s = 3 is v(max0, (1 + r)ai − h). Combining these generates a grid containing the valuefunction associated with each element of the idiosyncratic state variable grid (including assets (a)) for theT−j problem. This algorithm can be repeated for T−j problem incrementing j and iterating until T−j = t0.

4 Deriving the FOC and ECV

In this section we derive the FOC and analytical expressions for the continuation value for t ≤ T − 2

We proceed by deriving the FOCs assuming that the consumer chooses not to use government care, i.e.,G = 0. As is outlined above, the optimal choice of G is determined by comparing the value for G = 0

computed below to the value associated with G = 1. This problem, for j ≥ 2, is defined by

J(a, y, T − j, s, h, g) = maxa′,c

[Us(c)] + βE[V (a′, y, T − j + 1, s′, h′, g)].

Substituting in the expected continuation value yields

ECVT−j(a′, s) = β

∑s∈0,1,2,3

π(s′|s)

×∫

ΩH|T−j+1,s′

V (a′, y, T − j + 1, s′, h′, g)dH(h′|s′, T − j + 1)

= βE[V (a′, y, T − j + 1, s′, h′, g)].

Denoting Lagrange multipliers by λ in parenthesis, the optimization problem can be expressed as

J(a, y, T − j, s, h, g) = maxc,a′

Us(c) + ECVT−j(a′, s)

st.

(1 + r)a+ y(T − j)− a′ − c− h ≥ 0 (λT−jBC )

a′ ≥ 0 (λT−jW )

c ≥ χ if (G = 1 & s = 2). (λT−jχ )

4.1 FOCs

The FOCs for the given states are

s ∈ 0, 1

FOC(c):

d

dc[Us(c)]− λT−jBC = 0

FOC(a′):d

da′[ECVT−j(a

′, s)]− λT−jBC + λT−jW = 0.

12

Thus, there are 4 unknowns ( λT−jBC λT−jW , c, a′) and 4 equations:

d

dc[Us(c)]− λT−jBC = 0

d

da′[ECVT−j(a

′, s)]− λT−jBC + λT−jW = 0

λT−jBC ×((1 + r)a+ y(T − j)− a′ − c− h

)= 0

λT−jW × a′ = 0.

s = 2

FOC(eLTC)

d

dc[Us=2(c)]− λT−jBC + λT−jχ = 0

FOC(a′)d

da′[ECVT−j(a

′, s)]− λT−jBC + λT−jW = 0.

Thus, there are 5 unknowns ( λT−jBC λT−jW , λχ, c, a′) and 5 equations:

d

dc[Us=2(c)]− λT−jBC + λT−jχ = 0

d

da′[ECVT−j(a

′, s)]− λT−jBC + λT−jW = 0

λT−jBC ×((1 + r)a+ y(T − j)− a′ − c− h

)= 0

λT−jW × a′ = 0

λT−jχ × (c− χ) = 0.

4.1.1 Derivative of ECV

The endogenous grid method requires inverting the following relationships to obtain a value for expenditurec which can then yield initial assets a using the budget constraint

d

dc[Us(c)] =

d

da′[ECVT−j(a

′, s)].

The expression dda′ [ECVT−j(a

′, s)] can be written as

13

d

da′[ECVT−j(a

′, s)]

=d

da′β

∑s′∈0,1,2,3

π(s′|s)

∫ΩH|T−j+1,s′

V (a′, y, T − j + 1, s′, h′, g)dH(h′|s′, T − j + 1)

= β∑

s′∈0,1,2,3

π(s′|s)×

∫ΩH|T−j+1,s′

d

da′[V (a′, y, T − j + 1, s′, h′, g)

]dH(h′|s′, T − j + 1).

Assuming G = 0, substitute in the optimal policy function c.

For s′ ∈ 0, 1, by the envelope theorem,

d

da′[V (a′, y, T − j + 1, s′, h′, g)

]=

d

da′[Us(c

′) + βV (a′′, y, T − j + 2, s′′, h′′, g)

+λT−j+1BC ((1 + r)a′ + y′ − a′′ − c′ − h′)

+λT−j+1W (a′′)]

= λT−j+1BC (1 + r)

Plugging back in the equilibrium expression for λT−j+1BC shows that

d

da′[V (a′, y, T − j + 1, s′, h′, g)

]=

d

dc

[Us(c

′)]

(1 + r).

For s′ = 2,

d

da′[V (a′, y, T − j + 1, s′, h′, g)

]=

d

da′[Us=2(c′) + βV (a′′, y, T − j + 2, s′′, h′′, g)

+λT−j+1BC ((1 + r)a′ + y′ − a′′ − c′ − h′))

+λT−j+1W (a′′)

+λT−j+1χ (c′ − χ)]

= λT−j+1BC (1 + r).

Using the envelope theorem and substituting in λT−j+1BC = d

dc [Us=2(c′)] + λT−j+1χ yields

d

da′[V (a′, y, T − j + 1, s′, h′)

]= (1 + r)× d

dc

[Us=2(c′)

]+ (1 + r)λT−j+1

χ .

Since λχ = 0 if the constraint is not binding by the complementary slackness condition, the above simplifiesto

d

da′[V (a′, y, T − j + 1, s′, h′)

]=

(1 + r)× ddcUs=2(c′) if c > χ

(1 + r)× ddcUs=2(c′) + (1 + r)λT−j+1

χ if c = χ.

14

It remains to solve for λT−j+1χ . Since c′, a′′ are known due to backwards induction, the system reduces to

d

dc

[Us=2(c′)

]− λT−j+1

BC + λT−j+1χ = 0

d

da′[ECVT−j(a

′′, s′)]− λT−j+1

BC + λT−j+1W = 0

λT−j+1BC ×

((1 + r)a′ + y(T − j + 1)− a′′ − c′ − h′

)= 0

λT−j+1W × a′′ = 0

λT−j+1χ × (c− χ) = 0,

yielding for the three unknown multipliers:

− d

deLTC

[Us=2(c′)

]+ λT−jBC = λT−j+1

χ

d

da′[ECVT−j(a

′′, s′)]

+ λT−jW = λT−jBC .

Given that utility is strictly increasing in c or eLTC for the relevant health state, the budget constraint holdswith equality and λBC = 0 ∀ t.Thus,

λχ = − d

dc

[Us=2(c′)

]and

d

da′[V (a′, y, T − j + 1, s′ = 2, h′)

]=

(1 + r)× ddc [Us=2(c′)] if c > χ

0 if c = χ.

This expression provides a complete characterization of the continuation values as follows:

d

da′[V (a′, y, T − j + 1, s′ ∈ 0, 1, h′))

]= (1 + r)

d

dc

[Us(c

′)]

d

da′[V (a′, y, T − j + 1, s′ = 2, h′))

]=

(1 + r) ddc [Us=2(c′)] if c > χ

0 if c = χ

d

da′[V (a′, y, T − j + 1, s′ = 3, h′))

]=

dda′ [v((1 + r)a′ − h′)] if (1 + r)a′ − h′ > 0

0 if (1 + r)a′ − h′ ≤ 0.

Substituting the above three expressions into the below completely characterizes the derivative of theexpected continuation value. Thus, we have derived the following Euler equations:

s ∈ 0, 1 =⇒

15

d

dc[Us(c)] = β

∑s′∈0,1,2,3

π(s′|s)∫

ΩH|T−j+1,s′

d

da′[V (a′, y, T − j + 1, s′, h′, g)

]dH(h′|s′, T − j + 1) + λT−jW

s = 2=⇒

d

dc[Us=2(c)] = β

∑s′∈0,1,2,3

π(s′|s)∫

ΩH|T−j+1,s′

d

da′[V (a′, y, T − j + 1, s′, h′, g)

]dH(h′|s′, T − j + 1) + λT−jW − λT−jχ .

To handle the case in which the consumer chooses to use government care realize that for each period T−j+1

and for each element of the idiosyncratic state grid, there exists an aT−j+1 such that if aT−j+1 < aT−j+1

then G = 1. If G = 1, then

d

da′[V (a′, y, T − j + 1, , s, h, g)

]=

d

da′[[Us(ωG, ψG)] + βE[V (0, y, j + 1, s′, h′, g)]

]= 0,

since a marginal change in wealth does not affect the continuation value. Thus,

d

da′[ECVT−j(a

′, s)]

= β∑

s′∈0,1,2,3

π(s′|s)× IaT−j+1>aT−j+1s

∫ΩH|T−j+1,s′

d

da′[V (a′, y, T − j + 1, s′h′, g)

]dH(h′|s′, T − j + 1).

4.1.2 Summary of FOCs

The above work derived analytic expressions for the model objects that characterize the FOCs of an agent’sdecision problem in period t ≤ T − 2. Using the utility function associated with health state s, the FOCsare given by

d

dc[Us(c)] =

d

da′[ECVT−j(a

′, s)]

In the above sections we derived analytical expressions for ddat+1

[ECVt(at+1, s)]). These expressions arepresented below, and may be substituted into the body of this document to complete the specification of thealgorithm designed to compute optimal policies for the model in the main paper.

s ∈ 0, 1 =⇒

d

da′[ECVT−j(a

′, s)]

= β∑

s′∈0,1,2,3π(s′|s)× Ia′>a′s

∫ΩH|T−j+1,s′

d

da′[V (a′, y, T − j + 1, s′, h′, g)

]dH(h′|s′, T − j + 1) + λT−jW

s = 2 =⇒

d

da′[ECVT−j(a

′, s)]

= β∑

s′∈0,1,2,3π(s′|s)× Ia′>a′s

∫ΩH|T−j+1,s′

d

da′[V (a′, y, T − j + 1, s′, h′, g)

]dH(h′|s′, T − j + 1) + λT−jW − λT−jχ .

with the derivatives of the value function given by

16

d

da′[V (a′, y, T − j + 1, s′ ∈ 0, 1, h′, g)

]= (1 + r)

d

dc

[Us(c

′)]

d

da′[V (a′, y, T − j + 1, s′ = 2, h′, g)

]=

(1 + r) ddc [Us(c′)] if c > χ

0 if c = χ

d

da′[V (a′, y, T − j + 1, s′ = 3, h′, g)

]=

dda′ [v((1 + r)a′ − h′) if (1 + r)a′ − h′ > 0

0 if (1 + r)a′ − h′ ≤ 0.

To apply EGM to the above expressions, we will make the appropriate substitutions and ignore multipliers.Having solved the problem, we will then go back and check if the constraints are satisfied. If they are, thenwe have shown that the Lagrange multipliers are 0, as we assumed. If not, we impose the constraints, andproceed as before.

References

Ameriks, J., J. Briggs, A. Caplin, M. D. Shapiro, and C. Tonetti (2017): “Long-Term-Care Utility andLate-in-Life Saving,” Vanguard Research Initiative Working Paper.

(2018): “The Long-Term-Care Insurance Puzzle: Modeling and Measurement,” Vanguard Research InitiativeWorking Paper.

Fella, G. (2014): “A Generalized Endogenous Grid Method for Non-Smooth and Non-Concave Problems,” TheReview of Economic Dynamics, 17(2), 329–344.

17