Lecture notes on:

Premiums and benefits in Markov models

Ronnie Loeffen

November 28, 2014

Contents

1 Discrete time 1

1.1 Types of payments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Discounting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Expected present values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Reserves and the equivalence principle . . . . . . . . . . . . . . . . . . . . . 5

2 Continuous time 8

2.1 Sojourn and transition payments . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Discounting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 Expected present values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.4 Reserves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Multiple lives 14

Exercises 17

i

1 Discrete time

1.1 Types of payments

Insurance contracts or policies are made between an insurance company and the policy-holder(s). In such a contract the policyholder agrees to pay an amount or series of amountsto the insurance company, called premiums. In return the insurance company agrees to payan amount or amounts, called benefits, to the policyholder on the occurrence of a pre-specifiedevent. In the insurance policies that we will consider, we assume that the status of the pol-icyholder is partitioned into a finite number of possible states 1, 2, . . . , d with d ∈ N andthat the sizes and occurrence times of the payments just depend on the particular patternof states that the policyholder goes through during the contract period.

In this section we work in a discrete time framework, where time is split into a numberof periods (of the same size) and for which payments are made only at the beginning/end ofa period (note that the end of one period is the beginning of the next period). In particularwe work with the time line as depicted in Figure 1. We denote by Xn the status of thepolicyholder at time n, where n = 0, 1, 2, . . .. Note that, in accordance with the time line inFigure 1, time n corresponds to the beginning of period n + 1 (which is the same as the endof period n if n ≥ 1).

210

period 1 period 2 period n

n−1 n

Figure 1: A continuous time line starting at time 0 and split up in periods of length 1.

For now we distinguish between two types of payments, namely sojourn payments andtransition payments. A sojourn payment is a payment which is made when the policyholdervisits a particular state (note that sojourn means visit), whereas a transition payment is apayment made when the policyholder transitions from one state to another state. We denoteby βi(n) the size of the sojourn payment made at time n from the insurance company tothe policyholder when the policyholder is in state i at time n, where we understand thatβi(n) < 0 corresponds to a premium and βi(n) > 0 corresponds to a benefit. Further, wedenote by bij(n) the transition payment made at time n from the insurance company to thepolicyholder when the policyholder transitions from state i at time n − 1 to state j at timen. We will write ~β(n) = (β1(n), . . . , βd(n)) for the vector consisting of sojourn payments inperiod n and we will write B(n) for the matrix of transition payments from period n − 1 to

1

n, i.e. the d × d-matrix whose (i, j)the entry equals bij(n). If the payments do not depend

on n, we will simply write βi, bij , ~β and B. Benefit payments of sojourn type are known alsoas annuities, whereas benefit payments of transition type are known as assurances.

Example 1. Consider an insurance policy in which there are three possible states, 1:healthy,2:sick and 3:dead. We assume time is measured in years here. Further, it is agreed that abenefit of 1,000 pounds is paid out at the beginning of the year whenever the policyholder issick at that time and a benefit of 10,000 pounds is paid out at the end of the year of death

of the policyholder. Then we have ~β = (0; 1,000; 0) and B =(

0 0 10,0000 0 10,0000 0 0

)

. Suppose now that

in the next years the status of the policyholder evolves as in Table 1. Then the total amount

time n 0 1 2 3 4 5 6 7 8 9 10status Xn h h s s h s h d d d d

Table 1: Table corresponding to Example 1.

of benefits the insurance company will have to pay during those 10 years is 13,000 pounds.

Remark 2. At first sight it seems that a sojourn payment resulting from the policyholdervisiting state i can be interpreted as a transition payment which takes place when thepolicyholder transition to state i from any state in the state space. However, there is a smallissue with this observation when we consider the first period. In the first period a sojournpayment can take place but a transition payment cannot since no transition takes place atthe start of the policy. In continuous time the distinction between a sojourn and a transitionpayment will be much clearer.

1.2 Discounting

When comparing payments made at different time points, one should take into account thetime value of money. If you receive 1000 pounds today and you have to pay back 1000pounds exactly two years later, then on the first 1000 pounds you can earn interest at arate of say 2% per year, so that two years later the first 1000 pounds has accumulated to1000∗(1 + 0.02)2 = 1040.4 and so after you pay back the 1000 pounds, you have made aprofit. On the other hand, if we want to find out how much worth today, let’s denote thisnumber by x, is the 1000 pounds that we have to pay back in two years time, then we havex ∗ 1.022 = 1000 and so x = 1000 ∗ (1/1.02)2 = 961.17. We say that 961.17 is the presentvalue or discounted value of the 1000 pounds that we have to pay back in two years time. Inother words, if we have now 961.17 pounds, then this sum of money will have accumulatedto exactly 1000 pounds in two years time.

With r denoting the (constant) interest rate per period, we define the accumulation factor(per period) by 1 + r, the discount factor (per period) by 1/(1 + r) and the discount rate(per period) by 1 − 1/(1 + r) = r/(1 + r). Note that if r = 0.02, then the discount rate is

2

given by 0.02/1.02 = 0.0196, so the interest rate and the discount rate are not equal (unlessr = 0).

If we go back to Example 1 and assume that the interest rate is 2% per year, then thepresent value at the beginning of year 1 (i.e. at time 0) of the total amount of benefits paidout is given by

1000 ∗ v2 + 1000 ∗ v3 + 1000 ∗ v5 + 10000 ∗ v7 = 11514.82,

with v = 0.9804 denoting the discount factor.

1.3 Expected present values

The insurance company would like to know at the start of the policy how much benefits itwill have to pay out (and how much premium it will receive) during the contract period.Since in reality the policyholder moves in a random fashion through the states over time, itis impossible to determine precisely the future status of the policyholder and thus the exact(discounted) value of the benefits that the insurance company will have to pay. Thereforethe best thing to do is to set up some stochastic model of the status of the policyholderwhich on the one hand is realistic enough and on the other hand is simple enough so thatone can compute various quantities of interest. Since in a stochastic model the status of thepolicyholder is random, the corresponding (discounted) value of the sum of the sojourn andtransition payments will be a random variable. We might wish to work with a stochasticmodel that allows us to compute the distribution of this random variable, but the least weshould aim for in this direction is a model that allows us to compute the expected valueof this random variable. In this section we will model the status of the policyholder as a(discrete time) Markov chain as in such a model we can quite easily compute the expectedpresent value of the (random) premiums and benefits.

To this end, let X = {Xn : n ≥ 0} be a Markov chain with state space {1, . . . , d}.Recall that Xn models the status of the policyholder at time n. We denote by P(m, n) thed × d-matrix whose (i, j)th entry equals the transition probability pij(m, n) := Pr(Xn =

j|Xm = i). Let ~β(n) and B(n) be respectively, the vector of sojourn payments and thematrix of transition payments at time n. Recall that v denotes the (constant) discountfactor per period. The following theorem provides a formula for the computation of theexpected present value of the sojourn and the transition payments.

Theorem 3. Let 0 ≤ m ≤ n. Then given Xm = i, the expected present (at time 0) value(also called the actuarial value) of the sojourn payments made in the time interval [m, n] isgiven by

E

[

n∑

l=m

vlβXl(l)

∣

∣

∣

∣

∣

Xm = i

]

=

n∑

l=m

vld∑

j=1

βj(l)pij(m, l) (1)

3

and the expected present value of the transition payments made in the time interval (m, n] isgiven by

E

[

n∑

l=m+1

vlbXl−1Xl(l)

∣

∣

∣

∣

∣

Xm = i

]

=n∑

l=m+1

vld∑

k=1

d∑

j=1

bjk(l)pij(m, l − 1)pjk(l − 1, l). (2)

Here we make the usual convention that∑n

l=r · = 0 if r > n.

Note that βXl(l) is the random variable that equals

∑dj=1 βj(l)1{Xl=j} and bXl−1Xl

(l) is

the random variable that is equal to∑d

j=1

∑dk=1 bjk(l)1{Xl−1=j,Xl=k}.

Proof. Since it is allowed to switch finite sums and expectation, we have

E

[

n∑

l=m

vlβXl(l)

∣

∣

∣

∣

∣

Xm = i

]

=n∑

l=m

vlE [βXl

(l)|Xm = i]

=n∑

l=m

vld∑

j=1

βj(l)Pr(Xl = j|Xm = i)

=n∑

l=m

vld∑

j=1

βj(l)pij(m, l),

which proves the first identity. Similarly, we have for the second

E

[

n∑

l=m+1

vlbXl−1Xl(l)

∣

∣

∣

∣

∣

Xm = i

]

=n∑

l=m+1

vlE[

bXl−1Xl(l)|Xm = i

]

=

n∑

l=m+1

vl

d∑

j=1

d∑

k=1

bjk(l)Pr(Xl−1 = j, Xl = k|Xm = i)

=

n∑

l=m+1

vl

d∑

j=1

d∑

k=1

bjk(l)Pr(Xl−1 = j|Xm = i)

× Pr(Xl = k|Xl−1 = j)

=n∑

l=m+1

vld∑

k=1

d∑

j=1

bjk(l)pij(m, l − 1)pjk(l − 1, l),

whereby we have used the Markov property (see Theorem 2.2 in other lecture notes) in thethird equality. �

Returning to Example 1, suppose that we model the health status of the policyholder bya time homogeneous Markov chain {Xn : n ≥ 0} with transition matrix

P =

0.8 0.15 0.050.5 0.4 0.10 0 1

.

4

With interest rate equal to 2%, the expected present value (at the beginning of year 0) ofthe total benefits paid in the first 10 years (which is understood here to include the possibletransition payment made at time 10 but excluding the possible sojourn payment made attime 10) given that the policyholder in the beginning of year 1 is healthy, is given by

1,000 ∗

9∑

l=0

(0.9804)l(Pl)12 + 10,000 ∗

10∑

l=1

(0.9804)l[

(Pl−1)11 ∗ p13 + (Pl−1)12 ∗ p23

]

,

whereby pij is the (i, j)th entry of P and whereby we understand P0 = I, where I is theidentity matrix. Although it would take ages to do the above computation by hand, we caneasily let a computer do the work in order to get a numerical value.

1.4 Reserves and the equivalence principle

Let us assume that the policy duration of an insurance contract consists of N periods, sothat the policy starts at time 0 and ends at time N (just after potential payments at timeN are made). Then, with the notation as in the previous section, the prospective reserves attime n in state i is defined as the expected present value (EPV) of the future benefits minusthe future premiums, or more precisely,

Vi(n) := E

[

N∑

l=n

vl−nβXl(l) +

N∑

l=n+1

vl−nbXl−1Xl(l)

∣

∣

∣

∣

∣

Xn = i

]

.

Note that for our definition of the reserves at time n in state i, i.e. Vi(n), we do includethe sojourn payment made at time n but not the transition payment made at time n. Sothe vector of reserves ~V (n) = (V1(n), . . . , Vd(n)) indicates how much money the insurancecompany has to set aside at time n (depending on the status of the policyholder in thatperiod) in order to cover on average the future liabilities associated to the policy. We cancompute Vi(n) for different values of i and n by using Theorem 3, which leads to

Vi(n) =N∑

l=n

vl−nd∑

j=1

βj(l)pij(n, l) +N∑

l=n+1

vl−nd∑

k=1

d∑

j=1

bjk(l)pij(n, l − 1)pjk(l − 1, l).

Note that when we use the above expression to compute the reserves, we first need to computethe transition probabilities pij(n, l), which we can do recursively via matrix multiplicationof the one-step transition matrices P(n, n+1), n ≥ 0 (note that typically in applications weknow (possibly after first estimating them) what the one-step transition matrices are). Thensecond, we have to plug the transition probabilities pij(n, l) into the formulas (1) and (2) inorder to obtain the reserves. One might guess that it should also be possible to compute thereserves directly via a recursion involving only the one-step transition probabilities and thatsuch a recursive computation might be easier and quicker than the aforementioned two-stepprocedure in which first all the transition probabilities need to be calculated. The followingproposition shows that this is indeed the case.

5

Proposition 4. The reserves Vi(n) for all i = 1, . . . , d and 0 ≤ n ≤ N can be computed viathe following backward recursion.

Compute Vi(N) = βi(N), i = 1, . . . , d.

Compute sequentially for n = N − 1, n = N − 2, . . ., n = 1, n = 0:

Vi(n) = vd∑

j=1

pij(n, n + 1) [Vj(n + 1) + bij(n + 1)] + βi(n), i = 1, . . . , d.

Proof. From the definition of the reserves, we have

Vi(N) = E[βXN(N)|XN = i] = βi(N).

Further, for n ≤ N − 1, we have

Vi(n) =vE

[

N∑

l=n+1

vl−n−1βXl(l) +

N∑

l=n+2

vl−n−1bXl−1Xl(l)

∣

∣

∣

∣

∣

Xn = i

]

+ vE[

bXnXn+1(n + 1)∣

∣Xn = i]

+ βi(n)

=vd∑

j=1

E

[

N∑

l=n+1

vl−n−1βXl(l) +

N∑

l=n+2

vl−n−1bXl−1Xl(l)

∣

∣

∣

∣

∣

Xn+1 = j, Xn = i

]

× Pr(Xn+1 = j|Xn = i) + vd∑

j=1

bij(n + 1)Pr(Xn+1 = j|Xn = i) + βi(n)

=vd∑

j=1

E

[

N∑

l=n+1

vl−n−1βXl(l) +

N∑

l=n+2

vl−n−1bXl−1Xl(l)

∣

∣

∣

∣

∣

Xn+1 = j

]

pij(n, n + 1)

+ vd∑

j=1

bij(n + 1)pij(n, n + 1) + βi(n)

=vd∑

j=1

pij(n, n + 1) [Vj(n + 1) + bij(n + 1)] + βi(n),

where we used the law of total probability and the definition of conditional probability inthe second equality and the Markov property in the third equality. �

Typically when setting up an insurance contract, the benefit payments are first specifiedand then the premium payments are determined. These premiums are usually chosen accord-ing to some premium principle leading to a ‘fair’ premium. One very important premiumprinciple is the so-called equivalence principle, which says that the EPV (at the beginning ofthe contract) of the total premiums is equal to the EPV of the total benefits. So under theequivalence principle the premiums are chosen in such a way that the reserves Vi(0) equalszero where i is the initial and thus observable state of the policyholder. Note that the equiv-alence principle only requires that Vi(0) = 0 with i the initial status of the policyholder; it

6

might well be that Vj(0) is not equal to zero for j 6= i. In many insurance contracts, the pol-icyholder does not pay premiums during the contract period but pays only one (lump sum)premium at the start of the policy. Such a contract is called a single premium contract andthe corresponding premium is called the single premium. A single premium can be modelledwith our setup by setting βi(0) equal to the single premium, where the state i correspondsto the initial status of the policyholder.

Example 5. Consider a time inhomogeneous alive-dead model X = {Xn : n ≥ 0} wherestate 1 is alive and state 2 is dead and Xn gives the status of the policyholder at time n (i.e.at the beginning of period n + 1). Let for n ≥ 1, the one-step transition matrices be givenby

P(n − 1, n) =

(

0.99n 1 − 0.99n

0 1

)

.

Assume a policy with a duration of 4 periods where the policyholder, who is currently alive,receives 10,000 upon death and for which an initial premium of size ρ is paid. The interestrate is 2% per period.

Question: What is the value of the single premium ρ when the equivalence principle isapplied and what are the values of Vi(n) for i = 1 and n = 0, 1, 2, 3, 4?

Answer: We use the algorithm of Proposition 4 to determine the reserves. For the valuesof the parameters in the algorithm we have N = 4, the matrix of transition payments isB(n) =

(

0 10,0000 0

)

for n = 1, 2, 3, 4 and the only non-zero sojourn payment is β1(0) = −ρ.We have V1(4) = 0 and note that V2(n) = 0 for n = 1, . . . , 4 (if the policyholder is dead attime n, then no future payments (i.e. payments in the time period (n, N ] will be made).Then using the algorithm the reserves at time n = 3 are

V1(3) =0.9804 ∗ (p11(3, 4)V1(4) + p12(3, 4)[V2(4) + 10,000])

=0.9804 ∗ (1 − 0.994) ∗ 10,000 = 386.32.

For the end of period 2 we have

V1(2) =0.9804 ∗ (p11(2, 3)V1(3) + p12(2, 3)[V2(3) + 10,000])

=0.9804 ∗ (0.993 ∗ 386.32 + (1 − 0.993) ∗ 10,000) = 658.68.

One period earlier the reserves are

V1(1) =0.9804 ∗ (p11(1, 2)V1(2) + p12(1, 2)[V2(2) + 10,000])

=0.9804 ∗ (0.992 ∗ 658.68 + (1 − 0.992) ∗ 10,000) = 828.02.

Finally, at time 0 the reserves in state alive are

V1(0) =0.9804 ∗ (p11(0, 1)V1(1) + p12(0, 1)[V2(1) + 10,000]) − ρ

=0.9804 ∗ (0.99 ∗ 828.02 + (1 − 0.99) ∗ 10,000) − ρ

=901.72 − ρ.

Under the equivalence principle we must have V1(0) = 0 and thus the size of the singlepremium is ρ = 901.72.

7

2 Continuous time

In this section we look at premiums and benefits for continuous time Markov chains, alsoknown as Markov jump processes (MJP). So the set of times is now the positive halfline[0,∞).

2.1 Sojourn and transition payments

Working with the finite state space {1, . . . , d} we denote by βi(t) the sojourn payment rate(per unit of time) at time t when the policyholder is in state i. This means that if thepolicyholder is in state i from time s until time t, then the policyholder receives a paymentof

∫ t

s

βi(u)du.

We therefore assume that βi(t) is an integrable function. We further assume that βi(t) is alsoa bounded function. Similarly as before, βi(t) represents payments flowing from the insurancecompany to the policyholder, so a negative value of βi(t) means that the policyholder pays

the insurance company. The vector ~β(t) = (β1(t), . . . , βd(t)) is the vector of sojourn rates attime t.

We denote, for i 6= j, by bij(t) the size of the transition payment (again from the insurancecompany to the policyholder) when the policyholder transitions from state i to state j attime t. We assume that bij(t) is integrable and bounded. In contrast to βi(t) which is a rate,the transition payment bij(t) is a lump sum (i.e. a single sum of money). We denote byB(t) the d×d-matrix whose (i, j)th entry equals bij(t). Since in continuous time a transitioncannot occur from state i to state i, we always have that the diagonal elements of B(t) are

zero, i.e. bii(t) = 0. Similarly as before if βi(t), ~β(t), bij(t) or B(t) does not depend on t, we

will simply write respectively βi, ~β, bij or B.

Example 6. Consider a health-sick-dead policy in which the policyholder receives a benefitrate of 1,000 pounds per year during illnes and a lump sum of 10,000 pounds upon death.The health status of the policyholder over time (in years) is given in Figure 2.

Question: What is the total value of the benefits the policyholder receives during his/herlifetime?

Answer : With 1:healthy 2:sick and 3:dead, we have a sojourn rate vector of ~β =

(0; 1,000; 0) and a transition payment matrix of B =(

0 0 10,0000 0 10,0000 0 0

)

. The total value of the

benefits is then∫ 24

16

1,000du +

∫ 40

28.5

1,000du +

∫ 52

47

1,000du + 10,000 = 34,500.

8

healthy

t

sick

dead

0 16 24 28.5 40 47 52

Figure 2: Health status of the policyholder corresponding to Example 6.

2.2 Discounting

Again we have to take into account the time value of money. If we have a lump sum orpayment rate of 1,000 pounds receivable at time t, then the present value (at time 0) of thatpayment is vt ∗ 1,000 with 0 < v ≤ 1 the constant discount factor per unit of time. Sinceit is more convenient to work with exponentials when powers are involved, we introduce theso-called force of interest (per unit of time) δ := − log v ≥ 0, so that vt = e−δt. Going backto Example 6 and working with a force of interest of δ = 0.02, we have that the presentvalue at time 0 of the total benefits is given by

1,000

(∫ 24

16

e−δudu +

∫ 40

28.5

e−δudu +

∫ 52

47

e−δudu

)

+ 10,000e−δ∗52

=1,000

0.02

(

e−0.32 − e−0.48 + e−0.57 − e−0.8 + e−0.94 − e−1.04)

+ 10,000e−1.04 = 16,571.31.

2.3 Expected present values

We let X = {Xt : t ≥ 0} be a Markov jump process with state space {1, 2, . . . , d} generated(as in the beginning of Chapter 5 of the other notes) by the Q-matrices Q(t) = (µij(t))

di,j=1,

t ≥ 0. Here we assume that the transition rates µij(t) are bounded and continuous in t.Recall that an MJP has right-continuous sample paths. Further, P(s, t) = (pij(s, t))

di,j=1 is

the matrix of transition probabilities defined by

pij(s, t) = Pr(Xt = j|Xs = i),

9

where 0 ≤ s ≤ t. Note that P(s, t) can be computed from the Q-matrix Q(t) by Kolmogorov’s

forward equations. Recall that ~β(t) is the vector of sojourn payment rates at time t, B(t) isthe matrix of transition payments at time t and δ ≥ 0 the force of interest. Given Xs = i,the expected present value at time 0 of the sojourn payments from time s until time t (withs < t) is defined as

E

[∫ t

s

e−δuβXu(u)du

∣

∣

∣

∣

Xs = i

]

.

Note that βXu(u) =∑d

i=1 βi(u)1{Xu=i}. Further, given Xs = i, the expected present value(at time 0) of the transition payments from time s until time t is defined as

E

∑

u∈[s,t]:Xu−6=Xu

e−δubXu−Xu(u)

∣

∣

∣

∣

∣

∣

Xs = i

,

where Xu− := limt↑u Xt stands for the left-limit of Xu. Note that

bXu−Xu(u) =

d∑

i=1

d∑

j=1

bij(u)1{Xu−=i,Xu=j}

and that the set of discontinuities {u ∈ [s, t] : Xu− 6= Xu} over which the sum is taken,corresponds exactly to the set of jump times of the MJP X in the time interval [s, t]. In theliterature one can find several other (equivalent) ways of denoting the expected present valueof the transition benefits. We can compute both EPVs via the following formulas which arevery similar to formulas (1) and (2) in the discrete time case.

Theorem 7. The following two identities hold:

E

[∫ t

s

e−δuβXu(u)du

∣

∣

∣

∣

Xs = i

]

=d∑

j=1

∫ t

s

e−δuβj(u)pij(s, u)du, (3)

E

∑

u∈[s,t]:Xu−6=Xu

e−δubXu−Xu(u)

∣

∣

∣

∣

∣

∣

Xs = i

=d∑

k=1

d∑

j=1

∫ t

s

e−δubjk(u)pij(s, u)µjk(u)du. (4)

Proof. As the integrand is bounded, we are allowed to switch integral and expectation (byFubini’s theorem) and therefore

E

[∫ t

s

e−δuβXu(u)du

∣

∣

∣

∣

Xs = i

]

=

∫ t

s

e−δuE [βXu(u)|Xs = i] du

=

∫ t

s

e−δud∑

j=1

βj(u)Pr(Xu = j|Xs = i)du

=

d∑

j=1

∫ t

s

e−δuβj(u)pij(s, u)du,

10

which proves the first identity. The second indentity we prove by an approximation argumentinvolving formula (2). Define, for m ≥ 1, Am as the following discrete subset of [s, t],

Am := {s, s + 1/m, s + 2/m, . . . , s + ⌊m(t − s)⌋/m},

where ⌊x⌋ denotes the integer part of x. Then

E

∑

u∈[s,t]:Xu−6=Xu

e−δubXu−Xu(u)

∣

∣

∣

∣

∣

∣

Xs = i

=

d∑

k=1

d∑

j=1

E

∑

t∈[s,t]:Xu−6=Xu

e−δubjk(u)1{Xu−=j,Xu=k}

∣

∣

∣

∣

∣

∣

Xs = i

=

d∑

k=1

d∑

j=1

E

[

limm→∞

∑

θ∈Am

e−δθbjk(θ)1{Xθ=j,Xθ+1/m=k}

∣

∣

∣

∣

∣

Xs = i

]

=

d∑

k=1

d∑

j=1

limm→∞

E

[

∑

θ∈Am

e−δθbjk(θ)1{Xθ=j,Xθ+1/m=k}

∣

∣

∣

∣

∣

Xs = i

]

=d∑

k=1

d∑

j=1

limm→∞

E

⌊m(t−s)⌋∑

r=0

(

e−δ)s+r/m

bjk(s + r/m)1{Xs+r/m=j,Xs+(r+1)/m=k}

∣

∣

∣

∣

∣

∣

Xs = i

=d∑

k=1

d∑

j=1

limm→∞

⌊m(t−s)⌋∑

r=0

(

e−δ)s+r/m

bjk(s + r/m)pij(s, s + r/m)pjk(s + r/m, s + (r + 1)/m)

=d∑

k=1

d∑

j=1

limm→∞

∑

θ∈Am

e−δθbjk(θ)pij(s, θ)pjk(θ, θ + 1/m)

1/m1/m

=d∑

k=1

d∑

j=1

∫ t

s

e−δubjk(u)pij(s, u)µjk(u)du,

where we have used the dominated convergence theorem in the second and third equality, thechange of variables θ = s+r/m in the fourth and sixth equality, (2) in the fifth since X(m) ={Xs+r/m : 0 ≤ r ≤ ⌊m(t−s)⌋} is a discrete time Markov chain and the dominated convergence

theorem together with the fact that limm→∞pjk(θ,θ+1/m)

1/m= µjk(θ) (see Equation (4.4) in the

other set of lecture notes) in the last equality. Note that the dominated convergence theoremis applicable in each case because the transition payments bjk(u) and the transition ratesµjk(u) are assumed to be bounded in u. �

2.4 Reserves

Let T be the end date of the insurance contract so that after time T no premiums andbenefits are paid. Then the (prospective) reserves at time s ∈ [0, T ] in state i is defined as

11

the expected present value of the future benefits minus the future premiums, i.e.

Vi(s) = E

∫ T

s

e−δ(u−s)βXu(u)du +∑

u∈[s,T ]:Xu−6=Xu

e−δ(u−s)bXu−Xu(u)

∣

∣

∣

∣

∣

∣

Xs = i

Using Theorem 7, we see that

Vi(s) =

d∑

j=1

∫ T

s

e−δ(u−s)pij(s, u)

[

βj(u) +

d∑

k=1

bjk(u)µjk(u)

]

du. (5)

A similar story as in the discrete case holds for the continuous time case: in order to apply (5)to calculate the reserves, one first needs to compute the transition probabilities pij(s, u) bysolving a system of ODEs and then plug them into formula (5), which involves computing anintegral. Instead one can solve another system of ODEs which gives us directly the reserves.

Theorem 8. Assume that µij(u), βi(u), bij(u) are bounded and continuous in u. Then wehave that the reserves satisfy the following system of ODEs:

d

dsVi(s) = δVi(s) −

d∑

j=1

µij(s) [Vj(s) + bij(s)] − βi(s), 0 < s < T, i = 1, . . . , d. (6)

with boundary condition ~V (T ) := (V1(T ), . . . , Vd(T )) = 0.

Proof. It is clear that the boundary condition ~V (T ) = 0 is satisfied. From (5) we have forevery t ∈ [s, T ],

Vi(s) =

d∑

j=1

∫ t

s

e−δ(u−s)pij(s, u)

[

βj(u) +

d∑

k=1

bjk(u)µjk(u)

]

du

+

d∑

j=1

∫ T

t

e−δ(u−s)pij(s, u)

[

βj(u) +

d∑

k=1

bjk(u)µjk(u)

]

du

=

d∑

j=1

∫ t

s

e−δ(u−s)pij(s, u)

[

βj(u) +

d∑

k=1

bjk(u)µjk(u)

]

du

+ eδ(s−t)d∑

j=1

∫ T

t

e−δ(u−t)

[

d∑

m=1

pim(s, t)pmj(t, u)

][

βj(u) +d∑

k=1

bjk(u)µjk(u)

]

du

=d∑

j=1

∫ t

s

e−δ(u−s)pij(s, u)

[

βj(u) +d∑

k=1

bjk(u)µjk(u)

]

du + eδ(s−t)d∑

m=1

pim(s, t)Vm(t)du.

By continuity of the functions µij(u), βi(u), bij(u) and the differentiability of pij(s, u) in s,cf. Section 4.4 about Kolmogorov’s backward differential equations in the other notes, it

12

follows that Vi(s) is differentiable in s and by taking derivatives in s on both sides of aboveequation and then letting t ↓ s we get,

d

dsVi(s) = − βi(s) −

d∑

k=1

bik(s)µik(s) + δVi(s) −

d∑

m=1

Vm(s)

(

d∑

k=1

µik(s)pkm(s, t)

)

t↓s

= − βi(s) −d∑

k=1

bik(s)µik(s) + δVi(s) −d∑

m=1

Vm(s)µim(s),

where we used pij(s, s) = 1{i=j} and the Kolmogorov backward equations, see Section 4.4 inthe other notes. Note that switching of the derivative and the integral can be easily justifiedby the dominated convergence theorem as the functions µij(u), βi(u), bij(u) are assumed tobe bounded. �

Just as in the discrete time case, a typical way to set up premiums and benefits in aninsurance policy is such that the equivalence principle is satisfied, meaning that the EPV(at the beginning of the contract) of the total premiums is equal to the EPV of the totalbenefits. So under the equivalence principle and assuming that the insurance policy startsat time 0, we have either that the initial or single premium equals Vi(0), where i is the initialstate of the policyholder or, in the case that there is no single premium, we have Vi(0) = 0.

Example 9. Consider an alive-dead model where the set of Q-matrices is given by

Q(t) =

(

−µ(t) µ(t)0 0

)

.

Assume that the policyholder pays a premium at constant rate −β, where β < 0, during hislife and that a lump sum benefit of size b > 0 is paid out upon death of the policyholder. Inthat case we have V2(s) = 0 for all s ≥ 0 and V1(s) satisfies the differential equation:

d

dsV1(s) = δV1(s) + µ(s)V1(s) − bµ(s) − β.

Note that the sojourn payment rate vector is given by ~β = (β1, β2) = (β, 0) and so inparticular β1 < 0 as it represents money flowing from the policyholder to the insurancecompany. The above differential equation is called Thiele’s differential equation. Assumenow that µ(s) is constant in s and denote µ := µ(s). Then by solving first the homogeneouspart of the equation and then finding a particular solution, we have that, recalling theboundary condition V1(T ) = 0,

V1(s) =β + bµ

δ + µ

(

1 − e−(δ+µ)(T−s))

, 0 ≤ s ≤ T.

Of course we could also have used (5) to get the above expression for V1(s). We see that theequivalence principle is satisfied if and only if β + bµ = 0.

13

3 Multiple lives

Up till now we considered insurance policies involving one life. In this section we will look atpolicies that depend on the status of multiple (independent) lives. Here we will only considerpolicies in continuous time and with only one risk, namely the risk of dying. When only onelife is under consideration, one way to determine EPVs of premiums and benefits when theonly risk is the risk of dying, is to work with the Markov jump process X = {Xt : t ≥ 0}corresponding to the alive-dead model

a:alive d:dead-µ(t)

with force of mortality µ(t) and then apply the formulas from Section 2. Alternatively, onecan use lifetime random variables. To this end, let T be the survival time with force ofmortality µ(t), t ≥ 0. Recall from Section 4.3 in the other set of notes that we have thefollowing connection between T and the MJP X,

Pr(T ≤ x + t|T > x) = Pr(Xx+t = d|Xx = a) = 1 − exp

(

−

∫ x+t

x

µ(u)du

)

, t ≥ 0.

More convenient is actually to work with the residual lifetime random variable Tx at age xwhich, recall from Section 1.3 in the other notes, is defined as the random variable whosedistribution function is given by

Pr(Tx ≤ t) = Pr(T ≤ x + t|T > x), t ≥ 0.

Note further that the probability density function (pdf) of Tx, denoted by fTx , is given by

fTx(t) = µ(x + t) exp

(

−

∫ x+t

x

µ(u)du

)

, t ≥ 0.

The EPV of premiums and benefits corresponding to the alive-dead model for a life currentlyaged x can then be expressed as an expectation of (a function of the) random variable Tx.

Similarly, when considering two independent lives, there are two approaches that one canfollow in order to compute EPVs. One is to work with the MJP whose Q-matrices are givenby the picture

14

1 3

2 4

-µ(2)(t)

-µ(2)(t)

?

µ(1)(t)

?

µ(1)(t)

with states 1:life 1 and 2 both alive, 2:life 1 dead, life 2 alive, 3:life 1 alive, life 2 dead and4:life 1 and 2 both dead. Note that since the lives are independent, the dead of one life doesnot affect the lifetime of the other life, which means that µ12(t) = µ34(t) and µ13(t) = µ24(t),where recall µij(t) is the transition intensity from state i to state j. Alternatively, one canwork with two lifetime random variables, which is the approach that we will employ here.

In particular, we work with the following setup. We let T(1)x be the (residual) lifetime of

individual 1 currently aged x and T(2)y be the (residual) lifetime of individual 2 currently aged

y. The corresponding force of mortality functions might be different for the two individualsand we denote them by µ(1)(·) and µ(2)(·) for individual 1 and individual 2 respectively.Unlike in Section 2, here we do allow µ(x) to be a function with discontinuities. To be moreprecise, we only assume that µ(x) is integrable and bounded. In particular, the case whereµ(x) is a piecewise constant function is allowed. We assume further that the two individuals

are independent which means that the lifetimes T(1)x and T

(2)y are assumed to be independent.

Next we will illustrate how to calculate EPVs of benefits involving two lives by consideringa number of examples. We will work with a constant force of interest (per year) denoted byδ.

Example 10. Consider an annuity of rate 1 (per year) that is paid out as long both indi-viduals are alive. Then the EPV (in terms of the force of mortality functions µ(1)(·) andµ(2)(·)) is given by

E

[∫ ∞

0

e−δt1{t<T

(1)x ,t<T

(2)y }

dt

]

=

∫ ∞

0

e−δtPr(T (1)x > t, T (2)

y > t)dt

=

∫ ∞

0

e−δtPr(T (1)x > t)Pr(T (2)

y > t)dt

=

∫ ∞

0

e−δt exp

(

−

∫ x+t

x

µ(1)(u)du

)

exp

(

−

∫ y+t

y

µ(2)(u)du

)

dt,

where the first equality follows by Fubini and the second by independence of T(1)x and T

(2)y .

Example 11. Consider an assurance benefit of size 1 that is paid out upon the first deathamongst the two individuals, provided the first death occurs within 10 years time. Denote

15

T(1)x ∧ T

(2)y := min{T

(1)x , T

(2)y }. Then we have by independence

Pr(T (1)x ∧ T (2)

y > t) = Pr(T (1)x > t, T (2)

y > t)

= exp

(

−

∫ x+t

x

µ(1)(u)du

)

exp

(

−

∫ y+t

y

µ(2)(u)du

)

and so the pdf fT

(1)x ∧T

(2)y

of T(1)x ∧ T

(2)y is given by

fT

(1)x ∧T

(2)y

(t) =[

µ(1)(x + t) + µ(2)(y + t)]

exp

(

−

∫ x+t

x

µ(1)(u)du −

∫ y+t

y

µ(2)(u)du

)

.

Consequently, the EPV of the assurance is given by

E

[

e−δ(T(1)x ∧T

(2)y )1

{T(1)x ∧T

(2)y ≤10}

]

=

∫ ∞

0

e−δt1{t≤10}fT(1)x ∧T

(2)y

(t)dt

=

∫ 10

0

e−δt[

µ(1)(x + t) + µ(2)(y + t)]

exp

(

−

∫ x+t

x

µ(1)(u)du −

∫ y+t

y

µ(2)(u)du

)

dt.

Example 12. Consider an assurance benefit of size 1 that is paid out to individual 2 at thetime of death of individual 1, provided individual 2 is still alive then. With fZ denoting the(joint) pdf of the random vector Z, the corresponding EPV is given by

E

[

e−δT(1)x 1

{T(1)x <T

(2)y }

]

=

∫ ∞

0

∫ ∞

0

e−δt1{t<s}fT(1)x ,T

(2)y

(t, s)dsdt

=

∫ ∞

0

∫ ∞

0

e−δt1{t<s}fT(1)x

(t)fT

(2)y

(s)dsdt

=

∫ ∞

0

e−δtµ(1)(x + t) exp

(

−

∫ x+t

x

µ(1)(u)du

)∫ ∞

t

fT

(2)y

(s)dsdt

=

∫ ∞

0

e−δtµ(1)(x + t) exp

(

−

∫ x+t

x

µ(1)(u)du

)

exp

(

−

∫ y+t

y

µ(2)(u)du

)

dt,

where in the second equality we used the independence of T(1)x and T

(2)y .

Note that in all of the above examples we can work out the integral expressions in thecase where the two forces of mortality are constant.

16

Exercises

Exercise 1

Consider a 10-year insurance policy in which the health status of the policyholder is assessedonce a year and the policyholder receives a benefit of 1,000 pounds each time he/she isassessed as sick and receives once a benefit of 10,000 pounds upon death. The health statusof the policyholder evolves as in Figure 3. Working with an interest rate of 2% per year,calculate the present value (at time 0) of the total benefits in the following four cases.

(a) The assessments and payments take place at the beginning of each year.

(b) The assessments and payments take place at the end of each year.

(c) The assessments and payments take place exactly halfway during each year.

(d) The assessments are made at the beginning of each year and the payments take placeat the end of the year.

0 3 4 t1 2

s

h

d

Figure 3: The health status over time of the policyholder corresponding to Exercise 1.

Exercise 2

A theme park owner wants to buy insurance to cover a low turnout of visitors due to badweather during one week (Monday till Sunday) in the summer. The owner and the insurancecompany agree upon a two-state time homogeneous Markov model for the day-to-day statusof the weather with states 1:sunny and 2:rainy and transition matrix

P =

(

0.8 0.20.6 0.4

)

.

17

It is further agreed that for each rainy day in the week, the theme park owner receives abenefit of 500 pounds. Since the length of the insurance policy is so short, it is agreed towork with an interest rate of 0%.

(a) Assuming that it is sunny on Monday and that the theme park owner pays a fixed-sizepremium for each day of the week that the weather is sunny, what is the size of thepremium when the equivalence principle is applied?

(b) Assume that the probability of the weather on Monday being sunny is 75% and ofbeing rainy is 25% and that the theme park owner pays a single premium of size ρ atthe beginning of the week. What is the value of ρ when the equivalence principle isapplied?

Exercise 3

A motor insurance company offers its customers a premium discount of either 0%, 20% or40% depending on the driver’s claim record. A claims-free year results in the discount beingincreased to the next higher stage or in the retention of the maximum discount for the nextyear. A year with at least one claim results in the discount being reduced to the next lowerstate or in the retention of a zero discount. Each year a driver has probability 1/4 of makingat least one claim and a probability of 3/4 of making no claims. The premium, withoutdiscount is 500 pounds a year and is due at the beginning of each year. New customers donot receive a discount when they join. The interest rate is 2% per year.

(a) What is the expected present value of the total amount of premiums a new customerhas to pay in the first four years?

(b) Assume that when the new customer makes a claim in a year, the size of the claim ison average equal to µ. Assuming that the insurance company pays back a claim atthe end of the year, what is the value of µ such that the reserves, taking into accountonly the first four years, of the insurance company at the start of the policy is zero?

Exercise 4

In a particular law firm the people who work there can be categorised into four differentstates: 1:junior associate, 2:senior associate with possibility of becoming partner, 3:seniorassociate with no possibility of becoming partner and 4:partner. At this law firm, a juniorassociate receives a salary of £30,000 per year, a senior associate earns £40,000 per yearand a partner receives £60,000 per annum. Assume here that salaries for the whole year arepaid annually in advance, i.e. each employee receives his/her entire salary for the year at thestart of that year. At the end of each year the partners decide what the status in the newyear will be of each employee. If at this meeting the partners decide that an employee shouldbe promoted, then this employee will be immediately awarded with a one-time bonus. Inparticular, when a promotion is made from junior to senior associate the employee receivesa one-time bonus of £15,000, while a one-time bonus of £25,000 is handed out when theemployee becomes partner. Assume that the yearly status of an employee of this law firm

18

can be modelled by a time homogeneous discrete time Markov chain with transition matrix

P =

0.8 0.2 0 00 0.5 0.4 0.10 0 1 00 0 0 1

.

Further assume that the interest rate is 2% per annum.

(a) A person just got hired as a junior associate at the start of the year. Determine theexpected present value of the total amount of money this person will earn in the firstfive years of employment. (Note that this includes the possible bonus handed out atthe end of the fifth year, but does not include the salary corresponding to the sixthyear of employment.)

(b) Same as part (a), but now assume that there is a slight change in the salary structureof a junior associate. Namely, a junior associate receives a salary of £25,000 in his/herfirst year of being a junior associate and receives £30,000 per year in subsequent yearsas a junior associate.

(c) Same as part (b), but now assume that there is a slight change in the salary structureof a senior associate as well. Namely, a senior associate receives a salary of £40,000in his/her first year of being a senior associate and receives £45,000 per year insubsequent years as a senior associate.

Exercise 5

A university offers a four-year degree course. Given that a student does not die during theyear, the probability that a student progresses from one year of study to the next is 0.85 inthe first year of study, 0.9 in the second year and 0.95 in the third. Students who fail in anyyear may not continue in the degree. Assume that the lifetime of a student is modelled bya survival time with constant hazard rate/force of mortality µ = 0.001 and that given thatthe student is alive, his/her residual lifetime does not depend on whether or not the studentis still on the course. Further assume there are no means of leaving the course other thanby death or failure.

Students pay tuition fees at the start of each year. The tuition fee is £5000 per year.Assuming interest at 2% per year, calculate the expected present value of fee income to theuniversity for a new student.

Exercise 6

Consider the following MJP model which models the health status of a life:

19

h:healthy i:ill

d:dead

-�

α(t)

β(t)@

@@

@@

@Rµ(t)

��

��

��

ν(t)

Note that the transition rates depend on age t. Let δ > 0 be the constant force of interest.Provide integral expressions (in terms of the transition probabilities and the transition rates)for the expected present value of the following benefits for a life currently aged 45 and healthy:

(a) An illness benefit of 10,000 pounds payable continually while the life is ill but endingat age 65.

(b) A term assurance paying 30,000 pounds on death before age 65 if the life dies fromthe ill state.

(c) An illness benefit of 10,000 pounds payable continually while the life is ill but thebenefit only starts after the life has been ill for a consecutive period of 1 year (thisapplies each time that the life gets ill). Further the benefit ends at age 65. (Hint: useExercise 2 on Example sheet 5.)

(d) An endowment assurance paying 50,000 pounds on survival to age 65, provided thatsurvival occurs without the life having ever been ill.

(e) An endowment assurance paying 50,000 pounds on death before age 65, provided thatdeath occurs without the life having ever been ill.

(f) An endowment assurance paying 50,000 pounds on death before age 65 or on survivalto age 65, provided that death and survival occur without the life having ever beenill.

(g) A term assurance paying 40,000 pounds on death before age 65 if the life dies fromthe healthy state while having been ill before.

Exercise 7

Consider the following time homogeneous health-illness-death model with the addition of a’terminally ill’ state k. The transition rates are as given below.

20

h i k

d

-�

0.02

1.00

-0.15

@@

@@

@@@R

0.05

��

��

���

0.40

?

0.05

Assume a 10-year insurance policy with (i) a sojourn benefit of 1,000 pounds per year payablewhenever the life is ill, (ii) a sojourn benefit of 4,000 pounds per year payable whenever thelife is terminally ill, (iii) a lump sum benefit of 10,000 pounds upon death due to illness(independent of whether he/she was terminally ill or just ill) and (iv) a lump sum benefit of5,000 pounds upon death due to other reasons. The force of interest is 0.02.

(a) Let Vh(t), Vi(t) and Vk(t) be the value of the reserves at time t in state h, i and krespectively. Write down the system of ordinary differential equations with boundarycondition that is satisfied by (Vh(t), Vi(t), Vk(t)).

The Euler method of numerically solving a system of (autonomeous) ODEs with boundarycondition

~y(t) = ~F (~y(t)), ~y(t0) = ~y0,

goes as follows. (Note that F , t0 and y0 are assumed to be given here.) Choose a value ∆ forthe size of every step and set tn = t0 + ∆ · n, n = 1, 2, . . .. Now, the Euler method consistsof computing the vectors ~y1, ~y2, . . . recursively via

~yn+1 = ~yn + ∆ · ~F (~yn).

The value of ~yn is then an approximation of the solution to the ODE at time tn, i.e. ~yn ≈~y(tn). The error in the approximation gets smaller as the step size ∆ decreases.

(b) By implementing the Euler method on a computer (e.g. with the aid of Matlab),provide numerical plots (for a suitable small step size ∆) of the reserves Vh(t), Vi(t)and Vk(t) for 0 ≤ t ≤ 10.

(c) Assume a premium of size ρ is paid at the beginning of the policy. Determine ρ nu-merically under the equivalence principle if the insured life is healthy at the beginningof the policy.

Exercise 8

Consider the following graphical representation of a Q-matrix of a time homogeneous MJP,which is used for critical illness insurance.

21

1:ill

2:dead due to illness

3:dead by other causes

���������*

HHHHHHHHHj

0.80

0.02

Associated to this model is a singe premium insurance contract which specifies that aslong as the insured is ill, he/she gets a benefit of rate (1+0.4t)∗5000 pounds per annum/year,where time t is time in years after the start of the policy. Further at the time of death alump sum benefit of 10,000 pounds is paid out, provided the person dies due to the illness.Working with a force of interest of 0.05 (per annum), determine the single premium underthe equivalence principle.

Exercise 9

Let T(1)x and T

(2)y be the (residual) lifetimes of individual 1, currently aged x, and individual

2, currently aged y, respectively. Assume the force of mortality of individual 1 and 2 isconstant and equal to µ(1) for individual 1 and equal to µ(2) for individual 2. Assume thatT

(1)x and T

(2)y are independent and denote the constant force of interest by δ.

(a) Find the EPV of a last survivor annuity for which an annuity of size 1 is paid as longas at least one of the two individuals is alive.

(b) Find the EPV of a last survivor assurance for which a benefit of size 1 is paid out atthe time when the last survivor dies.

(c) Find the EPV of an annuity of size 1 payable to individual 2 that starts on the deathof individual 1 and then continues until the death of individual 2, provided thatindividual 1 dies within 10 years after the start of the policy.

(d) Find the EPV of an annuity of size 1 payable to individual 2 that starts on the deathof individual 1 and then continues until the death of individual 2 or until 10 yearshave expired after the death of individual 1, whichever event occurs first.

(e) Find the EPV of an annuity of size 1 payable to individual 2 that starts on the deathof individual 1 and then continues until the death of individual 2 or until 10 yearshave expired after the start of the policy, whichever event occurs first.

Exercise 10

22

A life insurance company issues a special annuity contract to a husband and wife. In thispolicy 10,000 per year is payable continuously for life to the survivor of the first death,provided the first death occurs within ten years of the start of the policy. Payments beginimmediately on death and are guaranteed for the remaining period of the original ten yearterm. (That is, once started, payments will continue until at least the end of the originalten year term even if the second life dies before that.) The company uses a model with thefollowing assumptions:

• the lifetimes of the husband and wife are independent;

• the husband has a constant force of mortality of 0.02;

• the wife has a constant force of mortality of 0.015;

• the force of interest equals 0.04.

(a) Calculate the expected present value of the annuity payments that take place in thefirst ten years.

(b) Calculate the expected present value (with respect to the start of the policy) of theannuity payments that take place after the first ten years.

(c) Determine the single premium that the couple has to pay at the start of the policy inorder for the equivalence principle to be satisfied.

23