shiryaev
DESCRIPTION
mathematics stochastic processes lectureTRANSCRIPT
Albert N. SHIRYAEV
Steklov Mathematical Institute,
Lomonosov Moscow State University
LECTURES on
FinancialStatistics, Stochastics, and Optimization
e-mail: [email protected]
1
TOPIC I. Financial models and innovations in stochastic
economics
1. The classical and neoclassical models of the dynamics ofthe prices driven by Brownian motion and Levy processes
2. Stylized facts3. Constructions based on the change of time, stochastic
volatility
TOPIC II. Technical Analysis. I
1. Kagi and Renko charts2. Prediction of time of maximum value of prices observable
on time interval [0, T ]3. Quickest detection of time of appearing of arbitrage4. Drawdowns as the characteristics of risk
TOPIC III. Technical Analysis. II
1. Buy and Hold2. Stochastic Buy an Hold (rebalancing of the portfolio)
TOPIC IV. General Theory of Optimal Stopping 2
TOPIC I. Financial models and innovationsin stochastic economics
1. The classical and neoclassical models of the
dynamics of the prices driven by Brownian
motion and Levy processes
2. Stylized facts
3. Constructions based on the change of time,
stochastic volatility
I-1
The construction of the right probability-statistical models of the
dynamics of prices of the basic financial instruments (bank account,
bonds, stocks, etc.) is undoubtedly one of important steps for successful
application of the results of mathematical finance and financial
engineering.
Without adequate models for prices there is no successful risk management,
portfolio optimization, allocation of funds, derivative pricing, etc.
The main accent in this lecture is made on the construction of the
HYPERBOLIC LEVY PROCESSES,
which are widely used in econometric models of the dynamics of the
financial indexes.
I-2
THE FIRST CLASSICAL MODELS FOR
PRICE DYNAMICS
In the sequel,
S = (St)t!0 is the price of (for simplicity) one asset
L. Bachelier (1900). Theorie de la speculation:
St = S0 + µt + !Bt ,
where B = (Bt)t!0 is a standard Brownian motion, i.e.,
a Gaussian process with independent
increments and continuous trajectories,
B0 = 0, EBt = 0, E(Bt " Bs)2 = t " s.
I-3
M. Kendall (1953). The analysis of economic time series.Part 1. Prices (J. Roy. Statist. Soc., 96, 11–25):
The empirical analysis of prices S = (Sn)n!0 for
– wheat (monthly average prices on the Chicago market, 1883–1934),– cotton (the New York Mercantile Exchange, 1816–1951)
did not reveal (contrary to common expectations) neither rhythms,nor cycles. The observed data look as if
“...the Demon of Chance drew a random number... andadded it to the current price to determine the next... price”:
Sn = S0eHn , where Hn = h1 + · · · + hn is the sum of
independent random variables
(“random walk hypothesis”)
I-4
M. F.M. Osborne (1959). Brownian motion in the stock market.
Operation Research, 7, 145–153: St = S0 + µt + !Bt .
P. A. Samuelson (1965). Proof that properly anticipated prices
fluctuate randomly. Industrial Management Rev., 6, 41–49:
St = S0eHt, Ht =!µ "
!2
2
"t + !Bt,
S = (St)t!0 is an economic (geometric) Brownian motion;
dSt = St(µ dt + ! dBt)
#This model underlies the Black–Scholes theory of option pricing.$
I-5
MARTINGALE APPROACH TO STUDYING THE MODELS
S = (Sn)n!0, Sn = S0eHn, Hn = h1 + · · · + hn
(hn = log(Sn/Sn"1) is a “return”, “logarithmic return”)
Doob’s decomposition. Assume that “stochastics” of the marketis described by a filtered probability space (!,F , (Fn)n!0, P) andE|Hn| < %, n ! 0.
Hn =n#
k=1
E(hk | Fk"1) +n#
k=1
!hk " E(hk | Fk"1)
", or
hn = E(hn | Fn"1)$ %& 'µn
are Fn"1-measurable
+ (hn " E(hn | Fn"1))$ %& '"n
are Fn-measurable,E("n | Fn"1) = 0,
("n) is a martingale di!erence
I-6
1970s#large time intervals –year, quarter, month$
: linear models like AR, MA, ARMA with
hn = µn + !n#n (i.e., "n = !n#n),
"#$ µn and !n are Fn"1-measurable,
#n & N (0,1) are independent, n ! 1.
AR(p) model:
µn = a0 + a1hn"1 + · · · + aphn"p, !n = const
MA(q) model:
µn = b0 + b1#n"1 + · · · + bq#n"q, !n = const
ARMA(p, q) model:
µn =(a0 + a1hn"1 + · · · + aphn"p
)
+(b0 + b1#n"1 + · · · + bq#n"q
), !n = const
I-7
1980s#analysis of day data$
: nonlinear models ARCH, GARCH, CRR
Sn = S0 exp{h1 + · · · + hn}
ARCH(p) model – AutoRegressive Conditional Heteroskedastic
model; P. Engle (1982):
hn = !n#n, !n =
*++,$0 +p#
i=1
$ih2n"i is random (!).
GARCH(p, q) model – Generalized ARCH model; T. Bollerslev
(1986):
hn = !n#n, !n =
*+++,
-
$0 +p#
i=1
$ih2n"i
.
+
- q#
j=1
%j!2n"j
.
.
Binary CRR-model – Cox, Ross, Rubinstein (1979):
hn = log(1 + &n), &n takes two values, &n > "1.
I-8
1990s#intraday data analysis$
: (a) Stochastic processes with discrete
intervention of chance (piecewise
constant trajectories with jumps at
“close” times '1, '2, . . .):
Ht =#
hkI('k ' t)
(b) Data come almost continuously.
I-9
WEAKNESS of the MODEL St = S0eHt, Ht = (µ"!2/2)t+!Bt,
based on a Brownian motion, i.e., dSt = St(µ dt + ! dBt) with a
constant volatility !.
Really observable smile e!ect says that the volatility ! is NOT a
constant.
Consider a call (buyer) option with pay-o! function (ST " K)+:
C(t, x) = EP
((ST " K)+ |St = x
).
By the Black–Scholes formula we find C(t, x) = CBS(t, x;T, K, !).
On option market there exist real prices
/C(t, x;T, K).
From CBS(t, x;T, K, !) ( /C(t, x;T, K) we calculate the implied volatility
! = !(t, x;T, K). Fix t, x, T . It turns out that !(K) has a U-form
(with Kmin ( x) – smile e!ect.
I-10
1st CORRECTION
(R. Merton, 1973)
! ") !(t)
!(t) ") !(t, St)
2nd CORRECTION
(B. Dupire, 1994)
Pricing with a smile,
RISK, 7, 18–20
I-11
One- and two-dimensional distributions of H = (Ht)t!0.
The observable properties of h(")t = log(St/St"")
A. The behavior of empirical densities p(")(x), constructed upon
h(")" , h(")
2" , . . ., is di!erent from that of normal distribution. In
a neighborhood of the central value, the densities p(")(x) are
peak-like, and “heavy tails” are observed as x ) ±%.
B. The empirical estimator of autocorrelation (t = k")
&(n") =0Eh(")
t h(")t+n" " Eh(")
t Eh(")t+n"
1203Dh(")
t Dh(")t+n"
1
shows that for small n" the value &(n") is negative, while most
of the values of &(n") are close to zero (noncorrelatedness).
C. Analogous estimators for autocorrelation of absolute values
|h(")t | and |h(")
t+n"| show that for small n" the autocorrelation
is positive (clustering e!ect).
I-12
HERE THE PICTURE
I-13
Searching for adequate statistical models which describe
dynamics of the prices S = (St)t!0 led to
LEVY PROCESSES.
Now these processes take the central place in modelling the
prices of financial indexes, the latter displaying the jump
character of changes.
I-14
MAIN MODELSbased on a Brownian motion
Exponential
Brownian model
St = S0 exp{µt + !Bt}
* *Exponential
INTEGRAL
Brownian model:
St = S0 exp45 t
0µs ds +
5 t
0!s dBs
6
Exponential
TIME-CHANGED
Brownian model:
St = S0 exp7µT(t) + BT(t)
8
I-15
Assuming that µ = 0, one can rewrite these models in a brief form:
S = S0e!B
* *S = S0e!·B S = S0eB+T
where • ! · B is the stochastic integral (5 ·
0!s dBs),
• B + T is a time change in Brownian motion (BT(t)).
I-16
A generalization of these “Brownian” models, which have being
predominating in financial modelling for a long time, is based on
the idea to replace
BROWNIAN MOTION
B = (Bt)t!0
by LEVY PROCESSES
L = (Lt)t!0
:
S = S0e!L
* *S = S0e!·L S = S0eL+T
I-17
LEVY PROCESS L = (Lt)t!0 is a process with stationary
increments, L0 = 0, which is continuous in probability.
Such processes have modifications whose trajectories
• are right-continuous (for t ! 0) and
• have limits from the left (for t > 0).
Kolmogorov-Levy-Khinchin’s formula for characteristic functions:
Eei(Lt = exp
9
t0i(b "
(2
2c +
5 !ei(x " 1 " i(h(x)
"F(dx)
1:
,
where: h(x) = xI(|x| ' 1) (classical “truncation” function),
F(dx) is a !-finite measure on R \ {0}such that
;min(1, x2)F(dx) < %,
b , R and c ! 0;
(b, c, F) =: T is a triplet of local characteristics of L.
I-18
The Levy–Ito representation for trajectories of L = (Lt)t!0:
Lt = Bt + Lct +
5 t
0
5h(x) d(µ " )) +
5 t
0
5(x " h(x)) dµ ,
• Bt = bt; • Lct is a continuous component of L
(Lct =
-c Wt, where Wt is a Wiener process);
• µ is the measure of jumps: for A , B(R \ {0})
µ(*; (0, t] . A) =#
0<s'tIA("Ls) ("Ls = Ls " Ls");
• ) is the compensator of the measure of jumps µ:
)((0, t] . A) = tF(A), F(A) =;A F(dx).
The measure µ is a Poissonian measure with
E exp4i#
k'n(kµ(Gk)
6= exp
4#k'n
(ei(k " 1))(Gk)6, n ! 1,
where Gk are sets from R+ . R and )(dt, dx) = dt F(dx).
I-19
EXAMPLES of LEVY PROCESSES :
• Brownian motion,
• Poisson process,
• compound Poisson process Lt =Nt#
k=1
+k, where
(Nt)t!0 is a Poisson process,
(+k)k!1 is a sequence of independent and identicallydistributed random variables
I-20
In connection with financial econometrics, these are
HYPERBOLIC Levy processes,
that are of a great interest, because they model well the really
observable processes H = (Ht)t!0 for many underlying financial
instruments (rate of exchange, stocks, etc.).
The credit of developing the theory of such processes and their
applications is due to E. Halphen, O. Barndor!-Nielsen, E. Eberlein.
We will construct these processes, following mostly Chapters 9
and 12 of the monograph: O. Barndor!-Nielsen, A. N. Shiryaev,
Change of Time and Change of Measures, World Scientific (in
print).
I-21
For a Levy process (Ht)t!0 we have
Eei(Ht = (Eei(H1)t.
The properties of Levy’s processes imply that the random variable
h = H1 is infinitely divisible, i.e., for any n one can find i.i.d. r.v.’s
+1, . . . , +n such that
Law(h) = Law(+1 + · · · + +n).
We will look for the infinitely divisible r.v.’s h having the form
h = µ + %!2 + !#,
where # is a standard Gaussian random variable, # & N (0,1),
! = !(*) is the “volatility” (which does not depend on #), for
whose square, !2, we will construct the special distribution
GIG – Generalized Inverse Gaussian distribution.
I-22
Strikingly, this distribution (on R+) is infinitely divisible and the
distribution of h = µ + %!2 + !# (on R) is infinitely divisible as well.
Hence there exist Levy processes T = (T(t))t!0 and H/ = (H/t )t!0
such that
Law(T(1)) = Law(!2) and Law(H/1) = Law(h).
As a realization of H/ = (H/t )t!0 one can take
Ht = µt + %T(t) + BT(t),
where the “time change” T = (T(t))t!0 and the Brownian motion
B = (Bt)t!0 are independent.
In the sequel, we do not distinguish between the processes H and H/.
This process H, remarkable in many respect, bears the name
L(GH) – Generalized Hyperbolic Levy process.
I-23
Let discuss the details of construction of GIG-distributions for !2.
Let W = (Wt)t!0 be a Wiener process (standard Brownian motion).
For A ! 0, B > 0 introduce
TA(B) = inf{s ! 0: As + Ws ! B}.
HERE MUST BE A PICTURE
I-24
The formula for the density pTA(B)(s) = dP(TA(B) ' s)/ds is well
known:
pTA(B)(s) =B
s,s(B " As), "#$ ,s(x) =
1-2-s
e"x2/(2s). (1)
Herefrom we find the Laplace transform:
Ee"(TA(B) = exp7AB(1 "
<1 + 2(/A2)
8.
Letting b = B2 > 0 and a = A2 ! 0, we find from (1) the following
formula for p(s; a, b) = pT-
a(-
b)(s):
p(s; a, b) = c1(a, b)s"3/2e"(as+b/s)/2 , "#$ c1(a, b) =
=b
2-e-
ab.
The distribution with density p(s; a, b) is named
IG = IG(a, b) – Inverse Gaussian distribution.
I-25
Next important step: one define ad hoc the function
p(s; a, b, )) = c2(a, b, )) s)"1e"(as+b/s)/2, (2)
where parameters a, b, ) , R are chosen in such a way that p(s; a, b, ))
be probability density on R+:
a ! 0, b > 0, ) < 0
a > 0, b > 0, ) = 0
a > 0, b ! 0, ) > 0
>
05 %
0s)"1e"(as+b/s)/2 ds < %
?
.
It is well known that K)(y) 1 12
5 %
0s)"1e"y(s+1/s)/2 ds is the modified
third-kind Bessel function of order ), which for y > 0 solves
y2f 22(y) + yf 2(y) " (y2 + )2)f(y) = 0.
The constant in (2) has the form c2(a, b, )) =(a/b))/2
2K)(-
ab).
I-26
The distribution on R+ with density
p(s; a, b, )) =(a/b))/2
2K)(-
ab)s)"1e"(as+b/s)/2
bears the name
GIG = GIG(a, b) – Generalized Inverse Gaussian distribution.
I-27
IMPORTANT PROPERTIES of GIG-DISTRIBUTION (for !2):
A. This distribution is infinitely divisible.
B. The density p(s; a, b, )) is unimodal with the mode
m =
@AB
AC
b /(2(1 " ))
), if a = 0,
(() " 1) +
<ab + () " 1)2
)/a, if a > 0.
C. Laplace’s transform L(() =;%0 e"(sp(s; a, b, )) ds is given by
L(() =D1 +
2(
a
E")/2K)(<
ab(1 + 2(/a))
K)(-
ab).
As a by-product, one deduces the representation for the density f(y)
of Levy measure F(dy). (Note: L(() = exp{;%0 (e"(y " 1)f(y) dy}.)
I-28
Particularly important SPECIAL CASES of GIG-distributions:
I. a ! 0, b > 0, ) = "1/2: in this case GIG(a, b,"1/2) = IG(a, b)
– Inverse Gaussian distribution.
Density: p(s; a, b) = c1(a, b)s"3/2e"(as+b/s)/2, c1(a, b) =<
b2- e
-ab.
Density of Levy’s measure: f(y) =<
b2-
e"ay/2
y3/2 .
II. a > 0, b = 0, ) > 0: in this case GIG(a,0, ))=Gamma(a/2, ))
– Gamma distribution. Density: p(s; a,0, )) = (a/2))
#()) s)"1e"as/2.
Density of Levy’s measure: f(y) = y"1)e"ay/2.
III. a > 0, b > 0, ) = 1: p(s; a, b,1) =
<a/b
2K1(-
ab)e"(as+b/s)/2
– PH – Positive Hyperbolic distribution, or H+-distribution.
I-29
Since GIG-distribution is infinitely divisible, if one take it as the
distribution of squared volatility !2,
Law(!2) = GIG,
then one can construct a nonnegative nonincreasing Levy process
T = (T(t))t!0 (a subordinator) such that
Law(T(1)) = Law(!2) = GIG.
In the subsequent constructions, this process plays the role of
change of time, operational time,
business time.
As was explained above, the next step in construction of the return
process H = (Ht)t!0, consists in the following.
I-30
Form the variable h = µ+%!2+!#, where Law(!2) = GIG, Law(#) =
N (0,1), !2 and # are independent. It is clear that
Law(h) = E!2N (µ + %!2, !2)
is a mixture of normal distributions, i.e., the density ph(x) of h is of
the form
ph(x) =5 %
0
1-
2-yexp
9
"(x " (µ + %y))2
2y
:
pGIG(y) dy.
Integrating and denoting ph(x) by p/(x; a, b, µ, %, )), we find that
p/(x; a, b, µ, %, )) = c3(a, b, %, ))K)"1/2
!$<
b + (x " µ)2"
!<b + (x " µ)2
"1/2")e%(x"µ) ,
where $ =<
a + %2 and c3(a, b, %, )) =(a/b))/2 $
12")
-2- K)(
-ab)
.
I-31
The obtained distribution Law(h) with density p/(x; a, b, µ, %, )) bears
the name
Generalized Hyperbolic distribution, GH = GH(a, b, µ, %, ))).
In the case ) = 1 the graph of the function
log p/(x; a, b, µ, %,1) = log c/(a, b, %) " $<
b + (x " µ)2 + %(x " µ)
is a hyperbola with asymptotes $|x " µ| + %(x " µ).
This is why the distribution for h in the case ) = 1 is called
hyperbolic, which explains the name “generalized hyperbolic distribution”
in the case of arbitrary ).
I-32
SOME PROPERTIES of GH-distribution (for h):
A*. This distribution is infinitely divisible.
B*. If % = 0, then the distribution is unimodal with mode m = µ.
(In the general case m is determined as a root
of a certain transcendental equation.)
C*. Laplace’s transform L/(() =5 %
0e(xp(x; a, b, µ, %, )) dx
(for complex ( such that |% + (| < $, $ =<
a + %2)
is given by
L/(() = e(µ0
a
$2 " (% + ()2
1)/2 K)(<
b[$2 " (% + ()2])
K)(-
ab).
I-33
THREE important SPECIAL CASES of GH-distributions:
I*. a ! 0, b > 0, ) = "1/2: in this case
GIG(a, b,"1/2) = IG(a, b) is Inverse Gaussian distribution.
The corresponding GH-distribution is commonly named
Normal Inverse Gaussian
and denoted by N + IG. The density has the form
p/(x; a, b, µ, %,"12) =
$b
-e-
ab K1($<
b + (x"µ)2)<
b + (x"µ)2e%(x"µ) , x , R.
Laplace’s transform:
L/(() = exp7(µ +
-b(-
a "<
a " 2%( " (2)8.
I-34
II*. a > 0, b = 0, ) > 0: in this case
GIG(a,0, )) = Gamma(a/2, )) – Gamma distribution.
The corresponding GH-distribution is named
Normal Gamma distribution
(notation: N + Gamma) or
VG-distribution
(notation: VG [Variance Gamma]). Density:
p/(x; a,0, µ, %, )) =a)
--#())(2$))"1/2
|x " µ|)"1/2
. K)"1/2($|x " µ|) e%(x"µ).
Laplace’s transform: L/(() = eµ((a/[a " 2%( " (2])).
I-35
III*. a > 0, b > 0, ) = 1: in this case
GIG(a, b,1) = H+(a, b) – Positive hyperbolic distribution.
The corresponding GH-distribution for h is the hyperbolic distribution
H called also
“NORMAL positive hyperbolic distribution”
(notation: H or N + H+). Density:
p/(x; a, b, µ, %,1) =a
2b$K1(-
ab)exp
4"$
<b + (x " µ)2 + %(x " µ)
6,
where $ =<
a + %2.
I-36
Having GIG-distributions for !2 and GH-distributions for h, we turnto construction of the Levy process H = (Ht)t!0 used in representationof the prices St = S0eHt, t ! 0.
TWO POSSIBILITIES! !
the fact that h has infinitely
divisible distribution allows
one to construct, using the
general theory, the Levy
process H/ = (H/t )t!0 such
that
Law(H/1) = Law(h)
using the already constructed
process T = (Tt)t!0, one forms the
process H = (Ht)t!0:
Ht = µt + %T(t) + BT(t),
where Brownian motion B and
process T are taken to be
independent.
The process H = (Ht)t!0 bears the name
L(GH) – “GENERALIZED hyperbolic Levy distribution”.
I-37
In the cases I*, II*, and III* mentioned above the corresponding
Levy processes have the special names:
L(N + IG)-process,
L(N + H+)- or L(H)-process,
L(N + Gamma)- or L(VG)-process.
It is interesting to note that L(N+ IG)- and L(N+Gamma)-processes
have an important property:
Law(Ht) belongs to the same type of distributions as Law(H1)
(this follows immediately from the formulae for Laplace’s transforms).
I-38
CONCLUDING NOTES.
Densities of distributions of h (= H1) are determined by FIVE
parameters (a, b, µ, %, )), that gives a great freedom in determining
parameters which would fit well the empirical data.
In this connection it is appropriate to recall that in statistics, there
exists a well-known method of “Pearson’s curves”, which is widely
used to construct (one-dimensional) densities of distributions upon
independent observations over a random variable +. K. Pearson itself
(1894) constructed such densities as solutions f(x) of the system
of nonlinear equations
f 2(x) =(x " a)f(x)
b0 + b1x + b2x2.
I-39
These densities are determined by FOUR parameters (a, b0, b1, b2).
The density p/(x; a, b, µ, %, )) of GH-distributions of (constructively
built) variables h = µ+%!2+!# is determined by FIVE parameters.
(It is known that these densities lie between Pearson’s curves of
type III and type V.)
The essential advantage of GH-distributions consists in their
infinite divisibility
(this is not the case for distributions from the Pearson system),
which enables us to construct processes H = (Ht)t!0 which describe
adequately the time dynamics of logarithmic return of the prices
S = (St)t!0.
I-40
TOPIC II. Technical Analysis. I
1. Kagi and Renko charts
2. Prediction of time of maximum value of prices
observable on time interval [0, T ]
3. Quickest detection of time of appearing of
arbitrage
4. Drawdowns as the characteristics of risk
II-1
The main motivation of this lecture is based on idea to obtain a
mathematical explanation of some practical methods (“when to buy,
when to sell”, etc.) of the Technical Analysis which have as usual
only a descriptive character.
As is well known, the “fundamentalists” are trying to explain
WHY the stock price moves;
they make their decisions by looking at the state of the “economy
at large”; they define a stock value and calculate proper stock prices
in view of its estimated future values; they build their analysis upon
the assumption that the actions of market operators are “rational”.
II-2
As to the “technicians” they concentrate on the local peculiarities
of the markets, they emphasize “mass behavior”, “market moods”;
they start their analysis from an idea that stock price movement
is “the product of supply and demand”; their basic concept is the
following: the analysis of past stock prices helps us to see future
prices because past prices take future prices into account; they try
to explain
HOW the stock prices move.
II-3
1. Kagi and Renko charts
Let X = (Xt)t!0 be a stock price.
The Japanese “Kagi chart” and “Renko chart” (also called the price
ranges) give methods to forecast price trends from price changes
which exceed either a certain range H or a certain rate H. The price
range or rate H is determined in advance. (In Japan, popular price
ranges are "5, 10, 20, 50, 100, 200.) Greater price ranges are used
for stocks with higher prices because their upward and downward
movements are larger.
II-4
R ) |X|, K ) maxX " X
RENKO construction: Step I: We construct (/&i):
/&0 = 0,
/&n+1 = inf7t > /&n : |Xt " X/&n
| = H8, n ! 1.
!
"
X
t
H
2H
3H
4H
!
/&0#
#$
!
/&1%%&
!
/&2 ##'
!
/&3((()
!
/&4*
*+
!
/&5##$
!
/&6 ,,-
!/&7,,-
!
/&8
!
/&9
II-5
Step II: Construction (/&n) ") (&m, &/m).
We look at all /&n such that!X/&n
" X/&n"1
"!X/&n"1
" X/&n"2
"< 0.
&/m"1
&m"1
&m
&/m t
X&m is a Markov time
&/m is a non-Markov time
II-6
KAGI construction: !0 = inf4u > 0 : max
[0,u]X " min
[0,u]X = H
6
!/0 =
@AAAB
AAAC
inf4u , [0, !0] : Xu = min
[0,!0]X6
if X!0 = max[0,!0]
X
inf4u , [0, !0] : Xu = max
[0,!0]X6
if X!0 = min[0,!0]
X
H H
!/0 !/
0!0 !0
tt
XX
II-7
Next step: we define by induction
!n+1 =
@AAAAB
AAAAC
inf4u > !n : max
[xn,u]X " Xu = H
6if X!n " X!/
n= H
inf4u > !n : max
[xn,u]X " Xu = H
6if X!n " X!/
n= "H
!/n+1 =
@AAAAB
AAAAC
inf4u , [!n, !n+1] : Xu = max
[!n,!n+1]X6
if X!n " X!/n= H
inf4u , [!n, !n+1] : Xu = min
[!n,!n+1]X6
if X!n " X!/n= "H
II-8
Kagi and Renko variation (on [0, T ]):
KT (X;H) =N#
n=1
|X!/n"X!/
n"1|, N =NT(X;H),
RT (X;H) =M#
n=1
|X&/n"X&/n"1|, M =MT(X;H).
Kagi and Renko volatilities (on [0, T ]):
kT (X;H) =KT (X;H)
MT(X;H),
rT (X;H) =RT (X;H)
MT(X;H).
II-9
THEOREM. If X = !B, then
1) kT (!B;H) & 2H,
NT &T!2
H2(P-a.s.),
KT = kTNTP&
2T!2
H2;
2) rT (!B;H)P& 2H,
MT &T!2
2H2(P-a.s.),
RT = rTMTP&
T!2
H.
II-10
Results of the statistical analysis
of some stock prices
X = (Xt)t!0 % Future on Index SP500 (Emini-SP500 Futures)
1 point = $ 50
2002-2003 (471 trading days)
" = 1 sec, Xt is the value of the last transaction at time t.
H 1 1.25 1.5 2 2.25 2.5 3 4rT(X;H)
H1.83 1.84 1.86 1.88 1.86 1.88 1.80 1.69
RENKO
H 1 1.25 1.5 2 2.25 2.5 3 4kT(X;H)
H1.83 1.85 1.85 1.89 1.91 1.93 1.92 1.87
KAGI
Almost the same results are valid for Futures on Index Nasdaq 100
(Emini-Nasdag100 Futures), 1 point = $ 20
II-11
0.5
1
1.5
2
2.5
3
0.75 1.25 1.75 2.25 2.75 3.25 3.75 4.251 1.5 2 2.5 3 3.5 4 4.5
Kagi
Renko
For EESR (United Energy System of
Russia)
rT (X;H)
H& 1.99 &
kT (X;H)
H.
II-12
Let us say that X-market has
r(H)-property if E rT (X;H) & r(H) · Hk(H)-property if E kT (X;H) & k(H) · H
(For a Brownian motion r(H) = k(H) = 2.)
II-13
Define Renko strategy .R = (.Rt )t!0 with
.Rt =
#
n!1
sgn!X&n"1 " X&/n"1
"I[&n"1,&/n"1)
(t)
.!I(k(H)!2) " I(k(H)<2)
", t ! 0,
and the corresponding capital
C.R
t =5 t
0.Ru dXu " (
5 t
0|d.R
u |.
Then
limt)%
EC.R
t
Mt= |r(H) " 2| · H " 2(.
The similar result is valid for the Kagi strategy .K = (.Kt )t!0.
II-14
&/m"1
&m"1
&m
&/m t
X
If R(H)>2, then
we buy at &m"2, &m, . . .
we sell at &m"1, &m+1, . . .
(33) (**)
If R(H)<2, then
we buy at &m"1, &m+1, . . .
we sell at &m"2, &m, . . .
(3*) (*3)
II-15
2. Prediction of time of maximum value of prices observable
on time interval [0, T ]
We would like to present now several our probability and statistical
approaches to solving some other problems of the technical analysis.
Problem. When to sell stock optimally?
We shall describe prices
by a Brownian motion
B = (Bt)0't'1; / is a
point of maximum of B:
B/ = max0't'1
Bt.
!
"..
...##/
/////0001
1111111110000**#
##*
*%%%%!
"
!
t
1'
/'
B/
II-16
Suppose that we begin to observe this process at time t = 0
(“morning time”), and, using only past observations, we stop at
time ' declaring “alarm” about selling. It is very natural to try to
solve the following problem: to find “optimal” times '/ and '// such
that either
inf0'''1
E |B/ " B' |2 = E |B/ " B'/|2
orinf
0'''1E |/ " ' | = E |/ " '//|.
For us it was a little bit surprising that here the optimal stopping
times coincide: '// = '/. The solution shows that
'/ = inf4t ' 1 : max
s'tBs " Bt ! z/
-1 " t
6,
where z/ is a certain (known) constant (z/ = 1.12 . . .).
II-17
This problem belongs to the theory of optimal stopping and methodof its solution is based on reducing to the special free-boundaryproblem.
1
1
τ
t z*√1- t
t*
It is interesting to note that
E '/ = 0.55 . . . , D'/ = 0.05 . . . .II-18
The cases Bµt = µt + Bt instead of Bt are more complicated.
If µ > 0 and µ is away from 0, then
'/ = inf{t ' 1 : b1(t) ' Sµt " Bµ
t ' b2(t)}
where • Bµt = µt + Bt, Sµ
t = maxu't Bµu,
• b1(t) and b2(t) have the following form:
b1
D
γ1
T0
b2
γ2
C
t*u*
C
#Here C is
the area of
continuation of
observations, D
is the stopping
area.$
II-19
If µ > 0 and µ is close to 0, then the corresponding picture has the
following form:
γ1
b2
γ2
C
u*
C
b1
D
T0
C
s*
C
II-20
For µ < 0 and if µ is far from 0, the picture is as follows:
T
b1C
D
γ1
0
C
II-21
In the considered problem, the time / is a “change point” of the
changing of the directions of trend
t1/
Bt
Solution of the problem
“ inf'
E |B' " B/|2 ”
or the problem “ inf' E |'"/|”depends, of course, on the
construction at any time t
a “good” prediction of the
change point /. The natural
estimate of / should be
based on the a posteriori
probability -t = P(/ ' t | FBt ), where FB
t = !(Bs, s ' t).
II-22
Stochastic analysis shows that
-t = 2,
>St " Bt-
1 " t
?
" 1, St = maxu't
Bu,
that explains appearing of the expression
St " Bt-1 " t
which is involved above in the definition of optimal stopping time
'/ = inf
9
t ' 1 :St " Bt-
1 " t! z/
:
.
Statistics St " Bt is appearing in many problems of the financial
mathematics and financial engineering (and, generally, in the mathematical
statistics under name CUSUM statistics).
Now we are going to tackle the following problem, which is interesting,
e.g., from the point of view of the quickest detection of arbitrage.
II-23
3. Quickest detection of time of appearing of arbitrage
Problem. Suppose we observe the prices
Xt = r(t"/)++!Bt
or
dXt =
@B
C! dBt, t ' /,
r dt + ! dBt, t > /./
Here a “change point” / is considered as a time of appearing of
arbitrage. (Brownian motion’s prices correspond to the non-arbitrage
situation. Brownian motion with drift corresponds to a case of
arbitrage.)
One very di&cult question here is “what is /?”. There are two
approaches. In the first one we assume that / is a random variable.
II-24
Suppose that ' is time of “alarm” /. Consider two events
{' < /} and {' ! /}.The set {' < /} is the event of a false alarm with a (false alarm)probability P(' < /).
# # #
'
/
'
{'</}& '$ %
$ %& '{'!/}
From a financial point of view
an interesting characteristic of
the event {' ! /} is a delay
time E('"/ | ' ! /) or E('"/)+.
These considerations lead to the following problem: in the classM$ = {' : P(' < /) ' $}, i.e., in the class of stopping times withthe probability of false alarm P(' < /) which less or equal the fixedlevel $, one need to find optimal stopping '/$ , M$ such that
inf',M$
E(' " /)+ = E('/$ " /)+.
II-25
It turned out that it is not a simple problem if we consider anarbitrary distribution for /. However, there exists one case when wemay solve this problem in implicit form. This case is the following.
Assume that / has the exponential distribution:
P(/ = 0) = - and P(/ > t | / > 0) = e"(t,
where ( is a given positive constant and - , [0,1). This assumptionis very reasonable. Indeed, for A < a < b < B
lim()0
P
!/ , (a, b) | / , (A, B)
"=
|b " a||B " A|
.
It means that in limit (( ) 0) the conditional distribution of / isuniform, that is, in some sense the worst possible case from pointof view of uncertainty of time of appearing of a change point /.
We describe now the results about structure of the optimal stoppingtime '/$.
II-26
Denote -t = P(/ ' t | FXt ), where FX
t = !(Xs, s ' t). This process
satisfies the following nonlinear stochastic di!erential equation:
d-t =D( "
r2
!2-2
t
E(1 " -t) dt +
r
!2-t(1 " -t) dX'
with -0 = -.
Then it turns out that an optimal stopping time '/$ is given by
'/$ = inf{t : -t ! B/$},
where (for case - = 0, for simplicity)
B/$ = 1 " $.
Second formulation of the quickest detection of arbitrage assumes
that / is simply a parameter from [0,%). In this case we denote by
P/ the distribution of the process X under the assumption that a
change point is occurred at time /.
II-27
By P% we denote the distribution of X under assumption that thereis no change point at all. Denote for given T > 0
MT = {' : E% ' ' T}the class of stopping time for which the mean time E% ' before(false) alarm is less or equal to T .
Put also
C(T) = inf',MT
sup/
ess sup*
E/
((' " /)+ | F/
)(*).
We proved that for each T > 0 in the class MT there exists anoptimal strategy with the following structure: declare alarm at time
'/T = inf4t : max
u'tXu " Xt ! a/(T)
6,
where a/(T) is a certain constant. It is interesting to note that (ifr2/(2!2) = 1)
C(T) & logT, T ) %.
II-28
The given method, based on the
“CUSUM statistics maxX " X”,
also is asymptotically optimal for more tractable criteria
D(T) = inf',MT
sup/
E/(' " / | ' ! /)
(We don’t know what is an optimal method for D(T)-criterion.)
Asymptotically, again
D(T) & logT, T ) %.
II-29
4. Drawdowns as the characteristics of risk
From given above exposition we observe importance of the “maxX"X”-characteristics for taking optimal decisions. Now we would like to
discuss that statistics and related ones in the problems of measure
of risk. There is a special terminology for such an object which
related to words “drawdown”, or “downfall”.
In practice a “drawdown” on time interval [0, t] is defined as the
percent change in a manager’s net asset value
– from any newly established peak to
a subsequent through,
– from a high “water mark” to the next
low “water mark”.
II-30
From the theoretical point of view,
Drawdown is a statistical measure of risk for investments; a
competitor to the standard measure of risk such
as return probability, VaR, Sharpe ration, etc.
There are various definitions of drawdown’s characteristics, whichmeasure the decline in net asset value from the historic high point.
In one financial paper we read that
. . . Measuring risk through extreme losses is a very
appealing idea. This is indeed how financial companies
perceive risks. This explains the popularity of loss
statistics as the maximum drawdown and maximum
loss. . .
and
. . . it does not seem possible to derive exact results for
the expected maximum drawdown.
II-31
Looking forward:
• What kinds of drawdowns should we expect over any given investment
horizon?
• How many drawdowns should be experienced?
• How big?
Under the
Commodity Futures Trading Commission’s
(CFTC)
mandatory disclosure regime managed futures advisors are obliged
to disclose, as part of their capsule performance records, their
“worst peak-to-valley drawdown”.
We shall demonstrate here some our theoretical calculations related
to drawdowns.II-32
Let Bµt = µt + !Wt be a Brownian motion with drift, W0 = 0.
There are several interesting characteristics related to
!
"
!
2
maxs't
Bµs
TµH
H
Range:
Rµt = max
s'tBµ
s " mins't
Bµs
Statistics T µH for Bµ:
TµH = inf
4t ! 0 : max
s'tBµ
s " Bµt ! H
6
Range,
Drawdowns,
Downfalls,. . .
II-33
If µ = 0:
E T0H =
>H
!
?2
, E maxt'T0
H
B0t = H,
DT0H =
2
3
>H
!
?4
, E e"(T0H =
1
cosh
>H
!
-2(
?
.
If µ 4= 0:
E TµH =
!2
2µ2
-
exp42µ
!2H6" 1 "
2µ
!2H
.
,
E maxt'Tµ
H
Bµt =
!2
2µ
-
exp42µ
!2H6" 1
.
.
II-34
Towards a problem from Kolmogorov’s diary (1944):
...For free (or not) random walk: How Xt drops when Xt falls for
the first time (on (t " ", t)) from above to some level +? To all
appearance, certainly very steeply!..
!
"
'(H)
H
0T
!
B(H)t = H + Bt, B(H)
0 = H, Bt = B0t
'(H) = inf{u : B(H)u = 0}
F(t) = P
!'(H) ' t | mins'T Bs ' 0
"
f(t) =dF(t)
dt
=H-
T
2G(H/-
T )t"3/2e"H2/2t, t ' T,
G(x) =5 %
xe"u2/2 du
II-35
The following three characteristics of drawdowns are the most important:
1 Maximumdrawdown
") Dt = max0's's2't
(Bs " Bs2)
(cf. Rt = max0's,s2't(Bs " Bs2); so Dt ' Rt).
"
!
t
B
Dt
II-36
2
Drawdown fromhigh “watermark” to thenext low “watermark”
")Dt = B!t " min
!t's2'tBs2
= max0's't
Bs " min!t's2't
Bs2
(where !t = inf{s ' t : Bs = maxu't
Bu}.
II-37
3
Drawdown fromprevious high“water mark”to the lowest“water mark”
") Dt = max0's'!2
t
Bs " B!2t
(where !2t = inf{s ' t : Bs = minu't Bu}.
"
!
!2t t
B
Dt
II-38
General results on D, D, D for B:
(1) Dt = Dt
(2) Dt = max(Dt, Dt)
(3) Dtlaw= max
s't|Bs|
(4) Dtlaw= max
gt's't|Bs|
where gt = sup{s ' t : Bs = 0}.
II-39
Distributional results on D1, D1 for a standard Brownian motion
B = B":
(5) P(D1'x) = P
Dmaxs'1
|Bs|'xE
=4
-
%#
n=0
("1)n
2n+1exp
4"
-2(2n+1)2
8x2
6
E D1 = Emaxs'1
|Bs| =<
-2 = 1.2533 . . .
E Dt = !-
t<
-2 (for !B+ on [0, t])
(6) P(D1 ' x) = P
Dmax
g1's'1|Bs| ' x
E= FD1
(x)
fD1(x) =
dFD1(x)
dx=<
8-
%#
k=1
("1)k"1ke"12k2x2
E D1 =<
8- log 2 = 1.1061
II-40
Note that
fR1(x) =
8-2-
%#
k=1
("1)k"1k2e"k2x2/2, x > 0,
FK1(x) = P
Dmaxs'1
|bs| ' xE,
fK1(x) =
dFK1(x)
dx= 8x
%#
k=1
("1)k"1k2e"k2x2,
where b = (bs)s'1 is a Brownian bridge (bs = Bs " sB1).
Since fR1(x) =
<2-
1x fK1
(x), we have E R1 =<
8- = 1.5957 . . . and
E D1 ' E D1 ' E R1=8
-log 2 '
3-
2'
=8
-1.1061 . . . ' 1.2533 . . . ' 1.5957 . . .
II-41
LEMMA.
(1) Dtlaw= Dt
(2) Dt =
@B
CDt = Dt on {!t ' !2
t}
max(Dt, Dt) on {!t > !2t}
(3) max(Dt, Dt) = Dt ' Rt
Known results about Rt and Td
R = R1: t = 1, µ = 0, ! = 1
W. Feller (1951) got for fR(x) =dP(R ' x)
dx, x > 0, the following
formula:
fR(x) =8-2-
%#
k=1
("1)k"1k2e"k2x2
2 .
II-42
REMARK. If b(t) = Bt " tB1, t ' 1, is a Brownian bridge, then forKolmogorov’s distribution FK(x) = P
!supt'1
|b(t)| ' x"
we have
FK(x) = 1 " 2%#
k=1
("1)k"1e"2k2x2=
-2-
x
%#
k=1
e"(2k"1)2-2/x2
5 (/-function)
fK(x) = 8x%#
k=1
("1)k"1k2e"2k2x2.
Since fR(x) =8-2-
%#
k=1
("1)k"1k2e"k2x2
2 , we get
fR(x) =
=2
-
1
xfK(x),
so
E R =
=8
-(= 1.5957691216 . . .)
II-43
THEOREM. (t = 1, µ = 0, ! = 1)
(a) D1law= max
0't'1|Bt|
(b) If FD1(x) = P(D1 ' x) then (it is well known)
FD1(x) = 1 "
1-2-
%#
k="%
5 x
"x
0e"
(y+4kx)2
2 " e"(y+2x+4kx)2
2
1dy
=4
-
%#
n=0
("1)n
2n + 1e"-2(2n+1)2
8x2
(c) E D1 = E max0't'1
|Bt| =<
-2
(E DT = !-
T<
-2)
II-44
Proof. (a): Denote
Mt = maxs't
Bs, Lt = lim#*0
1
2#
t5
0
I(|Bs| ' #) ds.
By Levy’s theorem
(Mt " Bt, Mt; t ' 1)law= (|Bt|, Lt; t ' 1).
Hence
D1 = max0's's2'1
(Bs"Bs2) = max0's2'1
!max
0's's2Bs " Bs2
"
= max0's2'1
(Ms2 " Bs2)law= max
0't'1|Bt|.
II-45
Proof. (c): We give two proofs. Let % = (%t)t!0 be a Brownian
motion. From self-similarity
(%at; t ! 0)law= (a1/2%t; t ! 0).
So if s1 = inf{t ! 0 : |%t| = 1}, then
P!
sup0't'1
|%t| ' x"= P
!supt'1
|%t/x2| ' 1"= P
!sup
t'1/x2|%t| ' 1
"
= P
Ds1 !
1
x2
E= P
D1
-s1
' xE,
i.e.,
supt'1
|%t|law=
1-
s1.
II-46
The normal distribution property=
2
-
%5
0
e" x2
2!2 dx = !
5
E D = E sup0't'1
|%t| = E1
-s1
=
=2
-
%5
0
E e"x2s12 dx
II-47
We have E e"(s1 =1
cosh-
2(. Hence
E D =
=2
-
5 %
0
dx
coshx= 2
=2
-
%5
0
ex dx
e2x + 1
= 2
=2
-
%5
1
dy
1 + y2= 2
=2
-arctan(x)
FFFFF
%
1
= 2
=2
-
-
4=3
-
20 E D =
3-
2
II-48
Second proof of the equality E D =3
-
2is based on the fact that
supt'1
|%t|law=
1
2
5 1
0
du
R(2)u
,
where R(2)s is a Bessel-2:
R(2)s = G%s +
1
2
5 s
0
du
R(2)u
.
Thus,
E D = E sup |%t| = E R(2)1 = E
<+21 + +22 =
3-
2,
+166 +2, +i & N (0,1).
II-49
THEOREM. (t = 1, µ = 0, ! = 1, D1 = B!1 " min!1's2'1 Bs2)
(a) D1law= sup
g1's'1|Bs|, where g1 = sup{t ' 1 : Bt = 0}.
(b) fD1
(x) =
=8
-
%#
k=1
("1)k"1ke"k2x2
2 , x > 0.
(c) E D1 =
=8
-log 2 (= 1.1061 . . .),
E D1 ' E D1 ' E R
<8- log 2 '
<-2 '
<8-
1.1061... ' 1.2533... ' 1.5957...
II-50
Proof: By Levy’s theorem
>Mt " Bt, Mt, Bt;
t ' 1
?law=
>|Bt|, Lt, Lt " |Bt|;t ' 1
?
5>
Mt " Bt, Mt, Bt;
!1 ' t ' 1
?law=
>|Bt|, Lt, Lt " |Bt|;
g1 ' t ' 1
?
where !1 = min7s ' 1 : Bs = max
u'1Bu
8.
II-51
Therefore!B!1, max
!1't'1(Mt " Bt " Mt)
"
law=
!Lg1 " |Bg1|, max
g1't'1(|Bt|" Lt)
"
=!Lg1, max
g1't'1|Bt|" Lg1
"
(since Bg1 = 0 and Lt = Lg1 for g1 ' t ' 1).
Finally,
D1 = B!1 " min!1't'1
Bt = B!1 + max!1't'1
("Bt)
= B!1 + max!1't'1
(Mt " Bt " Mt)
law= Lg1 + max
g1't'1|Bt|" Lg1 = max
g1't'1|Bt|.
II-52
TOPIC III. Technical Analysis. II
1. Buy and Hold
2. Stochastic Buy an Hold
(rebalancing of the portfolio)
III-1-1
TOPIC III. 1: BUY & HOLD
Consider (B, S)-market. For example: dB = rB dt
dS = S(µ dt + ! dWt)
Pt =St
Bt, MT = max
t'TPt, mT = min
t'TPt.
PROBLEMS:
Buying: (1) inf''T
E UD
S'
MT
E
(2) inf''T
E UD
S'
mT
E
Selling: (1) sup''T
E UD
S'
MT
E
(2) sup''T
E UD
S'
mT
E
where U = U(x) is a “utility function”.
III-1-2
Interesting cases: U(x) = x, U(x) = log x
For U(x) = x :
(1): inf''T
EUD
S'
MT
E& sup
''TEU
DMT " S'
MT
E
3Maximization of the expected relative
error between the buying price and the
highest possible stock price by choosing
a proper time to buy
(2): Maximization of the expected relative error between
the buying price and the lowest possible stock price
III-1-3
For U(x) = log x :
E logP'
MT= E logP' " E logMT, E log
P'/TMT
= E logP'/T" E logMT.
Take Pt = exp4D
µ " r "!2
2
Et + !Wt
6. Then
E logP' = E
0Dµ " r "
!2
2
E' + !W'
1,
sup''T
E logP' = sup''T
E
0Dµ " r "
!2
2
E' + !W'
1
= sup''T
E
0Dµ " r "
!2
2
E'1.
Thus, '/T =
@B
C0, µ " r " !2/2 ' 0,
T, µ " r " !2/2 > 0.(#)
III-1-4
Problem (1) for U(x) = x is more di&cult.
However, it is remarkable that answer is the same:
sup''T
EP'
MT= E
P'/TMT
, where '/T is given by (#).
The first step of solving the nonstandard problem sup''T
EP'
MTis its
reduction to the standard problem of the type
V = sup''T
EG(X')
for some Markov process X and Markov time ' (with respect to
(FXt )t!0, FX
t = !(Xs, s ' t)).
III-1-5
Taking E( · | F'), F' = FS' , where FS
t = !(Su, u ' t), we find
EP'
MT= E E
DP'
MT
FFFF F'
E= EG(', M)
' " W )' ),
where
) = µ " r " !2/2, W )t = )t + !Wt, M)
t = max0'u't
Wu,
G(t, x) = E(e"x 7 e"M)
T"t), t , [0, T ], x , [0,%).
For a given ), the process Xt = M)t " W )
t is a Markov process.
III-1-6
So the problem sup!$T
E(P!/MT ) is reduced to sup!$T
EG(!, X! ).
For G(t, x) we can find explicit expressions:
G(t, x) =2() " 1)
2) " 1e"()"1/2)(T"t)$
D"x + () " 1)(T " t)-T " t
E
+1
2) " 1e"(1"2))x$
D"x " )(T " t)-T " t
E
+ e"x$D
x " )(T " t)-T " t
E, if " %=
1
2,
G(t, x) =D1 + x +
T " t
2
E$D"x " (T " t)/2-
T " t
E
"=
T " t
2-exp
4"
(x + (T " t)/2)2
2(T " t)
6
+ e"x$D
x " (T " t)/2-T " t
E, if " =
1
2.
III-1-7
Solution of the problem “supt$T EG(!, X! )”:
By dynamic programming methods we must find for 0 ' t ' T
and x , [0,%)
V (t, x) = sup''T"t
ExG(t + ', Xxt+'), where Xx
t = x 7 M)t " W )
t .
It is clear that V (0,0) = sup''T E(P'/MT).
Introduce the sets
D =7(t, x) , [0, T ] . [0,%): V (t, x) = G(t, x)
8,
C =7(t, x) , [0, T) . [0,%): V (t, x) > G(t, x)
8;
the set D is a set of stopping observation and
the set C is a continuation area.
III-1-8
To find V = V (t, x) and sets C and D we usually consider the Stefan
(free-boundary) problem: to find GV = GV (t, x), GC, and HD such that
Lx GV (t, x) = 0, (t, x) , GC, GV (t, x) = G(t, x), (t, x) , HD,
where Lx is the infinitesimal operator of the process Xx = (Xxt )t!0,
x ! 0. We know that
Law(Xxt , t ' T) = Law(|Y x
t |, t ' T),
where Y x is a “bang-bang” process:
dY xt = ") sgnY x
t dt + dIWt, Y x0 = x.
III-1-9
By the Tanaka formula
|Y xt | = |x|+
5 t
0sgnY x
s dY xs +Lt(Y
x) = |x|")t+5 t
0sgnY x
s dIWs+Lt(Yx),
where Lt(Y x) is the local time of Y x at zero over time interval [0, t].
Since Law(Xu) = Law(|Y x|), from the previous formula we find that
Lxf(t, x) =0f
0t" )
0f
0x+
1
2
02f
0x2, t , (0, T), x , (0,%),
for f , C1,2 with reflection condition0f
0x(t,0+) = 0.
Free-boundary method can be applied here. However, it is more
convenient to use the following “direct” method.
III-1-10
If G(t, x) , C1,2, then, by the Ito formula,
G(t + s, |Y xs |) = G(t, s) +
5 s
0LxG(t + u, |Y x
u |) du
+5 s
0
0G
0x(t + u, |Y x
u |) dIWu
$ %& 'is a martingale,
where "1 ' 0G0x ' 0
+5 s
0
0G
0x(t + u, |Y x
u |) dLu(Yx)
$ %& '= 0,
since 0f0x(t,0+) = 0
.
So, V (t, x) = sup''T"t
ExG(t + ', Xx' ) = sup
''T"tExG(t + ', |Y x
' |)
= G(t, x) + sup''T"t
Ex
5 '
0H(t + u, |Y x
u |) du
= G(t, x) + sup''T"t
Ex
5 '
0H(t + u, Xx
u) du,
where H(t, x) 1 LxG(t, x) =0G
0t" )
0G
0x+
1
2
02G
0x2.
III-1-11
From explicit formulae for G = G(t, x) we find that
H(t, x) =D) "
1
2
EG(t, x) "
0G
0x(t, x).
If ) ! 1/2, then 0G/0x ' 0 and H(t, x) ! 0.
From V (t, x) = G(t, x)+ sup''T"t
Ex
5 '
0H(t+u, Xx
u) du we conclude that
if ) ! 1/2, then V (t, x) ! G(t, x)
if ) > 1/2, then V (t, x) > G(t, x).
III-1-12
As a result we see that
if " > 1/2, then the optimal !#T equals T ;
if " = 1/2, then any time !#T = t is optimal.
For " < &1/2 we find that
H(t, x) =D) +
1
2
EG(t, x) "
DG(t, x) +
0G
0x(t, x)
$ %& '> 0
E.
So, in this case V (t, x) ' G(t, x). From here we conclude that
if " < &1/2, then the optimal !#T equals zero
The case &1/2 < " < 1/2 is more complicated and can be investigatedby the “free-boundary methods”.
III-1-13
So, we have the following result:
the optimal stopping time !#T is DETERMINISTIC:
!#T =
@B
C0, if ) 1 µ " r " !2/2 ' 0,
T, if ) 1 µ " r " !2/2 > 0.
III-1-14
POPULAR STOCK MODELS IN FINANCE:
St = S0eµ(t)+!(t)Bt (exponential Brownian model);
St = S0eµ(t)+; t0 !(s)dBs (stochastic volatility model);
St = S0eµ(t)+BT (t) (change-of-time model).
Instead of bT we can take a Levy process Lt.
The similar model can be considered for case of discrete time.
III-1-15
For example, let’s take Gaussian-Inverse Gaussian process
Ht = µt + %T(t) + BT(t) ,
where T(t) = inf{s > 0: /Bs +-
as !-
bt},( /Bs) and (Bs) are independent Brownian motions.
For case
sup''T
ESt
MT, St = eHt, MT = exp
4supt'T
Ht
6
we find that
if µ ' 0, # ' 0, then the optimal stopping time is !#T = T
III-1-16
TOPIC III. 2: STOCHASTIC “BUY & HOLD”
(PORTFOLIO REBALANCING)
III-2-1
TOPIC IV. General Theory of Optimal Stopping
Lecture 1. Introduction, pp. 2–46.
Lectures 2-3. Theory of optimal stopping for discrete time
(finite and infinite horizons)
A) Martingale approach B) Markovian approach
(pp. 47–85) (pp. 86–104)
Lectures 4-5. Theory of optimal stopping for continuous time
(finite and infinite horizons)
A) Martingale approach B) Markovian approach
(pp. 105–119) (pp. 120–146)
Essential references p. 147.
IV-1-1
Lecture 1. INTRODUCTION.
1. Connections of the Optimal stopping theory and the Mathematical
analysis (especially PDE-theory) are as well illustrated by the
Dirichlet problem for the Laplace equation:
to find a harmonic function u = u(x) in the class C2 in the
bounded open domain C 8 Rd, i.e., to find a function u , C2
that satisfies the equation
"u = 0, x , C, (/)
and the boundary condition
u(x) = G(x), x , 0D, where D = Rd \ C. (//)
IV-1-2
Let
'D = inf{t : Bxt , D},
where
Bxt = x + Bt
and B = (Bt)t!0 is a d-dimensional standard Brownian motion.
Then the probabilistic solution of the Dirichlet problem
"u = 0, x , C,
u(x) = G(x), x , 0D,
is given by the formula
u(x) = EG(Bx'D
), x , C 9 0DDu(x) = ExG(B'D)
E.
IV-1-3
The optimal stopping theory operates with the optimization
problems, where
• we have a set of domains C =7C : C 8 Rd
8and
• we want to find the function
U(x) = sup'D
ExG(B'D) , where G = G(x) is given for all x , Rd,
D , D =JD = C : C , C
K
or, generally, to find the function
V (x) = sup'
ExG(B') , where ' is an arbitrary finite
stopping time defined by the
process B.
IV-1-4
2. The following scheme illustrates the kind of concrete problems
of general interest that will be studied in the courses of lectures:
A. Theory of probability
sharp inequalities
B. Mathematical statistics
sequential analysis
C. Financial mathematics
stochastic equilibria
The solution method for problems A, B, C consists in reformulation
to an optimal stopping problem and reduction to a free-boundary
problem as stated in the diagram:
IV-1-5
A, B, C
!#
"$
!"#$1 !"
#$4
Optimal stopping problems
!#
"$
!"#$2 !"
#$3
Free-boundary problems
IV-1-6
3. To get some idea of the character of problems A, B, C that will
be studied, let us begin with the following remarks.
(A) Let B = (Bt)t!0 be a standard Brownian motion. Then
Wald identities:EBT = 0 and EB' = 0 if E
-' < %
EB2T = T and EB2
' = E' if E' < %
From Jensen’s inequality and E|B' |2 = E' we get
E|B' |p ' (E')p/2 for 0 < p ' 2
E|B' |p ! (E')p/2 for 2 ' p < %
B. Davis (1976): E|B' | ' z/1E-
' , z/1 = 1.30693 . . .
IV-1-7
Now our main interest relates with the estimation of the expectations
Emaxt''
Bt and Emaxt''
|Bt|.
We have
maxBlaw= |B|.
So,
Emaxt'T
Bt = E|BT | =
=2
-T
and
Emaxt''
Bt = E|B' | '
@B
C
-E' ,
z/1E-
' , z/1 = 1.30993 . . .
IV-1-8
The case of max |B| is more di&cult. We know that
P
>
maxt'T
|Bt| ' x
?
=4
-
%#
n=0
("1)n
2n + 1exp
>
"-2(2n + 1)2
8x2
?
.
From here it is possible to obtain (but it is not easy!) that
Emaxt'T
|Bt| =3
-
2T
!= 1.25331 . . .
".
(Recall that E|BT | =<
2-T (= 0.79788 . . .).)
IV-1-9
SIMPLE PROOF:
(Bat; t ! 0)law= (
-aBt; t ! 0).
Take ! = inf {t > 0 : |Bt| = 1}. Then
P
Dsup
0't'1|Bt| ' x
E= P
Dsup
0't'1|Bt/x2| ' 1
E
= P
Dsup
0't'1/x2|Bt| ' 1
E= P
D! !
1
x2
E= P
>1-
!' x
?
,
that is,
sup0't'1
|Bt|law=
1-
!
IV-1-10
The normal distribution property:
=2
-
5 %
0Ee
" x2
2a2 dx = a , a > 0. (#)
So,
E sup0't'1
|Bt| = E1-
!
(#)=
=2
-
5 %
0Ee
"x2!2 dx.
Since Ee"(! = 1cosh
-2(
, we get
E sup0't'1
|Bt| =
=2
-
5 %
0
dx
coshx= 2
=2
-
5 %
0
ex dx
e2x + 1=
=2
-
5 %
1
dy
1 + y2
= 2
=2
-arctan(x)
FFFFF
%
1
= 2
=2
-·-
4=
3-
2.
IV-1-11
E sup0't'1
|Bt| =3
-
2E sup
0't'T|Bt| =
3-
2T
In connection with MAX the following can be interesting. In his
speech delivered in 1856 before a grand meeting at the St.-Petersburg
University the great mathematician
P. L. Chebyshev (1821–1894)
has formulated some statements about the “unity of theory and
practice”. In particular he emphasized that
“a large portion of the practical questions can be stated in the
form of problems of MAXIMUM and MINIMUM... Only the
solution of these problems can satisfy the requests of practice
which is always in search of the best and the most e&cient.”
IV-1-12
4. Suppose that instead of maxt'T |Bt|, where, as already known,
E max0' t'T
|Bt| =3
-
2T ,
we have some random time ' and we want to find
E max0' t' !
|Bt| = ?
It is clear that it is virtually impossible
• to compute this expectation for every stopping time ' of B.
Thus, as the second best thing, one can try
• to bound it with a quantity which is easier computed.
A natural candidate for the latter is E' at least when finite.
In this way a PROBLEM A has appeared.
IV-1-13
Problem A leads to the following maximal inequality:
E
Dmax0't''
|Bt|E' C
-E' (3)
which is valid for all stopping times ' of B with the best constant
C equal to-
2.
We will see that the problem A can be solved in the form (3) by
REFORMULATION to the following optimal stopping problem:
V/ = sup'
E
Dmax0't''
|Bt|" c'E
, (4)
where
• the supremum is taken over all stopping times ' of B
satisfying E' < %, and
• the constant c > 0 is given and fixed.
It constitutes Step 1 in the diagram above.
IV-1-14
If V/ = V/(c) can be computed, then from (4) we get
E
Dmax0't''
|Bt|E' V/(c) + c E' (5)
for all stopping times ' of B and all c > 0. Hence we find
E
Dmax0't''
|Bt|E' inf
c>0
!V/(c) + c E'
". (6)
for all stopping times ' of B. The RHS in (6) defines a function of
E' that, in view of (4), provides a sharp bound of the LHS.
Our lectures demonstrate that the
optimal stopping
problem (4)can be reduced to a
free-boundary
problem
This constitutes Step 2 in the diagram above.
IV-1-15
Solving the free-boundary problem one finds that V/(c) = 1/2c.
Inserting this into (6) yields
infc>0
E
!V/(c) + c E'
"=
-2 E' (7)
so that the inequality (6) reads as follows:
E
Dmax0't''
|Bt|E'
-2 E' (8)
for all stopping times ' of B.
This is exactly the inequality (3) above with C =-
2.
The constant-
2 is the best possible in (8).
IV-1-16
In the lectures we consider similar sharp inequalities for other stochastic
processes using ramifications of the method just exposed.
Apart from being able to
• derive sharp versions of known inequalities
the method can also be used to
• derive some new inequalities.
IV-1-17
(B) Classic examples of problems in SEQUENTIAL ANALYSIS:
• WALD’s problem (“Sequential analysis”, 1947) of sequential
testing of two statistical hypotheses
H0 : µ = µ0 and H1 : µ = µ1 (9)
about the drift parameter µ , R of the observed process
Xt = µt + Bt , t ! 0, where B = (Bt)t!0 is
a standard Brownian
motion.
(10)
• The problem of sequential testing of two statistical hypotheses
H0 : ( = (0 and H1 : ( = (1 (11)
IV-1-18
about the intensity parameter ( > 0 of the observed process
Xt = N(t , t ! 0, where N = (Nt)t!0 is a
standard Poisson process.
(12)
The basic problem in both cases seeks to find the
optimal decision rule ('/, d/)
in the class "($, %) consisting of decision rules
(d, '), where ' is the time of stopping and
accepting H1 if d = d1 or
accepting H0 if d = d0,
such that the probability errors of the first and second kind satisfy:
P(accept H1 | true H0) ' $ (13)
P(accept H0 | true H1) ' % (14)
and the mean times of observation E0' and E1' are as small as
possible.
It is assumed that $ > 0 and % > 0 with $ + % < 1.
IV-1-19
It turns out that with this (variational) problem
one may associate an optimal stopping (Bayesian) problem
which in turn can be reduced to a free-boundary problem .
This constitutes Steps 1 and 2 in the diagram above.
Solving the free-boundary problem leads to an optimal decision rule
('/, d/) in the class "($, %) satisfying (13) and (14) as well as the
following two identities:
E0' = inf(',d)
E0' (15)
E1' = inf(',d)
E1' (16)
where the infimum is taken over all decision rules (', d) in "($, %).
This constitutes Steps 3 and 4 in the diagram above.
IV-1-20
In our lectures we study these as well as closely related problems of
QUICKEST DETECTION.
(The story of creating of the quickest detection problem of randomly
appearing signal, its mathematical formulation, and the route of
solving the problem (1961) are also interesting.)
Two of the prime findings, which also reflect the historical development
of these ideas, are the
principles of SMOOTH and CONTINUOUS FIT
respectively.
IV-1-21
C) One of the best-known specific problems of
MATHEMATICAL FINANCE,
that has a direct connection with optimal stopping problems, is the
problem of determining the
arbitrage-free price of the American put option.
Consider the Black–Scholes model, where the stock price X =
(Xt)t!0 is assumed to follow a geometric Brownian motion:
Xt = x expD!Bt + (r " !2/2) t
E, (17)
where x > 0, ! > 0, r > 0 and B = (Bt)t!0 is a standard Brownian
motion. By Ito’s formula one finds that the process X solves
dXt = rXt dt + !Xt dBt with X0 = x. (18)
IV-1-22
General theory of financial mathematics makes it clear that the
initial problem of determining the arbitrage-free price of the American
put option can be reformulated as the following optimal stopping
problem:
V/ = sup'
Ee"r'(K " X')+ (19)
where the supremum is taken over all stopping times ' of X.
This constitutes Step 1 in the diagram above.
The constant K > 0 is called the strike price. It has a certain
financial meaning which we set aside for now.
IV-1-23
It turns out that the optimal stopping problem (19):
V/ = sup'
Ee"r'(K " X')+
can be reduced again to a free-boundary problem which can be
solved explicitly. It yields the existence of a constant b/ such that
the stopping time
'/ = inf { t ! 0 | Xt ' b/ } (20)
is optimal in (19).
This constitutes Steps 2 and 3 in the diagram above.
Both the optimal stopping point b/ and the arbitrage-free price V/can be expressed explicitly in terms of the other parameters in the
problem. A financial interpretation of these expressions constitutes
Step 4 in the diagram above.
IV-1-24
In the formulation of the problem (19) above:
V/ = sup'
Ee"r'(K " X')+
no restriction was imposed on the class of admissible stopping
times, i.e. for certain reasons of simplicity it was assumed there
that
' belongs to the class of stopping times
M = { ' | 0 ' ' < % } (21)
without any restriction on their size.
IV-1-25
A more realistic requirement on a stopping time in search for the
arbitrage-free price leads to the following optimal stopping problem:
V T/ = sup
',MTEe"r'(K " X')
+ (22)
where the supremum is taken over all ' belonging to the class of
stopping times
MT = { ' | 0 ' ' ' T } (23)
with the horizon T being finite.
The optimal stopping problem (22) can be also reduced to a free-
boundary problem that apparently cannot be solved explicitly.
IV-1-26
Its study yields that the stopping time
'/ = inf { 0 ' t ' T | Xt ' b/(t) } (24)
is optimal in (22), where b/ : [0, T ] ) R is an increasing continuous
function.
A nonlinear Volterra integral equation can be derived which characterizes
the optimal stopping boundary t :) b/(t) and can be used to compute
its values numerically as accurate as desired.
The comments on Steps 1–4 in the diagram above made in the
infinite horizon case carry over to the finite horizon case without
any change.
In our lectures we study these and other similar problems that arise
from various financial interpretations of options.
IV-1-27
5. So far we have only discussed problems A, B, C and their reformulations
as optimal stopping problems. Now we want to address the methods
of solution of optimal stopping problems and their reduction to free-
boundary problems.
There are essentially two equivalent approaches to finding a solution
of the optimal stopping problem. The first one deals with the problem
V/ = sup',M
EG' in the case of infinite horizon, (25)
or the problem
V T/ = sup
',MTEG' in the case of finite horizon, (26)
where M = { ' | 0 ' ' ' %}, and MT = { ' | 0 ' ' ' T }.
IV-1-28
In this formulation it is important to realize that
G = (Gt)t!0 is an arbitrary stochastic process defined on
a filtered probability space (!,F , (Ft)t!0, P), where it is
assumed that G is adapted to the filtration (Ft)t!0 which
in turn makes each ' from M or MT a stopping time.
Since the method of solution to the problems (25) and (26) is based
on results from the theory of martingales (Snell’s envelope, 1952),
the method itself is often referred to as the
MARTINGALE METHOD.
IV-1-29
On the other hand, if we are to take a state space (E,B) large
enough, then one obtains the
“Markov representation” Gt = G(Xt)
for some measurable function G, where X = (Xt)t!0 is a Markov
process with values in E. Moreover, following the contemporary
theory of Markov processes it is convenient to adopt the definition
of a Markov process X as the family of Markov processes
((Xt)t!0, (Ft)t!0, (Px)x,E) (27)
where Px(X0 = x) = 1, which means that the process X starts at
x under Px. Such a point of view is convenient, for example, when
dealing with the Kolmogorov forward or backward equations, which
presuppose that the process can start at any point in the state
space.
IV-1-30
Likewise, it is a profound attempt, developed in stages, to study
optimal stopping problems through functions of initial points in the
state space.
In this way we have arrived to the second approach which deals with
the problem
V (x) = sup'
ExG(X') (28)
where the supremum is taken over M or MT as above (Dynkin’s
formulation, 1963).
Thus, if the Markov representation of the initial problem is valid, we
will refer to the
MARKOVIAN METHOD of solution.
IV-1-31
6. To make the exposed facts more transparent, let us consider the
optimal stopping problem
V/ = sup'
E
Dmax0't''
|Bt|" c'E
in more detail.
Denote
Xt = |x + Bt| (29)
for x ! 0, and enable the maximum process to start at any point by
setting for s ! x
St = s 7D
max0'r't
Xr
E. (30)
IV-1-32
St = s 7D
max0'r't
Xr
E
The process S = (St)t!0 is not Markov, but
the pair (X, S) = (Xt, St)t!0 forms a Markov process
with the state space
E = { (x, s) , R2 |0 ' x ' s }.
The value V/ from (4) above: V/ = sup'
E
Dmax0't''
|Bt|" c'E
coincides
with the value function
V/(x, s) = sup'
Ex,s
!S'" c'
"(31)
when x = s = 0. The problem thus needs to be solved in this more
general form.
IV-1-33
The general theory of optimal stopping for Markov processes makes
it clear that the optimal stopping time in (31) can be written in the
form
'/ = inf { t ! 0 | (Xt, St) , D/} (32)
where D/ is a stopping set, and
C/ = E \ D/ is the continuation set.
In other words,
• if the observation of X was not stopped before time t
since Xs , C/ for all 0 ' s < t, and we have that Xt , D/,then it is optimal to stop the observation at time t,
• if it happens that Xt , C/ as well, then the observation
of X should be continued.
IV-1-34
"
#$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$
x
ss = x
% #%(xt, st)
D/ C/Heuristic considerations about the
shape of the sets C/ and D/makes it plausible to guess that
there exist a point s/ ! 0 and
a continuous increasing function
s :) g/(s) with g/(s/) = 0 such
thatD/ = { (x, s) , R
2 | 0 ' x ' g/(s) , s ! s/ } (33)
Note that such a guess about the shape of the set D/ can be made
using the following intuitive arguments. If the process (X, S) starts
from a point (x, s) with small x and large s, then it is reasonable to
stop immediately because to increase the value s one needs a large
time ' which in the formula (31) appears with a minus sign.
IV-1-35
At the same time it is easy to see that
if x is close or equal to s then it is reasonable to continue
the observation, at least for small time ", because s will
increase for the value-
" while the cost for using this
time will be c", and thus-
" " c" > 0 if " is small
enough.
Such an a priori analysis of the shape of the boundary between the
stopping set C/ and the continuation set D/ is typical to the act of
finding a solution to the optimal stopping problem. The
art of GUESSING
in this context very often plays a crucial role in solving the problem.
IV-1-36
Having guessed that the stopping set D/ in the optimal stopping
problem V/(x, s) = sup' Ex,s(S'" c') takes the form
D/ = { (x, s) , R2 | 0 ' x ' g/(s) , s ! s/ },
it follows that '/ attains the supremum, i.e.,
V/(x, s) = Ex,s
!S'/" c'/
"for all (x, s) , E. (34)
Consider V/(x, s) for (x, s) in the continuation set
C/ = C1/ 9 C2
/ (35)
where the two subsets are defined as follows:
C1/ = { (x, s) , R
2 | 0 ' x ' s < s/ } (36)
C2/ = { (x, s) , R
2 | g/(s) < x ' s , s ! s/ }. (37)
IV-1-37
Denote by
LX =1
2
02
0x2
the infinitesimal operator of the process X. By the strong Markov
property one finds that V/ solves
LXV/(x, s) = c for (x, s) in C/. (38)
If the process (X, S) starts at a point (x, s) with x < s, then during
a positive time interval the second component S of the process
remains equal to s.
This explains why the infinitesimal operator of the process (X, S)
reduces to the infinitesimal operator of the process X in the interior
of C/.
IV-1-38
On the other hand, from the structure of the process (X, S) it followsthat at the diagonal in R2
+
• the condition of normal reflection holds:
0V/0s
(x, s)
FFFFFx=s"
= 0. (39)
Moreover, it is clear that for (x, s) , D/• the condition of instantaneous stopping holds:
V/(x, s) = s. (40)
Finally, either by guessing or providing rigorous arguments, it isfound that at the optimal boundary g/
• the condition of smooth fit holds:
0V/0x
(x, s)
FFFFFx=g/(s)+
= 0. (41)
IV-1-39
This analysis indicates that the value function V/ and the optimal
stopping boundary g/ can be obtained by searching for the pair of
functions (V, g) solving the following free-boundary problem:
LXV (x, s) = c for (x, s) in Cg (42)0V
0s(x, s)
FFFFx=s"
= 0 (normal reflection) (43)
V (x, s) = s for (x, s) in Dg (instantaneous stopping) (44)0V
0x(x, s)
FFFFx=g(s)+
= 0 (smooth fit) (45)
where the two sets are defined as follows (g(s0) = 0):
Cg = { (x, s) , R2 | 0 ' x ' s < s0 or g(s) < x ' s, s ! s0 } (46)
Dg = { (x, s) , R2 | 0 ' x ' g(s) , s ! s0 } (47)
It turns out that this system does not have a unique solution so
that an additional criterion is needed to make it unique in general.
IV-1-40
Let us show how to solve the free-boundary problem (42)–(45) bypicking the right solution (more details will be given in the lectures).
From (42) one finds that for (x, s) in Cg we have
V (x, s) = cx2 + A(s) x + B(s) (48)
where A and B are some functions of s. To determine A and B aswell as g we can use the three conditions
0V
0s(x, s)
FFFFx=s"
= 0 (normal reflection)
V (x, s) = s for (x, s) in Dg (instantaneous stopping)0V
0x(x, s)
FFFFx=g(s)+
= 0 (smooth fit)
which yield
g2(s) =1
2(s " g(s)), for s ! s0. (49)
IV-1-41
It is easily verified that the linear function
g(s) = s "1
2c(50)
solves (49). In this way a candidate for the optimal stopping boundary
g/ is obtained.
For (x, s) , E with s ! 12c one can determine V (x, s) explicitly using
V (x, s) = cx2 + A(s) x + B(s)
and
g(s) = s "1
2c.
This in particular gives that V (1/2c,1/2c) = 3/4c.
IV-1-42
For other points (x, s) , E when s < 1/2c one can determine V (x, s)
using that the observation must be continued. In particular for x =
s = 0 this yields that
V (0,0) = V (1/2c,1/2c) " c E0,0(!) (51)
where ! is the first hitting time of the process (X, S) to the point
(1/2c,1/2c).
Because E0,0(!) = E0,0(X2!) = (1/2c)2 and V (1/2c,1/2c) = 3/4c,
we find that
V (0,0) =1
2c(52)
as already indicated prior to (7) above. In this way a candidate for
the value function V/ is obtained.
IV-1-43
The key role in the proof of the fact that
V = V/ and g = g/
is played by
Ito’s formula (stochastic calculus) and the
optional sampling theorem (martingale theory).
This step forms a VERIFICATION THEOREM that makes it
clear that
the solution of the free-boundary problem coincides
with the solution of the optimal stopping problem
IV-1-44
7. The important point to be made in this context is that the
verification theorem is usually not di&cult to prove in the cases
when a candidate solution to the free-boundary problem is obtained
explicitly.
This is quite typical for one-dimensional problems with infinite
horizon, or some simpler two-dimensional problems, as the one just
discussed.
In the case of problems with finite horizon, however, or other
multidimensional problems, the situation can be radically di!erent.
In these cases, in a manner quite opposite to the previous ones,
the general results of optimal stopping can be used to prove the
existence of a solution to the free-boundary problem, thus providing
an alternative to analytic methods.
IV-1-45
8. From the material exposed above it is clear that our basic interest
concerns the case of continuous time.
The theory of optimal stopping in the case of continuous time is
considerably more complicated than in the case of discrete time.
However, since the former theory uses many basic ideas from the
latter, we have chosen to present the case of discrete time first, both
in the martingale and Markovian setting, which is then likewise
followed by the case of continuous time. The two theories form
several my lectures.
IV-1-46
LECTURE 2–3:Theory of optimal stopping for discrete time.
A. Martingale approach.
1. Definitions
(!,F , (Fn)n!0, P), F0 8 F1 8 · · · 8 Fn 8 · · · 8 F , G = (Gn)n!0.
Gain Gn is Fn-measurable
Stopping (Markov) time ' = '(*):
' : ! ) {0,1, . . . ,%}, {' ' n} , Fn for all n ! 0.
M is the family of all finite stopping times
M is the family of all stopping times
MNn = {' , M |n ' ' ' N}
For simplicity we will set MN = M
N0 and Mn = M
%n .
IV-2/3-1
The optimal stopping problem to be studied seeks to solve
V/ = sup'
E G' . (53)
For the existence of E G' suppose (for simplicity) that
E sup0'k<%
|Gk| < % (54)
(then E G' is well defined for all ' , MNn , n ' N < %).
In the class MNn we consider
V Nn = sup
',MNn
E G' , 0 ' n ' N. (55)
Sometimes we admit that ' in (53) takes the value % (P(' = %) >0), so that ' , M. We put G' = 0 on {' = %}.
Sometimes it is useful to set G% = limsupn)%
Gn.
IV-2/3-2
2. The method of backward induction.
V Nn = sup
n'''NE G'
To solve this problem we introduce (by backward induction) a special
stochastic sequence SNN , SN
N"1, . . . , SN0 :
SNN = GN, SN
n = max{Gn, E(SNn+1 | Fn)},
n = N " 1, . . . ,0.
If n = N we have to stop and our stochastic gain SNN , equals GN .
#&&&&&&&&&' (
((
((
((
(() *********+,,,,,,- *********+ ,,,,,,,,,-
0 1 2 N " 2 N " 1 N
&
Stop at time N
!
IV-2/3-3
For n = N " 1 we can either stop or continue. If we stop, our gain
SNN"1, equals GN"1, and if we continue our gain SN
N"1 will be equal
to E(SNN | FN"1).
#&&&&&&&&&' ,,,,,,,,,- *********+
&&&&&&' ,,,,,,,,,- ..
..
..
../
0 1 2 N " 2 N " 1 N
!
either stop at time N " 1
or continue and stop at time N
&
&"
So,SN
N"1 = max{GN"1, E(SNN | FN"1)}
and optimal stopping time is
'NN"1 = min{N " 1 ' k ' N : SN
k = Gk}.
IV-2/3-4
Define now a sequence (SNn )0'n'N recursively as follows:
SNn = GN, n = N,
SNn = max{Gn, E(SN
n+1 | Fn)}, n = N"1, . . . ,0.
The described method suggests to consider the following stopping
time:
'Nn = inf{n ' k ' N : SN
k = Gk} for 0 ' n ' N .
The first part of the following theorem shows that SNn and 'N
n solve
the problem in a stochastic sense.
The second part of the theorem shows that this leads also to a
solution of the initial problem
V Nn = sup
n'''NE G' for each n = 0,1, . . . , N.
IV-2/3-5
Theorem 1. (Finite horizon)
I. For all 0 ' n ' N we have:
(a) SNn ! E(G' | Fn), ;' , M
Nn ;
(b) SNn = E(G'N
n| Fn).
II. Moreover, if 0 ' n ' N is given and fixed, then we have:
(c) 'Nn is optimal in V N
n = supn'''N
E G' ;
(d) if '/ is also optimal then 'Nn ' '/;
(e) the sequence (SNk )n'k'N is the smallest
supermartingale which dominates (Gk)n'k'N
(Snell’s envelope)
(f) the stopped sequence (SNk<'N
n)n'k'N is a
martingale.
IV-2/3-6
Proof of Theorem 1.
I. Induction over n = N, N"1, . . . ,0.
Conditions
(a) SNn ! E(G' | Fn), ;' , M
Nn ,
and
(b) SNn = E(G'N
n| Fn)
are trivially satisfied for n = N .
Suppose that (a) and (b) are satisfied for n = N, N"1, . . . , k, where
k ! 1, and let us show that they must then also hold for n = k"1.
IV-2/3-7
(a)!SN
n ! E(G' | Fn), ;' , MNn
": Take ' , MN
k"1 and set ' = ' 7 k;
then ' , MNk , and since {' !k} , Fk"1 it follows that
E(G' | Fk"1) = E[I(' =k"1)Gk"1 | Fk"1] + E[I(' !k)G' | Fk"1]
= I(' =k"1)Gk"1 + I(' !k) E[E(G' | Fk) | Fk"1].
By the induction hypothesis, (a) holds for n = k. Since ' , MNk this
implies that
E(G' | Fk) ' SNk . (56)
From SNn = max(Gn, E(SN
n+1 | Fn)) for n = k " 1 we have
Gk"1 ' SNk"1, (57)
E(SNk | Fk"1) ' SN
k"1. (58)
IV-2/3-8
Using (56)–(58) in (??) we get
E(G' | Fk"1) ' I(' =k"1)SNk"1 + I(' !k) E(SN
k | Fk"1)
' I(' =k"1) SNk"1 + I(' !k) SN
k"1 = SNk"1. (59)
This shows that
SNn ! E(G' | Fn), ;' , M
Nn
holds for n = k " 1 as claimed.
(b)!SN
n = E(G'Nn| Fn)
": To prove (b) for n = k " 1 it is enough to
check that all inequalities in (??) and (59) remain equalities when
' = 'Nk"1. For this, note that
'Nk"1 = 'N
k on {'Nk"1 ! k};
Gk"1 = SNk"1 on {'N
k"1 = k " 1};E(SN
k | Fk"1) = SNk"1 on {'N
k"1 ! k}.
IV-2/3-9
Then we get
E(G'N
k"1| Fk"1
)= I('N
k"1 = k " 1)Gk"1
+ I('Nk"1!k) E
(E(G'N
k| Fk) | Fk"1
)
= I('Nk"1 = k " 1)Gk"1 + I('N
k"1!k) E(SNk | Fk"1)
= I('Nk"1 = k " 1)SN
k"1 + I('Nk"1!k)SN
k"1 = SNk"1.
Thus
SNn = E
!G'N
n| Fn
"
holds for n = k " 1. (We supposed by induction that (b) holds for
n = N, . . . , k.)
IV-2/3-10
(c)!'Nn is optimal in V N
n = supn'''N
E G'
":
Take expectation E in SNn ! E(G' | Fn), ' , Mn
n. Then
E SNn ! E G' for all ' , M
Nn
and by taking the supremum over all ' , MNn we see that
E SNn ! V N
n
D= sup
',MNn
E G'
E.
On the other hand, taking the expectation in SNn = E(G'N
n| Fn) we
get
E SNn = E G'N
n
which shows that
E SNn ' V N
n
D= sup
',MNn
E G'
E.
IV-2/3-11
So,
E SNn = V N
n
and since E SNn = E G'N
n, we see that
V Nn = E G'N
n
implying the claim (c): “The stopping time 'Nn is optimal”.
(d)!
if '/ is also optimal then 'Nn ' '/
":
If we suppose that '/ is also optimal then 'Nn ' '/. We claim that
the optimality of '/ implies that SN'/ = G'/ (P-a.s.). Indeed,
for all n ' k ' N SNk ! Gk, thus SN
'/ ! G'/.
If SN'/ 4= G'/ (P-a.s.) then
P(SN'/ > G'/) > 0.
IV-2/3-12
It thus follows that
E G'/ < E SN'/
($)' E SN
n(#)= V N
n ,
where
($) follows by the supermartingale property of
(SNk )n'k'N (see (e)) and the optional sampling
theorem, and
(#) was obtained in (c).
The strict inequality E G'/ < V Nn , however, contradicts the fact that
'/ is optimal.
Hence SN'/ = G'/ (P-a.s.) and the fact that 'N
n ' '/ (P-a.s.) follows
from the definition
'Nn = inf{n ' k ' N : SN
k = Gk}.
IV-2/3-13
(e)!the sequence (SN
k )n'k'N is the smallest supermartingale
which dominates (Gk)n'k'N
":
From
SNk = max{Gk, E(SN
k+1 | Fk)}, k = N " 1, . . . , n,
we see that (SNk )n'k'N is a supermartingale:
SNk ! E(SN
k+1 | Fk).
Also we have SNk ! Gk. It means that (SN
k )n'k'N is a supermartingale
which dominates (Gk)n'k'N .
Suppose that ( /Sk)n'k'N is another supermartingale which dominates
(Gk)n'k'N , then the claim that /Sk ! SNk (P-a.s.) can be verified by
induction over k = N, N " 1, . . . , l.
IV-2/3-14
Indeed, if k = N then the claim follows by SNn = GN for n = N .
Assuming that /Sk ! SNk for k = N, N " 1, . . . , l with l ! n + 1 it
follows that
SNl"1 = max(Gl"1, E(SN
l | Fl"1))
' max(Gl"1, E( /Sl | Fl"1)) ' /Sl"1 (P-a.s.)
using the supermartingale property of (Sk)n'k'N . So, (SNk )n'k'N is
the smallest supermartingale which dominates (Gk)n'k'N
(Snell’s envelop).
IV-2/3-15
(f)!
the stopped sequence (SNk<'N
n)n'k'N is a martingale
":
To verify the martingale property
E
(SN(k+1)<'N
n| Fk
)= SN
k<'Nn
with n ' k ' N " 1 given and fixed, note that
E(SN(k+1)<'N
n| Fk
)= E
(I('N
n ' k)SNk<'N
n| Fk
)
+ E
(I('N
n ! k + 1)SNk+1 | Fk
)
= I('Nn ' k)SN
k<'Nn
+ I('Nn ! k + 1) E(SN
k+1 | Fk)
= I('Nn ' k)SN
k<'Nn
+ I('Nn ! k + 1)SN
k = SNk<'N
n
where we used that
SNk = E(SN
k+1 | Fk) on { 'Nn ! k + 1 }
and { 'Nn ! k + 1 } , Fk since 'N
n is a stopping time.
IV-2/3-16
Summary
1) The optimal stopping problem
V N0 = sup
',MN0
E G'
is solved inductively by solving the problems
V Nn = sup
',MNn
E G' for n = N, N " 1, . . . ,0.
2) The optimal stopping rule 'Nn for V N
n satisfies
'Nn = 'N
k on {'Nn ! k}
for 0 ' n ' k ' N when 'Nk is the optimal stopping rule for V N
k . In
other words, this means that if it was not optimal to stop within
the time set {n, n + 1, . . . , k " 1} then the same optimality rule for
V Nn applies in the time set {k, k + 1, . . . , N}.
IV-2/3-17
3) In particular, when specialized to the problem V N0 , the following
general principle (of dynamic programming) is obtained:
if the stopping rule 'N0 is optimal for V N
0 and it was notoptimal to stop within the time set {0,1, . . . , n " 1}, thenstarting the observation at time n and being based on theinformation Fn, the same stopping rule is still optimal forthe problem V N
n .
#
"
$$$
$$
$$
$$
$$
$$
$$$
$$
$$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$
$$
$$
$
$$
$$
$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$
$$$
$$$
$$$
$$
$$
$$$
$$
$
$$
$$
$$
$
C
D
00000012222223000000122222230000001******+00000012222223$$
$$
$$4
n " 1 n0
D
IV-2/3-18
3. The method of ESSENTIAL SUPREMUM
The method of backward induction by its nature requires that the
horizon N be FINITE so that the case of infinite horizon remains
uncovered.
It turns out, however, that the random variables SNn defined by the
recurrent relations
SNn = GN, n = N,
SNn = max{Gn, E(SN
n+1 | Fn)}, n = N"1, . . . ,0,
admit a di!erent characterization which can be directly extended to
the case of infinite horizon N .
This characterization forms the base of the SECOND method that
will now be presented.
IV-2/3-19
Note that the relations
(a) SNn ! E(G' | Fn) ;' , M
Nn ;
(b) SNn = E(G'N
n| Fn)
from Theorem 1 suggest that the following identity should hold:
SNn = sup
',MNn
E(G' | Fn) .
(!) Di"culty: sup',MNn
E(G' | Fn) need not define a
measurable function.
To overcome this di&culty it turns out that the concept of
ESSENTIAL SUPREMUM
proves useful.
IV-2/3-20
Lemma (about Essential Supremum).
Let {Z$, $ , A} be a family of random variables defined on (!,F , P)
where the index set A can be arbitrary.
I. Then there exists a countable subset J of A such that the random
variable Z/ : ! ) R defined by
Z/ = sup$,J
Z$
satisfies the following two properties:
(a) P(Z$'Z/) = 1, ;$ , A;
(b) If Z : ! ) R is another random variable
satisfying P(Z$ ' Z/) = 1, ;$ , A, then
P(Z/ ' Z) = 1.
IV-2/3-21
II. Moreover, if the family {Z$, $ , A} is upwards directed in thesense that
for any $ and % in A there exists . in A
such that max(Z$, Z%) ' Z. (P-a.s.),
then the countable set J = {$n, n ! 1} can be chosen so that
Z/ = limn)%Z$n (P-a.s.)
where Z$1 ' Z$2 ' · · · (P-a.s.).
Proof. (1) Since x :) 2- arctan(x) is a strictly increasing function
from R to ["1,1], it is no restriction to assume that |Z$| ' 1.
(2) Let C denote the family of all countable subsets C of A. Choosean increasing sequence {Cn, n ! 1} in C such that
adef= sup
C,CE
Dsup$,C
Z$
E= sup
n!1E
Dsup$,Cn
Z$
E.
IV-2/3-22
Then Jdef=
L%n=1 Cn is a countable subset of A and we claim that
Z/ def= sup
$,JZ$
satisfies the properties (a) and (b).
(3) To verify these claims take $ , A arbitrarily.
(a): If $ , J then Z$ ' Z/ so that (a) holds. If $ /, J and we assume
that P(Z$ > Z/) > 0, then
a < E(Z/ 7 Z$) ' a
since a = E Z/ , ["1,1] (by the monotone convergence
theorem) and J 9 {$} belongs to C. As the strict inequality
is impossible, we see that P(Z$ ' Z/) = 1, ;$ , A as claimed.
(b): follows from Z/ = sup$,J Z$ and (a): P(Z$'Z/) = 1, ;$ , A,
since J is countable.IV-2/3-23
Finally, assume that the condition in II is satisfied. Then the initial
countable set
J = {$1, $2, . . .}
can be replaced by a new countable set J+ = {$+1, $+
2, . . .} if we
initially set $+1 = $1, and then inductively choose $+
n+1 ! $+n 7 $n+1
for n ! 1, where . ! $ 7 % corresponds to Z$, Z% and Z. such that
Z. ! Z$ 7 Z% (P-a.s.). The concluding claim Z/ = limn)% Z$n in II
is then obvious, and the proof of the lemma is complete.
With the concept of essential supremum we may now rewrite
SNn ! E(G' | Fn) ;' , M
Nn ; SN
n = E(G'Nn| Fn)
in Theorem 53 above as follows:
SNn = ess sup
n'''NE(G' | Fn) for all 0 ' n ' N .
IV-2/3-24
This ess sup identity provides an additional characterization of the
sequence of r.v.’s (SNn )0'n'N introduced initially by means of the
recurrent relations
SNn = GN, n = N,
SNn = max{Gn, E(SN
n+1 | Fn)}, n = N"1, . . . ,0.
Its advantage in comparison with these recurrent relations lies in
the fact that the identity
SNn = ess sup
n'''NE(G' | Fn)
can naturally be extended to the case of INFINITE horizon N . This
programme will now be described.
Consider (instead of V Nn = sup',MN
nE G')
Vn = sup',M%
n
E G' .
IV-2/3-25
To solve this problem we will consider the sequence of r.v.’s (Sn)n!0defined as follows:
Sn = ess sup'!n
E(G' | Fn)
as well as the following stopping time:
'n = inf{k ! n |Sk = Gk} for n ! 0,
where inf " = % by definition.
The first part (I) of the next theorem shows that (Sn)n!0 satisfiesthe same recurrent relations as (SN
n )0'n'N .
The second part (II) of the theorem shows that Sn and 'n solve theproblem in a stochastic sense.
The third part (III) shows that this leads to a solution of the initialproblem Vn = sup'!n E G' .
The fourth part (IV) provides a supermartingale characterization ofthe solution.
IV-2/3-26
Theorem 2 (Infinite horizon).
Consider the optimal stopping problems
Vn = sup'!n
E G' , ' , M%n , n ! 0
assuming that the condition E sup0'k<%
|Gk| < % holds.
I. The following recurrent relations hold:
Sn = max{Gn, E(Sn+1 | Fn)}, ;n ! 0.
II. Assume moreover if required below that
P('n<%) = 1.
Then for all n ! 0 we have:
Sn ! E(G' | Fn) ;' , Mn, Sn = E(G'n | Fn).
IV-2/3-27
III. Moreover, if n ! 0 is given and fixed, then we have:
The stopping time 'n = inf{k ! n : Sk = Gk} is
optimal in Vn = sup'!n E G' .
If '/ is an optimal stopping time for Vn = sup'!n E G'then 'n ' '/ (P-a.s.).
IV. The sequence (Sk)k!n is the smallest supermartingale
which dominates (Gk)k!n (Snell’s envelope).
The stopped sequence (Sk<'n)k!n is a martingale.
Finally, if the condition P('n < %) = 1 fails so that
P('n = %) > 0, then there is NO optimal stopping time
in Vn = sup'!n E G' .
IV-2/3-28
Proof. I. We need prove the recurrent relations
Sn = max{Gn, E(Sn+1 | Fk)}, n ! 0.
Let us first show that
Sn ' max{Gn, E(Sn+1 | Fk)}.
For this, take ' , Mn and set ' = ' 7 (n + 1).
Then ' , Mn+1, and since {' ! n + 1} , Fn we have
E(G' | Fn) = E[I(' = n)Gn | Fn] + E[I(' ! n + 1)G' | Fn]
= I(' = n)Gn + I(' ! n + 1) E(G' | Fn)
= I(' = n)Gn + I(' ! n + 1) E[E(G' | Fn+1) | Fn]
' I(' = n)Gn + I(' ! n + 1) E Sn+1 | Fn)
' max{Gn, E(Sn+1 | Fn)}.
IV-2/3-29
From this inequality it follows that
Sn = ess sup'!n
E(G' | Fn) ' max{Gn, E(Sn+1 | Fn)}
which is the desired inequality.
For the reverse inequality, let us first note that Sn ! Gn (P-a.s.)
by the definition of Sn, so that it is enough to show (and it is the
most di"cult part of the proof) that
Sn ! E(Sn+1 | Fn)
which is the supermartingale property of (Sn)n!0. To verify this
inequality, let us first show that the family {E(G' | Fn+1); ' , Mn+1}is upwards directed in the sense that
for any $ and % in A there exists . in A
such that Z$ 7 Z% ' Z..(#)
IV-2/3-30
For this, note that if !1 and !2 are from Mn+1 and we set !3 =
!1IA + !2IA where
A = {E(G!1 | Fn+1) ! E(G!2 | Fn+1)},
then !3 , Mn+1 and we have
E(G!3 | Fn+1) = E(G!1IA+ G!2IA | Fn+1)
= IA E(G!1 | Fn+1) + IA E(G!2 | Fn+1)
= E(G!1 | Fn+1) 7 E(G!2 | Fn+1)
implying (#) as claimed. Hence by Lemma there exists a sequence
{!k, k ! 1} in Mn+1 such that
ess sup'!n+1
E(G' | Fn+1) = limk)%
E(G!k | Fn+1)
where
E(G!1 | Fn+1) ' E(G!2 | Fn+1) ' · · · (P-a.s.).
IV-2/3-31
Since
Sn+1 = ess sup'!n+1
E(G' | Fn+1),
by the conditional monotone convergence theorem we get
E(Sn+1 | Fn) = E
(lim
k)%E(G!k | Fn+1) | Fn
)
= limk)%
E(E(G!k | Fn+1) | Fn
)
= limk)%
E(G!k | Fn) ' Sn.
So, Sn = max{Gn, E(Sn+1 | Fn)} and the proof if I is complete.
II. The inequality Sn ! E(G' | Fn), ;' , Mn, follows from the definition
Sn = ess sup'!n E(G' | Fn).
For the proof of the equality Sn = E(G'n | Fn) we use the fact stated
below in IV that the stopped sequence (Sk<'n)k!n is a martingale.
IV-2/3-32
Setting G/n = supk!n |Gk| we have
|Sk| ' ess sup'!k
E
!|G' | | Fk
"' E(G/
n | Fk) (/)
for all k ! n. Since G/n is integrable due to E supk!n |Gk| < %, it
follows from (/) that (Sk)k!n is uniformly integrable.
Thus the optional sampling theorem can be applied to the martingale
(Mk)k!n = (Sk<'n)k!n and we get
Mn = E(M'n | Fn). (//)
Since Mn = Sn and M'n = S'n we see that (//) is the same as Sn =
E(G'n | Fn).
III: “The stopping time 'n is optimal in Vn = sup'!n E G' .”
The proof uses II and is similar to the corresponding proof in Theorem
1 (N < %).
IV-2/3-33
IV. “The sequence (Sk)k!n is the smallest supermartingale which
dominates (Gk)k!n” (Snell’s envelop).
We proved in I that (Sk)k!n is a supermartingale. Moreover, from
the definition
Sn = ess sup'!n
E(G' | Fn)
it follows that Sk ! Gk, k ! n, which means that (Sk)k!n dominates
(Gk)k!n. Finally, if (Sk)k!n is another supermartingale which dominates
(Gk)k!n, then from Sn = E(G'n | Fn) (Part II) we find
Sk = E(G'k | Fk) ' E(S'k | Fk) ' Sk, ;k ! n.
(The last inequality follows by the optional sampling theorem being
applicable since S"k ' G"
k ' G/n (= supk!n |Gk|) with G/
n integrable.)
IV-2/3-34
The statement
“The stopped sequence (Sk<'n)k!n is a martingale”
is proved in exactly the same way as for case N < %.
Finally, note that the final claim
“If the condition P('n < %) = 1 fails so that P('n = %) >
0, then there is NO optimal stopping time in the problem
Vn = sup'!n E G'”
follows directly from III (“If 'n is optimal stopping tome then 'n ' '/(P-a.s.) for the problem Vn = sup'!n E G'”).
IV-2/3-35
Remark. From the definition
Sn = ess supn'''N
E(G' | Fn)
it follows that
N :) SNn and N :) 'N
n
are increasing. So,
S%n = lim
N)%SN
n and '%n = limN)%
'Nn
exist P-a.s. for each n ! 0.
IV-2/3-36
Note also that from
V Nn = sup
n'''NE G'
it follows that N :) V Nn is increasing, so that V %
n = limN)% V Nn
exists for each n ! 0.
From SNn = ess supn'''N E(G' | Fn) and Sn = ess sup'!n E(G' | Fn)
we see that
S%n ' Sn and '%n ' 'n. (/)
Similarly,
V %n ' Vn
!= sup
'!nE G'
". (//)
If condition E supn'k<% |Gk| < % does not hold then the inequalities
in (/) and (//) can be strict.
IV-2/3-37
Theorem 3 (From finite to infinite horizon).
If E sup0'k<% |Gk| < % then in S%n ' Sn, '%n ' 'n and V %
n ' Vn we
have equalities for all n ! 0.
Proof. From
SNn = max{Gn, E(SN
n+1 | Fn)}, n ! 0,
we get
S%n = max{Gn, E(S%
n+1 | Fn)}, n ! 0.
So, (S%n )n!0 is a supermartingale.
Since S%n ! Gn we see that
(S%n )" ' G"
n ' supn!0
G"n , n ! 0.
So, ((S%n )")n!0 is uniformly integrable.
IV-2/3-38
Then by the optional sampling theorem we get
S%n ! E(S%
' | Fn) for all ' , Mn. (/)
Moreover, since S%k ! Gk, k ! n, it follows that S%
' ! G' for all
' , Mn, and hence
E(S%' | Fn) ! E(G' | Fn) (//)
for all ' , Mn. From (/), (//) and
Sn = ess sup'!n
E(G' | Fn)
we see that S%n ! Sn.
Since the reverse inequality holds in general as shown above, this
establishes that S%n = Sn (P-a.s.) for all n ! 0. From this it also
follows that '%n = 'n (P-a.s.), n ! 0. Finally, the third identity
V %n = Vn follows by the monotone convergence theorem.
IV-2/3-39
B. Markovian approach.
We will present basic results of optimal stopping when
the time is discrete and the process is Markovian.
1. We consider a time-homogeneous Markov chain X = (Xn)n!0
• defined on a filtered probability space (!,F , (Fn)n!0, Px)
• taking values in a measurable space (E,B)
where for simplicity we will assume that
(a) E = Rd for some d ! 1
(b) B = B(Rd) is the Borel !-algebra on Rd.
IV-2/3-40
It is assumed that the chain X starts at x under Px for x , E.
It is also assumed that the mapping x :) Px(F) is measurable for
each F , F.
It follows that the mapping x :) Ex(Z) is measurable for each random
variable Z.
Finally, without loss of generality we will assume that (!,F) equals
the canonical space (EN0,BN0) so that the shift operator /n : ! ) !
is well defined by
/n(*)(k) = *(n+k) for * = (*(k))k!0 , ! and n, k ! 0.
(Recall that N0 stands for N 9 {0}.)
IV-2/3-41
Given a measurable function G : E ) R satisfying the following
condition (with G(XN) = 0 if N = %):
Ex
Dsup
0'n'N|G(Xn)|
E< %
for all x , E, we consider the optimal stopping problem
V N(x) = sup0'''N
Ex G(X')
where x , E and the supremum is taken over all stopping times '
of X. The latter means that ' is a stopping time w.r.t. the natural
filtration of X given by
FXn = !(Xk; 0 ' k ' n) for n ! 0.
IV-2/3-42
Since the same results remain valid if we take the supremum in
V N(x) = sup0'''N
Ex G(X') (/)
over stopping times ' w.r.t. (Fn)n!0, and this assumption makes
final conclusions more powerful (at least formally), we will assume
in the sequel that the supremum in (/) is taken over this larger class
of stopping times.
Note also that in (/) we admit that N can be +% as well.
In this case, however, we still assume that the supremum is taken
over stopping times ' , i.e. over Markov times ' satisfying 0 ' ' < %.
In this way any specification of G(X%) becomes irrelevant for the
problem (/).
IV-2/3-43
To solve
V N(x) = sup0'''N
Ex G(X') (/)
when N < %, we may note that by setting Gn = G(Xn) for n ! 0
the problem reduces to the problem
V Nn = sup
n'''NEx G' . (//)
Having identified (/) as (//), we can apply the method of back-
ward induction which leads to a sequence of r.v.’s (SNn )0'n'N and
a stopping time 'Nn = inf{n ' k ' N : SN
k = Gk}.
The key identity is
SNn = V N"n(Xn) for 0 ' n ' N , Px-a.s.; x , E (///)
Once (///) is known to hold, the results of the Theorem 1 (finite
horizon) from the Martingale theory translate immediately into the
present Markovian setting and get a more transparent form.
IV-2/3-44
To get formulation, let us define
CNn = {x , E : V N"n(x) > G(x) }
DNn = {x , E : V N"n(x) = G(x) }
for 0 ' n ' N . We also define stopping time
'D = inf {0 ' n ' N : Xn , DNn }.
and the transition operator T of X
TF(x) = Ex F(X1)
for x , E whenever F : E ) R is a measurable function so that
F(X1) is integrable w.r.t. Px for all x , E.
IV-2/3-45
Theorem 4 (Finite horizon: The time-homogeneous case)
Consider the optimal stopping problems
V n(x) = sup0'''n
Ex G(X') (/)
assuming that Ex sup0'k'N
|G(Xk)| < %. Then
I. Value functions V n satisfy the “Wald–Bellman equation”
V n(x) = max(G(x), TV n"1(x)) (x , E)
for n = 1, . . . , N where V 0 = G.
II. The stopping time 'D = inf {0 ' n ' N : Xn , DNn } is
optimal in (/) for n = N .
III. If '/ is an optimal stopping time in (/) then 'D ' '/ (Px-a.s.)
for every x , E.
IV-2/3-46
IV. The sequence (V N"n(Xn))0'n'N is the smallest
supermartingale which dominates (G(Xn))0'n'N under Px
for x , E given and fixed.
V. The stopped sequence (V N"n(Xn<'D))0'n'N is a
martingale under Px for every x , E.
Proof. To verify the equality SNn = V N"n(Xn) recall that
SNn = Ex(G(X'N
n) | Fn) (i)
for 0 ' n ' N . Since SN"nk + /n = SN
n+k we get that 'Nn satisfies
'Nn = inf{n ' k ' N : SN
k = G(Xk)} = n + 'N"n0 + /n (ii)
for 0 ' n ' N (/n*(k) = *(k + n)).
IV-2/3-47
Inserting (ii) into (i) and using the Markov property we obtain
SNn = Ex
(G(X
n+'N"n0 +/n
) | Fn
)= Ex
(G(X
'N"n0
) + /n | Fn
)
= EXn G(X'N"n0
)($)= V N"n(Xn)
(iii)
where ($) follows by (i): SNn = Ex(G(X'N
n) | Fn), which imply
Ex SN"n0 = Ex G(X
'N"n0
) = sup0'''N"n
Ex G(X') = V N"n(x) (iv)
for 0 ' n ' N and x , E.
Thus SNn = V N"n(Xn) holds as claimed.
IV-2/3-48
To verify the “Wald–Bellman equation”, note that the equality
SNn = max{Gn, E(SN
n+1 | Fn)},
using the Markov property, reads as follows:
V N"n(Xn) = max7G(Xn), Ex
(V N"n"1(Xn+1) | Fn
)8
= max7G(Xn), Ex
(V N"n"1(X1) + /n | Fn
)8
= max7G(Xn), EXn V N"n"1(X1)
8
= max7G(Xn), TV N"n"1(Xn)
8
(/)
for all 0 ' n ' N . Letting n = 0 and using that X0 = x under Px we
see that (/) yields V n(x) = max{G(x), TV n"1(x)}.
The remaining statements of the theorem follow directly from the
Martingale Theorem (1). The proof is complete.
IV-2/3-49
The “Wald–Bellman equation” can be written in a more compact
form as follows. Introduce the operator Q by setting
QF(x) = max(G(x), TF(x))
for x , E where F : E ) R is a measurable function for which
F(X1) , L1(Px) for x , E. Then the “Wald–Bellman equation” reads
as follows:
V n(x) = QnG(x)
for 1 ' n ' N where Qn denotes the n-th power of Q. These
recursive relations form a constructive method for finding V N when
Law(X1 |Px) is known for x , E.
IV-2/3-50
TIME-INHOMOGENEOUS MARKOV CHAINS X = (Xn)n!0
Put Zn = (n, Xn).
Z = (Zn)n!0 is a time-homogeneous Markov chain.
Optimal stopping problem:
(/) V N(n, x) = sup0'''N"n
En,x G(n+', Xn+') , 0 ' n ' N.
We assume
(//) En,x
Dsup
0'k'N"n|G(n + k, Xn+k)|
E< %, 0 ' n ' N.
IV-2/3-51
Theorem 5 (Finite horizon: The time-inhomogeneous case)
Consider the optimal stopping problem (/) upon assuming that the
condition (//) holds. Then:
I. The function V n satisfies the “Wald–Bellman equation”
V N(n, x) = max(G(n, x), TV N(n, x))
for n = N"1, . . . ,0 where
TV N(n, x) = En,x V N(n + 1, Xn+1), n = N " 1, . . . ,0,
and
TV N(N"1, x) = EN"1,x G(N, XN);
IV-2/3-52
II. The stopping time
'ND = inf{n ' k ' N : (n + k, Xn+k) , D}
with
D =7(n, x) , {0,1, . . . , N}.E : V (n, x) = G(n, x)
8
is optimal in the problem (/):
V N(n, x) = sup0'''N"n
En,x G(n+', Xn+');
III. If 'N/ is an optimal stopping time in (/) then 'N
D ' 'N/
(Pn,x-a.s.) for every (n, x) , {0,1, . . . , N}.E;
IV-2/3-53
IV. The value function V N is the smallest superharmonic
function which dominates the gain function G on
{0, . . . , N}.E,
TV N(n, x) ' V N(n, x), V N(n, x) ! G(n, x);
V. The stopped sequence!V N((n+k) < 'N
D , X(n+k)<'ND)"
0'k'N"n
is a martingale under Pn,x for every (n, x) , {0,1, . . . , N}.E;
The proof is carried out in exactly the same way as the proof of
Theorem 4.
IV-2/3-54
Optimal stopping for infinite horizon (N = %):
V (x) = sup'
Ex G(X')
Theorem 6
Assume Ex supn!0 |G(Xn)| < %, x , E.
I. The value function V satisfies the “Wald–Bellman equation”
V (x) = max(G(x), TV (x)), x , E.
II. Assume moreover when required below that Px('D < %) = 1
for all x , E, where
'D = inf{t ! 0 : Xt , D}
with D = {x , E : V (x) = G(x)}. Then the stopping time 'D
is optimal.
IV-2/3-55
III. If '/ is an optimal stopping time then 'D ' '/ (Px-a.s. for
every x , E).
IV. The value function V is the smallest superharmonic function
(Dynkin’s characterization) (TV ' V ) which dominates the
gain function G on E, or, equivalently, (V (Xn))n!0 is the
smallest supermartingale (under Px, x , E) which dominates
(G(Xn))n!0.
V. The stopped sequence (V (Xn<'D))n!0 is a martingale under
Px for every x , E.
VI. If the condition Px('D < %) = 1 fails so that Px('D = %) >
0 for some x , E, then there is no optimal stopping time in
the problem V (x) = sup' Ex G(X') for all x , E.
IV-2/3-56
Corollary (Iterative method). We have
V (x) = limn)%QnG(x)
(a constructive method for finding the value function V ).
Uniqueness in the Wald–Bellman equation
F(x) = max(G(x), TF(x))
Suppose E supn!0 F(Xn) < %.
Then F equals the value function V if and only if the following
“boundary condition at infinity ” holds:
lim supn)%
F(Xn) = limsupn)%
G(Xn) Px-a.s. ;x , E.
IV-2/3-57
2. Given $ , (0,1] and bounded g : E ) R and c : E ) R+, consider
the optimal stopping problem
V (x) = sup'
Ex
D$'g(X') "
'#
k=1
$k"1c(Xk"1)E.
Let IX = (IXn)n!0 denote the Markov chain X killed at rate $. It
means that
/TF(x) = $ TF(x).
Then
V (x) = sup'
Ex
Dg(IX') "
'#
k=1
c(IXk"1)E.
The “Wald–Bellman equation” takes the following form:
V (x) = max7g(x), $TV (x) " c(x)
8.
IV-2/3-58
LECTURES 4–5.
Theory of optimal stopping for continuous time
A. Martingale approach
Let (!,F , (Ft)t!0, P) be a stochastic basis (a filtered probability
space with right-continuous family (Ft)t!0 where each Ft contains
all P-null sets from F.
Let G = (Gt)t!0 be a gain process. (We interpret Gt as the gain if
the observation of G is stopped at time t.)
DEFINITION.
A random variable ' : ! ) [0,%] is called a Markov time
if {' ' t} , Ft for all t ! 0.
A Markov time is called a stopping time if ' < % P-a.s.
IV-4/5-1
We assume that G = (Gt)t!0 is right-continuous and left-continuous
over stopping times (if 'n 3 ' then G'n ) G' P-a.s.).
We also assume that
E!
sup0't'T
|Gt|"
< % (GT = 0 if T = %).
BASIC OPTIMAL STOPPING PROBLEM:
V Tt = sup
t'''TE G' .
We shall admit that T = %. In this case the supremum is still taken
over stopping times ' , i.e. over Markov times ' satisfying t ' ' < %.
IV-4/5-2
Two ways to tackle the problem V Tt = supt'''T E G' :
(1) Discrete time approximation
[0, T ] ") T(n) =7t(n)0 , t(n)
1 , . . . , t(n)n
83 T is a dense
subset of [0, T ]
G ") G(n) = (Gt(n)i
)
with applying previous discrete-time results and then
passing to the limit n ) %;
(2) Straightforward extension of the method of essential
supremum. This programme will now be addressed.
We denote for simplicity of the notation
Vt = V Tt (T < % or T = %).
IV-4/5-3
Consider the process S = (St)t!0 defined as follows:
St = ess sup'!t
E(G' |Ft).
The process S is the Snell’s envelope of G.
Introduce
't = inf {u ! t |Su = Gu} where inf " = % by definition.
We shall see below that
St ! max{Gt, E(Su | Ft)} for u ! t.
The reverse inequality is not true generally.
However,
St = max{Gt, E(S!<'t | Ft)}
for every stopping time ! ! t and 't given above.
IV-4/5-4
Theorem 1. Consider the optimal stopping problem
Vt = sup'!t
E G' , t ! 0,
upon assuming E supt!0 |Gt| < %. Assume moreover when required
below that
P('t < %) = 1, t ! 0.
(Note that this condition is automatically satisfied when the horizon
T is finite.) Then:
I. For all t ! 0 we have
St ! E(G' | Ft) for each ' , Mt
St = E(G't| Ft)
where Mt = {' : ' ' T} if T < %,
Mt = {' : ' < %} if T = %.
IV-4/5-5
II. The stopping time 't = inf{u ! t : Su = Gu} is
optimal (for the problem Vt = sup'!t E G').
III. If '/t is an optimal stopping time as well then
't ' '/t P-a.s.
IV. The process (Su)u!t is the smallest right-
continuous supermartingale which dominates
(Gs)s!t.
V. The stopped process (Su<'t)u!t is a right-
continuous martingale.
VI. If the condition P('t < %) = 1 fails so that
P('t = %) > 0, then there is no optimal stopping
time.
IV-4/5-6
Proof. 1+. Let us first prove that S = (St)t!0 defined by
St = ess sup'!t
E(G' | Ft)
is a supermartingale.
Show that the family {E(G' | Ft) : ' , Mt} is upwards directed in the
sense that if !1 and !2 are from Mt then there exists !3 , Mt such
that
E(G!1| Ft) 7 E(G!2| Ft) ' E(G!3| Ft).
Put !3 = !1IA + !2IA where
A = {E(G!1|Ft) ! E(G!2| Ft)}.
Then !3 , Mt and
E(G!3| Ft) = E(G!1IA + G!2IA | Ft) = IA E(G!1| Ft) + IA E(G!2| Ft)
= E(G!1| Ft) 7 E(G!2| Ft).
IV-4/5-7
Hence there exists a sequence {!k; k ! 1} in Mt such that
(/) ess sup',Mt
E(G' | Ft) = limk)%
E(G!k | Ft)
where
E(G!1| Ft) ' E(G!2| Ft) ' · · · P-a.s.
From (/) and the conditional monotone convergence theorem (using
E supt!0 |Gt| < %) we find that for 0 ' s < t
E(St | Fs) = E
!lim
k)%E(G!k| Ft) | Fs
"
= limk)%
E[E(G!k| Ft) | Fs]
= limk)%
E(G!k| Fs) ' Ss
!= ess sup
'!sE(G' | Fs)
".
Thus (St)t!0 is a supermartingale as claimed.
IV-4/5-8
Note that from E supt!0 |Gt| < % and
St = ess sup'!t
E(G' | Ft),
ess sup'!t
E(G' | Ft) = limk)%
E(G!k| Ft)
it follows that
E St = sup'!t
E G' .
2+. Let us next show that the supermartingale S admits a right-
continuous modification /S = ( /St)t!0.
From the general martingale theory it follows that it su&ces to
check that
t% E St is right-continuous on R+.
IV-4/5-9
By the supermartingale property of S
E St ! · · · ! E St2 ! E St1, tn 3 t.
So, L := limn)% E Stn exists and
E St ! L.
To prove the reverse inequality, fix # > 0 and by means of E St =
sup'!t E G' choose ! , Mt such that
E G! ! E St " #.
Fix " > 0 and note that there is no restriction to assume that
tn , [t, t + "] for all n ! 1. Define
!n =
@B
C! if ! > tn,
t + ! if ! ' tn.
IV-4/5-10
Then for all n ! 1 we have
(/) E G!n = E G!I(! > tn) + E Gt+"I(! ' tn) ' E Stn
since !n , Mtn and E St = sup'!t E G' . Letting n ) % in (/) and
assuming that E sup0't'T |Gt| < % we get
E G!I(! > t) + E Gt+"I(! = t) ' L (= limn
E Stn).
Letting now " * 0 and using that G is right-continuous we obtain
E G!I(! > t) + E GtI(! = t) = E G! ' L.
From here and E G! ! E St " # we see that L ! E St " # for all # > 0.
Hence L ! E St and thus
limn)%E Stn = L = E St, tn 3 t,
showing that S admits a right-continuous modification /S = ( /St)t!0
which we also denote by S throughout.
IV-4/5-11
Let us prove property IV:
The process (Su)u!t is the smallest right-continuous
supermartingale which dominates (Gs)s!t.
For this, let GS = ( GSu)u!t be another right-continuous supermartingale
which dominates G = (Gu)u!t. Then by the optional sampling theorem
(using E supt!0 |Gt| < %) we have
GSu ! E( GS' | Fu) ! E(G' | Fu)
for all ' , Mu when u ! t. Hence by the definition Su = ess sup'!u
E(G' | Fu)
we find that Su ' GSu (P-a.s.) for all u ! t. By the right-continuity of
S and GS this further implies that
P(Su ' GSu for all u ! t) = 1
as claimed.
IV-4/5-12
Property I: for all t ! 0
(/) St ! E(G' | Ft) for each ' , Mt,
(//) St = E(G't| Ft).
The inequality (/) follows from the definition St = ess sup'!t E(G' | Ft).
The proof of (//) is the most di&cult part of the proof of the
Theorem.
The sketch of the proof is as follows.
IV-4/5-13
Assume that Gt ! 0 for all t ! 0.
($) Introduce, for ( , (0,1), the stopping time
'(t = inf{s ! t : (Ss ' Gs}
(Then (S'(t' G'(
t, '(
t+ = 't.)
(%) We show that
St = E(S'(t| Ft) for all ( , (0,1).
So St ' (1/() E(G'(t| Ft) and letting ( 3 1 we get
St ' E(G'1t| Ft)
where '1t = lim(31 '(
t ('(t 3 when ( 3).
(.) Verify that '1t = 't. Then St ' E(G't| Ft) and evidently
St ! E(G't| Ft). Thus St = E(G't| Ft).
IV-4/5-14
For the proof of property V:
The stopped process (Su<'t)u!t is a right-
continuous martingale
it is enough to prove that
E S!<'t = E St
for all bounded stopping times ! ! t.
The optional sampling theorem implies
E S!<'t ' E St. (60)
On the other hand, from St = E(G't | Ft) and S't = G't we see that
E St = E G't = E S't ' E S!<'t.
Thus, E S!<'t = E St and (Su<'t)u!t is a martingale.
IV-4/5-15
B. Markovian approach
Let X = (Xt)t!0 be a strong Markov process defined on a filtered
probability space
(!,F , (Ft)t!0, Px)
where x , E (= Rd), Px(X0 = x) = 1,
x ) Px(A) is measurable for each A , F.
Without loss of generality we will assume that
(!,F) = (E[0,%),B[0,%)) (canonical space)
Shift operator /t = /t(*): ! ) ! is well defined by
/t(*)(s) = *(t + s) for * = (*(s))s!0 , ! and t, s ! 0.
IV-4/5-16
We consider the optimal stopping problem
V (x) = sup0'''T
Ex G(X')
G(XT ) = 0 if T < %; Ex sup0't'T
|G(Xt)| < %.
Here ' = '(*) is a stopping time w.r.t.
(Ft)t!0 (FXt 8 Ft, FX
t = !(Xs; 0 ' s ' t)).
G is called the gain function,
V is called the value function.
IV-4/5-17
CASE T = %:V (x) = sup
'Ex G(X')
Px(X0 = x) = 1
Introduce
the continuation set C = {x , E : V (x) > G(x)} and
the stopping set D = {x , E : V (x) = G(x)}
NOTICE! If
V is lsc (lower semicontinuous)
$
%
&
G is usc (upper semicontinuous)
%
&
&
then
C is open and D is closed
IV-4/5-18
The first entry time
'D = inf{t ! 0 : Xt , D}
for closed D is a stopping time since both X and (Ft)t!0 are right-
continuous.
DEFINITION. A measurable function F = F(x) is said to be
superharmonic (for X) if
Ex F(X!) ' F(x)
for all stopping times ! and all x , E. (It is assumed that F(X!) ,L1(Px) for all x , E whenever ! is a stopping time.)
We have:
F is superharmonic i!(F(Xt))t!0 is a supermartingale
under Px for every x , E.
IV-4/5-19
The following theorem presents
NECESSARY CONDITIONS
for the existence of an optimal stopping time.
Theorem. Let us assume that there exists an optimal stopping time
'/ in the problem
V (x) = sup'
Ex G(X')
i.e. V (x) = Ex F(X'/). Then
(I) The value function V is the smallest superharmonic
function (Dynkin’s characterization) which dominates
the gain function G on E.
IV-4/5-20
Let us in addition to “V (x) = Ex F(X'/)” assume that
V is lsc and G is usc.
Then
(II) The stopping time 'D = inf{t ! 0 : Xt , D} satisfies
'D ' '/ (Px-a.s., x , E)
and is optimal;
(III) The stopped process (V (Xt<'D))t!0 is a right-continuous
martingale under Px for every x , E.
IV-4/5-21
Now we formulate
SUFFICIENT CONDITIONS
for the existence of an optimal stopping time.
Theorem. Consider the optimal stopping problem
V (x) = sup'
Ex G(X')
upon assuming that the condition
Ex supt!0
|G(Xt)| < %, x , E,
is satisfied.
IV-4/5-22
Let us assume that there exists the smallest superharmonic functionGV which dominates the gain function G on E.
Let us in addition assume that
GV is lsc and G is usc.
Set D = {x , E : GV (x) = G(x)} and let 'D = inf{t : Xt , D}.
We then have:
(a) If Px('D < %) = 1 for all x , E, then GV = V and 'D is
optimal in V (x) = sup' Ex G(X');
(b) If Px('D < %) < 1 for some x , E, then there is no
optimal stopping time in V (x) = sup' Ex G(X').
IV-4/5-23
Corollary (The existence of an optimal stopping time).
Infinite horizon (T = %). Suppose that V is lsc and G is usc. If
Px('D < %) = 1 for all x , E, then 'D is optimal. If Px('D < %) < 1
for some x , E, then there is no optimal stopping time.
Finite horizon (T < %). Suppose that V is lsc and G is usc. Then
'D is optimal.
Proof for T = %. (The case T < % can be proved in exactly the
same way as the case T = % if the process (Xt) is replaced by the
process (t, Xt).)
The key is to show that V is SUPERHARMONIC.
IV-4/5-24
If so, then evidently V is the smallest superharmonic function
which dominates G on E. Then the claims of the corollary follow
directly from the Theorem (on su&cient conditions) above.
For this, note that V is measurable (since it is lsc) and thus so is
the mapping
(/) V (X!) = sup'
EX! G(X')
for any stopping time ! which is given and fixed.
On the other hand, by the strong Markov property we have
(//) EX! G(X') = Ex [G(X!+'+/!) | F!]
for every stopping time ' and x , E. From (/) and (//) we see that
V (x!) = ess sup'
Ex [G(X!+'+/!) | F!]
under Px where x , E is given and fixed.
IV-4/5-25
We can show that the family7
E[X!+'+/! | F!] : ' is a stopping time8
is upwards directed: if &1 = !+'1 +/! and &2 = !+'2+/! then there
is & = ! + ' + /! such that
E[G(X&) | F!] = E[G(X&1) | F!] 7 E[G(X&2) | F!].
From here we can conclude that there exists a sequence of stopping
times {'n;n ! 1} such that
V (X!) = limn
Ex [G(X!+'n+/!) | Fn]
where the sequence {Ex [G(X!+'n+/!) | Fn]} is increasing Px-a.s.
IV-4/5-26
By the monotone convergence theorem using E supt!0 |Gt| < % we
can conclude
Ex V (X!) = limn
Ex G(X!+'n+/!) ' V (x)
for all stopping times ! and all x , E. This proves that V is
superharmonic.
REMARK 1. If the function
x :) Ex G(X')
is continuous (or lsc) for every stopping time ' , then x :) V (x) is lsc
and the results of the Corollary are applicable. This yields a powerful
existence result by simple means.
IV-4/5-27
REMARK 2. The above results have shown that the optimal stopping
problem
V (x) = sup'
Ex G(X')
is equivalent to the problem of finding the smallest superharmonic
function GV which dominates G on E. Once GV is found it follows
that V = GV and 'D = inf{t : G(Xt) = GV (Xt)} is optimal.
There are two traditional ways for finding GV :
(i) Iterative procedure (constructive but non-explicit)
(ii) Free-boundary problem (explicit or non-explicit).
IV-4/5-28
For (i), e.g., it is known that if G is lsc and
Ex inft!0
G(Xt) > "% for all x , E,
then GV can be computed as follows:
GV (x) = limn)% lim
N)%QN
n G(x)
where
QnG(x) := G(x) 7 Ex G(X1/2n)
and QNn is the N-th power of Qn.
IV-4/5-29
The basic idea (ii) is that
GV and C (or D)
should solve the free-boundary problem:
(/) LXGV ' 0
(//) GV ! G ( GV > G on C & GV = G on D)
where LX is the characteristic (infinitesimal) operator of X.
Assuming that G is smooth in a neighborhood of 0C the following
“rule of thumb” is valid.
IV-4/5-30
If X after starting at 0C enters immediately into int(D) (e.g. when X
is a di!usion process and 0C is su&ciently nice) then the condition
LXGV ' 0 under (//) splits into the two conditions:
LXGV = 0 in C
0 GV
0x
FFFF0C
=0G
0x
FFFF0C
(smooth fit).
On the other hand, if X after starting at 0C does not enter immediately
into int(D) (e.g. when X has jumps and no di!usion component
while 0C may still be su&ciently nice) then the condition LXGV ' 0
(i.e. (/)) under (//) splits into the two conditions:
LXGV = 0 in C
GVFFF0C
= GFFF0C
(continuous fit).
IV-4/5-31
Proof of the Theorem on NECESSARY conditions
Basic lines
(I) The value function V is the smallest superharmonic
function which dominated the gain function G on E.
We have by the strong Markov property:
Ex V (X!) = Ex EX! G(X'/) = Ex Ex[G(X'/) + /! | F!]
= Ex G(X!+'/+/!) ' sup'
Ex G(X') = V (x)
for each stopping time ! and all x , E.
Thus V is superharmonic.
IV-4/5-32
Let F be a superharmonic function which dominates G on E. Then
Ex G(X') ' Ex F(X') ' F(x)
for each stopping time ' and all x , E. Taking the supremum over all
' we find that V (x) ' F(x) for all x , E. Since V is superharmonic
itself, this proves that V is the smallest superharmonic function
which dominated G.
(II) Let us show that the stopping time
'D = inf{t : V (Xt) = G(Xt)}
is optimal (if V is lsc and G is usc).
We assume that there exists an optimal stopping time '/:
V (x) = Ex G(X'/), x , E.
IV-4/5-33
We claim that V (X'/) = G(X'/) Px-a.s. for all x , E.
Indeed, if Px{V (X'/) > G(X'/)} > 0 for some x , E, then
Ex G(X'/) < Ex V (X'/) ' V (x)
since V is superharmonic, leading to a contradiction with the fact
that '/ is optimal. From the identity just verified it follows that
'D ' '/ Px-a.s. for all x , E.
IV-4/5-34
By (I) the value function V is the superharmonic (Ex V (X!) ' V (X)
for all stopping time ! and x , E). Setting ! 1 s and using the
Markov property we get for all t, s ! 0 and all x , E
V (Xt) ! EXt V (Xs) = Ex [V (Xt+s) | Ft].
This shows that
The process (V (Xt))t!0 is a supermartingale
under Px for each x , E.
Suppose for the moment that V is continuous. Then obviously it
follows that (V (Xt))t!0 is right-continuous. Thus, by the optional
sampling theorem (using E supt!0 |G(Xt)| < %), we see that
Ex V (X') ' Ex V (X!) for ! ' ' .
IV-4/5-35
In particular, since 'D ' '/ we get
V (x) = Ex G(X'/) = Ex V (X'/)
' Ex V (X'D) = Ex G(X'D) ' V (x),
where we used that
V (X'D) = G(X'D)
Now it is easy to show that 'D is optimal if V is continuous.
IV-4/5-36
If V is only lsc, then again (see the lemma below) the process
(V (Xt))t!0 is right-continuous (Px-a.s. for each x , E), and the
proof can be completed as above.
This shows that 'D is optimal if V is lsc as claimed.
Lemma. If a superharmonic function F : E ) R is lsc, then
the supermartingale (F(Xt))t!0 is right-continuous
(Px-a.s. for each x , E).
We omit the proof.
IV-4/5-37
(III) The stopped process (V (Xt<'D))t!0 is a right-continuous
martingale under Px for every x , E.
PROOF. By the strong Markov property we have
Ex [V (Xt<'D) | Fs<'D] = Ex
(EXt<'D
G(X'D) | Fs<'D
)
= Ex
!Ex [G(X'D) + /t<'D | Ft<'D] | Fs<'D
"
= Ex
!Ex [G(X'D) | Ft<'D] | Fs<'D
"= Ex [G(X'D) | Fs<'D]
= EXs<'DG(X'D) = V (Xs<'D)
for all 0 ' s ' t and all x , E proving the martingale property. The
right-continuity of!V (Xt<'D)
"
t!0follows from the right-continuity
of (V (Xt))t!0 that we proved above.
The proof of the theorem on necessary conditions is complete.
IV-4/5-38
REMARK. The result and proof of the Theorem extend in exactly
the same form (by slightly changing the notation only) to the finite
horizon problem
VT(X) = sup0'''T
Ex G(X').
Now we formulate the theorem which provides
su"cient condition
for the existence of an optimal stopping time.
IV-4/5-39
THEOREM. Consider the optimal stopping problem
V (x) = sup'
Ex G(X')
upon assuming that Ex supt!0 |G(Xt)| < %, x , E. Let us assume
that
(a) there exists the smallest superharmonic function GV which
dominates the gain function G on E;
(b) GV is lsc and G is usc.
Set D = {x , E : GV (x) = G(x)} and 'D = inf{t : Xt , D}.
We then have:
(I) If Px('D < %) = 1 for all x , E, then GV = V and 'D is
optimal;(II) If Px('D < %) < 1 for some x , E, then there is no
optimal stopping time.
IV-4/5-40
SKETCH OF THE PROOF.
(I) Since GV is superharmonic majorant for G, we have
Ex G(X') ' Ex GV (X') ' V (x)
for all stopping times ' and all x , E. So
G(x) ' V (x) = sup'
Ex G(X') ' GV (x)
for all x , E.
Next step (di"cult!): assuming that Px('D < %) = 1 for all x , E,
we prove the inequality
GV (x) ' V (x)
and optimality of time 'D.
IV-4/5-41
(II) If Px('D < %) < 1 for some x , E then there is no optimal
stopping time.
Indeed, by “necessary-condition theorem” if there exists optimal
optimal '/ then 'D ' '/.
But 'D takes value % with positive probability for some x , E.
So, for this state x we have Px('/ = %) > 0 and '/ cannot be
optimal (in the class M = {' : ' < %}).
IV-4/5-42