lecture 6
TRANSCRIPT
Special Models
Cusp Estimation
Let the observed process be
dXt = a |Xt − ϑ|κ dt + h (Xt) dt + dWt, X0, 0 ≤ t ≤ T,
where κ ∈ (0, 1/2).
The MLE ϑT and BE (for quadratic loss function) ϑT are defined bythe usual relations
L(ϑT , XT
)= sup
θ∈ΘL
(θ,XT
),
ϑT =∫
Θ
θ p(θ|XT
)dθ, p
(ϑ|XT
)=
p(θ) L(θ, XT
)∫Θ
p(v) L (v, XT ) dv.
1
Introduce the process
Z (u) = exp{
WH (u)− 12|u|2H
}, u ∈ IR,
where WH (·) is fBm, H = κ + 1/2 and random variables
Z (u) = supu∈IR
Z (u) , u =
∫IR
u Z(u) du
∫IR
Z(v) dv.
and the constant
Γ2ϑ =
2a2
G (ϑ)Γ(1 + κ
)Γ(
12 − κ
)
22κ√
π(2κ + 1)[1− cos(πκ)
],
2
Put γϑ = Γ1/Hϑ . The lower bound for all estimators ϑT
limδ→0
limT→∞
sup|ϑ−ϑ0|<δ
T 1/HEϑ
(ϑT − ϑ
)2 ≥ Eu2
γ2ϑ0
,
The MLE and BE are consistent, have the following limits indistribution
Lϑ
{T 1/2H
(ϑT − ϑ
)}=⇒ Lϑ
{u
γϑ
},
Lϑ
{T 1/2H
(ϑT − ϑ
)}=⇒ Lϑ
{u
γϑ
},
and we have the convergence of moments too. Moreover, the BE areasymptotically efficient.
3
Introduce the normalized LR
ZT (u) =L
(ϑ + T−1/2Hu,XT
)
L (ϑ,XT ),
where u ∈ UT =(T 1/2H (α− ϑ) , T 1/2H (β − ϑ)
)and show that
• (ZT (u1) , . . . , ZT (uk)) ⇒ (Zϑ (u1) , . . . , Zϑ (uk)),
• Eϑ
∣∣∣ZT (u2)1/2 − ZT (u1)
1/2∣∣∣2
≤ C |u2 − u1|2H ,
• Pϑ
{ZT (u) > e−κ|u|2H
}≤ CN
|u|N
Here Zϑ (u) = Z (γ (ϑ)u).
4
The limit distribution of the MLE we obtained by the “usual” way.
Pϑ0
{T 1/2H
(ϑT − ϑ0
)< x
}=
= P
{sup
T 1/2H(θ−ϑ0)<x
L(ϑ,XT
)> sup
T 1/2H(θ−ϑ0)≥x
L(ϑ,XT
)}
= P
{sup
T 1/2H(θ−ϑ0)<x
L(ϑ, XT
)
L (ϑ0, XT )> sup
T 1/2H(θ−ϑ0)≥x
L(ϑ,XT
)
L (ϑ0, XT )
}
= P{
supu<x
ZT (u) > supu≥x
ZT (u)}→ P
{supu<x
Zϑ (u) > supu≥x
Zϑ (u)}
= P(
u
γ (ϑ0)< x
), i.e. T 1/2H
(ϑT − ϑ0
)=⇒ u
γ (ϑ0).
where we put ϑ = ϑ0 + T−1/2Hu.
5
ZT (u) ⇒ Zϑ (u)? Let us put δt (u) = S(ϑ + T−1/2Hu, Xt
)
−S (ϑ, Xt) . Then
ln ZT (u) =∫ T
0
δt (u) dWt − 12
∫ T
0
δt (u)2 dt.
For the ordinary integral (ϕT = T−1/2H)
JT =∫ T
0
δt (u)2 dt = a2
∫ T
0
(|Xt − ϑ− ϕT u|κ − |Xt − ϑ|κ)2 dt+o (1) .
Further∫ T
0
(|Xt − ϑ− ϕT u|κ − |Xt − ϑ|κ)2 dt
= T
∫ ∞
−∞(|x− ϑ− ϕT u|κ − |x− ϑ|κ)2 f◦T (x) dx
= T
∫ ∞
−∞(|x− ϑ− ϕT u|κ − |x− ϑ|κ)2 f (ϑ, x) dx + o (1)
6
We change the variable x = ϑ + sϕT u
a2T
∫ ∞
−∞(|x− ϑ− ϕT u|κ − |x− ϑ|κ)2 f (ϑ, x) dx
= |u|2κ+1a2ϕ2κ+1
T T f (ϑ, ϑ)∫ ∞
−∞(|s− 1|κ − |s|κ)2 ds + o (1)
−→ Γ2ϑ |u|2κ+1
, i.e. JT → Γ2ϑ |u|2H
,
because ϕ2κ+1T T = 1. By the CLT
∫ T
0
δt (u) dWt =⇒ N(0, Γ2
ϑ |u|2H)∼ WH (γ (ϑ) u) .
7
Delay Estimation
The observed process is
dXt = −γ Xt−ϑ dt + dWt, X0 (s) , −τ ≤ s ≤ 0,
where γ > 0, 0 < ϑ < π/2γ. Note that
Xt−ϑ = X0 + γ
∫ t−ϑ
0
Xs−ϑ ds + Wt−ϑ
Therefore the trend is as smooth w.r.t. ϑ as Wiener process w.r.t.time (nondiferentiable). The Gaussian process Xt has ergodicproperties.
8
Introduce the stochastic process
Z (u) = exp{
W (u)− |u|2
}, u ∈ IR
and the random variables
u = arg supu∈IR
Z (u) , u =
∫IR
u Z (u) du∫IR
Z (u) du.
Note that
Eu2 = 26, Eu2 = σ20 , σ2
0 = 16ζ (3) ' 19.3
9
The lower bound for all estimators ϑT is
limδ→0
limT→∞
sup|ϑ−ϑ0|<δ
T 2Eϑ
(ϑT − ϑ
)2 ≥ Eu2
γ2,
The MLE and BE are consistent, have the following limits indistribution
T(ϑT − ϑ
)=⇒ u
γ, T
(ϑT − ϑ
)=⇒ u
γ,
Moreover, the BE are asymptotically efficient. Introduce
ZT (u) =L
(ϑ + T−1u,XT
)
L (ϑ,XT )=⇒ Z (γu) ?
10
For the MLE we have once more :
Pϑ0
{T
(ϑT − ϑ0
)< x
}=
= P
{sup
T (θ−ϑ0)<x
L(ϑ, XT
)> sup
T (θ−ϑ0)≥x
L(ϑ,XT
)}
= P
{sup
T (θ−ϑ0)<x
L(ϑ,XT
)
L (ϑ0, XT )> sup
T (θ−ϑ0)≥x
L(ϑ,XT
)
L (ϑ0, XT )
}
= P{
supu<x
ZT (u) > supu≥x
ZT (u)}→ P
{supu<x
Z (γu) > supu≥x
Z (γu)}
= P(
u
γ< x
), i.e. T
(ϑT − ϑ0
)=⇒ u
γ.
where we put ϑ = ϑ0 + T−1u.
11
Why ZT (u) ⇒ Z (γu)? Put δt (u) = Xt−ϑ−u/T −Xt−ϑ. Then
δt(u) = −γ
∫ t−ϑ− uT
t−ϑ
Xs−ϑ ds +(Wt−ϑ− u
T−Wt−ϑ
)
=γ u
TXt−2ϑ (1 + o(1)) +
1√T
Wt(u).
Further ∫ T
0
δt (u)2 dt ∼ 1T
∫ T
0
Wt (u)2 dt −→ |u| .
Hence, by CLT∫ T
0
(Xt−ϑ−u/T −Xt−ϑ
)dWt =⇒ N (0, |u|) ∼ W (u)
12
Change-point estimation
Suppose that the observed diffusion process
dXt = S (ϑ,Xt) dt + σ (Xt) dWt, X0, 0 ≤ t ≤ T,
where the trend coefficient S (θ, x) is discontinuous function along thetwo curves
{x
(1)∗ (θ) , θ ∈ [α, β]
},
{x
(2)∗ (θ) , θ ∈ [α, β]
}.
Example. Let ϑ ∈ (α, β), α > 1
dXt = −Xt sgn(X2
t − 2 ϑXt + 1)
dt + σ dWt.
13
Γ2ϑ =
2∑
i=1
∣∣∣x(i)∗ (ϑ)
∣∣∣S+ (ϑ)− S− (ϑ)
σ(x
(i)∗ (ϑ)
)
2
fϑ,
where S± (ϑ) = S(ϑ, x
(i)∗ (ϑ)±
), fϑ = f
(ϑ, x
(i)∗ (ϑ)
),
x(i)∗ (ϑ) =
dx(i)∗
dϑ(ϑ) , ϑ ∈ [α, β]
and introduce the process
Z (u) = exp{
W (u)− |u|2
}, u ∈ IR,
where W (u) , u ∈ IR is two-sided Wiener process. The r.v.’s u and u
are defined with the help of Z (·).
14
Low bound
limδ→0
limT→∞
sup|ϑ−ϑ0|<δ
T 2Eϑ
(ϑT − ϑ
)2 ≥ Eu2
Γ2ϑ0
,
Theorem 1 The MLE ϑT and the BE ϑT are uniformly on
compacts K ⊂ Θ consistent, their limit distributions are
T(ϑT − ϑ
)=⇒ u
Γϑ, T
(ϑT − ϑ
)=⇒ u
Γϑ,
the moments converge and the estimator ϑT is asymptotically
efficient.
The normalized LR is
ZT (u) =L
(ϑ + T−1u,XT
)
L (ϑ,XT )=⇒ Z (Γϑu) ?
15
As usual to show that T(ϑT − ϑ0
)=⇒ u Γ−1
ϑ we write
Pϑ0
{T
(ϑT − ϑ0
)< x
}= P
{supu<x
ZT (u) > supu≥x
ZT (u)}
−→ P{
supu<x
Z (Γϑu) > supu≥x
Z (Γϑu)}
= P(
u
Γϑ< x
).
Examples.
dXt = − sgn (Xt − ϑ) dt + dWt, 0 ≤ t ≤ T,
dXt = −Xt
(a + b 1I{ϑ<Xt<c+ϑ}
)dt + dWt,
dXt = −Xt
(a + b 1I{ϑ1<Xt<ϑ2}
)dt + dWt.
16
Simple switching with f (ϑ, x) = e−2|x−ϑ|
dXt = − sgn (Xt − ϑ) dt + dWt
ln ZT (u) = −∫ T
0
1I{ϑ≤Xt≤ϑ+u/T} dWt − 12
∫ T
0
1I{ϑ≤Xt≤ϑ+u/T} dt
We have (u > 0)∫ T
0
1I{ϑ≤Xt≤ϑ+u/T} dt = T
∫ ∞
−∞1I{ϑ≤x≤ϑ+u/T}f◦T (x) dx
= T
∫ ϑ+u/T
ϑ
f◦T (x) dx → u f (ϑ, ϑ) = u
and ∫ T
0
1I{ϑ≤Xt≤ϑ+u/T} dWt =⇒ N (0, |u|) ∼ W (u)
17
Nonparametric estimation
Diffusion process
dXt = S(Xt) dt + σ(Xt) dWt, X0, t ≥ 0.
The functions S(·) and σ(·) > 0 are continuous and S (·) is unknown.
Conditions:∫ x
0
exp{−2
∫ y
0
S(v)σ2(v)
dv
}dy → ±∞, x → ±∞
and
G (S) =∫ ∞
−∞
1σ(x)2
exp{
2∫ x
0
S(v)σ2(v)
dv
}dx < ∞
The process Xt is ergodic.
18
The invariant density function
fS(x) =1
G(S)σ(x)2exp
{2
∫ x
0
S(v)σ2(v)
dv
}, FS (x) =
∫ x
−∞fS(y) dy.
Main Problems:
• Distribution function estimation FS (x)
• Invariant density estimation fS (x)
• Trend coefficient estimation S (x)
Fix some S∗(·) and introduce the set
Vδ = {S(·) : supx∈IR
|S(x)− S∗(x)| ≤ δ}
19
Distribution function estimation.Empirical distribution function is consistent
FT (x) =1T
∫ T
0
1I{Xt<x} dt −→ FS (x)
and asymptotically normal√
T(F
T(x)− F (x)
)=⇒ N
(0, IF (S, x)−1
)
where the Fisher information IF (S, x) = dF (S, x)−2 ,
dF (S, x)2 = 4 ES
(F (ξ ∧ x)− F (ξ)F (x)
σ (ξ) f (ξ)
)2
< ∞
20
It follows from the representation
√T
(F
T(x)− F (x)
)=
2√T
∫ XT
X0
F (v ∧ x)− F (v)F (x)σ (v)2 f (v)
dv
− 2√T
∫ T
0
F (Xt ∧ x)− F (Xt)F (x)σ (Xt) f (Xt)
dWt.
Is it possible to estimate better? Lower bound:
limδ→0
limT→∞
supS(·)∈Vδ
T ES
(FT (x)− F
S(x)
)2
≥ IF (S∗, x)−1.
21
The first step is
supS(·)∈Vδ
T ES
(FT (x)− F
S(x)
)2
≥ sup|ϑ−ϑ0|<δ
Eϑ
(FT (x)− F
ϑ(x)
)2
where the parametric family is
dXt = [S∗(Xt) + (ϑ− ϑ0)ψ(Xt) σ(Xt)2] dt + σ(Xt) dWt,
The DF can be expanded by ϑ− ϑ∗
Fϑ(x) = F (x) + 2 (ϑ− ϑ0) E
{[1I{ξ<x} − F (x)]
∫ ξ
0
ψ (y) dy
}+ o(1),
Introduce the class
K =
{ψ(·) : E
{[1I{ξ<x} − F (x)]
∫ ξ
0
ψ (y) dy
}=
12
}.
Then with ϑ0 = F (x) obtain Fϑ(x) = ϑ + o (1)
22
sup|ϑ−ϑ0|<δ
TEϑ
(FT (x)− F
ϑ(x)
)2
= sup|ϑ−ϑ0|<δ
TEϑ
(ϑT − ϑ
)2 ≥ I−1ψ
where Iψ = Eψ (ξ)2 σ (ξ)2. We have to find the least favorable Iψ∗ :
infψ(·)∈K
Iψ = Iψ∗ .
We obtain the inequality
Iψ ≥{
4E(
[1− F (ξ ∨ x)]F (ξ ∧ x)σ(ξ) f(ξ)
)2}−1
= IF (S∗, x)
Hence
Iψ∗ =
{4E
([1− F (ξ ∨ x)]F (ξ ∧ x)
σ(ξ) f(ξ)
)2}−1
= IF (S∗, x)
and EDF is asimptotically efficient.
23