structural break detection in time series models
DESCRIPTION
Structural Break Detection in Time Series Models. Richard A. Davis Thomas Lee Gabriel Rodriguez-Yam Colorado State University (http://www.stat.colostate.edu/~rdavis/lectures) This research supported in part by: National Science Foundation IBM faculty award - PowerPoint PPT PresentationTRANSCRIPT
1Corvallis 9/05
Structural Break Detection in Time Series ModelsStructural Break Detection in Time Series Models
Richard A. Davis
Thomas Lee
Gabriel Rodriguez-Yam
Colorado State University
(http://www.stat.colostate.edu/~rdavis/lectures)
This research supported in part by:• National Science Foundation• IBM faculty award
• EPA funded project entitled: Space-Time Aquatic Resources Modeling and Analysis Program (STARMAPSTARMAP))
The work reported here was developed under STAR Research Assistance Agreements CR-829095 awarded by the U.S. Environmental Protection Agency (EPA) to Colorado State University. This presentation has not been formally reviewed by EPA. EPA does not endorse any products or commercial services mentioned in this presentation.
2Corvallis 9/05
Illustrative Example
time
0 100 200 300 400
-6-4
-20
24
6
How many segments do you see?
1 = 51 2 = 151 3 = 251
3Corvallis 9/05
Illustrative Example
time
0 100 200 300 400
-6-4
-20
24
6
1 = 51 2 = 157 3 = 259
Auto-PARM=Auto-Piecewise AutoRegressive Modeling
4 pieces, 2.58 seconds.
4Corvallis 9/05
Introduction Setup Examples
AR GARCH Stochastic volatility State space model
Model Selection Using Minimum Description Length (MDL) General principles Application to AR models with breaks
Optimization using Genetic Algorithms Basics Implementation for structural break estimation
Simulation results
Applications
Simulation results for GARCH and SSM
6Corvallis 9/05
Examples
1. Piecewise AR model:
where 01m-1mn + 1, and {t} is
IID(0,1).
Goal: Estimate
m = number of segments
j = location of jth break point
j = level in jth epoch
pj = order of AR process in jth epoch
= AR coefficients in jth epoch
j = scale in jth epoch
, if , 111 jj-tjptjptjjt tYYYjj
),,( 1 jjpj
7Corvallis 9/05
Piecewise AR models (cont)
Structural breaks:
Kitagawa and Akaike (1978)
• fitting locally stationary autoregressive models using AIC
• computations facilitated by the use of the Householder transformation
Davis, Huang, and Yao (1995)
• likelihood ratio test for testing a change in the parameters and/or order of an AR process.
Kitagawa, Takanami, and Matsumoto (2001)
• signal extraction in seismology-estimate the arrival time of a seismic signal.
Ombao, Raz, von Sachs, and Malow (2001)
• orthogonal complex-valued transforms that are localized in time and frequency- smooth localized complex exponential (SLEX) transform.
• applications to EEG time series and speech data.
8Corvallis 9/05
Motivation for using piecewise AR models:
Piecewise AR is a special case of a piecewise stationary process (see Adak 1998),
where , j = 1, . . . , m is a sequence of stationary processes. It is
argued in Ombao et al. (2001), that if {Yt,n} is a locally stationary
process (in the sense of Dahlhaus), then there exists a piecewise
stationary process with
that approximates {Yt,n} (in average mean square).
Roughly speaking:{Yt,n} is a locally stationary process if it has a time-
varying spectrum that is approximately |A(t/n,|where A(u,is a
continuous function in u.
,)/(~
1),[, 1
m
j
jtnt ntIYY
jj
}{ jtY
, as ,0/ with nnmm nn
}~
{ ,ntY
9Corvallis 9/05
Examples (cont)
2. Segmented GARCH model:
where 01m-1mn + 1, and {t} is
IID(0,1).
3. Segmented stochastic volatility model:
4. Segmented state-space model (SVM a special case):
, if ,
,
122
1122
112
jj-qtjqtjptjptjjt
ttt
tYY
Y
jjjj
. if ,loglog log
,
122
112
jj-tjptjptjjt
ttt
t
Y
jj
. if ,
specified is )|(),...,,,...,|(
111
111
jj-tjptjptjjt
ttttt
t
ypyyyp
jj
10Corvallis 9/05
Model Selection Using Minimum Description Length
Basics of MDL:
Choose the model which maximizes the compression of the data
or, equivalently, select the model that minimizes the code length of
the data (i.e., amount of memory required to encode the data).
M = class of operating models for y = (y1, . . . , yn)
LF (y) = = code length of y relative to F MTypically, this term can be decomposed into two pieces (two-part code),
where
= code length of the fitted model for F
= code length of the residuals based on the fitted model
,ˆ|ˆ( ˆ()( )eL|y)LyL FFF
|y)L F(
)|eL Fˆ(
11Corvallis 9/05
Model Selection Using Minimum Description Length (cont)
Applied to the segmented AR model:
First term :
, if , 111 jj-tjptjptjjt tYYYjj
|y)L F(
m
jj
jm
jj
mmm
np
pnmm
yLyLppLLL(m)|y)L
12
1222
111
log2
2logloglog
)|ˆ()|ˆ(),,(),,(ˆ( F
Encoding: integer I : log2 I bits (if I unbounded)
log2 IU bits (if I bounded by IU)
MLE : ½ log2N bits (where N = number of observations used to
compute ; Rissanen (1989))
12Corvallis 9/05
Second term : Using Shannon’s classical results on
information theory, Rissanen demonstrates that the code length of
can be approximated by the negative of the log-likelihood of the
fitted model, i.e., by
)eL F|ˆ(
m
j
yL)eL1
12 )|ˆ(logˆ|ˆ( F
For fixed values of m, (1,p1),. . . , (m,pm), we define the MDL as
m
j
m
jjj
jm
jj
mm
yLnp
pnmm
ppmMDL
1 122
1222
11
)|ˆ(loglog2
2logloglog
)),(,),,(,(
The strategy is to find the best segmentation that minimizes
MDL(m,1,p1,…, m,pm). To speed things up for AR processes, we use
Y-W estimates of AR parameters and we replace
with jj n )ˆ2(log 22
)|ˆ(log2 yL j
13Corvallis 9/05
Optimization Using Genetic Algorithms
Basics of GA:
Class of optimization algorithms that mimic natural evolution.
• Start with an initial set of chromosomes, or population, of
possible solutions to the optimization problem.
• Parent chromosomes are randomly selected (proportional to
the rank of their objective function values), and produce
offspring using crossover or mutation operations.
• After a sufficient number of offspring are produced to form a
second generation, the process then restarts to produce a third
generation.
• Based on Darwin’s theory of natural selection, the process
should produce future generations that give a smaller (or
larger) objective function.
14Corvallis 9/05
Map the break points with a chromosome c via
where
For example,
c = (2, -1, -1, -1, -1, 0, -1, -1, -1, -1, 0, -1, -1, -1, 3, -1, -1, -1, -1,-1) t: 1 6 11 15
would correspond to a process as follows:
AR(2), t=1:5; AR(0), t=6:10; AR(0), t=11:14; AR(3), t=15:20
Application to Structural Breaks—(cont)
Genetic Algorithm: Chromosome consists of n genes, each taking
the value of 1 (no break) or p (order of AR process). Use natural
selection to find a near optimal solution.
),,,( )),(,),(,( n111 cppm mm
. isorder AR and at timepoint break if ,
,at point break no if ,1
1 jjjt ptp
t
),,,( )),(,),(,( n111 cppm mm
. isorder AR and at timepoint break if ,
,at point break no if ,1
1 jjjt ptp
t
17Corvallis 9/05
Simulation Examples-based on Ombao et al. (2001) test cases
1. Piecewise stationary with dyadic structure: Consider a time
series following the model,
where {t} ~ IID N(0,1).
,1024769 if ,81.32.1
,769513 if ,81.69.1
,5131 if ,9.
21
21
1
tYY
tYY
tY
Y
ttt
ttt
tt
t
Time
1 200 400 600 800 1000
-10
-50
510
,1024769 if ,81.32.1
,769513 if ,81.69.1
,5131 if ,9.
21
21
1
tYY
tYY
tY
Y
ttt
ttt
tt
t
18Corvallis 9/05
1. Piecewise stat (cont)
GA results: 3 pieces breaks at 1=513; 2=769. Total run time 16.31 secs
Time
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
Time
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
True Model Fitted Model
Fitted model:
1- 512: .857 .9945
513- 768: 1.68 -0.801 1.1134
769-1024: 1.36 -0.801 1.1300
19Corvallis 9/05 Time
1 200 400 600 800 1000
-6-4
-20
24
Simulation Examples (cont)
2. Slowly varying AR(2) model:
where and {t} ~ IID N(0,1).
10241 if 81. 21 tYYaY ttttt
)],1024/cos(5.01[8. tat
0 200 400 600 800 1000
time
0.4
0.6
0.8
1.0
1.2
a_t
10241 if 81. 21 tYYaY ttttt
)],1024/cos(5.01[8. tat
20Corvallis 9/05Time
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
2. Slowly varying AR(2) (cont)
GA results: 3 pieces, breaks at 1=293, 2=615. Total run time 27.45 secs
True Model Fitted Model
Fitted model:
1- 292: .365 -0.753 1.149
293- 614: .821 -0.790 1.176
615-1024: 1.084 -0.760 0.960
Time
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
24Corvallis 9/05
2. Slowly varying AR(2) (cont)
True Model Average Model
In the graph below right, we average the spectogram over the GA fitted models generated from each of the 200 simulated realizations.
Time
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
Time
Freq
uenc
y
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
27Corvallis 9/05
Examples
Speech signal: GREASY
Time
1 1000 2000 3000 4000 5000
-150
0-5
000
500
1000
1500
G R EA S Y
28Corvallis 9/05
Time
1 1000 2000 3000 4000 5000
-150
0-5
000
500
1000
1500
G R EA S Y
Time
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
Speech signal: GREASY
n = 5762 observations
m = 15 break points
Run time = 18.02 secs
29Corvallis 9/05
Application to GARCH (cont)
Garch(1,1) model:
.1000501 if ,6.1.4.
,5011 if ,5.1.4.2
121
21
212
tY
tY
tt
ttt
. if ,
IID(0,1)~}{ ,
12
121
2jj-tjtjjt
tttt
tY
Y
# of CPs GA%
AG%
0 80.4 72.0
1 19.2 24.0
2 0.4 0.4
AG = Andreou and Ghysels (2002)
.1000501 if ,6.1.4.
,5011 if ,5.1.4.2
121
21
212
tY
tY
tt
ttt
Time
1 200 400 600 800 1000
-4-2
02
CP estimate = 506
. if ,
IID(0,1)~}{ ,
12
121
2jj-tjtjjt
tttt
tY
Y
30Corvallis 9/05
time
y
1 100 200 300 400 500
05
1015
Count Data Example
Model: Yt | t Pois(exp{t}), t = t-1+ t , {t}~IID N(0,
Breaking PointM
DL
1 100 200 300 400 500
1002
1004
1006
1008
1010
1012
1014
True model:
Yt | t ~ Pois(exp{.7 + t }), t = .5t-1+ t , {t}~IID N(0, .3), t < 250
Yt | t ~ Pois(exp{.7 + t }), t = -.5t-1+ t , {t}~IID N(0, .3), t > 250.
GA estimate 251, time 267secs
31Corvallis 9/05
time
y
1 500 1000 1500 2000 2500 3000
-2-1
01
23
Model: Yt | t N(0,exp{t}), t = t-1+ t , {t}~IID N(0, )
SV Process Example
True model:
Yt | t ~ N(0, exp{t}), t = -.05 + .975t-1+ t , {t}~IID N(0, .05), t 750
Yt | t ~ N(0, exp{t }), t = -.25 +.900t-1+ t , {t}~IID N(0, .25), t > 750.
GA estimate 754, time 1053 secs
Breaking Point
MD
L
1 500 1000 1500 2000 2500 3000
1295
1300
1305
1310
1315
32Corvallis 9/05
time
y
1 100 200 300 400 500
-0.5
0.0
0.5
Model: Yt | t N(0,exp{t}), t = t-1+ t , {t}~IID N(0, )
SV Process Example
True model:
Yt | t ~ N(0, exp{t}), t = -.175 + .977t-1+ t , {t}~IID N(0, .1810), t 250
Yt | t ~ N(0, exp{t }), t = -.010 +.996t-1+ t , {t}~IID N(0, .0089), t > 250.
GA estimate 251, time 269s
Breaking Point
MD
L
1 100 200 300 400 500
-530
-525
-520
-515
-510
-505
-500
33Corvallis 9/05
SV Process Example-(cont)
time
y
1 100 200 300 400 500
-0.5
0.0
0.5
Fitted model based on no structural break:
Yt | t N(0, exp{t}), t = -.0645 + t-1+ t , {t}~IID N(0,
True model:
Yt | t ~ N(0, exp{at}), t = -.175 + .977t-1+ et , {t}~IID N(0, .1810), t 250
Yt | t N(0, exp{t}), t = -.010 t-1+ t , {t}~IID N(0, t > 250.
time
y
1 100 200 300 400 500
-0.5
0.0
0.5
1.0
simulated seriesoriginal series
34Corvallis 9/05
SV Process Example-(cont)
Fitted model based on no structural break:
Yt | t N(0, exp{t}), t = -.0645 + t-1+ t , {t}~IID N(0,
time
y
1 100 200 300 400 500
-0.5
0.0
0.5
1.0 simulated series
Breaking Point
MD
L
1 100 200 300 400 500
-478
-476
-474
-472
-470
-468
-466
35Corvallis 9/05
Summary Remarks
1. MDL appears to be a good criterion for detecting structural breaks.
2. Optimization using a genetic algorithm is well suited to find a near optimal value of MDL.
3. This procedure extends easily to multivariate problems.
4. While estimating structural breaks for nonlinear time series models is more challenging, this paradigm of using MDL together GA holds promise for break detection in parameter-driven models and other nonlinear models.