time series, spring 2014 - chalmersrootzen/timeseries/timeseries... · stochastic process if and...
TRANSCRIPT
![Page 1: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/1.jpg)
Time series, spring 2014 1
![Page 2: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/2.jpg)
The course• Time series, stochastic processes, existence theorem,
stationarity and strict stationarity, autocovariance functions, multivariate normal distribution
• Trends and seasonality• Hilbert spaces• ARMA processes• Spectral methods• Prediction• Estimation, model selection, and model checking• Multivariate time series (perhaps)• Kalman filters• ARIMA, unit root , and stochastic volatility models
![Page 3: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/3.jpg)
Course literature
(TS) Time Series: Theory and Methods, second edition (1991) P.J. Brockwell and R.A. Davis, Springer-Verlag, New York.
Complementary literature:(ITS): Introduction to Time Series and Forecasting, second edition (2002) P.J. Brockwell and R.A. Davis, Springer-Verlag, New York + material from new version of the book, soon to appear.
ITSM: The program package which comes with the books:http://www.stat.columbia.edu/~rdavis/Pune2015/itsm7pro.zip
Slides provided by Richard Davis
SSPSE: Stationary stochastic processes for scientists and engineers(2013) G. Lindgren, H. Rootzén, and M. Sandsten
![Page 4: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/4.jpg)
Slides for TS ch 1, p 1-37
Exercises: TS 1.2, 1.4, 1.5, 1.6, 1.7, 1.8, 1.10, 1.11, 1.12, 1.13,1.15, 1.18
![Page 5: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/5.jpg)
Australian red wine sales
(thou
sand
s)
1 9 8 0 1 9 8 2 1 9 8 4 1 9 8 6 1 9 8 8 1 9 9 0 1 9 9
0.5
1.0
1.5
2.0
2.5
3.0
Features: upward trendseasonal pattern (peak in July, trough in Jan)increase in variability
![Page 6: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/6.jpg)
Time Series Plots: Examine for
• trend over time (does the series increase or decrease with time)
• regular seasonal (or cyclical) components
• constant variability over time• other systematic features of the data
![Page 7: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/7.jpg)
Ex 1.1.4 (All-star baseball games, 1933-1995)
1 9 4 0 1 9 5 0 1 9 6 0 1 9 7 0 1 9 8 0 1 9 9 0
-2-1
01
2
1 if National league won-1 if American league won
![Page 8: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/8.jpg)
Ex 1.1.6 (Accidental deaths, USA; DEATHS.TSM)
(thou
sand
s)
1 9 7 3 1 9 7 4 1 9 7 5 1 9 7 6 1 9 7 7 1 9 7 8 1 9 7
78
910
11
Features: slight trendseasonal component (peak in July)
Monthly accidental deaths
![Page 9: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/9.jpg)
s.rv' N(0,.25) of sequence IIDan is N where2,...,200 1,= t, N )10cos(t/
t
t+=tX
Ex. (Signal Detection; SIGNAL.TSM)
Figure : red = estimated signal
black= true signal
0 5 1 0 1 5 2 0
-2-1
01
23
![Page 10: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/10.jpg)
Stochastic (or random) variableDescribes the mechanism of drawing a number randomly from a bag of numbers,With different probabilities for different numbers
Stochastic processDescribes the mechanism of drawing numbers randomly from many bags of numbers
or (equivalently) the mechanism of drawing a function randomly from a bag of functions
3.49
7.14 8.72 1.03
![Page 11: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/11.jpg)
Stochastic variables and processesΩ,𝐹𝐹,𝑃𝑃 Probability space. Ω is the ”bag of numbers”, 𝑃𝑃
determines the probability of choosing a particular set ofnumbers, 𝐹𝐹 contains all sets which have a probability
𝑋𝑋 Stochastic variable. A function Ω → 𝑹𝑹
𝑇𝑇 Parameter space. In this course 𝑇𝑇 is ”time”: 𝑹𝑹 or 0, 1, 2, … or 0, ±1, ±2, … or …
𝑋𝑋𝑡𝑡; 𝑡𝑡 ∈ 𝑇𝑇 Stochastic process. A collection (or family) of random variables
or, a function Ω → 𝑹𝑹𝑻𝑻 , i.e. a function-valued randomvariable
or, a distribution (probability measure) on the space offunctions T → 𝑹𝑹
![Page 12: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/12.jpg)
time
x_t
0 5 10 15 20
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
A realisation (or sample function, or sample path, or sample field, or observation, or trajectory, or …) is the function
𝑥𝑥⋅ 𝜔𝜔 : 𝑇𝑇 → 𝑹𝑹𝑑𝑑𝑡𝑡 → ft(𝜔𝜔)
for 𝜔𝜔 fixed.
An example: 5 realisations 𝑋𝑋𝑡𝑡 𝜔𝜔1 ,𝑋𝑋𝑡𝑡 𝜔𝜔2 ,𝑋𝑋𝑡𝑡 𝜔𝜔3 ,𝑋𝑋𝑡𝑡 𝜔𝜔4 , 𝑋𝑋𝑡𝑡 𝜔𝜔5 of the stochastic process 𝑋𝑋𝑡𝑡 𝜔𝜔 = 𝐴𝐴 𝜔𝜔 cos(𝜋𝜋
4𝑡𝑡 +
Θ 𝜔𝜔 , with 𝐴𝐴 > 0,Θ ∼ 𝑈𝑈(0, 2 𝜋𝜋)
![Page 13: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/13.jpg)
Terminology
random variablestochastic variable
random elementstochastic element
random processstochastic process
random fieldstochastic field
random vectorstochastic vector
![Page 14: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/14.jpg)
Objectives of Time Series AnalysisModelling paradigm:
• set up family of probability models to represent data • estimate parameters of model• check model for goodness of fit
Applications of models:
• provides a compact description of the data• interpretation • prediction• hypothesis testing
![Page 15: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/15.jpg)
Ex: Binary Process
Xt ~ IID
P[Xt = 1] = p, P[Xt = -1] = 1-p,
This is a family of SPs indexed by p. How do we know a probability space (Ω,F,P) exists? Answer given by Kolmogorov’s theorem (next slide). If p=.5, then
P[X1 = i1, X2 = i2 , . . .,Xn = in] =1/2n
for any n-tuple (i1, i2 , . . ., in) of ±1’s
Is this a good model for All Star baseball games?
Ex: (Random Walk). Let Xt ~ IID as in previous example
S0= 0, St= X1+ . . . + Xt , t ≥1
![Page 16: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/16.jpg)
The finite dimensional distributions (fidis) of a stochastic process Xt, t ∈ T are the collection of functions
where x = (x1, . . . , xn)’ and t = (t1, . . . , tn)’ are vectors in 𝑹𝑹𝑛𝑛
and 𝑇𝑇𝑛𝑛, with t1< t2< … < tn.
)...,,()( 21 21 nttt xXxXxXPFn
≤≤≤=xt
Kolmogorov’s consistency teorem
Three realizations of a stochasticprocess. 𝐹𝐹2,5,8 𝑥𝑥1, 𝑥𝑥2, 𝑥𝑥3 is the probability to obtain a sample path which passes through all three vertical lines
![Page 17: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/17.jpg)
Theorem (Kolmogorov): The family of probability distribution functions Ft, t∈ Tn , t1< t2< … < tn, n ≥1 are the fidis of a stochastic process if and only if
where t(i) and x(i) are the vectors of t and x with the ith
component removed (i=1,…,n).
(Nice) functions of stochastic processes are stochastic processes – this is not proved in book, but is typically how processes are shown to exist in course. Existence or not is a recurring and important problem in time series analysis.
))(()(lim )( iFF ixi
xx tt =∞↑
![Page 18: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/18.jpg)
E.g.
So one direction of the theorem is obvious (the fidis of a SP must satisfy the consistency condition). Kolmogorov says that the consistency condition is also sufficient.
One can easily check that IID sequences exist by using Kolmogorov’s theorem. (Do this!)
),,(),,(
),,,(lim),,,(lim
4214,2,1
442211
4433221143214,3,2,133
xxxFxXxXxXP
xXxXxXxXPxxxxFxx
=≤≤≤=
≤≤≤≤=∞↑∞↑
![Page 19: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/19.jpg)
Let Xt be a time series (or SP) with E Xt2 < ∞ . Then the
autovariance function (ACVF) is given by
The time series Xt is (weakly) stationary if
(i) E Xt2 < ∞ for all t
(ii) E Xt is independent of t (mean is constant)
(iii) γ(r,s)=γ(r+t,s+t) for all r, s, t (covariance only depends on distance between time points)
( )( )[ ]ssrrsr EXXEXXEXXCovsr −−==γ ),(),(
Stationarity and strict stationarity
![Page 20: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/20.jpg)
• Replacing t with –s gives that
γ(r,s) = γ(r-s,s-s) = γ(r-s,0)
so that γ is a function of the time gap r-s. We hence write
γ(h)= γ(h,0) = Cov(Xt+h , Xt).
• Autocorrelation function (ACF). Note that γ(0) = Cov(Xt) so that ρ(h) = Cor(Xt+h , Xt)
= γ(h)/γ(0).
• in the literature, stationarity = weak stationarity
= stationarity in the wide sense= second-order stationarity
![Page 21: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/21.jpg)
The time series Xt, t=0, ±1, ±2, …. is strictly stationary if for any k ≥ 1,
(X1, . . . , Xk) =d (X1+h, . . . , Xk+h)
for all integers h. (=d means the two vectors have the same distribution)
e.g.,
• (X1, X2 , X3 , X4)’ =d (X0, X1 , X2 , X3)’
=d (X200, X201 , X202 , X203)’
=d etc
• X1=d Xh for all integers h, so the stochastic variables in a strictly stationary stochastic process are identically distributed
• (X1, Xk)’ =d (X1+h, Xk+h)’
![Page 22: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/22.jpg)
• If the time series Xt is strictly stationary with E Xt2 < ∞ , then
a) X1=d X2=d X3 …. and hence the mean must be constant,EX1= EX2= EX3 ….
b) (Xr, Xs)’ =d (X0, Xs-r)’ which implies that
Cov(Xr, Xs)= Cov(X0 , Xs-r) = γ(s-r).
(a) and (b) imply that Xt is stationary.
• The converse is not necessarily true:
let Zt ~ IID N(0,1) and Yt ~IID Exp(1) and define
Then 𝑋𝑋𝑡𝑡 is weakly stationary, but not strictly stationary. (Check this!)
odd is t ifeven is t if
,1,
−=
t
tt Y
ZX
![Page 23: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/23.jpg)
Random walk
St= X1+ . . . + Xt , Xt ~ IID(0,σ2)
Is St stationary?
• ESt = 0
v(St+h , St) = Cov(X1+ . . . + Xt+h, X1+ . . . + Xt )
= Cov(X1+ . . . Xt+…+ Xt+h, X1+ . . . + Xt )
= Cov(St + St+h - St , St)
= Cov(St, St)
= t σ2
Since this ACVF depends on t, the process is not stationary.
![Page 24: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/24.jpg)
Cosine process.
Xt= A cos(θt) + Bsin(θt),
where A and B are independent random variables with mean 0 and variance 1, and 𝜃𝜃 is a constant
EXt = 0
Cov(Xt+h , Xt)
= Cov(A cos(θ(t+h)) + Bsin(θ(t+h)), A cos(θt) + Bsin(θt))= cos(θ(t+h))cos(θt) + sin(θ(t+h))sin(θt) =cos(θh)
The process is weakly stationary (and in fact also strictly stationary)
![Page 25: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/25.jpg)
IID noise
Xt ~ IID (0,σ2)
IID means independent and identically distributed. Both stationary and strictly stationary
White noise
Xt ~ WN (0,σ2)
means that the variables are uncorrelated but not necessarily IID.
IID WN
IID WN
0.h if 0h if
,0,
)(2
≠=
σ
=γ h
![Page 26: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/26.jpg)
Moving Average; MA(1).
Xt = Zt + θ Zt-1 , Zt ~ WN(0,σ2)
This implies that the time series is stationary. Moreover,
1 |h| if1 h if
0 h if
,0,
,)(1)( 2
22
>±=
=
θσσθ+
=γ h
1 |h| if1 h if
0 h if
,0),1/(
,1)( 2
>±=
=
θ+θ=ρ h
![Page 27: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/27.jpg)
Gaussian processesA random vector 𝐗𝐗 = 𝑋𝑋1, … ,𝑋𝑋𝑑𝑑 has a multivariate Gaussian distribution iff one of the following conditions hold:
• 𝛼𝛼, 𝑥𝑥 ≜ ∑𝑖𝑖𝑑𝑑 𝛼𝛼𝑖𝑖𝑋𝑋𝑖𝑖 has a univariate normal distribution for all 𝛼𝛼 ∈ 𝑅𝑅𝑑𝑑 .
• There exist a vector 𝝁𝝁 ∈ 𝑹𝑹𝑑𝑑 and a non-negative definite matrix Σ such that for all 𝜽𝜽 ∈ 𝑹𝑹𝑑𝑑
𝜙𝜙 𝜽𝜽 = 𝐸𝐸 𝑒𝑒𝑖𝑖𝜽𝜽𝑋𝑋 = e𝑖𝑖𝑖𝑖𝝁𝝁−12𝜽𝜽𝚺𝚺𝜽𝜽´
If Σ is positive definite then 𝑋𝑋 has the probability density1
2𝜋𝜋 𝑑𝑑 Σ 1/2 𝑒𝑒−12 𝒙𝒙−𝝁𝝁 Σ−1 𝒙𝒙−𝝁𝝁 ´
then 𝑋𝑋 is Gaussian.
Here 𝝁𝝁 = 𝐸𝐸 𝑿𝑿 and Σ = 𝐶𝐶𝐶𝐶𝐶𝐶 𝑿𝑿 .
![Page 28: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/28.jpg)
We write 𝐗𝐗~𝑁𝑁𝑑𝑑 𝝁𝝁, Σ if 𝐗𝐗 has a d-variate Gaussian distribution with mean 𝑚𝑚 and covariance matrix Σ.
Important results, read TS
(i) if 𝐗𝐗~𝑁𝑁𝑑𝑑 𝝁𝝁, Σ and 𝐴𝐴 is a 𝑑𝑑 × 𝑑𝑑 matrix, then 𝐗𝐗𝐴𝐴~𝑁𝑁𝑑𝑑 𝝁𝝁𝐴𝐴,𝐴𝐴𝐴Σ𝐴𝐴
(ii) If 𝑿𝑿 = 𝑿𝑿1,𝑿𝑿2 with 𝑿𝑿1 = 𝑋𝑋1, … ,𝑋𝑋𝑛𝑛 , 𝑿𝑿2 = 𝑋𝑋𝑛𝑛+1, … ,𝑋𝑋𝑑𝑑 ,with mean vectors 𝝁𝝁1 and 𝝁𝝁2 and covariance matrix Σ =Σ1,1 Σ1,2Σ2,1 Σ2,2
, then the conditional distribution of 𝑿𝑿1 given 𝑿𝑿2 is n-variate normal with mean
𝝁𝝁1|2 = 𝝁𝝁1 + (𝑿𝑿2−𝝁𝝁2)Σ2,2−1Σ2,1
and covariance matrix
Σ1|2 = Σ1,1 − Σ1,2Σ2,2−1Σ2,1
![Page 29: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/29.jpg)
• Xt is a Gaussian time series iff all the fidis are multivariate normal.
So if Xt is Gaussian, then for any 𝑛𝑛 the distribution of X= (X1, . . . , Xn) is multivariate normal with
mean vector: µn = E(X1, . . . , Xn) covariance matrix: Σn = Cov(X,X)
If Σn is nonsingular, then X has a pdf
f(x) = (2π)-n/2 |Σn|-1/2 exp-1/2 (x- µn)’ Σn-1 (x- µn)
• A Gaussian time series is completely determined by its mean and autocovariance functions (i.e. second order properties)
• A Gaussian time series is stationary iff it is strictly stationary
![Page 30: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/30.jpg)
Classical Decomposition:
Xt = mt + st + Yt
mt trend component (slowly changing function of t )
st seasonal component (periodic with period d )
Yt random noise component
Remove trend and seasonal component, and then model the result (the residuals) as a stationary time series
Trends and seasonal components
![Page 31: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/31.jpg)
Model with no seasonal component.
Xt = mt + Yt ,
where mt is a slowly varying function called the trend function and 𝑌𝑌𝑡𝑡 is the residual process. We assume E Yt = 0 -- this is a modelling assumption.
e.g. mt = a0 + a1 t + a2 t2
where coefficients are estimated by minimizing the sum of squares of the deviations between the observations 𝑥𝑥1, … , 𝑥𝑥𝑛𝑛and 𝑚𝑚, i.e. by minimizing
∑𝑡𝑡=1𝑛𝑛 𝑥𝑥𝑡𝑡 − 𝑚𝑚𝑡𝑡2
with respect to 𝑎𝑎0,𝑎𝑎1,𝑎𝑎2.
Method 1: trend estimation via least squares
![Page 32: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/32.jpg)
Model: Xt = a0 + a1 t + a2 t2 + Yt
Ex. Population of USA.; USPOP.TSM
a0 = 6.96x105 , a2= 6.51x 105
Figure 1.8.
a1= -2.16 x 106,
(Milli
ons)
1 8 0 01 8 2 01 8 4 01 8 6 01 8 8 01 9 0 01 9 2 01 9 4 01 9 6 01 9 8 0
020
4060
8012
016
020
024
0
Forecast for year 2000: m2000 = 274.35 x 106
^ ^ ^
^
![Page 33: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/33.jpg)
Ex. Lake Huron Levels (1875-1972); LAKE.TSM
Figure 1.9.
Model: Xt = a0 + a1 t + Yt
1 8 8 01 8 9 01 9 0 01 9 1 01 9 2 01 9 3 01 9 4 01 9 5 01 9 6 01 9 7 0
6.0
7.0
8.0
9.0
10.0
11.0
12.0
10.2-.0242 t
![Page 34: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/34.jpg)
Residuals from the LS fit of Lake Huron data
1 8 8 0 1 9 0 0 1 9 2 0 1 9 4 0 1 9 6 0
-2-1
01
2
Note: residuals do not look IID
![Page 35: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/35.jpg)
finite moving average filter: assume 𝑋𝑋𝑡𝑡 = 𝑚𝑚𝑡𝑡 + 𝑌𝑌𝑡𝑡, where 𝑚𝑚𝑡𝑡varies slowly (is smooth) and 𝑌𝑌𝑡𝑡 has mean zero
𝑊𝑊𝑡𝑡 =1
2𝑞𝑞 + 1𝑗𝑗 ≤𝑞𝑞
𝑋𝑋𝑡𝑡+𝑗𝑗
=1
2𝑞𝑞 + 1𝑗𝑗 ≤𝑞𝑞
𝑚𝑚𝑡𝑡+𝑗𝑗 +1
2𝑞𝑞 + 1𝑗𝑗 ≤𝑞𝑞
𝑌𝑌𝑗𝑗
≈ mt + 0
Method 2: smoothing to estimate trend
![Page 36: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/36.jpg)
linear filter𝑋𝑋𝑡𝑡 𝑚𝑚𝑡𝑡 = 𝑊𝑊𝑡𝑡 = ∑𝑗𝑗 𝑎𝑎𝑗𝑗𝑋𝑋𝑡𝑡−𝑗𝑗
5-term moving average of strikes data STRIKES.TSM
(thou
sand
s)
1 9 5 0 1 9 6 0 1 9 7 0 1 9 8 0
3.5
4.0
4.5
5.0
5.5
6.0
![Page 37: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/37.jpg)
Exponential smoothing
𝑚𝑚𝑡𝑡 = 𝑎𝑎𝑋𝑋𝑡𝑡 + 1 − 𝑎𝑎 𝑚𝑚𝑡𝑡−1, t= 2, … n𝑚𝑚1 = 𝑋𝑋1
0 1a
max smoothing no smoothing
Strike data a = .4
(thou
sand
s)
1 9 5 0 1 9 6 0 1 9 7 0 1 9 8 0
3.5
4.0
4.5
5.0
5.5
6.0
![Page 38: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/38.jpg)
Classical Decomposition Model:
𝑋𝑋𝑡𝑡 = 𝑚𝑚𝑡𝑡 + 𝑠𝑠𝑡𝑡 + 𝑌𝑌𝑡𝑡
where, for seasons of length 𝑑𝑑
𝑠𝑠𝑡𝑡 = 𝑠𝑠𝑡𝑡+𝑑𝑑 and 𝑡𝑡=1
𝑑𝑑
𝑠𝑠𝑡𝑡 = 0 ,
and𝐸𝐸 𝑌𝑌𝑡𝑡 = 0
Estimation of both trends and seasonality
![Page 39: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/39.jpg)
Step 1: Estimate the trend using a simple moving average of length q = d/2 (or (d-1)/2).
Step 2: Estimate sk, k =1,…,d using the average deviations from trend for each season.
Step 3: Deseasonalize the data by forming
dt = xt − st , t=1,…,n
Step 4: Fit a parametric function mt to the deseasonalizeddata dt.
Step 5: Calculate the estimated noise/residuals
Yt = xt − mt − st
^
^ ^ ^
![Page 40: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/40.jpg)
The accidentals deaths data.data = blue boxes
st = red lines
(thou
sand
s)
1 9 7 3 1 9 7 4 1 9 7 5 1 9 7 6 1 9 7 7 1 9 7 8 1 9 7
78
910
11
![Page 41: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/41.jpg)
Difference Operator 𝛻𝛻 ≔ 1 − 𝐵𝐵
𝛻𝛻𝑋𝑋𝑡𝑡 = 𝑋𝑋𝑡𝑡 − 𝑋𝑋𝑡𝑡−1𝛻𝛻2𝑋𝑋𝑡𝑡 = 𝛻𝛻(𝑋𝑋𝑡𝑡 − 𝑋𝑋𝑡𝑡−1)
= 𝑋𝑋𝑡𝑡 − 2𝑋𝑋𝑡𝑡−1 + 𝑋𝑋𝑡𝑡−2
Backward Shift Operator B :
B Xt = Xt-1
Bs Xt = Xt-s , s=0, + 1, . . ..
Method 3: differencing to eliminatetrends and seasonality
![Page 42: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/42.jpg)
Seasonal Differencing 𝛻𝛻𝑑𝑑 ≔ 1 − 𝐵𝐵𝑑𝑑
𝛻𝛻𝑑𝑑𝑋𝑋𝑡𝑡 = 𝑋𝑋𝑡𝑡 − 𝑋𝑋𝑡𝑡−𝑑𝑑If 𝑠𝑠𝑡𝑡 is a seasonal component with period 𝑑𝑑 then
𝛻𝛻𝑑𝑑𝑠𝑠𝑡𝑡 = 𝑠𝑠𝑡𝑡 − 𝑠𝑠𝑡𝑡−𝑑𝑑 = 0
𝛻𝛻 applied to a trend function.
mt = a0 + a1 t
𝛻𝛻mt = mt - mt-1 = a0 + a1 t - (a0 + a1 (t-1))
= a1
Generally, 𝛻𝛻𝑘𝑘 applied to a polynomial of degree k gives a constant.
![Page 43: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/43.jpg)
Differenced monthly accidental deaths.xt = xt - xt-12 , t = 13, . . . , 72.
(thou
sand
s)
1 9 7 4 1 9 7 5 1 9 7 6 1 9 7 7 1 9 7 8 1 9 7
-1.0
-0.5
0.0
0.5
∇12
Deseasonalized series still exhibits trend which we attempt to remove by differencing
![Page 44: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/44.jpg)
Detrended and deseasonalized accidental deaths.
xt = xt - xt-1 - xt-12 + xt-13 , t = 14, . . . , 72.∇12∇
1 9 7 4 1 9 7 5 1 9 7 6 1 9 7 7 1 9 7 8 1 9 7
-500
050
010
00
![Page 45: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/45.jpg)
Quantile regression
For heavytailed data it may sometimes be better to use quantile/median regression instead of least squares. On then proceeds exactly as in method 1, but uses quantile regression instead of least squares. References and some discussion may be found at https://en.wikipedia.org/wiki/Quantile_regression
State of the art seasonal adjustment program
X-13ARIMA-SEATS Seasonal Adjustment Program, available at https://www.census.gov/srd/www/x13as/
![Page 46: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/46.jpg)
If γ(h) = Cov(Xt+h, Xt ) is the ACVF of a stationary time series Xt, then
• γ(0) ≥ 0,
• |γ(h)| ≤ γ(0)
• γ(h) = γ(−h) (i.e. γ(.) is an even function)
Proof: (i) follows from the fact that Var(Xt) ≥ 0. (ii) follows from the Cauchy-Schwarz inequality
|Cov(Xt+h, Xt)| ≤ Var(Xt+h)1/2 Var(Xt)1/2 = γ(0).
(iii) Follows from the fact that
γ(−h) = Cov(Xt-h, Xt ) = Cov(Xt+h, Xt ) = γ(h).
The ACVF of a stationary process
![Page 47: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/47.jpg)
Non-negative definiteness: A real-valued function κ(h) defined on the integers is non-negative definite (nnd) iff
for all n and choices of vectors a = (a1, . . . , an)’ .
0)(1,
≥−κ∑=
j
n
jii ajia
Theorem: γ(h) is the ACVF of a/some stationary TS iff
1) γ(.) is an even function
2) γ(.) is nnd
Proof: 1) If γ(h) is the ACVF of a stationary TS Xt, then by previous result, γ(h) is even and with X = (X1, . . . , Xn)’, we have
0≤Var(a’X)=a’Cov(X,X)a = Σi,jaiCov(Xi,Xj)aj = Σi,jaiγ(i-j)aj
![Page 48: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/48.jpg)
Now suppose γ(h) is even and nnd. We construct a stationary zero mean Gaussian TS with γ as its ACVF using Kolmogorov’s theorem. For t = (t1, . . . , tn)’ with t1< t2< … < tn define Ft as the multivariate normal distribution with mean zero and covariance matrix Σ𝑛𝑛= [γ(ti-tj)]i,j=1,…n. That is, the characteristic function associated with Ft is
φt(u)=exp(-u’ Σ𝑛𝑛u/2).
Since γ(h) is nnd, Kn is a nnd definite matrix and hence this is a legitimate characteristic function. The consistency condition in Kolmogorov’s theorem is equivalent to
which is easy to check holds, and this completes the proof.
))(()(lim )(0iiui
uu tt φ=φ→
![Page 49: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/49.jpg)
For any given ACVF, we constructed a stationary Gaussian TS with the specified ACVF.
• It can be hard to verify that a given function γ(h) is an ACVF, by checking it is nnd. Instead, one might try to find a stationary TS with the designated ACVF.
Ex. Is γ(h) := cos (θh) an ACVF?
Ans. Yes. But verification of nnd property is difficult.
Easier to check that cos (θh) is the ACVF of
Xt = A cos (θh) + B sin (θh)
where A & B are uncorrelated (0,1) rv’s.
• The ACF ρ(h) has the same properties as γ(h) and ρ(0)=1.
![Page 50: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/50.jpg)
Ex. The function
is an ACVF iff |ρ| ≤ .5.
To see this, if |ρ| ≤ .5, then γ is the ACVF of the MA(1) process with σ2=(1+θ2)-1 and θ=(2ρ)-1(1- sqrt(1-4ρ2)).
On the other hand, if |ρ| > .5, let a be the n-vector
(1,-1,1,-1,…)’. Then, for n large,
otherwise 1 if
0 if
,0,,1
)( ±==
ρ=γ h
hh
0)1(2,...)1,1,1(
100
01001001
,...)1,1,1(' <ρ−−=−
ρ
ρρρ
ρ
−= nnKn
aa
![Page 51: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/51.jpg)
Observed data: x1 , . . . , xn
Sample mean :
Sample autocovariance function :
Sample autocorrelation function :
• Division by n instead of (n-h) ensures that sample cov
matrix is nnd.
∑=
−=n
ttxnx
1
1
0hfor ,))(()(ˆ1
1 ≥−−=γ ∑−
=+
−hn
thtt xxxxnh
)0(ˆ)(ˆ
)(ˆγγ
=ρhh
njin ji 1,)](ˆ[ˆ=−γ=Γ
The sample ACVF
![Page 52: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/52.jpg)
• γ(h) is approximately the sample covariance function of (x1 , x1+h), . . . ,(xn-h , xn)
• If data are observations from IID noise, then
ρ(h) is approx N(0, n -1)
• For IID noise,
|ρ(h)| < 1.96 n -.5 with probability .95
![Page 53: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/53.jpg)
Ex. 200 observations from IID N(0,1)).
0 5 0 1 0 0 1 5 0 2 0 0
-2-1
01
23
L a g
ACF
0 1 0 2 0 3 0 4 0
0.00.2
0.40.6
0.81.0
![Page 54: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/54.jpg)
Sample ACF of WINE.TSM
L a g
ACF
0 1 0 2 0 3 0 4 0
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
![Page 55: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/55.jpg)
Let residuals from the LS fit be denoted by
yt = xt - a0 - a1 t , t=1,...,98
L a g
ACF
0 1 0 2 0 3 0 4 0
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
ACF of residuals yt
A Model for the Lake Huron Data
![Page 56: Time series, spring 2014 - Chalmersrootzen/timeseries/timeseries... · stochastic process if and only if where . t (i) and . x (i) are the vectors of . t. and . x. with the i. th](https://reader035.vdocument.in/reader035/viewer/2022062919/5f01bf547e708231d400d7e1/html5/thumbnails/56.jpg)
Scatter plot of residuals (yt-1 , yt) showing regression line y = .791 x.
- 2 . 5 - 2 . 0 - 1 . 5 - 1 . 0 - 0 . 5 0 . 0 0 . 5 1 . 0 1 . 5 2 . 0 2 . 5
-2.5
-1.5
-0.5
0.5
1.5
2.5
Model: Yt = .791 Yt-1 + Zt , Zt ~ WN(0,σ2) (Autoregressive or AR(1) model)