Download - Stationary Time Series Slides
-
7/30/2019 Stationary Time Series Slides
1/17
Covariance Stationary Time Series
Stochastic Process: sequence of rvs ordered by time
{Yt}
= {. . . , Y
1, Y0, Y1, . . .}
Defn: {Yt} is covariance stationary if
E[Yt] = for all t
cov(Yt, Ytj) = E[(Yt )(Ytj )] = j forall t and any j
Remarks
j = jth lag autocovariance; 0 = var(Yt)
j = j/0 = jth lag autocorrelation
Example: Independent White Noise (IW N(0, 2))
Yt = t, t iid (0, 2)E[Yt] = 0, var(Yt) =
2, j = 0, j 6= 0
Example: Gaussian White Noise (GW N(0, 2))
Yt = t, t iid N(0, 2)
Example: White Noise (W N(0, 2))
Yt = t
E[t] = 0, var(t) = 2, cov(t, tj) = 0
-
7/30/2019 Stationary Time Series Slides
2/17
Nonstationary Processes
Example: Deterministically trending process
Yt = 0 + 1t + t, t
W N(0, 2)
E[Yt] = 0 + 1t depends on t
Note: A simple detrending transformation yield a station-
ary process:
Xt = Yt 0 1t = t
Example: Random Walk
Yt = Yt1 + t, t W N(0, 2), Y0 is fixed
= Y0 +tX
j=1
j var(Yt) = 2t depends on t
Note: A simple detrending transformation yield a station-
ary process:
Yt = Yt Yt1 = t
Wolds Decomposition Theorem
Any covariance stationary time series {Yt} can be repre-
sented in the form
Yt = + X
j=0
jtj, t W N(0, 2)
0 = 1,X
j=0
2j <
Properties:
E[Yt] =
0 = var(Yt) = 2X
j=0
2j <
j = E[(Yt )(Ytj )]= E[(t + 1t1 + + jtj + )
(t
j + 1t
j
1 + )
= 2(j + j+11 + )
= 2X
k=0
kk+j
-
7/30/2019 Stationary Time Series Slides
3/17
Autoregressive moving average models (ARMA) Models
(Box-Jenkins 1976)
Idea: Approximate Wold form of stationary time series
by parsimonious parametric models (stochastic difference
equations)
ARMA(p,q) model:
Yt = 1(Yt1 ) + + p(Yt2 )+t + 1t1 + + qtq
t W N(0, 2)Lag operator notation:
(L)(Yt ) = (L)t(L) = 1 1L pLp(L) = 1 + 1L + + qL
q
Stochastic difference equation
(L)Xt = wt
Xt = Yt , wt = (L)t
ARMA(1,0) Model (1st order SDE)
Yt = Yt1 + t, t W N(0, 2)Solution by recursive substitution:
Yt = t+1Y1 + tY0 + t0 + + t1 + t
= t+1Y1 +tX
i=0
iti
= t+1Y1 +tX
i=0
iti, i = i
Alternatively, solving forward j periods from time t :
Yt+j = j+1Yt1 + jt + + t+j1 + t+j
= j+1Yt1 +jX
i=0
it+ji
Dynamic Multiplier:
dYjd0
= dYt+jdt
= j = j
-
7/30/2019 Stationary Time Series Slides
4/17
Impulse Response Function (IRF)
Plot j vs. j
Cumulative impact (up to horizon j)
jXi=1
j
Long-run cumulative impact
Xi=1
j = (1)
= (L) evaluated at L = 1
Stability and Stationarity Conditions
If || < 1 then
limj
j = limj
j = 0
and the stationary solution (Wold form) for the AR(1)
becomes.
Yt =X
j=0
jtj =X
j=0
jtj
This is a stable (non-explosive) solution. Note that
(1) =X
j=0
j =1
1 <
If = 1 then
Yt = Y0 +
tXj=0
j, j = 1, (1) =
which is not stationary or stable.
-
7/30/2019 Stationary Time Series Slides
5/17
AR(1) in Lag Operator Notation
(1 L)Yt = tIf || < 1 then
(1 L)1 = Xj=0
jLj = 1 + L + 2L2 +
such that
(1 L)1(1 L) = 1Trick to find Wold form:
Yt = (1 L)1(1 L)Yt = (1 L)1t=
Xj=0
jLjt
=X
j=0
jtj
= X
j=0jtj, j = j
Moments of Stationary AR(1)
Mean adjusted form:
Yt
= (Yt
1
) + t, t
W N(0, 2), || < 1
Regression form:
Yt = c + Yt1 + t, c = (1 )Trick for calculating moments: use stationarity properties
E[Yt] = E[Ytj] for all j
cov(Yt, Ytj) = cov(Ytk, Ytkj) for all k, j
Mean of AR(1)
E[Yt] = c + E[Yt1] + E[t]= c + E[Yt]
E[Yt] =c
1 =
-
7/30/2019 Stationary Time Series Slides
6/17
Variance of AR(1)
0 = var(Yt) = E[(Yt )2] = E[((Yt1 ) + t)2]= 2E[(Yt1 )2] + 2E[(Yt1 )t] + E[2t ]= 20 +
2 (by stationarity)
0 =2
1 2Note: From the Wold representation
0 = var
X
j=0
jtj
= 2
Xj=0
2j =2
1 2
Autocovariances and Autocorrelations
Trick: multiply Yt by Ytj and take expectations
j = E[(Yt
)(Yt
j
)]
= E[(Yt1 )(Ytj )] + E[t(Ytj )]= j1 (by stationarity)
j = j0 = j2
1 2Autocorrelations:
j = j0=
j
00
= j = j
Note: for the AR(1), j = j. However, this is not true
for general ARMA processes.
Autocorrelation Function (ACF)
plot j vs. j
-
7/30/2019 Stationary Time Series Slides
7/17
half-life
0.99 68.970.9 6.580.75 2.410.5 1.000.25 0.50
Table 1: Half lives for AR(1)
Half-Life of AR(1): lag at which IRF decreases by one
half
j = j = 0.5
j ln = ln(0.5) j = ln(0.5)
ln
The half-life is a measure of the speed of mean reversion.
Application: Half-Life of Real Exchange Rates
The real exchange rate is defined as
zt = st pt + ptst = log nominal exchange rate
pt = log of domestic price level
pt = log of foreign price level
Purchasing power parity (PPP) suggests that zt should
be stationary.
-
7/30/2019 Stationary Time Series Slides
8/17
ARMA(p, 0) Model
Mean-adjusted form:
Yt = 1(Yt
1 ) + + p(Ytp ) + tt W N(0, 2)
E[Yt] =
Lag operator notation:
(L)(Yt ) = t(L) = 1
1L
pL
p
Unobserved Components representation
Yt = + Xt
(L)Xt = t
Regression Model formulation
Yt = c + 1Yt1 + + pYtp + t(L)Yt = c + t, c = (1)
Stability and Stationarity Conditions
Trick: Write pth order SDE as a 1st order vector SDE
Xt
Xt1Xt2
...Xtp+1
=
1 2 p
1 0 00 1 . . . 0... ... . . . ...0 0 1 0
Xt1
Xt2Xt3
...Xtp
+
t
00...0
or
t(p1)
= F(pp)
t1(p1)
+ vt(p1)
Use insights from AR(1) to study behavior of VAR(1):
t+j = Fj+1t1 + F
jvt + + Fvt+j1 + vt
Fj = F F F (j times)
Intuition: Stability and stationarity requires
limjFj = 0
Initial value has no impact on eventual level of series.
-
7/30/2019 Stationary Time Series Slides
9/17
Example: AR(2)
Xt = 1Xt1 + 2Xt2 + t
or
" XtXt1
#="
1
21 0# " Xt1Xt2 # + "
t0#
Iterating j periods out gives"Xt+j
Xt+j1
#=
"1 21 0
#j+1 "Xt1Xt2
#
+ " 1 21 0
#j
" t0# + + " 1 2
1 0# " t+j10 # + "
t+j
0
First row gives Xt+j
Xt+j = [f(j+1)11 Xt1 + f
(j+112 Xt2] + f
(j)11 t
+ + f11t+j1 + t+jf
(j)11 = (1, 1) element ofF
j
Note:
F2 =
"1 21 0
# "1 21 0
#=
"21 + 2 12
1 2
#
Result: The ARMA(p, 0) model is covariance stationary
and has Wold representation
Yt = +X
j=0
jtj, 0 = 1
with j = (1, 1) element ofFj provided all of the eigen-
values ofF have modulus less than 1.
Finding Eigenvalues
is an eigenvalue ofF and x is an eigenvector if
Fx = x (F Ip)x = 0 F Ip is singular det(F Ip) = 0
Example: AR(2)
det(F I2) = det1 21 0 ! "
00 #!
= det
1 2
1
!
= 2 1 2
-
7/30/2019 Stationary Time Series Slides
10/17
The eigenvalues ofF solve the reverse characteristic
equation
2 1 2 = 0Using the quadratic equation, the roots satisfy
i =1
q21 + 42
2, i = 1, 2
These root may be real or complex. Complex roots induce
periodic behavior in yt. Recall, if i is complex then
i = a + bi
a = R cos(), b = R sin()
R =q
a2 + b2 = modulus
To see why |i| < 1 implies limjFj = 0 considerthe AR(2) with real-valued eigenvalues. By the spectral
decomposition theorem
F = TT1, T1 = T0
="
1 00 2
#, T =
"t11 t12t21 t22
#,
T1 =
"t11 t12
t21 t22
#
Then
Fj = (TT
1) (TT
1)
= TjT1
and
limj
Fj= T lim
j
jT1 = 0
provided |1| < 1 and |2| < 1.
-
7/30/2019 Stationary Time Series Slides
11/17
Note:
Fj = TjT1
=
"t11 t12t21 t22
# "
j1 0
0 j2
# "t11 t12
t21 t22
#
so that
f(j)11 = (t11t
11)j1 + (t12t
22)j2
= c1j1 + c2
j2 = j
where
c1 + c2 = 1
Then,
limj
j = limj
(c1j1 + c2
j2) = 0
Examples of AR(2) Processes
Yt = 0.6Yt1 + 0.2Yt2 + t1 + 2 = 0.8 < 1
F = " 0.6 0.21 0
#The eigenvalues are found using
i =
q21 + 42
2
1 =0.6 +
q(0.6)2 + 4(0.2)
2
= 0.84
2 =0.6
q(0.6)2 + 4(0.2)
2= 0.24
j = c1(0.84)j + c2(0.24)j
-
7/30/2019 Stationary Time Series Slides
12/17
Yt = 0.5Yt1 0.8Yt2 + t1 + 2 = 0.3 < 1
F = " 0.5 0.81 0
#Note:
21 + 42 = (0.5)2 4(0.8) = 2.95
complex eigenvaluesThen
i = a bi, i =1
a =12
, b =
q(21 + 42)
2
Here
a =0.5
2= 0.25, b =
2.95
2= 0.86
i = 0.25 0.86i
modulus = R = qa2 + b2 = q(0.25)2 + (0.86)2 = 0.895Polar co-ordinate representation:
i = a + bi s.t. a = R cos(), b = R sin()
= R cos() + R sin()i = R ei
Frequency satisfies
cos() = aR = cos1
aR
period =2
-
7/30/2019 Stationary Time Series Slides
13/17
Here,
R = 0.895
= cos1
0.25
0.985
= 1.29
period =
2
1.29 = 4.9Note: the period is the length of time required for the
process to repeat a full cycle.
Note: The IRF has the form
j = c1
j
1 + c2
j
2 Rj[cos(j) + sin(j)]
Stationarity Conditions on Lag Polynomial (L)
Consider the AR(2) model in lag operator notation
(1 1L 2L2)Xt = (L)Xt = tFor any variable z, consider the characteristic equation
(z) = 1 1z 2z2 = 0By the fundamental theorem of algebra
1 1z 2z2 = (1 1z)(1 2z)
so that
z1 =1
1, z2 =
1
2
are the roots of the characteristic equation. The values
1 and 2 are the eigenvalues ofF.
Note: If1+2 = 1 then (z = 1) = 1(1+2) = 0and z = 1 is a root of (z) = 0.
-
7/30/2019 Stationary Time Series Slides
14/17
Result: The inverses of the roots of the characteristic
equation
(z) = 1 1z 2z2 pzp = 0are the eigenvalues of the companion matrix F. There-
fore, the AR(p) model is stable and stationary provided
the roots of (z) = 0 have modulus greater than unity
(roots lie outside the complex unit circle).
Remarks:
1. The reverse characteristic equation for the AR(p) is
zp 1zp1 2zp2 p1z p = 0This is the same polynomial equation used to find the
eigenvalues ofF.
2. If the AR(p) is stationary, then
(L) = 1 1L pLp
= (1 1L)(1 2L) (1 pL)where |i| < 1. Suppose, all i are all real. Then
(1 iL)1 =X
j=0
ji L
j
(L)1 = (1 1L)1 (1 pL)1
=
Xj=0
j1L
j
Xj=0
j2L
j
. . .
Xj=0
j2L
j
The Wold solution for Xt may be found using
Xt = (L)1t
=
X
j=0
j1L
j
X
j=0
j2L
j
. . .
X
j=0
j2L
j
t
-
7/30/2019 Stationary Time Series Slides
15/17
3. A simple algorithm exists to determine the Wold form.
To illustrate, consider the AR(2) model. By definition
(L)1 = (1 1L 2L2) = (L),
(L) =
Xj=0
jLj
1 = (L)(L) 1 = (1 1L 2L2)
(1 + 1L + 2L2 + )
Collecting coefficients of powers of L gives
1 = 1 + (1 + 1)L + (2 11 2)L2 + Since all coefficients on powers of L must be equal to
zero, it follows that
1 = 1
2 = 11 + 2
3 = 12 + 21...
j = 1j1 + 2j2
Moments of Stationary AR(p) Model
Yt = 1(Yt1 ) + + p(Ytp ) + tt W N(0, 2)
or
Yt = c + 1Yt1 + + pYtp + tc = (1 ) = 1 + 2 + + p
Note: if = 1 then (1) = 1 = 0 and z = 1 isa root of (z) = 0. In this case we say that the AR(p)
process has a unit root and the process is nonstationary.
-
7/30/2019 Stationary Time Series Slides
16/17
Straightforward algebra shows that
E[Yt] =
0 = 11 + 22 + + pp + 2
j = 1j
1 + 2j
2 + + pj
p
j = 1j1 + 2j2 + + pjp
The recursive equations for j are called the Yule-Walker
equations.
Result: (0, 1, . . . , p1) is determined from the firstp elements of the first column of the (p2 p2) matrix
2[Ip2 (F F)]1
where F is the state space companion matrix for the
AR(p) model.
ARMA(0,1) Process (MA(1) Process)
Yt = + t + t1 = + (L)t(L) = 1 + L, t W N(0, 2)
Moments:
E[Yt] =
var(Yt) = 0 = E[(Yt )2]= E[(t + t1)2]= 2(1 + 2)
1 = E[(Yt
)(Yt
1
)]
= E[(t + t1)(t + t1)]= 2
1 =10
=
1 + 2
j = 0, j > 1
-
7/30/2019 Stationary Time Series Slides
17/17
Remark: There is an identification problem for
0.5 < 1 < 0.5The values and 1 produce the same value of 1. Forexample, = 0.5 and 1 = 2 both produce 1 = 0.4.
Invertibility Condition: The MA(1) is invertible if || < 1
Result: Invertible MA models have infinite order AR rep-
resentations
(Yt ) = (1 + L)t, || < 1= (1 L)t, = (1 L)1(Yt ) = t
Xj=0
()jLj(Yt ) = t
so that
t = (Yt) + (Yt1) + ()2 (Yt2) +