spectral density estimation via autoregressive modeling
TRANSCRIPT
Spectral Density Estimation via Autoregressive Modeling
Marie Forbelska
Masaryk University Brno, Department of Mathematics and Statistics
Podlesı, 3. 9. – 6. 9. 2013
Introduction Characteristics of Time Series
Characteristics of Time Series
Time Series (Discrete–Time Stochastic Processes)
A time series is a sequence of random variables{Yt , t = 0,±1,±2, . . .}
Beveridge Wheat Price Index, 1500-1869
n = 370
1500 1550 1600 1650 1700 1750 1800 1850
0
50
100
150
200
250
300
350
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 2 / 83
Introduction Weak Stationarity
Second Order Statistical Description
Definition
Stochastic process {Yt , t ∈ T} is said to be a second-order process ifEY 2
t <∞ for all t ∈ T .
Mean
Mean EYt = µt <∞ for all t ∈ T .
Autocovariance and Autocorelation
Autocovariance CY (t, s) of a random process {Yt , t ∈ Z} is definedas the covariance of Yt and Ys :
CY (t, s) = E (Yt − EYt)(Ys − EYs)
In particular, when t = s, we haveCY (t, t) = E (Yt − EYt)
2 = DYt
Autocorrelation coefficient is defined asRY (t, s) = CY (t,s)√
DYtDYs
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 3 / 83
Introduction Weak Stationarity
Weak Stationarity
We introduce weak stationarity which require that time series exhibitcertain time-invariant behavior.
Definition
A time series {Yt , t ∈ Z} is (weak) stationary if EYt <∞ for each t,and
(i) EYt = µ is a constant, independent of t, and(ii) CY (t, t + k) is independent of t for each k .
Notation
If {Yt , t ∈ Z} is (weak) stationary denote byγY (k) = CY (t, t + k)ρY (k) = RY (t, t + k)
for all t.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 4 / 83
Introduction Weak Stationarity
Spectral theory
Spectral density
Let {Yt , t ∈ Z} be a zero mean stationary random sequence with theautocovariance function satisfying
∞∑t=−∞
|γ(t)| <∞.
Then the spectral density function is the continuous function f (λ) givenby the uniformly convergent series
f (λ) =∞∑
t=−∞γ(t)e−iλt
(see Doob 1953, p. 476).
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 5 / 83
Introduction Weak Stationarity
White Noise
Definition
The process {εt , t ∈ T} is said to be an White Noise
if εt are uncorrelated random variables,
each with zero mean and variance σ2ε > 0
Notation: εt ∼WN(0, σ2ε).
Definition
If εt are also independent and identically distributed, then the process{εt , t ∈ T} is said to be an IID process.
Notation: εt ∼ IID(0, σ2ε).
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 6 / 83
Introduction Weak Stationarity
Gaussian White Noise
εt = ηt − µ ∼ WN(0, σ2ε)
where ηt ∼ N(µ = 1, σ2 = 1)
density: fη(x) = 1√2πσ2
exp {−12
(x−µ)2
σ2 } pro x ∈ R
mean: Eηt = µ
variance: Dηt = σ2 = σ2ε
−2 0 2 4
0.0
0.1
0.2
0.3
0.4
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−3
−2
−1
01
23
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 7 / 83
Introduction Weak Stationarity
Exponential White Noise
εt = ηt − µ ∼ WN(0, σ2ε)
where ηt ∼ Exp(µ = 1)
density: fη(x) = 1µ exp {− 1
µx} pro x ≥ 0
mean: Eηt = µ
variance: Dηt = µ2 = σ2ε
0 2 4 6
0.0
0.2
0.4
0.6
0.8
1.0
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
0 50 100 150 200 250 300
−1
01
23
45
6
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 8 / 83
Introduction Weak Stationarity
Beta-distributed White Noise
εt = ηt − µ ∼ WN(0, σ2ε)
where ηt ∼ Beta(a = 1.25, b = 1.25)
density: fη(x) = Γ(a+b)Γ(a)Γ(b) xa−1(1− x)b−1 pro x ∈ (0, 1)
mean: Eηt = µ = aa+b
variance: Dηt = ab(a+b)2(a+b+1)
= σ2ε
−0.4 −0.2 0.0 0.2 0.4
0.0
0.2
0.4
0.6
0.8
1.0
1.2
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−0.
4−
0.2
0.0
0.2
0.4
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 9 / 83
Introduction Weak Stationarity
ARMA Process
Definition
The process {Yt , t ∈ Z} is said to be an ARMA(p, q) process
if {Yt , t ∈ Z} is stationary and
if for every t ∈ Z,
Yt − ϕ1Yt−1 − · · · − ϕpYt−p = εt + θ1εt−1 + · · ·+ θqεt−q
where εt ∼WN(0, σ2ε).
We say that {Yt , t ∈ Z} is an ARMA(p, q) process with mean µ
if {Yt − µ, t ∈ Z} is an ARMA(p, q) process.
Special cases
If p = 0 then Yt is said to be moving average process MA(q).
If q = 0 then Yt is said to be autoregressive AR(p).
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 10 / 83
Introduction Weak Stationarity
Backshift Operators and Characteristic Polynomials
Backshift Operator B such that
BYt = BYt−1 and BkYt = BYt−k for all k ∈ Z
ARMA notation using backshift operators
Yt ∼ ARMA(p, q) : Φ(B)Yt = Θ(B)εt
Characteristic polynomials
AR part Φ(z) = 1− ϕ1z − · · · − ϕpzp
MA part Θ(z) = 1 + θ1z + · · ·+ θqzp
defined on |z | < 1.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 11 / 83
AR(2) : Yt = 0.5Yt−1 + 0.2Yt−2 + εt , εt ∼ N(0, 1)
0 50 100 150 200 250 300−4
−2
0
2
4
MA(2) : Yt = εt − 0.5εt−1 − 0.2εt−1, εt ∼ N(0, 1)
0 50 100 150 200 250 300−4
−2
0
2
4
ARMA(2, 2) : Yt = 0.5Yt−1 + 0.2Yt−2 + εt − 0.4εt−1 + 0.3εt−1, εt ∼ N(0, 1)
0 50 100 150 200 250 300−4
−2
0
2
4
6
Introduction Weak Stationarity
Causality and Invertibility of ARMA Processes
Definition
An ARMA(p, q) process is said to be causal (relative to {εt}) if there
exists a sequence of constants {ψi} such that∞∑i=0
|ψi | <∞ and
Yt =∞∑i=0
ψiεt−i , t ∈ Z
Which is equivalent to the condition
Φ(z) = 1− ϕ1z − . . .− ϕpzp 6= 0, ∀ |z | < 1
A similar definition for the invertibility of an ARMA(p, q) process relativeto εt can be presented if we interchange the role of {Yt} with {εt}. Thenthe invertibility is equivalent to the condition
Θ(z) = 1 + θ1z + . . .+ θpzq 6= 0, ∀ |z | < 1
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 13 / 83
Introduction Weak Stationarity
Spectral density of ARMA process
Spectral density of a MA(q) process
fY (ω) = σ2ε
2π
∣∣Θ (e−iω)∣∣2 for ω ∈ 〈−π, π〉
Spectral density of a AR(p) process
fY (ω) = σ2ε
2π1
|Φ(e−iω)|2for ω ∈ 〈−π, π〉
Spectral density of a ARMA(p, q) process
fY (ω) = σ2ε
2π|Θ(e−iω)|2|Φ(e−iω)|2
for ω ∈ 〈−π, π〉
where
Θ(z) = 1 + θ1z + . . .+ θqzq and Φ(z) = 1− ϕ1z − . . .− ϕpzp.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 14 / 83
Introduction Weak Stationarity
Moments of the AR(p) process
To calculate the mean we need causal AR(p) process:
EYt = E∑∞
j=0 ψjεt−j =∑∞
j=0 ψjEεt−j = 0.
Calculation of the autocovariance function is complicated:first equation Yt = ϕ1Yt−1 + · · ·+ ϕpYt−p + εtmultiplied by a term Yt−k and calculate the mean values of both sides, i.e.
EYtYt−k︸ ︷︷ ︸=γ(k)
= ϕ1 EYt−1Yt−k︸ ︷︷ ︸=γ(k−1)
+ · · ·+ ϕp EYt−pYt−k︸ ︷︷ ︸=γ(k−p)
+EεtYt−k .
then we compute
EYt−kεt = E (∞∑j=0
ψjεt−j−k)εt =∞∑j=0
ψjEεt−j−kεt =∞∑j=0
ψjσ2εδj+k
=
{σ2ε k = 0,
0 otherwise
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 15 / 83
Introduction Weak Stationarity
Yule–Walker equations
By simple modifications of the previous equations we get Yule–Walkerequations
for the autocovariance function
for k = 0: γ(0) − ϕ1γ(1) − · · ·− ϕpγ(p) = σ2ε
for k 6= 0: γ(k) − ϕ1γ(k−1) − · · ·− ϕpγ(k−p) = 0
for the autocorrelation function
for k = 0: ρ(0)︸︷︷︸=1
− ϕ1ρ(1) − · · ·− ϕpρ(p) = σ2ε
γ(0)
for k 6= 0: ρ(k) − ϕ1ρ(k−1) − · · ·− ϕpρ(k−p) = 0 (YW∗)
Yule–Walker equation is a widely used method to estimate the coefficientsof the AR(p) models.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 16 / 83
Introduction Weak Stationarity
Limit properties of the ρ(k) of the AR(p) process
Solution of the homogeneous differential equation, which we marked witha (YW∗), we get in addition to recurrent relationship too explicit form ofthe autocorrelation function
ρAR(p)(k) =m∑j=1
(pj−1∑s=0
cjsks
)λkj =
m∑j=1
(pj−1∑s=0
cjsks
)rkj e ikθj ,
where cjs are constants determined by the initial conditions and λj = rjeiθj
are the inverse of the roots of the Φ(z) = 1− ϕ1z − . . .− ϕpzp withmultiplicities pj . Because holds
|λj | = rj < 1, kde Φ(z0j) = 0 pro z0j = 1λj,
we get here, that ρ(k) decreases for k →∞ exponentially to zero, i.e.
ρ(k) −−−→k→∞
0,
which is a very important property identification autoregressive AR(p)processes.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 17 / 83
Introduction Weak Stationarity
AR(1) : Yt = 0.5Yt−1 + εt , εt ∼ N(0, 1)
0 50 100 150 200 250 300
−4
−3
−2
−1
0
1
2
3
4
ACF
−10 −8 −6 −4 −2 0 2 4 6 8 10
0
0.2
0.4
0.6
0.8
1fAR(ω) = σ2
ε2π
1|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0.1
0.2
0.3
0.4
0.5
0.6
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 18 / 83
Introduction Weak Stationarity
AR(1) : Yt = −0.5Yt−1 + εt , εt ∼ N(0, 1)
0 50 100 150 200 250 300
−4
−3
−2
−1
0
1
2
3
4
ACF
−10 −8 −6 −4 −2 0 2 4 6 8 10
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
fAR(ω) = σ2ε
2π1
|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0.1
0.2
0.3
0.4
0.5
0.6
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 19 / 83
Introduction Weak Stationarity
AR(1) : Yt = 0.95Yt−1 + εt , εt ∼ N(0, 1)
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●●●
●
●●
●
●
●●
●
●●
●
●
●●●
●●
●
●
●
●●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●●●
●
●
●
●
●●
●
●
●●
●
●●●
●
●●●
●●●
●●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●●
●●●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
0 50 100 150 200 250 300
−4
−2
0
2
4
6
8
ACF
−30 −20 −10 0 10 20 30
0.2
0.4
0.6
0.8
1.0
fAR(ω) = σ2ε
2π1
|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0
10
20
30
40
50
60
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 20 / 83
Introduction Weak Stationarity
AR(1) : Yt = −0.95Yt−1 + εt , εt ∼ N(0, 1)
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−6
−4
−2
0
2
4
6
ACF
−30 −20 −10 0 10 20 30
−1.0
−0.5
0.0
0.5
1.0
fAR(ω) = σ2ε
2π1
|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0
10
20
30
40
50
60
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 21 / 83
Introduction Weak Stationarity
AR(2) : Yt = −0.75Yt−1 − 0.75Yt−2 + εt , εt ∼ N(0, 1)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−4
−2
0
2
4
ACF
−10 −5 0 5 10
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
fAR(ω) = σ2ε
2π1
|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 22 / 83
Introduction Weak Stationarity
MA(2) : Yt = εt − 0.2279εt−1 + 0.2488εt−2, εt ∼ N(0, 1)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300−3
−2
−1
0
1
2
3
ACF
−30 −20 −10 0 10 20 30
−0.2
0.0
0.2
0.4
0.6
0.8
1.0fMA(ω) = σ2
ε2π
∣∣Θ (e−iω)∣∣2
−3 −2 −1 0 1 2 3
0.10
0.15
0.20
0.25
0.30
0.35
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 23 / 83
Introduction Weak Stationarity
MA(4) : Yt = εt + 0.8εt−1 + 0.2εt−4, εt ∼ N(0, 1)
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
0 50 100 150 200 250 300
−2
0
2
4
ACF
−30 −20 −10 0 10 20 30
0.0
0.2
0.4
0.6
0.8
1.0fMA(ω) = σ2
ε2π
∣∣Θ (e−iω)∣∣2
−3 −2 −1 0 1 2 3
0.0
0.1
0.2
0.3
0.4
0.5
0.6
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 24 / 83
Introduction Weak Stationarity
MA(12) : Yt = εt + 0.8εt−1 + 0.2εt−12, εt ∼ N(0, 1)
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−4
−2
0
2
4
ACF
−30 −20 −10 0 10 20 30
0.0
0.2
0.4
0.6
0.8
1.0fMA(ω) = σ2
ε2π
∣∣Θ (e−iω)∣∣2
−3 −2 −1 0 1 2 3
0.0
0.1
0.2
0.3
0.4
0.5
0.6
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 25 / 83
Introduction Weak Stationarity
ARMA(1, 1) : Yt = 0.5Yt−1 + εt + 0.5εt−1, εt ∼ N(0, 1)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
0 50 100 150 200 250 300
−2
0
2
4
ACF
−30 −20 −10 0 10 20 30
0.0
0.2
0.4
0.6
0.8
1.0fARMA(ω) = σ2
ε2π|Θ(e−iω)|2|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 26 / 83
Introduction Weak Stationarity
ARMA(2, 1) : Yt = 0.2Yt−1 + 0.7Yt−1 + εt + 0.5εt−1, εt ∼ N(0, 1)
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
0 50 100 150 200 250 300
−3
−2
−1
0
1
2
3
ACF
−30 −20 −10 0 10 20 30
0.0
0.2
0.4
0.6
0.8
1.0fARMA(ω) = σ2
ε2π|Θ(e−iω)|2|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0
1
2
3
4
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 27 / 83
Introduction Weak Stationarity
ARMA(1, 2) : Yt = −0.75Yt−1 + εt − εt−1 + 0.25εt−2, εt ∼ N(0, 1)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−5
0
5
ACF
−30 −20 −10 0 10 20 30
−0.5
0.0
0.5
1.0fARMA(ω) = σ2
ε2π|Θ(e−iω)|2|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0
2
4
6
8
10
12
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 28 / 83
Introduction Weak Stationarity
ARMA(1, 3) : Yt = −0.6Yt−1 + εt − 0.7εt−1 + 0.4εt−2 + 0.4εt−3, εt ∼ N(0, 1)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−6
−4
−2
0
2
4
ACF
−30 −20 −10 0 10 20 30
−0.5
0.0
0.5
1.0fARMA(ω) = σ2
ε2π|Θ(e−iω)|2|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0.0
0.5
1.0
1.5
2.0
2.5
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 29 / 83
Introduction Weak Stationarity
ARMA(2, 2) : Yt = 0.8897Yt−1 − 0, 4858Yt−2 + εt − 0.2279εt−1 + 0.2488εt−2, εt ∼ N(0, 1)
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−3
−2
−1
0
1
2
3
4
ACF
−30 −20 −10 0 10 20 30
−0.2
0.0
0.2
0.4
0.6
0.8
1.0fARMA(ω) = σ2
ε2π|Θ(e−iω)|2|Φ(e−iω)|2
−3 −2 −1 0 1 2 3
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 30 / 83
Introduction Stationary seasonal linear models
Seasonal linear models
So far we have discussed the links between neighboring random variables
. . . ,Yt ,Yt+1,Yt+2, . . . .
If a random process also includes seasonal fluctuations, it is necessary tonotice the dependencies between random variables, which divides seasonlength L.
. . . ,Yt ,Yt+L,Yt+2L, . . . .
First, we introduce seasonal differential operator of length L > 0:∆LYt =Yt − Yt−L = (1− BL)Yt
∆2LYt =∆L(∆LYt) = ∆L(Yt−Yt−L)
=(Yt−Yt−L)−(Yt−L−Yt−2L)=Yt−2Yt−L+Yt−2L = (1−BL)2Yt
...∆D
L Yt =(1− BL)DYt
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 31 / 83
Introduction Stationary seasonal linear models
Construction seasonal models
To better understand the structure of seasonal patterns in the B–Jmethodology, divide, for example, monthly data (L = 12) for r yearsin the following table.
Year January February · · · December
1 Y1 Y2 · · · Y12
2 Y13 Y14 · · · Y24...
......
......
r Y1+12(r−1) Y2+12(r−1) · · · Y12+12(r−1)
For each column j ∈ {1, . . . , 12} separately consider a ARMA(P,Q)model of the same type:
Yj+12t = π1Yj+12(t−1) + · · ·+ πPYj+12(t−1)+
ηj+12t + ψ1ηj+12(t−1) + · · ·+ ψQηj+12(t−1)
Because all 12 random processes is of the same type, we can writeπ(B12)Yt = Ψ(B12)ηt .
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 32 / 83
Introduction Stationary seasonal linear models
Remap white noise to the new process
When 12 white noise of the same type {η1+12t} ∼ WN(0, σ2η)
{η2+12t} ∼ WN(0, σ2η)
......
...{η12+12t} ∼ WN(0, σ2
η)
sequentially assemble in time and create a single random process
{η∗t , t = 0,±1,±2, . . .},
we do not get white noise, it is recalled that:
Eη∗t η∗t+h = 0 only where h that are multiples of 12
Eη∗t ηt+h 6= 0 may occur for any other h,
therefore model the process ηt as a general ARMA(pq) process
Φ(B)η∗t = Θ(B)εt , εt ∼WN(0, σ2ε).
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 33 / 83
Introduction Stationary seasonal linear models
Stationary SARMA models
General stationary seasonal mixed SARMA model:
Φ(B)π(BL)Yt = Θ(B)Ψ(BL)εt ∼ SARMA(p, q)× (P,Q)L
kde
Φ(B) = 1− ϕ1B − · · · − ϕpBp
π(BL) = 1− π1BL − · · · − πPBPL
Θ(B) = 1 + θ1B + · · ·+ θqBp
Ψ(BL) = 1 + ψ1BL + · · ·+ ψQBQL
MA homogeneous seasonal models
Yt = Θ(B)Ψ(BL)εt ∼ SARMA(0, q)× (0,Q)L.
AR homogeneous seasonal models
Φ(B)π(BL)Yt = εt ∼ SARMA(p, 0)× (P, 0)L
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 34 / 83
Introduction Stationary seasonal linear models
SARMA model as a special type of ARMA model
Consider a simple example SARMA(1, 0)× (1, 0)12 model:
Φ(B)π(B12)Yt = εt
(1− ϕ1B)(1− π1B12)Yt = εt
(1− ϕ1B − π1B12 + ϕ1π1B13)Yt = εt
Yt − ϕ1Yt−1 − π1Yt−12 + ϕ1π1Yt−13 = εt
We see that it is a special case of AR(13) model in which:
10 coefficients are zero,three remaining non-zero coefficients were created on two parameters:.
Relationship between SARMA and ARMA models
Model SARMA(p, q)× (P,Q)L is actually ARMA(p + PL,QL + q) modelwith additional conditions on AR and MA coefficients.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 35 / 83
Introduction Stationary seasonal linear models
Pure seasonal homogeneous model with MA parts: Yt = Ψ(B12)εt
SARMA(0, 0)(0, 1)12 : Yt = (1− 0.95B12)εt : Yt = εt − 0.95εt−12, εt ∼ N(0, 1)
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−4
−2
0
2
ACF
−60 −40 −20 0 20 40 60
−0.5
0.0
0.5
1.0
fSMA(ω) = σ2ε
2π
∣∣Ψ (e−i12ω)∣∣2
−3 −2 −1 0 1 2 3
0.0
0.1
0.2
0.3
0.4
0.5
0.6
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 36 / 83
Introduction Stationary seasonal linear models
Pure seasonal homogeneous model with AR parts: π(B12)Yt = εt
SARMA(0, 0)(0, 1)12 : (1− 0.95B12)Yt = εt : Yt = 0.95Yt−12 + εt , εt ∼ N(0, 1)
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
0 50 100 150 200 250 300
−5
0
5
ACF
−60 −40 −20 0 20 40 60
0.0
0.2
0.4
0.6
0.8
1.0
fSAR(ω) = σ2ε
2π1
|π(e−i12ω)|2
−3 −2 −1 0 1 2 3
0
10
20
30
40
50
60
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 37 / 83
Introduction Stationary seasonal linear models
Pure seasonal homogeneous model with AR parts: π(B12)Yt = εt
SARMA(0, 0)(2, 0)12 : (1− 0.3B12 + 0.1B24)Yt = εt : Yt = 0.3Yt−12 − 0.1Yt−24 + εt , εt ∼ N(0, 1)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
0 50 100 150 200 250 300
−3
−2
−1
0
1
2
3
ACF
−60 −40 −20 0 20 40 60
0.0
0.2
0.4
0.6
0.8
1.0
fSAR(ω) = σ2ε
2π1
|π(e−i12ω)|2
−3 −2 −1 0 1 2 3
0.10
0.15
0.20
0.25
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 38 / 83
Introduction Stationary seasonal linear models
Seasonal homogeneous model with MA parts: Yt = Θ(B)Ψ(B12)εt
SARMA(0, 1)(0, 1)12 : Yt = (1 + 0.9B)(1− 0.4B12)εt : Yt = εt + 0.9εt−1 − 0.4εt−12 − 0.36εt−13, εt ∼ N(0, 1)
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300−4
−2
0
2
4
ACF
−60 −40 −20 0 20 40 60
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
fSMA(ω) = σ2ε
2π
∣∣Θ (e−iω)∣∣2∣∣Ψ (e−i12ω)∣∣2
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 39 / 83
Introduction Stationary seasonal linear models
Seasonal homogeneous model with MA parts: Yt = Θ(B)Ψ(B12)εt
SARMA(0, 1)(0, 1)12 : Yt = (1 + 0.9B)(1 + 0.4B12)εt : Yt = εt + 0.9εt−1 + 0.4εt−12 + 0.36εt−13, εt ∼ N(0, 1)
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
0 50 100 150 200 250 300
−4
−2
0
2
4
ACF
−60 −40 −20 0 20 40 60
0.0
0.2
0.4
0.6
0.8
1.0
fSMA(ω) = σ2ε
2π
∣∣Θ (e−iω)∣∣2∣∣Ψ (e−i12ω)∣∣2
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 40 / 83
Introduction Stationary seasonal linear models
Seasonal homogeneous model with AR parts: Φ(B)π(B12)Yt = εt
SARMA(1, 0)(1, 0)12 : (1− 0.5B)(1− 0.7B12)Yt = εt : Yt = 0.5Yt−1 + 0.7Yt−12 − 0.35Yt−13 + εt , εt ∼ N(0, 1)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 50 100 150 200 250 300
−2
0
2
4
ACF
−60 −40 −20 0 20 40 60
0.0
0.2
0.4
0.6
0.8
1.0
fSAR(ω) = σ2ε
2π1
|Φ(e−iω)|21
|π(e−i12ω)|2
−3 −2 −1 0 1 2 3
0
1
2
3
4
5
6
7
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 41 / 83
Introduction Stationary seasonal linear models
Seasonal homogeneous model with AR parts: Φ(B)π(B12)Yt = εt
SARMA(1, 0)(1, 0)12 : (1− 0.9B)(1− 0.7B12)Yt = εt : Yt = 0.9Yt−1 + 0.7Yt−12 − 0.63Yt−13 + εt , εt ∼ N(0, 1)
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●●
●
●
●
●●●
●
●●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●●●
●
●
●●●
●
●
●●
●●
●
●●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
0 50 100 150 200 250 300
−5
0
5
ACF
−60 −40 −20 0 20 40 60
0.2
0.4
0.6
0.8
1.0
fSAR(ω) = σ2ε
2π1
|Φ(e−iω)|21
|π(e−i12ω)|2
−3 −2 −1 0 1 2 3
0
50
100
150
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 42 / 83
Introduction Stationary seasonal linear models
Seasonal mixed model: Φ(B)π(B12)Yt = Θ(B)Ψ(B12)εt
SARMA(1, 1)(1, 1)12 : (1− 0.5B)(1− 0.7B12)Yt = (1 + 0.9B)(1− 0.4B12)εtYt = 0.5Yt−1 + 0.7Yt−12 − 0.35Yt−13 + εt + 0.9εt−1 − 0.4εt−12 − 0.36εt−13, εt ∼ N(0, 1)
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
0 50 100 150 200 250 300
−4
−2
0
2
4
ACF
−60 −40 −20 0 20 40 60
0.0
0.2
0.4
0.6
0.8
1.0
fSARMA(ω) = σ2ε
2π|Θ(e−iω)|2|Φ(e−iω)|2
|Ψ(e−i12ω)|2|π(e−i12ω)|2
−3 −2 −1 0 1 2 3
0
2
4
6
8
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 43 / 83
Introduction Stationary seasonal linear models
Seasonal mixed model: Φ(B)π(B12)Yt = Θ(B)Ψ(B12)εt
SARMA(1, 1)(1, 1)12 : (1− 0.5B)(1− 0.7B12)Yt = (1 + 0.9B)(1 + 0.4B12)εtYt = 0.5Yt−1 + 0.7Yt−12 − 0.35Yt−13 + εt + 0.9εt−1 + 0.4εt−12 + 0.36εt−13, εt ∼ N(0, 1)
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
0 50 100 150 200 250 300
−5
0
5
ACF
−60 −40 −20 0 20 40 60
0.0
0.2
0.4
0.6
0.8
1.0
fSARMA(ω) = σ2ε
2π|Θ(e−iω)|2|Φ(e−iω)|2
|Ψ(e−i12ω)|2|π(e−i12ω)|2
−3 −2 −1 0 1 2 3
0
10
20
30
40
50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 44 / 83
Introduction Stationary seasonal linear models
Seasonal mixed model: Φ(B)π(B12)Yt = Θ(B)Ψ(B12)εt
SARMA(1, 1)(1, 1)12 : (1− 0.9B)(1− 0.7B12)Yt = (1 + 0.9B)(1− 0.4B12)εtYt = 0.9Yt−1 + 0.7Yt−12 − 0.63Yt−13 + εt + 0.9εt−1 − 0.4εt−12 − 0.36εt−13, εt ∼ N(0, 1)
●
●
●
●●●●●
●●●
●●
●
●
●●
●
●
●●
●●●
●
●
●●●
●●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●●
●●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●●●●●
●
●
●●●
●●●
●
●
●
●●
●
●●●
●●
●●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●●
●●
●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●●●
●●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●●
●●●●
●●●●
●
●
0 50 100 150 200 250 300
−10
−5
0
5
10
ACF
−60 −40 −20 0 20 40 60
0.2
0.4
0.6
0.8
1.0
fSARMA(ω) = σ2ε
2π|Θ(e−iω)|2|Φ(e−iω)|2
|Ψ(e−i12ω)|2|π(e−i12ω)|2
−3 −2 −1 0 1 2 3
0
50
100
150
200
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 45 / 83
Introduction Stationary seasonal linear models
Seasonal mixed model: Φ(B)π(B12)Yt = Θ(B)Ψ(B12)εt
SARMA(1, 1)(1, 1)12 : (1− 0.9B)(1− 0.7B12)Yt = (1 + 0.9B)(1 + 0.4B12)εtYt = 0.9Yt−1 + 0.7Yt−12 − 0.635Yt−13 + εt + 0.9εt−1 + 0.4εt−12 + 0.36εt−13, εt ∼ N(0, 1)
●
●
●
●
●●
●●
●
●●●
●●
●
●●●
●
●●
●●●
●
●
●
●●●
●●●
●●
●
●
●
●
●●
●●
●●
●●
●
●●
●●
●
●●●
●
●●●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●●●●●
●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●●●
●
●●
●
●●
●●
●●
●
●
●●
●●●
●
●●
●
●●●
●
●●
●●●
●●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●●
●
●●●
●
●
●
●
●
●
●●
●
●●●
●●●
●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
0 50 100 150 200 250 300
−20
−10
0
10
20
ACF
−60 −40 −20 0 20 40 60
0.4
0.6
0.8
1.0
fSARMA(ω) = σ2ε
2π|Θ(e−iω)|2|Φ(e−iω)|2
|Ψ(e−i12ω)|2|π(e−i12ω)|2
−3 −2 −1 0 1 2 3
0
200
400
600
800
1000
1200
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 46 / 83
Estimation of moments
Estimation of moments
Let Y = (Y1, . . . ,Yn)T be a time series observed at equally-spaced timepoints t1, . . . , tn. We consider the problem of using these data to forecastYn+1 at time tn+1.
-�∆
t1 t2 · · ·
-�∆
ti ti+1 · · ·
-�∆
tn−1 tn
Denote by ∆ = ti+1 − ti . Thenti = t1 + (i − 1)∆ for i = 2, . . . , ni = ti−t1
∆ + 1
Without loss of generality, we can therefore assume that ti = i .
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 47 / 83
Estimation of moments
Estimation of the second order moments
Suppose we have data Y1, . . . ,Yn from a stationary time series. We canestimate
Empirical Mean Estimator
Y = 1n
n∑t=1
Yt
Empirical Autocovariance Function Estimator
Ck = γ(k) = 1n−k
n−k∑t=1
(Yt − Y )(Yt+k − Y ) for k = 0, 1, . . . , n − 1
Empirical Autocorrelation Function Estimator
ρ(k) = γ(k)γ(0)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 48 / 83
Estimation of moments
Example with ACF estimates for AR(2)
AR(2) : (1− 1.5B + 0.75B2)Yt = εt : Yt = 1.5Yt−1 − 0.75Yt−2 + εt , εt ∼ N(0, 1)
0 100 200 300 400 500
−5
05
10
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●●
●
●●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●●
●●
●
●
●●
●
●●●
●●
●
●
●
●●
●
●
●●
●
●●
●●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●●
●●
●
●
●●●
●
●●●●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●●●
●
●
●●
●●
●
●●
●
●●
●●
●●●
●
●
●●●
●●
●●
●●●●
●●●
●●
●●
●●●
●
●●
●
●●●●●
●
●
●
●
●
●
●●●
●●
●
●
●●●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●
●●
●●
●
●
●●
●
●
●●●
●●
●
●●
●
●●
●●●●
●●●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●●
●●
●
●●
●
●
●●
●●●
●
●●
●
●●
●
●●●●●
●
●●●
●
●
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 49 / 83
Estimation of moments
Example with ACF estimates for AR(2)
Empirical estimate for ACFn = 100
0 5 10 15 20 25 30
−0.5
0.0
0.5
1.0
Empirical estimate for ACFn = 200
0 5 10 15 20 25 30
−0.5
0.0
0.5
1.0
red . . . theoretical values
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 50 / 83
Estimation of moments
Example with ACF estimates for AR(2) (cont.)
Empirical estimate for ACFn = 300
0 5 10 15 20 25 30
−0.5
0.0
0.5
1.0
Empirical estimate for ACFn = 500
0 5 10 15 20 25 30
−0.5
0.0
0.5
1.0
red . . . theoretical values
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 51 / 83
Estimation of moments
Example with ACF estimates for AR(2) (cont.)
Empirical estimate for ACFn = 1000
0 5 10 15 20 25 30
−0.5
0.0
0.5
1.0
Empirical estimate for ACFn = 5000
0 5 10 15 20 25 30
−0.5
0.0
0.5
1.0
red . . . theoretical values
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 52 / 83
Estimation of moments
Monte Carlo study for the 1000 replication
Empirical estimate for ACF●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●
●
●●
●●
●●●
●●●
●
●●●
●●●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●●● ●
● ●●●●●
●●
●
●●
●
●
●●●
●
●●
●
●
●●
●
●
●
● ●
●
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
−0.
50.
00.
51.
0 n = 100●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ●●
●●
● ● ● ●● ● ● ●
Empirical estimate for ACF●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●
●
●●
●●
● ●
●
●●
●●
●
●●●
●●
●
●
●
●
●
●
●●●
●
●●
●
●●●●
●●
●●
●●●
●●●
●●●
●
●●
●●●
●
●●
●
●
●
●
●●
●●●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●●
●
●●●●
●●●●●●
●
●●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●●
●●●
●
●●●●●
●● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●●
●
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
−0.
50.
00.
51.
0 n = 200●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ●●
●●
● ● ● ●● ● ● ●
red . . . theoretical values
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 53 / 83
Estimation of moments
Monte Carlo study for the 1000 replication (cont. 1)
Empirical estimate for ACF●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●●●●●●●●
●
●●●
● ●
●
●
●●
●●●
●
●●
●
●●●●
●
●
●
●
●●●
●●●
●●●●
●●●●
●
●●
●●●●
●
●
●● ●
●
●●●●
●●●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●● ●
●
●
●
●●
●
●
●●
●●
●
●
●●●●
●●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●●●
●
●
●
●●● ●●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
−0.
50.
00.
51.
0 n = 300●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
● ● ● ●● ● ● ●
Empirical estimate for ACF●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●
●●
●
●●●
●
●●
●●●●●
●
●●
●●●●●
●
●
●●●●●
●●●●●●●
●●●
●●●●
●
●
●
●●
●●●●
●●
●
●
●
●
●
●
●●
●●●●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●●
●●●●
●●
●●●
●●●
●
● ●●
●●●
●●
●
●
●●●
●●
●●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●●
●●●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●●●●●●
●
●●●●●●●●●
●●●
●●
●● ●●
●
●●
●●
●
●●
●●
●
●●●
●
●
●●
●
●●
●●
●●
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
−0.
50.
00.
51.
0 n = 500●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
● ● ● ●● ● ● ●
red . . . theoretical values
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 54 / 83
Estimation of moments
Monte Carlo study for the 1000 replication (cont. 2)
Empirical estimate for ACF●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●
●●●
●●
●●
●
●
●
●●●
●
●
●●
●●
●
●●
●●
●
●
●●
●
●●●●
●
●
●
●●●●
●●
●
●
●
●●
●●
●
●●
●●
●●●
● ●
●●
●
●
●
●
●●●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●●●
●
●●
●
●
●
●
●
●
●●
●●●
●●
●
●●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●●●
●
●●●
●
●
●●
●
●
●
●
●●●●● ●
●●●
●
●●
●●
● ●
●
●
●
●
●●●●
●
●●●●
●●
●
●
●
●
●●●
●●
●
●●
●●●●
●
●
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
−0.
50.
00.
51.
0 n = 1000●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
● ● ● ●● ● ● ●
Empirical estimate for ACF●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●
●●●●●●
●
●●
●
●●●●
●
●
●●●
●●
●●
●
●●
●●●
●
●
●●●
●●
●●●
●
●
●
●
●●●
●●
●
●●●●
●●●●
●
●●●
●●
●●●●●
●●
●●
●
●●
●●
●
●
●●●
●
●
●
●●●●
●
●
●
●
● ●
●●●●
●
●
●●●
● ●●●●●
●
●●
●●●
●
●
●
●
●
●●●
●
●●●
●
● ●
●
●●
●●
●
●● ●●
●
●●
●
●
●
● ●
●
●
●
●
●●
●●
●
●●
●
●●●●
●
●
●
●●●●
●●
●
●●●
●●●
●●●
●
●●● ●●●●●●●●●
●
●
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
−0.
50.
00.
51.
0 n = 5000●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
● ● ●●
●● ● ●
red . . . theoretical values
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 55 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Total Mortality (weeklydata)
n = 508
1970 1972 1974 1976 1978 1980
140
160
180
200
220
x
Den
sity
HistogramKernel Density EstimateNormal Density
140 160 180 200 220
Assessment of seasonality
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
140
160
180
200
220
140
160
180
200
220
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ● ●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
1970
1971
1972
1973
1974
1975
1976
1977
1978
0 10 20 30 40 50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 56 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Total Mortality
Spectral Density
0.0 0.5 1.0 1.5 2.0 2.5 3.0
510
2050
100
200
Angular Frequency
Spe
ctra
l Den
sity
Inf
Parametric AR(2) estimateParametric ARMA(2,0) estimateParametric SARMA(3,0)(2,2)[52] estimate
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 57 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Total Mortality●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
Empirical ACFTheoretical ACF for AR(2)Theoretical ACF for ARMA(2,0)Theoretical ACF for SARMA(3,0)(2,2)[52]
0 20 40 60 80 100 120
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
●
●●
●● ●
●● ●
●● ● ● ●
●●
● ●●
●● ●
● ●●
●● ●
● ●●
●●
● ●●
●● ●
●
0 10 20 30 40
4.35
4.40
4.45
4.50
AIC criterion for choice of AR order
●
2
Optimal order of AR = 2
AIC = n(log σ2ε + 1) + 2(p + 1)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 58 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Cardiovascular Mortalityn = 508
1970 1972 1974 1976 1978 1980
70
80
90
100
110
120
130
x
Den
sity
HistogramKernel Density EstimateNormal Density
70 80 90 100 110 120 130
Assessment of seasonality
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
7080
9010
011
012
013
0
7080
9010
011
012
013
0
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
1970
1971
1972
1973
1974
1975
1976
1977
1978
0 10 20 30 40 50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 59 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Cardiovascular MortalitySpectral Density
0.0 0.5 1.0 1.5 2.0 2.5 3.0
25
1020
5010
020
050
010
0020
00
Angular Frequency
Spe
ctra
l Den
sity
52.5
Parametric AR(28) estimateParametric ARMA(2,0) estimateParametric SARMA(2,2)(2,2)[52] estimate
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 60 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Cardiovascular Mortality
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●●●●●●●
●●
●●
●●
●●
●●
Empirical ACFTheoretical ACF for AR(28)Theoretical ACF for ARMA(2,0)Theoretical ACF for SARMA(2,2)(2,2)[52]
0 20 40 60 80 100 120
−0.5
0.0
0.5
1.0
●
●●
●● ●
● ●●
● ●●
●●
●
● ● ● ●●
● ●●
● ●●
●
●● ●
●●
● ●●
●●
●●
●
0 10 20 30 40
3.50
3.55
3.60
3.65
AIC criterion for choice of AR order
●
28
Optimal order of AR = 28
AIC = n(log σ2ε + 1) + 2(p + 1)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 61 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Temperature (weekly data)n = 508
1970 1972 1974 1976 1978 1980
50
60
70
80
90
100
x
Den
sity
HistogramKernel Density EstimateNormal Density
50 60 70 80 90 100
Assessment of seasonality
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
5060
7080
9010
0
5060
7080
9010
0
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●●
●●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
1970
1971
1972
1973
1974
1975
1976
1977
1978
0 10 20 30 40 50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 62 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Temperature
Spectral Density
0.0 0.5 1.0 1.5 2.0 2.5 3.0
510
2050
100
200
500
1000
Angular Frequency
Spe
ctra
l Den
sity
49.9
Parametric AR(22) estimateParametric ARMA(1,2) estimateParametric SARMA(1,1)(2,0)[52] estimate
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 63 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Temperature●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
Empirical ACFTheoretical ACF for AR(22)Theoretical ACF for ARMA(1,2)Theoretical ACF for SARMA(1,1)(2,0)[52]
0 20 40 60 80 100 120
−0.5
0.0
0.5
1.0
●
●
●
●●
● ● ● ●
●
●
● ●● ● ● ● ● ● ● ●
●●
● ● ● ●● ●
● ● ●● ●
● ● ● ● ●●
0 10 20 30 40
3.65
3.70
3.75
3.80
3.85
3.90
3.95
AIC criterion for choice of AR order
●
22
Optimal order of AR = 22
AIC = n(log σ2ε + 1) + 2(p + 1)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 64 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Relative Humidityn = 508
1970 1972 1974 1976 1978 1980
20
40
60
80
x
Den
sity
HistogramKernel Density EstimateNormal Density
20 40 60 80
Assessment of seasonality
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
2040
6080
2040
6080
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1970
1971
1972
1973
1974
1975
1976
1977
1978
0 10 20 30 40 50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 65 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Relative Humidity
Spectral Density
0.0 0.5 1.0 1.5 2.0 2.5 3.0
1020
3040
Angular Frequency
Spe
ctra
l Den
sity
Inf
Parametric AR(4) estimateParametric ARMA(4,3) estimateParametric SARMA(2,5)(1,1)[52] estimate
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 66 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Relative Humidity (weeklydata)
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
Empirical ACFTheoretical ACF for AR(4)Theoretical ACF for ARMA(4,3)Theoretical ACF for SARMA(2,5)(1,1)[52]
0 20 40 60 80 100 120
0.0
0.2
0.4
0.6
0.8
1.0
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
0 10 20 30 40
4.93
4.94
4.95
4.96
4.97
4.98
4.99
AIC criterion for choice of AR order
●
4
Optimal order of AR = 4
AIC = n(log σ2ε + 1) + 2(p + 1)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 67 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Carbon Monoxide (weeklydata)
n = 508
1970 1972 1974 1976 1978 1980
5
10
15
20
x
Den
sity
HistogramKernel Density EstimateNormal Density
5 10 15 20
Assessment of seasonality
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
510
1520
510
1520
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ● ●●
●
●
● ●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
1970
1971
1972
1973
1974
1975
1976
1977
1978
0 10 20 30 40 50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 68 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Carbon Monoxide (weekly data)
Spectral Density
0.0 0.5 1.0 1.5 2.0 2.5 3.0
0.5
1.0
2.0
5.0
10.0
20.0
50.0
100.
020
0.0
Angular Frequency
Spe
ctra
l Den
sity
49.9
Parametric AR(16) estimateParametric ARMA(1,2) estimateParametric SARMA(2,3)(2,0)[52] estimate
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 69 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Carbon Monoxide (weeklydata)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●●●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●●●●●●
●●
●●
●●
●●
●●
●●
●●
●●●●
Empirical ACFTheoretical ACF for AR(16)Theoretical ACF for ARMA(1,2)Theoretical ACF for SARMA(2,3)(2,0)[52]
0 20 40 60 80 100 120
−0.5
0.0
0.5
1.0
●
●
● ● ● ● ●●
● ●
●
● ●● ●
● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●
●
0 10 20 30 401.
71.
81.
92.
02.
1
AIC criterion for choice of AR order
●
16
Optimal order of AR = 16
AIC = n(log σ2ε + 1) + 2(p + 1)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 70 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Hydrocarbonsn = 508
1970 1972 1974 1976 1978 1980
20
40
60
80
100
x
Den
sity
HistogramKernel Density EstimateNormal Density
20 40 60 80 100
Assessment of seasonality
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
2040
6080
100
2040
6080
100
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
● ●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1970
1971
1972
1973
1974
1975
1976
1977
1978
0 10 20 30 40 50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 71 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: HydrocarbonsSpectral Density
0.0 0.5 1.0 1.5 2.0 2.5 3.0
510
2050
100
200
500
1000
2000
Angular Frequency
Spe
ctra
l Den
sity
56.8
Parametric AR(22) estimateParametric ARMA(1,2) estimateParametric SARMA(2,2)(2,2)[52] estimate
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 72 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Hydrocarbons●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●●●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●●●●●●
●●
●●
●●
●
●●
●
●●
●●
●●
●●
●●
●●●●●●●●●●●●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●●●●●●●●●
Empirical ACFTheoretical ACF for AR(22)Theoretical ACF for ARMA(1,2)Theoretical ACF for SARMA(2,2)(2,2)[52]
0 20 40 60 80 100 120
−0.4
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
●
●
●
● ● ● ●●
● ●
●● ●
● ●
●●
● ● ●●
●●
●●
●● ●
●● ● ● ● ●
●●
● ● ●●
0 10 20 30 40
4.80
4.85
4.90
4.95
5.00
5.05
AIC criterion for choice of AR order
●
22
Optimal order of AR = 22
AIC = n(log σ2ε + 1) + 2(p + 1)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 73 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Nitrogen Dioxiden = 508
1970 1972 1974 1976 1978 1980
5
10
15
20
25
x
Den
sity
HistogramKernel Density EstimateNormal Density
5 10 15 20 25
Assessment of seasonality
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
510
1520
25
510
1520
25
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
1970
1971
1972
1973
1974
1975
1976
1977
1978
0 10 20 30 40 50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 74 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Nitrogen Dioxide
Spectral Density
0.0 0.5 1.0 1.5 2.0 2.5 3.0
25
1020
50
Angular Frequency
Spe
ctra
l Den
sity
Inf
Parametric AR(6) estimateParametric ARMA(2,2) estimateParametric SARMA(3,3)(2,0)[52] estimate
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 75 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Nitrogen Dioxide)●
●
●
●
●
●
●
●●
●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
Empirical ACFTheoretical ACF for AR(6)Theoretical ACF for ARMA(2,2)Theoretical ACF for SARMA(3,3)(2,0)[52]
0 20 40 60 80 100 120
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
0 10 20 30 40
2.75
2.76
2.77
2.78
2.79
AIC criterion for choice of AR order
●
6
Optimal order of AR = 6
AIC = n(log σ2ε + 1) + 2(p + 1)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 76 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Nitrogen Dioxiden = 508
1970 1972 1974 1976 1978 1980
5
10
15
20
25
x
Den
sity
HistogramKernel Density EstimateNormal Density
5 10 15 20 25
Assessment of seasonality
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
510
1520
25
510
1520
25
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
1970
1971
1972
1973
1974
1975
1976
1977
1978
0 10 20 30 40 50
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 77 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Nitrogen Dioxide
Spectral Density
0.0 0.5 1.0 1.5 2.0 2.5 3.0
25
1020
50
Angular Frequency
Spe
ctra
l Den
sity
Inf
Parametric AR(6) estimateParametric ARMA(2,2) estimateParametric SARMA(3,3)(2,0)[52] estimate
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 78 / 83
LA Pollution-Mortality Study: Spectral Density
LA Pollution-Mortality Study: Nitrogen Dioxide●
●
●
●
●
●
●
●●
●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
Empirical ACFTheoretical ACF for AR(6)Theoretical ACF for ARMA(2,2)Theoretical ACF for SARMA(3,3)(2,0)[52]
0 20 40 60 80 100 120
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
0 10 20 30 40
2.75
2.76
2.77
2.78
2.79
AIC criterion for choice of AR order
●
6
Optimal order of AR = 6
AIC = n(log σ2ε + 1) + 2(p + 1)
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 79 / 83
LA Pollution-Mortality Study: Spectral Density
Literatura
Andel, J. Statisticka analyza casovych rad . Praha. SNTL 1976.
Altman, N. S. (1990): Kernel Smoothing of Data With CorrelatedErrors. Journal of the American Statistical Association. Vol. 85, No.411, pp. 749–759.
Anderson, T.W. The Statistical Analysis of Time Series. John Wiley& Sons Inc. 1971.
Box, G., Jenkins, G. Time series analysis - forecasting and control .Holden-Day 1976.
Brabanter, K., Brabanter, J., Suykens, J. A. K. (2011): KernelRegression in the Presence of Correlated Errors. Journal of MachineLearning Research 12, pp. 1955–1976.
Brillinger, D. R. (1975): Time Series. Data Analysis and Theory. SanFrancisco: Holden–Day.
Brockwell, P.J., Davis, R.A. Time Series: Theory and Methods.Springer–Verlag, New York, 1991.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 80 / 83
LA Pollution-Mortality Study: Spectral Density
Literatura (pokracovanı)
Brockwell, P.J., Davis, R.A. Introduction to time series andforecasting . Springer-Verlag, New York, 2002.
Cipra, T. Analyza casovych rad s aplikacemi v ekonomii . SNTL,Praha, 1986.
Doob, J.L. Stochastic processes. New York, Wiley 1953.
Damsleth, E., Spjøtvoll, E. Estimation of Trigonometric Componentsin Time Series, J. Amer. Statistics, Assoc. 77. 1982, pp. 382–387.
Daniels, H.E. Rank correlation and population models. Journals ofthe Royal Statistical Society, B, 12 1950. 171–181.
Doob, J.L. Stochastic processes. New York, Wiley 1953.
Fisher, R.A. Tests of significance in harmonic analysis. Proc. RoyalSoc. A 125, 1929, 54–59.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 81 / 83
LA Pollution-Mortality Study: Spectral Density
Literatura (pokracovanı)
Forbelska, M. Detekce periodicity v hydrologickych datech. In XIII.letnı skola bometriky, Biometricke metody a modely v soucasne vedea vyzkumu. 1. vyd. Brno: UKZUZ Brno, 1998. s. 173–178.
Forbelska, M. Stochasticke modelovanı jednorozmernych casovychrad. Munipress, 2009.
Geweke, J.F., Meese, R. Estimating Regression Models of Finite butUnknown Order. International Economic Review, 22, 1981. 55–70.
Gichman, I.I., Skorochod, A.V. Teorija slucajnych processov . Moskva.Nauka 1971.
Hamilton, J.D. Time Series Analysis. Princeton University Press.1994.
Hannan, E.J., Quinn, B. G. The Determination of the Order of anAutoregression, Journal of the Royal Statistical Society, Series B, 41,No.2, 1979, 190–195.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 82 / 83
LA Pollution-Mortality Study: Spectral Density
Literatura (pokracovanı)
Michalek, J., Budıkova, M., Brazdil, R. Metody odhadu trendu casoverady na prıkladu stredoevropskych teplotnıch rad. 1. vyd. Praha :Cesky hydrometeorologicky ustav, 1993, 53 s.
Priestley, M. Spectral analysis and time series. Academic Press 1989.
Stulajter, F. Odhady v nahodnych procesoch. Alfa. Bratislava. 1989.
Vesely, V. Knihovna programu TSA-M pro analyzu casovych rad. Ed.P.Flak. In XIV. letna skola biometriky, Biometricke metody a modelyv podohospodarskej vede, vyskume a vyuke. Nitra: AgenturaSlovenskej akademie podohospodarskych vied, 2000. s. 239–248.
Vesely, V. Uvod do casovych rad. In Proceedings ANALYZADAT’2003/II. Pardubice (Czech Rep.): Trilobyte, Ltd., 2004. od s.7–31.
Zvara, K. Regresnı analyza Praha. Academia. 1989.
Marie Forbelska (MU – UMS) Spectral Density Estimation via AR Modeling Podlesı, 3. 9. – 6. 9. 2013 83 / 83