Yule-Walker Equations and Moving Average Models

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Outline

1 §7.1: Yule-Walker Equations

2 §3.2 Moving Average Processes

3 Homework 2c

Arthur Berg Yule-Walker Equations and Moving Average Models 2/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Yule-Walker Equations

Start with the mean zero AR(p) model:

Zt = φ1Zt−1 + φ2Zt−2 + · · ·+ φpZt−p + at (?)

Multiply both sides of (?) by Zt−h for h = 1, . . . , p:

ZtZt−h = φ1Zt−1Zt−h + φ2Zt−2Zt−h + · · ·+ φpZt−pZt−h + atZt−h (??)

Take expectations throughout:

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1)

Now take the expectation of (??) with h = 0:

γ(0) = φ1γ(1) + φ2γ(2) + · · ·+ φpγ(p) + E(Ztat)︸ ︷︷ ︸σ2

(? ? ?)

Rearranging (? ? ?) givesσ2 = γ(0)− φ1γ(1)− · · · − φpγ(p) (2)

Equations (1) and (2) are the Yule-Walker Equations.Arthur Berg Yule-Walker Equations and Moving Average Models 3/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Yule-Walker Equations

Start with the mean zero AR(p) model:

Zt = φ1Zt−1 + φ2Zt−2 + · · ·+ φpZt−p + at (?)

Multiply both sides of (?) by Zt−h for h = 1, . . . , p:

ZtZt−h = φ1Zt−1Zt−h + φ2Zt−2Zt−h + · · ·+ φpZt−pZt−h + atZt−h (??)

Take expectations throughout:

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1)

Now take the expectation of (??) with h = 0:

γ(0) = φ1γ(1) + φ2γ(2) + · · ·+ φpγ(p) + E(Ztat)︸ ︷︷ ︸σ2

(? ? ?)

Rearranging (? ? ?) givesσ2 = γ(0)− φ1γ(1)− · · · − φpγ(p) (2)

Equations (1) and (2) are the Yule-Walker Equations.Arthur Berg Yule-Walker Equations and Moving Average Models 3/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Yule-Walker Equations

Start with the mean zero AR(p) model:

Zt = φ1Zt−1 + φ2Zt−2 + · · ·+ φpZt−p + at (?)

Multiply both sides of (?) by Zt−h for h = 1, . . . , p:

ZtZt−h = φ1Zt−1Zt−h + φ2Zt−2Zt−h + · · ·+ φpZt−pZt−h + atZt−h (??)

Take expectations throughout:

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1)

Now take the expectation of (??) with h = 0:

γ(0) = φ1γ(1) + φ2γ(2) + · · ·+ φpγ(p) + E(Ztat)︸ ︷︷ ︸σ2

(? ? ?)

Rearranging (? ? ?) givesσ2 = γ(0)− φ1γ(1)− · · · − φpγ(p) (2)

Equations (1) and (2) are the Yule-Walker Equations.Arthur Berg Yule-Walker Equations and Moving Average Models 3/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Yule-Walker Equations

Start with the mean zero AR(p) model:

Zt = φ1Zt−1 + φ2Zt−2 + · · ·+ φpZt−p + at (?)

Multiply both sides of (?) by Zt−h for h = 1, . . . , p:

ZtZt−h = φ1Zt−1Zt−h + φ2Zt−2Zt−h + · · ·+ φpZt−pZt−h + atZt−h (??)

Take expectations throughout:

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1)

Now take the expectation of (??) with h = 0:

γ(0) = φ1γ(1) + φ2γ(2) + · · ·+ φpγ(p) + E(Ztat)︸ ︷︷ ︸σ2

(? ? ?)

Rearranging (? ? ?) givesσ2 = γ(0)− φ1γ(1)− · · · − φpγ(p) (2)

Equations (1) and (2) are the Yule-Walker Equations.Arthur Berg Yule-Walker Equations and Moving Average Models 3/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Yule-Walker Equations

Start with the mean zero AR(p) model:

Zt = φ1Zt−1 + φ2Zt−2 + · · ·+ φpZt−p + at (?)

Multiply both sides of (?) by Zt−h for h = 1, . . . , p:

ZtZt−h = φ1Zt−1Zt−h + φ2Zt−2Zt−h + · · ·+ φpZt−pZt−h + atZt−h (??)

Take expectations throughout:

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1)

Now take the expectation of (??) with h = 0:

γ(0) = φ1γ(1) + φ2γ(2) + · · ·+ φpγ(p) + E(Ztat)︸ ︷︷ ︸σ2

(? ? ?)

Rearranging (? ? ?) givesσ2 = γ(0)− φ1γ(1)− · · · − φpγ(p) (2)

Equations (1) and (2) are the Yule-Walker Equations.Arthur Berg Yule-Walker Equations and Moving Average Models 3/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Yule-Walker Equations

Start with the mean zero AR(p) model:

Zt = φ1Zt−1 + φ2Zt−2 + · · ·+ φpZt−p + at (?)

Multiply both sides of (?) by Zt−h for h = 1, . . . , p:

ZtZt−h = φ1Zt−1Zt−h + φ2Zt−2Zt−h + · · ·+ φpZt−pZt−h + atZt−h (??)

Take expectations throughout:

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1)

Now take the expectation of (??) with h = 0:

γ(0) = φ1γ(1) + φ2γ(2) + · · ·+ φpγ(p) + E(Ztat)︸ ︷︷ ︸σ2

(? ? ?)

Rearranging (? ? ?) givesσ2 = γ(0)− φ1γ(1)− · · · − φpγ(p) (2)

Equations (1) and (2) are the Yule-Walker Equations.Arthur Berg Yule-Walker Equations and Moving Average Models 3/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Y-W Equations in Matrix Form

From the recurrence

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1),

extract the p equationsγ(1) = φ1γ(0) + φ2γ(1) + · · ·+ φpγ(p− 1)γ(2) = φ1γ(1) + φ2γ(0) + · · ·+ φpγ(p− 2)

...γ(p) = φ1γ(p− 1) + φ2γ(p− 2) + · · ·+ φpγ(0)

γ(1)γ(2)

...γ(p)

︸ ︷︷ ︸

γp

=

γ(0) γ(1) · · · γ(p− 1)γ(1) γ(0) · · · γ(p− 2)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .γ(p− 1) γ(p− 2) · · · γ(0)

︸ ︷︷ ︸

Γp

φ1φ2...φp

︸ ︷︷ ︸

φ

Hence Γpφ = γp where Γp = {γ(k − j)}pj,k=1.

Arthur Berg Yule-Walker Equations and Moving Average Models 4/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Y-W Equations in Matrix Form

From the recurrence

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1),

extract the p equationsγ(1) = φ1γ(0) + φ2γ(1) + · · ·+ φpγ(p− 1)γ(2) = φ1γ(1) + φ2γ(0) + · · ·+ φpγ(p− 2)

...γ(p) = φ1γ(p− 1) + φ2γ(p− 2) + · · ·+ φpγ(0)

γ(1)γ(2)

...γ(p)

︸ ︷︷ ︸

γp

=

γ(0) γ(1) · · · γ(p− 1)γ(1) γ(0) · · · γ(p− 2)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .γ(p− 1) γ(p− 2) · · · γ(0)

︸ ︷︷ ︸

Γp

φ1φ2...φp

︸ ︷︷ ︸

φ

Hence Γpφ = γp where Γp = {γ(k − j)}pj,k=1.

Arthur Berg Yule-Walker Equations and Moving Average Models 4/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Y-W Equations in Matrix Form

From the recurrence

γ(h) = φ1γ(h− 1) + φ2γ(h− 2) + · · ·+ φpγ(h− p) (1),

extract the p equationsγ(1) = φ1γ(0) + φ2γ(1) + · · ·+ φpγ(p− 1)γ(2) = φ1γ(1) + φ2γ(0) + · · ·+ φpγ(p− 2)

...γ(p) = φ1γ(p− 1) + φ2γ(p− 2) + · · ·+ φpγ(0)

γ(1)γ(2)

...γ(p)

︸ ︷︷ ︸

γp

=

γ(0) γ(1) · · · γ(p− 1)γ(1) γ(0) · · · γ(p− 2)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .γ(p− 1) γ(p− 2) · · · γ(0)

︸ ︷︷ ︸

Γp

φ1φ2...φp

︸ ︷︷ ︸

φ

Hence Γpφ = γp where Γp = {γ(k − j)}pj,k=1.

Arthur Berg Yule-Walker Equations and Moving Average Models 4/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Estimation Using the Y-W Equations

Under the method of moments approach, we estimate φ = (φ1, φ2, . . . , φp) with

φ̂ = Γ̂p−1

γ̂p

and also

σ̂2 = γ̂(0)− φ̂1γ̂(1)− φ̂2γ̂(2)− · · · − φ̂pγ̂(p)

= γ̂(0)− φ̂′γ̂p

= γ̂(0)−(

Γ̂p−1

γ̂p

)′γ̂p

= γ̂(0)− γ̂p′Γ̂p−1

γ̂p

where Γ̂p = {γ̂(k − j)}pj,k=1 and γ̂p = (γ̂(1), . . . , γ̂(p))′.

Arthur Berg Yule-Walker Equations and Moving Average Models 5/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Estimation Using the Y-W Equations

Under the method of moments approach, we estimate φ = (φ1, φ2, . . . , φp) with

φ̂ = Γ̂p−1

γ̂p

and also

σ̂2 = γ̂(0)− φ̂1γ̂(1)− φ̂2γ̂(2)− · · · − φ̂pγ̂(p)

= γ̂(0)− φ̂′γ̂p

= γ̂(0)−(

Γ̂p−1

γ̂p

)′γ̂p

= γ̂(0)− γ̂p′Γ̂p−1

γ̂p

where Γ̂p = {γ̂(k − j)}pj,k=1 and γ̂p = (γ̂(1), . . . , γ̂(p))′.

Arthur Berg Yule-Walker Equations and Moving Average Models 5/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Outline

1 §7.1: Yule-Walker Equations

2 §3.2 Moving Average Processes

3 Homework 2c

Arthur Berg Yule-Walker Equations and Moving Average Models 6/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Moving Average Model — MA(q)

Definition (Moving average model — MA(q))The moving average model of order q is defined to be

Zt = µ+ at + θ1at−1 + θ2at−2 + · · ·+ θqat−q

where θ1, θ2, . . . θq are parameters in R.

The above model can be compactly written as

Zt = µ+ θ(B)at

where θ(B) is the moving average operator.

Definition (Moving Average Operator)The moving average operator is

θ(B) = 1 + θ1B + θ2B2 + · · ·+ θqBq

Arthur Berg Yule-Walker Equations and Moving Average Models 7/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Moving Average Model — MA(q)

Definition (Moving average model — MA(q))The moving average model of order q is defined to be

Zt = µ+ at + θ1at−1 + θ2at−2 + · · ·+ θqat−q

where θ1, θ2, . . . θq are parameters in R.

The above model can be compactly written as

Zt = µ+ θ(B)at

where θ(B) is the moving average operator.

Definition (Moving Average Operator)The moving average operator is

θ(B) = 1 + θ1B + θ2B2 + · · ·+ θqBq

Arthur Berg Yule-Walker Equations and Moving Average Models 7/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Moving Average Model — MA(q)

Definition (Moving average model — MA(q))The moving average model of order q is defined to be

Zt = µ+ at + θ1at−1 + θ2at−2 + · · ·+ θqat−q

where θ1, θ2, . . . θq are parameters in R.

The above model can be compactly written as

Zt = µ+ θ(B)at

where θ(B) is the moving average operator.

Definition (Moving Average Operator)The moving average operator is

θ(B) = 1 + θ1B + θ2B2 + · · ·+ θqBq

Arthur Berg Yule-Walker Equations and Moving Average Models 7/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Outline

1 §7.1: Yule-Walker Equations

2 §3.2 Moving Average Processes

3 Homework 2c

Arthur Berg Yule-Walker Equations and Moving Average Models 8/ 9

§7.1: Yule-Walker Equations §3.2 Moving Average Processes Homework 2c

Homework 2c

Read §3.3 and §3.4.Do exercise #3.1(a,b) and #3.5.

Arthur Berg Yule-Walker Equations and Moving Average Models 9/ 9