introduction to time series analysishalweb.uc3m.es/esp/personal/personas/jtena/eng/... ·...

Introduction to Time Series

Analysis

Juan de Dios Tena Horrillo

Office. 10.1.02B

jtena@est-econ.uc3m.es

ductio

time series (2

OUTLINE

Chapter 1. Univariate ARIMA models.

Chapter 2. Model fitting and checking.

Chapter 3. Prediction and model selection.

Chapter 4. Outliers and influential

observations.

Chapter 5. Vector Autoregressive and Vector

Error Correction Models.

Chapter 6. Nonlinear Time Series Modelling.

ductio

time series (2

TEXTBOOKS

Peña, D., Tiao, G.C. and Tsay, R.S. (2000). A Course in Time Series Analysis. Wiley & sons.

Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press.

Box, G.E.P., Jenkins, G.M. and Reinsel, G. (1994). Time Series Analysis: Forecasting and Control, 3rd. Ed. Prentice-Hall, Englewood Cliffs, NJ.

Enders, W. (2004). Applied Econometric Time Series . Wiley & sons.

Lütkepohl, H. and Krätzig, M. (2004). Applied Time Series Econometrics. Cambridge University Press.

Franses, P.H. and van Dick, D. (2000). Non-linear time series models in empirical finance. Cambridge University Press.

ductio

time series (2

GRADING

Final exam (70%).

Class participation and empirical project (30%).

ductio

time series (2

WEB RESOURCES

Global Insight.

Time series data library.

http://datamarket.com/data/list/?q=provider:tsdl

Macroeconomic time series

http://www.fgn.unisg.ch/eurmacro/macrodata/index.html

Instituto Nacional de Estadística

http://www.ine.es

Eurostats

http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home/

ductio

time series (2

MOTIVATION

Time series: sequence of observations taken at

regular intervals of time

Data in bussines, engineering, enviroment,

medicine, etc.

ductio

time series (2

MOTIVATION.

ductio

time series (2

SUICIDES IN SPAIN, TOTAL NUMBER

Source: INE

95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10

SUICIDES

MOTIVATION.

ductio

time series (2

CO2 EMISSIONS IN THE USA

Source: World Databank

2,800,000

3,200,000

3,600,000

4,000,000

4,400,000

4,800,000

5,200,000

5,600,000

6,000,000

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005

MOTIVATION.

ductio

time series (2

INDUSTRIAL PRODUCTION INDEX (SPAIN)

Source: ine

1975 1980 1985 1990 1995 2000 2005 2010

MOTIVATION.

ductio

time series (2

MOTIVATION.

ductio

time series (2

OVERNIGHT STAYS IN SPANISH HOTELS

Source: ine

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

7,000,000

8,000,000

9,000,000

10,000,000

11,000,000

99 00 01 02 03 04 05 06 07 08 09 10 11 12

TURISTS

MOTIVATION.

ductio

time series (2

CONSUMER PRICE INDEX IN GERMANY AND SPAIN

Source: Eurostats

1996 1998 2000 2002 2004 2006 2008 2010 2012

GERMAN_CPI SPANISH_CPI

MOTIVATION.

ductio

time series (2

CONSUMER PRICE INDEX FOR FOOD AND NON-ALCOHOLIC

BEVERAGES IN GERMANY AND SPAIN

Source: Eurostats

1 9 9 6 1 9 9 8 2 0 0 0 2 0 0 2 2 0 0 4 2 0 0 6 2 0 0 8 2 0 1 0 2 0 1 2

S P A N IS H _ F O O D G E R M A N _ F O O D

MOTIVATION.

ductio

time series (2

Source: Instituto Flores de Lemus

2008 2009 2010 2011 2012 2013 2014

YEAR-ON-YEAR RATE OF INFLATION IN SPAIN AND CONTRIBUTIONS OF MAIN COMPONENTS

ENERGÍA ALIMENTOS NO ELABORADOS

INFLACIÓN SUBYACENTE IPC TOTAL

MOTIVATION.

ductio

time series (2

Source: Instituto Flores de Lemus

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

INFLATION IN MADRIDAnnual rates of growth

80% 60% 40% 20%Confidence intervals:

Average inflation(1998-2011): 2,7%

MOTIVATION.

ductio

time series (2

MOTIVATION.

ductio

time series (2

MOTIVATION.

Knowledge of the dynamic structure will help to

produce accurate forecasts of future observations

and design optimal control schemes.

ductio

time series (2

MOTIVATION

Main objectives

Understanding the dynamic structure of the

observations in a single series.

Ascertain the leading, lagging and feedback

relationships among several variables.

Analyzing the impact of other variables that

can be used for policy decisions.

What to do when there are breaks in the

evolution of the series. 19

ductio

time series (2

Chapter 1

Univariate ARIMA Models

Recommended readings: chapters 1 to 3 of Peña et al. (2000)

ductio

time series (2

CHAPTER 1. CONTENTS.

1.1. Introduction.

Definitions, examples,

1.2. Properties of univariate ARMA.

ARMA weights.

Stationarity and covariance structure.

Autocorrelation function.

Partial autocorrelación function.

1.3. Model especification.

1.4. Examples.

ductio

time series (2

INTRODUCTION

ductio

time series (2

Stochastic Process. Real-valued random

variable 𝑍𝑡 that follows a distribution 𝑓𝑡(𝑍𝑡) .

The T-dimensional variable 𝑍𝑡1, 𝑍𝑡2

, … , 𝑍𝑡𝑇

will have a joint distribution that depends on

𝑡1, … , 𝑡𝑇 .

INTRODUCTION

ductio

time series (2

Time series 𝑧𝑡1, 𝑧𝑡2

, … , 𝑧𝑡𝑇 will denote a

particular realization of the stochastic process.

To simplify notation 𝑧1 , 𝑧2 , … , 𝑧𝑇 .

INTRODUCTION

ductio

time series (2

Assumptions

1. The process is stationary.

2. The joint distribution of 𝑧1 , 𝑧2 , … , 𝑧𝑇 is a

multivariate normal distribution.

INTRODUCTION

Implications of stationarity

The joint distribution remains constant over time

It is then true that

ductio

time series (2

013 ) ),...,,(),...,,( 2121 kTkkT zzzfzzzf

;ztEz zt VVz

INTRODUCTION

Stationarity implies a constant mean and bounded

deviations from it.

Strong requirement, few actual economic series will

satisfy it.

ductio

time series (2

INTRODUCTION

ductio

time series (2

INTRODUCTION

ductio

time series (2

INTRODUCTION

ductio

time series (2

INTRODUCTION

Transformations to achieve stationarity

Constant variance: log/level plus outlier correction.

Stationary in mean: differencing.

ductio

time series (2

ductio

time series (2

ductio

time series (2

INTRODUCTION

Differencing

Let B be the backward operator

We shall use the operators:

Regular difference

Seasonal difference

ductio

time series (2

jttj zzB

INTRODUCTION

ductio

time series (2

If 𝑧𝑡 is a deterministic trend (𝑧𝑡 = 𝑎 + 𝑏𝑡)

∇𝑧𝑡 = 𝑏

∇2𝑧𝑡 = 0

∇2𝑧𝑡 = ∇(∇𝑧𝑡)

In general, ∇d will reduce a polynomial of degree d

to a constant.

INTRODUCTION

ductio

time series (2

Example: for a quarterly series 𝑧𝑡 , ∇4𝑧𝑡 = 𝑧𝑡 − 𝑧𝑡−4

will cancel a constant, but it will also cancel other

deterministic periodic functions.

INTRODUCTION

ductio

time series (2

Homogeneous difference equation

∇4𝑧𝑡 = (1 − 𝐵4)𝑧𝑡 = 𝑧𝑡 − 𝑧𝑡−4

Characteristic equation: 𝑟4 − 1 = 0

Solution is given by 𝑟 = 14

(four roots on the unit

circle)

𝑟1 = 1, 𝑟2 = −1, 𝑟3 = 𝑖, 𝑟4 = −𝑖

Two real roots and two complex conjugates with

modulus 1 and frequency 𝜔 = 𝜋/2.

INTRODUCTION

Therefore,

1. One in the zero frequency (trend)

2. One in the twice-a-year seasonality

3. Associated with once-a-year seasonality

ductio

time series (2

)1)(1)(1( 24 BBB

ductio

time series (2

To see this just recall that the complementary function to

the difference equation 𝑧𝑡 = 𝑏𝑧𝑡−1 + 𝑐𝑧𝑡−2 will eventually be

of the form:

𝑧𝑡 = 𝑟𝑡 𝐴 cos 𝜃𝑡 + 𝐵 sin 𝜃𝑡

where 𝑟 is a positive constant and 𝜃 is an angle measured

in radians. 𝐴 and 𝐵 are arbitrary constants to enable the

solution to satisfy any starting point of 𝑧𝑡 .

Hence, when the two solutions of the characteristic

equations are 𝑖 and – 𝑖, sin 𝜃 = 1 and 𝜃 =𝜋

UNIVARIATE ARMA MODELS

“Building block” is the white noise process

Implications:

ductio

time series (2

),0( 2Niidat

0,...),,/()( 321 ttttt aaaaEaE

0),(),( jttjtt aaCovaaE

221 ,...),/(),( atttjtt aaaVaraaVar

INTRODUCTION

ductio

time series (2

General ARMA(p,q) processes:

𝜙 𝐵 𝑧𝑡 = 𝑐 + 𝜃(𝐵)𝑎𝑡

𝑧𝑡 is the observable time series.

𝑎𝑡 is sequence of white noise.

𝑐 is the constant term.

INTRODUCTION

Autoregressive polynomial

Moving-average polynomial

The AR and MA polynomials are assumed to have no

common factor

ductio

time series (2

ppBBBB ...1)( 2

qqBBBB ...1)( 2

INTRODUCTION

ductio

time series (2

Stationarity implies that all zeros of 𝜙 𝐵 are

restricted to lie outside the unit circle and in this case:

𝑐 = 1 − 𝜙1 − ⋯− 𝜙𝑝 𝜇

where 𝜇 is the mean of the series.

Overall behavior of the series remains the same

over time.

INTRODUCTION

ductio

time series (2

Overall behavior of the series remains the same

over time.

In real world, however, time series data often

exhibits a drifting behavior. Nonstationarity can be

modeled allowing some of the zeroes in 𝜙 𝐵 to be

equal to one. Thus,

𝜙 𝐵 (1 − 𝐵𝑑)𝑧𝑡 = 𝑐 + 𝜃(𝐵)𝑎𝑡

This is known as the ARIMA (p,d,q) model.

INTRODUCTION

Some special cases of ARIMA(p,d,q):

ARMA(1,1)

IMA(1,1)

ARIMA(0,1,1)(0,1,1)

ductio

time series (2

INTRODUCTION

ductio

time series (2

AR(1) to MA(∞) by recursive substitution.

𝑧𝑡 = 𝜙𝑧𝑡−1 + 𝑎𝑡

𝑧𝑡 = 𝜙 𝜙𝑧𝑡−2 + 𝑎𝑡−1 + 𝑎𝑡 = 𝜙2𝑧𝑡−2 + 𝜙𝑎𝑡−1 + 𝑎𝑡

𝑧𝑡 = 𝜙𝑘𝑧𝑡−𝑘 + 𝜙𝑘−1𝑎𝑡−𝑘+1 + ⋯ + 𝜙2𝑎𝑡−2 + 𝜙𝑎𝑡−1 + 𝑎𝑡

INTRODUCTION

ductio

time series (2

If 𝜙 < 1, then

𝑧𝑡 = 𝜙𝑗𝑎𝑡−𝑗

𝑗 =0

which is a MA(∞)

INTRODUCTION

Some considerations:

ARMA models are not unique (variety of possible

representations.

Parsimony is always a desirable property.

AR representations are easiest to estimate (OLS) and

to be interpretated.

ductio

time series (2

PROPERTIES

For simplicity, we will assume zero mean and a starting point m. The ARIMA (p,d,q)

Can be rewritten as:

where,

are r=max(p,q) initial values.

ductio

time series (2

tt aBzB )()(

)',....,( tm zzz )',...,( tm aaa

)'0,...,0,,...,( 1 rmm ww

1,..., rmm ww

PROPERTIES

ductio

time series (2

1.........

.........

......

.........

1.........

.........

......

.........

)1)(1( mtmt

PROPERTIES

ARMA weights

Where,

ductio

time series (2

11 DaDDz

PROPERTIES

The -weights can be obtained by equating

coefficients of powers of B from the relations:

ductio

time series (2

)()()( BBB

...1)( 221 BBB

PROPERTIES

The ARIMA (p,d,q) model can then be rewritten in the

MA form as:

ductio

time series (2

hhthtt aaz

PROPERTIES

In the same way,

Where,

ductio

time series (2

aDzDD 11

PROPERTIES

The AR form of the model is then,

ductio

time series (2

).()()( BBB

hhtht azz

PROPERTIES

These two expressions are of fundamental importance in understanding the nature of the model.

The MA form with the weights, shows how the observation is affected by current and past shocks or innovations.

The AR form with the weigths, indicates how the observation is related to its own past values.

ductio

time series (2

PROPERTIES

Examples : AR(1) model

MA representation:

AR representation:

ductio

time series (2

....22

11 tttt aaaz

ttt azz 1

PROPERTIES

Examples : MA(1) model

MA representation:

AR representation:

ductio

time series (2

013 ) tjt

jtt azzz .......1

1 ttt aaz

PROPERTIES

Examples : IMA(1,1) model

MA representation:

AR representation:

ductio

time series (2

ttttt azzzz ...)1()1()1( 32

....))(1( 21 tttt aaaz

STATIONARITY CONDITION

Consider,

Since the innovations are assumed normally

distributed, it follows that the observations are also

normally distributed.

It is seen that, if in the characteristic equation:

ductio

time series (2

hhthtt aaz

kk rArA 0011 ...

(Where the A’s denote polinomials in k and the r’s are

the distinct zeroes of )

Then as we have that

ductio

time series (2

,1jrpoj ,...,1

,0)( tzE

2),cov(h

khhaktt zz

So that will be stationary in this asymptotic sense.

This is the stationarity condition of an ARMA model,

and is equivalent to require that the zeroes of

are lying outside the unit circle.

ductio

time series (2

LAG K AUTOCOVARIANCE

Let us denote,

For an alternative expressions of the autocovariance

function in terms of the ARMA parameters,

ductio

time series (2

)(),cov(),cov()( kzzzzk kttktt

)...( 11 ptpttkt zzzz

)...( 11 qtqttkt aaaz

LAG K AUTOCOVARIANCE

By taking expectations on both sides and using the

MA form, we obtain, for

Where,

ductio

time series (2

hh ghkk

,...2,1,00

AUTOCORRELATION FUNCTION

The autocorrelation function is defined as

By substitution of , we obtain that

ductio

time series (2

,...2,1,0,)0(

)()( k

AUTOCORRELATION FUNCTION

When , in a MA(q) model, then,

That is, the autocorrelation function cuts off after lag

ductio

time series (2

qkk qq

,)...1()(

PARTIAL AUTOCORRELATION FUNCTION

Intuition

AR(1):

AR(2):

ductio

time series (2

tttt zzzz 123

The autocorrelation function only takes into account

that and are related in both cases.

But if we want to measure the direct relationship

(without the intermediate ), we found that it is

zero for the AR(1) and different from zero for the

AR(2) model

ductio

time series (2

tz 2tz

The partial autocorrelation function is, therefore, a

measure of the linear relation among observations k-

periods apart, independently of the intermediate

values.

ductio

time series (2

Consider first an stationary AR(p) model. The

autoregressive coefficients are related to the

autocorrelations by the Yule-Walker equations,

where, is the matrix,

ductio

time series (2

pG )( pxp

ductio

time series (2

1)1(...)2()1(

)1(1)2(

)2(1)1(

)1()2(...)1(1

Regarding this as system of p equations and p

unknowns (the coefficients), the solution is, for p>1,

the ratio of two determinants

ductio

time series (2

ppp GH /

ductio

time series (2

)()1(...)2()1(

)1(1)2(

)2(1)1(

)1()2(...)1(1

ductio

time series (2

•A pxp matrix, and for p=1. This

leads to define, for any stationary model

•Which is known as the Partial Autocorrelation

Function.

ductio

time series (2

It has the property that, for a stationary AR(p)

model,

In other words, vanishes for k>p when the

model is AR(p).

PROPERTIES

ductio

time series (2

SAF PAF

AR(P) slow decay

towards zero

p different

from zero

MA(q) q different

from zero

slow decay

towards zero

ARMA(p,q) Slow decay

towards zero

Slow decay

towards zero

EXAMPLES

AR(1) model

Model:

Variance:

ductio

time series (2

ttt azz 1

2222azz

EXAMPLES

Autocovariance function:

ductio

time series (2

EXAMPLES

Autocorrelation function:

Partial autocorrelation:

ductio

time series (2

)1()1(

EXAMPLES

AR(2) model

Model:

Variance:

ductio

time series (2

tttt azzz 2211

EXAMPLES

ductio

time series (2

22211 kkkk

EXAMPLES

MA(1) model

Model:

Variance:

ductio

time series (2

1 ttt aaz

2222aaz

)1( 222 az

EXAMPLES

Autocovariance function:

ductio

time series (2

EXAMPLES

ductio

time series (2

SPECIFICATION

The class of ARMA(p,q) models is extensive.

Guidelines are needed in selecting a member of the

class to represent the time series data at hand

ductio

time series (2

SPECIFICATION

Box and Jenkins (1976) proposed an iterative model

building strategy.

Tentative specification or identification of a model.

Efficient estimation of model parameters.

Diagnostic checking of fitted model for further

improvement.

ductio

time series (2

SPECIFICATION

Tentative specification. The aim is to employ statistics

1. Can be readily calculated from the

2. Allow the user to tentatively select a

model .

ductio

time series (2

SPECIFICATION

Three methods:

The Sample Autocorrelation Function

The Sample Partial Autocorrelation

Function (SPAF)

Use of diagnostic tools (AIC,BIC) to

find the best model (TRAMO-

automatic).

ductio

time series (2

SPECIFICATION

Sample Autocorrelation Function. The SACF of are defined as

and the sample mean of the n available observations

ductio

time series (2

,...2,1/ˆ 0 kCCkk

zzzzC jt

SPECIFICATION

Properties:

1.For stationary models,

2. When there exists a unit root in the AR polynomial

ductio

time series (2

nkk ̂

SPECIFICATION

If the SACF of the original series is persistently close

to 1 as k increases, one forms the first difference ,

, and studies its SACF to determine whether further

differencing is called for.

ductio

time series (2

SPECIFICATION

Once stationary is achieved, a cutting off pattern after,

say a lag “q”, in the SACF will then lead to tentative

specification of a MA(q) model.

ductio

time series (2

SPECIFICATION

The Sample Partial Autocorrelation Function:

Are obtained by replacing the in

by their sample estimates

ductio

time series (2

,...2,1ˆ kk

SPECIFICATION

Properties:

1.For stationary models,

2. The are asymptotically normally distributed

3. For a stationary AR(p) model

ductio

time series (2

nkk ̂

pknVar k 1)ˆ(

SPECIFICATION

Properties 1, 2 and 3 make SPACF a convenient tool

for specifying the order or a stationary AR model

(cutting off pattern after lag p)

Not valid for non-stationary models.

ductio

time series (2

SPECIFICATION

Weakness of the SACF and SPACF.

1. Subjective judgment is often required to decide on

the order of differencing.

2. For stationary ARMA models, both SACF and

SPACF tend to exhibit a gradual “tapering off”

behavior, making the specification very difficult.

ductio

time series (2

SPECIFICATION

Diagnostic Tools

The program (TRAMO), in an automatic way,

specifies a set of possible models, estimate them and

select the best one based on AIC and BIC criteria.

However an accurate jugment is always necessary to

interprect the output of TRAMO.

ductio

time series (2

introduction to time series analysishalweb.uc3m.es/esp/personal/personas/jtena/eng/... ·...

Documents

tena product catalog - motion specialties · tena ®...

tena koe todd - taupo district council

nzalt, 6 july 2010 keynote address teachers’ lives, loves...

from the principal tena koutou katoa

tena comfort mini - media.allaboutincontinence.co.uk ·...

animals mamífers ainhoa tena

tena koe - childrenlovetosing.com · tena koe tena koe –...

now with subtle wetness indicator tena pants new … ·...

healthy eating (irene lendinez, cintia tena)

tena product catalogue - motion21).pdf · the tena® brand...

the test and training enabling architecture...

version finale tena izy

gene hudgins tena development lead 4 march 2003 test and...

test and training enabling architecture (tena)...

performance testing of the tena middleware

february...if you can help please notify the school office...

test & training enabling architecture (tena) overview ·...

whiringa-ä-nuku / october 2008 “if you are a social...

catalogue of tena virgin hair 2015

dipòsit digital de documents de la uab · 2013. 2. 5. ·...