Univariate Time seriesMethods of Economic InvestigationLecture 18

Last TimeMaximum Likelihood EstimatorsLikelihood function: useful for estimating non-linear functionsVarious test statisticsLR has nice properties and W/LM converge to LR in large samplesProvides a clear way to test between models

Todays ClassWhy might you need to test between models?Error term or dependent variable doing something unconventionalCorrelation across observations Usually this occurs in time series data

Data StructuresRemember back to the beginning of the term: two types of data:Uni-dimensionalAcross individualsAcross timeMulti-dimensionalPanel data (repeated observations on same individual over time)Repeated Cross-section (observations in various time periods on different individuals

Time Series Processtime series as set of repeated observations of the same variable. A good example of this is GNP or a stock returns. Define {x1, x2, . . . xT } = {xt}, for the time periods t = 1, 2, . . . T. xt is a random variableUnder the usual conditions we could estimate yt = xt + t, E(t | xt) = 0, and OLS provides a consistent estimate of .

Data Generating ProcessesWe did maximum likelihood alreadyThink of the true data generating process as generating different streams of potential data setsWe only observe one of these data setsKey insight: we want to understand how our data relates to the true data generating process so that we can ensure any estimation doesnt omit key variables in the DGP

Strict (or Strong) StationarityA process {xt} is strictly (or strongly) stationary if the joint probability distribution function of {xts,,xt,, xt+s} is independent of t for all s.

Notice that strict stationarity requires that f(xT) is the same regardless of the starting/ending point of your series. This is saying that the data generating process is constant over time.

Weak StationarityA process xt is weakly stationary or covariance stationary if E(xt), E(xt2) are finite and E(xt,xtj) depends only on j and not on t.Weak stationarity is a different concept. It requires finite and well defined first and second moments, which are defined only by the distance between to observations, not the absolute time when those observations exist.

Comparing types of stationaritystrict stationarity does not imply weak stationarityCauchy process which does not have moments. A process could be drawn from an iid Cauchy distribution and therefore be strongly stationary but not weakly stationary. Contant data generating process that is constant over time with finite first and second moments, then strong stationarity will imply weak stationarity. Weak stationarity does not imply strong stationarity. most distributions are characterized by more parameters than the mean and variance and thus weak stationarity allows that other moments depend on t. The special case is a normal distribution which is fully characterized by its mean and variance and thus weak stationarity plus normality will imply strong stationarity.

Why we like stationarityStationarity is a useful concept when thinking about estimation because it suggests that something is fixed across the sequence of random variables. This means if we could observe a process for long enough, we can learn something about the underlying data generating process.

ErgodicityThis is actually not sufficient though for our purposes. To ensure that this long enough is finite, we add the additional condition of ergodicity. Ergodicity is a condition that restricts the memory of the process. It can be defined in a variety of ways. A loose definition of ergodicity is that the process is asymptotically independent. That is, for sufficiently large T, xt and xt+n are nearly independent.

Lag operatorThe lag operator, L, takes one whole time series {xt} and produces another; the second time series is the same as the first, but moved backwards in date. L2xt = LLxt = Lxt1 = xt2 Lj xt = xtj ; Lj xt = xt+j . It will be useful also to note that we can define lag polynomials so thata(L)xt = (a0L0 + a1L1 + a2L2)xt = a0xt + a1xt1 + a2xt2.

Convergence to True MomentsA stationary, ergodic process is that with finite persistence, we can estimate our standard sample moments and as the sample goes to infinity, these sample moments will converge in the usual way to the true moments.

This is typically called the ergodic theorem

White NoiseA white noise process is just like our random error terms. random variable t. ~ i.i.d. N(0, 2 ). E(t) = E(t |all information at t 1) = 0. This simply means that seeing all the past s will not help predict what the current will be. If all processes were like we wouldnt need to worry about persistence or serial correlation but in practice, of course, processes like are rare.

Moving Average or MA(q) ProcessesMoving average processes are ones in which the dependent variable is a function of past realizations of the error We can write this in either polynomial or lag notation as:xt = t + 1t-1 + + qt-q OR xt = (1+1L + . . . qLq)t OR xt = b(L)tEx MA(1):xt = t + t-1 OR xt = (1+L)tSometimes, we define infinite moving averages, which we denote MA():

Absolutely SummableFor MA process to be well defined we require that its parameter is absolutely summable or:

A necessary condition for the js to be absolutely summable is that which is like the ergodicity condition For technical reasons we will actually require square summability, which is that

AR(p) Autoregressive processes are ones in which the dependent variable is function of previous periods realizations.For example, GDP today is a function of past GDP levels. We can write this in either polynomial or lag-operator notation as:xt =1 xt-1 + 2 xt-2 + + p xt-p + t (1 + 1L + 2L2 + . . . + pLp)xt = t a(L)xt = tEx AR(1): xt = xt-1 + t OR (1 L)xt = t

Invertibility We typically require that AR(p) processes be invertible. To understand this, lets return to our lag polynomialsConsider a specific examplext = 1xt1 + 2xt2 + t or (1 1L 2L2)xt = tfactor the quadratic lag polynomial with 1 and 2 such that(1 1L 2L2) = (1 1L)(1 2L). This implies that 12 = 2 and 1 + 2 = 1.

Invertibility - 2Now we need to invert: (1 1L)(1 2L)xt = twe get: xt = (1 1L)1(1 2L)1tOr

for our lag polynomial to be invertible To see why, notice xt = (1 1L)1(1 2L)1t would not be well defined. we call a process invertible if the roots of its parameter values are less than 1 in absolute value. This is convenient because it will also imply stationarity.

Autocovariance Generating Functionsthe autocovariance generating function (AGF) of series xt is defined as j = cov(xt, xtj) =E(xtxtj) this mapping of time to covariance relies on the fact that that the covariance depends on the time in between two xs and not on the absolute date t. This should sound familiarit is our stationarity property and invertibility is a sufficient condition for a stationary series.

Example: MA(q)MA(q) process, xt = (1+1L + . . . qLq)t Var(t) = 2 for all t, ThenLE(xt) = E[ 0t + 1t-1 + + qt-q]= = 0E(t) + 1E(t-1) + + qE(t-q)= 0 . In general of course, this could equal some mean , which we have for ease set to zero. Var(xt) = E[(xt )2] = E[(0t + 1t-1 + qt-q)2] = (1 + 12 + + q2)2 1 = Cov(xt,xt-1) = E(xt,xt-1) = E[(0t ++qt-q)(0t-1+1t-2 ++qt-q-1)] = (10 + + q-1q)2

Next TimeWhy are ARMA processes usefulWold Decomposition

Estimating Stationary Univariate Time SeriesGLSModel Selection

*