ch3. trends - missouri state universitypeople.missouristate.edu/songfengzheng/teaching/mth548... ·...

Ch3. TRENDS

Time Series Analysis

Time Series Analysis Ch3. TRENDS

3.1 Deterministic Versus Stochastic Trends

The simulated random walk in Exhibit 2.1 shows a “upwardtrend”. However, it is caused by a strong correlation between theseries at nearby time points. The true process is Yt = Yt−1 + et orYt =

∑ti=1 et , which has zero mean and increasing variance over

time. We call such a trend a stochastic trend.

By comparison, the average monthly temperature series in Exhibit1.7 shows a cyclical trend. We may model it by Yt = µt + Xt ,where µt is a deterministic function with period 12 (µt = µt−12),and Xt is a zero mean unobserved variation (or noise). We saythat this model has a deterministic trend.


3.2 Estimation of a Constant Mean

Consider a model with a constant mean function

Yt = µ+ Xt , E (Xt) = 0.

The most common estimate of µ is the sample mean

Y =1

n

n∑t=1

Yt . Then E (Y ) = µ. So Y is a unbiased estimator

of µ. If {Yt} is stationary with acf ρk , Ex 2.17 shows that

VarY =γ0

n

[1 + 2

n−1∑k=1

(1− k

n

)ρk

]. (1)

For many stationary processes, the acf decays quickly that∞∑k=0

|ρk | <∞. Under this assumption, we have

VarY ≈ γ0n

[ ∞∑k=−∞

ρk

]for large n.


So for many stationary processes {Yt}, the variance of Y isinversely proportional to n. Increasing n will improve theestimation.

Ex. We use (1) to investigate Y :

1 If {Xt} is white noise, then ρk = 0 for k > 0. So VarY =γ0n

.

2 If ρ1 6= 0 but ρk = 0 for k > 1, then VarY → γ0n (1 + 2ρ1) as

n→∞.

3 If {Yt} is a MA(1) model: Yt = et + θet−1, then ρ1 =θ

1 + θ2

and ρk = 0 for k > 1. We have

VarY =γ0n

[1 +

2θ

1 + θ2

(n− 1

n

)]≈ (1 + θ)2

1 + θ2γ0n.

When −1 < θ < 0, say θ = −0.5, VarY ≈ 0.2γ0n

. Comparing

with the white noise, the negative correlation ρ1 improves theestimation significantly.

When 0 < θ < 1, say θ = +0.5, VarY ≈ 1.8γ0n

. The positive

correlation ρ1 reduces the accuracy of estimation.Time Series Analysis Ch3. TRENDS

Ex. For an order one autoregressive process

Yt = φYt−1 + et , −1 < φ < 1,

we can show that ρk = φ|k| for all k. Then

VarY ≈ γ0n

[ ∞∑k=−∞

φ|k|

]=

(1 + φ)

(1− φ)

γ0n.

We plot in R the function(1 + φ)

(1− φ)for −1 < φ < 1:

phi=seq(-.95,.95,0.05)

factor=(1+phi)/(1-phi)

plot(y=factor,x=phi,type=’l’)

abline(h=1,col=’red’)

When φ→ 1−, the variance of Y dramatically increases. In fact,φ = 1 is the nonstationary random walk process Yt = Yt−1 + et .The variance of Y can be very different for a nonstationary process.


3.3 Regression Methods

We use classical regression analysis to estimate the parameters inthe nonconstant mean trend models Yt = µt + Xt , where µt isdeterministic and E(Xt) = 0. These include linear, quadratic,seasonal means, and cosine trends.

3.3.1 Linear and Quadratic Trends: The deterministic trendlinear model is the most widely used model:

µt = β0 + β1t

we call β0 the intercept, and β1 the slope, resp. The classicalleast squares regression is to find β̂0 and β̂1 to minimize

Q(β0, β1) =∞∑t=1

[Yt − (β0 + β1t)]2 .

Use partial derivatives to solve that (formulas are not required):

β̂1 =

∑nt=1(Yt − Y )(t − t)∑n

t=1(t − t)2, β̂0 = Y − β̂1t, where t =

n + 1

2.


Exhibit. SCL in the Capital Asset Pricing Model (CAPM)


The function lm fits a linear model (a regression model) with its first

argument being a formula, i.e., an expression including a tilde sign ( ∼ ),

the left-hand side of which is the response variable and the right-hand

side are the covariates or explanatory variables (separated by plus

signs if there are two or more covariates). The intercept term can be

removed by adding the term “-1” on the right-hand side of the tilde sign.

The covariates must be separated with a plus sign (+).


The function abline is a low-level graphics function. If a fitted simple regression model is passed to it, it adds the

fitted straight line to an existing graph. Any straight line of the form y = β0 + β1x can be superimposed on the

graph by running the command

abline(a=beta0,b=beta1)


3.3.2 Cyclical or Seasonal Trends

Suppose that a seasonal trend, such as the average monthlytemperature data in Exhibit 1.7, can be modeled as

Yt = µt + Xt , E (Xt) = 0 for all t.

One suitable model may be a seasonal mean model, where thereare 12 constants (parameters), β1, β2, · · · , β12, giving the expectedaverage temperature for each of the 12 months, such that

µt =

β1 t = 1, 13, 25, · · ·β2 t = 2, 14, 26, · · ·...

β12 t = 12, 24, 36, · · ·


chap3.R:

Exhibit 3.3 Regression Results for the Seasonal Means Model

Exhibit 3.4 Results for Seasonal Means Model with an Intercept


R: The expression season(tempdub) outputs the monthly index oftempdub as a factor, and saves it into the object month.. A factoris a kind of data structure for handling qualitative (nominal) datathat do not have a natural ordering like numbers do. However, forpurposes of summary and graphics, the user may supply the levelsargument to indicate an ordering among the factor values.See TS-ch3.R


Exhibit. We may analyze a data set “Monthly milk production:pounds per cow. Jan 62 - Dec 75” (Source: Time Series DataLibrary (citing: Cryer (1986))) imported fromhttp://datamarket.com/data/list/?q=provider:tsdl. Save it as“milk.csv”, and analyze by the R codes in TS-ch3.R.See TS-ch3.R

The model milk.m2 is a combination of linear model with seasonalmean model. We may decompose a time series into a sum orproduct of trend, seasonal, and stochastic components by“decompose” command.See TS-ch3.R


http://datamarket.com/data/list/?q=provider:tsdl

3.3.3 Cosine Trends:In some cases, seasonal trends can be modeled economically withcosine curves (that incorporate the smooth change expected fromone time period to the next while still preserving the seasonality.)The simplest such model is Yt = µt + Xt with

µt = β0 + β cos(2πft + Φ) = β0 + β1 cos(2πft) + β2 sin(2πft)

where β1 = β cos Φ and β2 = −β sin Φ. We call β(> 0) theamplitude, f the frequency, and Φ the phase of the curve. 1/fis called the period of the cosine wave.


Exhibit 3.5 Cosine Trend Model for Temperature Series (chap3.R)

Exhibit 3.6 Cosine Trend for the Temperature Series (chap3.R)

The harmonic function will return a matrix of the values ofcos(2πt) and sin(2πt).See TS-ch3.R for some basic commands of matrix


3.4 Reliability and Efficiency of Regression Estimates

Reliability and efficiency of estimates lie on the variances of theestimates. Let us analyze the raliability of estimates for somedeterministic trends.

Assumption: the series is represented as Yt = µt + Xt , where

1 µt is a deterministic trend of the kind linear, seasonal means,or cosine curves, and

2 {Xt} is a zero-mean stationary process with autocovarianceand autocorrelation functions γk and ρk , respectively.

Parameters in a linear model are estimated according to ordinaryleast squares regression.


3.4.1 The seasonal meansFor N complete years of monthly data, the estimate for the meanfor the jth month is

β̂j =1

N

N−1∑i=0

Yj+12i

By (1),

Varβ̂j =γ0N

[1 + 2

N−1∑k=1

(1− k

Nρ12k

)], for j = 1, 2, · · · , 12.

(2)Only the seasonal autocorrelations, ρ12, ρ24, · · · , enter into the

above equation. If {Xt} is white noise, then Varβ̂j =γ0N

.


3.4.2 The cosine trends µt = β0 + β1 cos(2πft) + β2 sin(2πft)For any frequency of the form f = m/n, where m is an integersatisfying 1 ≤ m < n/2, the estimates β̂1 and β̂2 are

β̂1 =2

n

n∑t=1

[cos

(2πmt

n

)Yt

], β̂2 =

2

n

n∑t=1

[sin

(2πmt

n

)Yt

],

(3)Because these are linear functions of {Yt}, their variances can becomputed by (1).

1 If {Xt} is white noise, we get Varβ̂1 = Varβ̂2 = 2γ0/n.2 If m/n = 1/12 and ρk = 0 for k > 1, then

Varβ̂1 =2γ0n

[1 +

4ρ1n

n−1∑t=1

cos

(πt

6

)cos

(πt + 1

6

)]When n→∞, we have (see (3.4.5) in the textbook)

Varβ̂1 →2γ0n

(1 + 2ρ1 cos

(π6

))≈ 2γ0

n(1 + 1.732ρ1) .

Varβ̂2 has the same asymptotic behavior.Time Series Analysis Ch3. TRENDS

3.4.3 The seasonal mean trends v.s. the cosine trendsSeasonal means and cosine trends are often competing models fora cyclical trend.

Consider the two estimates for the trend in µ1 (January).

For the seasonal means, the variance is given by (2).

With the cosine trend model, the estimate is

µ̂1 = β̂0 + β̂1 cos

(2π

12

)+ β̂2 sin

(2π

12

).

The terms β̂0, β̂1, β̂2 are uncorrelated by (3). So

Varµ̂1 = Varβ̂0 + Varβ̂1

[cos

(2π

12

)]2+ Varβ̂2

[sin

(2π

12

)]2.

(4)


Consider two scenarios:

1. Assume that {Xt} is white noise. The seasonal means modelgives Varµ̂1 = γ0/N. The cosine model hasVarβ̂1 = Varβ̂2 = 2γ0/n and so

Varµ̂1 =γ0n

{1 + 2

[cos(π6

)]2+ 2

[sin(π6

)]2}= 3

γ0n.

Since N = 12n, the ratio of the standard deviation in thecosine model to that in the seasonal means model is√

3γ0/n

γ0/N=

√3N

n= 0.5.


2. Assume that {Xt} has ρk = 0 for k > 1. The seasonal meansmodel still has Varµ̂1 = γ0/N. The cosine trend model forlarge n has

Varβ̂0 ≈γ0n

(1+2ρ1), Varβ̂1 ≈ Varβ̂2 ≈2γ0n

(1 + 2ρ1 cos

(π6

)).

So (4) implies that

Varµ̂1 =γ0

n

{1 + 2ρ1 + 2

[1 + 2ρ1 cos

(π6

)]}=

γ0

n

{3 + 2ρ1

[1 + 2 cos

(π6

)]}.

If ρ1 = −0.4, then we get 0.814γ0/n, and the ratio of thestandard deviation in the cosine case to the standard deviationin the seasonal means case is√[

(0.814γ0)/n

γ0/N

]=

√0.814N

n= 0.26, a big reduction.


3.4.4 The linear time trends µt = β0 + β1tThe least squares estimate of the slope is

β̂1 =

∑nt=1(t − t)Yt∑nt=1(t − t)2

.

Since β̂1 is a linear combination of Y ’s, we have

Varβ̂1 =12γ0

n(n2 − 1)

[1 +

24

n(n2 − 1)

n∑s=2

s−1∑t=1

(t− t)(s− t)ρs−t

]For the case ρk = 0 for k > 1, we get

Varβ̂1 =12γ0

n(n2 − 1)

[1 + 2ρ1

(1− 3

n

)]≈ 12γ0(1 + 2ρ1)

n(n2 − 1)for large n.

In this situation, the variance increases as ρ1 increases. The caseρ1 = 0 is the white noise case, where

Varβ̂1 =12γ0

n(n2 − 1). (5)


3.4.5 Best linear unbiased estimates (BLUE, or GLS)


3.5 Interpreting Regression Output

In many statistical softwares, some properties of the regressionoutput depend heavily on the regression assumption that {Xt} iswhite noise, and some depend on the further assumption that {Xt}is approximately normally distributed. These assumptions are falsefor many time series!

Therefore, we shall be very careful in interpreting the regressionoutputs for time series.


Consider the deterministic trend model Yt = µt + Xt . Let µ̂t denote theestimated trend. We estimate Xt by Yt − µ̂t , and the standard deviation√γ0 of Xt by the residual standard deviation

s =

√√√√ 1

n − p

n∑t=1

(Yt − µ̂t)2 (6)

where p is the number of parameters estimated in µt and n − p is thedegree of freedom for s. The value of s (depent on the unit) gives anabsolute measure of the goodness of fit of the estimated trend.

A unitless measure of the goodness of fit is the R2. (the coefficient ofdetermination, or multiple R-squared).

R2 is the square of the sample correlation coefficient between theobserved series and the estimated trend.

R2 is also the fraction of the variation in the series that is explainedby the estimated trend.


3.6 Residual Analysis

In the modelYt = µt + Xt ,

the unobserved stochastic component {Xt} can be predicted bythe residual

X̂t = Yt − µ̂t .

We may investigate the standardized residual X̂t/s. (moststatistics software will produce standardized residuals using a morecomplicated standard error in the denominator that takes intoaccount the specific regression model being fit.)


The expression rstudent(model3) returns the (externally)studentized residuals from the fitted model. [whereasrstandard(model3) gives the (internally) standardized residuals.]

Exhibit 3.8 Residuals versus Time for Temperature Seasonal Means (chap3.R)


There are no apparent patterns relating to different months of theyear.

Exhibit 3.9 Residuals versus Time with Seasonal Plotting Symbols (chap3.R)


We look at the standardized residuals versus the correspondingtrend estimate (fitted value). No dramatic patterns are found.

Exhibit 3.10 Standardized Residuals versus Fitted Values for the Temperature Seasonal Means Model(chap3.R)


The normality of the standardized residuals may be checked byhistogram and QQ plot.

Gross nonnormality can be assessed by plotting a histogram of theresiduals or standardized residuals.


Normality can be checked more carefully by plotting the so-callednormal scores or quantile-quantile (QQ) plot. Such a plot displaysthe quantiles of the data versus the theoretical quantiles of anormal distribution. The straight-line pattern here supports theassumption of a normally distributed stochastic component in thismodel.

qqnorm(rstudent(model3))

A reference straight line can be superimposed on the Q-Q normalscore plot by

qqline(rstudent(model3))


An excellent test of normality is known as the Shapiro-Wilk test. Itessentially calculates the correlation between the residuals and thecorresponding normal quantiles.

shapiro.test(rstudent(model3))

The runs test tests the independence of a sequence of randomvariables by checking whether there are too many or too few runsabove (or below) the median.

runs(rstudent(model3))


The Sample Autocorrelation Function

Another very important diagnostic tool for examining dependenceis the sample autocorrelation function.

Def. If {Yt} is a sequence of data, we define the sampleautocorrelation function rk at lag k as

rk =

∑nt=k+1(Yt − Y )(Yt−k − Y )∑n

t=1(Yt − Y )2, for k = 1, 2, · · · (7)

A plot of rk versus lag k is often called a correlogram.

When {Yt} is a stationary process, rk gives an estimate of ρk .


win.graph(width=4.875, height=3, pointsize=8)

acf(rstudent(model3))

Exhibit 3.13 Sample Autocorrelation of Residuals of Seasonal Means Model (chap3.R)


We now consider the standardized residuals from fitting a straightline to the rwalk dataset.

Exhibit 3.14 Residuals from Straight Line Fit of the Random Walk (chap3.R)


Exhibit 3.15 Residuals versus Fitted Values from Straight Line Fit (chap3.R)


Exhibit 3.16 Sample Autocorrelation of Residuals from Straight Line Model (chap3.R)


Let us turn to the annual rainfall in Los Angeles (Exhibit 1.1). We found

no evidence of dependence in that series, but we now look for evidence

against normality. Exhibit 3.17 displays the normal quantile-quantile plot

for that series. We see considerable curvature in the plot.

Exhibit 3.17 Quantile-Quantile Plot of Los Angeles Annual Rainfall Series (chap3.R)


Exercises 3.6 & 3.12: The data file beersales contains monthly U.S.beer sales (in millions of barrels) for the period January 1975 throughDecember 1990.

(a) Display and interpret the time series plot for these data.

(b) Now construct a time series plot that uses separate plotting symbolsfor the various months. Does your interpretation change from thatin part (a)?

(c) Use least squares to fit a seasonal-means trend to this time series.Interpret the regression output. Save the standardized residualsfrom the fit for further analysis.

(d) Construct and interpret the time series plot of the standardizedresiduals from part (c). Be sure to use proper plotting symbols tocheck on seasonality in the standardized residuals.

(e) Use least squares to fit a seasonal-means plus quadratic time trendto the beer sales time series. Interpret the regression output. Savethe standardized residuals from the fit for further analysis.

See TS-ch3.R


(f) Construct and interpret the time series plot of the standardizedresiduals from part (e). Again use proper plotting symbols to checkfor any remaining seasonality in the residuals.

(g) Perform a runs test on the standardized residuals from part (e) andinterpret the results.

(h) Calculate and interpret the sample autocorrelations for thestandardized residuals.

(i) Investigate the normality of the standardized residuals (error terms).Consider histograms and normal probability plots. Interpret theplots.

See TS-ch3.R


ch3. trends - missouri state universitypeople.missouristate.edu/songfengzheng/teaching/mth548... ·...

Documents