automatic forecasting algorithms - rob j. hyndman · 2017-11-18 · motivation common in business...

Automatic Forecasting Algorithms 1

2017 Beijing Workshop onForecasting

AutomaticForecastingAlgorithmsRob J Hyndman

robjhyndman.com/beijing2017

Outline

1 Motivation

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms Motivation 2

Motivation

Common in business to have over 1000 products thatneed forecasting at least monthly.Forecasts are often required by people who areuntrained in time series analysis.

SpecificationsAutomatic forecasting algorithms must:

å determine an appropriate time series model;å estimate the parameters;å compute the forecasts with prediction intervals.

Motivation

Common in business to have over 1000 products thatneed forecasting at least monthly.Forecasts are often required by people who areuntrained in time series analysis.

SpecificationsAutomatic forecasting algorithms must:

å determine an appropriate time series model;å estimate the parameters;å compute the forecasts with prediction intervals.

Outline

1 Motivation

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms ETS 5

Exponential smoothing methodsSeasonal Component

Trend N A MComponent (None) (Additive) (Multiplicative)

N (None) (N,N) (N,A) (N,M)A (Additive) (A,N) (A,A) (A,M)Ad (Additive damped) (Ad,N) (Ad,A) (Ad,M)

(N,N): Simple exponential smoothing(A,N): Holt’s linear method(Ad,N): Additive damped trend method(A,A): Additive Holt-Winters’ method(A,M): Multiplicative Holt-Winters’ method(Ad,M): Damped multiplicative Holt-Winters’ method

There are also multiplicative trend methods (not recommended).Automatic Forecasting Algorithms ETS 6

Methods V ModelsExponential smoothing methods

Algorithms that return point forecasts.

Innovations state space modelsGenerate same point forecasts but can also generateforecast intervals.A stochastic (or random) data generating process thatcan generate an entire forecast distribution.Allow for “proper” model selection.ETS(Error,Trend,Seasonal):

Error = {A,M}Trend = {N,A,Ad}Seasonal = {N,A,M}.

Methods V ModelsExponential smoothing methods

Algorithms that return point forecasts.

Innovations state space modelsGenerate same point forecasts but can also generateforecast intervals.A stochastic (or random) data generating process thatcan generate an entire forecast distribution.Allow for “proper” model selection.ETS(Error,Trend,Seasonal):

Error = {A,M}Trend = {N,A,Ad}Seasonal = {N,A,M}.

Exponential smoothing methodsSeasonal Component

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

Exponential smoothing modelsSeasonal Component

General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

There are 9 separate exp. smoothing methods.Each can have an additive or multiplicative error, giving 18separate models.Only 15 models are numerically stable.Additive and multiplicative error models give same pointforecasts but different prediction intervals.All models can be written in innovations state space form.

Exponential smoothing modelsAdditive Error Seasonal Component

N (None) A,N,N A,N,A A,N,M

A (Additive) A,A,N A,A,A A,A,M

Ad (Additive damped) A,Ad,N A,Ad,A A,Ad,M

Multiplicative Error Seasonal ComponentTrend N A M

Component (None) (Additive) (Multiplicative)

N (None) M,N,N M,N,A M,N,M

A (Additive) M,A,N M,A,A M,A,M

Ad (Additive damped) M,Ad,N M,Ad,A M,Ad,MAutomatic Forecasting Algorithms ETS 11

ETS state space modelsxt−1

State space modelxt = (level, slope, seasonal)

xt yt+1

εt+1 xt+1

xt yt+1

εt+1 xt+1 yt+2

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3 yt+4

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3 yt+4

EstimationCompute likelihood L fromε1, ε2, . . . , εT.Optimize L wrt model parameters.

ETS state space modelsLet xt = (`t, bt, st, st−1, . . . , st−m+1) and εt

iid∼ N(0, σ2).

yt = h(xt−1)︸︷︷︸ + k(xt−1)εt︸︷︷︸ Observation equationµt et

xt = f(xt−1) + g(xt−1)εt State equation

Additive errors:k(xt−1) = 1. yt = µt + εt.

Multiplicative errors:k(xt−1) = µt. yt = µt(1 + εt).εt = (yt − µt)/µt is relative error.

Innovations state space models

Estimation

L∗(θ, x0) = n log( n∑

t=1ε2t /k

2(xt−1))

+ 2n∑t=1

log |k(xt−1)|

= −2 log(Likelihood) + constant

Estimate parameters θ = (α, β, γ, φ) and initial statesx0 = (`0, b0, s0, s−1, . . . , s−m+1) by minimizing L∗.

Automatic forecasting

From Hyndman et al. (IJF, 2002):

Apply each model that is appropriate to the data.Optimize parameters and initial values using MLE.Select best method using AICc.Produce forecasts using best method.Obtain forecast intervals using underlying state spacemodel.

Method performed very well in M3 competition.

Example: Asian livestock

livestock %>% ets() %>% forecast() %>% autoplot()

1960 1980 2000

Forecasts from ETS(M,A,N)

Example: drug sales

h02 %>% ets() %>% forecast() %>% autoplot()

1995 2000 2005 2010

Forecasts from ETS(M,Ad,M)

www.OTexts.org/fpp2

Outline

1 Motivation

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms ARIMA models 19

ARIMA models

yt−1

yt−2

yt−3yt

Inputs Output

ARIMA models

yt−1

yt−2

yt−3

Inputs Output

Autoregression (AR) model

ARIMA models

yt−1

yt−2

yt−3

εt−1

εt−2

Inputs Output

Autoregression movingaverage (ARMA) model

ARIMA models

yt−1

yt−2

yt−3

εt−1

εt−2

Inputs Output

EstimationCompute likelihood L fromε1, ε2, . . . , εT.Use optimization algorithm tomaximize L.

ARIMA models

yt−1

yt−2

yt−3

εt−1

εt−2

Inputs Output

EstimationCompute likelihood L fromε1, ε2, . . . , εT.Use optimization algorithm tomaximize L.

ARIMA modelAutoregression movingaverage (ARMA) modelapplied to differences.

Auto ARIMAlivestock %>% auto.arima() %>% forecast() %>% autoplot()

1960 1980 2000

Forecasts from ARIMA(0,1,0) with drift

Auto ARIMAh02 %>% auto.arima() %>% forecast() %>% autoplot()

1995 2000 2005 2010

Forecasts from ARIMA(3,1,3)(0,1,1)[12]

How does auto.arima() work?A non-seasonal ARIMA process

φ(B)(1− B)dyt = c + θ(B)εtNeed to select appropriate orders p, q, d, and whether toinclude c.

Algorithm choices driven by forecast accuracy.

Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select p, q, c by minimising AICc.Use stepwise search to traverse model space, startingwith a simple model and considering nearby variants.

How does auto.arima() work?A seasonal ARIMA process

Φ(Bm)φ(B)(1− B)d(1− Bm)Dyt = c + Θ(Bm)θ(B)εtNeed to select appropriate orders p, q, d, P,Q,D, andwhether to include c.

Hyndman & Khandakar (JSS, 2008) algorithm:

Select no. differences d via KPSS unit root test.Select D using OCSB unit root test.Select p, q, P,Q, c by minimising AICc.Use stepwise search to traverse model space, startingwith a simple model and considering nearby variants.

Outline

1 Motivation

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms STLM 25

STL decompositionSTL: “Seasonal and Trend decomposition using Loess”,Very versatile and robust.Unlike X-12-ARIMA, STL will handle any type ofseasonality.Seasonal component allowed to change over time,and rate of change controlled by user.Smoothness of trend-cycle also controlled by user.Robust to outliersNot trading day or calendar adjustments.Only additive.Take logs to get multiplicative decomposition.Use Box-Cox transformations to get otherdecompositions.

STL decompositionfit <- stl(elecequip, s.window=5, robust=TRUE)autoplot(fit) +

ggtitle("STL decomposition of electrical equipment index")

2000 2005 2010

STL decomposition of electrical equipment index

STL decompositionfit <- stl(elecequip, s.window="periodic", robust=TRUE)autoplot(fit) +

ggtitle("STL decomposition of electrical equipment index")

2000 2005 2010

−10−5

STL decomposition of electrical equipment index

Forecasting and decomposition

Forecast seasonal component using seasonal naivemethod.Forecast seasonally adjusted data using non-seasonaltime series method. E.g., ETS or ARIMA.Combine forecasts of seasonal component withforecasts of seasonally adjusted data to get forecastsof original data.

Electrical equipmentfit <- stl(elecequip, s.window=7)fit %>% seasadj() %>% ets() %>% forecast() %>% autoplot() + ylab("seasonally adjusted elecequip")

2000 2005 2010 2015

Forecasts from ETS(A,Ad,N)

Electrical equipmentfit %>% forecast() %>%

autoplot()

2000 2005 2010 2015

p level

Forecasts from STL + ETS(A,Ad,N)

Forecasting and decompositionelecequip %>% stlf() %>%

autoplot() + ylab('elecequip')

2000 2005 2010 2015

p level

Forecasts from STL + ETS(A,Ad,N)

Decomposition and prediction intervals

It is common to take the prediction intervals from theseasonally adjusted forecasts and modify them withthe seasonal component.This ignores the uncertainty in the seasonalcomponent estimate.It also ignores the uncertainty in the future seasonalpattern.

Outline

1 Motivation

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms TBATS 34

Complex seasonality

1990 2000 2010 2020

Gasoline data

Complex seasonality

1990 2000 2010 2020

Forecasts from TBATS(1, {0,0}, −, {<52.18,9>})

Complex seasonality

1990 2000 2010 2020

Forecasts from TBATS(1, {0,0}, −, {<52.18,9>})

fit <- tbats(gasoline)fcast <- forecast(fit)autoplot(fcast)

Complex seasonality

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Number of calls to large American bank {7am..9pm}

Complex seasonality

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Forecasts from TBATS(1, {3,1}, 0.8, {<169,6>, <845,4>})

Complex seasonality

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Forecasts from TBATS(1, {3,1}, 0.8, {<169,6>, <845,4>})

fit <- tbats(calls)fcast <- forecast(fit)autoplot(fcast)

TBATS model

yt = observation at time t

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt

dt =p∑i=1φidt−i +

q∑j=1θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)

j,t−1 cosλ(i)j + s∗(i)

j,t−1 sinλ(i)j + γ(i)

1 dts(i)j,t = −s(i)

j,t−1 sinλ(i)j + s∗(i)

j,t−1 cosλ(i)j + γ(i)

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)

Box-Cox transformation

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)

M seasonal periods

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)

M seasonal periods

global and local trend

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)

M seasonal periods

ARMA error

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)

M seasonal periods

ARMA error

Fourier-like seasonal terms

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)

M seasonal periods

ARMA error

Fourier-like seasonal terms

TBATSTrigonometricBox-CoxARMATrendSeasonal

TBATS modelTBATSTrigonometric terms for seasonalityBox-Cox transformations for heterogeneityARMA errors for short-term dynamicsTrend (possibly damped)Seasonal (including multiple and non-integer periods)

Handles non-integer seasonality, multiple seasonalperiods.Entirely automatedPrediction intervals often too wideVery slow on long series

Outline

1 Motivation

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms FASSTER 43

Pedestrian counts

Jul 2016 Oct 2016 Jan 2017 Apr 2017

Hourly pedestrian traffic at Southern Cross Station

Apr 01 Apr 15 May 01 May 15 Jun 01

Hourly pedestrian traffic at Southern Cross Station

Pedestrian counts

Weekday Weekend

Seasonality in pedestrian traffic at Southern Cross Station

Switching StructureFASSTER extends current state-space approaches byswitching between states.Dynamic linear model

yt = Ftθt + vt, vt ∼ N (0,V)θt = Gθt−1 + wt, wt ∼ N (0,W)

Switch between two groups:

yt = It∈G1Ftθ(1)t + It∈G2Ftθ

(2)t + vt vt ∼ N (0,V)

Groups G1 and G2 define the switching rule (say weekdaysand weekends).

(2)t + vt vt ∼ N (0,V)

Application to pedestrian trafficThis series contains a switching daily pattern overweekdays and weekends. Each group can be modelledusing level and hourly seasonal states.

Jan 30 Feb 06 Feb 13 Feb 20

Pedestrian Traffic at Southern Cross Station

Application to pedestrian traffic

An appropriate model can be constructed using switchingstates:

yt = It∈WeekdayFtθ(Weekday)t + It∈WeekendFtθ(Weekend)

t + vt

where Ftθ(i)t = `(i)

t + f(i)t and vt ∼ N (0,V)

`t is a level componentft is a seasonal component based on Fourier terms

FASSTER allows flexible use of:

seasonal factorsfourier seasonal termspolynomial trendsBoxCox transformationsexogenous regressorsARMA processesstate switching

General measurement equation

yt = F(0)t θ

(0)t +

k∑j=1

It∈GjF(j)t θ

(j)t + vt, vt ∼ N (0,V)

where k is the number of switching combinations.Automatic Forecasting Algorithms FASSTER 50

FASSTER allows flexible use of:

seasonal factorsfourier seasonal termspolynomial trendsBoxCox transformationsexogenous regressorsARMA processesstate switching

General measurement equation

yt = F(0)t θ

(0)t +

k∑j=1

It∈GjF(j)t θ

(j)t + vt, vt ∼ N (0,V)

where k is the number of switching combinations.Automatic Forecasting Algorithms FASSTER 50

FASSTER and Dynamic Linear ModelsA FASSTER model can be represented as a time-varyingDLM.

where θ0 ∼ N (m0, C0)

There are too many parameters to easily estimate.E.g., pedestrian model has 48 states:2 groups× (1 level + 23 fourier states).We use a “heuristic estimation” approach involvingonly two passes through the data.

FASSTER and Dynamic Linear ModelsA FASSTER model can be represented as a time-varyingDLM.

where θ0 ∼ N (m0, C0)

There are too many parameters to easily estimate.E.g., pedestrian model has 48 states:2 groups× (1 level + 23 fourier states).We use a “heuristic estimation” approach involvingonly two passes through the data.

Usage (Pedestrian Counts)SthCross_fasster_fit <- SthCross_Ex %>%

fasster(Hourly_Counts ~ DayType %S% (poly(1) + trig(24)))

0 100 200 300 400 500

Series

Response

Fitted

Pedestrian Traffic at Southern Cross Station

Forecasts (Pedestrian counts)

Outline

1 Motivation

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms Comparisons 54

Pharmaceutical sales

1995 2000 2005 2010

Forecast

FASSTER

R packages

https://github.com/earowang/tsibble

http://pkg.earo.me/sugrrants

https://github.com/mitchelloharawild/fasster

http://pkg.robjhyndman.com/forecast

http://pkg.earo.me/hts

Textbook

OTexts.org/fpp2

automatic forecasting algorithms - rob j. hyndman · 2017-11-18 · motivation common in business...

Documents

forecasting using - rob j hyndman · forecasting using r...

forecasting inflation and growth: do private forecasts match...

functional time series forecasting - semantic scholar ›...

jon faust and jonathan wright. forecasting inflation a...

forecasting using - rob j hyndman · pdf...

forecasting february 26, 2007. laws of forecasting three...

chapter 4 forecasting production planning overview what is...

forecasting using r - rob j hyndman · outline 1time series...

forecasting the unpredictable: a review of forecasts on...

forecasting the forecasts of others: implications for

forecasting hierarchical time series - rob j hyndman · pdf...

forecasting - rob j hyndman · vector ar allow for feedback...

econometric forecasts package: short-run forecasting models...

forecasting using r - rob j. hyndman · fit3

weather forecasting for load forecasts

forecasting - rob j hyndman · fertilizer sales. brief bio...

forecasting uncertainty in electricity demand - … ·...

forecasting using - rob j hyndman · pdf file ·...

coherent mortality forecasting: the product-ratio...

forecasting using r - rob j. hyndman · 2016-10-18 · case...