visualization and forecasting of big time series data
TRANSCRIPT
Rob J Hyndman
Visualizing and forecasting
big time series data
20
00
20
10
Ho
liday
20
00
20
10
VF
R
20
00
20
10
Bu
sin
ess
20
00
20
10
BA
A
BA
B
BA
C
BB
A
BC
A
BC
B
BC
C
BD
A
BD
B
BD
C
BD
D
BD
E
BD
F
BE
A
BE
B
BE
C
BE
D
BE
E
BE
F
Oth
er
BE
G
Victoria: scaled
Outline
1 Examples of biggish time series
2 Time series visualisation
3 BLUF: Best Linear Unbiased Forecasts
4 Application: Australian tourism
5 Fast computation tricks
6 hts package for R
7 References
Visualising and forecasting big time series data Examples of biggish time series 2
1. Australian tourism demand
Visualising and forecasting big time series data Examples of biggish time series 3
1. Australian tourism demand
Visualising and forecasting big time series data Examples of biggish time series 3
Quarterly data on visitor night from1998:Q1 – 2013:Q4From: National Visitor Survey, based onannual interviews of 120,000 Australiansaged 15+, collected by Tourism ResearchAustralia.Split by 7 states, 27 zones and 76 regions(a geographical hierarchy)Also split by purpose of travel
HolidayVisiting friends and relatives (VFR)BusinessOther
304 bottom-level series
2. Labour market participation
Australia and New Zealand StandardClassification of Occupations
8 major groups43 sub-major groups
97 minor groups– 359 unit groups
* 1023 occupations
Example: statistician2 Professionals
22 Business, Human Resource and MarketingProfessionals224 Information and Organisation Professionals
2241 Actuaries, Mathematicians and Statisticians224113 Statistician
Visualising and forecasting big time series data Examples of biggish time series 4
2. Labour market participation
Australia and New Zealand StandardClassification of Occupations
8 major groups43 sub-major groups
97 minor groups– 359 unit groups
* 1023 occupations
Example: statistician2 Professionals
22 Business, Human Resource and MarketingProfessionals224 Information and Organisation Professionals
2241 Actuaries, Mathematicians and Statisticians224113 Statistician
Visualising and forecasting big time series data Examples of biggish time series 4
3. Spectacle sales
Visualising and forecasting big time series data Examples of biggish time series 5
Monthly UK sales data from 2000 – 2014Provided by a large spectacle manufacturerSplit by brand (26), gender (3), price range(6), materials (4), and stores (600)About 1 million bottom-level series
3. Spectacle sales
Visualising and forecasting big time series data Examples of biggish time series 5
Monthly UK sales data from 2000 – 2014Provided by a large spectacle manufacturerSplit by brand (26), gender (3), price range(6), materials (4), and stores (600)About 1 million bottom-level series
3. Spectacle sales
Visualising and forecasting big time series data Examples of biggish time series 5
Monthly UK sales data from 2000 – 2014Provided by a large spectacle manufacturerSplit by brand (26), gender (3), price range(6), materials (4), and stores (600)About 1 million bottom-level series
3. Spectacle sales
Visualising and forecasting big time series data Examples of biggish time series 5
Monthly UK sales data from 2000 – 2014Provided by a large spectacle manufacturerSplit by brand (26), gender (3), price range(6), materials (4), and stores (600)About 1 million bottom-level series
Outline
1 Examples of biggish time series
2 Time series visualisation
3 BLUF: Best Linear Unbiased Forecasts
4 Application: Australian tourism
5 Fast computation tricks
6 hts package for R
7 References
Visualising and forecasting big time series data Time series visualisation 6
Kite diagrams0
00
Line graph profile
Duplicate & fliparound the hori-zontal axis
Fill the colour
Visualising and forecasting big time series data Time series visualisation 7
Kite diagrams: Victorian tourism20
0020
10
Hol
iday
2000
2010
VF
R
2000
2010
Bus
ines
s
2000
2010
BA
A
BA
B
BA
C
BB
A
BC
A
BC
B
BC
C
BD
A
BD
B
BD
C
BD
D
BD
E
BD
F
BE
A
BE
B
BE
C
BE
D
BE
E
BE
F
Oth
er
BE
G
Victoria
Visualising and forecasting big time series data Time series visualisation 8
Kite diagrams: Victorian tourism
Visualising and forecasting big time series data Time series visualisation 8
Kite diagrams: Victorian tourism
Visualising and forecasting big time series data Time series visualisation 8
Kite diagrams: Victorian tourism20
0020
10
Hol
iday
2000
2010
VF
R
2000
2010
Bus
ines
s
2000
2010
BA
A
BA
B
BA
C
BB
A
BC
A
BC
B
BC
C
BD
A
BD
B
BD
C
BD
D
BD
E
BD
F
BE
A
BE
B
BE
C
BE
D
BE
E
BE
F
Oth
er
BE
G
Victoria: scaled
Visualising and forecasting big time series data Time series visualisation 8
An STL decompositionSTL decomposition of tourism demandfor holidays in Peninsula
5.0
6.0
7.0
data
−0.
50.
5
seas
onal
5.8
6.1
6.4
tren
d
−0.
40.
0
2000 2005 2010
rem
aind
er
timeVisualising and forecasting big time series data Time series visualisation 9
Seasonal stacked bar chart
Place positive values above the originwhile negative values below the originMap the bar length to the magnitudeEncode quarters by colours
Visualising and forecasting big time series data Time series visualisation 10
Seasonal stacked bar chart
Place positive values above the originwhile negative values below the originMap the bar length to the magnitudeEncode quarters by colours
−1.0
−0.5
0.0
0.5
1.0
Holiday
BAA BAB BAC BBABCABCBBCCBDABDBBDCBDDBDEBDF BEA BEBBECBEDBEE BEFBEGRegions
Sea
sona
l Com
pone
nt
Qtr
Q1
Q2
Q3
Q4
Visualising and forecasting big time series data Time series visualisation 10
Seasonal stacked bar chart: VIC
Visualising and forecasting big time series data Time series visualisation 11
Corrgram of remainder
Visualising and forecasting big time series data Time series visualisation 12
Compute the correlationsamong the remaindercomponents
Render both the sign andmagnitude using a colourmapping of two hues
Order variables according tothe first principal component ofthe correlations.
Corrgram of remainder: VIC
Visualising and forecasting big time series data Time series visualisation 13−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
BE
EH
olB
EF
Oth
BE
EO
thB
DE
Oth
BE
BO
thB
EA
Bus
BE
FB
usB
DC
Oth
BA
CH
olB
EB
Bus
BE
AV
isB
BA
Hol
BD
EH
olB
AB
Oth
BA
AV
isB
AA
Hol
BD
CH
olB
BA
Bus
BC
BH
olB
EG
Bus
BD
DV
isB
AB
Vis
BD
AV
isB
EA
Oth
BD
FH
olB
EE
Bus
BA
AO
thB
AC
Oth
BD
AO
thB
DE
Bus
BC
BO
thB
AC
Bus
BE
BV
isB
AC
Vis
BC
AO
thB
EF
Vis
BC
BV
isB
ED
Hol
BE
GO
thB
DB
Hol
BA
BB
usB
EB
Hol
BD
FB
usB
EC
Hol
BC
AH
olB
DB
Oth
BE
AH
olB
DC
Bus
BE
CV
isB
DB
Vis
BC
CH
olB
BA
Vis
BA
BH
olB
BA
Oth
BC
CO
thB
CB
Bus
BC
CV
isB
EG
Vis
BD
DH
olB
EC
Oth
BD
CV
isB
AA
Bus
BC
CB
usB
EC
Bus
BC
AV
isB
DF
Vis
BE
GH
olB
DD
Oth
BE
DO
thB
ED
Vis
BD
DB
usB
DE
Vis
BE
FH
olB
EE
Vis
BD
BB
usB
DA
Bus
BD
AH
olB
CA
Bus
BD
FO
thB
ED
Bus
BEEHolBEFOthBEEOthBDEOthBEBOthBEABusBEFBusBDCOthBACHolBEBBusBEAVisBBAHolBDEHolBABOthBAAVisBAAHolBDCHolBBABusBCBHolBEGBusBDDVisBABVisBDAVisBEAOthBDFHolBEEBusBAAOthBACOthBDAOthBDEBusBCBOthBACBusBEBVisBACVisBCAOthBEFVisBCBVisBEDHolBEGOthBDBHolBABBusBEBHolBDFBusBECHolBCAHolBDBOthBEAHolBDCBusBECVisBDBVisBCCHolBBAVisBABHolBBAOthBCCOthBCBBusBCCVisBEGVisBDDHolBECOthBDCVisBAABusBCCBusBECBusBCAVisBDFVisBEGHolBDDOthBEDOthBEDVisBDDBusBDEVisBEFHolBEEVisBDBBusBDABusBDAHolBCABusBDFOthBEDBus
Corrgram of remainder: TAS
Visualising and forecasting big time series data Time series visualisation 14−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
FC
AH
ol
FB
BH
ol
FB
AH
ol
FAA
Hol
FC
BH
ol
FC
AV
is
FB
BV
is
FAA
Vis
FC
BB
us
FAA
Oth
FC
AO
th
FB
BO
th
FB
AB
us
FB
AO
th
FC
BV
is
FC
AB
us
FB
AV
is
FC
BO
th
FB
BB
us
FAA
Bus
FCAHol
FBBHol
FBAHol
FAAHol
FCBHol
FCAVis
FBBVis
FAAVis
FCBBus
FAAOth
FCAOth
FBBOth
FBABus
FBAOth
FCBVis
FCABus
FBAVis
FCBOth
FBBBus
FAABus
Feature analysis
Summarize each time series with a featurevector:
strength of trendlumpiness (variance of annual variances ofremainder)strength of seasonalitysize of seasonal peaksize of seasonal troughACF1linearity of trendcurvature of trendspectral entropy
Do PCA on feature matrix
Visualising and forecasting big time series data Time series visualisation 15
Feature analysis
Visualising and forecasting big time series data Time series visualisation 16
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
3031
32
33
34
35
36
37
38
39
40
41
42
4344
45
46
47
48
49
50
51
52
53
54
55
56 57
58
59
6061
62
63
64
65
6667
68
69
70
71
7273
7475
76
77
78
79
80
81
82 83
84
85
86
87
8889
90 91
9293
94
9596
97
98
99
100
101
102
103
104
105
106 107
108109
110
111
112
113
114
115
116
117118
119
120
121
122
123
124
125
126
127
128
129130
131
132
133134
135
136
137
138
139
140
141
142143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158159
160
161
162
163
164
165
166
167
168
169170 171
172
173
174
175
176
177
178
179
180
181182
183
184
185
186187
188
189
190
191
192
193
194 195
196
197 198
199
200
201
202
203
204
205
206207
208
209210
211212
213
214
215
216
217
218219
220
221
222
223
224
225
226227
228
229
230
231
232
233
234
235236
237
238
239
240
241
242 243 244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262263
264
265
266 267
268
269
270
271
272
273
274275
276
277
278
279280
281
282
283
284
285
286
287
288289
290
291
292
293
294
295296
297
298
299300
301 302303304
lum
p
season
spike
entropy
fo.acf
peaktrough
linearity
curvature
−2
0
2
4
−2 0 2standardized PC1 (43.2% explained var.)
stan
dard
ized
PC
2 (1
8.6%
exp
lain
ed v
ar.)
Feature analysis
Visualising and forecasting big time series data Time series visualisation 16
0
200
400
600
800
0
250
500
750
0
100
200
300
400
0
500
1000
1500
BE
GB
usB
CC
Bus
CC
AO
thC
AC
Oth
2000 2005 2010Time
valu
e
Feature analysis
Visualising and forecasting big time series data Time series visualisation 16
400
800
1200
1600
250
500
750
1000
500
1000
1500
2000
1000
2000
3000
4000
AD
AH
olB
DC
Hol
BB
AH
olA
CA
Hol
2000 2005 2010Time
valu
e
Feature analysis
Visualising and forecasting big time series data Time series visualisation 16
NSW NT QLD SA
TAS VIC WA
−2
0
2
4
6
−2
0
2
4
6
−7.5−5.0−2.5 0.0 2.5 5.0−7.5−5.0−2.5 0.0 2.5 5.0−7.5−5.0−2.5 0.0 2.5 5.0PC1
PC
2
Feature analysis
Visualising and forecasting big time series data Time series visualisation 16
Bus Hol
Oth Vis
−2
0
2
4
6
−2
0
2
4
6
−7.5 −5.0 −2.5 0.0 2.5 5.0−7.5 −5.0 −2.5 0.0 2.5 5.0PC1
PC
2
Outline
1 Examples of biggish time series
2 Time series visualisation
3 BLUF: Best Linear Unbiased Forecasts
4 Application: Australian tourism
5 Fast computation tricks
6 hts package for R
7 References
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 17
Hierarchical time series
A hierarchical time series is a collection ofseveral time series that are linked together ina hierarchical structure.
Total
A
AA AB AC
B
BA BB BC
C
CA CB CC
ExamplesNet labour turnoverTourism by state and region
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 18
Hierarchical time series
A hierarchical time series is a collection ofseveral time series that are linked together ina hierarchical structure.
Total
A
AA AB AC
B
BA BB BC
C
CA CB CC
ExamplesNet labour turnoverTourism by state and region
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 18
Hierarchical time series
A hierarchical time series is a collection ofseveral time series that are linked together ina hierarchical structure.
Total
A
AA AB AC
B
BA BB BC
C
CA CB CC
ExamplesNet labour turnoverTourism by state and region
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 18
Hierarchical time series
Total
A B C
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 19
Yt : observed aggregate of allseries at time t.
YX,t : observation on series X attime t.
Bt : vector of all series atbottom level in time t.
Hierarchical time series
Total
A B C
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 19
Yt : observed aggregate of allseries at time t.
YX,t : observation on series X attime t.
Bt : vector of all series atbottom level in time t.
Hierarchical time series
Total
A B C
yt = [Yt, YA,t, YB,t, YC,t]′ =
1 1 11 0 00 1 00 0 1
YA,tYB,tYC,t
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 19
Yt : observed aggregate of allseries at time t.
YX,t : observation on series X attime t.
Bt : vector of all series atbottom level in time t.
Hierarchical time series
Total
A B C
yt = [Yt, YA,t, YB,t, YC,t]′ =
1 1 11 0 00 1 00 0 1
︸ ︷︷ ︸
S
YA,tYB,tYC,t
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 19
Yt : observed aggregate of allseries at time t.
YX,t : observation on series X attime t.
Bt : vector of all series atbottom level in time t.
Hierarchical time series
Total
A B C
yt = [Yt, YA,t, YB,t, YC,t]′ =
1 1 11 0 00 1 00 0 1
︸ ︷︷ ︸
S
YA,tYB,tYC,t
︸ ︷︷ ︸
Bt
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 19
Yt : observed aggregate of allseries at time t.
YX,t : observation on series X attime t.
Bt : vector of all series atbottom level in time t.
Hierarchical time series
Total
A B C
yt = [Yt, YA,t, YB,t, YC,t]′ =
1 1 11 0 00 1 00 0 1
︸ ︷︷ ︸
S
YA,tYB,tYC,t
︸ ︷︷ ︸
Btyt = SBt
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 19
Yt : observed aggregate of allseries at time t.
YX,t : observation on series X attime t.
Bt : vector of all series atbottom level in time t.
Hierarchical time seriesTotal
A
AX AY AZ
B
BX BY BZ
C
CX CY CZ
yt =
YtYA,tYB,tYC,tYAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
=
1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1
︸ ︷︷ ︸
S
YAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
︸ ︷︷ ︸
Bt
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 20
Hierarchical time seriesTotal
A
AX AY AZ
B
BX BY BZ
C
CX CY CZ
yt =
YtYA,tYB,tYC,tYAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
=
1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1
︸ ︷︷ ︸
S
YAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
︸ ︷︷ ︸
Bt
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 20
Hierarchical time seriesTotal
A
AX AY AZ
B
BX BY BZ
C
CX CY CZ
yt =
YtYA,tYB,tYC,tYAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
=
1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1
︸ ︷︷ ︸
S
YAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
︸ ︷︷ ︸
Bt
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 20
yt = SBt
Forecasting notation
Let yn(h) be vector of initial h-step forecasts,made at time n, stacked in same order as yt.(They may not add up.)
Reconciled forecasts are of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecastsyn(h) to get bottom-level forecasts.
S adds them up
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 21
Forecasting notation
Let yn(h) be vector of initial h-step forecasts,made at time n, stacked in same order as yt.(They may not add up.)
Reconciled forecasts are of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecastsyn(h) to get bottom-level forecasts.
S adds them up
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 21
Forecasting notation
Let yn(h) be vector of initial h-step forecasts,made at time n, stacked in same order as yt.(They may not add up.)
Reconciled forecasts are of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecastsyn(h) to get bottom-level forecasts.
S adds them up
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 21
Forecasting notation
Let yn(h) be vector of initial h-step forecasts,made at time n, stacked in same order as yt.(They may not add up.)
Reconciled forecasts are of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecastsyn(h) to get bottom-level forecasts.
S adds them up
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 21
Forecasting notation
Let yn(h) be vector of initial h-step forecasts,made at time n, stacked in same order as yt.(They may not add up.)
Reconciled forecasts are of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecastsyn(h) to get bottom-level forecasts.
S adds them up
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 21
General properties: bias
yn(h) = SPyn(h)
Assume: base forecasts yn(h) are unbiased:E[yn(h)|y1, . . . , yn] = E[yn+h|y1, . . . , yn]
Let Bn(h) be bottom level base forecastswith βn(h) = E[Bn(h)|y1, . . . , yn].Then E[yn(h)] = Sβn(h).We want the revised forecasts to beunbiased: E[yn(h)] = SPSβn(h) = Sβn(h).
Revised forecasts are unbiased iff SPS = S.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 22
General properties: bias
yn(h) = SPyn(h)
Assume: base forecasts yn(h) are unbiased:E[yn(h)|y1, . . . , yn] = E[yn+h|y1, . . . , yn]
Let Bn(h) be bottom level base forecastswith βn(h) = E[Bn(h)|y1, . . . , yn].Then E[yn(h)] = Sβn(h).We want the revised forecasts to beunbiased: E[yn(h)] = SPSβn(h) = Sβn(h).
Revised forecasts are unbiased iff SPS = S.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 22
General properties: bias
yn(h) = SPyn(h)
Assume: base forecasts yn(h) are unbiased:E[yn(h)|y1, . . . , yn] = E[yn+h|y1, . . . , yn]
Let Bn(h) be bottom level base forecastswith βn(h) = E[Bn(h)|y1, . . . , yn].Then E[yn(h)] = Sβn(h).We want the revised forecasts to beunbiased: E[yn(h)] = SPSβn(h) = Sβn(h).
Revised forecasts are unbiased iff SPS = S.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 22
General properties: bias
yn(h) = SPyn(h)
Assume: base forecasts yn(h) are unbiased:E[yn(h)|y1, . . . , yn] = E[yn+h|y1, . . . , yn]
Let Bn(h) be bottom level base forecastswith βn(h) = E[Bn(h)|y1, . . . , yn].Then E[yn(h)] = Sβn(h).We want the revised forecasts to beunbiased: E[yn(h)] = SPSβn(h) = Sβn(h).
Revised forecasts are unbiased iff SPS = S.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 22
General properties: bias
yn(h) = SPyn(h)
Assume: base forecasts yn(h) are unbiased:E[yn(h)|y1, . . . , yn] = E[yn+h|y1, . . . , yn]
Let Bn(h) be bottom level base forecastswith βn(h) = E[Bn(h)|y1, . . . , yn].Then E[yn(h)] = Sβn(h).We want the revised forecasts to beunbiased: E[yn(h)] = SPSβn(h) = Sβn(h).
Revised forecasts are unbiased iff SPS = S.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 22
General properties: bias
yn(h) = SPyn(h)
Assume: base forecasts yn(h) are unbiased:E[yn(h)|y1, . . . , yn] = E[yn+h|y1, . . . , yn]
Let Bn(h) be bottom level base forecastswith βn(h) = E[Bn(h)|y1, . . . , yn].Then E[yn(h)] = Sβn(h).We want the revised forecasts to beunbiased: E[yn(h)] = SPSβn(h) = Sβn(h).
Revised forecasts are unbiased iff SPS = S.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 22
General properties: bias
yn(h) = SPyn(h)
Assume: base forecasts yn(h) are unbiased:E[yn(h)|y1, . . . , yn] = E[yn+h|y1, . . . , yn]
Let Bn(h) be bottom level base forecastswith βn(h) = E[Bn(h)|y1, . . . , yn].Then E[yn(h)] = Sβn(h).We want the revised forecasts to beunbiased: E[yn(h)] = SPSβn(h) = Sβn(h).
Revised forecasts are unbiased iff SPS = S.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 22
General properties: variance
yn(h) = SPyn(h)
Let variance of base forecasts yn(h) be givenby
Σh = Var[yn(h)|y1, . . . , yn]
Then the variance of the revised forecasts isgiven by
Var[yn(h)|y1, . . . , yn] = SPΣhP′S′.
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 23
General properties: variance
yn(h) = SPyn(h)
Let variance of base forecasts yn(h) be givenby
Σh = Var[yn(h)|y1, . . . , yn]
Then the variance of the revised forecasts isgiven by
Var[yn(h)|y1, . . . , yn] = SPΣhP′S′.
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 23
General properties: variance
yn(h) = SPyn(h)
Let variance of base forecasts yn(h) be givenby
Σh = Var[yn(h)|y1, . . . , yn]
Then the variance of the revised forecasts isgiven by
Var[yn(h)|y1, . . . , yn] = SPΣhP′S′.
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 23
BLUF via trace minimizationTheoremFor any P satisfying SPS = S, then
minP
= trace[SPΣhP′S′]
has solution P = (S′Σ†hS)−1S′Σ†h.
Σ†h is generalized inverse of Σh.
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Revised forecasts Base forecasts
Equivalent to GLS estimate of regressionyn(h) = Sβn(h) + εh where ε ∼ N(0,Σh).
Problem: Σh hard to estimate.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 24
BLUF via trace minimizationTheoremFor any P satisfying SPS = S, then
minP
= trace[SPΣhP′S′]
has solution P = (S′Σ†hS)−1S′Σ†h.
Σ†h is generalized inverse of Σh.
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Revised forecasts Base forecasts
Equivalent to GLS estimate of regressionyn(h) = Sβn(h) + εh where ε ∼ N(0,Σh).
Problem: Σh hard to estimate.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 24
BLUF via trace minimizationTheoremFor any P satisfying SPS = S, then
minP
= trace[SPΣhP′S′]
has solution P = (S′Σ†hS)−1S′Σ†h.
Σ†h is generalized inverse of Σh.
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Revised forecasts Base forecasts
Equivalent to GLS estimate of regressionyn(h) = Sβn(h) + εh where ε ∼ N(0,Σh).
Problem: Σh hard to estimate.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 24
BLUF via trace minimizationTheoremFor any P satisfying SPS = S, then
minP
= trace[SPΣhP′S′]
has solution P = (S′Σ†hS)−1S′Σ†h.
Σ†h is generalized inverse of Σh.
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Revised forecasts Base forecasts
Equivalent to GLS estimate of regressionyn(h) = Sβn(h) + εh where ε ∼ N(0,Σh).
Problem: Σh hard to estimate.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 24
BLUF via trace minimizationTheoremFor any P satisfying SPS = S, then
minP
= trace[SPΣhP′S′]
has solution P = (S′Σ†hS)−1S′Σ†h.
Σ†h is generalized inverse of Σh.
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Revised forecasts Base forecasts
Equivalent to GLS estimate of regressionyn(h) = Sβn(h) + εh where ε ∼ N(0,Σh).
Problem: Σh hard to estimate.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 24
BLUF via trace minimizationTheoremFor any P satisfying SPS = S, then
minP
= trace[SPΣhP′S′]
has solution P = (S′Σ†hS)−1S′Σ†h.
Σ†h is generalized inverse of Σh.
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Revised forecasts Base forecasts
Equivalent to GLS estimate of regressionyn(h) = Sβn(h) + εh where ε ∼ N(0,Σh).
Problem: Σh hard to estimate.Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 24
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 1: OLSAssume εh ≈ SεB,h where εB,h is theforecast error at bottom level.
Then Σh ≈ SΩhS′ where Ωh = Var(εB,h).
If Moore-Penrose generalized inverse used,then (S′Σ†hS)
−1S′Σ†h = (S′S)−1S′.
yn(h) = S(S′S)−1S′yn(h)
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 25
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 1: OLSAssume εh ≈ SεB,h where εB,h is theforecast error at bottom level.
Then Σh ≈ SΩhS′ where Ωh = Var(εB,h).
If Moore-Penrose generalized inverse used,then (S′Σ†hS)
−1S′Σ†h = (S′S)−1S′.
yn(h) = S(S′S)−1S′yn(h)
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 25
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 1: OLSAssume εh ≈ SεB,h where εB,h is theforecast error at bottom level.
Then Σh ≈ SΩhS′ where Ωh = Var(εB,h).
If Moore-Penrose generalized inverse used,then (S′Σ†hS)
−1S′Σ†h = (S′S)−1S′.
yn(h) = S(S′S)−1S′yn(h)
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 25
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 1: OLSAssume εh ≈ SεB,h where εB,h is theforecast error at bottom level.
Then Σh ≈ SΩhS′ where Ωh = Var(εB,h).
If Moore-Penrose generalized inverse used,then (S′Σ†hS)
−1S′Σ†h = (S′S)−1S′.
yn(h) = S(S′S)−1S′yn(h)
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 25
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 1: OLSAssume εh ≈ SεB,h where εB,h is theforecast error at bottom level.
Then Σh ≈ SΩhS′ where Ωh = Var(εB,h).
If Moore-Penrose generalized inverse used,then (S′Σ†hS)
−1S′Σ†h = (S′S)−1S′.
yn(h) = S(S′S)−1S′yn(h)
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 25
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 1: OLSAssume εh ≈ SεB,h where εB,h is theforecast error at bottom level.
Then Σh ≈ SΩhS′ where Ωh = Var(εB,h).
If Moore-Penrose generalized inverse used,then (S′Σ†hS)
−1S′Σ†h = (S′S)−1S′.
yn(h) = S(S′S)−1S′yn(h)
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 25
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 2: WLSSuppose we approximate Σ1 by itsdiagonal.Easy to estimate, and places weight wherewe have best forecasts.Empirically, it gives better forecasts thanother available methods.
yn(h) = S(S′ΛS)−1S′Λyn(h)Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 26
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 2: WLSSuppose we approximate Σ1 by itsdiagonal.Easy to estimate, and places weight wherewe have best forecasts.Empirically, it gives better forecasts thanother available methods.
yn(h) = S(S′ΛS)−1S′Λyn(h)Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 26
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 2: WLSSuppose we approximate Σ1 by itsdiagonal.Easy to estimate, and places weight wherewe have best forecasts.Empirically, it gives better forecasts thanother available methods.
yn(h) = S(S′ΛS)−1S′Λyn(h)Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 26
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 2: WLSSuppose we approximate Σ1 by itsdiagonal.Easy to estimate, and places weight wherewe have best forecasts.Empirically, it gives better forecasts thanother available methods.
yn(h) = S(S′ΛS)−1S′Λyn(h)Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 26
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 2: WLSSuppose we approximate Σ1 by itsdiagonal.Easy to estimate, and places weight wherewe have best forecasts.Empirically, it gives better forecasts thanother available methods.
yn(h) = S(S′ΛS)−1S′Λyn(h)Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 26
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Optimal combination forecasts
Revised forecasts Base forecasts
Solution 2: WLSSuppose we approximate Σ1 by itsdiagonal.Easy to estimate, and places weight wherewe have best forecasts.Empirically, it gives better forecasts thanother available methods.
yn(h) = S(S′ΛS)−1S′Λyn(h)Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 26
yn(h) = S(S′Σ†hS)−1S′Σ†hyn(h)
Challenges
Computational difficulties in bighierarchies due to size of the S matrix andsingular behavior of (S′ΛS).
Loss of information in ignoring covariancematrix in computing point forecasts.
Still need to estimate covariance matrix toproduce prediction intervals.
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 27
yn(h) = S(S′ΛS)−1S′Λyn(h)
Challenges
Computational difficulties in bighierarchies due to size of the S matrix andsingular behavior of (S′ΛS).
Loss of information in ignoring covariancematrix in computing point forecasts.
Still need to estimate covariance matrix toproduce prediction intervals.
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 27
yn(h) = S(S′ΛS)−1S′Λyn(h)
Challenges
Computational difficulties in bighierarchies due to size of the S matrix andsingular behavior of (S′ΛS).
Loss of information in ignoring covariancematrix in computing point forecasts.
Still need to estimate covariance matrix toproduce prediction intervals.
Visualising and forecasting big time series data BLUF: Best Linear Unbiased Forecasts 27
yn(h) = S(S′ΛS)−1S′Λyn(h)
Outline
1 Examples of biggish time series
2 Time series visualisation
3 BLUF: Best Linear Unbiased Forecasts
4 Application: Australian tourism
5 Fast computation tricks
6 hts package for R
7 References
Visualising and forecasting big time series data Application: Australian tourism 28
Australian tourism
Visualising and forecasting big time series data Application: Australian tourism 29
Australian tourism
Visualising and forecasting big time series data Application: Australian tourism 29
Hierarchy:States (7)
Zones (27)
Regions (82)
Australian tourism
Visualising and forecasting big time series data Application: Australian tourism 29
Hierarchy:States (7)
Zones (27)
Regions (82)
Base forecastsETS (exponentialsmoothing) models
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: Total
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
6000
065
000
7000
075
000
8000
085
000
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: NSW
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
1800
022
000
2600
030
000
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: VIC
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
1000
012
000
1400
016
000
1800
0
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: Nth.Coast.NSW
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
5000
6000
7000
8000
9000
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: Metro.QLD
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
8000
9000
1100
013
000
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: Sth.WA
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
400
600
800
1000
1200
1400
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: X201.Melbourne
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
4000
4500
5000
5500
6000
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: X402.Murraylands
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
010
020
030
0
Base forecasts
Visualising and forecasting big time series data Application: Australian tourism 30
Domestic tourism forecasts: X809.Daly
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
020
4060
8010
0
Reconciled forecasts
Visualising and forecasting big time series data Application: Australian tourism 31
Tota
l
2000 2005 2010
6500
080
000
9500
0
Reconciled forecasts
Visualising and forecasting big time series data Application: Australian tourism 31
NS
W
2000 2005 2010
1800
024
000
3000
0
VIC
2000 2005 20101000
014
000
1800
0
QLD
2000 2005 2010
1400
020
000
Oth
er2000 2005 201018
000
2400
0
Reconciled forecasts
Visualising and forecasting big time series data Application: Australian tourism 31
Syd
ney
2000 2005 20104000
7000
Oth
er N
SW
2000 2005 2010
1400
022
000
Mel
bour
ne
2000 2005 2010
4000
5000
Oth
er V
IC
2000 2005 2010
6000
1200
0
GC
and
Bris
bane
2000 2005 2010
6000
9000
Oth
er Q
LD2000 2005 201060
0012
000
Cap
ital c
ities
2000 2005 2010
1400
020
000
Oth
er
2000 2005 2010
5500
7500
Forecast evaluation
Select models using all observations;
Re-estimate models using first 12observations and generate 1- to8-step-ahead forecasts;
Increase sample size one observation at atime, re-estimate models, generateforecasts until the end of the sample;
In total 24 1-step-ahead, 232-steps-ahead, up to 17 8-steps-ahead forforecast evaluation.
Visualising and forecasting big time series data Application: Australian tourism 32
Forecast evaluation
Select models using all observations;
Re-estimate models using first 12observations and generate 1- to8-step-ahead forecasts;
Increase sample size one observation at atime, re-estimate models, generateforecasts until the end of the sample;
In total 24 1-step-ahead, 232-steps-ahead, up to 17 8-steps-ahead forforecast evaluation.
Visualising and forecasting big time series data Application: Australian tourism 32
Forecast evaluation
Select models using all observations;
Re-estimate models using first 12observations and generate 1- to8-step-ahead forecasts;
Increase sample size one observation at atime, re-estimate models, generateforecasts until the end of the sample;
In total 24 1-step-ahead, 232-steps-ahead, up to 17 8-steps-ahead forforecast evaluation.
Visualising and forecasting big time series data Application: Australian tourism 32
Forecast evaluation
Select models using all observations;
Re-estimate models using first 12observations and generate 1- to8-step-ahead forecasts;
Increase sample size one observation at atime, re-estimate models, generateforecasts until the end of the sample;
In total 24 1-step-ahead, 232-steps-ahead, up to 17 8-steps-ahead forforecast evaluation.
Visualising and forecasting big time series data Application: Australian tourism 32
Hierarchy: states, zones, regions
MAPE h = 1 h = 2 h = 4 h = 6 h = 8 AverageTop Level: Australia
Bottom-up 3.79 3.58 4.01 4.55 4.24 4.06OLS 3.83 3.66 3.88 4.19 4.25 3.94WLS 3.68 3.56 3.97 4.57 4.25 4.04Level: States
Bottom-up 10.70 10.52 10.85 11.46 11.27 11.03OLS 11.07 10.58 11.13 11.62 12.21 11.35WLS 10.44 10.17 10.47 10.97 10.98 10.67Level: Zones
Bottom-up 14.99 14.97 14.98 15.69 15.65 15.32OLS 15.16 15.06 15.27 15.74 16.15 15.48WLS 14.63 14.62 14.68 15.17 15.25 14.94Bottom Level: Regions
Bottom-up 33.12 32.54 32.26 33.74 33.96 33.18OLS 35.89 33.86 34.26 36.06 37.49 35.43WLS 31.68 31.22 31.08 32.41 32.77 31.89
Visualising and forecasting big time series data Application: Australian tourism 33
Outline
1 Examples of biggish time series
2 Time series visualisation
3 BLUF: Best Linear Unbiased Forecasts
4 Application: Australian tourism
5 Fast computation tricks
6 hts package for R
7 References
Visualising and forecasting big time series data Fast computation tricks 34
Fast computation: hierarchical data
Total
A
AX AY AZ
B
BX BY BZ
C
CX CY CZ
yt =
YtYA,tYB,tYC,tYAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
=
1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1
︸ ︷︷ ︸
S
YAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
︸ ︷︷ ︸
Bt
Visualising and forecasting big time series data Fast computation tricks 35
yt = SBt
Fast computation: hierarchical data
Total
A
AX AY AZ
B
BX BY BZ
C
CX CY CZ
yt =
YtYA,tYAX,tYAY,tYAZ,tYB,tYBX,tYBY,tYBZ,tYC,tYCX,tYCY,tYCZ,t
=
1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 01 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 1 10 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1
︸ ︷︷ ︸
S
YAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t
︸ ︷︷ ︸
Bt
Visualising and forecasting big time series data Fast computation tricks 36
yt = SBt
Fast computation: hierarchies
Think of the hierarchy as a tree of trees:
Total
T1 T2 . . . TK
Then the summing matrix contains k smaller summingmatrices:
S =
1′n1
1′n2· · · 1′nK
S1 0 · · · 00 S2 · · · 0...
... . . . ...0 0 · · · SK
where 1n is an n-vector of ones and tree Ti has niterminal nodes.
Visualising and forecasting big time series data Fast computation tricks 37
Fast computation: hierarchies
Think of the hierarchy as a tree of trees:
Total
T1 T2 . . . TK
Then the summing matrix contains k smaller summingmatrices:
S =
1′n1
1′n2· · · 1′nK
S1 0 · · · 00 S2 · · · 0...
... . . . ...0 0 · · · SK
where 1n is an n-vector of ones and tree Ti has niterminal nodes.
Visualising and forecasting big time series data Fast computation tricks 37
Fast computation: hierarchies
S′ΛS =
S′1Λ1S1 0 · · · 0
0 S′2Λ2S2 · · · 0... ... . . . ...0 0 · · · S′KΛKSK
+λ0 Jn
λ0 is the top left element of Λ;Λk is a block of Λ, corresponding to tree Tk;Jn is a matrix of ones;n =
∑k nk.
Now apply the Sherman-Morrison formula . . .
Visualising and forecasting big time series data Fast computation tricks 38
Fast computation: hierarchies
S′ΛS =
S′1Λ1S1 0 · · · 0
0 S′2Λ2S2 · · · 0... ... . . . ...0 0 · · · S′KΛKSK
+λ0 Jn
λ0 is the top left element of Λ;Λk is a block of Λ, corresponding to tree Tk;Jn is a matrix of ones;n =
∑k nk.
Now apply the Sherman-Morrison formula . . .
Visualising and forecasting big time series data Fast computation tricks 38
Fast computation: hierarchies
(S′ΛS)−1 =
(S′1Λ1S1)
−1 0 · · · 00 (S′2Λ2S2)
−1 · · · 0...
.... . .
...0 0 · · · (S′KΛKSK)
−1
−cS0
S0 can be partitioned into K2 blocks, with the (k, `)block (of dimension nk × n`) being
(S′kΛkSk)−1Jnk,n`(S
′`Λ`S`)
−1
Jnk,n` is a nk × n` matrix of ones.
c−1 = λ−10 +
∑k
1′nk(S′kΛkSk)
−11nk .
Each S′kΛkSk can be inverted similarly.S′Λy can also be computed recursively.
Visualising and forecasting big time series data Fast computation tricks 39
Fast computation: hierarchies
(S′ΛS)−1 =
(S′1Λ1S1)
−1 0 · · · 00 (S′2Λ2S2)
−1 · · · 0...
.... . .
...0 0 · · · (S′KΛKSK)
−1
−cS0
S0 can be partitioned into K2 blocks, with the (k, `)block (of dimension nk × n`) being
(S′kΛkSk)−1Jnk,n`(S
′`Λ`S`)
−1
Jnk,n` is a nk × n` matrix of ones.
c−1 = λ−10 +
∑k
1′nk(S′kΛkSk)
−11nk .
Each S′kΛkSk can be inverted similarly.S′Λy can also be computed recursively.
Visualising and forecasting big time series data Fast computation tricks 39
The recursive calculations can bedone in such a way that we neverstore any of the large matricesinvolved.
Fast computation
A similar algorithm has been developed forgrouped time series with two groups.When the time series are not strictlyhierarchical and have more than two groupingvariables:
Use sparse matrix storage and arithmetic.
Use iterative approximation for invertinglarge sparse matrices.
Paige & Saunders (1982)ACM Trans. Math. Software
Visualising and forecasting big time series data Fast computation tricks 40
Fast computation
A similar algorithm has been developed forgrouped time series with two groups.When the time series are not strictlyhierarchical and have more than two groupingvariables:
Use sparse matrix storage and arithmetic.
Use iterative approximation for invertinglarge sparse matrices.
Paige & Saunders (1982)ACM Trans. Math. Software
Visualising and forecasting big time series data Fast computation tricks 40
Fast computation
A similar algorithm has been developed forgrouped time series with two groups.When the time series are not strictlyhierarchical and have more than two groupingvariables:
Use sparse matrix storage and arithmetic.
Use iterative approximation for invertinglarge sparse matrices.
Paige & Saunders (1982)ACM Trans. Math. Software
Visualising and forecasting big time series data Fast computation tricks 40
Outline
1 Examples of biggish time series
2 Time series visualisation
3 BLUF: Best Linear Unbiased Forecasts
4 Application: Australian tourism
5 Fast computation tricks
6 hts package for R
7 References
Visualising and forecasting big time series data hts package for R 41
hts package for R
Visualising and forecasting big time series data hts package for R 42
hts: Hierarchical and grouped time seriesMethods for analysing and forecasting hierarchical and groupedtime series
Version: 4.5Depends: forecast (≥ 5.0), SparseMImports: parallel, utilsPublished: 2014-12-09Author: Rob J Hyndman, Earo Wang and Alan LeeMaintainer: Rob J Hyndman <Rob.Hyndman at monash.edu>BugReports: https://github.com/robjhyndman/hts/issuesLicense: GPL (≥ 2)
Example using Rlibrary(hts)
# bts is a matrix containing the bottom level time series# nodes describes the hierarchical structurey <- hts(bts, nodes=list(2, c(3,2)))
Visualising and forecasting big time series data hts package for R 43
Example using Rlibrary(hts)
# bts is a matrix containing the bottom level time series# nodes describes the hierarchical structurey <- hts(bts, nodes=list(2, c(3,2)))
Visualising and forecasting big time series data hts package for R 43
Total
A
AX AY AZ
B
BX BY
Example using Rlibrary(hts)
# bts is a matrix containing the bottom level time series# nodes describes the hierarchical structurey <- hts(bts, nodes=list(2, c(3,2)))
# Forecast 10-step-ahead using WLS combination method# ETS used for each series by defaultfc <- forecast(y, h=10)
Visualising and forecasting big time series data hts package for R 44
forecast.gts functionUsageforecast(object, h,method = c("comb", "bu", "mo", "tdgsf", "tdgsa", "tdfp"),fmethod = c("ets", "rw", "arima"),weights = c("sd", "none", "nseries"),positive = FALSE,parallel = FALSE, num.cores = 2, ...)
Argumentsobject Hierarchical time series object of class gts.h Forecast horizonmethod Method for distributing forecasts within the hierarchy.fmethod Forecasting method to usepositive If TRUE, forecasts are forced to be strictly positiveweights Weights used for "optimal combination" method. When
weights = "sd", it takes account of the standard deviation offorecasts.
parallel If TRUE, allow parallel processingnum.cores If parallel = TRUE, specify how many cores are going to be
used
Visualising and forecasting big time series data hts package for R 45
Outline
1 Examples of biggish time series
2 Time series visualisation
3 BLUF: Best Linear Unbiased Forecasts
4 Application: Australian tourism
5 Fast computation tricks
6 hts package for R
7 References
Visualising and forecasting big time series data References 46
ReferencesRJ Hyndman, RA Ahmed, G Athanasopoulos, andHL Shang (2011). “Optimal combination forecasts forhierarchical time series”. Computational statistics &data analysis 55(9), 2579–2589.RJ Hyndman, AJ Lee, and E Wang (2014). Fastcomputation of reconciled forecasts for hierarchicaland grouped time series. Working paper 17/14.Department of Econometrics & Business Statistics,Monash UniversityRJ Hyndman, AJ Lee, and E Wang (2014). hts:Hierarchical and grouped time series.cran.r-project.org/package=hts.RJ Hyndman and G Athanasopoulos (2014).Forecasting: principles and practice. OTexts.OTexts.org/fpp/.
Visualising and forecasting big time series data References 47
ReferencesRJ Hyndman, RA Ahmed, G Athanasopoulos, andHL Shang (2011). “Optimal combination forecasts forhierarchical time series”. Computational statistics &data analysis 55(9), 2579–2589.RJ Hyndman, AJ Lee, and E Wang (2014). Fastcomputation of reconciled forecasts for hierarchicaland grouped time series. Working paper 17/14.Department of Econometrics & Business Statistics,Monash UniversityRJ Hyndman, AJ Lee, and E Wang (2014). hts:Hierarchical and grouped time series.cran.r-project.org/package=hts.RJ Hyndman and G Athanasopoulos (2014).Forecasting: principles and practice. OTexts.OTexts.org/fpp/.
Visualising and forecasting big time series data References 47
å Papers and R code:
robjhyndman.com
å Email: [email protected]