exploring the feature space of large collections of time series

Page 1

Exploring the feature space of large time series collections 1

Exploring thefeature space oflarge collectionsof time series

Rob J Hyndman

Page 2

Exploring the feature space of large time series collections 1

Exploring thefeature space oflarge collectionsof time series

Rob J Hyndman

with Earo Wang, Nikolay LaptevYanfei Kang, Kate Smith-Miles

Page 3

Outline

1 M3 competition data

2 Yahoo web traffic

3 What next?

Exploring the feature space of large time series collections M3 competition data 2

Page 4

M3 forecasting competition

Exploring the feature space of large time series collections M3 competition data 3

Page 5

M3 forecasting competition

Exploring the feature space of large time series collections M3 competition data 3

Page 6

M3 forecasting competition

“The M3-Competition is a final attempt by the authors tosettle the accuracy issue of various time series methods. . .The extension involves the inclusion of more methods/researchers (in particular in the areas of neural networksand expert systems) and more series.”

Makridakis & Hibon, IJF 2000

3003 series

All data from business, demography, finance andeconomics.

Series length between 14 and 126.

Either non-seasonal, monthly or quarterly.

All time series positive.

Exploring the feature space of large time series collections M3 competition data 4

Page 7

M3 forecasting competition

Exploring the feature space of large time series collections M3 competition data 5

●

Page 8

Key idea

Examples for time series

lag correlationsize and direction of trendstrength of seasonalitytiming of peak seasonalityspectral entropy

Called “features” or “characteristics” in themachine learning literature.

Exploring the feature space of large time series collections M3 competition data 6

John W Tukey

Cognostics

Computer-produced diagnostics(Tukey and Tukey, 1985).

Page 9

Key idea

Examples for time series

lag correlationsize and direction of trend

strength of seasonalitytiming of peak seasonalityspectral entropy

Called “features” or “characteristics” in themachine learning literature.

Exploring the feature space of large time series collections M3 competition data 6

John W Tukey

Cognostics

Computer-produced diagnostics(Tukey and Tukey, 1985).

Page 10

Key idea

Examples for time series

lag correlationsize and direction of trend

strength of seasonalitytiming of peak seasonalityspectral entropy

Called “features” or “characteristics” in themachine learning literature.

Exploring the feature space of large time series collections M3 competition data 6

John W Tukey

Cognostics

Computer-produced diagnostics(Tukey and Tukey, 1985).

Page 11

An STL decompositionQuarterly visitor nights: Mornington Peninsula

Yt = St + Tt + Rt St is periodic with mean 0

5.0

6.0

7.0

data

−0.

50.

5

seas

onal

5.8

6.1

6.4

tren

d

−0.

40.

0

2000 2005 2010

rem

aind

er

timeExploring the feature space of large time series collections M3 competition data 7

Page 12

Candidate features

STL decompositionYt = St + Tt + Rt

Seasonal period

Strength of seasonality: 1− Var(Rt)Var(Yt−Tt)

Strength of trend: 1− Var(Rt)Var(Yt−St)

Spectral entropy: H = −∫ π−π fy(λ) log fy(λ)dλ,

where fy(λ) is spectral density of Yt.Low values of H suggest a time series that iseasier to forecast (more signal).

Autocorrelations: r1, r2, r3, . . .

Optimal Box-Cox transformation parameter λExploring the feature space of large time series collections M3 competition data 8

Page 13

Candidate features

STL decompositionYt = St + Tt + Rt

Seasonal period

Strength of seasonality: 1− Var(Rt)Var(Yt−Tt)

Strength of trend: 1− Var(Rt)Var(Yt−St)

Spectral entropy: H = −∫ π−π fy(λ) log fy(λ)dλ,

where fy(λ) is spectral density of Yt.Low values of H suggest a time series that iseasier to forecast (more signal).

Autocorrelations: r1, r2, r3, . . .

Optimal Box-Cox transformation parameter λExploring the feature space of large time series collections M3 competition data 8

Page 14

Candidate features

STL decompositionYt = St + Tt + Rt

Seasonal period

Strength of seasonality: 1− Var(Rt)Var(Yt−Tt)

Strength of trend: 1− Var(Rt)Var(Yt−St)

Spectral entropy: H = −∫ π−π fy(λ) log fy(λ)dλ,

where fy(λ) is spectral density of Yt.Low values of H suggest a time series that iseasier to forecast (more signal).

Autocorrelations: r1, r2, r3, . . .

Optimal Box-Cox transformation parameter λExploring the feature space of large time series collections M3 competition data 8

Page 15

Candidate features

STL decompositionYt = St + Tt + Rt

Seasonal period

Strength of seasonality: 1− Var(Rt)Var(Yt−Tt)

Strength of trend: 1− Var(Rt)Var(Yt−St)

Spectral entropy: H = −∫ π−π fy(λ) log fy(λ)dλ,

where fy(λ) is spectral density of Yt.Low values of H suggest a time series that iseasier to forecast (more signal).

Autocorrelations: r1, r2, r3, . . .

Optimal Box-Cox transformation parameter λExploring the feature space of large time series collections M3 competition data 8

Page 16

Candidate features

STL decompositionYt = St + Tt + Rt

Seasonal period

Strength of seasonality: 1− Var(Rt)Var(Yt−Tt)

Strength of trend: 1− Var(Rt)Var(Yt−St)

Spectral entropy: H = −∫ π−π fy(λ) log fy(λ)dλ,

where fy(λ) is spectral density of Yt.Low values of H suggest a time series that iseasier to forecast (more signal).

Autocorrelations: r1, r2, r3, . . .

Optimal Box-Cox transformation parameter λExploring the feature space of large time series collections M3 competition data 8

Page 17

Candidate features

STL decompositionYt = St + Tt + Rt

Seasonal period

Strength of seasonality: 1− Var(Rt)Var(Yt−Tt)

Strength of trend: 1− Var(Rt)Var(Yt−St)

Spectral entropy: H = −∫ π−π fy(λ) log fy(λ)dλ,

where fy(λ) is spectral density of Yt.Low values of H suggest a time series that iseasier to forecast (more signal).

Autocorrelations: r1, r2, r3, . . .

Optimal Box-Cox transformation parameter λExploring the feature space of large time series collections M3 competition data 8

Page 18

Candidate features

STL decompositionYt = St + Tt + Rt

Seasonal period

Strength of seasonality: 1− Var(Rt)Var(Yt−Tt)

Strength of trend: 1− Var(Rt)Var(Yt−St)

Spectral entropy: H = −∫ π−π fy(λ) log fy(λ)dλ,

where fy(λ) is spectral density of Yt.Low values of H suggest a time series that iseasier to forecast (more signal).

Autocorrelations: r1, r2, r3, . . .

Optimal Box-Cox transformation parameter λExploring the feature space of large time series collections M3 competition data 8

Page 19

Dimension reduction for time series

Exploring the feature space of large time series collections M3 competition data 9●

Page 20

Dimension reduction for time series

Exploring the feature space of large time series collections M3 competition data 9●

SpecEntr

0.0 0.4 0.8 2 6 10 0.0 0.4 0.8

0.5

0.9

0.0

0.6

Trend

Season

0.0

0.6

28 Freq

ACF

−0.

40.

6

0.5 0.7 0.9

0.0

0.6

0.0 0.4 0.8 −0.4 0.2 0.8

Lambda

Featurecalculation

Page 21

Dimension reduction for time series

Exploring the feature space of large time series collections M3 competition data 9●

SpecEntr

0.0 0.4 0.8 2 6 10 0.0 0.4 0.8

0.5

0.9

0.0

0.6

Trend

Season

0.0

0.6

28 Freq

ACF

−0.

40.

6

0.5 0.7 0.9

0.0

0.6

0.0 0.4 0.8 −0.4 0.2 0.8

Lambda

● ●●

●

●

●●● ●●

● ●

●

●

●

●●

●

●

●

●

●

●

●● ●●●

●

●●●

●

●●● ●

●●● ●

●● ●●● ●●●●

●

●●

●●●●●●

●

●●●●

●●●

●

●

●●

●

●●●

●●●

● ●

●

●

●●

●

●

●

●●●

●●

●●

●

●●

●●

●●●●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●●●● ●●

●

●

●●●

●●●●●●●

●●●

●

●

●●

●●●

●●●●

●●●●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●●

●●

●●

●

●●

●●

●

●

●

●

●

● ●

●●●●●

●● ●

●

●

●

●

●

● ●

●

●

●●

●

●

●●

●●

●

●●

●●

●

●●●●●●●●

●

●●●●●

●●●●●

●●●●●●●●●●

●

●

●● ●

●●●

●

●●● ●

●● ●

●●

●●●

●●●●

●●●●●

● ●●●●●●● ●

●●

●●●

●● ●●●●

●

●

●●●

●●

●

●

●●●● ●

●●

●

●●●

●●●

●

●●●●●●●●

●●●

●

●

●

●

●

● ●●

●

●●●

●●●●

●

●●●

●●●

●

●●

●

●

●●

●

●

●●●●●

●●

●●

●

●

●●●

● ●●

●

●●

●●

●

● ●

● ●●●

●

●

●●●●●●

●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●

●

●●●

●●

●

●●●

●

●●●

●●●●●

●

●

●●●

●

● ●●●●

●●

●

●●●●●●

●●●●

● ●●

●●

●●

●●

●●

●●

●●● ●

●●

●●●●

●●

●

●

●

●●●

●

●●●● ●●

●●

●

●●

●

●

●●

●

●●

●●●

● ●●

●

●

●●●

●

●●

●●● ●●

●●● ●●●

●

●●

●

●●

●●●

●

●●

●

●

●●● ●●

●

●●

●●

●●

●●●●●●

●●●

●

●●

●

●●●●●●●●●

●

●

●

●●

●

●

●

●●

●

●

● ● ●●●

●

●●

●

●

●● ●●●

●

●

●

●

●

●

●●

●

●● ●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●●●

● ●●

●

● ●

●

● ●●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●● ●

●

●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●●●●

●●

●●

●●

●

●●

●

●

●● ●●●● ●

● ●●

●

●

●

●

●

●● ●● ●

●

●●●

●

●

●●● ●

●● ●

●●

●

●

●● ●●

●●●

●●

●●●●● ●

●●●

●

●●

●

●●

●

●●●●

●●●

●

●

● ●●●●●●

●

●●

●●● ●●●●●● ●●●

●

●●●

●

●

●● ●●●●● ●

●●

●

●●●●

●●

●

●● ●

●

● ●

●

●

●●

●●

●

●

●

● ●

●● ●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●●●●

●

●●●

●

●●●

●

●●● ●●

●●

●●●

●●●

●●●

●

●

●●●

●●

●

●●

●

●

●●●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●●●

●●

●●

●

●

●●

●

●●

● ●

●

●

●

●●

●

●● ●●●

●

●●●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●●●●●

●

●● ●●

●●

●●●

●●

●

●

●● ●

●●

●●●

●

●

●

●

●●●

●

●● ●

●

●●●

●●●●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●●● ●●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●●

●●

●● ●

●●

●

●

●

●●

●

●●

●●

●● ●

●●●

●

●

●●

●

●●

●●

●●●

●●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

● ●

●

● ●

●●

●

●

●

●

●

●

●

●●

●●●

●

●●

● ●●

●

●

●●

●●

● ●

●

●●

●

●●

●

●●

●

●●

●

●

●

●●●

●●

●● ●

●

●●

●

●

●

●

●

●●

●

● ●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●● ●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●●●

●

●

●

●●

●●

●

●●

●●●

●

●

●●

●●

●●

●●

●

● ●● ●

●●

●● ●●

●●

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●●

●

●

●

●

●●

● ●●●

●

●

●●

●

●

●

● ●● ●

●●

●

●

●●●

●

●

●

●●

●

●●

●●

●

●●

●●●

●

●

●●●

●

● ●

● ●●

● ●

●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●●● ●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

● ●●

●●

●

● ●●

●

●●

●

● ●● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

● ●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●●

●

●

● ●●

●●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

● ●●

●

●

●

●●

●

●

● ●

●

●

●●●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●●

●

●

●●●

●

●

●

● ●

●

●

●●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●●

●

● ●●

●

●●●

●●

●

●

●

●

●

●

●●

●

●●

●

●●●

●

●●

●

●●

●

●

●

●

●● ●●●●

●●

● ●

●

●●

●

●

●●

●●

●

●

●●

●

●

●●

●

●

●●

●

●●

●●

●

●●

● ●

● ●

●

●●

●●

●●

●●

●●

●

●●

●

●

●

● ●

●●

●●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●● ●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

● ●●● ●●

● ●●

●

●

●

●

●●●●

●

●

●●

●

●●

●

●●

●●●●●

●

●

●●●● ●

●

●●●● ●

●

●

●

●

●

● ●●●●●

●●

●

●

●●

●●

●●

●

●●

●

●●

●●

●

●

● ●

●

●● ●

●

●

●

●●●

●

●

●

●

●● ●●● ●●●● ●●●●

●

●●

●●●●●●

●

●●●●

●●

●●

● ●●

●

●

●

●●●

●

●

●●

●

●

● ●●

●

●●●● ●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●●

●

●●

●

●●●●●

●

●●

● ●

● ●

●

●●

●●●

●●

●

●●●

●●

● ●●

●●●

●

●

●

●

●●

● ●●

●●●

●

●

●●●

●●

●

●

●●

● ●●

●

●

●●

●

●●

●●●

●●

●

●

●

●●

●

●●●

●●

●

●

●●

●● ●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●●●●

●

●

●●

●

●●●

●

●●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●●● ●●

● ●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

● ●

●

●●

●

●

●●

●●●

● ●

●

●●

●

●●●●●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●●

● ●●●

●

●●

●

●

●

●

●

● ●

● ●

●●

●

●●

●●

●●●

●●

●● ● ●●●

●●

●●●●●

●

●●●●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●●●●●

●●

●

●●●

●●

●●

●●●●●

●

●

●

●

●

● ●

●●●●●●

●●

●

●●●●● ●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●●

●

● ●● ●●

●

●●●●

●●

●

●

●

●●●

●

●

●

● ●●

●●

●

●●

● ●● ●●●

●●

●●●

● ●●●

●●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●●

●●●

●

●●

●

●●●

●

●●●●

●

●

●

●

●●●

●●●

●

●

●

●●

● ●●

●

●● ●● ● ●

●

●● ●●

●●●

●

●

●

●

●●●●

●●●

●●●

●

●

●

●●

● ●●●●●●●●●

●

●●●●●●●●●●●●●●●●●

● ●●

●●

●

●

●●●●●

●●●

●

●

●●●●●●

●

●

●●●●

●●

●

●●

●

●

●●

●

●●

●●●●●●

●●●●●

●

●●

●●●●●●●

●●●

●

●●

●

●●

●●●

●●

−3

−2

−1

0

1

2

3

−2 0 2 4PC1

PC

2

Featurecalculation

Principalcomponentdecomposition

Page 22

Feature space of M3 data

Exploring the feature space of large time series collections M3 competition data 10

● ●●

●

●

●●● ●●

● ●

●

●

●

●●

●

●

●

●

●

●

●● ●●●

●

●●●

●

●●● ●

●●● ●

●● ●●● ●●●●

●

●●

●●●●●●

●

●●●●

●●●

●

●

●●

●

●●●

●●●

● ●

●

●

●●

●

●

●

●●●

●●

●●

●

●●

●●

●●●●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●●●● ●●

●

●

●●●

●●●●●●●

●●●

●

●

●●

●●●

●●●●

●●●●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●●

●●

●●

●

●●

●●

●

●

●

●

●

● ●

●●●●●

●● ●

●

●

●

●

●

● ●

●

●

●●

●

●

●●

●●

●

●●

●●

●

●●●●●●●●

●

●●●●●

●●●●●

●●●●●●●●●●

●

●

●● ●

●●●

●

●●● ●

●● ●

●●

●●●

●●●●

●●●●●

● ●●●●●●● ●

●●

●●●

●● ●●●●

●

●

●●●

●●

●

●

●●●● ●

●●

●

●●●

●●●

●

●●●●●●●●

●●●

●

●

●

●

●

● ●●

●

●●●

●●●●

●

●●●

●●●

●

●●

●

●

●●

●

●

●●●●●

●●

●●

●

●

●●●

● ●●

●

●●

●●

●

● ●

● ●●●

●

●

●●●●●●

●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●

●

●●●

●●

●

●●●

●

●●●

●●●●●

●

●

●●●

●

● ●●●●

●●

●

●●●●●●

●●●●

● ●●

●●

●●

●●

●●

●●

●●● ●

●●

●●●●

●●

●

●

●

●●●

●

●●●● ●●

●●

●

●●

●

●

●●

●

●●

●●●

● ●●

●

●

●●●

●

●●

●●● ●●

●●● ●●●

●

●●

●

●●

●●●

●

●●

●

●

●●● ●●

●

●●

●●

●●

●●●●●●

●●●

●

●●

●

●●●●●●●●●

●

●

●

●●

●

●

●

●●

●

●

● ● ●●●

●

●●

●

●

●● ●●●

●

●

●

●

●

●

●●

●

●● ●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●●●

● ●●

●

● ●

●

● ●●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●● ●

●

●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●●●●

●●

●●

●●

●

●●

●

●

●● ●●●● ●

● ●●

●

●

●

●

●

●● ●● ●

●

●●●

●

●

●●● ●

●● ●

●●

●

●

●● ●●

●●●

●●

●●●●● ●

●●●

●

●●

●

●●

●

●●●●

●●●

●

●

● ●●●●●●

●

●●

●●● ●●●●●● ●●●

●

●●●

●

●

●● ●●●●● ●

●●

●

●●●●

●●

●

●● ●

●

● ●

●

●

●●

●●

●

●

●

● ●

●● ●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●●●●

●

●●●

●

●●●

●

●●● ●●

●●

●●●

●●●

●●●

●

●

●●●

●●

●

●●

●

●

●●●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●●●

●●

●●

●

●

●●

●

●●

● ●

●

●

●

●●

●

●● ●●●

●

●●●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●●●●●

●

●● ●●

●●

●●●

●●

●

●

●● ●

●●

●●●

●

●

●

●

●●●

●

●● ●

●

●●●

●●●●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●●● ●●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●●

●●

●● ●

●●

●

●

●

●●

●

●●

●●

●● ●

●●●

●

●

●●

●

●●

●●

●●●

●●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

● ●

●

● ●

●●

●

●

●

●

●

●

●

●●

●●●

●

●●

● ●●

●

●

●●

●●

● ●

●

●●

●

●●

●

●●

●

●●

●

●

●

●●●

●●

●● ●

●

●●

●

●

●

●

●

●●

●

● ●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●● ●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●●●

●

●

●

●●

●●

●

●●

●●●

●

●

●●

●●

●●

●●

●

● ●● ●

●●

●● ●●

●●

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●●

●

●

●

●

●●

● ●●●

●

●

●●

●

●

●

● ●● ●

●●

●

●

●●●

●

●

●

●●

●

●●

●●

●

●●

●●●

●

●

●●●

●

● ●

● ●●

● ●

●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●●● ●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

● ●●

●●

●

● ●●

●

●●

●

● ●● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

● ●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●●

●

●

● ●●

●●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

● ●●

●

●

●

●●

●

●

● ●

●

●

●●●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●●

●

●

●●●

●

●

●

● ●

●

●

●●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●●

●

● ●●

●

●●●

●●

●

●

●

●

●

●

●●

●

●●

●

●●●

●

●●

●

●●

●

●

●

●

●● ●●●●

●●

● ●

●

●●

●

●

●●

●●

●

●

●●

●

●

●●

●

●

●●

●

●●

●●

●

●●

● ●

● ●

●

●●

●●

●●

●●

●●

●

●●

●

●

●

● ●

●●

●●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●● ●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

● ●●● ●●

● ●●

●

●

●

●

●●●●

●

●

●●

●

●●

●

●●

●●●●●

●

●

●●●● ●

●

●●●● ●

●

●

●

●

●

● ●●●●●

●●

●

●

●●

●●

●●

●

●●

●

●●

●●

●

●

● ●

●

●● ●

●

●

●

●●●

●

●

●

●

●● ●●● ●●●● ●●●●

●

●●

●●●●●●

●

●●●●

●●

●●

● ●●

●

●

●

●●●

●

●

●●

●

●

● ●●

●

●●●● ●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●●

●

●●

●

●●●●●

●

●●

● ●

● ●

●

●●

●●●

●●

●

●●●

●●

● ●●

●●●

●

●

●

●

●●

● ●●

●●●

●

●

●●●

●●

●

●

●●

● ●●

●

●

●●

●

●●

●●●

●●

●

●

●

●●

●

●●●

●●

●

●

●●

●● ●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●●●●

●

●

●●

●

●●●

●

●●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●●● ●●

● ●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

● ●

●

●●

●

●

●●

●●●

● ●

●

●●

●

●●●●●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●●

● ●●●

●

●●

●

●

●

●

●

● ●

● ●

●●

●

●●

●●

●●●

●●

●● ● ●●●

●●

●●●●●

●

●●●●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●●●●●

●●

●

●●●

●●

●●

●●●●●

●

●

●

●

●

● ●

●●●●●●

●●

●

●●●●● ●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●●

●

● ●● ●●

●

●●●●

●●

●

●

●

●●●

●

●

●

● ●●

●●

●

●●

● ●● ●●●

●●

●●●

● ●●●

●●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●●

●●●

●

●●

●

●●●

●

●●●●

●

●

●

●

●●●

●●●

●

●

●

●●

● ●●

●

●● ●● ● ●

●

●● ●●

●●●

●

●

●

●

●●●●

●●●

●●●

●

●

●

●●

● ●●●●●●●●●

●

●●●●●●●●●●●●●●●●●

● ●●

●●

●

●

●●●●●

●●●

●

●

●●●●●●

●

●

●●●●

●●

●

●●

●

●

●●

●

●●

●●●●●●

●●●●●

●

●●

●●●●●●●

●●●

●

●●

●

●●

●●●

●●

−3

−2

−1

0

1

2

3

−2 0 2 4PC1

PC

2

First two PCs explain 68% of variation.

Page 23

Feature space of M3 data

Exploring the feature space of large time series collections M3 competition data 10

● ●●

●

●

●●● ●●

● ●

●

●

●

●●

●

●

●

●

●

●

●●●●●

●

●●●

●

●●● ●

●●● ●

●● ●●● ●●●●

●

●●

●●●●●●

●

●●●●

●●●

●

●

●●

●

●●●

●●●

● ●

●

●

●●

●

●

●

●●●

●●

●●

●

●●

●●

●●●●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●●● ●●

●

●

●●●

●●●●●●●

●●●

●

●

●●

●●●

●●●●

●●●●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●●

●●

●●

●

●●

●●

●

●

●

●

●

●●

●●●●●

●● ●

●

●

●

●

●

● ●

●

●

●●

●

●

●●

●●

●

●●

●●

●

●●●●●●●●

●

●●●●●

●●●●●

●●●●●●●●●●

●

●

●●●

●●●

●

●●● ●

●● ●

●●

●●●

●●●●

●●●●●

● ●●●●●●● ●

●●

●●●

●● ●●●●

●

●

●●●

●●

●

●

●●●● ●

●●

●

●●●

●●●

●

●●●●●●●●

●●●

●

●

●

●

●

● ●●

●

●●●

●●●●

●

●●●

●●●

●

●●

●

●

●●

●

●

●●●●●

●●

●●

●

●

●●●

● ●●

●

●●

●●

●

● ●

● ●●●

●

●

●●●●●●

●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●

●

●●●

●●

●

●●●

●

●●●

●●●●●

●

●

●●●

●

● ●●●●

●●

●

●●●●●●

●●●●

● ●●

●●

●●

●●

●●

●●

●●●●

●●

●●●●

●●

●

●

●

●●●

●

●●●● ●●

●●

●

●●

●

●

●●

●

●●

●●●

● ●●

●

●

●●●

●

●●

●●●●●

●●● ●●●

●

●●

●

●●

●●●

●

●●

●

●

●●● ●●

●

●●

●●

●●

●●●●●●

●●●

●

●●

●

●●●●●●●●●

●

●

●

●●

●

●

●

●●

●

●

● ● ●●●

●

●●

●

●

●● ●●●

●

●

●

●

●

●

●●

●

●● ●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●●●

● ●●

●

● ●

●

● ●●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●●●●

●●

●●

●●

●

●●

●

●

●●●●●● ●

● ●●

●

●

●

●

●

●● ●● ●

●

●●●

●

●

●●● ●

●● ●●●

●

●

●● ●●

●●●

●●

●●●●● ●

●●●

●

●●

●

●●

●

●●●●

●●●

●

●

● ●●●●●●

●

●●

●●● ●●●●●● ●●●

●

●●●

●

●

●● ●●●●● ●

●●

●

●●●●

●●

●

●● ●

●

●●

●

●

●●

●●

●

●

●

● ●

●● ●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●●●●

●

●●●

●

●●●

●

●●● ●●

●●

●●●

●●●

●●●

●

●

●●●

●●

●

●●

●

●

●●●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●●●

●●

●●

●

●

●●

●

●●

● ●

●

●

●

●●

●

●● ●●●

●

●●●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●●●●●

●

●● ●●

●●

●●●

●●

●

●

●● ●

●●

●●●

●

●

●

●

●●●

●

●●●

●

●●●

●●●●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●●● ●●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●●

●●

●● ●

●●

●

●

●

●●

●

●●

●●

●● ●

●●●

●

●

●●

●

●●

●●

●●●

●●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

● ●

●

● ●

●●

●

●

●

●

●

●

●

●●

●●●

●

●●

● ●●

●

●

●●

●●

●●

●

●●

●

●●

●

●●

●

●●

●

●

●

●●●

●●

●● ●

●

●●

●

●

●

●

●

●●

●

● ●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●● ●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●●●

●

●

●

●●

●●

●

●●

●●●

●

●

●●

●●

●●

●●

●

● ●● ●

●●

●● ●●

●●

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●●

●

●

●

●

●●

● ●●●

●

●

●●

●

●

●

● ●●●

●●

●

●

●●●

●

●

●

●●

●

●●

●●

●

●●

●●●

●

●

●●●

●

● ●

● ●●

● ●

●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●●● ●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●●●

●●

●

● ●●

●

●●

●

● ●● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

● ●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●●

●

●

● ●●

●●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

● ●●

●

●

●

●●

●

●

● ●

●

●

●●●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●●

●

●

●●●

●

●

●

● ●

●

●

●●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●●

●

● ●●

●

●●●

●●

●

●

●

●

●

●

●●

●

●●

●

●●●

●

●●

●

●●

●

●

●

●

●● ●●●●

●●

● ●

●

●●

●

●

●●

●●

●

●

●●

●

●

●●

●

●

●●

●

●●

●●

●

●●

● ●

● ●

●

●●

●●

●●

●●

●●

●

●●

●

●

●

● ●

●●

●●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●● ●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

● ●●●●●

● ●●

●

●

●

●

●●●●

●

●

●●

●

●●

●

●●

●●●●●

●

●

●●●● ●

●

●●●● ●

●

●

●

●

●

● ●●●●●

●●

●

●

●●

●●

●●

●

●●

●

●●

●●

●

●

● ●

●

●● ●

●

●

●

●●●

●

●

●

●

●● ●●● ●●●●●●●●

●

●●

●●●●●●

●

●●●●

●●

●●

● ●●

●

●

●

●●●

●

●

●●

●

●

● ●●

●

●●●● ●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●●

●

●●

●

●●●●●

●

●●

● ●

● ●

●

●●

●●●

●●

●

●●●

●●

● ●●

●●●

●

●

●

●

●●

● ●●

●●●

●

●

●●●

●●

●

●

●●

● ●●

●

●

●●

●

●●

●●●

●●

●

●

●

●●

●

●●●

●●

●

●

●●

●● ●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●●●●

●

●

●●

●

●●●

●

●●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●●● ●●

● ●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

● ●

●

●●

●

●

●●

●●●

● ●

●

●●

●

●●●●●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●●

● ●●●

●

●●

●

●

●

●

●

● ●

● ●

●●

●

●●

●●

●●●

●●

●● ● ●●●

●●

●●●●●

●

●●●●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●●●●●●●

●

●●●

●●

●●

●●●●●

●

●

●

●

●

● ●

●●●●●●

●●

●

●●●●● ●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●●

●

● ●● ●●

●

●●●●

●●

●

●

●

●●●

●

●

●

● ●●

●●

●

●●

●●● ●●●

●●

●●●

● ●●●

●●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●●

●●●

●

●●

●

●●●

●

●●●●

●

●

●

●

●●●

●●●

●

●

●

●●

● ●●

●

●● ●● ● ●

●

●● ●●

●●●

●

●

●

●

●●●●

●●●

●●●

●

●

●

●●

●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●

●●●

●●

●

●

●●●●●

●●●

●

●

●●●●●●

●

●

●●●●

●●

●

●●

●

●

●●

●

●●

●●●●●●

●●●●●

●

●●

●●●●●●●

●●●

●

●●

●

●●

●●●

●●

−3

−2

−1

0

1

2

3

−2 0 2 4PC1

PC

2

3

6

9

12value

Freq

Page 24

Feature space of M3 data

Exploring the feature space of large time series collections M3 competition data 10

● ●●

●

●

●●● ●●

● ●

●

●

●

●●

●

●

●

●

●

●

●●●●●

●

●●●

●

●●● ●

●●● ●

●● ●●● ●●●●

●

●●

●●●●●●

●

●●●●

●●●

●

●

●●

●

●●●

●●●

● ●

●

●

●●

●

●

●

●●●

●●

●●

●

●●

●●

●●●●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●●● ●●

●

●

●●●

●●●●●●●

●●●

●

●

●●

●●●

●●●●

●●●●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●●

●●

●●

●

●●

●●

●

●

●

●

●

●●

●●●●●

●● ●

●

●

●

●

●

● ●

●

●

●●

●

●

●●

●●

●

●●

●●

●

●●●●●●●●

●

●●●●●

●●●●●

●●●●●●●●●●

●

●

●●●

●●●

●

●●● ●

●● ●

●●

●●●

●●●●

●●●●●

● ●●●●●●● ●

●●

●●●

●● ●●●●

●

●

●●●

●●

●

●

●●●● ●

●●

●

●●●

●●●

●

●●●●●●●●

●●●

●

●

●

●

●

● ●●

●

●●●

●●●●

●

●●●

●●●

●

●●

●

●

●●

●

●

●●●●●

●●

●●

●

●

●●●

● ●●

●

●●

●●

●

● ●

● ●●●

●

●

●●●●●●

●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●

●

●●●

●●

●

●●●

●

●●●

●●●●●

●

●

●●●

●

● ●●●●

●●

●

●●●●●●

●●●●

● ●●

●●

●●

●●

●●

●●

●●●●

●●

●●●●

●●

●

●

●

●●●

●

●●●● ●●

●●

●

●●

●

●

●●

●

●●

●●●

● ●●

●

●

●●●

●

●●

●●●●●

●●● ●●●

●

●●

●

●●

●●●

●

●●

●

●

●●● ●●

●

●●

●●

●●

●●●●●●

●●●

●

●●

●

●●●●●●●●●

●

●

●

●●

●

●

●

●●

●

●

● ● ●●●

●

●●

●

●

●● ●●●

●

●

●

●

●

●

●●

●

●● ●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●●●

● ●●

●

● ●

●

● ●●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●●●●

●●

●●

●●

●

●●

●

●

●●●●●● ●

● ●●

●

●

●

●

●

●● ●● ●

●

●●●

●

●

●●● ●

●● ●●●

●

●

●● ●●

●●●

●●

●●●●●●

●●●

●

●●

●

●●

●

●●●●

●●●

●

●

● ●●●●●●

●

●●

●●● ●●●●●● ●●●

●

●●●

●

●

●● ●●●●● ●

●●

●

●●●●

●●

●

●● ●

●

●●

●

●

●●

●●

●

●

●

● ●

●● ●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●●●●

●

●●●

●

●●●

●

●●● ●●

●●

●●●

●●●

●●●

●

●

●●●

●●

●

●●

●

●

●●●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●●●

●●

●●

●

●

●●

●

●●

● ●

●

●

●

●●

●

●● ●●●

●

●●●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●●●●●

●

●● ●●

●●

●●●

●●

●

●

●● ●

●●

●●●

●

●

●

●

●●●

●

●●●

●

●●●

●●●●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●●● ●●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●●

●●

●● ●

●●

●

●

●

●●

●

●●

●●

●● ●

●●●

●

●

●●

●

●●

●●

●●●

●●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

● ●

●

● ●

●●

●

●

●

●

●

●

●

●●

●●●

●

●●

● ●●

●

●

●●

●●

●●

●

●●

●

●●

●

●●

●

●●

●

●

●

●●●

●●

●● ●

●

●●

●

●

●

●

●

●●

●

● ●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●● ●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●●●

●

●

●

●●

●●

●

●●

●●●

●

●

●●

●●

●●

●●

●

● ●● ●

●●

●● ●●

●●

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●●

●

●

●

●

●●

● ●●●

●

●

●●

●

●

●

● ●●●

●●

●

●

●●●

●

●

●

●●

●

●●

●●

●

●●

●●●

●

●

●●●

●

● ●

● ●●

● ●

●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●●● ●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●●●

●●

●

● ●●

●

●●

●

● ●● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

● ●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●●

●

●

● ●●

●●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

● ●●

●

●

●

●●

●

●

● ●

●

●

●●●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●●

●

●

●●●

●

●

●

● ●

●

●

●●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●●

●

● ●●

●

●●●

●●

●

●

●

●

●

●

●●

●

●●

●

●●●

●

●●

●

●●

●

●

●

●

●● ●●●●

●●

● ●

●

●●

●

●

●●

●●

●

●

●●

●

●

●●

●

●

●●

●

●●

●●

●

●●

● ●

● ●

●

●●

●●

●●

●●

●●

●

●●

●

●

●

● ●

●●

●●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●● ●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

● ●●●●●

● ●●

●

●

●

●

●●●●

●

●

●●

●

●●

●

●●

●●●●●

●

●

●●●● ●

●

●●●● ●

●

●

●

●

●

● ●●●●●

●●

●

●

●●

●●

●●

●

●●

●

●●

●●

●

●

● ●

●

●● ●

●

●

●

●●●

●

●

●

●

●● ●●● ●●●●●●●●

●

●●

●●●●●●

●

●●●●

●●

●●

● ●●

●

●

●

●●●

●

●

●●

●

●

● ●●

●

●●●● ●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●●

●

●●

●

●●●●●

●

●●

● ●

● ●

●

●●

●●●

●●

●

●●●

●●

● ●●

●●●

●

●

●

●

●●

● ●●

●●●

●

●

●●●

●●

●

●

●●

● ●●

●

●

●●

●

●●

●●●

●●

●

●

●

●●

●

●●●

●●

●

●

●●

●● ●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●●●●

●

●

●●

●

●●●

●

●●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●●● ●●

● ●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

● ●

●

●●

●

●

●●

●●●

● ●

●

●●

●

●●●●●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●●

● ●●●

●

●●

●

●

●

●

●

● ●

● ●

●●

●

●●

●●

●●●

●●

●● ● ●●●

●●

●●●●●

●

●●●●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●●●●●●●

●

●●●

●●

●●

●●●●●

●

●

●

●

●

● ●

●●●●●●

●●●

●●●●● ●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●●

●

● ●● ●●

●

●●●●

●●

●

●

●

●●●

●

●

●

● ●●

●●

●

●●

●●● ●●●

●●

●●●

● ●●●

●●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●●

●●●

●

●●

●

●●●

●

●●●●

●

●

●

●

●●●

●●●

●

●

●

●●

● ●●

●

●● ●● ● ●

●

●● ●●

●●●

●

●

●

●

●●●●

●●●

●●●

●

●

●

●●

●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●

●●●

●●

●

●

●●●●●

●●●

●

●

●●●●●●

●

●

●●●●

●●

●

●●

●

●

●●

●

●●

●●●●●●

●●●●●

●

●●

●●●●●●●

●●●

●

●●

●

●●

●●●

●●

−3

−2

−1

0

1

2

3

−2 0 2 4PC1

PC

2

0.00

0.25

0.50

0.75

value

Season

Page 25

Feature space of M3 data

Exploring the feature space of large time series collections M3 competition data 10

● ●●

●

●

●●● ●●

● ●

●

●

●

●●

●

●

●

●

●

●

●●●●●

●

●●●

●

●●● ●

●●● ●

●● ●●● ●●●●

●

●●

●●●●●●

●

●●●●

●●●

●

●

●●

●

●●●

●●●

● ●

●

●

●●

●

●

●

●●●

●●

●●

●

●●

●●

●●●●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●●● ●●

●

●

●●●

●●●●●●●

●●●

●

●

●●

●●●

●●●●

●●●●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●●

●●

●●

●

●●

●●

●

●

●

●

●

●●

●●●●●

●● ●

●

●

●

●

●

● ●

●

●

●●

●

●

●●

●●

●

●●

●●

●

●●●●●●●●

●

●●●●●

●●●●●

●●●●●●●●●●

●

●

●●●

●●●

●

●●● ●

●● ●

●●

●●●

●●●●

●●●●●

● ●●●●●●● ●

●●

●●●

●● ●●●●

●

●

●●●

●●

●

●

●●●● ●

●●

●

●●●

●●●

●

●●●●●●●●

●●●

●

●

●

●

●

● ●●

●

●●●

●●●●

●

●●●

●●●

●

●●

●

●

●●

●

●

●●●●●

●●

●●

●

●

●●●

● ●●

●

●●

●●

●

● ●

● ●●●

●

●

●●●●●●

●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●

●

●●●

●●

●

●●●

●

●●●

●●●●●

●

●

●●●

●

● ●●●●

●●

●

●●●●●●

●●●●

● ●●

●●

●●

●●

●●

●●

●●●●

●●

●●●●

●●

●

●

●

●●●

●

●●●● ●●

●●

●

●●

●

●

●●

●

●●

●●●

● ●●

●

●

●●●

●

●●

●●●●●

●●● ●●●

●

●●

●

●●

●●●

●

●●

●

●

●●● ●●

●

●●

●●

●●

●●●●●●

●●●

●

●●

●

●●●●●●●●●

●

●

●

●●

●

●

●

●●

●

●

● ● ●●●

●

●●

●

●

●● ●●●

●

●

●

●

●

●

●●

●

●● ●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●●●

● ●●

●

● ●

●

● ●●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●●●●

●●

●●

●●

●

●●

●

●

●●●●●● ●

● ●●

●

●

●

●

●

●● ●● ●

●

●●●

●

●

●●● ●

●● ●●●

●

●

●● ●●

●●●

●●

●●●●●●

●●●

●

●●

●

●●

●

●●●●

●●●

●

●

● ●●●●●●

●

●●

●●● ●●●●●● ●●●

●

●●●

●

●

●● ●●●●● ●

●●

●

●●●●

●●

●

●● ●

●

●●

●

●

●●

●●

●

●

●

● ●

●● ●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●●●●

●

●●●

●

●●●

●

●●● ●●

●●

●●●

●●●

●●●

●

●

●●●

●●

●

●●

●

●

●●●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●●●

●●

●●

●

●

●●

●

●●

● ●

●

●

●

●●

●

●● ●●●

●

●●●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●●●●●

●

●● ●●

●●

●●●

●●

●

●

●● ●

●●

●●●

●

●

●

●

●●●

●

●●●

●

●●●

●●●●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●●● ●●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●●

●●

●● ●

●●

●

●

●

●●

●

●●

●●

●● ●

●●●

●

●

●●

●

●●

●●

●●●

●●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

● ●

●

● ●

●●

●

●

●

●

●

●

●

●●

●●●

●

●●

● ●●

●

●

●●

●●

●●

●

●●

●

●●

●

●●

●

●●

●

●

●

●●●

●●

●● ●

●

●●

●

●

●

●

●

●●

●

● ●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●● ●

●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●●●

●

●

●

●●

●●

●

●●

●●●

●

●

●●

●●

●●

●●

●

● ●● ●

●●

●● ●●

●●

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●●

●

●

●

●

●●

● ●●●

●

●

●●

●

●

●

● ●●●

●●

●

●

●●●

●

●

●

●●

●

●●

●●

●

●●

●●●

●

●

●●●

●

● ●

● ●●

● ●

●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●●● ●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●●●

●●

●

● ●●

●

●●

●

● ●● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

● ●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●●

●

●

● ●●

●●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

● ●●

●

●

●

●●

●

●

● ●

●

●

●●●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●●

●

●

●●●

●

●

●

● ●

●

●

●●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●●

●

● ●●

●

●●●

●●

●

●

●

●

●

●

●●

●

●●

●

●●●

●

●●

●

●●

●

●

●

●

●● ●●●●

●●

● ●

●

●●

●

●

●●

●●

●

●

●●

●

●

●●

●

●

●●

●

●●

●●

●

●●

● ●

● ●

●

●●

●●

●●

●●

●●

●

●●

●

●

●

● ●

●●

●●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●● ●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●●

●

● ●

●

●

●

●

●

●

●

●●

●●

●●●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

● ●●●●●

● ●●

●

●

●

●

●●●●

●

●

●●

●

●●

●

●●

●●●●●

●

●

●●●● ●

●

●●●● ●

●

●

●

●

●

● ●●●●●

●●

●

●

●●

●●

●●

●

●●

●

●●

●●

●

●

● ●

●

●● ●

●

●

●

●●●

●

●

●

●

●● ●●● ●●●●●●●●

●

●●

●●●●●●

●

●●●●

●●

●●

● ●●

●

●

●

●●●

●

●

●●

●

●

● ●●

●

●●●● ●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●●

●

●●

●

●●●●●

●

●●

● ●

● ●

●

●●

●●●

●●

●

●●●

●●

● ●●

●●●

●

●

●

●

●●

● ●●

●●●

●

●

●●●

●●

●

●

●●

● ●●

●

●

●●

●

●●

●●●

●●

●

●

●

●●

●

●●●

●●

●

●

●●

●● ●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●●●●

●

●

●●

●

●●●

●

●●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●●● ●●

● ●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

● ●

●

●●

●

●

●●

●●●

● ●

●

●●

●

●●●●●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●●

● ●●●

●

●●

●

●

●

●

●

● ●

● ●

●●

●

●●

●●

●●●

●●

●● ● ●●●

●●

●●●●●

●

●●●●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●●●●●●●

●

●●●

●●

●●

●●●●●

●

●

●

●

●

● ●

●●●●●●

●●●

●●●●● ●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●●

●

● ●● ●●

●

●●●●

●●

●

●

●

●●●

●

●

●

● ●●

●●

●

●●

●●● ●●●

●●

●●●

● ●●●

●●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●●

●●●

●

●●

●

●●●

●

●●●●

●

●

●

●

●●●

●●●

●

●

●

●●

● ●●

●

●● ●● ● ●

●

●● ●●

●●●

●

●

●

●

●●●●

●●●

●●●

●

●

●

●●

●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●

●●●

●●

●

●

●●●●●

●●●

●

●

●●●●●●

●

●

●●●●

●●

●

●●

●

●

●●

●

●●

●●●●●●

●●●●●

●

●●

●●●●●●●

●●●

●

●●

●

●●

●●●

●●

−3

−2

−1

0

1

2

3

−2 0 2 4PC1

PC

2

0.25

0.50

0.75

value

Trend