segmenting electric load time series?...an empirical evaluation of forecasting hourly uk load data 2...

Segmenting Electric Load Time Series? An empirical evaluation of forecasting hourly UK load data

2 August 2011, IJCNN 2011, San Jose, CA

Dr. Sven F. Crone

Dr. Nikolaos Kourentzes

Agenda

1. Why segment Electricity Load Data? • The challenge: high freq data • Best practices for Neural? • How to segment Time Series?

2. Empirical Evaluation of Accuracy • Experimental Design • Neural v. Statistical Algorithms • Experimental Results

3. Conclusions

Load Forecasting

Challenges in Electricity Forecasting Data Conditions for Modelling

• Hourly load (7.5 years = 66,505 observations) triple seasonality (high freq!)

Annual (week in the year) & Weekly (day in week) & Daily (Hour in day) Seasonality Requires dozens of binary dummy variables for causal modelling high dim!

Only recent extensions for Exponential Smoothing … novel methods

• Single seasonal HW-Exponential Smoothing intraweek cycles s2=336 half hours

• Double seasonal HW-Exponential Smoothing intraweek & intraday cycles s1=48 & s2=336 half hours

• Triple seasonal HW-Exponential Smoothing by Taylor (2010) EJOR

intraweek & intraday & intravear cycles s1 = 48 & s2 = 336 & s3 = 17,472

IMA(d,q) models (+ AR-term “for correction of 1st order autocorrelation”) Extended to IC (intraday cycle) Exponential smoothing

Triple Smoothing Statistical Benchmark Methods

#1 Accuracy

Publications Review of NN setups

Every NN model is unique – few commonalities / agreement on “how to …” Most forecast profiles use at least historic Electric Load & Temperature (+ …)

All but 3 papers segment time series into MULTIPLE SUBSERIES !

[Hippert, Pedreira, Souza (2001) Neural networks for short time load forecasting a review and evaluation, IEEE Trans. On Power Systems]

5 to 107 input nodes

6 to 90 hidden nodes

1 to 48 output nodes

27 to 32,000 parameters

All (but 3) segment by calendar days

But no consensus: 1 to 105

seperate NN models


Segmentation of time series into 2 subgroups: 2 series of Weekdays vs. weekends 24h of [Mon] x [Tue, Wed, Thu] x [Friday] x [Sat & Sun]? Summer & Winter? Summer & Fall & Winter & Spring? Bank holidays? …


Segmentation of time series into homogeneous subseries: 7 daily time series 24h of [Mon ] x [Tue] x [Wed]…

Successful simplification of modelling eliminates 1 seasonality


Segmentation into 24 subgroups: 24 time series of hourly data [01:00 – 02:00] x [02:00 – 03:00] x …

Successful simplification of modelling eliminates 2 seasonalities


Segmentation into 168 subgroups: 168 time series of hourly data per weekday [15:00-16:00 on Mondays] x [15:00-16:00 on Tuesdays] x …

Successful simplification of modelling eliminates 3 seasonalities

Agenda



3. Conclusions

Load Forecasting

Segmentation of MLPS Different Forms of Segmentation

Unsegmented time series vs. 3 different setups of segmentation MLP feedforward neural net vs. statistical benchmarks

Daily UK electricity demand (01.04.2001 01:00 – 01.11.2008 01:00=66,505 hours=2,771 days = 7.6 years ) 34,345 train (2001-2004) + 16,080 valid (2005-2006) + 16,080 test (2007-2008)

Source: UK National Grid website

Univariate MLP

NAR(p)

Multivariate MLP

NARX(p)

Univariate Regression

AR(p)

Multivariate Regression

ARX(p)

4 Naive

I(d)

5 Exponent. Smoothing IMA(d,q)

Fixed horizon h=24 t+1, t+2, …, t+24 = multiple step ahead „trace forecasts“ Rolling time origin evaluation: forecast + roll 1 hour forward average of 16,080 errors!

Accuracy measured on MAPE (meanabsolute percent error)

No segmentation

Y

Segment into 7 daily time series

Y7i

Segment into 24 hourly time series

Y24j

Segment into 168 hour/day series

Y168k

Triple seasonality: hourly+daily+annual

Double seasonality: hourly + annual

Double seasonality: daily + annual

Single seasonality: annual

Univariate MLP trained on 4 time

series representations

0

50

100

150

200

... ...

yt

yt-1

u1

un+m+1

un+3

un+2

un+1

u2

yt-2

yt-n-1

u3

un

...

un+m

un+m+2

un+m+h

θ n+1

q n+2

q n+3

q n+m

q n+m+1

q n+m+2

q n+m+h...

...

1ˆ

ty

2ˆ

ty

ˆt hy

fu(·)

fu(·)

fu(·)

fu(·)

fu(·)

fu(·)

fu(·)

Modelling of nonlinear, autoregressive Processes Each neuron = NARX(p) process Hierarchy of NARX(p)-processes

Only one (!) possible architecture simple, well known

1ˆ , ,...,t h t t t n t hy f x x x

Multilayer Perceptrons Class of statistical methods for regression Combination of simple processing units complex system behaviour

Quadratische Fehlerfunktion SE(et)

2 1 1 2ep

2

1.5

1

0.5

0.5

1

1.5

2

epR

R 4

R 2 SE e

R 1 AE e

Verlustfunktion

Multilayer

Perceptron

(MLP)

MLP for Forecasting Simple architecture

Agenda



3. Conclusions

Load Forecasting

Experimental Results Univariate & Multivariate Methods

MLPs on Y outperforms all other methods novel Triple EXSM ranks only 2nd !!! Univariate: 15.8% improvement on linear Regression (1.69% v. 2.33% ) Multivariate: 20.2% improvement on multiple Regression (1.82% v. 2.28%)

Table 1: MAPE per univariate algorithm for original and segmented time series

MAPE Complete Series Y Segmented Series Y7i Segmented Series Y24j Segmented Series Y168k

Model Train Valid Test Train Valid Test Train Valid Test Train Valid Test

Naive 17.21% 18.41% 19.17% 16.95% 18.21% 18.89% 9.57% 9.05% 8.61% 12.96% 13.15% 12.56%

Naive Daily* 5.78% 5.50% 5.39% 3.98% 3.66% 3.55% - - - - - -

Naive Weekly**

4.04% 3.78% 3.66% - - - 5.12% 4.88% 4.45% - - -

Naive Annual***

3.53% 3.51% 4.09% 3.50% 3.47% 4.05% 3.53% 3.47% 4.03% 3.47% 3.16% 3.72%

EXSM Daily* 7.27% 6.90% 7.15% 4.29% 3.85% 3.93% - - - - - -

EXSM Weekly** 3.39% 3.43% 3.44% - - - 4.74% 4.42% 4.07% - - -

EXSM Annual*** 1.69% 2.55% 3.39% 2.30% 3.36% 4.08% 2.99% 3.91% 4.41% 2.39% 3.64% 4.00%

EXSMTriple 1.42% 1.79% 2.00% - - - - - - - - -

EXSMDouble**** 2.14% 2.08% 2.13% 2.38% 3.34% 3.82% 2.36% 3.19% 3.52% - - -

Regr Univariate 2.31% 2.18% 2.33% 3.03% 3.03% 3.47% 3.25% 3.41% 3.88% 2.93% 3.00% 3.68%

NN Univariate 1.50% 1.69% 1.96% 2.79% 3.25% 3.82% 3.00% 3.39% 3.85% 3.08% 3.18% 3.93%

Daily: s = 24; Weekly: s = 168 and 7 for Y and Y24j respectively; Annual: s = 8736, 1248, 364 and 52 for Y, Y7i, Y24j and Y168k;

Table 2: MAPE per multivariate algorithm for original and segmented time series



Best Univariate 1.42% 1.69% 1.96% 2.30% 3.03% 3.47% 2.36% 3.19% 3.52% 2.39% 3.00% 3.68%

Regr Multivariate 2.21% 2.19% 2.28% 2.82% 2.98% 3.27% 2.89% 3.17% 3.72% 2.92% 2.99% 3.67%

NN Multivariate 1.37% 1.60% 1.82% 2.17% 3.02% 3.36% 2.27% 2.76% 3.33% 2.89% 3.21% 3.80%

Naive Daily @ Y7 is best Naive no annual seasonality

EXSM Annual @ Y is best EXSM all 3

seasonalities Tripe EXSM @ Y

is best new EXSM all 3 seasonalities

MLP @ Y outperforms all on

valid & test

Segmenting works for simple methods

Segmenting fails for complex

methods

Identical (relative) improvement for multivariate MLPs

Experimental Results by Forecasting Horizon

NN multivariate >> NN univariate >> EXSM triple >> REG multivariate >> REG univariate

Results are consistent across forecasting horizons in & out of sample Temperature only impacts for forecasts of longer horizons

2 4 6 8 10 12 14 16 18 20 22 240

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Forecast Horizon

MA

PE

%

Validation Set

2 4 6 8 10 12 14 16 18 20 22 240

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Forecast Horizon

MA

PE

%

Training Set

NaiveDaily-Y

7i

EXSMTriple-Y

RegUnivariate-Y

RegrMultivariate-Y

NNUnivariate-Y

NNMultivariate-Y

2 4 6 8 10 12 14 16 18 20 22 240

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Forecast Horizon

MA

PE

%

Test Set

Experimental Results by Number of Hidden Nodes

Relative ranking is consistent across increasing number of hidden nodes Non-segmented NNs are also more robust to overfitting

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000

10

20

30

Hidden Nodes

MA

PE

%

Training Set MAPE %

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000

10

20

30

Hidden Nodes

MA

PE

%

Validation Set MAPE %

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000

10

20

30

Hidden Nodes

MA

PE

%

Test Set MAPE %

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000

5

10

15

20

25

30

Hidden Nodes

MA

PE

%

Training Set: Model Selection Error

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000

5

10

15

20

25

30

Hidden Nodes

MA

PE

%

Validation Set: Model Selection Error

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1000

5

10

15

20

25

30

Hidden Nodes

MA

PE

%

Test Set: Model Selection Error

NNY

NNY7i

NNY24j

NNY168k

NNYT NN

Y7iT NN

Y24jT NN

Y168kT

Grey bar indicates the best model

Increased MLP size Increasing errors

(overfitting?)

Increased MLP size increasing errors

(overfitting?)

Increased MLP size constant errors!

• Neural Networks outperform all other algorithms – Using 1 single continuous time series >> segmenting series

– MLPs are accurate and robust • Robust results across forecasting horizons • Robust results across number of hidden nodes

• (most) Multivariate methods outperform univariate methods – Multivariate Regression better than univariate Regression – BUT: Tripe Exponential Smoothing better than multivariate Regressions

• Segmenting time series is detrimental to accuracy

– Destroys autoregressive information between consecutive observations – This information is valuable in load data see tripe EXSM

Extended PAPER is available … we welcome you comments! CHALLENGE US! Get the data and do better … ;-)

Conclusions NN vs. Statistics?

Novel insights into best practices how to model MLPs But WHY?

Experimental Results Segmenting Load Forecasting

Different input vectors selected, according to inherent seasonality Segmenting time series losses AR(p) information of omitted seasonality

If (!) relevant changes occur on t-1…t-24 etc. these will be captured later

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

48

72

96

120

144

164

165

166

167

168

169

170

171

172

336

504

672

840

1008

1176

8568

8736

8760

8784

8904

9072

Hourly Lags

Y

Y7i

Y24j

Y168k

Univariate lags - Electricity

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

48

72

96

120

144

167

168

169

170

171

172

336

504

672

840

1008

1176

8568

8736

8760

8784

8808

8904

9072

Hourly Lags

YY

7iY

24jY

168k

Multivariate lags - Electricity

0.20.40.60.81

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

48

72

96

120

144

167

168

169

170

171

172

336

504

672

840

1008

1176

8568

8736

8760

8784

8808

8904

9072

Hourly Lags

TT

7iT

24jT

168k

Multivariate lags - Temperature

0.20.40.60.81

Sven F. Crone Lancaster University Management School Centre for Forecasting - Lancaster, LA1 4YX email: [email protected]

Nikos Kourentzes Lancaster University Management School Centre for Forecasting - Lancaster, LA1 4YX email: [email protected]

Questions?

Copyright

These slides are from a workshop on statistical forecasting. You may use these slides for internal purpose, so long as they are clearly identified as being created and copyrighted © by Dr. Sven F. Crone and Dr. Nikolaos Kourentzes, Lancaster Centre for Forecasting. You may not use the text and images in a paper, tutorial or external training without explicit prior permission from Dr. Crone.


Annual (week in the year) & Weekly (day in week) & Daily (Hour in day) Seasonality Only recent extensions for Exponential Smoothing, comparatively new topic! Requires dozens of binary dummy variables for causal modelling high dim!

• High Frequency Data mulitple seasonality


Non-constant mean requires adaptive models stochastic formulations Non-constant variance (heteroscedasticity) requires GARCH / Tsay correction

• Non-constant mean & variance


External variables need to be incorporated Causal models? Additional complexity to identify for future & correct in the past!

• Calendar Events & Exogenous Effects

Christmas


Unclear how to overcome limitation of statistical modelling Automatic model building not feasible

• High frequency data creates problems

• Long series many observations (e.g. 22,806 obs)

• Statistical tests everything becomes significant

0 500 1000 1500 2000

-2

-1

0

1

2

Sample size

Confidence I

nte

rval

Effect of sample size on confidence interval

0 10 20 30 40-1

-0.5

0

0.5

1

Lag

Sam

ple

Auto

corr

ela

tion

120 observations

Non signif icant Signif icant CI

0 10 20 30 40-1

-0.5

0

0.5

11200 observations


Requires outlier removal Requires use of nonlinear models or stepwise linearisation

• Data contains Outliers and Nonlinear Interactions

Henley, Pearson (1997) Nonlinearities in Electricity Demand


Requires novel techniques not yet developed (forefront of research) Functional Data transition function between functional shapes


Different Regions exhibit different patterns require individual modelling


Highest level of forecasting modelling complexity

• Electrical Load & Price data show high complexity

• Time series show nonlinearities & causal effects

• Require nonlinear causal methods (NN, SVR) complex

• Requires many dummy variables for multiple Seasonality

• Similar time series exhibit different patterns

• Require time series specific model building

• Too many observations void statistical tests

• Automatic specification not feasible manual experts

© Sven Crone - Lancaster Centre for Forecasting

Segment into subseries train & predict subseries recombine

Different ways of segmenting time series hour; hour per week …

Day

Week

Intraday

9:00-10:00

Mon, Tue, …, Sun

Intraweek

9:00-10:00

at weekly seasonality

Intraday & Intraweek

20:00-21:00 on Friday Evening

Segmenting Time Series hour of day, hour per week etc.

Experimental Design Segmenting Load Forecasting

E-Load forecasting is most prominent & sophisticated discipline No possibility to take best practices from elsewhere

Apr May Jun Jul Aug Aug Sep Oct Nov Dec Jan Feb Mar

40000

60000

80000

100000

120000

Annual seasonality

6 121824 6 121824 6 121824 6 121824 6 121824 6 121824 6 121824

40000

60000

80000

100000

120000

Weekly seasonality

Sun Mon Tue Wed Thu Fri Sat

Hour

2 4 6 8 10 12 14 16 18 20 22 24

40000

60000

80000

100000

120000

Daily seasonality

Hour

Experimental Results Segmenting Load Forecasting

…

Table 4: MAPE of sparse and full input vector NN models



NNUnivariate 1.50% 1.69% 1.96% 2.79% 3.25% 3.82% 3.00% 3.39% 3.85% 3.08% 3.18% 3.93%

NNFUnivariate 1.75% 1.86% 2.02% 2.13% 3.48% 3.89% 2.93% 3.37% 3.73% 2.95% 3.46% 3.95%

Difference -0.24% -0.17% -0.06% 0.67% -0.23% -0.07% 0.07% 0.02% 0.12% 0.13% -0.29% -0.02%

NNMultivariate 1.37% 1.60% 1.82% 2.17% 3.02% 3.36% 2.27% 2.76% 3.33% 2.89% 3.21% 3.80%

NNFMultivariate 1.60% 1.80% 1.97% 2.48% 4.14% 4.42% 2.26% 2.84% 3.41% 2.94% 4.22% 4.50%

Difference -0.23% -0.20% -0.15% -0.31% -1.12% -1.06% 0.01% -0.08% -0.08% -0.05% -1.00% -0.69%

Experimental Results Results by Data Conditions

NN multivariate outperforms all methods on Off-peak & Peak Load NN multivariate outperforms all methods on Winter v. Summer months

Table 5: MAPE for off-peak and peak hours

MAPE Off-Peak Peak

Model Train Valid Test Train Valid Test

NaiveDaily 3.96% 3.65% 3.52% 4.13% 3.72% 3.82%

EXSMTriple 1.41% 1.78% 2.01% 1.43% 1.79% 1.90%

RegrUnivariate 2.30% 2.18% 2.33% 2.39% 2.16% 2.30%

NNUnivariate 1.49% 1.67% 1.97% 1.60% 1.80% 1.90%

RegrMultivariate 2.20% 2.19% 2.29% 2.29% 2.14% 2.25%

NNMultivariate 1.38% 1.57% 1.82% 1.34% 1.77% 1.82%

Table 8: MAPE for winter and summer days

MAPE Winter Summer

Model Train Valid Test Train Valid Test

NaiveDaily 4.81% 4.61% 4.71% 3.41% 3.16% 2.92%

EXSMTriple 1.54% 1.92% 2.14% 1.33% 1.71% 1.92%

RegrUnivariate 2.45% 2.50% 2.76% 2.21% 2.00% 2.08%

NNUnivariate 1.76% 2.00% 2.28% 1.33% 1.52% 1.78%

RegrMultivariate 2.15% 2.11% 2.36% 2.25% 2.23% 2.24%

NNMultivariate 1.57% 1.72% 2.06% 1.24% 1.53% 1.68%

Experimental Results Statistical tests of Results

Results are statistically significant

Table 3: Nemenyi Test Model Ranking

Train* Valid** Test**

Model Mean Rank Model Rank Mean Rank Model Rank Mean Rank Model Rank

NaiveDaily 4.59 5 4.41 6 4.17 6

EXSMTriple 2.99 2 3.25 3 3.31 3

RegrUnivariate 4.00 4 3.66 5 3.66 5

NNUnivariate 2.96 2 3.11 2 3.21 2

RegrMultivariate 3.81 3 3.59 4 3.54 4

NNMultivariate 2.65 1 2.98 1 3.11 1

*Critical distances are 0.06, 0.05, 0.04 for 1%, 5% and 10% significance levels respectivelly;

**Critical distances are 0.07, 0.06, 0.05 for 1%, 5% and 10% significance levels respectivelly

segmenting electric load time series?...an empirical evaluation of forecasting hourly uk load data 2...

Documents