Challenges in forecastingpeak electricity demand

Rob J Hyndman

Challenges in forecasting peak electricity demand 1

Part 2

Challenges

1 How to evaluate forecast distributions

2 How to select the best forecasting method

3 How to account for off-grid generation

4 How to use smart metre data and networkdata when forecasting

5 How to improve forecasts

Challenges in forecasting peak electricity demand 2

Forecast accuracy measures

MAE: Mean absolute errorMSE: Mean squared errorMAPE: Mean absolute percentage error

å Good when forecasting a typical future value(e.g., the mean or median).

å Useless for evaluating forecast percentiles(probability of exceedance values) and forecastdistributions.

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 3

Evaluating forecast distributions

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 4

PoE (annual interpretation)

Year

PoE

Dem

and

2.0

2.5

3.0

3.5

4.0

98/99 00/01 02/03 04/05 06/07 08/09 10/11

10 %50 %90 %

●

●

●

●

●

●

●●

● ●

●

●

●

●

10 out of 13 above 90% PoE

5 out of 13 above 50% PoE

0 out of 13 above 10% PoE

Evaluating forecast distributions

Qt(p) = PoE of yt, to be exceeded with per-centage p.

G(p) = percentage of times yt greater thanQt(p) in the historical data.

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 5

If Qt(p) is accurate, then G(p) ≈ p

Evaluating forecast distributions

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 6

020

4060

8010

0

p = Forecast percentage exceedance

G(p

) =

Act

ual p

erce

ntag

e ex

ceed

ance

0 10010% 50% 90%

5/13

10/13

Evaluating forecast distributions

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 7

0 20 40 60 80 100

020

4060

8010

0

p = Forecast percentage exceedance

G(p

) =

Act

ual p

erce

ntag

e ex

ceed

ance

KS

KS = maxp |G(p)− p|

KS = Kolmogorov-Smirnov statistic= largest difference between G(p) and p.

Evaluating forecast distributions

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 8

0 20 40 60 80 100

020

4060

8010

0

p = Forecast percentage exceedance

G(p

) =

Act

ual p

erce

ntag

e ex

ceed

ance

Area = MAEP: Mean Absolute Excess Probability

MAEP =∫ 10 |G(p)− p|dp

Evaluating forecast distributions

MAEP more sensitive and less variable than KS.

Weekly or monthly maximums are betterbecause there are more of them to evaluate.

What is a good value of KS or MAEP?

We could restrict the range of p to “interesting”values such as p > 0.5.

These only measure whether the PoEs wereexceeded, not how much they were exceeded.

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 9

Forecast scoring

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 10

0 1 2 3 4 5 6

Demand distribution

Demand (GWh)

50%

50% PoE

Score for 50% PoEEquivalent toabsolute error

Forecast scoring

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 11

0 1 2 3 4 5 6

Demand distribution

Demand (GWh)

10%

10% PoE

Score for 10% PoE

Forecast scoring

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 12

0 1 2 3 4 5 6

Demand distribution

Demand (GWh)

75%

75% PoE

Score for 75% PoE

Forecast scoring

Let Qt(1), . . . ,Qt(99) be the PoEs of the forecastdistribution for probabilities 1%,. . . ,99%. Then thescore for observation y is

S(Qt(i), yt) =

{1

100 i(Qt(i)− yt) if yt < Qt(i)1

100(100− i)(yt − Qt(i)) if yt ≥ Qt(i)

Scores are averaged over all observed data foreach i to measure the accuracy of the forecastsfor each percentile.Average score over all percentiles gives thebest distribution forecast.Takes account of how far PoEs are exceeded.

Challenges in forecasting peak electricity demand How to evaluate forecast distributions 13

Forecasting the past

Traditional evaluation

Time series cross-validation

GEFCOM 2012 competition

Challenges in forecasting peak electricity demand How to select the best forecasting method 14

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● timeTraining data Test data

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

Also known as “Evaluation ona rolling forecast origin”

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● time

GEFCom2012

Challenges in forecasting peak electricity demand How to select the best forecasting method 15

Accuracy measuredby weighted root MSE.

Methodspublishedin IJF,April 2014

GEFCom2014

Probabilistic forecasting of demand, price,wind, and solar.

Forecasts to be submitted in the form ofpercentiles of future distributions.

Accuracy measured by scoring.

Rolling forecasts with incremental data updateon a weekly basis.

Prizes for student teams, and for best methods.

Winning methods to be published in the IJF.

Challenges in forecasting peak electricity demand How to select the best forecasting method 16

How to account for off-grid generation?

Locally generated power may not be recorded.But forecasts better if they are total demand.Need a model for PV generation that is linked tothe model for demand.

Challenges in forecasting peak electricity demand How to account for off-grid generation 17

Better: measure the off-grid generation viasmart metres.

How to use smart metre data?

Challenges in forecasting peak electricity demand How to use smart metre data 18

Smart metre data allow predictionof usage at household level basedon household characteristics:number of occupants, ages, etc.

So we could build a model forindividual usage, and scale it upfor the entire network.

How to allow for demandresponse?

Need aggregate information onhousehold characteristics for thenetwork.

Combine network and smartforecasts to improve accuracy

Ten steps to improving your forecasts

1 Look after your data

2 Understand how your forecasts will be used

3 Find the right forecasting tools

4 Use appropriate accuracy measures

5 Do not set targets

6 Do not adjust dishonestly

7 Keep it simple, stupid

8 Combine forecasts

9 Share ideas and mix with other forecasters

10 Adopt a process of continuous improvement

Challenges in forecasting peak electricity demand How to improve forecasts 19

Some resourcesBlogs

robjhyndman.com/hyndsight/blog.drhongtao.com/

OrganizationsInternational Institute of Forecasters:forecasters.orgIEEE Working Group on Energy Forecasting:linkedin.com/groups/IEEE-Working-Group-on-Energy-4148276

BooksDickey and Hong (2014) Electric loadforecasting: fundamentals and best practices,OTexts. www.otexts.org/book/elf

Challenges in forecasting peak electricity demand How to improve forecasts 20