has wind energy forecasting solved the challenge posed by ... pdfs... · actual wind energy and a...

Has Wind Energy Forecasting Solved the Challenge Posed by Intermittency?

Evidence from the United KingdomKevin F. Forbes

Ernest M. ZampelliSchool of Business and Economics

The Catholic University of [email protected]

9th Annual Trans-Atlantic InfradayFederal Energy Regulatory Commission

Washington, DC30 October 2015

2

The Organization of this Talk

1)Why is Forecasting Important?

2) The Literature on Wind Energy Forecast Accuracy

3) What is the level of forecast skill ? Specifically, what does the Mean Squared Error Skill Score (MSESS) indicate about the solar and wind energy forecasts? How does this level of accuracy compare to the accuracy of the load forecasts?

4)From the point of view of a system operator, how does wind energy compare with conventional forms of generation?

5)What are the prospects for improving the accuracy of the wind, solar, and load forecasts?

3

1)Why is Forecasting Important?

• The stability of the power grid is enhanced when forecasts are more accurate. This is important because blackouts have very high societal costs

• Some forms of balancing technologies such as open-cycle gas turbines have above average emissions factors.

• The market price of upward balancing power can be very costly.

4

Errors in the Day-Ahead Load Forecast for New York City and the Differential between the Real-Time and Day-Ahead Prices in New York City, 6 August 2009 – 30 June 2013.

Note: Excludes the period of time when operations were affected by Superstorm Sandy in late October 2012

5

2) The Literature on Forecast Accuracy Some researchers calculate a root-mean-squared error of the forecasts and then weight it by the capacity of the equipment used to produce the energy. The reported capacity weighted root mean squared errors (CWRMSE) are usually less than 10 percent. Adherents of this approach include Lange, et al. (2006, 2007), Cali et al. (2006), Krauss, et al. (2006), Holttinen, et al. (2006), Kariniotakis, et al. (2006), and even NERC (2010, p. 9).

In a publication entitled, “Wind Power Myths Debunked,” Milligan, et al. (2009) draw on research from Germany to argue that it is a fiction that wind energy is difficult to forecast. In their words: “In other research conducted in Germany, typical wind forecast errors for a single wind project are 10% to 15% root mean-squared error (RMSE) of installed wind capacity (emphasis added) but drop to 5% to 7% for all of Germany.” (Milligan, et al. 2009, p. 93)

The UK’s Royal Academy of Engineering (2014, p. 33) has noted that wind energy’s capacity weighted forecast error of about five percent is evidence that that the wind energy forecasts are highly accurate.

A report by the IPCC ( 2012 p, 623) on renewable energy indicates that wind energy is moderately predictable as evidenced by a capacity weighted RMS forecast error that is less than 10%. Solar energy is reported to be even more accurate.

6

The Literature on Forecast Accuracy (Continued)NREL (2013) implicitly endorses capacity weighted RMSEs for wind energy but makes use of energy weighted RMSEs when discussing the accuracy of load forecasts.

In contrast, Forbes et. al. (2012) calculate a root-mean-squared forecast error for wind energy in nine electricity control areas. The RMSEs are normalized by the mean level of wind energy that is produced. The reported energy weighted root mean squared errors (EWRMSE) are in excess of 20 %.

7

3) Using The Mean-Squared-Error Skill Score (MSESS) to Assess Forecast AccuracyA useful alternative to both the energy weighted and capacity weighted RMSE is the mean-squared-error skill score (MSESS). With this metric, one can evaluate the skill of a forecast relative to a persistence forecast, a persistence forecast being a period-ahead forecast that assumes that the outcome in period t equals the outcome in period t-1. The MSESS with the persistence forecast as a reference is calculated as follows:

Where is the mean squared error of the forecast that is being evaluated and is the mean squared error a persistence forecast. A perfect forecast would have a MSESS equal to one. A MSESS equal to zero indicates that the forecast skill is equal to that of a persistence forecast. A negative MSESS indicates that the forecast under evaluation is inferior to a persistence forecast.

•

8

How accurate are the forecasts?• MSESS were computed for the following zones and/or control areas:

• Bonneville Power Administration• CAISO: SP15 and NP15• MISO• PJM• 50Hertz in Germany• Amprion in Germany• Elia in Belgium• RTE in France• National Grid in Great Britain• Finland• Sweden• Norway• Eastern Denmark• Western Denmark

• When possible the MSESS are reported for Wind, Solar, and Load

9

Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference

Control Area/Zone Forecast Type Sample Period Observations Granularity

MSESS

50Hertz (Germany) Day-Ahead Load

1Jan2011 – 31Dec2013

104,590Quarter-Hour -62.7486

Day-Ahead Wind

1Jan2011 – 31Dec2013

104,590Quarter-Hour -31.3501

Day-Ahead Solar

1Jan2011 – 31Dec2013

54,545Quarter-Hour -5.26831

Amprion (Germany) Day-Ahead Load

1Jan2011 – 31Dec2013

103,326 Quarter-Hour-12.3308

Day-Ahead Wind

1Jan2011 – 31Dec2013

103,326 Quarter-Hour-14.5887

Day-Ahead Solar1Jan2011 – 31Dec2013

55,498 Quarter-Hour-11.20691

10

Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference (Continued)

Control Area/Zone Forecast Type Sample Period Observations Granularity

MSESS

California ISO Day-Ahead Load1Jan2013 – 31Dec2013

8,760 Hourly0.6026

NP15 Day-Ahead Wind1Jan2013 – 31Dec2013 8,704 Hourly -6.1401

NP15 Hour-Ahead Wind1Jan2013 – 31Dec2013 8,704 Hourly -2.3605

NP15 Day-Ahead Solar1Jan2013 – 31Dec2013 8,666 Hourly -3.2002

NP15 Hour-Ahead Solar1Jan2013 – 31Dec2013 8,666 Hourly -2.4846

SP15 Day-Ahead Wind1Jan2013 – 31Dec2013 8,752 Hourly -4.8210

SP15 Hour-Ahead Wind1Jan2013 – 31Dec2013 8,752 Hourly -2.1894

SP15 Day-Ahead Solar1Jan2013 – 31Dec2013 8,752 Hourly 0.7050

SP15 Hour-Ahead Solar1Jan2013 – 31Dec2013 8,752 Hourly 0.7972

11


Control Area/Zone Forecast Type Sample Period Observations Granularity MSESSBelgium

Day-Ahead Solar 1Jan2013 – 31Dec2013 17,921 Quarter-Hour -12.2621

Intra-Day Solar 1Jan2013 – 31Dec2013 11,278 Quarter-Hour -9.7931

France Day-Ahead Load 1Jan2012 – 31Dec2013 35,088 Half-Hourly

0.3842

Day-Ahead Wind 1Jan2012 – 31Dec2013 17,349 Hourly -5.7375

Hour 1 Same Day, Wind

1Jan2012 – 31Dec201315,109 Hourly -5.2889

Norway Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.1870

Sweden Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.2008

Finland Day-Ahead Load 1Jan2011 – 31Dec2013 26,159 Hourly 0.0486

Eastern Denmark Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.3953

Day-Ahead Wind 1Jan2011 – 31Dec2013 26,107 Hourly -2.7507

Western Denmark Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.6560

Day-Ahead Wind1Jan2011 – 31Dec2013

26,105 Hourly-3.6749

12


Control Area/Zone Forecast Type Sample Period Observations Granularity MSESS

MISO Day-Ahead Wind Energy 1Jan2011 – 31Dec2013 26,303 Hourly -4.3873

PJM Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.4727

New York City Day-Ahead Load 1Jan2011 – 31Dec2013 25,675 Hourly 0.1703

Bonneville Power Five Minute-Ahead Wind 1Jan2012 – 31Dec2013 206,477 Five minutes -36.25762

Hour-Ahead Wind1Jan2012 – 31Dec2013 16,847 Hourly -63.57602

Great Britain Day-Ahead Load 1Jan2012 – 31Dec201330,477

Half-Hourly 0.62

Day-Ahead Wind1Jan2012 – 31Dec2013

30,477 Half-Hourly

-19.032

1 Daylight portion of the sample period

2MSESS calculation excludes periods in which wind energy production was curtailed by the system operator.

13

4)From the point of view of a system operator, how does wind energy compare with conventional forms of generation? Evidence from Great Britain

• In Great Britain, each generating station informs the system operator of its intended level of generation one hour prior to real-time. This value is known as the final physical notification (FPN).

• Generators also submit bids (a proposal to reduce generation) and offers (a proposal in increase generation) to provide balancing services

• During real-time, the system operator accepts the bids and offers based on system conditions.

• In short, the revised generation schedule equals the FPN plus the level of balancing services volume requested by the system operator.

• Failure to follow the revised generation schedule gives rise to an electricity market imbalance that needs to be resolved by other generators.

14

The Revised Generation Schedules vs Actual Generation: The Case of Coal in Great Britain

020

0040

0060

0080

0010

000

1200

0

Met

ered

Gen

erat

ion

(MW

h)

0 2000 4000 6000 8000 10000 12000

Scheduled Generation including Balancing Actions (MWh)

EWRMSE = 2.5 %

15

The Revised Generation Schedules vs Actual Generation: The Case of Combined Cycle Gas Turbines in Great Britain

025

0050

0075

0010

000

1250

0

Met

ered

Gen

erat

ion

(MW

h)

0 2500 5000 7500 10000 12500


EWRMSE = 5.6%

16

Actual vs. Scheduled Generation: The Case of Nuclear Energy in Great Britain

010

0020

0030

0040

0050

00

Met

ered

Gen

erat

ion

(MW

h)

0 1000 2000 3000 4000 5000

Scheduled Generation (MWh)

EWRMSE = 7.4 %

17

The Revised Generation Schedules vs Actual Generation: The Case of Wind Energy in Great Britain, 1 Jan 2012 – 31 Dec 2013

050

010

0015

0020

0025

0030

00

Met

ered

Gen

erat

ion

(MW

h)

0 500 1000 1500 2000 2500 3000


EWRMSE= 18 %

18

Average Imbalances by Fuel in Great Britain, 1 Jan 2012- 31 December 2013

19

A Closer look at the Wind Energy Imbalances, 1 Jan 2012 – 30 June 2014

20

5) The Prospects for Improving the Forecasts

• Significant improvements in day-ahead forecasts will probably require major advances in meteorological research. One obvious place to begin is to note that the heat trapping properties of Greenhouse gases most likely have implications for wind speeds.

• Significant improvements in very short run forecasts (e.g. one or two hours ahead) are possible by exploiting the systematic nature of the existing forecast errors.

21

The Systematic Nature of the Existing Day-Ahead Forecast Errors for Wind Energy: Evidence from Great Britain

22

Out of Sample Results for Wind Energy in Great Britain, 1 Jan 2014 – 30 June 2014Forecast Type Number of Observations MSESS EWRMSE

Day-Ahead Wind Forecast

8,571 -35.05 31.9 %

Forecast equal to the levels of generation declared by operators one hour prior to real-time

8,571 -19.71 24.2 %

Modified Forecast: available to system operator 30 min prior to real-time

8,571 -1.95 9.1 %

23

Further Analysis

• What about other Power Systems ?

• What about Solar Energy ?

24

Hour-Ahead Forecasted and Actual Wind Energy in SP15 in CAISO, 1 January – 30 September 2015

EWRMSE = 37.1 % MSESS = -2.62

25

Actual Wind Energy and a Revised Hour-Ahead Wind Energy Forecast for SP15 in CAISO), 1 January – 30 September 2015

EWRMSE = 15.8 % MSESS = 0.34

26

Actual Solar Energy in 50Hertz in Germany and an Out-of-Sample Econometrically Modified Solar Energy Forecast, 1 July 2013 – 3 March 2014

For the daylight period:

EWRMSE = 4.8 %

MSESS = 0.768

27

Summary and Conclusions•With few exceptions, the load forecasts examined in this study have positive skill scores relative to a persistence load forecast.

•With few exceptions, the solar and wind forecasts examined in this study have negative skill scores relative to the corresponding persistence forecasts.

•Evidence has been presented that the forecast errors have a systematic component

•Evidence has also been presented that econometric modelling of this systematic component can yield short-run solar and wind energy forecasts that are significantly more accurate. This does not resolve the challenge of intermittency but may mitigate matters.

•The modelling approach can also be applied to improve load forecasts. See http://dialogue.usaee.org/index.php/day-ahead-market-prices-of-electricity-and-economic-fundamentals-preliminary-evidence-from-new-york-city

28

ReferencesGodfrey Boyle, 2010. Renewable energy technologies for electricity generation, in Harnessing Renewable Energy in Electric Power Systems, Boaz Moselle, Jorge Padilla, and Richard Schmalenese (eds.), RFF Press, Washington, DC, 2010, at 7-29.

California Independent System Operator, ISO New England, Midwest Independent Transmission System Operator, New York Independent System Operator , PJM Interconnection, and Southwest Power Pool, 2010. 2010 ISO/RTO Metrics Report. At http://www.isorto.org/atf/cf/%7B5B4E85C6-7EAC-40A0-8DC3-003829518EBD%7D/2010%20ISO-RTO%20Metrics%20Report.pdf <last accessed 15 feb 2012>

Ümit Cali, Bernhard Lange, Rene Jursa, Kai Biermann, 2006. Short-term prediction of distributed generation – Recent advances and future challenges, Elftes Kasseler Symposium Energie-Systemtechnik. At http://www.iset.uni-kassel.de/public/kss2006/KSES_2006.pdf <last accessed 15 feb 2012>

Mark A. Delucchi and Mark Z. Jacobson, 2011. Providing all global energy with wind, water, and solar power, Part II: Reliability, system and transmission costs, and policies. Energy Policy, 39, at 1170-1190.

European Wind Energy Association, 2007. Debunking the Myths. At http://www.ewea.org/fileadmin/ewea_documents/documents/publications/wind_benefits/Windpower_is_unreliable.pdf <last accessed 15 feb 2012>

Kevin Forbes, Marco Stampini, and Ernest M. Zampelli, 2012a. Are Policies to Encourage Wind Energy Predicated on a Misleading Statistic?, The Electricity Journal, Volume 25, Issue 3, pp. 42-54

Kevin Forbes, Marco Stampini, and Ernest M. Zampelli, 2012b. Do Policies to Encourage Wind Energy Inadvertently Pose Challenges to Electric Power Reliability? Evidence from the 50Hertz Control Area in Germany, The Electricity Journal, November 2012, Volume 25, Issue 9, pp. 37-42

GE Energy, 2010. Western Wind and Solar Integration Study, NREL/SR-550-47434, National Renewable Energy Laboratory, Golden, Colorado, May. At http://www.nrel.gov/wind/systemsintegration/pdfs/2010/wwsis_final_report.pdf <last accessed 15 feb 2012>

Gregor Giebel, Richard Brownsword, George Kariniotakis, Michael Denhard, and Caroline Draxl, 2011. The State-Of-The-Art in Short-Term Prediction of Wind Power A Literature Overview, 2nd Edition. Project report for the Anemos.plus and SafeWind projects. 109 pp. Risø, Roskilde, Denmark. Available at http://130.226.56.153/zephyr/publ/GGiebelEtAl-StateOfTheArtInShortTermPrediction_ANEMOSplus_2011.pdf <last accessed 15 feb 2012>

Hannale Holttinen, Peter Meibom, Antje Orths, Frans van Hulle, Bernhard Lange, Mark O’Malley, Jan Pierik, Bart Ummels, John Olav Tande,Ana Estanqueiro, Manuel Matos, Emilio Gomez, Lennart Söder, Goran Strbac, Anser Shakoor, Joao Ricardo, J. Charles Smith, Michael Milligan, and Erik Ela, 2009. IEA WIND Task 25: Design and operation of power systems with large amounts of wind power. At http://www.vtt.fi/inf/pdf/tiedotteet/2009/T2493.pdf <last accessed 15 feb 2012>

29

References (Continued)

Hannale Holttinen, Pirkko Saarikivi, Sami Repo, Jussi Ikäheimo, Goran Koreneff, 2006. Prediction Errors and Balancing Costs for Wind Power Production in Finland. Global Wind Power Conference, Adelaide

Intergovernmental Panel on Climate Change, 2012, Renewable Energy Sources and Climate Change Mitigation Special Report of the Intergovernmental Panel on Climate Change. At http://srren.ipcc-wg3.de/report/IPCC_SRREN_Full_Report.pdf

George Kariniotakis, 2006. State of the art in wind power forecasting, 2nd International Conference on Integration of Renewable Energies and Distributed Energy Resources, Napa, California/USA, 4-8 December.

Mattias Lange and Ulrich Focken, 2005. State-of-the-Art in Wind Power Prediction in Germany and International Developments. Prediction of Wind Power and Reducing the Uncertainty for Grid Operators, Second Workshop of International Feed-In Cooperation, Berlin (DE) http://www.energymeteo.de/media/fic_eeg_article.pdf <last accessed 15 feb 2012>

Bernhard Lange, Kurt Rohrig, Bernhard Ernst, Florian Schlögl, Umit Cali, Rene Jursa, and Javad Moradi, 2006. Wind power prediction in Germany – Recent advances and future challenges. European Wind Energy Conference and Exhibition, Athens (GR).

Bernhard Lange, Kurt Rohrig, Florian Schlögl, Umit Cali, and Rene Jursa,2006. Wind Power Forecasting. in: Boyle, G.(Ed.), Renewable Electricity and the Grid. Earthscan, London, England, at 95-120.

Bernhard Lange, Arne Wessel, Jan Dobschinski, and Kurt Rohrig, 2009. Role of Wind Power Forecasts in Grid Integration Kasseler Symposium Energie-Systemtechnik, at 118-130 http://www.iset.uni-kassel.de/public/kss2009/2009_KSES_Tagungsband.pdf <last accessed 15 feb 2012>

Bernhard Lange, Kurt Rohrig, Bernhard Ernst, Florian Schlögl, Umit Cali, Rene Jursa, and Javad Moradi, 2006. Wind power prediction in Germany – Recent advances and future challenges, Zeitschrift für Energiewirtschaft, vol. 30, no 2, at115-120. At http://www.iset.uni-kassel.de/abt/FB-I/publication/Lange-et-al_2006_EWEC_paper.pdf <last accessed 15 feb 2012>

30


Bernhard Lange, Kurt Rohrig, Florian Schlögl, Umit Cali,and Rene Jursa, 2007. Wind Power Forecasting, in Renewable Electricity and the Grid, Godfrey Boyle, Ed. Sterling,VA: Earthscan, London, at 95-120.David Milborrow, 2007. Wind Power on the Grid, in Renewable Electricity and the Grid, Godfrey Boyle, Ed. Sterling,VA: Earthscan, London, at 31-54 Michael Milligan,Kevin Porter, Edgar DeMeo, Paul Denholm, Hannele Holttinen, Brendan Kirby, Nicholas Miller, Andrew Mills, Mark O’Malley, Matthew Schuerger, and Lennart Soder , 2009. Wind Power Myths Debunked, IEEE Power and Energy, November/December vol 7 no 6, at 89-99.National Grid, 2009. Operating the Electricity Transmission Networks in 2020: Initial Consultation. At http://www.nationalgrid.com/NR/rdonlyres/32879A26-D6F2-4D82-9441-40FB2B0E2E0C/39517/Operatingin2020Consulation1.pdf <last accessed 15 feb 2012>North American Electric Reliability Corporation, 2009b. Accommodating High Levels of Variable Generation, April. At http://www.nerc.com/files/IVGTF_Report_041609.pdf <last accessed 15 feb 2012>NERC, 2010. IVGTF Task 2.1 Report: Variable Generation Power Forecasting for Operations. At http://www.nerc.com/files/Varialbe%20Generationn%20Power%20Forecasting%20for%20Operations.pdf <last accessed 15 feb 2012>Jennifer Rodgers and Kevin Porter, 2009. Central Wind Power Forecasting Programs in North America by Regional Transmission Organizations and Electric Utilities, NREL/SR-550-46763. Available at http://www.nrel.gov/docs/fy10osti/46763.pdf <last accessed 15 feb 2012>

31


National Grid, 2009. Operating the Electricity Transmission Networks in 2020: Initial Consultation. At http://www.nationalgrid.com/NR/rdonlyres/32879A26-D6F2-4D82-9441-40FB2B0E2E0C/39517/Operatingin2020Consulation1.pdf <last accessed 15 feb 2012>

North American Electric Reliability Corporation, 2009b. Accommodating High Levels of Variable Generation, April. At http://www.nerc.com/files/IVGTF_Report_041609.pdf <last accessed 15 feb 2012>

NERC, 2010. IVGTF Task 2.1 Report: Variable Generation Power Forecasting for Operations. At http://www.nerc.com/files/Varialbe%20Generationn%20Power%20Forecasting%20for%20Operations.pdf <last accessed 15 feb 2012>

Jennifer Rodgers and Kevin Porter, 2009. Central Wind Power Forecasting Programs in North America by Regional Transmission Organizations and Electric Utilities.

Royal Academy of Engineering, 2014, Wind Energy : Implications of Large-Scale Deployment on the GB Electricity System http://www.raeng.org.uk/publications/reports/wind-energy-implications-of-large-scale-deployment

has wind energy forecasting solved the challenge posed by ... pdfs... · actual wind energy and a...

Documents