feature engineering to improve time series forecasting › uploads › 1 › 3 › 4 › 0 ›...
TRANSCRIPT
![Page 1: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/1.jpg)
Ricardo BessaINESC TEC, Portugal
Feature Engineering to Improve Time Series Forecasting
EES-UETP: Advanced Data Analytics for Energy Systems
September 3-5, 2018Porto, Portugal
![Page 2: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/2.jpg)
Course Module Outline
1 Feature Engineering Concept
2 Manual and Automated Feature Engineering
3 Case Study: Solar and Wind Power Forecasting
4 Case Study: Electricity Price Forecasting
5 Other Use Cases and Concluding Remarks
![Page 3: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/3.jpg)
Feature Engineering Concept
![Page 4: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/4.jpg)
Feature Engineering
Motivation
Improve forecasting skill by extracting
additional information from raw data
(case studies ahead…)
Explain data and uncover relevant
information from unsupervised learning
(one example in this tutorial…)
Process information from
continuous streaming data
internet-of-things
➢ Aggregate information from
multiple sensors
➢ Explore multilevel information
(hierarchical features)
![Page 5: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/5.jpg)
Feature Engineering
Concept
CRoss Industry Standard Process for
Data MiningCRISP-DM
Feature engineering area of the process
OPTIONS
➢ Manual creation of features with domain knowledge
➢ Automatic extraction of features, e.g. deep learning techniques
![Page 6: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/6.jpg)
Feature Engineering
Toy Example
Predicting the life time of a specific component within the car or the equipment → forecast horizon is longer-term (days, weeks or months)
Sensors will produce data every second (atomic level)
Data needs to be aggregated over time to understand meaningful trends and changes that will signify an impending failure (aggregated level)
features feeding the statistical learning model
Example later on electricity price forecasting
![Page 7: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/7.jpg)
Manual and Automated Feature Engineering
![Page 8: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/8.jpg)
Manual Feature Engineering
Temporal Aggregation
Temporal aggregation of data
Overlapping windowOne may use various definitions of creating the time windows and then naturally these windows will overlap
Fixed time windowAggregation is performed over a specific, uniform time interval, e.g. 15-min
Variable time windowVarious measures can be used to generate variable time windows. In most cases, a specific number of occurrences of events are used to determine the window size
Exponentially expanding or exponentially contracting time windowsAggregate the near-term data across more granular time windows, while data that is further off, may be aggregated across a wider window [exponentially expanding windows]
The opposite is referred to as exponentially contracting window
1
![Page 9: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/9.jpg)
Manual Feature Engineering
Basic Features
Creation of basic features2
Simple features involving one type of signal▪ Change over time: Cm+1 = (Xm+1 – Xm)/(tm+1 – tm)▪ Rate of change over time: RTm= (Cm+1 – Cm)/(tm+1 – tm)▪ Growth or decay: Gm+1 = (Xm+1 – Xm)/Xm
▪ Rate of growth or decay: RGm= (Gm+1 – Gm)/(tm+1 – tm)▪ Count of values above or below a threshold value▪ Moving average = Average of (Xm-p to Xm)▪ Moving standard deviation = Standard deviation of (Xm-p to Xm)▪ Relative average = Moving average / Global average▪ Relatives standard deviation = Moving standard deviation / Global standard deviation▪ Ratio of changes, growth rate etc. with standard deviation▪ Features involving trend of values across various aggregation windows: change and rate of change in
average, standard deviation etc. across windows
![Page 10: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/10.jpg)
Manual Feature Engineering
Basic Features
Creation of basic features2
Using multiple time series to create large number of combined features▪ Simple ratio between the two series▪ Ratio of changes, rate of change and growth between the two series▪ Ratio of moving average and moving standard deviation▪ Ratio of relative averages and relative standard deviation▪ Relative first difference: RV1m+1 = (XAm+1 – XAm)/(XBm+1 – XBm)▪ Relative second difference: RV2m+1 = (GAm+1 – GAm)/(GBm+1 – GBm)▪ Count of cases where Growth of both series is positive or negative▪ Count of cases where Growth of both series is in opposite direction▪ Count of cases where the first series is above a threshold and the second below a threshold and vice-a-versa
![Page 11: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/11.jpg)
Manual Feature Engineering
Advanced Features
Features based on higher order statistics3
▪ Moments (mean, variance, skewness and kurtosis etc.) are calculated within the aggregation window▪ In non-Gaussian time series, cumulants are used rather than the moments for obtaining information
about the nature of the distribution within the window
Features based on series transformation4
Mathematical transformations can be used to decompose the time series into a set of simpler functions. Usually the following transformations are attempted:▪ Fast Fourier Transform▪ Hilbert Huang Transform▪ Wigner Ville Distribution▪ Wavelet Transformation
![Page 12: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/12.jpg)
Automated Feature Engineering
Available Software
➢ Automatically create features from a set of related tables
➢ Method known as Deep Feature Synthesis (Kanter and Veeramachaneni, 2015)➢ Deep feature synthesis stacks multiple
transformation and aggregation operations (which are called feature primitives **analogues to basic features in previous slides) to create features from data spread across many tables
Source: Kanter, J. M., & Veeramachaneni, K. (2015, October). Deep feature synthesis: Towards automating data science endeavors. In Data Science and Advanced Analytics (DSAA), 2015. 36678 2015. IEEE International Conference on (pp. 1-10). IEEE.
![Page 13: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/13.jpg)
Automated Feature Engineering
Representation Learning Framework
CLASSICALPrincipal Components Analysis (PCA)(+) simplicity(-) linear (non-linear versions require parameter tuning)
Source: Angermueller, C., Pärnamaa, T., Parts, L., & Stegle, O. (2016). Deep learning for computational biology. Molecular systems biology, 12(7), 878.
![Page 14: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/14.jpg)
Automated Feature Engineering
Convolutional Extraction
Source: Angermueller, C., Pärnamaa, T., Parts, L., & Stegle, O. (2016). Deep learning for computational biology. Molecular systems biology, 12(7), 878.
Input Image
![Page 15: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/15.jpg)
Automated Feature Engineering
Stack of Restricted Boltzmann Machines(RBM)
Source: Testolin, A., Stoianov, I., De Filippo De Grazia, M., & Zorzi, M. (2013). Deep unsupervised learning on a desktop PC: a primer for cognitive scientists. Frontiers in psychology, 4, 251..
Source: Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. science, 313(5786), 504-507.
![Page 16: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/16.jpg)
Case Study: Solar and Wind Power Forecasting
![Page 17: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/17.jpg)
Case-Study
PV Installation in INESC TEC Building & Sotavento Wind Farm in Spain
Data available upon request
Raw NWP dataset: 2704 variables for the wind power plant and 1014 variables for the PV site
Clear case for feature engineering: how much information can be extracted from this raw data?
![Page 18: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/18.jpg)
Local NWP Information
PV Forecasting
Clear SkyPartially Cloud Cover OvercastLo
cal N
WP
info
rmat
ion
Tem
po
ral Varian
ce
![Page 19: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/19.jpg)
Local NWP Information
PV Forecasting
Clear SkyPartially Cloud Cover Overcast
Loca
l NW
P in
form
atio
n
![Page 20: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/20.jpg)
Grid NWP Information
PV Forecasting
Clear SkyPartially Cloud Cover OvercastSp
atia
l Gri
d N
WP
info
rmat
ion
![Page 21: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/21.jpg)
List of Created Features
PV Forecasting
![Page 22: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/22.jpg)
PV Forecasting Framework
Model Chain
NWP for the client location
Grid of NWP
Gradient Boosting Trees
PV Power Forecasts
(point & probabilistic)
Temporal Features+ Point Forecasts Spatial Features
![Page 23: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/23.jpg)
Forecast Example
PV Forecasting
Probabilistic forecasts• Uncertainty better modeled around the observed values• Some of the abnormal high uncertainty verified for clear-sky days is removed
Point forecasts• Some of the over/underestimation situations are resolved• Improvements on the power peak forecasts for some clear-sky days
Temporal Information Temporal & Spatial Information
![Page 24: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/24.jpg)
Numerical Results
PV Forecasting
Local Temporal Information
Grid Spatial Information
Combination temporal & spatial
inputs
(best overall model)
24h Forecast Horizon
𝐼𝑚𝑝𝑟𝑜𝑣𝑒𝑚𝑒𝑛𝑡 = 1 −𝑚𝑒𝑡𝑟𝑖𝑐𝑚𝑜𝑑𝑒𝑙
𝑚𝑒𝑡𝑟𝑖𝑐𝑏𝑎𝑠𝑒∙ 100%
![Page 25: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/25.jpg)
Local NWP Information
Wind Power Forecasting
![Page 26: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/26.jpg)
Local NWP Information
Wind Power Forecasting
![Page 27: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/27.jpg)
Spatial NWP Information
Wind Power Forecasting
Power and Spatial NWP Grid Comparison
![Page 28: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/28.jpg)
List of Created Features
Wind Power Forecasting
Lo
ca
l In
form
ati
on
Do
ma
in K
no
wle
dg
e
![Page 29: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/29.jpg)
Automated Features from NWP Grid
Wind Power Forecasting
Without considering the spatial relationbetween the variables
extractedfeatures
extractedfeatures
with the spatial relationbetween the variables
![Page 30: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/30.jpg)
Automated Features from NWP Grid
Wind Power Forecasting
Lo
ca
l In
form
ati
on
Au
to-E
nc
od
ers
![Page 31: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/31.jpg)
Numerical Results
Wind Power Forecasting
24h Forecast Time Horizon
𝐼𝑚𝑝𝑟𝑜𝑣𝑒𝑚𝑒𝑛𝑡 = 1 −𝑚𝑒𝑡𝑟𝑖𝑐𝑚𝑜𝑑𝑒𝑙
𝑚𝑒𝑡𝑟𝑖𝑐𝑏𝑎𝑠𝑒∙ 100%
![Page 32: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/32.jpg)
Numerical Results
Wind Power Forecasting
![Page 33: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/33.jpg)
Forecast Example
Wind Power Forecasting
Base model
Model spatial & temporal data
![Page 34: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/34.jpg)
The Impact of Feature Engineering
In Contrast to Probabilistic Forecasts Generated with Weather Ensembles
Probabilistic forecast generated with a weather ensemble model
Probabilistic forecast generated with a feature engineering + GBT
![Page 35: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/35.jpg)
Case Study: Electricity Price Forecasting
![Page 36: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/36.jpg)
Electricity Price Forecasting
Motivation
Statistical learning methods heavily dependent onthe availability of sufficient historical data withhigh (or low) price regimes
highest prices during thistwo years period
![Page 37: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/37.jpg)
Feature Engineering Approach
Electricity Price Forecasting
combine additional (in addition to exogenous variables) information with the statistical model
forecast the daily average price including information from daily futures contracts
(proxy forecast of spot price)
![Page 38: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/38.jpg)
Feature Engineering Approach
Statistical Models
Linear Median Regression
➢ Day of the week, month of the year➢ Past daily average prices➢ Daily futures contracts➢ D-1 generation: coal, wind and PV
Forecast Daily Average Price
Gradient Boosting Trees
➢ Day of the week, hour of the day, month of the year➢ Past daily average prices➢ D-1 generation: coal, hydro w/reservoir➢ Forecasts: load, wind power penetration, PV, solar thermal
Forecast Day-ahead Price
The analysis of cross-effects between variables needs to be analyzed within the model (model-specific)
![Page 39: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/39.jpg)
Feature Engineering Approach
The Impact of the Average Price Feature in the Forecasting Skill
without rescaling
with rescaling
➢ Low price regime
![Page 40: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/40.jpg)
Feature Engineering Approach
The Impact of the Average Price Feature in the Forecasting Skill
without rescaling
with rescaling
➢ High price regime
![Page 41: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/41.jpg)
Day-ahead Price Forecasting
Results
without rescaling with rescaling
![Page 42: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/42.jpg)
Features for Intraday Price Forecasting
Data Analysis
All intraday sessions are strongly influenced by the day-ahead prices
Each intraday session is highly correlated with the previous one
![Page 43: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/43.jpg)
Features for Intraday Price Forecasting
Model
Linear Quantile Regression
i-th intraday session
𝑆𝑖 𝜏 = ൞
𝑐 + 𝛽7 × 𝑃𝐷𝐴, 𝑖𝑓 𝑖 = 1
𝑐 + 𝛽7 × 𝑃𝐷𝐴 +𝑗=1
𝑖−1
𝛽8+𝑗−1 × 𝐼𝐷 𝑗 , 𝑖𝑓 𝑖 > 1
where c comprises calendar variables
𝑐 = 𝛽0 + 𝛽1 × 𝐶𝐻,𝑐𝑜𝑠 + 𝛽2 × 𝐶𝐻,𝑠𝑖𝑛+ 𝛽3 × 𝐶𝑊𝑑,𝑐𝑜𝑠+ 𝛽4 × 𝐶𝑊𝑑,𝑠𝑖𝑛+ 𝛽5 × 𝐶𝑀,𝑐𝑜𝑠 + 𝛽6 × 𝐶𝑀,𝑠𝑖𝑛
![Page 44: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/44.jpg)
Intraday Price Forecasting
Results
![Page 45: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/45.jpg)
Deep Learning for Electricity Markets
Curve Forecasting
CONCEPT
GOALForecast the 24 residual demand curves from the day-ahead market
Demand
Supply Residual demand
![Page 46: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/46.jpg)
Type of Features
Curve Forecasting
Long Short Term Memory networks
1-Dimensional input vector
-0.49446
-0.407376
-0.407376
…
…
0.39716
0.420512
0.731689
![Page 47: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/47.jpg)
Type of Features
Curve Forecasting
Long Short Term Memory networks
2-Dimensional input vector
![Page 48: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/48.jpg)
Automatic Feature Extraction with LSTM
1-D and 2-D Approaches
1-D 2-D
Exogenous variables(solar, load, windforecast, wind andsolar penetration).
![Page 49: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/49.jpg)
Illustrative Results (1-D)
Iberian Electricity Market
![Page 50: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/50.jpg)
Illustrative Results (2-D)
Iberian Electricity Market
![Page 51: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/51.jpg)
Other Use Cases and Concluding Remarks
![Page 52: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/52.jpg)
Distribution Grids
Visualization of Low Voltage Grid Operation
21 22 23
24 25 26 27 28
1
2 3 4
5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20
29 30 31 32
33
Distributed Stochastic Neighbor Embedding (t-SNE)
2125 belief states
PCA (no information!)
![Page 53: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/53.jpg)
Classify Events in Transmission Networks
Using Data Collected by Phasor Measurement Units (PMU)
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
Generator tripping Load shedding
Line tripping Oscillation
❑ Events collected at several PMUs simultaneously in Brazil (MedFasee project)❑ Frequency data collected at the rate of 1 measurement per 1/60 second
60.4 Hz
59.5 Hz
60 Hz
60.4 Hz
59.5 Hz
60 Hz
60.4 Hz
59.5 Hz
60 Hz
60.4 Hz
59.5 Hz
60 Hz
![Page 54: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/54.jpg)
Classify Events in Transmission Networks
Results
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60
model error in event recognition
Deeper Feedforward ANN 1,5 %
Deep Belief Networks 8,5 %
Convolutional NN 30x40 0 %
Look: can you see?
Layer 1 Layer 2 Layer 32D image
![Page 55: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/55.jpg)
• Feature engineering can result in significant improvements for renewable energy forecasting
• #personal opinion# more important than the choice of the statistical learning model
• Deep learning (frame representation) and manual feature construction can be combined to extract meaningful information from the raw NWP dataset
• Several business cases exist for feature engineering, including data-driven optimization (e.g. reinforcement learning)
• Additional information is very critical when producing uncertainty forecasts
• Feature engineering can also be used for machine learning model interpretation and big data visualization
Concluding Remarks
![Page 56: Feature Engineering to Improve Time Series Forecasting › uploads › 1 › 3 › 4 › 0 › 13407469 › feature... · Feature Engineering to Improve Time Series Forecasting EES-UETP:](https://reader033.vdocument.in/reader033/viewer/2022060211/5f04be397e708231d40f7ba4/html5/thumbnails/56.jpg)
J.R. Andrade, R.J. Bessa, “Improving renewable energy forecasting with a grid of numericalweather predictions”, IEEE Transactions on Sustainable Energy, vol. 8, no. 4, pp. 1571-1580,Oct. 2017.
J.R. Andrade, J.M. Filipe, M. Reis, R.J. Bessa, “Probabilistic price forecasting for day-ahead andintraday markets: Beyond the statistical model,” Sustainability, vol. 9, no. 11, pp. 1990, 2017.
R.J. Bessa, C. Möhrlen, V. Fundel, M. Siefert, J. Browell, S. Haglund El Gaidi, Bri-Mathias Hodge,U. Cali, and G. Kariniotakis, “Towards improved understanding of the applicability ofuncertainty forecasts in the electric power industry,” Energies, vol. 10, no. 9, pp. 1402, 2017.
L. Cavalcante, R. J. Bessa, M. Reis, J. Dowell, “LASSO vector autoregression structures for veryshort-term wind power forecasting,” Wind Energy, vol. 20, no. 4, pp. 657-675, April 2017.
V. Miranda, P. Cardoso, R.J. Bessa, “Through the looking glass: seeing events in power systemsdynamics,” working paper, 2018.
M. Pereira, R.J. Bessa, C. Gouveia Moura, “Low voltage grid data visualization with abiologically inspired cognitive architecture,” working paper, 2018.
References
INESC TEC Work