smart grids: the big data challenge - université de...

Smart Grids: the Big Data challenge

JOURNÉE ORES DU 19 NOVEMBRE 2015

Zacharie DE GREVE, Lazaros EXIZIDIS, Martin HUPEZ, Vasiliki KLONARI, Benjamin PICART, Jean-François TOUBEAU, François VALLEE

[email protected]

Electrical Power Engineering Unit, Faculty of Engineering, University of Mons

GREDOR Project

mailto:[email protected]

https://gredor.be/

Université de Mons

One selected message from CIRED 2015

2Z. De Grève | Electrical Power Engineering Unit

« … What are the [new] roles of DSOs in the transformation of the energy system ?• DSOs will need to actively manage and operate smarter grids• DSOs will implement the roll out of smart metering• DSOs will become data managers […and enter the world of Big Data…]• DSOs will play a key role in the design and implementation of local energy

policies and the development of smart cities…»

Philippe Monloubou, CEO at ERDF, « Power distribution at the heart of the energy transition »,

CIRED 2015, 15th of June 2015, Lyon

(23rd International Conference and Exhibition on

Electricity Distribution, June 2015, Lyon, France)

Université de Mons

Massive roll out of Smart Metering devices


ERDF (principal DSO in France) Linky project: up to 35 million of metering devices until 2021

ORES: Smart Metering (SM) devices for all the customers in 15 years(starting from 2019) [O. Durieux, Journée ORES du 20/11/2014,

Faculty of Engineering, UMONS]

[P. Monloubou, CIRED2015, Lyon, France]

… (Enexis in The Netherlands, Enel in Italy, etc.)

Université de Mons

Outline

Why do we need data in Smart Grids?

Some data related challenges

Focus on Big Data Analytics in Smart Grids

A. Data characteristics

B. Two fundamental problems and illustrations

Conclusion and perspectives


Université de Mons

Outline








Université de Mons

A great wise man once said…


Why do we need data in Smart Grids ?

Some data relatedchallenges

Focus on Big Data analytics Conclusion and perspectives

A close monitoring of the electrical consumption of households will help improving the energy efficiency

« The <to be completed> energy is the one we do not use. »

• <cheapest> if you are an economist,• <greenest> if you are an ecologist,• <most efficient> if you are an engineer,• …

Université de Mons

From classical networks to Smart Grids

Massive integration of Renewable Energy Sources (RES), typically windor solar, in electricity distribution networks


Towards a coordinated « smart » management of the network to avoidtechnical problems (e.g. overvoltages, congestions)

• A guideline: consume the energy when it is produced, locally if possible (flexibility of demand using financial incentives, storage, etc.)




• Recourse to advanced optimization algorithms (dynamic optimization, in an uncertain environment, with nonlinear equations, and a mix between continuous and integer variables)…

Uncertainty of electrical quantities

• Wind and solar production

• Electrical consumption (or load)

Strongly relies on the observability of the distribution network !

Université de Mons 8Z. De Grève | Electrical Power Enigineering Unit

A simple scenario-oriented Monte Carlo approach

8

RES

PriceLoad

StochasticModels

Trajectories

(or scenarios)

• Probabilities of congestion

Indicators

• Probabilities of over/undervoltage

• Reliability indexes (LOLE, etc.)

Sampling

I.

Power flow

computation

II.

Networkmodel

0. Build stochasticmodels

Historical data



Focus on Big Data analytics


Data for analyzing modern distribution grids

Université de Mons

For building stochastic models of electricalquantities, we need data…

9F. Vallée & Z. De Grève | Electrical Power Engineering Unit

RES TTF/TTR

PriceLoad

SequentialModels

Trajectories

(or scenarios)

• Probabilities of congestion

Indicators

• Probabilities of over/undervoltage

• Reliability indexes (LOLP, etc.)

Sampling

I.

Power flow

computation

II.

Networkmodel

0. Build

Historical data





stochasticmodels

Data for analyzing modern distribution grids

Université de Mons

Outline








Université de Mons

The technical challenges related to data are numerous…

Communication between devices





Data storage

Privacy and security

• Power Line Communication – or PLC – and GPRS (e.g. ERDF in France, ORES, Enel in Italy, etc.)

• Radio transmission (e.g. Enexis in The Netherlands)

• …

Metering technology

[D. Lonneke, Journée ORES du 20/11/2014,

Faculty of Engineering, UMONS]

Université de Mons

The technical challenges related to data are numerous…





Investment strategies for IT infrastructures

Analysis of the recorded data (Big Data Analytics)

• For DSOs: better coordination of the production/consumptionin a Smart Grid context

• For consumers: improve energy efficiency by closelymonitoring consumption

• For energy suppliers: establish client typical profiles

• …

Focus of the restof this talk

Université de Mons

Outline








Université de Mons 14Z. De Grève | Electrical Power Engineering Unit




A. Data characteristicsConclusion and

perspectives

RES and load data are intrisicallysequential

Example of wind speed

Autocorrelation function:

A (biased) estimator:

with:

A “non sequential” signal

A “sequential” signal

Autocorrelation functions of typical wind days:

Day/night cycles= seasonality

KNMI database

http://www.knmi.nl/samenw/hydra/







perspectives

RES and load data are intrisicallysequential

0,0E+00

1,0E-05

2,0E-05

3,0E-05

4,0E-05

5,0E-05

1 8 15 22 29 36 43 50 57 64 71 78 85 92

Synthetic Load Profiles (SLPs)

Source: Synergrid (residential customer)http://www.synergrid.be/index.cfm?PageID=16896&language_code=FRA

(To be multiplicatedby annual indexes)

Market quantities (prices, etc.)

http://www.synergrid.be/index.cfm?PageID=16896&language_code=FRA






perspectives

The sequentiality needs to bemodeled !

To evaluate techno-economic strategies which imply a time coupling (e.g. flexibility)

• Storage

Need to store an history, in order to:

• remain within capacity limits,• control the number of cycles (for a

battery), etc.

• Load shifting

with LS

without LSLoad rebound !






perspectives

Modeling the sequentialityAbundant literature on the topic of stochastic modeling of RES, at various time scales

• Statistical/data mining: Artificial Neural Networks (ANNs), KalmannFilters, Markov chains (hidden or not), etc.

• Physical (e.g. meteo): computational fluid dynamics for wind, etc.

• Hybrid approaches, etc.

Time Series models: a simple and efficient way for generating future RES trajectories/scenarios• Able to mimic the statistical properties of real data

• Stored in a compact form. Ex:

Ex: wind speed

AutoRegressive Moving Average (ARMA)

MA(q) process(innovation)

AR(p) process

Université de Mons

Outline












B. Two fund. problemsand illustrations


Two fundamental problemsI. High dimensionnality of the underlying optimization problems

… ……

……

Scenario tree (e.g. for wind/PV production, load, etc.)

Probability of occurrence of scenario

Computational burden !

1. Clustering techniques to limit the number of scenarii

Ex. 1: a clustering example on wind data

…

2. Orient the sampling by modeling dependencies inherent to data

Ex. 2: wind geographical

correlation

Wind speed for site 2

[m/s]

Wind speed for site 1 [m/s]


Why do we needdata in Smart Grids ?


Focus on BigData analytics


Ex. 1: clusteringon wind speed


Clustering for merging scenariiExample 1: clustering on wind speed

Wind speed

Wind speed

Wind speed

Wind speed

Wind speed

Wind speed

t

t

t

t

t

t

Clustering

X N

…

Wind speed

Wind speed

Wind speed

t

t…

• Generation of a reduced number K of typical days of wind, starting from N days of historical data (N >> K)

• Use of K-means, K-medoids algorithms

[B. Picart et al, to be submitted]








Clustering for merging scenariiKNMI database

• Hourly values of the wind speed and direction

• Since 1950

• 65 stations in Holland

• Open access at:











Power Spectral Density (PSD): describes how the power of a signal is distributed over the different frequencies

Estimated here using the periodogram method

• Why typical days ?

Zoom

Analysis of Schiphol station (1981-1990) shows a peak at a frequency f ≈ 11,56*10-6 Hz, which corresponds to a period T = 1/f ≈ 24h










• Methodology1. Feature selection using Principal Component Analysis (PCA)2. Clustering using K-means/K-medoids3. Associate a typical day to each cluster (centroids ?)

3 dimensional vectors instead of 24 !










• Methodology1. Feature selection using Principal Component Analysis (PCA)2. Clustering using K-means/K-medoids3. Associate a typical day to each cluster (centroids ?)

Step 1 Step 2 Step 3 Step 4

K-means algorithm:

Centroid

Initialize centroids Assign object to nearest centroid

Compute new centroids

Re-assign










• Methodology1. Feature selection using Principal Component Analysis (PCA)2. Clustering using K-means/K-medoids3. Associate a typical day to each cluster (centroids here)

Similar performance than classical k-means but:

+ 3D instead of 24D vectors+ Physical interpretation of the 3 dominant directions (modeling ramps)

To be addressed:

• “Smoothness” of typical days (centroids…)

• “Extreme” scenario

Ongoing: power network reliability analysis(See B. Picart poster for more details)








Two fundamental problemsI. High dimensionnality of the underlying optimization problems

… ……

……

Scenaro tree (e.g. for wind/PV production, load, etc.)

Probability of occurrence of scenario

Computationnalburden !

1. Clustering techniques to limit the number of scenarii

2. Orient the sampling by modeling dependencies inherent to data

Ex. 1: a clustering example on wind data

Ex. 2: wind geographical

correlation

…

Wind speed for site 2

[m/s]

Wind speed for site 1 [m/s]






Ex. 2: modelingwind correlation


« Smart » model sampling strategiesExample 2: modeling wind correlation

• Using the Cholesky decomposition [Villanueva et al, IEEE Trans. on Sus. Energy, 2012]

ARMA model 1

ARMA model 3

ARMA model 2

1. Identify n ARMA models separately, based on historical data

2. Compute the (n x n) correlation matrix R, from historical data

Pearson cofficient

3. Compute the Cholesky decomposition of R









• Using the Cholesky decomposition• Using the Cholesky decomposition [Villanueva et al, IEEE Trans. on Sus. Energy, 2012]

ARMA model 1

ARMA model 3

ARMA model 2

4. Generate correlated wind speed time series

ARMA 1 ARMA 2 ARMA 3

uncorrelated

Ex: KNMI database, Schiphol and Ijmuijgen sites:









• Impact of the correlation on power system reliability indices

[Z De Grève et al, EnergyCon2016, Submitted]

• Ongoing work

Non linear correlation (copula based methods) Complete review of the SoA and comparison on the same test

case Time varying correlation

(Test network with two generation units subject to failures, two wind farms, two loads. Collaboration with Tractebel Engineering)

LOLP: Loss of Load Probability EENS: Expected Energy Not Supplied







Two fundamental problemsII. The case of missing or incomplete data

• SM devices are not installed everywhere,

• sensor failures may generate “holes” in the historical database, etc.

Strategy: extrapolate the missing data based on the available

Ex. 3: a low voltage example using reference Cumulative Distribution Functions (CDFs)

Ex. 4: an approach for filling “holes” in electrical consumption

Université de Mons

And if no SM data is available ?


Example 3: a Low Voltage example using reference CDFs

• (Smart) Metering devices are currently not installed everywhere

SM

HV/MV

MV/LV

SM

MV/LV

Strategy: take advantage of what we already have…

Lack of data more particularly at MV/LV substations (and in LV networks)

• Cluster power of a given area into c= cL + cG categories: cL Demand Components (DCs): residential load, tertiary, industrial,

etc. cG Dispersed Generation Components (DGCs): photovoltaïc, wind,

etc.

LVnetwork

LVnetwork





Ex. 3: building reference CDFs


Université de Mons










• Build c reference Cumulative Distribution Functions (CDFs)

1. normalize energy recordings for nodes withSMs, based on annual produced/consumedenergies,

2. compute c ref CDFs,3. assign the CDFs to nodes without SMs, and

perform analysis (denormalize !).

[Toubeau et al, EnergyCon2016, Submitted]

Available

Data

Université de Mons










• A Low Voltage network with PV (ORES, Flobecq, Belgium)

MV/LVtransformer

1D clustering on annual indexes


Université de Mons










• Comparison with measured data and SLPs on the power exchanged at the MV/LV substation during the month of July

Box plot

CDFs


Université de Mons

Sensors may have failures

35

Client 1 X X ? X

Client 2 X X X X

… X ? X X

? X X X

X X X ?

≈





Ex. 4: fillingholes in data


Example 4: matrix factorization to fill the holesOngoing work with F. Lecron, Management

and Computer Science Group, FPMs

.

• Real data is projected on a space of dimension f < m and f < n• Compute matrixes W and H from the incomplete version of X• W and H yield the missing elements of X thereafter• First tests are ongoing…

Z. De Grève | Electrical Power Engineering Unit

…

Université de Mons

Outline






Conclusions and perspectives


Université de Mons

Conclusions and perspectives

An improved observability of distribution grids is needed to implementSmart strategies: towards the world of Big Data


• Reducing dimensionality (and avoid non realistic states) by using clustering techniques and by orienting the sampling

• Missing (or incomplete) data

• Other issues will appear… (profiling clients ?)

Analysis of metering data (Big data analytics)

New competences for analyzing data in Smart Grids

Signal Processing

Probabilities and Statistics

Machine Learning

Time Series Modeling

Université de Mons

Thank you for your attention


Special thanks to:Lazaros Exizidis(3), Martin Hupez(1), Vasiliki Klonari(1), Fabian Lecron(2), Benjamin

Picart(3), Jean-François Toubeau(4), François Vallée(2)

GREDOR Project

(1) (2) (3) (4)

https://gredor.be/

smart grids: the big data challenge - université de...

Documents