s.2.e specifications for data ingestion via sunshine ftp

CIP-ICT-PSP-2012-6 – 325161

of 6

"This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the

Competitiveness and Innovation Framework Programme by the European Community"

(http://ec.europa.eu/ict_psp).

Annex to deliverable D2.4

Specifications for the ingestion of pilot's

consumption and indoor sensor data

via Sunshine's FTP

Revision: v1.0

http://ec.europa.eu/ict_psp

CIP-ICT-PSP-2012-6 – 325161

of 6




REVISION HISTORY AND STATEMENT OF ORIGINALITY

Revision Date Author Description

v0.1 5th May 2014 Luca Giovannini Document created.

V1.0 21st May 2014 Luca Giovannini Document reviewed and updated.

Statement of originality:

This deliverable contains original unpublished work except where clearly indicated otherwise.

Acknowledgement of previously published material and of the work of others has been made through

appropriate citation, quotation or both.

Moreover, this deliverable reflects only the author’s views. The European Community is not liable for any

use that might be made of the information contained herein.


CIP-ICT-PSP-2012-6 – 325161

of 6




Table of contents

1 General principles ............................................................................................................... 4

1.1 Meter mapping file .................................................................................................................. 4

1.2 Consumption and sensor measurement data files .................................................................... 5


CIP-ICT-PSP-2012-6 – 325161

of 6




1 General principles

These guidelines have the purpose of defining how pilot should deliver consumption and indoor sensor

data to Sunshine’s FTP server to be then stored into the central data repository.

The guidelines are applicable to energy consumption readings and indoor sensors measurements from pilot

buildings of Scenario 2 and to electrical energy consumption readings from light lines of Scenario 3. Data

delivery via FTP will be available to pilots both in the baseline data gathering phase (up to Month 21) and in

the subsequent piloting phase of Sunshine’s application.

Data delivery via FTP is just one of the two possible data delivery means offered by the Sunshine’s

infrastructure. Pilot that are able and willing to put in place a web service to expose this kind of data can

instead refer to the guideline that defines how to interface the pilot’s web services with Sunshine’s

platform via ESPI – Green Button data exchange protocol.

The following sections describe in details the type of data to deliver and its format. First of all, pilots have

to compile a mapping file that has the purpose of describing the basic properties of the meter/sensor and

allow to identify which pilot building or light line does it belong to (section 1.1). Then, consumption

readings and sensor measurements have to be grouped together, stored in CSV files and put on the

project’s FTP space (section 1.2).

1.1 Meter mapping file

The mapping file has the purpose of describing the basic properties of the meter/sensor and identifying

which pilot building or light line does it belong to.

A copy of this file has been put on the main pilot folder of the project’s FTP server, already filled with some

example lines: ftp://sunshine.dedacenter.it/PilotName/meter_mapping.xlsx

The part of the mapping file that has to be filled by pilots is in the tab “mapping” and has the following

fields:

Meter/Sensor ID. A unique ID that identifies the meter (or the sensor).

o The format is PPP-nnn, where PPP is a label referring to the pilot (see table in tab “codelists”) and nnn

is a progressive numerical id.

o Example: FER-001.

Feature of Reference. The string identifier of the pilot building the meter/sensor is attached to, as it

appears in the file “Pilot building properties survey.xlsx" compiled by the pilot. For meters attached to

light lines used in Scenario 3, this field has to be filled with a reference name for the light line.

o Example: Scuola Materna Pacinotti


ftp://sunshine.dedacenter.it/PilotName/meter_mapping.xlsx

CIP-ICT-PSP-2012-6 – 325161

of 6




Observable Property. The type of property observed.

o Combo box with selection limited to: Electrical energy, Gaseous fuel, Liquid fuel, Solid fuel, Steam fuel,

Thermal energy, Temperature.

Consumption Reading Type. Refers only to observed properties of energy consumption (so it is not

applicable to Temperature observations) and it specifies whether if readings are absolute (e.g. the

cumulative amount of cubic meters of gas measured by a gas meter from the time it was installed) or

relative (e.g. the amount of cubic meters consumed between the two last measurements).

Unit. The unit the observed property is measured into.

o Combo box with selection limited to: kWh, m3, L, Kg, ton, °C

Measuring Frequency . Describes if consumption readings and sensor measurement are taken regularly

or not and at which frequency.

o Combo box with selection limited to: 15 = every 15 minutes, 1h = hourly, 1d = daily, 1w = weekly, 1m =

monthly, 1y = yearly, ir = irregular intervals.

o If the data collection frequency is not among the options, mark it as irregular.

Cost availability [€/unit]. States if data about energy cost are available in parallel with consumption

measures. This is obviously not applicable to temperature sensor measures. Energy cost data is always

relative to the period of time between the current observation and the previous.

o Combo box with selection limited to: yes, no, n.a.

Description and ID. A description of the sensor nature, location and ID.

o Example: Gas meter for the central heating system [ID: IT221E00001234]

Data File Name. This field is automatically filled as soon as the other field in the same row are. It

provides the string to be used as file name for the consumption/sensor data files described below.

o Example: FER-001_GASR_MCU_1h_nnnnnnn.csv

1.2 Consumption and sensor measurement data files

Data files contain the actual consumption readings (or sensor measurement) for a specific meter for a

specific period of time. Readings have to be grouped together, stored in CSV files and put on the project’s

FTP space where a specific folder has been set up for each pilot:

ftp://sunshineftp.sinergis.it/PilotName/Dynamic_data.

Readings have to be provided with the highest available frequency and can be collected at irregular

intervals.


ftp://sunshineftp.sinergis.it/PilotName/Dynamic_data

CIP-ICT-PSP-2012-6 – 325161

of 6




During baseline data collection the grouping of single readings in a single file can be on a monthly basis,

while during Sunshine’s platform piloting period the grouping have to be done on a daily basis.

The format of the consumption file name is: MMM-MMM_OOOO_UUU_FF_nnnnnnnn.csv

o Where MMM-MMM stands for the meter/sensor ID, OOOO for the observable property, UUU for the

unit, FF for the data collection frequency and nnnnnnnn is a progressive numerical id.

o The basic string for the filename is automatically generated for each meter/sensor in the mapping file,

if the values in the table are properly filled. Example: FER-001_GASR_MCU_1h_nnnnnnn.csv

The format of the data contained in the CSV file is: timestamp;consumption;cost

o Timestamp has to be in UTC and has the format yyyy-mm-dd hh:mm:ss

o Consumption and cost are float values with a point as decimal separator

o Dataline example: 2013-11-01 01:15:00;0.9570;0.23

Cost data can be omitted if unavailable (as specified in the mapping file).

o Dataline example without cost: 2013-11-01 01:15:00;0.9570

No string header must be put in the CSV file

Each relative consumption reading (as well as cost) is an integrated value describing the energy

consumed during an interval of time. Relative consumption reading is assumed to refer to the interval

between the timestamp of the previous reading and the timestamp of the current reading.

Here follows an example of relative consumption data with cost:

[...]

2013-11-01 01:00:00;0.9570;0.23

2013-11-01 02:00:00;0.8340;0.22

2013-11-01 03:00:00;0.9933;0.24

2013-11-01 04:00:00;0.7750;0.19

[...]

And here is an example, for an indoor temperature sensor:

[...]

2013-11-01 15:00:00;22.5

2013-11-01 16:00:00;21.340

2013-11-01 17:00:00;20.9933

2013-11-01 18:00:00;20.7750

[...]


s.2.e specifications for data ingestion via sunshine ftp

Technology