Swiss Experiment EPFL-LSIR Report

Hoyoung Jeung

SwissEx Annual Meeting, Zurich 15th June

Deliverable Overview

Framework Overview

capturedcaptured

queriedqueried

processedprocessed

Data Integration

Requires close collaboration with end-users– BIG efforts to find appropriate solutions to very specific needs– Continuous migrations of data along with hardware evolution and

addition of new types of sensors.

in 2010– SwissEx general (hosted by SLF) – http://montblanc.slf.ch– LTE – http://ltepc.epfl.ch– APUNCH – http://ifu-apunch.ethz.ch:22001/– PermaSense – http://data.permasense.ch

Continuing in 2011– CCES general portal - http://montblanc.slf.ch:22001– CCES public data - http://montblanc.slf.ch:22002/– Radar images and data from APUNCH: http:// ifu-

apunch.ethz.ch:22001– Disdrometer and radar data from Environmental Remote Sensing

Laboratory: http://ltepc3.epfl.ch:22001

Data Repository/Mirroring Services

In addition to the aforementioned instances of GSN, mirror some of the data using additional machines at EPFL.– Near real-time data duplication.– Multipurpose: backup, query load distribution.– Accessible only for authorized users

LSIR repository– Mainly data from SLF, backup purpose

Lafouly repository– Data from a catchment used by the EU project Hydrosys in

Lafouly region. Mainly used for GEOtop model simulations.

PlanetData repository– Mirroring Wannengrat public data at SLF for query load distribution.

Spatial Data Support

Extending GSN for handling geographical data

Support spatially-enabled processing – Data types: point, polyline, polygon,…– Spatial queries: e.g., average wind speed in Lausanne yesterday?

Two implementations– Java topology suite (JTS) integration: used for any DB– PostGIS integration

• Only for PostgreSQL DB• Support spatial index

Output in GML format

Sensor Data Search

Processing in a federated network– Wiki: central point, easy to use GUI based on Google Maps interface– GSN instances: distributed, very fast data visualization

Ontology-based– “temp=temperature=air temp”– W3C SSN + NASA SWEET– On-going work

Plan to scale up– Search beyond swissex data

User-Interactive Data Cleaning

Model-based anomaly detection– User can select model/parameters, parametric/non-parametric

original data stream↓

approximation using user-selected models↓

detecting anomalies↓

user confirmation: anomaly is an actual error?

New Wrapper Developments

Grid wrapper– Reads ESRI grids (e.g., digital elevation models) from files and stores as

binary objects in DB, to be queried using a new web method (/griddata)– Returns grids (ESRI ASCII format or rendered images) for a given time

range or a stream of values for a single cell within a grid (as a time series).

JDBC wrapper– Connects to remote DB directly, without going through standard GSN communica

tion channels– More efficient processing, used in PermaSense

Sensorscope wrapper– The previous Sensorscope wrapper was extended to handle the new

packet format and new sensor types (e.g. Decagon 10HS and Apogee SP-212).

Tweeter wrapper– Enable data fusion between sensor data and Tweeter data in real time

Various Technical Support

Data access via web services– A standard web service was developed for accessing data in GSN

from external tools like Geotop and Alpine3D. – A C++ client for data access using user’s existing programs.

Continuation of the access control component in GSN– GSN discretionary access control for each virtual sensor.  – Access can be made through web interface (using interactive login)

or through passing of parameters using REST web calls or standard web services.

– Direct passing of parameters without going through the interactive web sessions was needed for automatic tools such as Mathlab or R.

Install, configuration, patches

Continuous development of SwissEx wiki

Planned Work

Linking Swissex data – e.g., Linked Open Data (LOD) data– PlanetData (FP7 NoE) consortium– OpenIoT (FP7 strep)– Scalable solution necessary

Completion of ontology-based sensor data search

Cloud-based sensor data management– Mirroring the current data with public/private cloud services

Questions ?