DATABANKONDERGRONDVLAANDERENPlace du DOV dans l'infrastructure

d'information géographique en Région flamande,

accessible via Geopunt.

06/11/2014 - Marleen Van Damme - DOV-coördinator

DOV, soil and subsoil in FlandersState of play

Technologies and new tools in place to deliver INSPIRE

INSPIRE Conference Barcelona, 30/09/2016Marleen Van DammeVlaamse Overheid – Dep. LNEDOV-coordinator

OverviewSituation of DOV

What is DOV? Which data ?1. Infrastructure and new applications in place

“DOV-Verkenner” and “Delfstoffenverkenner”Mercurius

2. Experience so far with data harmonisation3. Enhancing spatial web services of LNE with analitics

Sesam – Kibana - ZabbixGoogle monitorSpatineo Monitor

Conclusions

What is DOV?

URL: http://dov.vlaanderen.beDOV is a cooperation in Flanders:

First established in 1996Open data online since 20023 entities are the core partners:

Mobility and Public Works Department (MOW)Environment, Nature and Energy Department (LNE)Flemish Environment Agency (VMM)

Which data in DOV?DOV offers data and information on thematic areas related tosoil and subsoil:

groundwatergeology

geotechnics

soil

All data is published “as is” within the INSPIRE deadlines

and is available as open data.Re-use is allowed

“gratis open data licentie Vlaanderen V1.2”

Results of a first test available on data harmonisation

DOV data is related to 9 INSPIRE themes

Position in the SDI in Flandersand Europe

Geopunt

DOV

INSPIRE Geoportal

GS Soil

EGDI

DOV : Infrastructure and new applications in place

Database: PostgreSQL, PostGISWeb Applications (“Verkenner”): based on open source software

Java, GWTGeotoolsGeomajas

INSPIRE – Services:Catalog (Geonetwork): CSW >1015 metadata filesView- and download-services (Geoserver) : WMS, WFS/WCS > 788 layers

Application supporting the publication proces (ETLP-tool)

“Mercurius” for DOV - “Ariadne” for MercatorNet

DOV-verkenner

the application is used both by public and for internaldatamangement

public user is an anonymous useraccess for known users goes via e-IDEasy switch between several thematic viewers withspecific tools (ex. Mineral resources – silt model)

Simple and powerfull advanced search possibilities on underlying database (JSON) and on each WFS-layer activatedduring a session

Access to data objects via a permanent URL (starting point for further evolution towards linked open data)

DOV-Verkenner DOV Delfstoffenverkenner

Visualisation of Loess Voxel Model (van Haren et al., 2015

Mercurius / AriadneApplication developed to manage the publication proces of

datasets to INSPIRE compliant webservicesManage the datasets in databases on different levelsManage raster filesManage SLD’sCreating metadataCreating object catalogPublishing OGC services (with automatic updates ifconfigured so)

Testing the publication in a work environmentPublishing either for internal usePublishing for public use

Experience so far withdata harmonisation ? 1 test – 6 steps

Test has be done from technical point of view in order to define a possible workflow, to detect what works and what not, to make an inventory of issues to be cleared out before going in more detail and involving the thematical experts intensively

Analitics – several tools in use

Sesam since 2011• Logging of all requests on

Geonetwork and geoserver

• Own application developed in 2011 –still running

Monitoring Website’s Uptime with Google Docs

• Real time uptime check, requestevery 5 min

Spatineo since 2015• Real time monitoring of response

times, request every 5 min

• Usage analitics : based on ownlogfiles, shipped to Spatineo

Kibana• Logging• Real time monitoring

Zabbix• monitoring

Usage of DOV –Sesam compared to Spatineo

SESAM

Jaar CSW WMS WFS WCS zips

2011 250.000 500.000 0 0 na

2012 1.114.595 2.826.549 23.318 0 na

2013 24.135 8.789.788 2.561.662 2.281 na

2014 1.201.828 47.206.065 26.815.979 6.337 na

2015 1.504.838 60.401.191 26.723.257 2.148.405 1.531

Differences comes from different point of view on recognizing requests

All warnings combined in onemailbox – who’s the first ?

Zabbix versus Spatineo

Spatineo - Performance testing

TestscenarioDOV: 50 req/s in totalMercatorNet: 56 req/s in total

First test on 19/04/2016Initial set up knowing existence of performance issues

Second test on 24/05/2016MercatorNet – no changes –Geoserver running on 2 nodesDOV – Geoserver running on 4 nodes in stead of 2

Vector complex legend

WFS – GML2

Cap. 470 layers

raster

Vector light legend

Vector light legend

WFS –app/gml+xml

DO

VM

erca

torN

et

Spatineo - Performance testing –results on 19/04/2016

DOV

MercatorNet

Spatineo - Performance testing -results on 24/05/2016

DOV

MercatorNet

Kibana during the test comparedto regular usage

DOV - Requestsfrom spatineomonitoring andperformance test included

DOV – requestsfrom “real” users during the testing time window

Kibana versus Spatineo

Spatineo monitor included

Spatineo monitor not included

Max: 3500 requests per min >>> almost 60 requests per second

10.000 requests from Spatineo during the visible time frame

Corresponding views in Spatineo for the same day

Spatineo: Only req. on WMS DOV

Kibana: all req. on geoserver DOV

Conclusions

Providing open data is challengingPositive feedback from users is motivating

Having the possibility to use DOV, our users tell us that they “enjoy” paying taxes

Due to INSPIRE obligations and the open data policy in Europe – governmental tasks related to it have become vital for the whole society. Costs are worth spending due to gain for all involved target users.

Using the services that had been set up for publication of data in own applications leads to even more need to have insight in usage and performance

Several tools from different points of view have to be in place

VISIT US !

www.dov.vlaanderen.be/portaal

QUESTIONS ?Contact us at dov@vlaanderen.be