ESDIN/Quality

Antti Jakobsson, WP 8 leader

QKEN meeting in Brussels 5-7 May 2010

ESDIN project info (www.esdin.eu)

Interactive Instruments

Bundesamt für Kartographie

und Geodäsie

Lantmäteriet

National Technical University of Athens

IGN Belgium

Bundesamt für Eich- und

Vermessungswesen

Universität Berlin

EDINA, University Edinburgh

National Agency for Cadastre and

Real Estate Publicity Romania

Helsinki University of Technology

IGN France

Kadaster

Kort & Matrikelstyrelsen

Geodan Software Development & Technology

1Spatial

The Finnish Geodetic Institute

National Land Survey of Finland

Institute of Geodesy,

Cartography and Remote

Sensing

Statens kartverk Statens kartverk

EuroGeographics

Key Goals

Further the ambition of the European Commission to create a European Spatial Data Infrastructure (ESDI) building on the National Spatial Data Infrastructures (NSDI) in Member States. Help member states, candidate countries and EFTA States prepare their reference data for INSPIRE related to Annex I themes in co-ordinate reference systems, administrative units, transport network, cadastral parcels,hydrography and geographical names

ExM Specifications

ERM EGM EBM

INSPIRE DataSpecification

EXM DataSpecification

1. Migration of EuroGeoGraphics harmonised data specifications to INSPIRE

2. Add support for datasets on large scales to

EXM

Large ScaleData Spec.NMCA A

Large ScaleData Spec.NMCA B

Large ScaleData Spec.NMCA C

NMCA BDatasets based on NMCA data specs

NMCA BDatasets based on NMCA data specs

NMCA ADatasets based on NMCA data specs

NMCA ADatasets based on NMCA data specs

DatasetLarge scaleDataset

Large scaleDataset

Medium scaleDataset

Medium scaleDataset

Small scaleDataset

Small scaleDataset

Large scaleDataset

Large scaleDataset

Medium scaleDataset

Medium scaleDataset

Small scaleDataset

Small scale

4. Specify and implement INSPIRE transformation and

content access services

5. Harmonise and implement

licensing and pricing policies

3. Adapt NMCA data maintenance processes to new

requirements

VirtualDataset

Large Scale

VirtualDataset

Large Scale

VirtualDataset

Medium Scale

VirtualDataset

Medium Scale

VirtualDataset

Small Scale

VirtualDataset

Small Scale

Specifications describe harmonised

virtual European EXM datasets,

accessible through distributed INSPIRE Network Services

ERM EGM EBM

INSPIRE DataSpecification

EXM DataSpecification

1. Migration of EuroGeoGraphics harmonised data specifications to INSPIRE

2. Add support for datasets on large scales to

EXM

Large ScaleData Spec.NMCA A

Large ScaleData Spec.NMCA B

Large ScaleData Spec.NMCA C

NMCA BDatasets based on NMCA data specs

NMCA BDatasets based on NMCA data specs

NMCA ADatasets based on NMCA data specs

NMCA ADatasets based on NMCA data specs

DatasetLarge scaleDataset

Large scaleDataset

Medium scaleDataset

Medium scaleDataset

Small scaleDataset

Small scaleDataset

Large scaleDataset

Large scaleDataset

Medium scaleDataset

Medium scaleDataset

Small scaleDataset

Small scale

4. Specify and implement INSPIRE transformation and

content access services

5. Harmonise and implement

licensing and pricing policies

3. Adapt NMCA data maintenance processes to new

requirements

VirtualDataset

Large Scale

VirtualDataset

Large Scale

VirtualDataset

Medium Scale

VirtualDataset

Medium Scale

VirtualDataset

Small Scale

VirtualDataset

Small Scale

Specifications describe harmonised

virtual European EXM datasets,

accessible through distributed INSPIRE Network Services

ExM Specifications and Guidelines• EGM/ERM/EBM specifications migrated into framework of the INSPIRE Data

Specifications – ExM Data specification (medium/small scale)• National specifications migrated into framework of the INSPIRE Data Specifications

– ExM Data specification (large scale)• General Guidelines and rules completed by developed scripts and tools for Edge-

matching and Map generalisation• General specifications and guidelines for sustainable maintenance at the European

level for Stable Unique Identifiers, Incremental update delivery and Geo Rights Management services

• Data policy and pricing guidelines• Prepare guidelines for the creation of discovery metadata and data evaluation• Develop a quality model.

Page 6

Configuration:Transformationbetween EXM and NMCA A specifications

Configuration:Transformationbetween EXM and NMCA A specifications

DatasetNMCA B

Large scale

DatasetNMCA B

Large scale

DatasetNMCA B

Medium scale

DatasetNMCA B

Medium scale

DatasetNMCA B

Small scale

DatasetNMCA B

Small scale

Transforming WFS / WMS(Download / View Service)Transforming WFS / WMS

(Download / View Service)

ESDI Applications(Test environment)

Configuration:Transformationbetween EXM and NMCA B specifications

Configuration:Transformationbetween EXM and NMCA B specifications

tests the INSPIRE Data Specification as well as

the transformation to and from the underlying

NMCA data

Generalisation as part of NMCA

business processes

Dataset is virtual only and information derived

on-the-fly by a generalisation service

DatasetNMCA A

Medium scale

DatasetNMCA A

Medium scale

DatasetNMCA A

Small scale

DatasetNMCA A

Small scale

Transforming WFS / WMS(Download / View Service)Transforming WFS / WMS

(Download / View Service)

Dataset is virtual only and information derived

on-the-fly by a generalisation service

Coordinate transformation serviceCoordinate transformation service

Rights management servicesRights management services

Edge-matching serviceEdge-matching service

Quality evaluation serviceQuality evaluation service

Service Infrastructure

Background

The ”famous” concept of quality in ISO 19113

Present ISO 19113

Basic quality measures in ISO 19138

ISO TS 19158 Quality assurance of data supply

• Project accepted 2009 and TS expected 2011• A framework that facilitates the production or update of a product to meet

quality requirements• A quality evaluation process may be used in different phases of a product

life cycle, having different objectives in each phase. The phases of the life cycle considered here are specification, production, delivery, use and update

• An organisation applying the TS has to consider

– Quality requirements– Identification of processes– How to measure quality during production or update– Introduce accredition of its supply procesess and personell

Terminology

ESDIN approach

ESDIN approach to quality

ESDIN approach to quality

Quality Evaluation process according to ESDIN Quality Model (D8.1)

Check conformancewith AQLsIf not acceptablethen correct

Metadata Scope dataset

Scope: dataset

Conformance levelsdescribedby error rate (AQLs), errorcountand CE95

Select from tables qualitymeasures for feature typesand attributes to beevaluated

Test completeness, thematic accuracyand positionalaccuracy bysampling

Test report

Test logicalconsistence and temporal accuracywith full inspection

Report errorcount=0

Report errorrate as DQL/ and CE95 based on requirements

CompletenessThematic accuracyPositional accuracy

Reporterror rate/ and CE95

feedback

Scope: subset(sample)

Quality Model content (D8.1)A. Identification of the objectives

and requirements of quality at feature type level

– Analysis and identification of quality requirements

– Selection of quality parameters requires the (prior) identification of:

• the data quality elements and subelements (ISO 19113)

• the quality measures (ISO 19138) • the acceptance criteria and

conformity levels of quality. These conformity levels may be set as declared quality levels (DQLs) which then are reported in metadata.

 

B. Evaluation of the spatial data The evaluation procedures according to ISO

19114. The production and metadata recording

process (ISO 19115 or quality report).

Future editing• Quality Benchmark with EuroGeographics Quality KEN done• Quality requirements for ESDIN datasets togther with WP6 and

WP7• Small scale version after WP6 has done the specification (on going)

ESDIN approach to quality

ESDIN Metadata Framework• DoW obj WP8

– To define data quality reporting and metadata guidelines for large medium and small scale topographic and administrative reference information.

• Objectives WP8 regarding metadata (from Dow)– To provide users of reference information harmonized metadata

concepts for discovery and data evaluation purposes. Guidelines for the creation of discovery metadata and data evaluation for the data providers of reference information will be developed.

• Standard orientated approach– Foundation standards from the ISO 19100 series

Toward a general use case

data application

user

Metadatabrowser

<<Actor>>Register

<<Actor>>Metadata

Repository

search engine

<<Actor>>Repository

<<Actor>>Data

Repository

Discovery

Exploration/Evaluation

Exploitation/Use

Management

ManagementTools

manager

ESDIN metadata vision

• ESDIN metadata vision established (draft document)

• Requirements analysis for ESDIN metadata content ongoing

• ESDIN metadata Guidelines draft established ( january 2010)

– Content• ESDIN Discovery metadata (based on INSPIRE IR)• ESDIN Evaluation metadata (NMCAs req + INSPIRE TWG (v3))• Tbd : ESDIN DQ elements, ESDIN WP3, ESDIN WP7

• Final deliverable scheduled 31 March 2010-> June 2010

ESDIN Metadata status

ESDIN approach to quality

Where you utilize quality webservices?• If you are a data provider for SDI

– For quality control during production (automated) called here conformance testing

– For quality evaluation after the production (semi-automated)

• If you are the SDI co-ordinator or data custodian

– For quality audit for process accreditation or data certification doing either conformance testing and/or quality evalution

• If you are customer or data user

– To evaluate usability using metadata information

EB

ME

BM

NMCA1NMCA1

GeoRM Layers

Applications and Geoportals

NMCAMaster

data

NMCAMaster

dataExMLarge

ExMSmall

Trans-formation

Trans-formation

Dow

nloa

dD

ownl

oad

TransformationTransformation

Quality evaluation

Quality evaluation

Conformance testing

Conformance testing

Edge-matching

Edge-matching

Conformance testing

Conformance testing

Edge-matching

Edge-matching

GeneralizationGeneralization

Conformance testing

Conformance testing

Generalization

Generalization

NMCAMaster

data

NMCAMaster

dataExMLarge

ExMSmall

Trans-formation

Trans-formation

Quality evaluation

Quality evaluation

Conformance testing

Conformance testing

Conformance testing

Conformance testing

Conformance testing

Conformance testing

Generalization

Generalization

NMCA2NMCA2

GeoRM Layers

ERM

ERM

EBM

EBM

EGM

EGM

ER

ME

RM

Dow

nloa

dD

ownl

oad

EG

ME

GM

Dow

nloa

dD

ownl

oad D

ownload

Dow

nload

EB

ME

BM

ER

ME

RM

Dow

nloadD

ownload

EG

ME

GM

Dow

nloadD

ownload

Euro-Geographics

Euro-Geographics

Dow

nloa

dD

ownl

oad

EB

ME

BM

ER

M<

ER

M<

Dow

nloa

dD

ownl

oad

EG

ME

GM

Dow

nloa

dD

ownl

oad

GeneralizationGeneralization

ExMMedium

ExMSmall

ExMLarge

GeoRM Layers

Datasets downloaded for user applications

EBM, ERM and EGM are just specific download services from respective ExMsEG services are ’cascading’ to NMCA servicesEG GeoRM is a ’broker’ for NMCA GeoRM

This NMCA derives both ExM Large and ExM Medium from its Master data

This NMCA derives ExM Large from its Master data, and generalize ExM Medium from ExM Large

Edge-matching does not necessarily need to be performed twice, generalization of edge-matching should still be edgematched

ExMMetadata

ExMMetadata

MetadataMetadata

MetadataMetadata

ExMMedium

ExMMedium

Addi-tional data

Addi-tional data

Conformance testing

Conformance testingEBMEBM ERMERM EGMEGM

Demonstration

• Goal1: to meet INSPIRE/ESDIN quality requirements and to report conformance (producer)

• Goal2: To evaluate usability of reference data for GMES reference data services

Goal1:Quality Evaluation • National Land Survey of Finland is the official source of INSPIRE

Annex I reference data in Finland. It will utilize the web service to evaluate if they have met the quality model requirements of INSPIRE/ESDIN before publising data in ESDIN services

• The same procedure may be used for GMES reference data provision when NLS is the accredited provider for GMES

Demo: Semi-Manual Quality Evaluation Service

• Based on server/client architecture:

– Server: Open source 52North Web Processing Server (WPS)

http://52north.org/

– Client: Open source OpenJUMP GIShttp://www.openjump.org/

• Demo Test Case: Positional Accuracy– Test Dataset: Point dataset

– Test Item: GM_Point

– Test Purpose: To determine and test the process for ISO 3951 based variable sampling and inspection

– Test Method: An implementation of a quality measure ID 45 (circular error at 95 % significance level (CE95)), ISO 3951 based sampling and quality conformance evaluation

Demonstration• demo_projekti_4.htm

Goal1: Conformance testing• IGN Belgium and Dutch Kadaster are providing Esdin medium scale

data for the EuroRegionalMap. Goal is to check conformance during and after the generalization and edge-matching processes

• The same procedure may be utlized when IGN Belgium and Dutch Kadaster are accredited provider for GMES services

Automatic Conformance Testing& Quality Assured Data Processing

Radius StudioDemonstration

Goal2: Evaluation usability for GMES reference data services

• Two possibilities: taking data that is not from accredited sources:

– Usability criteria taken from GMES specifications– Deriving data quality criteria from usability criteria– Checking that minimum criteria is met from metadata

• If using accredited sources then just checking the conformance is met

Conclusions• Introducing a quality model which uses a same principles for all

Annex I themes -> we will suggest this a guideline for INSPIRE implementation

• Introducing comformance levels that can be evaluated using semi-automated or automated based on ISO standards

• Metadata Guidelines for both discovery and evaluation metadata• Automation of quality evaluation and conformance testing• Considering accreditation principles and usability