building a validation database for land cover products...

Building a validation database for land cover products from high spatial resolution images

The Land Cover project of the ESA Climate Change Initiative

Bontemps Sophie1, Achard Frédéric2, Lamarche Céline1, Mayaux Philippe2, Seifert Frank-Martin3, Arino Olivier3, Defourny Pierre1 1: UCLouvain, Belgium; 2: Joint Research Center, Italy; 3: ESA, ESRIN, Italy

Land cover as an operational product

What about products quality?

Validation

• “The process of assessing, by independent means, the quality of

the data products derived from the system outputs” (CEOS-WGCV

definition)

• CEOS 4-level validation approach according to the temporal and spatial coverage of available reference data

CEOS validation

stage Characteristics

Stage 1 Product accuracy is assessed from a small (typically < 30) set of locations and time periods by comparison

with in-situ or other suitable reference data.

Stage 2

Product accuracy is estimated over a significant set of locations and time periods by comparison with

reference in situ or other suitable reference data. Spatial and temporal consistency of the product and with

similar products has been evaluated over globally representative locations and time periods. Results are

published in the peer-reviewed literature.

Stage 3

Uncertainties in the product and its associated structure are well quantified from comparison with

reference in situ or other suitable reference data. Uncertainties are characterized in a statistically

robust way over multiple locations and time periods representing global conditions. Spatial and temporal

consistency of the product and with similar products has been evaluated over globally representative

locations and periods. Results are published in the peer-reviewed literature.

Stage 4 Validation results for stage 3 are systematically updated when new product versions are released and as

the time-series expands.

The CCI Land Cover project

• ESA Climate Change Initiative to provide a comprehensive and timely response to the need for long-term satellite-based products in the climate domain

• Land Cover project to generate global LC maps consistent over time, to match the needs of key users of the climate modelling community.

• Phase 1 (2011-2013) – Phase 2 (2014-2016)

http://www.esa-landcover-cci.org/

The CCI Land Cover project

Envisat MERIS Full & Reduced Resolution 2002-2012 300-1000m

SPOT-Vegetation 1 & 2 1998-2012 1000m

Envisat ASAR 2005-2012 Mainly Wide Swath Mode (150m)

• 3 consistent global LC maps

CCI-LC validation strategy

CCI-LC quantitative accuracy

assessment

1. Stratified random sampling of sites

2. Collection of high resolution imagery

3. Pre-processing of high resolution imagery

4. Development of a validation dedicated tool

5. Generation of the reference dataset (image interpretation)

6. Comparison between reference dataset and CCI land cover products

Stratified random sampling

1. Stratified random sampling of sites – Sampling design : systematic + 2-stage stratification + random

Systematic sampling of the FAO FRA RSS / JRC TREES + diminution of sample intensity with higher latitudes (for equal-probability of samples selection) + stratification for lower intensity in very

homogenous desert areas

1. Stratified random sampling of high-resolution sites – Sampling design : systematic + 2-stage stratification + random

Final random selection => 2600 sampling units distributed globally in a systematic stratified random manner

Stratified random sampling 2

0 k

m

8 km

8 k

m

Collection of high resolution imagery


2. Collection of high resolution imagery To allow the validation of the 3 epochs (2010, 2005, 2000)

Relying on Google Earth facilities

Collecting Landsat imagery for years 2000, 2005 and 2010 (GLS)

2010

2005

2000

Imagery pre-processing



3. Pre-processing of high resolution imagery Per-object approach: selection of segmentation parameters (object size, compactness, etc.) suitable for a wide variety of landscapes to avoid under-and over-segmentation

Under segmentation

Imagery segmentation

Over segmentation


Correct segmentation


Validation interface





On-line interpretation interface derived

from the GlobCover tool


1. Layer box 2. Zooms

3. Tools (navigation, NDVI, select object, paint class)

4. Legend 5. Comment

Step 1: use all information for assign class in the 2010 period Step 2: indicate the level of certainty for the entire interpretation


Step 3: evaluate the change between the 3 epochs, by « back-dating » the 2010 interpretation Step 4: indicate the level of certainty for the entire interpretation


Insular SE Asia: Forest Class

Thailand: mixed land use

Image interpretation





5. Generation of the reference dataset

High-resolution image interpretation

by experts in a standardized manner

on the developed tool

Already a great success … with

great possibilities for future

• All experts involved in validation quite happy with the tool: friendly and easy to use, clear interpretation framework, on-line

• Resulting database: – Plenty information that need to be fully exploited

• 2010 interpretation + level of certainty

• 2010 - 2005 - 2000 change information + level of certainty

• Each sample as a set of objects

– Homogenity of the sample

– Interpretation of all LC classes in the unit in a single LC class, in a mosaic, in LC fractions, etc.

– Used in the CCI-LC project but the use of VHR images coupled with a per-object should allow using it for other maps with lower-higher spatial resolution

Sentinel-2 opportunities

• Use of S2 imagery instead of Google Earth – Less variability in the images to segment (sensor, dates, pre-processing

performance, etc.) => better segments delineation

– Deriving NDVI time profiles from high resolution dataset (S2 being compatible with other HR missions – Landsat 8), to use temporal information inside the sample unit

Broadleaf or needleleaf? Rainfed or irrigated?

Perspectives

• Validation tool based on GLC2000, GlobCover, FAO-FRA RSS / JRC-TREES previous projects lessons learned

• Precursor for operational LC validation activities in collaboration with international scientific community – As a concept:

• Per-object approach inside a given area ensure the scalability of the resulting database => could be used to validate Sentinel-2 LC maps

• Relies on the availability of a large amount of good quality reference imagery at high spatial resolution => boosted by the unique amount of Sentinel-2 data.

– As a tool: could be of use in many other types of activities (validation, in-situ data collection, crowd-sourcing, etc.), all of them being in link with high spatial resolution imagery

• Sentinel-2 to make a significant difference as data source and as application

Thank you for your attention

building a validation database for land cover products...

Documents