Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

KIM Validation for EO Archived Data

Exploitation Support (KIMV)

Mihai DatcuDLR Oberpfaffenhofen

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

1997 first tests (ETH Zurich) what’s that? ¼ scene

1999 MMDEMO (ETH Zurich) I2M exists and works! 10 scenes

2002 KIM (ESA, DLR, ETHZ, NREC, EUSC) system, appl. Use 20 GB

2003/4 use KIM in projects: SMART, PRESENCE

2003/4 Information Theory for evaluation.

2005 KIMV: operational system, bugs, enhancements accuracy, application scenarios…1 TB, 1m – 1Km, optical and SAR….

….more TB…more users…more sensors TerraSAR

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

KIM/KES system concept

•Knowledge-driven Information Mining (KIM)

•Knowledge enabled services (KES)

•KIM and KES are based on Human Centred Concepts

•Implements improved feature extraction

search on a semantic levelavailability of collected knowledgeinteractive knowledge discoveryshare knowledgenew visual user interfaces

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

KIM and KES systems

1. A library of algorithms which is used to extract the primitive features

2. A machine learning (Bayesian network) algorithm to generate interactively image classifications

3. A data base management system for the image content information catalogue and semantics and knowledge

The systems are helping the user in his analytical task to extract the information; the system records the knowledge and can reuse or communicate it. In addition, KIM and KES adapt to the user conjecture and are designed to operate very fast on large image volumes.

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

System complexity

KIM/KES integrate

natural languagetextnumerical recordsGIS spatial data representationdatabase and visual capabilitiesanalysis of multidimensional pictorial

structurescomputer vision pattern recognition relational data modelsknowledge representation and bases

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

System complexity

Important complexity factors

is the unbalanced ratio between the huge information volume of EO data (i.e. enormous image archives) and the sequential, mainly linguistic, and limited capacity of people to access information

perception of information as ``signals-signs-symbols'' is generally not dependent on the form in which the information is presented but rather on the context in which it is perceived, i.e. upon the intentions and expectations of the perceiver.

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

Validation procedure

objective evaluation of system performance

relevance in real applications with users in the loop, i.e. validation from the “subjective” perspective of the users interested in specific data and applications.

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

The expert evaluators

KIM/KES systems respond to existing and new requirements of a very broad range of applications

aerospace agencies (ESA, CNES, NASA, DLR) satellite centres (EUSC, ARCS) universities and research unitsindustry data providers

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

The tasks

access to information in large EO archivesimage interpretationunderstanding phenomenatarget or objects detectioninformation mining and knowledge discovery

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

The data sets

Sensor Collection TilesERS ers_GEC 571ERS + LANDSAT Mozambique 448HYPERSPECTRAL Presence 4HYPERSPECTRAL/R

Smart 24IKONOS Ikonos 207IKONOS ikonos_geo 651LANDSAT Switzerland 184LANDSAT 5/7 land_IT 468LANDSAT 5/7 ls_urbex 468LANDSAT 7 landsat7 168MERIS meris_120 1099MERIS MerSelectFrame 1039MIXED Nepal 188SPOT 5 cnes_eval 423SPOT 5 cnes_spot50 45SPOT 5 cnes_spotM 45SPOT 5 geo_spot 9SPOT 5 mihai_spot 9

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

The operation modes

Content Based Image Retrieval (CBIR)

CBIR is based on utilization of semantic queriesCBIR enables an operator to “see” into a large volume

Data/Information mining

explore the information content of the images probabilistic image retrieval integrated with interactive learning and image classification

Scene understanding

derive knowledge, interpret or understand the structures and objects

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

The questionnaire

evaluation for information retrieval systems, manmachine communication and image classification

rank the user satisfaction on scale with 4 qualitative

values (very good, good, acceptable, uncertain)

semantic differentials for questionnaire-based system

validation (for characterization of the task, searchprocess, retrieved result and system behaviour)

evaluation of the man-machine communication ( extent

of system functionalities, effects on the user, specific

system like items, and general score)

guideline for a general assessment of the validation

results and suggestions.

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

CBIR results analysis

28%

40%

18%

14%

Very Good

Good

Acceptable

Uncertain

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

I2M results analysis

12%

23%

31%

34%

Very Good

Good

Acceptable

Uncertain

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

SU/Classification results analysis

41%

32%

18%

9%

Very Good

Good

Acceptable

Uncertain

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

Man-Machine Communication

21%

54%

22%

3%

Very Good

Good

Acceptable

Uncertain

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

MERIS: the classification

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

The method

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

Results

    l2_flags classification

  cloud water land

cloud 96% 0,6% 3,4%

water

0,1% 99,9% 0%

land 20,3% 0,5% 79,2%

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

Results

    cloud_type

    cloud water

  

cloud

98% 2%

water

0% 100%

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

Results

    Meris Level 2 Product

  Cloud Water Land

Cloud 97,3% 1,2% 4,3%

Water 3,8% 96,2% 0%

Land 19,1% 0,3% 80,7%

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

Conclusions          cloud and water the classification given by

training the system is more than 90% similar to level 2 product in most of the cases.

          land classification is not as similar as for cloud

and water, this is due to two facts:         land could be covered by cloud

land is a very general concept          ice classification a big difference is detected.

level 2 product classification is considering ice over the water and for as it is a classification of snow.

 snow classification: level 2 product is including the snow in cloud class, meanwhile KIM can separate snow and cloud as two different classes

Rem

ote

Sen

sin

g T

ech

nolo

gy I

nsti

tute

Feature constancy (data models)

Gemetry (HR vs. LR)

SPOT (CNES) data quality

# semantic labels grows with higherresolution

MERIS Landsat ERS SPOT IKONOS

10 100 10 k100 1000