kes: knowledge enabled services for better eo information …csiss.gmu.edu/dad/igarss04_08.pdf ·...

KES: Knowledge Enabled Services for better EO Information Use

Andrea ColapicchioniAdvanced Computer Systems

Space [email protected]

IGARSS 2004 - Image Information Mining

The problem

n During the last decades, the satellite image catalogues have stored huge quantity of data

n State of the art catalogues permit only to specify location, time of interest, metadata like platform, sensor, acquisition mode…


The interpretation task

n The interpretation of EO images requiresn Fusion of data/information for better

understanding of structuresn Aggregation with existing knowledge

specific to the application fields (at higher level)


A little bit of history..

n 1996-2000 IIM (Image Information Mining) (http://isis.dlr.de/mining)

n 2001: KIM (Knowledge Information Mining) (http://www.acsys.it:8080/kim)

n 2002: KES (Knowledge Enabled Services)n 2003: KIMV (KIM Validation) n 2004: KEO (Knowledge-centred Earth Observation)

1996 2006

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

1996 - 2000IIM (DLR, ETH)

2001 - 2002KIM (ACS, DLR, ETH)

2002 - 2004KES (DLR, ACS)

2004 - 2006KEO (ACS, DLR, GTD, CNES)

2003 - 2004KIMV (ACS, DLR)


KIM: Knowledge Driven Information Mining

Images

PrimitiveFeatureExtraction(at pixel level)

010010101101110010

010010101101110010

PrimitiveFeatures:- Texture- Colour- Shape


More in detail


KIM Interactive learning


From data to semantic

Images PrimitiveFeatureExtraction

010010101101110010

010010101101110010

PrimitiveFeatures:- Texture- Colour- Shape

InteractiveLearning

River

Forest

Cloud

Features

DATA SEMANTIC


From KIM to KES (Knowledge Enabled Services)

n Image interpretation is not a simple task. Each user needs a set of accessory data, as for example GIS layers or texts obtained through Internet.

n Yet, the amount of available information makes searches a demanding and expensive task.

n An environment where images are at the focal point, and where each user can navigate through a taxonomically structured knowledge, could be of extreme value.


Other semantic

ImagesPrimitiveFeature

extraction

PrimitiveFeatures

Image Features

Text

GISLayers

InteractiveLearning

Interaction

Interaction

Text Features

GIS Features

DATA SEMANTIC


KES: New interface


Semantic Grouping

n In KIM it is simple to define a feature by identifying a river through positive and negative examples.

n However, the “river" might become part of a wider concept (e.g.: water). This should be implemented without retraining the system for all possible water types.


Aggregated Featuresn In KES a new kind of grouping, called

“aggregated features”, has been introduced.n In a similar way as in defining positive and

negative examples on the image, it will be possible to define the concept “water” as:

n Positive examples: n sea + river + lake + water reservoir

n Negative examples: n streets + houses + mountains


Semantic Grouping

ImagesPrimitiveFeature

extraction

PrimitiveFeatures

Image Features

Text

GISLayers

Interaction

Text Features

GIS Features

DATA SEMANTIC

SemanticGrouping

AggregatedImage Features

InteractiveLearning

Interaction

ObjectDescriptor

Descriptive

GroupingsDescriptive

Groupings

DescriptiveGroupings



Ontology

n Ontology is the specification of a conceptualisation. It can be related to a system (System Ontology, which can be domain independent and reused for different domains) or to a domain (Domain Ontology, specific for that domain).


Domain Ontology

n Domain ontology is the set of definitions and concepts pertaining and belonging to a specific domain (and shared by concerned people).

n Different domains have generally different ontologies. As an example, a climatology expert could have a different vision of (and terms to describe) water compared to that of an oceanography expert.


TaxonomyDomain 1 Domain n

…

Sub domain Sub domain Sub domain

AggregatedImage Feature

ImageFeature

Image Feature

AggregatedImage Feature

Image Feature

Image Image Image Image

TextFeature

GISFeature

Text TextGISLayer

GISLayer



Data / Semantic / Ontology

Images PrimitiveFeatures

Image Features

Text

GISLayers Interaction

Text Features

GIS Features

DATA SEMANTIC

SemanticGrouping

AggregatedImage

Features

Interaction

ObjectDescriptor

Descriptive

Groupings



PrimitiveFeature

extraction

InteractiveLearning

Domains

DOMAINONTOLOGY

ONTOLOGY


Implicit and explicit knowledge transfer

n The KIM prototype permits to associate weighted combinations of primitive features to image features. While defining features, the user explicitly transfers knowledge to the system, which is stored and made available to the same or other users.

n In addition there is a further type of knowledge (implicit) that could be discovered: if the user domain of interest is known, by observing the user interactions with the system during search and browsing, it is possible to infer the data of likely user interest and link it with the pertinence domain.


Knowledge transfer

Lake

Sea

Roads

ERSImage

LandsatImage Meris

Image AlgaeBlooming

ClimateChanges

Cities

Temperature

CensusMap

IKONOSImage Roads

UrbanAreas

Factories

PoliticalFacts


Knowledge Graph

www.a.com

MERISImage

www.a.com

Spectral(feature)

www.a.comIkonosImage

WaterGIS

Water

River

RiverGIS

www.b.com

Hydrology


Knowledge Discovery

Exploring this graph means discovering the user's knowledge

KIMV (KIM Validation)

A practical case study:Automatic cloud classification in

MERIS images


Cloud Cover Characterization

MERIS Level 1 – ReducedResolution

“cloud”

KIMV System


Cloud cover characterization

Map ofCloudobject


MAP Object Extraction

Binarized, closedMAP of “cloud” label


Tiling

(x1, y1)-(x2,y2)…2012345

(x1,y1)-(x2,y2)…. 6012345

ShapeVoteIMG


MASS


Linux Cluster

n Cluster nodes: 10 Dell 1750, dual Xeon 3 GHz

n Database and Application Server: Dell 6600, 4 Xeon 2.8 GHz

RACK 01

Power Vault(14x146 GB)

PowerEdge 1750 (x 5)

PowerEdge 2650

6 6 0 F

1

2

3

4

5

6

1

2

3

4

5

6

PowerVault

3 6 G B

1 0 K rpm F C

36 GB10 K rp m FC 36 GB10 K rp m FC 3 6GB1 0K rp m FC 3 6GB1 0K rp m FC 3 6 GB1 0K rp m F C 3 6 GB1 0K r pm F C 3 6 GB10 K r pm F C 36 GB10 K r pm F C 36 GB10 K rpm F C 36 GB10 K rp m FC 3 6GB1 0K rp m FC 3 6GB1 0K rp m FC 3 6GB1 0K rp m FC

7

121

6

19 241 3 18

3C17203 Super Stack 3

1 2 3 4 5 6 7 8 9 1 0 11 12

1 31 4 15 16 17 18 19 20 2 12 2 23 24

1 2 3 4 5 6 7 8 9 1 0 11 12

1 31 4 15 16 17 18 19 20 2 12 2 23 24

Status

P a c k e t

Status

P a c k e t

Packet - Green = Full Duplex

Green = 100MbpsYellow = 10MbpsStatus -

Yellow = Half Duplex

on = enabled, l ink okflashing = disabled

42

3

1

8

6

7

5Module 1 Module 2

P o w e r / S e l f T e s t

Switch 4400

Giga Switch

17 50P o we rE dg e

X EO N

17 50P ow e rE dg e

XE ON

17 50P ow e rE dg e

XE ON

17 50P ow e rE dg e

XE ON

KVM Switch

Rackable Monitor & Keyboard (1U)


System Context

KIMSYSTEM

Internet

ParallelIngestion Chain

Expert userDirect Connection

MASSTOOLBOX

RDBMS

KIM Server

Internet

Ingestion chain monitor

Direct ConnectionJDBC

WEB SERVICE

Ingestiondaemon

WorkflowEngine

User Client

Internet

MERISARCHIVE FACILITY

MASSPORTAL


KIM prototype

n http://www.acsys.it:8080/kim

n Andrea Colapicchionin [email protected]

Visit the ACS stand for a demo

kes: knowledge enabled services for better eo information …csiss.gmu.edu/dad/igarss04_08.pdf ·...

Documents