infrastructure for service support may 2004 service infrastructure within the esa gs pier giorgio...

May 2004Infrastructure for Service Support

Service InfrastructureService Infrastructurewithin the ESA GSwithin the ESA GS

Pier Giorgio Marchetti – ESA / ESRIN+39 0694180413

Cell. +39 [email protected]

Ground Segment Department


adding value to data…adding value to data…means supporting servicesmeans supporting services

Archiving

Acquisition

Processing

Dissemination

Catalogue

Ordering

DataEnvironment

Mission Planning

Quality Control

Support 4 Service Providers

Service Support

Support 4 users

ServiceEnvironment

Order Desk

UserEnvironment

Help Desk


SER

VIC

ES

IN

FR

AS

TR

UC

TU

RE

Oxygen Ground Segment ModelOxygen Ground Segment Model

Help Desk

Web Portal

Catalogue

Data Ordering Use

r Serv

ices

Mission Planning

Quality Control

CO-ORDINATION CENTRECO-ORDINATION CENTRE

Network Network

Acquisition

Mon

itori

ng &

C

ontr

ol

FACILITYFACILITY

Archive

ProductionDissemination Info

rmati

on

Min

ing (

futu

re)


Services Infrastructure – From …Services Infrastructure – From …

Non EO Data

Service Provider

Value Adder

Service Provider

Distri-butor

Value Adder

Value Adder

Data Environment

Data Environment

Data Environment

User Envir.

User Envir.End

User

EndUser


……To - Services InfrastructureTo - Services Infrastructure

Data Environment

Data Environment

Non EO Data

Service Provider

Data Environment

User Envir.

EndUser

Value Adder

EndUser

Service Provider

Distri-butor

Value Adder

Value Adder

User Envir.

IIM

IIM

IIM

IIM


Service ????Service ???? Service

A Service is the repeated, regulated (usually by means of a Service Level Agreement) and agreed (via contract or agreement) or declared (with defined quality parameters) exploitation (for commercial or non-commercial purposes) of a function, processor or application. The output of a service includes data, products, information and solutions.

Web Services Please refer to W3C http://w3c.org


Service Infrastructure – the usersService Infrastructure – the users

Users

Service Providers

Service Consumers


Service Infrastructure – the interactionsService Infrastructure – the interactions

Users

Service Providers

B2B

Co-operation

B2C

Service Exploitation

Service Consumers


SI – high level requirementsSI – high level requirements

Empower Service Providers:• Service Providers maintain full control over their own

infrastructure

• “Neutral” service infrastructure

Empower GS harmonisation• Simple catalogue service

• Order service

• Generic service activation

Shortcut data “format” issue:• Concentrate on supply chain orchestration

and flow control

• Let Users (and COTS) rule data format issues

Online, nrt or offline service execution are all the same

Foster long term research & industry growth

• rely on widely accepted and open standards (see OGC, W3C and ISO TCs)

Support requirements from relevant actors and European Programmes

• EC-DG INFSO, JRC, EUSC

• GMES, INSPIRE,…

Users

Service Providers

B2B

Co-operation

B2C


Service Consumers


SI – the functional objectivesSI – the functional objectives

Service support environment for the integration of Earth Observation and GIS supply chains

Support for data providers, service providers and end-users alike

Supply chain orchestration uses web services and workflow strds

Open and distributed architecture, allowing to add new supply chains

User driven/selected service delivery

Users

Service Providers

B2B

Co-operation

B2C


Service Consumers


SI – the user…SI – the user…

Multiple utilisation scenarios• Service Provider co-operation

• Test and fast/cheap deployment of new supply chains

• Simplified user access

• …

Multiple user types:• anonymous user: can only activate free

services access via the Global, Regional or

Thematic Portal

• registered user: can activate free or paying services

access via the Global, Regional or Thematic Portal

• service provider: can add new services access via the Global, Regional or

Thematic Portal access via workflow client tools

• helpdesk, administrator access Admin functionality on the

Global, Regional or Thematic Portal

Users

Service Providers

B2B

Co-operation

B2C


Service Consumers


SI – the architecture …SI – the architecture …

Users

Service Providers

B2B

Co-operation

B2C


Service Consumers Global

EO

Service

Infrastructure

Regional

or

Thematic

Service

Infrastructure


Legacy Service

SOAP

Toolbox

Catalogue Service

OGC WMS Server(s)

Web Browser

Workflow Editor

SOAP

Mobile Client

HTTP

OGC Gazetteer Server

HTTP

User

Legacy Service

Catalogue Service

ServiceProvider

AOI Server

Workflows, service data,order data ...

Portal

SOAPServiceProvider

ServiceInfrastructure

Regional or Thematic Service Infrastructure


NPP*

Service VITO

Format Conversion Service GIM

AOI selection(GIM)

Clipping Service GIM

NPP-Europe-GeoTIFF

NPP-Belgium-GeoTIFF1

NPP input + AOI description + required format

NPP-Europe-HDF

Supply Chain Orchestration … 1 Supply Chain Orchestration … 1

(*) NPP or Net Primary Productivity represents repeatable estimates of the net flux of carbon. Product is based on Végétation S10.


Combining service

• GIS additional way to combine services.

• Useful for vulnerability maps, hazard maps, rescue resources map, crisis map etc.

Supply Chain Orchestration… 2Supply Chain Orchestration… 2

Digital Terrain Model

Fire risk map

Community boundaries

Roads

Fire stations

Risk MapNDVI

LST

Chaining Overlaying


Internet

User FTP Server

Portal & Workflow

EngineMASS User

Order A1

(2)

Service Provider A2 (e.g. GIM)

Service Provider A1 (e.g. VITO)

(4)

(3)(6)

Order A2

(7)

(5)

Order A

Result A

Result A1 Result A2

(1)

(8)

(9)

Control flow Control flow : control flow: data flow: XML


User InteractionUser Interaction

Select service of interest

Identify time and AOI (method of selection is chosen by SP)

Select parameters (e.g. format)

Get RFQ (if applicable)

Issue request

Check status of orders/requests

Get the output (delivery options identified by SP)


Selection of time, AOI and parametersSelection of time, AOI and parameters


Check status of orders/requestsCheck status of orders/requests


Get results (service output)Get results (service output)


Oil spill detection (Spacetec)


Ship detection (Spacetec)


Simple Catalogue ServiceSimple Catalogue Service

Objective: Proof and test catalogue service within environment for the orchestration of EO services

Technology: Web Services, Workflow engine

Preliminary results:• Simple catalogue access and interoperability is made easy

• Technological homogeneity makes integration of (different) service elements simple and their orchestration straightforward

• Exploitation of state of the art OGC standards allows seamless integration into mapping services


Spot “search” results (GML)

Catalogue search results (textual)

Envisat “search” results (GML)


Back up plan

DEALERINVENTORY

SERVICE

SEARCH PROVIDER

1 … N

receiveinitiate( [in] searchCriteria collectionId)

invokeonResult( [in] resultList )

invoke

ASYNC WRAPPER

receive

invoke

reply

assign

receive

onResult( [in] completeList )

While CollectionId in request

While CollectionId in request

assignDynamic partnerLink


Service ProviderService Provider

Enters in partnership with other service providers

orchestrates the supply chain defining which elementary services are part of it…

authorises users to activate “his” supply chain

publishes the services from “his” infrastructure (over which he maintains the full control)

monitors “his” service execution, state, on the workflow, on his infrastructure

is responsible for• defining and publishing “his” own SLA

• monitoring and respecting “his” QoS; invoicing (where applicable)


Workflow editorto chain services

Workflow console to monitor service

instances


EOService

Infrastructure

ServiceProviderLegacy

Infrastructure

Client HTTP

ToolboxSOAP / HTTP

Legacy

Interface vs. SP legacy Interface vs. SP legacy InfrastructureInfrastructure


The Toolbox ConceptThe Toolbox Concept

Allows easy “publishing” of any service, with minimum assumptions on the SP available infrastructure (including NO infrastructure)

Provide a generic gateway toward service providers’ infrastructure

Generic/Customisable Service Interface: different Back End communication and interactions supported.

Customisation towards the SP infrastructure based on XML scripting


TOOLBOX Overview (1)TOOLBOX Overview (1)

Based on free and Open Source software• No additional licences are needed

Front-End supporting the communication with MASS.• Based on Java and the SUN JWSDP

TOMCAT JAXP (XML handling library)

• It implements all the MASS interfaces Order (synchronous and asynchronous) Request For Quotation (synchronous and asynchronous) Search (synchronous) Present (synchronous)

• Supports a generic service integration

Automatic WSDL creation


TOOLBOX Overview (2)TOOLBOX Overview (2)

Back-end supporting the most common “communication mechanism”• FTP file transfer

• HTTP

• File exchange

• Email notification

• JDBC

• Command and script invocation

• SOAP over HTTP

Products delivery supported by an internal built-in FTP server


Service Integration on the Service Provider Side -1Service Integration on the Service Provider Side -1

Service Configuration and test• Possibility to create and configure a service by means of a Web

Application Access control A Simple SOAP client is provided for test purposes. It includes a Catalogue test page. On-line Service Definition Script validation.


Service Integration on the Service Provider Side - 2Service Integration on the Service Provider Side - 2

Service Execution• Triggered by SOAP messages coming from EO Service Infrastructure

• Support for both synchronous and asynchronous communication mechanisms.

Service Monitoring and Control• Possibility to monitor the status of the service by means of a Web

Application Requests cancellation Requests monitor

• Service log Monitoring Log based on XML XSLT transformations.


Service Testing: SOAP clientService Testing: SOAP client


Service Test: Simple Catalogue Service Test: Simple Catalogue


Service Monitoring: LogService Monitoring: Log


2 - define interaction methods

1 - select service category 3 - describe service

4 - here the techs!

Service Definition Panel

5 – register!


Operational DemonstrationOperational Demonstration Operational Demonstration :

• Starts in October 2004

• >30 services

• >15 service providers

• Countries already involved: Belgium, France, Germany, Italy,(Norway), UK, Switzerland, The Netherlands, Portugal, … others

• Open to any user and additional service providers


Conclusions…Conclusions… Service Infrastructure facilitates:

• Integration of wide range of heterogeneous EO and GIS services including catalogues (open to geospatial local/global data/services)

• contacts between service providers and users more coherent (e.g. single service responsibility)

• Working relationships among service providers via the flow control (basic cooperation environment)

SI empowers service providers• enabling to combine and chain service elements within the supply chain

• Allows re-use generic or basic services

• Efforts potential service providers limited to specific skills and added value

• Allows to broaden and improve existing services

SI empowers users• offering services that are closer to the user’s expectations.


Conclusions…Conclusions… Future work

• Consolidate ongoing work on simple catalogue access and order management Interface Control Documents discussed with CNES, DLR, ASI, + others

• Get feedback on open source TOOLBOX

• Expand institutional cooperation at national and international level EC-DGINFSO, EUSC, JRC, …

• Involve federating agencies EEA, …

• Extend collaboration with standardization bodies OGC, ISO, …

• Consolidate working relationships with relevant programmes GMES, GSE, …

• ICD Documents & TOOLBOX available at http://earth.esa.int/rtd


Image Information Mining Coordination GroupImage Information Mining Coordination Group

ASI, CNES, CNR, DLR, EC-IST, ESA, ETHZ, EUSCASI, CNES, CNR, DLR, EC-IST, ESA, ETHZ, EUSChttp://earth.esa.int/rtd/IIMCG http://earth.esa.int/rtd/IIMCG

IIMCG terms of reference Charter

For Image Information Mining (IIM) applied to Earth Observation images, the IIMCG shall:

Promote European research and development of IIM techniques

Interface with European and National programmes in the field, like the O2 (Open and Operational) ESA initiative, the EC FP6 activities, etc.

Foster, within the members, the possibility to:

Share scientific and technical experience, data, information and applications

Identify relevant scientific and technical issues

Suggest technical directions

Suggest solutions for funding and coordination of activities

Provide a forum where interested bodies may:• Constantly be kept updated on ongoing activities in the field and on relevant scientific and technical issues

• Exchange information (including gathering of suggestions and requirements)

• Be informed on available demonstrators, prototypes, products, etc.

• Identify and suggest / promote appropriate standards

• Facilitate and promote the use of resulting products


Image Information Mining Coordination GroupImage Information Mining Coordination Group

ASI, CNES, CNR, DLR, EC-IST, ESA, ETHZ, EUSCASI, CNES, CNR, DLR, EC-IST, ESA, ETHZ, EUSChttp://earth.esa.int/rtd/IIMCG http://earth.esa.int/rtd/IIMCG

IIMCG terms of reference

Establish and maintain liaison with other interested bodies and with new

sensors’ research and design (in particular for the identification of short /

medium / long term needs)

Identify research and technological issues

Define possible strategies and solutions

Suggest activities and coordination methods

Suggest standards and interfaces for possible technologies

infrastructure for service support may 2004 service infrastructure within the esa gs pier giorgio...

Documents

service level agreement

offline service execution

data providers

end user end user slide

iim slide

gis supply chains support

iso tcs support requirements

cots rule data format