esa archives and vo activities - obspm.freuropean space astronomy centre (esac) villafranca del...

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 1

IVOA SPM, Moscow, 21/09/2006

ESA Archives and VO activities

http://esavo.esac.esa.int/

Christophe ArvisetESA-VO, Spain



Presentation Overview

ESA Archives at ESAC

ESA VO Project

Some ESA VO Tools

Conclusion



ESACEuropean Space Astronomy Centre• Located near Madrid,

Spain

ESAC default location for:• science operations,

• long history with astronomical missions, starting with IUE,

• now expanding with solar system missions.

• science archives

ESAC to be the main European VO node for space-based astronomy http://www.esa.int/SPECIALS/ESAC/



Astronomy Archives at ESACISO Data Archive,

• Available since Dec. 1998 : http://iso.esac.esa.int/ida/• Last main version, IDA 10.0 on 29/06/2006

XMM-Newton Science Archive• Available since April 2002: http://xmm.esac.esa.int/xsa/• Active development, XSA 3.5 on 28/08/2006

Integral Science Data Archive• Available internally since Dec. 2003, • Open publicly since July 2005 , http://integral.esac.esa.int/isda/• Active development, ISDA 2.5.2 on 27/06/2006

Herschel, Planck, GAIA, … to come in the future



Same Look & Feel for ESA Science Archives

Similar User Interface• Powerful and easy to

use• Panels aimed at

general users and expert users

Same functionalities across projects

Better corporate image for ESA



ESA Planetary Science Archive

All Planetary missions into a single archive (PSA)• Available since March 2004, http://www.rssd.esa.int/PSA• Active development, PSA 2.7 on 01/08/2006• Mars Express, Smart-1, Giotto, Huygens, Rosetta data• Venus Express, BepiColombo and future missions to be included in

the future

Also based on ESA astronomy archive infrastructure

SOHO Archive to be developed in 2006/2007



PSA User Interfaces

Classical User Interface Map Based Interface (MEX)



One Team, Several ProjectsA core Science Archives and VO Team (~14 people) in ESAC• Support many projects (horizontal support vs vertical organization)• Organized by functions: database, user interface, data distribution,

inter-operability

Advantages• Manpower shared between projects, small fraction for old projects• Knowledge transfer between projects• Re-use of existing systems for new projects• Easier and cheaper maintenance of existing systems• Ensure for long term preservation of data holdings

1 Archive Scientist per project to consolidate User requirements• Expert of the science field, Close to the community



One Team, Several ProjectsNot responsible for data pipeline development, nor for the data quality

Responsible for Archive “accessibility tools” and VO access

Main people involved (so far) in the VO• Christophe Arviset (team leader)• Pedro Osuna (ESA VO Technical

Coordinator)• Matteo Guainazzi (ESA VO Scientist)• Jesus Salgado, Inaki Ortiz, Aurelien

Stebe, Isa Barbarisi

Other people working mainly on archives



Existing Access – GUI

Proj

ect

Busine

ss L

ogic

TranslationLayer

Meta Data in Database

Data Productson hard disks

ProductRequestProducts

GUI Database/Product Request

GUI Database Result / Product

DB/Product Request

DB Result / Product

User friendly web access : Java Applet Powerful queries, organized by panel, hundreds of queriable and displayable parametersPublic data accessible to all, proprietary data accessible only to privilege usersImages preview (icons, full image)Quick download (1 click) at various processing levels, shopping basket retrievalRemote access to other archives (ADS, and tools (Aladin, VOSpec)

Project prior to the VO• Existing data model, data format, access

mechanisms



Existing Access : Scriptable I/F

Proj

ect

Busine

ss L

ogic

TranslationLayer






DB/Product Request

DB Result / Product

Script

able

Inte

rfac

e

Project Product Request

Project Productforeach obs in (observation_list)

AIOgetfile (obs, file_required)analyse_process file_requiredsave resultsdelete file_required

end

DB/Product Request

DB Result / Product

Access to metadata (via XML scripting)Access to data through socket (direct data stream) or FTP (several files or individual file)

Own Interoperability standards and needs as required by the projectNot using VO protocols, but ready for it…



VO compliant Architecture

Proj

ect

Busine

ss L

ogic

TranslationLayer






DB/Product Request

DB Result / Product


Project Product

DB/Product Request

DB Result / Product

SIAP/SSA Request

VOTable / Product

TranslationLayer

Script

able

Inte

rfac

e


Project Product

(Scriptable Interface)foreach obs in (observation_list)

AIOgetfile (obs, file_required)analyse_process file_requiredsave resultsdelete file_required

end

DB/Product Request

DB Result / Product

Existing scriptable I/F already carried all the functionalities required by the new VO standards• Access to metadata (database)• Access to products (data repository)

By using “Translation Layer”, easy to convert the VO standards interfaces into our existing I/F• Little effort required• Translation Layer easy to code using XML

As VO standards evolve, our scriptable I/F system remains, but we just have to adapt our translation layers





ESA VO Project

Some ESA VO Tools

Conclusion



ESA VO Main ObjectivesESAC to be the main European VO node for space-based astronomy • Link to all the astronomy projects at ESAC (ISO, XMM-Newton,

Integral, Herschel, Planck, Gaia, …)

All ESAC Archives to be VO-compliant

Development of VO-compliant tools:• VOSpec, VOQuest• Registry, DMMapper, DALtoolkit, …

Active member of the EURO-VO and IVOA

Science research through the VO



ESA is an active member of EURO-VO

Member of EURO-VO Data Centre Alliance• Development of the ESAC Astronomical Archives• Make these archives accessible through VO standards• VO Support to other European Data Centre• Data Processing Centre, through the ESA-VO GRID

• Processing the data where they reside, instead of moving the data around

Member of EURO-VO Technology Centre• Technology research and prototyping on VO standards• Link with VOTECH project• Link with IVOA corresponding working groups



EURO-VO Facility CentreFunded by ESA and ESO

WebPortal, user friendly for Scientists• VO tools, User recipes and User manuals, …

WebPortal, user friendly for Technical people• VO development tools, tutorials, links to VO

standards, …

Support for VO technology take-up in Europe• Scientific and technical VO Workshops

Registry of VO resourceshttp://www.euro-vo.org/



VO Recipes

http://www.euro-vo.org/pub/fc/recipes.html





ESA VO Project

Some ESA VO Tools

Conclusion



ESA VO RegistryFull Searchable Registry • implemented in Java with Sybase RDBMS• Being upgraded to IVOA RI 1.0 specs

Fully compliant OAI interface for harvesting

Services accessible by Web Service, HTTP POST or GET

User friendly Interface with Web Pages for Search and Insert/Update

http://esavo.esac.esa.int/registry



VOSpecA tool to retrieve, display, manipulate spectra and lines coming from various VO resources

Main functionalities:• automatic conversion of units via dimensional

equation analysis• Fitting, de-redening capabilities• interface with theoretical models• interface with atomic and molecular databases

and on-the-fly identification on SEDs• Some more analysis functions

http://esavo.esac.esa.int/vospec



VOSpec : Real and Theoretical Spectra

Select Source

Find/Select in the VORegistry

VO Spectral Servers

IUEspectra

ISOspectra

VEGATheoretical

Model

DisplayTheoretical Spectra

and Observational Spectra

IUEspectra

ISOspectra

VEGATheoretical

Model

NormalizeTheoretical

Spectra

IUEspectra

ISOspectra

VEGATheoretical

Model

Theoretical SpectraFits

Observational Spectra



VOSpec and Line AccessVOSpec implements the SLAP (Simple Line Access Protocol) VO standard, which allows to publish atomic and molecular line databases in the VO

This means that – for these databases – VOSpec allows you to:• List the atomic and molecular transitions which may occur in a given

wavelength band• Superpose the closest atomic or molecular transition to any line

features in an observed spectrum displayed with VOSpec

Demo



VOQuest (=== Still Prototype ===)

VOQuest allows you to search for catalogues, which support certain type of queries, such as:• Coordinates, Fluxes, Photometry• Still a prototype, interface to be improved

The same ADQL query is sent to several VO-compliant catalogue servers and query results come back in the same format• Using Source Catalogue Data Model

Further actions are then feasible using other VO tools• Plotting, column manipulation, X-matching with

TOPCAT(Astrogrid)

See talk at IVOA @ Victoria• http://www.ivoa.net/internal/IVOA/InterOpMay2006Applications/VOQuest.pdf



DAL ToolkitPurpose : help astronomers to easily publish their data through VO protocols

DALingestor• Takes FITS spectra as inputs• Create necessary metadata for SSA• In the future, will also support images

(SIAP) and lines (SLAP)

DALserver• Publish data through SSAP, SIAP,

SLAP



ConclusionsESAC is a major Data Centre and its goal is to become the VO node in Europe for space astronomical data• ISO, XMM-Newton, Integral, Herschel, Planck, Gaia, …

ESA is putting a significant effort into the VO• At a European (EURO-VO) and worldwide level (IVOA)

VO tools developed at ESA• ESA VO Registry – a registry of VO resources• VOSpec – a tool to consume SSAP, TSAP, SLAP aware services • VOQuest – a tool to consume Source Catalog DM aware services• DALToolkit – a toolkit to publish basic resources through SIAP, SSAP,

SLAP



Acknowledgements

ESAC Science Archives and VO Team

esa archives and vo activities - obspm.freuropean space astronomy centre (esac) villafranca del...

Documents