1 p.quinn tivo june10-14 2002 drivers, status and planning

P.Quinn TIVO June10-14 20021

Drivers, Status and Planning


Overview• The need for a new research

infrastructure in astronomy• AVO

– Work Program– Status– Future

• The International VO Alliance


The winds of change

“Internet computing and Grid technologies promise to change the way we tackle complex problems. They will enable large-scale aggregation and sharing of computational, data and other resources across institutional boundaries. And harnessing these new technologies effectively will transform scientific disciplines ranging from high-energy physics to the life sciences.” Dr. Ian Foster, Co-leader GLOBUS Project


Particle Physics Problems

The LHC Detectors

CMSATLAS

LHCb

~8-10 PetaBytes /year/on tape~1 PetaByte/year/on disk

~8000 kSI95 (~300,000 PC2000)


What has this to do with astronomy?

• Astronomy has become a BIG international science– Gemini, VLT, ALMA, SKA, NGST, ELTs

• Astronomy projects involve– International coordinated research efforts – Distributed multi-wavelength teams, resources and

data – Data volumes with doubling times < 12 months

• Astronomy service organizations need to – provide their communities with access to software

tools, high quality raw and processed data in the face of desktop computing power and network bandwidths with doubling times > 18 months


Science Data VolumeArchive Volume (GB)

1

10

100

1000

10000

1992199319941995199619971998199920002001

Archive Volume (GB)

1

10

100

1000

10000

1992199319941995199619971998199920002001

Data Requested (GB)

1

10

100

1000

10000

1996 1997 1998 1999 2000 2001

Data Requested (GB)

1

10

100

1000

10000

1996 1997 1998 1999 2000 2001

T2 < 12 months

ESO/STECF Science Archive Facility


Projected GrowthT2 < 12 months


Astronomical Strategies

PROBLEM SOLUTIONSlow CPU growth Distributed ComputingLimited storage Distributed DataLimited bandwidth Information

Hierarchies - Move only what you

need

Data diversity InteroperabilityVO


Distributed Computing at Work

• Virtual and collaborative exploration of the Universe

Floating Point Operations

Total CPU time

Results received

4.260259e+1849.31 TFLOPs/sec

1.502416e+21

1662.448 years954229.737 years

1092374491854017

50753675440Users

Last 24 HoursTotal


An Integrated European Approach

Information Society TechnologiesGRID startups (today~40M Euro)

[3.6B Euro 2003-2007]

ScienceIndustry / business

ApplicationsApplications

MiddlewareMiddleware& Tools& Tools

Underlying Underlying InfrastructuresInfrastructures

CROSSGRID

DATATAG

GRIDLAB

EGSOGRIA

GRIP EUROGRID

DAMIEN GR

IDS

TA

RT

AVO

DATAGRID


The AVO Proposal• EU RTD Proposal submitted 15 February 2001 following

OPTICON recommendations for a European VO effort (similar effort in the US [NVO] proposed by Decadal Report)

• ESO PI (Quinn) + STECF(Benvenuti), CDS (Genova), TERAPIX (Mellier), ASTROGRID (Lawrence) and Jodrell Bank (Diamond) + NVO affiliates

• Asked for funds for a three year Phase-A study of an Astrophysical Virtual Observatory – 7.2 Million € (50% from EU and 50% from organizations)

• Focus on – Multi-wavelength science case demonstrations (ASTROVIRTEL

basis)– Interoperability demonstrations (CDS/All)– Technology assessments and testbeds (ASTROGRID/ESO)


AVO STATUS•AVO approved with EU funds ~2 Million € (total budget ~ 4M €)• Contract start on 15 November 2001 - 3 Year Phase A study• 9 NEW POSITIONS for 3 years over 6 institutions - total 18 FTE (~ 50 people)•Total VO funding AVO+NVO+ASTROGRID = $21 million (US)•3 Year target :

•Build VO 1.0 among the 6 partner archive sets by•Defining and executing trial science cases•Defining, developing and deploying new interoperability standards and tools•Developing and deploying new Grid-based services


AVO Work Program

WP0.1AVO Integration

WP Manager M.DolenskyAVO Systems Engineer

WP0.2Phase-B Plan

WP Manager : M.Dolensky

WP1.1AVO SRM

WP Manager : AVO Scientist TBD

WP1.2Science Requirements


WP1.3Implement Science Cases


Work Area 1Science

WA Manager P.BenvenutiSTECF

WP2.1Archive Inclusion

WP Manager : F.Ochsenbein

WP2.2Test and EvaluationWP Manager : M.Allen

WP2.3New Functionality

WP Manager : X.Derriere

WP2.4Tool Evaluation

WP Manager : F.Genova

Work Area 2Interoperability

WA Manager F.GenovaCDS

WP3.1Grid Technologies

WP Manager : G.Rixon

WP3.2Storeage and Compute Tech .

WP Manager : A. Wicenec

WP3.3Database Technology

WP Manager : C.Page

Work Area 3Technology

WA Manager A.LawrenceASTROGRID

Work Area OAVO Programme Management

P.QuinnDeputy : Benoit Pirenne


Lofti BEN JAFFEL Institute d’Astrophysique, Paris

Andrea CIMATTI Arcetri Observatory, Florence

Luiz DA COSTA ESO GarchingRichard DE GRIJS Institute of Astronomy, CambridgeDaniel EGRET Observatoire de Strasbourg

Giuseppina FABBIANO Center for Astrophysics, Cambridge, MAGerry GILMORE Institute of Astronomy, CambridgeAna I. GOMEZ DE CASTRO Universidad Complutense de MadridKrzysztof GORSKI ESO GarchingPreben GROSBØL ESO Garching

Bob HANISCH ST ScI, BaltimoreUlrich HOPP Universitäts-Sternwarte MunichFlorian KERBER ST-ECFMary KONTIZAS University of Athens

AVO Science WG

Patrick LEAHY Jodrell Bank, ManchesterBruno LEIBUNDGUT ESO, GarchingTommaso MACCACARO Brera Observatory, MilanJanet MATTEI American Assoc. of Va riable Star Observers

Yannick MELLIER Institute d’Astrophysique, ParisTom OOSTERLOO NFRA (ASTRON), DwingelooPatricio ORTIZ Osservatorio di CapodimonteFabio PASIAN Osservatorio Astronomico, TriesteEmanuela POMPEI ESO Paranal

Philippe PRUGNIEL Observatoire de LyonTimo PRUSTI ESA / ESTECWolfgang VOGES MPE GarchingNicholas WALTON Isaac Newton Group, La PalmaHans ZINNECKER Astronomical Institute, Potsdam

See Piero’s talk on ASTROVIRTEL development

Selection of preliminary science cases in June’02


Interoperability

• Major step forward : VOTable 1.0 • See F.Ochsenbein talk


Technology

• Scalable storage and computing (see talk by Andreas Wicenec on NGAST)

• Web services prototypes (ASTROGRID talk by Andy Lawrence)

• Tool prototypes : QUERATOR (talk by F. Pierfederici)


VO ChallengesAVO: the promise

Bob Fosbury JENAM 2001(1) To take the data collected by teams for specific projects

and manage/massage it in such a way as to be of general use

Comments - • If the data in an archive are neither well-calibrated nor

well-described, they are of little value for the AVO• If the data are in good order, the AVO is unnecessary — it

is just a layer of middle-men• If I were a funding agency, why should I support the

‘middle-man’ when I could fund the original science project?


A VO RoadmapQuinn ADASS’01

• To address Bob’s comments the international VO effort needs to define a unified and astronomically accepted roadmap for the next three years

• Without a publicly visible direction, accountability and receptiveness to requirements we will fail -

• This roadmap must be in defined/produced by the next international VO meeting


The International Virtual Observatory Alliance


IVOA• IS NOT ANOTHER PROJECT • IS an alliance of existing and future

national and international projects to– Define the common ground needed to make

an operational and scientifically effective IVO

– Reach agreement on standards and interoperability

– Allow the international scientific communities resident in the national projects, and elsewhere, a global channel for comment, criticism and guidance


An Agreed Roadmap - 1

January 2002 Initiate international dialog on interoperability.

OPTICON Interoperability Working Group meeting, Strasbourg.

+ Discussion/revision draft VOTable std.

April 15, 2002Reach agreement on VOTable 1.0.

June 10-14, 2002 Formation of IVOA


Early Progress

• VOTable: A Proposed XML Format for Astronomical Tables

• Roy Williams, California Institute of Technology, USA NVO• François Ochsenbein, Observatoire Astronomique de Strasbourg, France AVO • Clive Davenhall, University of Edinburgh, UK AVO• Daniel Durand, Canadian Astronomy Data Centre, Canada CADC• Pierre Fernique, Observatoire Astronomique de Strasbourg, France AVO• David Giaretta, Rutherford Appleton Laboratory, UK STARLINK• Robert Hanisch, Space Telescope Science Institute, USA NVO• Tom McGlynn, NASA Goddard Space Flight Center, USA NVO• Alex Szalay, Johns Hopkins University, USA NVO• Andreas Wicenec, European Southern Observatory, Germany AVO

• Version 1.0 (15 Apr 2002)

• Document repository: http://cdsweb.u-strasbg.fr/doc/VOTable/• Comments: [email protected]


An Agreed Roadmap - 2January 2003 Coordinated initial science

demonstrations by IVOA members

January, 2003 IVOA agreement on initial suite of interoperability Standards and tools

May 2003 Working Published Web Services

August 2003 Coordinated intermediate science demonstrations including

international access at IAU General Assembly


An Agreed Roadmap - 3October 2003 Astronomical Query LanguageJanuary 2004 Coordinated intermediate

demonstrations + GridMay 2004 Resource Discovery 1.0July 2004 IVO 2005+ roadmap October 2004 Compound Web Services

and Ontology Service 1.0

January 2005 Coordinated complex science demonstrations


An Invitation

The ASTROGRID, AVO and NVO projects take the opportunity of the Munich VO meeting to formally announce the IVOA and would like to extend an invitation to all VO projects to join this alliance for the pursuit of an international virtual observatory and the expansion of astronomical research capabilities in the 21st century.


ESO Archive Policyhttp://archive.eso.org/Archive_Access_Policy.htmlESO is fully aware of the importance of international collaborations in achieving the

scientific goals and break-throughs necessary for astronomy to advance in the 21st century. The scientific potential of facilities like ALMA and OWL will only be realized by coordinated international efforts and the free exchange of astronomical ideas and data. To this end, ESO is preparing to provide open international access to all its archival data. Before this can happen, the ESO data holdings must be of a uniform high quality, in a form appropriate for recalibration and archival research and be supported by adequate operational manpower. ESO is taking the lead in Europe in defining data quality and interoperability standards through its coordination and funding efforts for an Astrophysical Virtual Observatory. The aim of the AVO is to provide the European and global astronomical community with an archival and data handling resource that can meet the needs of 21st century astronomy. The AVO program aims to join data from ESO telescopes with space and ground archives from Europe in an AVO Phase A facility by the end of 2003. From this time, ESO data will start to become freely available to the international astronomical community in a phased manner, following standards for data centers in the global virtual observatory and the availability of new VLT instruments.

1 p.quinn tivo june10-14 2002 drivers, status and planning

Documents