A PPARC funded project

Tony LindeProgramme Manager

eScience meets eFrameworks

28th April 2006NeSC,Edinburgh

28-Apr-2006 eScience meets eFrameworks 2Tony Linde

AstroGrid overviewVirtual Observatory infrastructure“The VO vision can be summed up as the desire to make all archives speak the same language − all searchable and analysable by the same tools, all data sources accessible through a common interface, all data held in distributed databases that appear as one.”Andy Lawrence, 09/2003

AstroGrid approach Build infrastructure first VObs Web Mix-and-match, plug-and-play, …

28-Apr-2006 eScience meets eFrameworks 3Tony Linde

AstroGrid projectDuration: Sept 2001 – Dec 2007Funding: £7.7M (PPARC)Personnel: ~26 (23.4 FTE)Goal: Develop VObs Infrastructure

Deploy UK VObsScope: Astrophysics, Solar, STP, …

Optical, X-Ray, Radio, …AG1 Phase A: Sept 2001 – Dec 2002

Analysis, R&D, ArchitectureAG1 Phase B: Jan 2003 – Dec 2004

Build, test & deliver V1.0AstroGrid-2: Jan 2005 – Dec 2007

Develop & deploy

28-Apr-2006 eScience meets eFrameworks 4Tony Linde

AstroGrid consortiumEdinburghLeicesterCambridgeMSSLJBORALQUB

From 2005, AstroGrid-2 + Bristol Exeter Leeds Portsmouth

28-Apr-2006 eScience meets eFrameworks 5Tony Linde

Wider involvementEurope Euro-VO: http://euro-vo.org VOTech project: http://www.eurovotech.org

FP6 funded: €6M: 2005-7 Follows on from AVO (FP5 funded)

International IVOA: http://ivoa.net Closer collaborations

RVO JVO SAAO NVO

28-Apr-2006 eScience meets eFrameworks 6Tony Linde

Project ApproachAgile: iterative & incremental development

Open project: Static http://www.astrogrid.org Wiki http://wiki.astrogrid.org Forum http://forum.astrogrid.org Jabber via jabber.astrogrid.org

Open Source code Academic Free License

http://www.opensource.org/licenses/academic.php AstroGrid releases

http://software.astrogrid.org

28-Apr-2006 eScience meets eFrameworks 7Tony Linde

AstroGrid infrastructure

28-Apr-2006 eScience meets eFrameworks 8Tony Linde

Deployments 1

28-Apr-2006 eScience meets eFrameworks 9Tony Linde

Deployments 2

28-Apr-2006 eScience meets eFrameworks 10Tony Linde

Deployments 3

28-Apr-2006 eScience meets eFrameworks 11Tony Linde

How do the VO components that implement these

interfaces relate to each other?

28-Apr-2006 eScience meets eFrameworks 12Tony Linde

DSA: Architecture

SkyNode

SIAP/

SSAP

Web Svc

(CEA)

ADQL

Translator

Plugins:

JDBC/Db

FITS etc

VOTable

DSA

28-Apr-2006 eScience meets eFrameworks 13Tony Linde

Registry

We can access data but how do we know where it can be found?

Knowledge of the location and nature of resources is critical for all this to workThe AstroGrid Infrastructure supplies a Registry (IVOA compliant of course!) which contains this informationRegistry is searchable by humans and by machines.

28-Apr-2006 eScience meets eFrameworks 14Tony Linde

Common Execution Architecture

Why would we want CEA/UWS?

Same reason we need universal access to dataAstroGrid supports the concept of workflowEssentially, remote execution of special scripts or jobsAllows data access and processing to be undertaken at source or where the compute power liesThe Grid concept!The AG Infrastructure helps make the details transparent, keeping focus on the science.

28-Apr-2006 eScience meets eFrameworks 15Tony Linde

WorkflowWorkflow

28-Apr-2006 eScience meets eFrameworks 16Tony Linde

Compute grid

JES

Registry

CommonExecutionConnector

Application as.exe file (local)

CommonExecutionConnector

CommonExecutionConnector

CommonExecutionConnector

Application as.exe file (grid)

Application as.HTTP service

Application asJava class

28-Apr-2006 eScience meets eFrameworks 17Tony Linde

MySpaceIf all this happens remotely, where do the results end up?

Distributed data access and processing requires distributed data storageMySpace is the AstroGrid solutionVOSpace/VOStore under review at IVOADistributed, virtual data storage that appears to the user as a homogeneous collection of filesDatabase tables coming soon!

28-Apr-2006 eScience meets eFrameworks 18Tony Linde

Distributed Storage

DSA Process Process Desktop

MySpace

28-Apr-2006 eScience meets eFrameworks 19Tony Linde

MySpace

28-Apr-2006 eScience meets eFrameworks 20Tony Linde

AstroGrid ActorsAstronomers Incl PIs, management structure, …

Instrument operators, …Data curatorsPublishersAdministratorsSystems managersDevelopers

Standards definers …

28-Apr-2006 eScience meets eFrameworks 21Tony Linde

AstroGrid and eFrameworks:Issues

Security Shibboleth authentication:

Single-signon Authorisation

Common policy requirements?

Service discovery Resource registries

Need for commonality? Will others need to get at astro resources?

Common identifiers?

28-Apr-2006 eScience meets eFrameworks 22Tony Linde

AstroGrid and eFrameworks:Other Issues

Service executionWorkflowShared storageAccountingNaming! e.g. VO

Bottom line Standards & interoperability How do we get W3C, GGF, IVOA, IE & project

standards to work?