databases and the grid ogsa-dai architecture & status malcolm atkinson ogsa-dai chief architect...

Databases and the Grid

OGSA-DAIArchitecture & Status

Malcolm AtkinsonOGSA-DAI Chief Architect

forall members of the OGSA-DAI team

Director of National e-Science Centrewww.nesc.ac.uk

3rd September 2002UK e-Science All Hands Meeting

Sheffield Hallam University

Overview

Database Task Force & GGF DAIS-WGOGSA-DAI Project

Scope, Scale, Participants, PlansArchitectureStatusRelationship with OGSA

Data Access & Integration

Central to e-ScienceCollaboration

Shared Databases Curated Knowledge Accumulated Observations Accumulated Simulations

Computation Data mining Input to models Calibration of models

Presentation Publication of results Visualisation

UK DBTF

http://www.cs.man.ac.uk/grid-db

Malcolm Atkinson (NESC)Vijay Dialani (Southampton

Uni.)Norman Paton (Manchester

Uni.)Dave Pearson (Oracle UK)Tony Storey (IBM Hursley)Paul Watson (Newcastle

Uni.)

Membership

GGFDAIS-WG

OGSA-DAICore Programme

Project

GGF DAIS WGChairs

Norman Paton (Manchester Uni.)Leanne Guy (CERN)Dave Pearson (Oracle UK)

ActivityBoF GGF4 TorontoWG Meeting GGF5 EdinburghWorkshops & Mail lists

GoalsAgree Standards for Database Access & IntegrationFreely available reference implementations

OGSA-DAI one source & focus for discussions

CambridgeOxford

Glasgow

Cardiff

Southampton

London

Belfast

Daresbury Lab

RALHinxton

OGSA-DAI Partners

EPCC & NeSC

Newcastle

IBMUSA

IBM Hursley

Oracle

Manchester

EPCC & NeSCIBM UKIBM USAManchester e-SCNewcastle e-SCOracle $5 million, 18 months, started 1st February 2002

OGSA-DAI Scope

Definition and development of generic Grid data services which provide access to and integration of data held in databases, and the management of data within a distributed environment. Database

A stored, structured collection of dataAccessed using an API that takes account of the structure of the data stored

IncludesRelational and object databasesXML repositoriesAdequately described & managed collections of files

OGSA-DAI Plan

Two PhasesPhase 1: Started Feb 02 ends 30th SeptemberDetailed Plan –

Requirements, Designs & Prototypes 6 Work Packages Project Management (Oracle, EPCC) Architecture (NeSC, DBTF) XML Data Management (NeSC & EPCC) Distributed Query Systems (Manchester & Newcastle) Metadata & Registries (NeSC & EPCC) Relational Databases (IBM UK)

Phase 2: 12 months Structure and Objectives to be Refined in Major Review GGF5 DAIS WG meeting a major input

OGSA-DAI Time Line

Feb ’02 May ’02 Jul ’02 Sep ’02 Dec ’02 Feb ’03 May ’03 Sep ’03

Ship Alpha Release for GT3 Integration

RDB + GT2 / OGSA Prototypes Available

XML + OGSA Prototype Available

Design Documents & Demos for DAIS WG @ GGF5

XML + OGSA Prototypes for Early Adopters

WS + GSI UK support ( > 100 downloads)

Phase 2 StartsPhase 1 Starts

Presentation & Beta @ GGF7

GGF6 WG Papers & Prototypes

Productisation, RAMPS &Extension

DAI Key Components

GridDataService GDS Access to data & DB operations

GridDataServiceFactory GDSF Makes GDS

GridDataServiceRegistry GDSR Discovery of GDS(F) & Data

GridDataTransportVehicle GDTV Connects components + Moves Data

GridDataTransportDepot GDTD GDTV with persistence

OGSA Relationship

Class GridService Registry NotificationConsumer NotificationProducer

GDS Mandatory   Optional Normal

GDSF Mandatory   Optional Normal

GDSR Mandatory Mandatory   Normal

GDTS Mandatory      

GDTV        

GDTD Mandatory   Optional Normal

DAI portType Usage

Class GridDataService GridDataTransport Factory

GDS Mandatory Normal  

GDSF Optional Normal Mandatory

GDSR Optional    

GDTS Optional Mandatory  

GDTV      

GDTD Optional Mandatory  

OGSA-DAI Architecture

1requestforfactory

DSR

GDSF

client

OGSA-DAI Architecture

2responsewithGDSFsGSHs

1requestforfactory

DSR

GDSF

client

OGSA-DAI Architecture

2responsewithGDSFsGSHs

1requestforfactory

3script for 3GDSs

DSR

GDSF

client

4creation of 3GDSs

OGSA-DAI Architecture

2responsewithGDSFsGSHs

1requestforfactory

3script for 3GDSs

DSR

GDSF

GDS1

GDS2

GDS3

client

4creation of 3GDSs

OGSA-DAI Architecture

5responsewith 3GSHs

2responsewithGDSFsGSHs

1requestforfactory

3script for 3GDSs

DSR

GDSF

GDS1

GDS2

GDS3

client

4creation of 3GDSs

OGSA-DAI Architecture

6scriptsrequestingDB actions

5responsewith 3GSHs

2responsewithGDSFsGSHs

1requestforfactory

3script for 3GDSs

DSR

GDSF

GDS1

GDS2

GDS3

client

4creation of 3GDSs

OGSA-DAI Architecture

6scriptsrequestingDB actions

5responsewith 3GSHs

2responsewithGDSFsGSHs

1requestforfactory

3script for 3GDSs

DSR

GDSF

GDS1

GDS2

GDS3

client

7transfer databatch to GDS2

stream to GDS3

4creation of 3GDSs

OGSA-DAI Architecture

6scriptsrequestingDB actions

5responsewith 3GSHs

2responsewithGDSFsGSHs

1requestforfactory

3script for 3GDSs

DSR

GDSF

GDS1

GDS2

GDS3

client

7transfer databatch to GDS2

stream to GDS3

8stream datato GDS2

4creation of 3GDSs

OGSA-DAI Architecture

6scriptsrequestingDB actions

5responsewith 3GSHs

2responsewithGDSFsGSHs

1requestforfactory

3script for 3GDSs

DSR

GDSF

GDS1

GDS2

GDS3

client

9transfer databatchto client

7transfer databatch to GDS2

stream to GDS3

8stream datato GDS2

OGSA-DAI Architecture4creation of 3GDSs

6scriptsrequestingDB actions

5responsewith 3GSHs

2responsewithGDSFsGSHs

1requestforfactory

3script for 3GDSs

DSR

GDSF

GDS1

GDS2

GDS3

client

9transfer databatchto client

7transfer databatch to GDS2

stream to GDS3

8stream datato GDS2

10stream datato specifieddestination

Status

Teams & project coordination effectiveRelationship with Early Adopterswith Grid Support Centre with Globus

High rates of interaction

XML GDS & GDSF prototypes availableRDB demo availableDistributed Query demo availablePapers & designs presented at GGF5

www.nesc.ac.uk

Functional Scope & Architecture for Phase 2

Drafts & Intensive discussions

OGSA-DAI & OGSA <((-:}

Description, e.g. portType Works WellExpect to make extensive use of

Data Service Elements Special to DBs: Static & Dynamic

Component ManagementNotificationGrid-FTPAccountingSecurity:

Authentication, Authorisation & Privacy

Reliable invocation…

OGSA-DAI Summary

On Schedule & Going WellContributions via DAIS-WG @ GGF5, 6, 7, …Coordinating with GT3 ReleasesEnding Phase 1 (Design Exploration)

Testing Architectural DesignUsing OGSAWorking with Early Adopter Pilot Projects

AstroGrid & MyGrid and others Many requests for access to the software

Releasing prototypes

Influence OGSA-DAI directionVia DAIS-WG & as Prototype users