GT Components

Globus Toolkit

• A “toolkit” of services and packages for creating the basic grid computing infrastructure

• Higher level tools added to this infrastructure• Version 4 is web-services based• Some non-web services code exists from

earlier versions (legacy) or where not appropriate (for efficiency, etc.).

• Each part comprises a set of web services and/or non-web service components.

• Some built upon earlier versions of Globus.

Data Management

SecurityCommonRuntime

Execution Management

Information Services

Web Services

Components

Non-WS

Components

Pre-WSAuthenticationAuthorization

GridFTP

GridResource

Allocation Mgmt(Pre-WS GRAM)

Monitoring& Discovery

System(MDS2)

C CommonLibraries

GT2

WSAuthenticationAuthorization

ReliableFile

Transfer

OGSA-DAI[Tech Preview]

GridResource

Allocation Mgmt(WS GRAM)

Monitoring& Discovery

System(MDS4)

Java WS Core

CommunityAuthorization

ServiceGT3

ReplicaLocationService

XIO

GT3

CredentialManagement

GT4

Python WS Core[contribution]

C WS Core

CommunitySchedulerFramework

[contribution]

DelegationService

GT4

Globus Open Source Grid Software

Java Services in Apache AxisPlus GT Libraries and Handlers

YourJava

Service

YourPythonService

YourJava

Service RF

T

GR

AM

Del

egat

ion

Inde

x

Trig

ger

Arc

hive

r

pyGlobusWS Core

YourC

Service

C WS Core

RLS

Pre

-WS

MD

S

CA

S

Pre

-WS

GR

AM

Sim

pleC

A

MyP

roxy

OG

SA

-DA

I

GT

CP

Grid

FT

P

C Services using GT Libraries and Handlers

SERVER

CLIENT

InteroperableWS-I-compliant

SOAP messaging

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

X.509 credentials =common authentication

Python hosting, GT Libraries

Another view of GT4 Components

CustomWeb

ServicesWS-Addressing, WSRF,

WS-Notification

CustomWSRF Web

Services

GT4WSRF Web

Services

WSDL, SOAP, WS-Security

User Applications

Reg

istr

yA

dmin

istr

atio

n

GT

4 C

onta

iner

GT4 Web Services Core

Execution Management

Key component

GRAM (Grid Resource Allocation Manager)

• For submitting executable jobs

• May interface to a local job scheduler

GRAMservices

GT4 Java Container

GRAMservices

Delegation

RFT FileTransfer

Transferrequest

GridFTPRemote storage element(s)

Localscheduler

Userjob

Compute element

GridFTP

sudo

GRAMadapter

FTPcontrol

Local jobcontrol

Delegate

FTP data

Client

Job

functions

Delegate

Service host(s) and compute element(s)

GT4 GRAM Structure:

Data management components

I Foster

Security ComponentsAddresses the security requirements of grid computing. Three important factors are:

• Authorization– Process of deciding whether a particular identity can

access a particular resource

• Authentication– Process of deciding whether a particular identity is

who he says he is (applies to humans and systems)

• Delegation (somewhat specific to grid computing)– Process of giving authority to another identity

(usually a computer/process) to act on your behalf.

Security continued

• Security aspects complicated by the fact that virtual organization members and resources can be in different administrative domains.

Data Management

SecurityCommonRuntime

Execution Management

Information Services

Web Services

Components

Non-WS

Components

Pre-WSAuthenticationAuthorization

GridFTP

GridResource

Allocation Mgmt(Pre-WS GRAM)

Monitoring& Discovery

System(MDS2)

C CommonLibraries

GT2

WSAuthenticationAuthorization

ReliableFile

Transfer

OGSA-DAI[Tech Preview]

GridResource

Allocation Mgmt(WS GRAM)

Monitoring& Discovery

System(MDS4)

Java WS Core

CommunityAuthorization

ServiceGT3

ReplicaLocationService

XIO

GT3

CredentialManagement

GT4

Python WS Core[contribution]

C WS Core

CommunitySchedulerFramework

[contribution]

DelegationService

GT4

Security

GT4’s Use of Security Standards

I Foster

GT4 Data Management

• Move large data to/from nodes• Replicate data for performance &

reliability• Locate data of interest• Provide access to different data sources

– File systems, parallel file systems, hierarchical storage (GridFTP)

– Databases (OGSA DAI)

Data Management

SecurityCommonRuntime

Execution Management

Information Services

Web Services

Components

Non-WS

Components

Pre-WSAuthenticationAuthorization

GridFTP

GridResource

Allocation Mgmt(Pre-WS GRAM)

Monitoring& Discovery

System(MDS2)

C CommonLibraries

GT2

WSAuthenticationAuthorization

ReliableFile

Transfer

OGSA-DAI[Tech Preview]

GridResource

Allocation Mgmt(WS GRAM)

Monitoring& Discovery

System(MDS4)

Java WS Core

CommunityAuthorization

ServiceGT3

ReplicaLocationService

XIO

GT3

CredentialManagement

GT4

Python WS Core[contribution]

C WS Core

CommunitySchedulerFramework

[contribution]

DelegationService

GT4

GridFTP and Reliable File Transfer

GridFTP• Built on FTP using separation of data and

control channels• Provides features for

– Large data transfers– Secure transfers– Fast transfers– Reliable transfers– Third party transfers

• Not a web service– RTF (Reliable File Transfer) service provided WS-

level interface

Third party transfers

PI = FTP Protocol InterpreterDTP= FTP Data Channel Process

PI

DTP DTP

PI

PI PI

Client

Server Server

Control channels

Data channel

Performing a third-party transfer

1. Client establishes control channel with server2. Using control channel, client sets up transfer

parameters and requests data channel creation

3. Data channel established,4. Client sends transfer command over control

channel,5. Data transfer starts through data channel.

Either client or server can send.

Parallel transfers and striping

• Using multiple (virtual) connections for transfer– Same external network– Speed improvement possible, but limited by

network card

• Striping– a version of parallel transfers that can use

separate hardware interfaces– Implemented in GT 4.

GridFTP and RFT

WS ClientRFT service

(Java)

Client API(Java)

XIO based (C) XIO based (C)

Control channel

Data channel

Control channel

GridFTP server GridFTP server

From Gridwise

GT 4 Replica Location Service

• Identify location of files via logical to physical name map

• Distributed indexing of names, fault tolerant update protocols

IndexIndex

I Foster

Data Management

SecurityCommonRuntime

Execution Management

Information Services

Web Services

Components

Non-WS

Components

Pre-WSAuthenticationAuthorization

GridFTP

GridResource

Allocation Mgmt(Pre-WS GRAM)

Monitoring& Discovery

System(MDS2)

C CommonLibraries

GT2

WSAuthenticationAuthorization

ReliableFile

Transfer

OGSA-DAI[Tech Preview]

GridResource

Allocation Mgmt(WS GRAM)

Monitoring& Discovery

System(MDS4)

Java WS Core

CommunityAuthorization

ServiceGT3

ReplicaLocationService

XIO

GT3

CredentialManagement

GT4

Python WS Core[contribution]

C WS Core

CommunitySchedulerFramework

[contribution]

DelegationService

GT4

Monitoring and Discovery

Monitoring and Discovery• WSRF provides common mechanisms for

monitoring and discovering a service:• GT4 “aggregator” services within MDS:

– MDS-Index: collects state information from registered resources and makes it available as XML document

– MDS-Trigger: passes this information to an executable

– MDS-Archive: archives state information (awaiting implementation)