ogsaevolution.ppt

Grid Computing

Ecole DRUIDE 2004 © 2004 IBM Corporation

Open Grid Services Architecture Evolution

Jean-Pierre ProstEMEA Design Center for ebusiness on demandPSSC, IBM Montpellier, France

Grid Computing

© 2004 IBM CorporationEcole DRUIDE 2004

Outline

On Demand and Grid

Service Oriented Architecture

Open Grid Services Infrastructure (OGSI)

OGSA Architected Services

OGSA Evolution: Grid vs. Web Services

Web Services Resource Framework (WSRF)

Globus Toolkit

Grid Computing


An enterprise whose business processes - integrated end-to-end across the company and with key partners, suppliers and customers - can respond with speed to any customer demand, market opportunity or external threat.

business on demand

On Demand Enterprise

Grid Computing


Flexible Financial & Delivery Options

Becoming an On Demand Enterprise

Innovativebusiness designs that sharpen focusand accelerate growth

Integrated, end-to-end business processes that are built to change

An IT operating environment

optimized for flexibility and resilience

BusinessTransformation

On DemandOperating

Environment

Business P

rocesses BusinessTransformation

Grid Computing


On Demand Operating Environment

On Demand Operating Environment

Business Flexibility

IT Simplification

Open standards are table stakes for an on demand operating environment

Horizontal Process

Business flexibility through integration of people, processes and information within

and beyond the enterprise

Integration

Infrastructure Management

IT simplification through automation and virtualization, enables access to and creates a consolidated, logical

view of resources across a network

Grid Computing


Grid Computing Defined

StorageStorage

ApplicationsApplications

ProcessingProcessing

Operating SystemOperating System

DataData

I/OI/O

Distributed computing based on open standards enabling

heterogeneous operations

Availability and Cost of Bandwidth

Availability and Cost of Bandwidth

Industry Acceptance of

Open Standards

OGSA

LINUX

WSDL

TCP/IP

Industry Acceptance of

Open Standards

OGSA

LINUX

WSDL

TCP/IP

Increased Software Sophistication

Workload Mgmt.

Provisioning

Scheduling

Billing and Metering

Increased Software Sophistication

Workload Mgmt.

Provisioning

Scheduling

Billing and Metering

Key Enablers

Grid Computing


Outline

On Demand and Grid






Globus Toolkit

Grid Computing



Interface

Bindings

Service implementation

Adaptation layer

Native platform function

Local/Remote Transparency

WSDL

Separate interface from access and implementation

Container

Grid Computing


Open Grid Services Architecture Objectives

Manage resources across distributed heterogeneous platforms

Deliver seamless QoS

Provide a common base for autonomic management solutions

Define open, published interfaces

Exploit industry-standard integration technologies

– Web Services, SOAP, XML,...

Integrate with existing IT resources

Grid Computing


Web Services

Business integration

Secure and universal access

Applications on demand

Grid Protocols

Vast resourcescalability

Global Accessibility

Resourceson demand

access manageshare

The Best of Two Worlds

Open Grid Services Architecture

Grid Computing


Pro

fessio

nal S

erv

ices

Network

OGSA Enabled

Storage

OGSA Enabled

Servers

OGSA Enabled

Messaging

OGSA Enabled

Directory

OGSA EnabledFile

Systems

OGSA Enabled

Database

OGSA EnabledWorkflo

w

OGSA Enabled

Security

OGSA Enabled

Open Grid Services Architecture Framework

Web Services

OGSI – Open Grid Services Infrastructure


Applications

Au

ton

om

ic C

ap

ab

ilit

ies

Grid Computing


Outline

On Demand and Grid






Globus Toolkit

Grid Computing



HandleMapNotificationFactoryService GroupsState Management.Lifecycle


Exploits existing web services properties

– Interface abstraction (WSDL)

– Protocol, language, hosting platform independence

Enhancement to web services

– Dynamic Creation

– State Management

– Event Notification

– Referenceable Handles

– Lifecycle Management

– Service Groups

Grid Computing


Web Services


HandleMapNotifyFactoryService GroupsState Mgt.LifecycleDiscovery



Hosting platform

(Java)

Other Possible Hosting

Platforms(environments)

Microsoft (.NET)Globus (C/C++)

Berkley Lab (Python)

the globus project tm

www.globus.org

• Open source• Reference

implementation

Grid Computing


Outline

On Demand and Grid






Globus Toolkit

Grid Computing


Security

•Registries and Discovery Services (SG)

• Attribute Propagation and Query• Service Domain

•Service Orchestration •Metering & Accounting

• Installation & Deployment

• Messaging and Queuing Services

• Event Services• Distributed

Secure Logging Service


Policy ManagementService CommunicationService Management

Grid Data Services Grid Core ServicesGrid Program Execution Services

Domain Specific Services

• Authentication• Authorization &

Access Control• Credential

Validation & Transformation

• Trust Broker

•Policy Service Manager•Policy Agent•Policy Transformation

Service•Policy Resolution Service•Policy Validation Service•Policy Administration

Services and Negotiation Framework

Grid Computing



Grid Data Services Grid Program Execution Services

Domain Specific Services

• Job Scheduler & Queuing Services

• Resource Reservation Services

• Workload Managers and Micro-Scheduling Services

• Data Access Services• Data Transformation &

Federation Services• Data Replication Service• Data Caching Service• MetaData Catalog

Services

Grid Core Services

Grid Computing


The Path to Standardization

• IBM is working on OGSA with Globus at GGF• IBM is also working within other standards bodies to define new

or extend existing standards to support required distributed services

• IBM will accelerate delivery through partners

the globus project tm

www.globus.org

Grid Computing


Outline

On Demand and Grid






Globus Toolkit

Grid Computing


Grid and Web Services: Convergence?

However, despite enthusiasm for OGSI, adoption within Web Services community turned out to be problematic

Started far apart in

applications &

technology

Grid

OGSI

GT2

GT1

WebHTTP

WSDL,

WS-*

WSDL 2,

WSDM

Have beenconverging ?

Grid Computing


A family of Web Services specification proposals

– Introduces a design pattern to specify how to use Web Services to access “stateful” components(WS Resource Framework - IBM, Globus, HP)

– Introduces message based publish-subscribe to Web Services (WS-Notification - IBM, Globus, Akamai, HP, SAP, Tibco, Sonic)

WS-

Serv

ice

Gro

up

WS-RenewableReferences

WS-

Not

ifica

tion

Modeling Stateful

Resources with Web Services

WS-B

ase Faults

WS-ResourceProperties W

S-Resource

Lifetime

IntroducedFirst

To be developed

January 20th Announcement

Grid Computing


Evolution of the specifications

WS-B

ase

Faults


WS

-Res

ou

rce

Pro

per

ties

Modeling Stateful Resources with Web Services

Publish-Subscribe Notification for

Web servicesW

S-Service

Groups

WS-T

opics

WS-BaseNotification

WS

-Reso

urce

Life

time

WS-Brokered

Notification

This is an open process of evolution

NotificationRe-factored

Still to be published

Grid Computing


•OGSA Services can be defined and implemented asWeb Services

•OSGA can take advantage of other Web Services standards

•OGSA can be implemented using standard Web Services development tools

•Grid applications will NOT require special Web Services infrastructure

Network

OGSA Enabled

Storage

OGSA Enabled

Servers

OGSA Enabled

Messaging

OGSA Enabled

Directory

OGSA EnabledFile

Systems

OGSA Enabled

Database

OGSA EnabledWorkflo

w

OGSA Enabled

Security

OGSA Enabled

Web Services

WS-Resource Framework & WS-Notification are an evolution of OGSI


How These Proposals Relate to OGSA

Web Services


Applications

WS-B

ase

Faults


WS

-Res

ou

rce

Pro

pe

rtie

s

Modeling Stateful Resources with Web

Services

Publish-Subscribe Notification for

Web services

WS-Service

Groups

WS-T

opics

WS-BaseNotification

WS

-Res

ou

rceL

ifetim

e

WS-Brokered

Notification

Grid Computing


The definition of WSRF means that Grid and Web Services communities can move forward on a common base

Grid

OGSI

GT2

GT1

WebHTTP

WSDL,

WS-*

WSDL 2,

WSDM

Have beenconverging WSRF

Started far apart in

applications &

technology

Grid and Web Services: Convergence

Grid Computing


Outline

On Demand and Grid






Globus Toolkit

Grid Computing


WS-Resource framework exploits WS-Addressing

– Web Services and WS-Resources are referenced using an “Endpoint Reference”

– Web Services that create or locate WS-Resources returnEndpoint References

Web Service and WS-Resource are separate

– A Web Service is stateless

– A WS-Resource provides a context for stateful execution

– Different entities, different lifetimes, different capabilities

The WS-Resource Framework Model

Grid Computing


Run-time environment

Inte

rface

WebService


Web Service

WSDL


Grid Computing


context

Inte

rface

WebService

messageid

message

Using a Web service to access a WS-Resource

id

address

resource

Endpoint Reference



Grid Computing

context

Inte

rface

WebService

messageid

message

Using a Web service to access a WS-Resource

id

address

resource

resource

Endpoint ReferenceEndpoint Reference




Grid Computing

Inte

rface

WebService

message

message

Creating / Locating a WS-Resource

address

Endpoint Reference

resource

Endpoint Reference

Web Service either locates or creates a

WS-Resource

address

id



Grid Computing


Examples of WS-Resources: – Physical entities (e.g.. processor, communication link, disk drive)

or Logical construct (e.g.. agreement, running task, subscription)

– Real or virtual

– Static (long-lived, pre-existing) orDynamic (created and destroyed as needed)

– Simple (one), or Compound (collection)

Unique - Has a distinguishable identity and lifetime

Stateful - Maintains a specific state that can be materialized using XML

May be accessed through one or more Web Services

resource

What is a WS-Resource?

Grid Computing


IBM

WS-Resource Properties– Resource state and metadata “Projected” as an XML

document– Query and Set operations

WS-Resource LifeTime– Explicit destruction or

“Soft state” time-to-live – Provides for cleanup

of resource instances

resource

<ProcessorProperties><ProcID>5A34C1DE03</ProcID><ProcArchitecture>Power6.2</ProcArchitecture><ProcSpeedMIPS>400</ProcSpeed><ProcCacheMB>256<ProcCache><ProcRunning>1</ProcRunning>

</ProcessorProperties>


Grid Computing


Subscriber indicates interest in a particular “Topic” by issuing a “subscribe” request

Broker (intermediary) permits decoupling Publisher and Subscriber

“Subscriptions” are WS-Resources

– Various subscriptions are possible

Publisher need NOT be a Web Service

Notification may be “triggered” by:– WS-Resource Property value changes– Other “situations”

Broker examines current subscriptions

Brokers may

– “Transform” or “interpret” topics– Federate to provide scalability

Broker

Subscriber

WS-Notification

Publisher

subscribe

subscribe

S S S

notify

notify

notify

notify

Grid Computing


OGSI to WSRF: Refactoring & Evolution

WS-BaseFaultsBase fault type

WS-ServiceGroupServiceGroup portTypes

Treated as a patternFactory portType

WS-NotificationNotification portTypes

WS-ResourceLifetimeGridService lifecycle mgmt

WS-ResourcePropertiesService data definition & access

WS-RenewableReferencesHandleResolver portType

WS-Addressing Endpoint ReferenceGrid Service Handle

WS-Addressing Endpoint ReferenceGrid Service Reference

WS-Resource framework (WSRF)OGSI

Grid Computing


Service Composition

Transports

Messaging

Description

Quality ofExperience(QoX)

HTTP/HTTPS SMTP RMI / IIOP

XSD WSDL

SOAPXML WS-Addressing WS-Renewable References

WS-Metadata ExchangeWS-Policy

WS-Service Group

WS-Resource Properties

JMS

WS-Security

WS-Reliable Messaging WS-Transaction

WS-Resource Lifetime

WS-Base Faults

WS-Notification BPEL4WS

How these proposals relate to other Web Services standards

Grid Computing


GridScheduler

A

ServiceLevel

WS-Resource framework & WS-NotificationScenario: Grid Resource Management & Scheduling

Cluster

RRR

Mainframe

RRRIBM

IBM

Blades

RRR

Notification WS-Resource used to “model” physical

processor resources

Local processor manageris “front-ended” with aWeb Service interface

Other kinds of processors are also “modeled” as same type of WS-Resources

JJ

J

WS-Notification can be used to “inform” the

scheduler when processor utilization

changes

Grid “Jobs” and “tasks” are also modeled using

WS-Resources and Resource Properties

Grid Scheduleris a

Web Service

Service Level Agreement

is modeled as a WS-Resource

Lifetime of SLA Resource tied to the duration

of the agreement

WS-Resource Properties project processor status (like utilization)

Grid Computing


Outline

On Demand and Grid






Globus Toolkit

Grid Computing


Globus Toolkit 2.0 Components

Globus SecurityInfrastructure

MDS client API callsto locate resources

Query current statusof resource

Create

RSL Library

Parse

Allocate &create processes

ProcessMonitor &

control

Site boundary

Client MDS: Grid Index Info Server

Gatekeeper

MDS: Grid Resource Info Server

Local Resource Manager

MDS client API callsto get resource info

GRAM client API statechange callbacks

Process

Process

Job Manager

GRAM client API calls to request resource allocation

and process creation.

Request

Grid Computing


From GT2 to GT3

Grid Computing


Globus Toolkit® and WS-Resource Framework

3.2

Improved robustness, scalability, performance,

usability

3.2March

4.0 Q2

4.0Q3

4.2Q1 ‘05

4.2 Q4

Numerous new WSRF-based services

4.2

4.0

WSRF; some new functionality; further usability, performance enhancements

2004 2005

Note: Globus is not waiting for finalizationof WSRF specs

Grid Computing


IBM Grid Toolbox V3 for Multiplatforms

A commercial release of the Globus Toolkit Version 3.0 with IBM Value Add

–A platform for the Grid Developer to develop and test grid services and grid applications

–A platform for the Grid Deployer/Administrator to host and manage grid services and grid applications

Grid Computing


IBM Grid Toolbox V3 for Multiplatforms Features Integrated Wizard Based Installation A grid service runtime environment based on the GGF Grid Service (OGSI)

specification for hosting grid services A web-based management application used to manage services within the

runtime environment Complete set of configuration and administration commands Development tools to build, package and use grid services Common (base) grid services for:

– GT3.0 Program Execution Service ( GT3.0 GRAM - MMJFS, MJS, UHE)– GT3.0 Information Services (GT3.0 MDS – Index Services)– GT3.0 Data Management Services (RFT, Multi-RFT)– Discovery via Service Group – Policy Management– Common Management Model (CMM)

Information Center including tutorials to assist with the education and understanding of the technologies and capabilities packaged within the product

– Sample grid services and applications demonstrating key capabilities

Grid Computing


Current Platforms SupportedProduct Flyer: IBM Grid Toolbox V3 for Multiplatforms

–http://www-1.ibm.com/grid/pdf/grid_toolbox.pdf

IBM Grid Toolbox V3 for Linux on xSeries

–Order online:

• US, UK: – http://www-132.ibm.com/content/home/store_IBMPublicUSA/en_US/ready2buy.html

and specify part number 52P1214

• France: – http://www-134.ibm.com/webapp/wcs/stores/servlet/CategoryDisplay?

catalogId=-250&storeId=250&categoryId=3160611&langId=-2&dualCurrId=287290

IBM Grid Toolbox V3 for AIX on pSeries

–Order through your local pSeries representative

• Product number 5765-G22

Grid Computing


IBM Virtualization Engine (recently announced)

IBM Virtualization Systems Services

IBM Virtualization Engine

EnterpriseWorkloadManager

SystemsProvisioning

IBM Director Multiplatform

Grid Toolbox

File Aggregation

(SAN File System)

Block Virtualization(SAN Volume

Controller)

StorageInfrastructure Management(Productivity

Center)

eServer and TotalStorage

IBM Virtualization Systems

Technologies

z/OS AIX 5L Linux i5/OS Windows

Hypervisors, Virtual IO, Virtual LAN

Open Based Interfaces and Standards

Open Based Interfaces and Standards

Grid Computing


Summary

OGSA is a service oriented architecture framework, which is the foundation for the On Demand Operating Environment

At its core, there is the Open Grid Services Infrastructure, which specifies how to make a Web Service manageable

– A Grid Service is today a manageable Web Service

Through the WS Resource Framework and WS-Notification specifications, Grid Services and Web Services will converge

OGSA architected services are being defined at the Global Grid Forum: core services, program execution services, and data services

The IBM Grid Toolbox V3 for Multiplatforms today offers a commercial release of the Globus Toolkit Version 3 (i.e. OGSI and higher level services) for Linux on xSeries and AIX on pSeries

Grid Computing


Additional Links OGSA and OGSI

– https://forge.gridforum.org/projects/ogsa-wg

– https://forge.gridforum.org/projects/ogsi-wg WS-RF and WS-Notification

– http://www.globus.org/wsrf/– 3.9.0 Java WSRF Core Preview now available (since 05/21/04)

Globus Toolkit

– http://www-unix.globus.org/toolkit/ IBM Grid Toolbox

– http://www-1.ibm.com/grid/solutions/grid_toolbox.shtml IBM Grid Computing

– http://www-1.ibm.com/grid/

Grid Computing


[email protected]

Thank You

ogsaevolution.ppt

Technology

grid computing

messaging ogsa

storage ogsa

security ogsa

database ogsa

directory ogsa

servers ogsa

workflow ogsa