04/21/23 B. Ramamurthy 1

An Overview of Grid Computing and its Impact on Information Technology

Bina RamamurthyBina RamamurthyPartially Supported by NSF DUE CCLI A&I Grant 0311473

04/21/23 B. Ramamurthy 2

Topics for Discussion

The beginningsThe scientific problems and high performance computing (HPC)The Condor gridThe Globus grid

04/21/23 B. Ramamurthy 3

How did we get here?

Time (years)1970 1980 1990 2000

scale

EUNETMILNET

Sp

ee

dN

um

be

r o

f h

ost

s

Defense:ARPANET

Academic Research:NSFNET

Web applicationInternet Commercialization

Search;Social networkingUnstructuredcontent

Virtualization;Cloud computing

04/21/23 B. Ramamurthy 4

Beginnings of The Grid

Beginnings of the grid in Search for Extra Terrestrial Intelligence (seti@home project)http://planetary.org/html/UPDATES/seti/index.html

The Wow signal http://planetary.org/html/UPDATES/seti/SETI@home/wowsignal.html

Condor High Throughput Computing (HTC) (1996 –Large collections of distributed heterogeneous computing resources.Use of unused cycles in general purpose systems HTC is effective management and exploitation of all available computing resources. For scientific applications requiring lots of flopsURL: http://www.cs.wisc.edu/condor/htc.htmlKeeps track of resources; schedules them on demandDistributed ownership of resources

04/21/23 B. Ramamurthy 5

Scientific Applications

National Fusion collaboratory: http://arxiv.org/ftp/cs/papers/0301/0301033.pdf

Globus Toolkit (GT) Argonne National lab Introduced grid services GT2 C-based GT3 Java/OO based GT4 WS based –lead to unification to

W3C’s WS standard

04/21/23 B. Ramamurthy 6

04/21/23 B. Ramamurthy 7

Issues

Modernizing legacy systemsWeb-enabling business servicesAutomating business processesIntegration within organizationData and application integration Collaboration with outside organizationsSecurity, accountability, availability, reliability, …

04/21/23 B. Ramamurthy 8

Web Services Standard

A common operation on the Internet is search, the results of which is consumed by humans.Develop complex multi-business applications that are beyond the current search-type applicationsWebservices (WS) is a standard that has been introduced by W3 consortium to address this important transition.Grid takes the web services to the next level: a grid service (GS) is a web service.

Grid/GS …… Web/WS ...... Internet

Technology Pipeline

04/21/23 B. Ramamurthy 9

Grid Organizations

The Globus Alliance: www.globus.org conducts research and development to

create fundamental technologies behind the "Grid," which lets people share computing power, databases, and other on-line tools securely across corporate, institutional, and geographic boundaries without sacrificing local autonomy. 

04/21/23 B. Ramamurthy 10

Grid Technology

Natural evolution of distributed systems and the Internet.Middleware supporting network of systems to facilitate sharing, standardization and openness.Infrastructure and application model dealing with sharing of compute cycles, data, storage and other resources.NSF supported Tera-gridPublicized by prominent industries as on-demand computing, utility computing, etc.

04/21/23 B. Ramamurthy 11

Types of Grid

Batch-oriented1. High performance

applications2. High Throughput

applications3. Example: Condor

Service-Oriented 1. View all the resources

and functions as services.

2. Build application models around services.

3. Anatomy of the grid 4. Physiology of the grid 5. Example: Globus

04/21/23 B. Ramamurthy 12

Service-oriented Standards

Open Grid Services Architecture (OGSA)Open Grid Services Infrastructure (OGSI)Globus Toolkit (GT3) is a reference implementation

04/21/23 B. Ramamurthy 13

OGSA, OGSI and WSOGSI has been replaced by WSRF (WS Resource Framework)

04/21/23 B. Ramamurthy 14

Features of Grid Service

Security

Routing

Persistence

ServiceData

Notification

Logging

Basic Service

Logger object; Levels of logging:Info, .. Warn, Error, FatalFiltering and redirecting to file, console

Stores service properties andStates; for discovery, monitoring,negotiations, etc.

Provides notification of events

Permanent services such as naming service thatget activated and terminated with the container

…

Services with abilityto migrate

ProvidesStandard security

04/21/23 B. Ramamurthy 15

Sample Grid Service: Notification

Foundational concepts: messaging, queues, source and sink for messages, subscription model, loose coupling, push and pull notificationGrid related concepts: Service data element (SDE), OGSINotification APIImplement a service that is a producer of notification.Implement a client application that invokes a service that produces notification; an associated listener that consumes the notification. Extend the application to illustrate push and pull model.

04/21/23 B. Ramamurthy 16

Notification Explained

Grid Service

Grid Service

Service Data Element (SDE)

Service Data Element (SDE)

Server Client

Client Application

GS Listener

3: notifyChange()

2: invoke method

1: subscribe to notification 4: process notification

Notes: Example: Grid service (GS) can be a Math Service with notifyChange to SDE on invocation of add Subtract methods.GWSDL file: extends=“ogsi”: GridServiceogsi:NotificationSource (declarative vs programmatic)Listener has: NotificationSinkManager to which is added a listener to Math Service’s GSH and SDE.Listener has deliveryNotification() method to process notification.

04/21/23 B. Ramamurthy 17

Higher Level Grid Concepts

Virtualization of services and resourcesFederation of DataProvisioningLifecycle ManagementVirtual OrganizationOrchestration of servicesChoreography of servicesWeb Services Orchestration and Choreography

04/21/23 B. Ramamurthy 18

Virtualization

Encapsulating service operations behind a common message-oriented service interface is called service virtualization.Isolates users from details of service implementation and location.Assumes support of a standard architecture.Webservices (WS) can do this, however grid life cycle management, fault handling and other features we have seen in the GT3 tutorial are not available with WS.OGSI specification addresses these issues using a core set of standard services.

04/21/23 B. Ramamurthy 19

Virtual Organization

Factory

Factory Mapper

Registry

Service Service Service……

Hardware

04/21/23 B. Ramamurthy 20

Application: Tax Return Filer

Registry

IRSServiceHandleMap

IRSServcieFactory

IRS TAX FilerHostingEnvironment

Registry

EMPServiceHandleMap

EMPServcieFactory

EMPLOYEMENT HostingEnvironment PerService

HandleMap

PERServcieFactory

PERSONAlHostingEnvironment

Registry

BNKServiceHandleMap

BNKServcieFactory

BANK HostingEnvironment

TAX client

Registry

Concepts illustrated: Virtual organization (VO) called IRS/Tax Filer that brings together virtualized capabilitiesof physical organizations of banking, personal profiles, and employment.Grid service handle (GSH) and Grid service reference (GSR), registry and handlemap, discovery of services, index services, application of notification, logging.High trust computing is the social imperative of grid.