Grid

Andrew Grimshaw

September, 2005

What is a Grid System?

A Grid system is a collectionof distributed resources connected by a network.

Examples of Distributed Resources: Desktop Handheld hosts Devices with embedded processing resources

such as digital cameras and phones Tera-scale supercomputers

A grid enables users to collaborate securely by sharing processing, applications, work flows and processes, and data across heterogeneous systems and administrative domains for collaboration, faster application execution, and easier access to data.

What is a Grid? A grid is all about gathering together resources (CPU, data, policy, devices, …) and making them accessible to users and applications.

Grid System

Characteristics of Grid systems

Numerous ResourcesOwnership by

MutuallyDistrustful

Organizations & Individuals

Faulty Resources

Different Security

Requirements & Policies Required

Resources areHeterogeneous

GeographicallySeparated

Different Resource

ManagementPolicies

Connected byHeterogeneous, Multi-Level

Networks

Characteristics of a Grid system

Numerous ResourcesOwnership by

MutuallyDistrustful

Organizations & Individuals

Faulty Resources

Different Security

Requirements & Policies Required

Resources areHeterogeneous

GeographicallySeparated

Different Resource

ManagementPolicies

Connected byHeterogeneous, Multi-Level

Networks

Requirements for robust, enterprise solutions

• Interoperable implementations• Standards-based• Simple• Secure• Scalable• Extensible• Site Autonomy• Multi-Language• Legacy Support• Transparency in multiple dimensions

• Naming schemes critical• Fault-tolerance & Exception Management• Modular and composable

Success Requires an integrated model at the foundation.

OGSAComplexity Management!!

Users view

Site 0 Site 1 Site 2 Site 3

SP2

SP2

HPSS

UsersUsers

Grid

Runprograms

AccessData Collaborate

Provideshared

services

Evolution in action

Bare Metal Programming

50’s

Batch OS

Multi-UserTimeshare

60’s to 80’s

Low Level Network

Programming

Today

Grid & WS

Now & Future!

Grid Computing Scenarios

Desktop Cycle Aggregation• Limited acceptance in commercial enterprises

Cluster Grids• Single owner, department, project • Single domain, file system• LAN connection

Campus/Enterprise Grids• Multiple owners, domains• Multiple file systems• WAN connection

Partner Grids• Multiple owners, sites, domains• Multiple file systems• Internet connectivity

Legion Grid

Software – C

ompute

and Data G

rid

Grid “component” models• Most grids today have a service oriented architecture

where (grid) resources are endpoints with• An interface – usually WSDL• Some meta-data – usually resource properties• A base set of functions that can be assumed, e.g., lifetimes,

notification, etc.• Some form of factory model

• Most grids today have• Some form of information discovery model• Some means of moving data around implicitly or explicitly• Some means establishing identity and mapping between

identity spaces• Can operate between administrative domains• Can operate through firewalls

Vendor neutral Grid buzz words• GGF – Global Grid Forum

• OGSA – Open Grid Services Architecture• EMS – Execution Management Services

• BES, RSS,• Data

• ByteIO• Naming

• RNS• WS-Names (identities)

• Provisioning and deployment• JSDL – Job Submission Description Language• WS-Agreement

• WS-Addressing• WSRF/WSN• WSDM

Context Services Info

Services

InfraServices

SecurityServices

Rsrc Mgmt Services

Execution Mgmt

Services

DataServices

PolicyMgmt

VOMgmt

Access

Integration

Transfer

Replication

BoundaryTraversal

Integrity

Authorization

Authentication

WSRF WSN WSDM

EventMgmt

Monitoring Discovery

JobMgmt

Logging

ExecutionPlanning

WorkflowMgmt

WorkloadMgmt

Provisioning

Execution

DeploymentConfigurationReservation

Naming

Self MgmtServices

HeterogeneityMgmt

Service LevelAttainment

QoSMgmt

Optimization

Information Services

Infrastructure Services

SelfMgmtServices

SecurityServices

Resource Mgmt Services

Execution Mgmt Services

DataServices

Context Services

UVA and Grid• Pioneered grid/metasystems with Legion

• Integrated OGSA-like system• Stress on ease of use

• Motto - “by default the use does not have to think”

• First operational in 1994• Grounds-wide virtual computer in 1996 • NPACINet & DoD MSRC grid• Dozens of applications

NPACINet - 2000

NPACI - SDSC, UCSD, Caltech, UT, UCB, UM, UVaDoD MSRC’s - NAVO & ARL, NASA - Ames

The Global Bio Grid

Tomorrow

• $1000/sequnce for humans – becomes standard clinical practice

• Global interconnected networks – grids• Provide transparent, secure, access to data,

applications, and on-demand compute.

• Research using not just your data, but all trusted data, not just your applications, but any trusted application.

Tomorrow - Clinical

• Unified clinical records

• Data mining of integrated clinical and scientific databases

What this will facilitate

• Discovery of new diagnostics and predictors for prescriptive medicine

• Discovery of new treatments to complex genetic diseases

• Push back expected lifetimes and improve quality of life

• Deliver better health care by identifying problems earlier

• Wide spread epidemiological research

There are a number of “catches”

• So much data!

• So many organizations with so little trust!

• So much complexity!

This is where grids enter the picture

• Federated access to multiple • Data sources

• Public databases• Commercial databases• In-house databases, annotations, etc.

• Application suites (including processes and workflows)• Compute resources

• Shared among collaborative research teams• Multiple research locations• Virtual organizations

• Heterogeneous compute environments• OS, geography, administrative domains, policies

• Fine-grain modular security with work towards HIPAA and CFR 21 part 11

• Built on evolving standards (GGF, WS-*)

GBG concept

GBG Today

• Datagrid using Avaki DG technology• UVA, NCBIO, U-Texas, Texas Tech• Already operational• Flat file and relational• Replacing with open-source data grid

• Compute grid at UVA on-line• IBM, 64 node, cluster as keystone• Four clusters on-line• 300+ lab PC’s running Windows XP