SPARRO Group, University of Regina 1

Portal Software: Browser-based Monte Carlo

Zisis PapandreouUniversity of Regina

GlueX Collaboration MeetingJLab, December 11-13, 2003

SPARRO Group, University of Regina2

Outline

Grid2003 Conference Grid Services

Regina Portal Application framework, Globus Meta queuing

Collaboration tools: GlueX Forum, Database Administrator

SPARRO Group, University of Regina3

Grid2003 Conference (Ed, Zisis)

Phoenix, AZ, November 16-17, 2003 Concurrent with SC2003 (8,000 hackers) Interactive and coached tutorial:

How to Build a Grid Service using the Globus Toolkit® 3 (Argonne people)

Written in Java Built (JDK) and run on Ed’s RH9.0 linux laptop

SPARRO Group, University of Regina4

GT3 Architecture Core: Grid Services Security: SSL, X.509 Base: Job Services,

Index Services, RFT Data: Replica

Management (data subsets)

Other: foreign grid services

OGSA

SPARRO Group, University of Regina5

javac Math.javajava org.apache.axis.wsdl.Java2WSDL-P MathPortType -S MathService -l http://localhost/ogsa/services/core/first/MathService -n http://first.core.gt3tutorial/Math gt3tutorial.core.first.impl.Math

package gt3tutorial.core.first.impl;public interface Math { public int add(int a, int b); public int subtract(int a, int b); public int multiply(int a, int b); public float divide(int a, int b); }

Creating a Grid Service

Write WSDL directly Generate WSDL from

interface language (Java) Decorate it Generate stubs Implement it

Web Services Description Language

SPARRO Group, University of Regina6

Exercises accomplished Grid Services / GT3 FileShare Service

deployment File sharing with

identifiers Adding service data Register with

community index

Lifetime management File searching Security Transience: create

and destroy FileShares

Grid Service Publishing

SPARRO Group, University of Regina7

Demonstration:FileShare Grid Service

(Ed Brash)

SPARRO Group, University of Regina8

Regina Portal: Beefed up

Glasgow: job submission, review (PHP)

New features: ColdFusion Application Framework (CFML) User registration & authentication Integration with GridPort TK, Globus Upload XML Geometry and compile Experiment or commit executables to db

https://www.phys.uregina.ca/GlueX/

SPARRO Group, University of Regina9

Demonstration:Regina Portal(Zisis Papandreou)

SPARRO Group, University of Regina10

Grid Computing “The Grid”: distributed computing infrastructure (both

software and hardware) that provides access to high-end computational capabilities

Grid computing enables the shared use of computational resources such as data archives and simulations.

Resource owners offer resources subject to constraints Resource consumers also place constraints on properties

SPARRO Group, University of Regina11

Queuing, Scheduling &Meta Scheduling

(Aadila Soobrattee)

Regina openSpace cluster (50 CPUs) is governed by OpenPBS local queuing software

Need capabilities of scheduling workload among a computer grid where local scheduling software and local policies may differ: meta-scheduler

Schemes: Centralized: surface info, locals dispatch jobs Hierarchical: meta and local share info; no flexible re-routing Distributed: scheduler/dispatcher model; flexible, scalable

SPARRO Group, University of Regina12

Distributed Scheduling

Meta-scheduler

Local Scheduler/ Dispatcher

Local Scheduler/ Dispatcher

Local Scheduler/ Dispatcher

Jobs

Job flowInformation flow

Meta schedulers at every site Jobs submitted to meta scheduler where job originates Meta schedulers query each other to collect instantaneous load

info; job is transferred to the site with the lowest load Dagman, Condor/G, Nimrod/G, GT3/Platform, Silver/Maui

SPARRO Group, University of Regina13

Silver/Maui

Silver provides grid scheduling services for clusters using OpenPBS, can operate on all major UNIX operating systems and is capable of operating with or without Globus

Jobs Jobs

Jobs

Meta-Scheduler

Local Scheduler

Meta-Scheduler

Local Scheduler

Meta-Scheduler

Local Scheduler

Silver:•reduces average job queue time by enabling access to a larger set of computer resources•intelligently decomposes jobs into multi-system sub-jobs to improve job turnaround time•manages site-dependent job environment and resource requirements•supports jobs which require co-allocated resources which expand multiple systems•supports jobs too large to run at any single site by spanning jobs across multiple sites

Silver:•reduces average job queue time by enabling access to a larger set of computer resources•intelligently decomposes jobs into multi-system sub-jobs to improve job turnaround time•manages site-dependent job environment and resource requirements•supports jobs which require co-allocated resources which expand multiple systems•supports jobs too large to run at any single site by spanning jobs across multiple sites

http://www.supercluster.org/

SPARRO Group, University of Regina14

1. User logs into Globus and gets Globus credential2. User submits job from site A via sqsub client3. sqsub translates job to global job and stores job and credential in SILVER's global queue4. SILVER determines list of feasible resources based on resources requested5. SILVER queries Globus and determines machines that support the specific credential6. SILVER queries each feasible resource and obtains matrix of resource availability times7. SILVER determines resource associated with earliest job completion time (site B)8. SILVER reserves resources on site B9. SILVER stages input data and executable to site B using Globus file transfer (gssftp)10. SILVER translates global job to RSL and submits job to Globus with 'run on site B' request11. Globus translates RSL job to resource manager command language and submits the job to resource manager on site B.12. MAUI detects job, executes it on reserved resources13. job completes14. MAUI creates accounting record, removes reservation, and updates SILVER15. Output data is staged to requested destination using Globus file transfer

SPARRO Group, University of Regina15

Collaboration Tools GlueX Forum (at UConn)

http://zeus.phys.uconn.edu/forums/gluex.org/index.php Web-based email forum (written in PHP) Category-based email, threads, moderators

PostgreSQL Database Administration pgAdminIII (win & unix) Remote connection to db servers Create/delete tables, enter data, view data

SPARRO Group, University of Regina16

What’s next? Expand portal further:

Install Meta Queuing System at Regina Web/Grid Services (document uploading, indexing and

searching) XML Data Transfer (Web Distributed Data Exchange –

WDDX) Communications:

Teleconferencing: AT&T Videoconferencing: VRVS or Polycom Email threaded discussions: GlueX Forum