CEOS Grid Task Team

Yonsook EnloeYonsook@harp.gsfc.nasa.gov

September 2004

CE SCommittee on Earth Observation SatellitesWorking Group on Information Systems and Services

Session Topics• Grid Task Team Status – Yonsook – 15 min• EO Research Group in Global Grid F – Yonsook – 5 min• Workflow & Globus Toolkit – Allan – 25 min• Chinese Grid Activities/Workflow – Dr. Huang – 20 min• GMU prototype – Liping Di – 20 min• ADG Prototype – Ken McDonald – 10 min• NOMADS – Dave Clark – 10 min• ESA Grid Update – Ivan Petiteville – 20 min• CNES Grid Update – Paul Kopp – 20 min• GISTDA Grid Activities – Pakorn – 15 min

(Dutch Space & UAH not presenting)

What Did We Do in Past Year?

• Held series of individual telecons, at least one with each of the 7 Application Projects. – Discussed common interests, potential partners, how to facilitate

success of project• Investigating Workflow Tools• Completed Metadata Catalog Service (MCS) 2.0 evaluation• Documented Metadata use across several Application projects• MapCenter Monitoring tool and MDS (resource directory)• Continue Technology Watch• Lost our IPG partner due to NASA funding cuts

– Prototyped the CEOS CA – Looking for new Grid organization partner for US projects – Earth

Systems Grid has been mentioned

Catalog & Metadata• MCS 2.0 released last fall by Ewa Deelman and her team,

partly in response to CEOS catalog requirements• Archie Warnock completed evaluation of MCS 2.0

– cumbersome & difficult installation– lack of metadata population tools – minimum bounding rectangle spatial search – supports standard spatial relations used for comparing spatial

structures (contains, overlaps, disjoint, ...)– slow performance (may be due to MySQL & hardware platform)** but free open source software!

• Documented the metadata standards used by several Application Projects. Team felt that there is no need for additional coordination on metadata beyond simple documentation at this point.

CEOS Certificate Authority (CA)• IPG lost NASA funding and most of the IPG team has

disbanded. IPG no longer exists in function since July 2004.• CEOS Grid was using IPG CA• Team decided to prototype CEOS CA. Ananth Rao doing the

implementation• Issues :

– CEOS CA issued certificates not (yet) widely used and accepted unlike earlier IPG certificates. Other organizations need to accept CEOS certificates

– Current CEOS CA is a prototype system – at current resource levels, will not scale if required to operate in a large Grid system

– Need to use certificates from multiple sources. . – Cannot give the Certificate software to partners outside the US. Each

country must implement their own CA.

CEOS CA Implementation

• Certificate Authority (CA)– Generates and signs its own certificate– Certificate Issuance - signs user, host, ldap certificates– Other functions include certificate renewal and revocation

Uses Globus SimpleCA package to implement back-end CA functionality

• Registration Authority (RA)– has responsibility for checking certificate requesting entity’s identification– CA signs request only after RA has confirmed requesting entity is valid

RA functionality provided through auto-email facility. All incoming certificate requests go through CEOS RA for vetting.

• Certificate Repository (CR)– is a repository of issued and revoked certificates

Set of wrapper scripts (written by IPG and modified by CEOS) provides client access for certificate issue and retrieval

Technology Watch

• Keeping updated a CEOS Grid toolkit of baselined Grid components. – Stability over bleeding edge– Grid Implementation Cookbook – add documentation to

facilitate implementation if documentation lagging

• Globus Toolkit installation Cookbook at http://grid-tech.ceos.org/gridwiki/GlobusToolkitInstallation

• CEOS Toolkit http://grid-tech.ceos.org/gridwiki/CEOSGridToolkit

Technology Watch (cont)

• Globus Toolkit 3.x vs 4.0 discussion – attendees of Globus World (Jan 2004) conference –

huge ripple caused by move towards Web Services Specs from OGSI – Globus 3.x vs 4.x

– And CEOS Grid was still using GT 2.x, waiting for GT 3.x to stabilize before moving to it. What to do now that GT 4.x is on the horizon?

– March Task Team telecon, Carl Kesselman (IPG) gave an excellent talk on Grid Web Services. Allan will give update today.

– Many pointers to current articles at http://grid-tech.ceos.org/gridwiki/GridInformation team website

Workflow Tiger Team

• Workflow Tools Tiger Team : – GridAssist – Dutch Space team (funded by ESA) – Mark ter Linden,

Ruud Grim– NCSA workflow tool (working with UAH) – Jay Alameda– Pegasus Workflow tool – Ewa Deelman– GMU workflow application – Liping Di

• Currently :– Discuss/compare tools, workflow scripts, etc.– How to compare workflow tools? Can we look at workflow

patterns? Led to looking at use cases that describe the Grid Application projects. – Allan will give a talk on this topic today.

• Can we install a workflow tool on CEOS Grid and test ?• Can we use a workflow tool to design a multi-agency Grid

demo?

Map Center & MDS Map Center Tool – basic infrastructure monitoring• Monitors hosts machines on CEOS Grid• Was modified to include Globus Infrastructure Testing

– Checks if Globus installed correctly?– Checks authentication (Certificates)– Checks if simple jobs & shell scripts can be run on connected host

machines– Checks if GridFTP works

MDS (Monitoring and Discovery Service) – resource directory for CEOS Grid– Each machine must populate metadata describing resources available on

that machine and the list of rules to use it– Users can search MDS for certain type resources (e.g. linux systems

having at least 512 MB memory and 2 GB disk) before remote jobs– Can be used by workflow tools to identify resources to run remote jobs

Demo : http://arao2.sgt-inc.com:8090/mapcenter/

Future Work

• Investigate partnering with Earth Systems Grid (ESG) in lieu of IPG

• Continue current activities• Facilitate success of Application Projects and provide

forum where additional partnerships can be forged• Facilitate support of science programs• Common Demo? Consider using an installation of a

Workflow tool in the CEOS Grid to do an across agency CEOS Grid demo at a future meeting (2005?)

Website urls

• Public Task Team website at http://lennier.gsfc.nasa.gov/grid

• Password protected working website at

http://grid-tech.ceos.org/gridwiki