constructing campus grids experiences adapting myvocs to uabgrid john-paul robinson high performance...

Constructing Campus GridsExperiences adapting myVocs to UABgrid

John-Paul RobinsonHigh Performance Computing Services

Office of the Vice President for Information TechnologyUniversity of Alabama at Birmingham

Internet2 Spring Member MeetingApril 2007

Overview

UAB CyberInfrastructure UABgrid myVocs myVocs box myVocs box on UABgrid Setting Up a VO Future Directions

UAB CyberInfrastructure

UAB HPC Resources Shared HPC Facility has 4 clusters Computer Science HPC Facility has 2 clusters UAB overall HPC computing power has been

tripling approximately on a 2 year cycle during the past 4 years

Optical Networks – campus & regional UABgrid – a campus computing and

collaboration environment

UAB HPC Resources

IBM BlueGene/L System (most recent) 2 Dell Xeon 64-bit Linux Clusters

128 nodes 4 TB disk storage Gigabit and Infiniband interconnect

2 Verari Opteron 64-bit Linux Clusters 64 and 32 nodes 2 GB RAM per node Gigabit interconnect

IBM Xeon 32-bit Linux Cluster 64 Nodes, Gigabit interconnect

UAB 10GigE Research Network

Build high bandwidth network linking UAB compute clusters

Leverage network for staging and managing grid-based compute jobs

Connect directly to high-bandwidth regional networks

UABgrid

Common interface for access to HPC infrastructure

Leverage UAB identity management system for consistent identity across resources

Provide access to regional, national, and international collaborators using Shibboleth identity framework

Support research collaboration through autonomous virtual organizations

UABgrid Architecture

Leverages IdM investments via InCommon

Provides collaboration environment for autonomous virtual organizations

Supports integration of local, shared, and regional resources

UAB Office of the VP of IT CyberInfrastructure Vision

10 Gigabit Ethernet optical network links major research areas in state

High performance computation resources distributed across state

Campus grids like UABgrid provide uniform access to computational resources

Regional grids like SURAgrid provide access to aggregate computational power and unique resources

Alabama Regional Optical Network

Alabama RON is a very high bandwidth lambda network. Operated by SLR.

Connects major research institutions across state

Connects Alabama to National Lambda Rail and Internet2 – projected completion for 2007

Aggregating Resources

UABgrid 2.0, powered by myVocs, to begin pilot operation Summer 2007

Exploring grid interconnection with Alabama Supercomputer Authority and UA System to aggregate resources in state

Continuing participation with SURAgrid to aggregate resources in region

UABgrid Background

Project grew out of NMI Testbed participation, complemented by participation in developing SURAgrid

Initially an integration of campus identity with grid credentials using Pubcookie to issue certificates from UABgrid CA

Initial tool integration based exclusively on identity

UABgrid CA: credentials used by grid computing courses; part of SURAgrid Bridge CA

Limitations of Initial Version

No virtual organization support or other authorization attributes

UABgrid CA key escrow limits trust Support for non-UAB users limited Inter-domain trust via web user interface

doesn't scale well

Complimentary Activities

“NMI Enabled Open Source Collaboration Tools for Virtual Organization” grant explores middleware integration (2003)

Mailing list system integration discussions in Internet2 Mlist working group leads to “Shibboleth Systems” insights (2004)

myVocs.org developed as demonstration of Shibboleth system (2005)

GridShib collaboration expands system reach to Globus-based grid resources (2006)

myVocs box built to ease deployment (2006)

“Shibboleth System”

Simplified, strict “federation” of one identity provider (IdP) with many resources providers reflects trust model of traditional system environments

Using Shibboleth for intra-system attribute transfer supports applications distributed across domain boundaries

The system can receive outside attributes from standard Shibboleth IdP federations

Essentially a proxy identity provider

myVocs

Demonstration virtual organization collaboration environment at myVocs.org

Use Shibboleth for identity management and attribute distribution

Leverage wealth of open source web applications for VO collaboration tools

Globus provides distributed computation foundation

GridShib binds Shibboleth and Globus for common attribute foundation

myVocs Solves the Attribute Puzzle

IdP1IdP1

IdP1IdP2

IdP1IdPn

Identity Providers


IdP1IdP1

IdP1IdP2

IdP1IdPn

Identity Providers

UnivAttributes


IdP1IdP1

IdP1IdP2

IdP1IdPn

Identity Providers

UnivAttributes

VOAttributes


IdP1IdP1

IdP1IdP2

IdP1IdPn

App1

Appn

App2

Identity Providers Applications

UnivAttributes

VOAttributes

A Look Inside myVocs

UABIdP

OtherIdPs

OpenIdP

UIUCIdP

Shibboleth SP

VO IdP with GridShib

VO Attribute Store

VO SPVO SPVO SP

MailList

VO SP

Wiki

VO SP VO SPVO SPVO SP

CMS


GridApps

Globus SP

myVocs

myVocs

myVocs is a “modern application environment” (in spirit of RL Bob's Middleware picture from this morning)

Collaboration application scalability Many users, many organizations, many tools,

many kinds of existing infrastructure Deployment manages application access

myVocs box

A virtual machine instance of myvocs.org Instantiates working federated platform Allows stand-alone exploration of

federation middleware Simplify construction of federated system

environments Support development of federated

applications Conceptualize complex federations as

simple federations in layers

myVocs box Contents

Debian GNU/Linux minimal system install Shibboleth IdM infrastructure Simplified group management with

Sympa Dynamically allocated collaboration tools GridShib CA and IdP interfaces Short-circuit identity provider Basic tools to support stand-alone

operation

Running myVocs box

Download virtual machine image from http://myvocs-box.myvocs.org

Run it with VMware Player or Server Put myvocs-box IP in /etc/hosts Point browser at http://myvocs-box Explore VO management & sample web

tools

http://myvocs-box.myvocs.org/

http://myvocs-box/

UABgrid 2.0

Use of myVocs collaboration environment architecture resolves limitations of initial version

Leverage myVocs box instance as the VO management platform

UABgrid CA aligned with PKI-lite GridShib CA supports grid credential

assignment without key escrow InCommon federation supplies identities

and other useful attributes

UABgrid and myVocs

UABIdP

OtherIdPs

Shibboleth SP

VO IdP with GridShib

VO Attribute Store

VO SPVO SPVO SP

WebApps


GridApps

Globus SP

UABgrid running myVocs box

Know the network profile configuration Import myVocs box into local namespace Integrate with local trust environment Hook in identity providers Establish virtual organizations Migrate existing resources Integrate new resources

Network Profile

Default ports HTTP, HTTPS, SSH. OK No firewall rules. OK Public default root password. Not OK

Import into Namespace

“Import” into namespace means assign appropriate local host name

Host name change affects system, web server, Shibboleth, and messaging

System name is standard host name change process

Web server has static rule with default host name Shibboleth has host name in config and metadata Messaging requires Sendmail to masquerade as

new host name and to listen on external interface

Integrate with Local Trust Environment

UABgrid CA defines PKI trust environment for hosts and users on UABgrid

UABgrid CA will define trust foundation for myVocs box and UABgrid metadata

Migration from default myVocs box trust configuration delayed temporarily to speed exploration of other parts of implementation

Default myVocs config “works” with a false sense of self

Hook in Identity Providers

The goal is to make UABgrid an InCommon application

InCommon will be primary identity federation for UABgrid

UABgrid operating policy for InCommon is being developed

Initial draft awaiting review Two levels of access with different attribute

requirements: collab tools & compute resources

OpenIdP.org in use for initial testing

Establish Virtual Organization

VOs are easy to create by way of the Sympa interface

HPC Services group has existing virtual organization called the Advanced Technology Lab (@lab)

@lab selected for migration to UABgrid VO (Drupal, mailing list, Connotea, Trac, etc)

6 core members with additional affiliates @lab will be used to manage UABgrid

using UABgrid (eat own dog food)

UABgrid Management Project

cfengine for configuration management All nodes will need Globus + GridShib

stack to accept “management” jobs Authorization to execute jobs comes from

@lab VO role Taking system perspective provides a

simplistic model to support construction of infrastructure

Still early on, but grid management using the grid infrastructure is the goal

Experience: Authentication

Shibboleth clearly sufficient for web applications

User certs via GridShib CA interface good for non-web applications

Flexible yet consistent session lifetime management needed – can be achieved for now via published practices

Essentially, authentication needs can be pretty well satisfied with existing technology

Experience: Authorization

Default myVocs authz roles OK for smaller groups (only 3 roles)

No central PDP (each app decides meaning of roles) good for enabling integration rather than enforcing it (applications just receive consistent attributes)

Managing multiple apps independently can be time consuming, use a small number

Experience: Applications

Sample applications in myVocs box are OK for working groups due to scale

Sample web applications dated – the current sample apps need to be updated to latest releases and modernized

Management of some application features requires file system access – need owner/admin file UI for web applications

Need registration UI for additional apps GridShib for Globus is for WS (ie. not SSH)

Experience: Final Thought

Don't get lost in the technology. Shibboleth and Globus are just the means

to building user-driven, federated system environments

Remaining Tasks

Integrate myVocs box with UABgrid trust fabric

Migrate existing applications used by @lab – requires some development work to address Shibboleth support

Integrate additional resources – on-going evaluation of application needs for this and other VOs

Migrate other existing working groups to UABgrid 2.0 (a.k.a. buy-in)

The Future

UABgrid 2.0 Pilot begins summer 2007 Explore grid-based integration with UA System

and Alabama Supercomputer Authority Recruiting additional manpower

myVocs box Will continue to be leveraged on UABgrid for

development efforts and improved as VO management platform

Performance of VM analyzed Ease of administration improved

Shibboleth trust management, additional attributes

Acknowledgments

NSF ANI-0330543 “NMI Enabled Open Source Collaboration Tools for Virtual Organization”

Office of the Vice President for Information Technology, University of Alabama at Birmingham

Projects: SURAgrid, GridShib, Internet2 People: Jill Gemmill, Tom Scavo, Von

Welch, Jim Phelps, Michael Schiffers, David Shealy

References

UAB CyberInfrastructure Planning http://www.uab.edu/it/CyberInfrastructure

UABgrid http://uabgrid.uab.edu

myVocs & myVocs box http://myvocs.org

OpenIdP.org http://openidp.org