VisPortal Projectdeveloper’s experience
C.E.Siegerist, J. Shalf, E.W. BethelNERSC/LBNL Visualization Group
T.J. Jankun-Kelley, O. Kreylos, K.L. MaCIPIC/UC Davis
Overview• Grid Introduction • Portal Architecture/Software
components. • Visportal Services/Example Pages• User’s perspective• Conclusions/Future/Caveats• References
The Grid• The Grid is an abstraction that allows
transparent use of distributed computing resources.
• The problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations.
VisPortal Project• The LBNL/NERSC VisPortal effort explores
ways to deliver Grid-based advanced visualization and data analysis capabilities through a Web Portal interface.
• Using standard Globus Grid middleware and off the shelf web automation, the VisPortal hides the underlying complexity of resource selection and distributed application management.
VisPortal ServicesMotivation: simplify remote visualization, offer point an click solutions for complex distributed applications
• File Management• Vis Applications: Visapult, AMR Renderer,
Web VisSheet.• One button Mpeg• …
Globus• The Globus Project provides software
tools (Globus toolkit) that make it easier to build computational grids and grid-based applications.
Globusrun Example of a file conversion RSL script
% globusrun –r escher.nersc.gov –f convert.rsl -w
& (executable=$GLOBUSRUN_GASS_URL # “/home/cristina/convertscript”)
(enviroment=(GLOBUSRUN_GASS_URL $(GLOBUS_GASS_URL))
#!/sbin/sh GLOBUS_LOCATION=/usr/local/globusexport GLOBUS_LOCATIONPATH=$GLOBUS_LOCATION/bin:$GLOBUS_LOCATION/sbinexport PATHLD_LIBRARY_PATH=$GLOBUS_LOCATION/libexport LD_LIBRARY_PATHIMAGE_MAGICK=/usr/freeware/bin export IMAGE_MAGICKglobus-gass-server -t -u -r -w -c > $mytmpgassfile &sleep 3set remote_gass_server_url=`cat $mytmpgassfile`rm $mytmpgassfile $GLOBUS_LOCATION/bin/globus-url-copy $GLOBUSRUN_GASS_URL/home/cristina/file.jpg file:/tmp/file.jpg$IMAGE_MAGICK/convert /tmp/file.jpg /tmp/file.bmp$GLOBUS_LOCATION/bin/globus-url-copy file:/tmp/file.bmp $GLOBUSRUN_GASS_URL/home/cristina/file.bmpglobus-gass-server-shutdown $remote_gass_server_url rm /tmp/file.bmprm /tmp/file.jpg
GPDK• The Grid Portal Development Kit (GPDK)
is a set of modular, reusable components for accessing Grid services in the form of Java Beans. It derives most of its functionality from the Globus Java Commodity Grid (Java CoG) toolkit.
• Other toolkits: GridPort, GridSphere
Portal ArchitectureHPC
Resources
WebPortal
Parallel VisualAnalysis Server
User client 1
Archival Storage
& Data Cache
User client n
User site(s)HPC Facility
Audio& video
DataGRAMGSIMDS
GRAMGSI
GSIFTP
httpsSOAPGridFTP
CoG
Portal Architecture (thick client)
HPCResources
WebPortal
Parallel VisualAnalysis Server
User client 1
Archival Storage
& Data Cache
User client n
User site(s)HPC Facility
Audio& video
DataGRAMGSIMDS
GRAMGSI
GSIFTP
httpsSOAPGridFTP
CoG
Thick Client
Resources | Users
Internal ArchitectureWeb Server
Tomcat
GPDKPortal Engine
(Servlets)Application Logic Presentation
(JSP)
Grid Service BeansSecurity Job
SubmissionInformation
ServicesData Transfer
User Profiles
Grid Middleware Libraries(e.g. CoG)
globus
Software Components
• Apache 1.3.27• Tomcat 3.3.1• gpdk cvs version• Java CoG v1.0a• mod_jk 1.3
• mod_ssl 2.8.12
• openssl 0.9.6h • globus 2.x• java 1.4.x• MyProxy 1.0• log4j_1.2.7
GPDKDemo Portal
Login Page
• Myproxy: username and password of delegated credential
• Local: load from credential file (grid-proxy-init)
• Update user’sresourcesjob historyemail address
Profile
File Mgmt.
• Foreground transfers
• Background transfers
• Globus File Yanker(Shreyas Cholia)
Launch ApplicationsVisapult
Launch ApplicationsAMR Renderer
Visportal WebSheet
MPEG Generator
• Example of a user driven service: the user connects to our Onyx to run an IDL batch job to produce a movie with data that he has in an IBM SP • The portal provides an interface to select the data file and the mpeg parameters. It transfers the file, launches the IDL job and sends an email to the user with a URL to retrieve the movie.
The portal is running (most of the time)
https://frost.lbl.gov/visportal/servlet/visportal
User’s perspective• Accounts in all the resources.
• Request a user certificate: http://www.doegrids.org/pages/cert-request.html
• Be in the grid-map files of the resources (automated through NIM).
• Create a proxy certificate and login.
Challenges
• Difficult GUI Implementation• Error Recovery• Support• Middleware is a moving target• Resource management
Future• Deploying a subset of services to
NERSC users.• Implementing specific user
workflows. • Upgrade to OGSA/GT3.0• Explore Gridlab/Gridsphere,
portlets.
Soon:visportal.nersc.gov
References• The anatomy of the Grid: Enabling Scalable Virtual Organizations, Intl. J.
Supercomputer Applications, 2001.• The Grid Portal Development Kit, J.Novotny, Cuncurrency: Pract. Exper.
2000; 00:1-7• How the Grid will affect the Architecure of Future Visualization Systems,
J.Shalf and W. Bethel, http://www-vis.lbl.gov/Publications• Building a Portal Using GPDK: A Developers Tutorial, J. Novotny,
http://doesciencegrid.org//public/events/GPDW/slides/gpdk-dev.pdf• Globus, A Metacomputing Infrastructure Toolkit. I. Foster, C.
Kessselman, Internation Journal of Supercomputing applications, 1997.• Deploying Web-based Visual Exploratin Tools on the Grid. T.J.Jankun-
Kelly, O. Kreylos, J. Shalf, K-L. Ma, B. Hammann, K. Joy, E.W. Bethel. IEEE Computer Graphics and Applications, march/april 2003.
• VisPortal, Deploying grid-enabled visualization tools through a web-portal interface. http://www-library.lbl.gov/library/docs/LBNL/529/40/PDF/LBNL-52940.pdf
• Globus: www.globus.org• GPDK: www.doesciencegrid.org
