application portlets within the progress hpc portal michał kosiedowski kat@man.poznan.pl

Application portlets within the PROGRESS HPC Portal

Michał Kosiedowski

kat@man.poznan.pl

Agenda

•Grid portals•PROGRESS HPC Portal•Specialized application portlets

•Conclusions

Grid portals: today

• Target: access to distributed grid services from multiple portals

• TACC GridPort 3.0• NCSA Alliance Portal• PROGRESS HPC Portal

PROGRESS

Grid Service Provider

• Job Submission Service– computing job building, submitting jobs to the grid for

execution and monitoring the execution– grid job descriptions in the XRSL language are transferred to

the GRB for the execution of the job– possibility to prepare workflows

• Application Management Service– manages the PROGRESS application repository– an application descriptor contains a reference to the

application’s executable and a set of its available, required or optional arguments, required environment variables as well as input and output files

– one executable may be referenced by many applications– different application configurations are recognized as

independent applications

Data Management System

• A distributed system enabling the management of grid data files

• Storing files in distributed storage modules of various types: generic filesystems, archivers, relational databases

• Uses metadata to describe files• Allows access to data banks like a mirror of

Sequence Retrieval System• Exposes its functionality within the Data

Broker Service

PROGRESS HPC Portal

Core portlets

• “My computing jobs”:– creation, configuration and execution of grid jobs on the grid– downloading and visualizing the results– utilizes the functionality of the GSP Job Submission and

Application Management services and the DMS Data Broker. • “Applications”:

– management of grid-enabled applications within the GSP application repository

– co-operates with the GSP Application Management Service and the DMS Data Broker.

• “My data”:– management of the files stored within the DMS– uploading and downloading files from the DMS– communicates with the DMS Data Broker.

“My computing jobs” portlet (1)

“My computing jobs” portlet (2)

“My computing jobs” portlet (3)

“My computing jobs” portlet (4)

“My computing jobs” portlet (5)

“My computing jobs” portlet (6)

“My computing jobs” portlet (7)

“My computing jobs” portlet (8)

“My computing jobs” portlet (9)

“My computing jobs” portlet (10)

“My computing jobs” portlet (11)

• Exposes the whole functionality of the Job Submission Service

• Enables submission of any job utilizing any available application

• Great for experienced users allowing them to fulfill their requirements

• Too complicated for use by non-advanced users

• Solution: specialized application portlets utilizing the same services as core portlets and familiar with the corresponding grid-enabled application structure

“Gaussian” portlet (1)

“Gaussian” portlet (2)

“Gaussian” portlet (3)

“Gaussian” portlet (4)

“DNA Assembly” portlet (1)

“DNA Assembly” portlet (2)

“DNA Assembly” portlet: Visualization

Application portlets

• Provide users with interfaces that are easy to use and require no grid knowledge

• Grid jobs may be configured very quickly with an intuitive wizard

• Easy and quick access to results: application portlets are familiar with corresponding applications

• Problem: How to quickly build such specialized portlets?

PROGRESS Portlet Framework

• The Framework allows preparing a high-level interface to Web Services with special beans to store the data obtained from WS responses and classes used to translate HTTP requests into the invocation of proper WS methods.

• This allows to quickly build new portlets on top of those classes to utilize the already existing functionality that they provide.

Provider ...Provider Provider ProviderProvider

HTTP

Content Generator

Content Generator

Content Generator

Content Generator

Content Generator

Request Handler

Request Handler

Request Handler

Request Handler

Request Handler

Request Handler...

Web Service Proxy ...Web Service

ProxyWeb Service

ProxyWeb Service

ProxyWeb Service

Proxy

...

Web Services Invocation Framework

SOAP (to GSP, DMS)

Portlets

Communication with Web Services (1)

• Web Services Invocation Framework is used for communication with Web ServicesObject executeMethod(

String methodName, ArgumentTable methodArguments)

• AXIS toolkit is used to generate the binding classes for complex class types mapping

• Each Web Service is described in an XML configuration file

<?xml version="1.0" encoding="UTF-8"?><web-services> <web-service id="appmgmt"> <name>Application Management</name> <wsdl-location>ApplicationManagementWS.wsdl</wsdl-location> <port-type-name>

http://viola.man.poznan.pl:5021/axis/services/ApplicationManagement </port-type-name>

<port-type-namespace>ApplicationCtrl</port-type-namespace> <description>GSP Application Management Service</description> </web-service> <web-service id="jobs"> <name>Job Submission</name> <wsdl-location>JobSubmission.wsdl</wsdl-location> <port-type-name>

http://viola.man.poznan.pl:5021/axis/services/JobSubmission </port-type-name>

<port-type-namespace>JobSubmissionCtrl</port-type-namespace> <description>GSP Job Submission Service</description> </web-service> <web-services>

Communication with Web Services (2)

Content generation (1)

• The content is generated based on two HTTP request parameters (page and action). 1. page e.g. ApplicationList, JobList2. action e.g. removeApplication, submitJob

• Actions are defined in actions.xml, pages in pages.xml (the content of those files is read by appropriate classes, ContentGenerator reads Action and Page objects from those classes). Actions are executed before page content is generated; pages usually contain default actions to generate their content.

• Actions are usually page independant, they can also replace the default page action

• An action is associated with one or more RequestHandler methods,or with methods of many RequestHandlers

• The Java reflection mechanism is used to call RequestHandler methods

Content generation (2)

• Page content is delivered by RequestHandlers in form of PageBean classes

• Each page can contain one or more classes implementing the PageBean interface (PageBean classes know how to deliver their content in form of a DOM)

• The page generates its content by converting its PageBeans into a DOM and then transforming it using an XSLT template

Page

PageBean1

RequestHandler1

RequestHandler2

RequestHandler3

PageBean2

PageBean3

PageBean4

DOM1

DOM2

DOM3

DOM4

DOM

Content Generator

HTML

Content generation (3)

• Pages are defined in an XML config file:

<pages>

<page name="ApplicationList">

<display-name>Application List</display-name>

<description>List of PROGRESS applications</description>

<xsl-file>application-list.xsl</xsl-file>

<action name="ApplicationList">

<request-handlers>

<request-handler class="ApplicationGetRequestHandler">

<method name="getApplicationList"/>

</request-handler>

</request-handlers>

</action>

</page>

...

</pages>

Content generation (4)

• Invocation of many RequestHandlers

<page name="EditApplicationExecutable">

<display-name>Edit Application Executable</display-name>

<description>

Edit application executable file - choose/change the file.

The executable can also be uploaded to SZD.

</description>

<xsl-file>edit-application-executable.xsl</xsl-file>

<action name="EditApplicationExecutable">

<request-handlers>

<request-handler class="ApplicationGetRequestHandler">

<method name="getApplication"/>

</request-handler>

<request-handler class="SzdGetRequestHandler">

<method name="getUserDirectoryListing"/>

</request-handler>

</request-handlers>

</action>

</page>

Content generation (5)

• RequestHandler ”workflow” <page name="JobList">

<display-name>Job List</display-name>

<description>List of jobs for a given user</description>

<xsl-file>job-list.xsl</xsl-file>

<scripts>

<script language="JavaScript" src="jscript/job-submission/job_list.js" />

</scripts>

<action name="JobList">

<request-handlers>

<request-handler class="JobGetRequestHandler">

<method name="getJobsOfUser" output-name="userJobs"/>

<method name="filterJobs"

input-name="userJobs" output-name="filteredJobs"/>

<method name="sortJobs" input-name="filteredJobs"/>

</request-handler>

</request-handlers>

</action>

</page>

Content generation (6)

• Page error redirection – defined in an error-redirections.xml file:

<?xml version="1.0" encoding="UTF-8"?>

<redirections><error-redirection-group id="EditJob:saveJob">

<error-redirection type="validation" redirect-to-page="EditJob" />

</error-redirection-group>

<error-redirection-group id="EditTaskInfo:saveTaskInfo">

<error-redirection type="validation" redirect-to-page="EditTaskInfo" />

</error-redirection-group>

</redirections>

Framework advantages

• It’s easy to reuse the portlets in different portal frameworks and on various portals and websites– The top-level Provider class can be easily adjusted to

implement any portal framework API– The content generated by a portlet can be easily

changed by playing with configurations files

• The same RequestHandlers can be used by mutliple portlets thus enabling easy creation of new portlets communicating with the same WS services– specialized application portlets, e.g.: “Gaussian” portlet,

“DNA Assembly” portlet

Conclusions

• Our experience shows it’s important to provide users with interfaces to grid-enabled applications that are easy to use and do not require deep knowledge about the grid and/or the application.

• It’s now easy to create such interfaces having powerful grid services on hand and tools like PROGRESS Portlet Framework that facilitate building these interfaces.

PROGRESS

http://progress.psnc.pl/

http://progress.psnc.pl/portal/

progress@psnc.pl