application porting using drmaa and globus gridway on the egee infrastructure
DESCRIPTION
Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure. J.L. Vázquez-Poletti (UCM) EGEE08 (Istambul). MaRaTra. Massive Ray Tracing in Fusion Plasmas Traces each ray contained in a microwave beam which heats fusion plasma Finds best “shooting” position - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/1.jpg)
EGEE-III INFSO-RI-222667
Enabling Grids for E-sciencE
www.eu-egee.org
EGEE and gLite are registered trademarks
J.L. Vázquez-Poletti (UCM)
EGEE08 (Istambul)
Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure
![Page 2: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/2.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
MaRaTra
• Massive Ray Tracing in Fusion Plasmas– Traces each ray contained in a microwave beam which heats fusion plasma – Finds best “shooting” position
Needs to be executed as much as possible in less time• Fusion Physics (Fusion VO) – Proposed by CIEMAT (Spain)• Parameter Sweep (with pre and postprocess)• 1 ray = 1 job = 9’ on a Pentium 4
– Upgrade: Using chunks reduces transfer and queue overhead• CLI developed using OGF DRMAA C API provided by GridWay
– lcg2.1.69 User Interface C++ API was tested at the beginning (couldn’t handle big collections of jobs)
• Production Status: up to 1001 Rays in 2h 10’
![Page 3: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/3.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
MaRaTra
5 Rays 10 Rays 15 Rays
0
5
10
15
20
25
30
35
101 Rays
Chunks
Time (min)
5 Rays 10 Rays 15 Rays
0
50
100
150
200
250
1001 Rays
Chunks
Time (min)
10R
2R 2R2R
2R
2R 2R
10R 5R 5R
![Page 4: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/4.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
CD-HIT Grid
• CD-HIT: Cluster Database at High Identity with Tolerance– Compares protein DB entries and eliminates redundancies– Example: UniRef data sets generated by UniProt– Infeasible to be executed on a single machine (memory requirements and total
execution time)• Bioinformatics (BioMed VO) – Proposed by CNIO (Spain)• Intree Workflow (with pre and postprocess)
– Grain can be adjusted– Upgrade: optimization heuristics for efficient execution have been added
• CLI developed using OGF DRMAA C API provided by GridWay• Production State: Up to 1.7GB DB processed in 7h 45’
![Page 5: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/5.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
CD-HIT Grid
Agg. Level
Tim
e (h
ours
)
![Page 6: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/6.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
SGE Cluster
Users
PBS Cluster PBS Cluster
gLite
GridWay
gLite gLite
• Services: BDII, GRAM, GridFTP• EGEE Resource Broker
• DRMAA interface• VO Schedulers
GridWay
Users
BiomedFusion
EGEE RB
Massive Ray Tracing CD-HIT workflow
Grid Execution Architecture
![Page 7: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/7.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
The GridWay Metascheduler
ExecutionManager
TransferManager
InformationManager
DispatchManager
RequestManager
Scheduler
Job Pool Host Pool
DRMAA library CLI
GridWay Core
GridFile Transfer
Services
GridExecutionServices
GridFTP RFTpre-WSGRAM
WSGRAM
GridInformation
Services
MDS2MDS2GLUE
MDS4
Resource DiscoveryResource MonitoringResource DiscoveryResource Monitoring
Job PreparationJob TerminationJob Migration
Job PreparationJob TerminationJob Migration
Job SubmissionJob MonitoringJob ControlJob Migration
Job SubmissionJob MonitoringJob ControlJob Migration
![Page 8: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/8.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
• Distributed Resource Management Application API– http://www.drmaa.org/
• Open Grid Forum Standard• Homogeneous interface to different Distributed Resource
Managers (DRM):– SGE– Condor– PBS/Torque– GridWay
C JAVA Perl Ruby Python
What is DRMAA?
The GridWay Metascheduler
![Page 9: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/9.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
• Community – Open Source Project. Globus Development Philosophy
• Development Infrastructure (thanks to Globus Project!)
• Mailing Lists
• Bugzilla
• CVS
• You are very welcome to contribute:
• Reporting Bugs ([email protected])
• Making feature requests for the next GridWay release ([email protected])
• Contributing your own developments (bug fixes, new features, documentation)
• Detailed Roadmap:
• GridWay Campaigns at bugzilla.mcs.anl.gov/globus/query.cgi
• www-unix.mcs.anl.gov/~bacon/cgi-bin/big-roadmap.cgi#Gridway
Development Process
The GridWay Metascheduler
![Page 10: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/10.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Supported Application Porting
• Chemistry– GAMESS and GAUSSIAN at group QCyCAR (UCLM)– Chemical Reactor Design in BE14 (“New Product & Process Development”) of
BEinGRID project• Life Sciences
– Dynamic BLAST at UABGrid– Multi-Resolution Docking at Centro de Investigaciones Biológicas (CSIC)– Verification and Optimization of Radiotherapy Treatments in BE25
(“BEinEIMRT: Business Experiment in Enhanced IMRT planning using Grid services on demand with SLAs”) of BEinGRID project
• Astronomy– XMM-Newton Data Processing at ESAC (ESA)– Use of Special Hardware for N-Body Dynamics in Galactic Nuclei at
AstroGrid-D• Climate
– Global Atmospheric Climate Simulations at EELA– Weather Research and Forecasting at LAGrid
• Geology– Petroleum Reservoir Simulations at TIGRE
![Page 11: Application Porting using DRMAA and Globus GridWay on the EGEE Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022051316/56814ad7550346895db7f079/html5/thumbnails/11.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Teşekkür ederim!
Questions?