![Page 1: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/1.jpg)
Capacity and Capability Computing using Legion
Anand Natrajan([email protected])
The Legion Project, University of Virginia(http://legion.virginia.edu)
![Page 2: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/2.jpg)
Capacity and Capability Computing
• Capacity Computing: Conduct larger computational experiments by expending more resources– single problem– multiple, independent problems
• Capability Computing: Conduct experiments with new mechanisms– Heterogeneity– Security– Collaboration
![Page 3: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/3.jpg)
Grid Environment
Computers Networks People Data Devices
Disjoint file systems Disjoint namespaces Multiple
administration domains
Unpredictable load, availability, failures
Security problems
![Page 4: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/4.jpg)
Grid OS Requirements
• Wide-area• High Performance• Complexity
Management• Extensibility• Security• Site Autonomy• Input / Output• Heterogeneity
• Fault-tolerance• Scalability• Simplicity• Single Namespace• Resource
Management• Platform
Independence• Multi-language• Legacy Support
![Page 5: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/5.jpg)
Legion - A Grid OS
![Page 6: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/6.jpg)
Tools
• MPI / PVM• P-space studies -
multi-run• Parallel C++• Parallel object-based
Fortran• CORBA binding• Object migration• Accounting
• Remote builds and compilations
• Fault-tolerant MPI libraries
• Post-mortem debugger
• Console objects• Parallel 2D file objects• Collections• Licence support
![Page 7: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/7.jpg)
Protein Folding with CHARMMMolecular
Dynamics Simulations
100-200 structures to sample
(r,Rgyr ) space
Rgyr
![Page 8: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/8.jpg)
IBM Blue HorizonSDSC
375MHz Power3512/1184
IBM Blue HorizonSDSC
375MHz Power3512/1184
Resources Available
HP SuperDomeCalTech
440 MHz PA-8700128/128
HP SuperDomeCalTech
440 MHz PA-8700128/128
IBM SP3UMich
375MHz Power324/24
IBM SP3UMich
375MHz Power324/24
IBM AzureUTexas
160MHz Power232/64
IBM AzureUTexas
160MHz Power232/64
Sun HPC 10000SDSC
400MHz SMP32/64
Sun HPC 10000SDSC
400MHz SMP32/64
DEC AlphaUVa
533MHz EV5632/128
DEC AlphaUVa
533MHz EV5632/128
![Page 9: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/9.jpg)
Transparent Remote Execution
• User initiates “run”• User/Legion selects site• Legion copies binaries• Legion copies input files• Legion starts job(s)• Legion monitors progress• Legion copies output files
![Page 10: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/10.jpg)
Mechanics of CHARMM Runs
Leg
ion
Register binaries
Create taskdirectories &specification
Dispatchruns
Dispatchmore runs
77%
20%
1%
2%
0%
0%Blue Horizon
CalTech
UTexas
DEC Alpha
UMich
Sun HPC
![Page 11: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/11.jpg)
Types Of Applications
• Legacy applications• Legion-aware applications
– I/O library– 2D file object
• Applications Using Stdgrid• Parameter Space Studies• Parallel Programs
– MPI, PVM, MPL, Basic Fortran Support (BFS)
![Page 12: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/12.jpg)
Computing in the Near Future
• Security• Fault-tolerance• Heterogeneity• Collaboration• …
• Legion supports these and other needs
![Page 13: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/13.jpg)
Heterogeneous Runs
BT-Med Ocean Model
![Page 14: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/14.jpg)
Cross-Organisation Collaboration
• Different companies• Proprietary simulations and data• Each needs the other• Form virtual partnership
![Page 15: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/15.jpg)
Flexible Context Space
Context
Context Context
Context Directory
Directory Directory
Directory
Disk
Disk e ftp
legion_export_dirlegion_import_tree
SambaNFS HTTP
FTP
![Page 16: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/16.jpg)
Interfaces
• Samba, NFS, FTP, HTTP interfaces to distributed file system
• Windows interface for file sharing• Command-line through Unix-like tools• Web interface through browser• Programmatic interfaces through
system calls in C, C++, Fortran, Java
![Page 17: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/17.jpg)
Platforms
• Windows NT, 2K, 98, 95• Sun (Solaris)• SGI (Irix, Origin)• Intel (Linux, Free BSD)• DEC (Unix, Linux)• Cray (T90, T3E) • IBM (AIX, SP-2)• HP (HPUX)
• Nimrod• Codine• LoadLeveler• Maui• PBS• NQS• LSF
![Page 18: Capacity and Capability Computing using Legion](https://reader033.vdocument.in/reader033/viewer/2022061614/5681495d550346895db6b06d/html5/thumbnails/18.jpg)
Applications
• Biochemistry and Molecular Science• Information Retrieval• Materials Science• Climate Modelling• Neuroscience• Aerospace• Astronomy• Graphics
NPACI - SDSC, UCSD, Caltech, UTexas, Umich, UCB, UVa. DoD MSRCs - NAVO & ARL, NASA Ames