globus grid middleware toolkit otto sievert cse 225 8 june, 2000
Post on 28-Dec-2015
216 Views
Preview:
TRANSCRIPT
Globus
Grid Middleware Toolkit
Otto Sievert
CSE 225
8 June, 2000
Globe
and other European Grid Activities
Otto Sievert
CSE 225
8 June, 2000
EGRID
• European Grid Community
• Collaborative community, not a standards group
• Commercial and Academic Interests
• www.egrid.org
European Tour
• Netherlands• Germany• Poland• Italy• Sweden
Grid Theme 1
• Be very (very) careful when choosing a project name.
Amsterdam: Globe
• Vrije Universiteit– Maarten van Steen– Andrew Tanenbaum
• “Middleware to facilitate
large-scale distributed applications”– Web focus– object-based coherency
Globe Uniqueness
• Too much data, too little resources (bandwidth, etc.)
• Caching helps
• Data Coherency integral to Cache Policy
• Release constraint of a single replication/ distribution policy for all objects– example: web pages
IMAGES
COUNTS
HTML
Globe Object
• Physically Distributed• Replicated
• Distribution Policy
Globe Local Object
• 4 subobjects (minimum)
• Modularity
IMAGES
SEMANTICS
REPLICATION
CONTROL
COMMUNICATION
Globe Binding
1. Name server
2. Object handle
3. Location service
4. Contact points
5. Choose point
6. Repository
7. Protocol
8. Bind!
NAMINGSERVICE
LOCATIONSERVICE
IMPLEMENTATIONREPOSITORY
CLIENTPROCESS
1
2
3
4
8
6
7
DISTRIBUTEDSHAREDOBJECT
Legion Binding
• Two-stage name resolution
• Binding agent• No implementation
repository
BINDINGAGENT
CLIENTOBJECT
1
2
3
4
5
SERVEROBJECT
Autopilot Binding
1. Sensor registers with the sole manager
2. AP client requests sensors from the manager
3. Manager returns available sensors
4. Client and sensor communicate directly
MANAGERCLIENT
1
2
3
4 SENSOR
Globe Claim: Remote Object Model Lacks Replication
• Globe– objects can be
replicated
– still maintain one state
– allows complex coherency policies
• Legion– in theory, supports
replication
– replicated state
– allows some but not all coherency policies
– in practice is not allowed
NAMINGSERVICE
LOCATIONSERVICE
IMPLEMENTATIONREPOSITORY
CLIENTPROCESS
1
2
3
4
8
6
7
DISTRIBUTEDSHAREDOBJECT
Globe Architecture
• Why all these servers?– separate naming from
locating– allow extensible
binding protocols (?)
Grid Theme 2: Communication
Centralized– NWS
– Globus MDS*
• Simple Management• Single Point of Failure
Distributed– NetSolve
– Fran’s Sensor Net
• Complex Management
• Scalable
Mixed– Legion
– Globe
– Autopilot
*
***
*** * *
*
*
***
*
Globe Implementation Example
• Set of HTML/image/ Java files
• One semantics subobject• browsers aren’t that
extensible, so …use gateway
SEMANTICS
REPLICATION
CONTROL
COMMUNICATION
browser
gateway
http://globe.foo.edu:8989/dir/file.html
Globe Example (cont’d)
• Replication Policies– Object-based
• “Permanent store”
• “Object-initiated store”
– Client-based• “Client-initiated store”
• How is this any better than what we have now?
Globe Live Demo ...
Globe Location Service
• Scalability Questions
• Tree Heirarchy– again Legion-like in its worst-case behavior
• Typical sol’n assumes mobile client
• Globe sol’n assumes mobile software
Does This Work?
• Single experiment - 5 wk. web server trace
• compare– no caching– various complex replication/coherency policies– automatic adaptive policy
• Results– (essentially) any replication scheme wins big– individual object adaptivity didn’t perform well
Globe: Conclusion
• Explicit coherency is clearly a Good Thing
• Security?
• Representative implementation?
Germany: Cactus
• Albert Einstein Institute, Potsdam– Thomas Radke
– Ed Seidel
• Distributed Astrophysics• Software Engineering• NCSA “hot code”
Cactus
• Separate CS from disciplinary science– Flesh = CS architecture
– Thorns = science modules
• 2-stage compilation– encapsulation
– modularity
– reuse
Cactus Compilation
• Two stage– Permanently bind thorns [Perl]– Compile binary [C++/F77]
• Efficient– Don’t carry unneeded thorn info
Grid Theme 3: Applications
• Numerical, or Non– Computation vs. Specialization
• Performance Measures:– Execution Time– Scale– Efficiency– Distribution
Grid Theme 4: Transparency
• Ease of Use vs. High Performance– As system becomes opaque, EoU increases– As system becomes opaque, Perf decreases– Where is the balance?
Germany: Gridware
• 1999 San Jose-based merger of two companies: Genias GmbH and Chord (U.S.)
• CoDINE– Compute farm load balancing system– Recently adopted by Sun™
• PaTENT– WinNT MPI
Grid Theme 5: Commoditization
• Reuse is strong in the Grid– Resources (Beowulf)– Middleware (Globus, PaTENT)– Applications (Cactus)
• Industry is influential– Largest grid apps in use today are commercial– Grid-ernet is profitable
To This Point ...
Resource
Middleware X X
Application X X
Commercial X
Germany: Unicore
• UNIform Computer Resources (German SC access)• Goal is to provide uniform access to high performance
computers - painful to learn– OS details
– Data storage conventions
– Administration policies
• 3 phase project:I self-contained jobs
II remote data access
III concurrent remote execution
Unicore (cont’d)
• How is this done?– Web (Java) user interface– X.509 authentication– Network Job Supervisor
• interprets Abstract Job Objects
• manages jobs and data
• interfaces with local batch systems (like LoadLeveler and CoDINE)
• vs. Globus?
Poland: POL-34
• National Grid• Very like the system used
by Unicore, a collection of widely-distributed parallel computers
• Tree-connected ATM network
POL-34
• Yellow = 2 Mb/s• Red = 34 Mb/s• Cyan = 155 Mb/s• Single administrative
domain via Virtual Users (skirting the grid issue)
• Use Load Sharing Facility (LSF)
Italy: SARA
• University of Lecce, Italy Giovanni Aloisio (with Roy Williams of Caltech)
• Synthetic Aperature Radar Atlas– Distributed data-intensive app
– Alan Su and the UCSD AppLeS group is involved
SARA Architecture
• The goal: easy, fast, efficient retrieval and processing of SAR image data
• Issues– data is distributed, stored
in tracks
– complex hierarchical system
• Prototypical grid app
Sweden: Computational Steering
• Parallelldatorcentrum Royal Institute of Technology Per Oster
• Using Globus and the Visualization Toolkit (VTK) to steer a single CFD code.
• Little data available• Eclipsed by Autopilot
Grid Theme 6: Heterogeneity
• Some attempt to hide it– Globus, CORBA, Java/Jini
• Some take advantage of it– Netsolve, Ninf
• Some characterize and manage it– AppLeS (SARA), NWS
To This Point ...
Resource X
Middleware X X X X
Application X X X X
Commercial X
Conclusion
• Explored Globe, Cactus, and other EuroGrid favorites in the context of– Communication architectures– Grid application characteristics– Grid transparency– Commodity computing influence– Grid heterogeneity
Network Weather Service (NWS)
• Rich Wolski, U. Tenn.• Monitors and Predicts
Grid Resources– network latency,
bandwidth
– CPU load, avail. Mem.
• Central NWS data server
• nws.npaci.edu/NWS
top related