evgm081 multi-site virtual cluster: a user-oriented, distributed deployment and management mechanism...
TRANSCRIPT
![Page 1: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/1.jpg)
EVGM08 1
Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management
Mechanism for Grid Computing Environments
Takahiro Hirofuchi, Takeshi Yokoi, Tadashi Ebara,Yusuke Tanimura, Hirotaka Ogawa, Hidetomo Nakada,
Yoshio Tanaka, and Satoshi Sekiguchi
National Institute of Advanced IndustrialScience and Technology (AIST)
![Page 2: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/2.jpg)
EVGM08 2
Background
Grid Computing Large-scale Distributed Heterogeneous
Grid Middleware Globus toolkit gLite
![Page 3: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/3.jpg)
EVGM08 3
Software Deployment and Management
Problems Software complexity
Dependency Configuration
Resource diversity Hardware
• x86, x64, memory, storage, … Software
• CentOS 4, CentOS 5, Solaris, …• Library versions
Organizations• Administration policy
x Sites
Management CostExplosion
![Page 4: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/4.jpg)
EVGM08 4
My Experience
VOMS Installation at 2007 spring Virtual organization management system Supports only SL3 (based on RHEL3 since 2003-12) It was nightmare
Debian Etch CentOS 5
VOMS depends on GT4.VOMS requires old GCC for C+
+.
GT4 requires new GCC for its patched Open
SSL
Me, too…?
![Page 5: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/5.jpg)
EVGM08 5
Our Concept
Virtualization Isolate resources Capsulate environments Create new administrative domains
Site A
Site B
Site CCreate virtual machinesat each site
![Page 6: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/6.jpg)
EVGM08 6
Our Concept
Virtualization Isolate resources Capsulate environments Create new administrative domains
Site A
Site B
Site C Group distributed VMsfor a Virtual Organization
![Page 7: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/7.jpg)
EVGM08 7
Our Concept
Virtualization Isolate resources Capsulate environments Create new administrative domains
Site A
Site B
Site C Multi-Site Virtual Cluster
![Page 8: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/8.jpg)
EVGM08 8
Multi-Site Virtual Cluster Integrate distributed VMs
Single cluster view. Allow single administrative domain
OS installation and full configuration Enable easy system deployment
Large-scale nodes
Application
Site A Site B
Application
Scientific applicationEmulation testbed
Deploymentand configuration
![Page 9: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/9.jpg)
EVGM08 9
System Components (1)
Site A
Site B
Site C
Resource VirtualizationMechanism
![Page 10: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/10.jpg)
EVGM08 10
System Components (2)
Site A
Site B
Site C
Web Service APIfor Virtualized Resource Control
![Page 11: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/11.jpg)
EVGM08 11
System Components (3)
Site A
Site B
Site C
Easy management systemfor large-scale, distributed nodes
![Page 12: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/12.jpg)
EVGM08 12
Resource Virtualization Mechanism
Design criteria Create completely-isolated VMs (Virtual Cluster)
Our virtual cluster system Physical nodes VMware Server and Xen VLAN LVM and iSCSI Free and Open Source
http://code.google.com/p/grivon
![Page 13: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/13.jpg)
EVGM08 13
Allocating a Virtual Cluster (1)
ClusterManager VMM VMM VMM
Private Network
Public Network
![Page 14: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/14.jpg)
EVGM08 14
Allocating a Virtual Cluster (2)
ClusterManager VMM VMM VMM
Create a new VLAN (eth0.1234)
Bridge to VLAN
![Page 15: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/15.jpg)
EVGM08 15
Allocating a Virtual Cluster (3)
ClusterManager VMM VMM VMM
Create new storage volumes
LVM
Attach via iSCSI
![Page 16: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/16.jpg)
EVGM08 16
Allocating a Virtual Cluster (4)
ClusterManager
VMM VMM VMM
Launch VMs on VMM
LVM
Attach via iSCSI
VM VM VM
![Page 17: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/17.jpg)
EVGM08 17
Web Service API
REST API https://example.com/api/vc/ /api/vc
• List virtual clusters (GET), Create a new virtual cluster (POST) /api/vc/1234/
• Get the status of VC 1234 (GET) /api/vc/1234/vm
• List the VMs (GET), Add/Delete a new VM(POST) /api/vc/1234/vm/{0, 1, 2, 3, 4, 5}
• Get the status of a VM, Start/Stop a VM (POST) /api/vc/1234/vpn /api/vc/1234/vpn/{0, 1, 2}
![Page 18: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/18.jpg)
EVGM08 18
Distributed Node Management
Exploit an existing cluster management system for the inside of a multi-site virtual cluster Designed for physical clusters Powerful node management
Node database, parallel command execution, automatic node installation
Integrate distributed VMs by Ethernet VPN Bridge internal networks of single-site virtual clusters Transparency for admins and users
Transparent package caching at each site Package-based software installer Quick installation and reconfiguration Flexible customizability
![Page 19: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/19.jpg)
EVGM08 19
VM VM VM
VM VM VM
VM VM VMEthernet VPN among site-local VLANs
Package cacherepository
Managed by Rocks toolkit
Frontend Node Console Node DB PXE installation server
![Page 20: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/20.jpg)
EVGM08 20
PrototypeImplementation
•Reservation portal Site resource monitoring Automatic reservation
• NPACI Rocks 4.2• VMware Server• VLAN, iSCSI• OpenVPN 2.0• Squid 3.0
![Page 21: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/21.jpg)
EVGM08 21
Evaluation
Demo over the Pacific Software configuration Parallel command line tools Node status monitoring Condor job sumission
Evaluate ScalabilityWANLarge number of VMs
Reconfiguration time Network traffic
![Page 22: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/22.jpg)
EVGM08 22
Experiment Setting
AMD Opteron 244, 3GB Mem, Gb Eth x2
16 nodes 134 nodes, reconfigured with 900MByte packages
AMD Opteron 246, 6GB Mem, Gb Eth x2
Node Config. DB
![Page 23: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/23.jpg)
EVGM08 23
134 Node Reinstallation over WAN
Only 20 minutes for a134-nodes virtual cluster
900MB programs per node
![Page 24: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/24.jpg)
EVGM08 24
30MB/s
800KB/s
10MB/s
Cache Enabled
Pre Cached
Cache DisabledWAN Traffic over VPNRTT 20ms
A cache server minimizesVPN traffic for reinstallation.
![Page 25: EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,](https://reader036.vdocument.in/reader036/viewer/2022062713/56649f425503460f94c61c54/html5/thumbnails/25.jpg)
EVGM08 25
Conclusion
Multi-site virtual cluster Virtualization for Grid computing Isolate resources Give independent administrative domains Easy-to-use UI
Future work Amazon EC2 support Live Migration (demo sc08@Authtin)