achieving power-efficiency in clusters without distributed file system complexity
DESCRIPTION
Achieving Power-Efficiency in Clusters without Distributed File System Complexity. Hrishikesh Amur, Karsten Schwan Georgia Tech. http://img.all2all.net/main.php?g2_itemId=157. Green Computing Research Initiative at GT. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/1.jpg)
Achieving Power-Efficiency in Clusters without Distributed
File System ComplexityHrishikesh Amur, Karsten Schwan
Georgia Tech
![Page 2: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/2.jpg)
Green Computing Research Initiative at GT
Circuit level: DVFS, power states, clock gating (ECE)
Chip and Package: power multiplexing, spatiotemporal migration (SCS, ECE)
Board: VirtualPower, scheduling/scaling/operating system… (SCS, ME, ECE)
Rack: mechanical design, thermal and airflow analysis, VPTokens, OS and management (ME, SCS)
Powe
r dist
ribut
ion
and
deliv
ery
(ECE
)
http://img.all2all.net/main.php?g2_itemId=157
Datacenter and beyond: design, IT management, HVAC control… (ME, SCS, OIT…)
focus of our work:
![Page 3: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/3.jpg)
Data-intensive applications that use distributed storage
Focus
![Page 4: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/4.jpg)
CPUMemoryPCI slotsMotherboardDisksFan
Per-system Power Breakdown
![Page 5: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/5.jpg)
Power off entire nodes
Approach to Power-Efficiency of Cluster
![Page 6: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/6.jpg)
Turning Off Nodes Breaks Conventional DFS
![Page 7: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/7.jpg)
Turning Off Nodes Breaks Conventional DFS
![Page 8: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/8.jpg)
Turning Off Nodes Breaks Conventional DFS
![Page 9: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/9.jpg)
Turning Off Nodes Breaks Conventional DFS
![Page 10: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/10.jpg)
Turning Off Nodes Breaks Conventional DFS
![Page 11: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/11.jpg)
Turning Off Nodes Breaks Conventional DFS
![Page 12: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/12.jpg)
Turning Off Nodes Breaks Conventional DFS
![Page 13: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/13.jpg)
One replica of all data placed on a small set of nodes
Primary replica maintains availability, allowing nodes storing other replicas to be turned off [Sierra, Rabbit]
Modifications to Data Layout Policy
![Page 14: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/14.jpg)
Where is new data to be written when part of the cluster is turned off?
Handling New Data
![Page 15: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/15.jpg)
New Data: Temporary Offloading
![Page 16: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/16.jpg)
Temporary off-loading to ‘on’ nodes is a solution
Cost of additional copying of lots of data
Usage of network bandwidth
Increased complexity!!
New Data: Temporary Offloading
![Page 17: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/17.jpg)
Failure of primary nodes cause a large number of nodes to be started up to restore availability
To solve this, additional groups with secondary, tertiary etc. copies have to be made.
Again, increased complexity!!
Handling Primary Failures
![Page 18: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/18.jpg)
Making a DFS power-proportional increases its complexity significantly
![Page 19: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/19.jpg)
Provide fine-grained control over what components to turn off
Our Solution
![Page 20: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/20.jpg)
Switch between two extreme power modes: max_perf and io_server
How do we save power?
![Page 21: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/21.jpg)
Fine-grained control allows all disks to be kept on maintaining access to stored data
How does this keep the DFS simple?
![Page 22: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/22.jpg)
Prototype Node Architecture
SATA Switch
Asterix Node
Obelix Node
![Page 23: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/23.jpg)
Prototype Node Architecture
SATA Switch
Asterix Node
Obelix Node
VMM
![Page 24: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/24.jpg)
max_perf Mode
SATA Switch
Asterix Node
Obelix Node
VM
![Page 25: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/25.jpg)
io_server Mode
SATA Switch
Asterix Node
Obelix Node
VM
![Page 26: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/26.jpg)
1 2 3 40
102030405060708090
ObelixAsterix-II
Servers in max_perf mode
Thro
ughp
ut/W
att
(MB/
s/W
)
Increased Performance/Power
![Page 27: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/27.jpg)
1 2 3 40
102030405060708090
ObelixAsterix-II
Servers in max_perf mode
Thro
ughp
ut/W
att
(MB/
s/W
)
Increased Performance/Power
![Page 28: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/28.jpg)
1 2 3 40
102030405060708090
ObelixAsterix-II
Servers in max_perf mode
Thro
ughp
ut/W
att
(MB/
s/W
)
Increased Performance/Power
![Page 29: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/29.jpg)
1 2 3 40
102030405060708090
ObelixAsterix-II
Servers in max_perf mode
Thro
ughp
ut/W
att
(MB/
s/W
)
Increased Performance/Power
![Page 30: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/30.jpg)
Obelix Asterix0
102030405060708090
LinuxdomUdom0domU*
Thro
ughp
ut
(MB/
s)
Virtualization Overhead: Reads
![Page 31: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/31.jpg)
Obelix Asterix0
1020304050607080
LinuxdomUdom0domU*
Thro
ughp
ut
(MB/
s)
Virtualization Overhead: Writes
![Page 32: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/32.jpg)
Turning entire nodes off complicates DFS
Good to be able to turn components off, or achieve more power-proportional platforms/components
Prototype uses separate machines and shared disks
Summary
![Page 33: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/33.jpg)
Load Management Policies Static
◦ e.g., DFS, DMS, monitoring/management tasks… Dynamic
◦ e.g., based on runtime monitoring and management/scheduling…
◦ helpful to do power metering on per process/VM basis
X86+Atom+IB…
![Page 34: Achieving Power-Efficiency in Clusters without Distributed File System Complexity](https://reader036.vdocument.in/reader036/viewer/2022081604/56816658550346895dd9d884/html5/thumbnails/34.jpg)
VM-level Power Metering: Our Approach
Built power profiles for various platform resources◦ CPU, memory, cache, I/O…
Utilize low-level hardware counters to track resource utilization on per VM basis◦ xenoprofile, IPMI, Xen tools…◦ track sets of VMs separately
Maintain low/acceptable overheads while maintaining desired accuracy◦ limit amount of necessary information, number of monitored
events: use instructions retired/s and LLC misses/s only
◦ establish accuracy bounds
Apply monitored information to power model to determine VM power utilization at runtime◦ in contrast to static purely profile-based approaches