Slide 1

Conquest-2: Improving Energy Efficiency and Performance Through a Disk/RAM Hybrid File System An-I Andy Wang Florida State University (NSF CCR-0098363, CNS-0410896) Slide 2 2 Conquest-2 Team Members FSU An-I Andy Wang (PI), Charles Weddle, Cory Fox, Jin Qian, Dragan Lojpur, Mark Carpenter, Ryan Fishel UCLA Peter Reiher (Co-PI), Erik Kline Harvey Mudd College Geoff Kuenning Former members: Mathew Oldham, Noriel Lu, RuGang Xu Slide 3 3 Motivation Computers are becoming cheaper Energy is not Energy consumption by storage devices 8% for laptops 24% for Web servers 77% for proxy servers 27% of the operating costs for data centers Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 4 4 Laptops: 8% 20 min of battery life Proxy server: Higher energy cost higher cooling cost lower density of servers more space cost Disk Energy Consumption Disk % of system power 5-yr cost of disk power Mobile Intel Pentium III laptop8%$5 Pentium 4 web server24%$120 Pentium 4 web proxy server77%$1,300 Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 5 5 Performance vs. Energy Benefits Performance More relevant during peak loads Energy savings Realized instantaneously Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 6 6 Roadmap Conquest Existing energy-saving approaches Emergence of memory-rich storage era Conquest-2 Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 7 7 Conquest A disk/persistent-RAM hybrid file system Deliver all file system services from memory, with the exception of high-capacity storage Two separate and specialized data paths Benefits: Simplicity Performance Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 8 8 Hardware Evolution 19902000 1 KHz 1 MHz 1 GHz CPU (50% /yr) Memory (50% /yr) Disk (15% /yr) Accesses Per Second (Log Scale) 10 5 10 6 1995 (1 sec : 6 days)(1 sec : 3 months) Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 9 [Caceres et al., 1993; Hillyer et al., 1996; Qualstar 1998; Tanisys 1999; Micron Semiconductor Products 2000; Quantum 2000]9 Storage Media Alternatives accesses/sec (log) $/MB (log) 10 0 10 3 persistent RAM Magnetic RAM? (write once) flash memory disk tape battery-backed DRAM 10 -3 10 6 Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 10 [Grochowski 2000]10 Price Trend of Persistent RAM 19952005 10 0 Year $/MB (log) 2000 10 -2 10 -1 10 1 10 2 paper/film 3.5 HDD 2.5 HDD 1 HDD Persistent RAM Booming of digital photography 4 to 10 GB of persistent RAM Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 11 [Iram 1993; Douceur et al., 1999; Roselli et al., 2000]11 User Access Patterns Small files Take little space (10%) Represent most accesses (90%) Large files Take most space Mostly sequential accesses Except database applications Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 12 12 Files Stored in Persistent RAM Small files (< 1MB) No seek time or rotational delays Fast byte-level accesses Metadata Fast synchronous update No dual representations Executables and shared libraries In-place execution Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 13 [Devlinux.com 2000]13 Large-File-Only Disk Storage Allocate in big chunks No fragmentation management No tricks for small files Storing data in metadata Wrapping a balanced tree onto disk cylinders Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 14 [Katcher 1997; Sweeney et al., 1996; Card et al., 1999; Namesys 2002]14 Conquest is comparable to ramfs At least 24% faster than the LRU disk cache ISP workload (emails, web-based transactions) PostMark Benchmark 40 to 250 MB working set with 2 GB physical RAM Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 15 15 When working set > RAM, Conquest is 1.4 to 2 times faster than ext2fs, reiserfs, and SGI XFS PostMark Benchmark 10,000 files, 3.5 GB working set with 2 GB physical RAM Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 16 16 Conquest-2 Conquest has made advancements in terms of better performance Can we extend Conquest to improve performance and reduce energy consumption at the same time? Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 17 17 Conquest-Based Numbers A UCLA Webserver Single disk File size threshold of 32KB Spin down whenever the disk idle time > 10s Conquest: 84% energy savings LRU: 64% energy savings Greater benefits for multiple disks Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 18 18 Existing Approaches Provide degraded service Reduced disk rotation speed Speculative methods Predicting idle periods for shutting down a disk Not suitable for servers High loads Uniform data striping among disks Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 19 19 Cannot simply replace server drives with laptop ones Just Use Laptop Drives? Typical server driveTypical laptop drive Power consumption active13 W2 W Performance average latency4 ms7 ms sustained transfer rate30 60 MB/s35 MB/s spin-up time10 s1.6 s Cost $/GB$1/GB$4/GB Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 20 20 RAM performance/energy savings and disk capacity? Persistent RAM Storage? Typical server driveTypical RAM Power consumption active12.5 W735mW Performance average latency4.2 ms1.4 2.8 ns sustained transfer rate32 58 MB/s240 MB/s spin-up time10 s72 200 ns Cost $/GB$1.2/GB$154/GB Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 21 21 Why not Conventional Caching? High overhead to access data stored in RAM storage 90% cache hit rate 90% disk idle time 10% of cache misses can keep a drive spinning all the time e.g., multimedia workloads Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 22 22 What if you have multiple disks? Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 23 23 And access patterns are skewed Access patterns Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 24 24 Better Off Caching Cold Disks Spin down cold disks Access patterns Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 25 25 Conquest-2 Approach Strategic use of memory storage Improve performance Energy-aware memory manager Power down unused banks Power-aware RAIDs (PARAIDs) Gear-shift individual drives according to performance demands Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 26 26 New Roles of Memory Shaping the frequency, timing, and predictability of disk accesses Low frequency of disk access Better performance Energy savings Predictability Hide the latency to spin a disk up Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 27 27 File Access Characterizations FrequencyArrival timesPredictabilitySizeLocation.tar.gzLowBulkLowLargeDisk.mpgLowScatteredHighLargeDisk.cHighScatteredLowSmallRAM locate.dbHighBulkLowLargeRAM Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 28 28 Energy-Aware Memory Management indices data frequently used (index, data) infrequently used (index, data) Conceptually simple, but difficult in practice Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 29 29 Linux Memory Manager (1) Page allocator maintains individual pages Page allocator Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 30 30 Linux Memory Manager (2) Zone allocator allocates memory in power-of- two sizes Page allocator Zone allocator Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 31 31 Linux Memory Manager (3) Slab allocator groups allocations by sizes to reduce internal memory fragmentation Page allocator Zone allocator Slab allocator Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 32 32 Linux Memory Manager (4) Difficult to collocate information according to energy constraints Page allocator Slab allocator Zone allocator Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 33 33 Conventional RAID load time drives Slide 34 34 Power-Aware RAID load time drives load time drives Slide 35 35 Challenges Energy Not enough opportunities to spin down RAIDs Performance Essential for peak loads Reliability Server-class drives are not designed for frequent power switching Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 36 36 Power-Aware RAID Observations RAIDs are configured for peak performance Uniform striping keeps all drives spinning for light loads Over-provision of storage capacity Unused storage can be traded for energy savings Cyclic fluctuation of loads Infrequent on-off power transitions can be effective Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 37 37 Cyclic Fluctuation of Loads load time utilization threshold gear 2 gear 1 Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 38 38 Skewed Striping for Energy Saving Use over-provisioned spare storage Can use fewer drives for light loads gear 1 soft-state block replication gear 2 disk 1disk 2disk 3disk 4 gear 3 RAID-5 layout Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 39 39 Preserving Peak Performance Based on RAID-5 All drives on for peak loads Full parallelism Fewer drives on for light loads Lower latency for small files Degraded throughput for large files Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 40 40 Reliability Drives have a limited number of power cycles Form bi-modal distribution of busy/idle drives Motivation Conquest Conquest-2 Power-Aware RAID Conclusion busy disks power cycled disks idle disks role exchange Disk 1 Gear 1 Gear 2 Gear 3 Disk 2Disk 3Disk 4Disk 5Disk 6 Slide 41 41 Reliability Drives have a limited number of power cycles Form bi-modal distribution of busy/idle drives Rotate drives with more power cycles Motivation Conquest Conquest-2 Power-Aware RAID Conclusion % of power cycles 100% 0% 020,000 power cycles gear 1 gear 2 gear 3 Slide 42 42 Reliability Drives have a limited number of power cycles Form bi-modal distribution of busy/idle drives Rotate drives with more power cycles Ration number of power cycles Distributed parity (RAID-5) Tolerate single-disk failures Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 43 43 Other Issues Update propagations Gear-shifting policies Motivation Conquest Conquest-2 Power-Aware RAID Conclusion disk utilization gear 2 utilization threshold time gear shift gear 1 utilization threshold disk utilization utilization threshold time downshift 300s, 60s, 10s moving averages Slide 44 44 Gear-Shifting Policies Ideal In practice time load time load Slide 45 45 Empirical Measurements Servers are not measurement friendly Time consuming Cannot easily apply the trick of skipping idle times Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 46 46 Workload Selection Need to match with the hardware setup energy consumption load linear scaling everything on or off geared switching no choices time Slide 47 47 Experiment Set 1 Workload FSU CS Department Web Server trace A single day trace Hardware Dell 2600 with 5 drives PARAID 2 gears (3-disk RAID-0 and 5-disk RAID-0) No energy-aware memory management Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 48 48 Web Trace Replay 512x 1920x1024x Speed-upPower savings (+stdev) 1920x (290 req/sec)15% (+2.2%) 1024x (144 req/sec)25% (+1.3%) 512x (72 req/sec)34% (+1.1%) Slide 49 49 Experiment Set 2 Workload Cello99 server I/O trace from HP A 50-hr trace Hardware Dell 2600 with 5 drives PARAID 2 gears (3-disk RAID-5 and 5-disk RAID-5) No energy-aware memory management Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 50 50 Cello99 50hr Trace Speed-upPower savings (+stdev) 128x (1024 req/sec)7.8% (+0.21%) 64x (548 req/sec)12% (+2.8%) 32x (274 req/sec)13% (+0.26%) 32x 64x128x Slide 51 51 Experiment Set 3 Workload PostMark benchmark (ISP workload) Hardware Dell 2600 with 5 drives PARAID 2 gears (3-disk RAID-5 and 5-disk RAID-5) No energy-aware memory management Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 52 52 PostMark Slide 53 53 Conquest-2 Current Status PARAID Implementing reliability mechanisms Energy-aware memory manager Integrating the memory and the disk components Empirical measurements Exploring different server loads Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 54 54 Conclusion Energy efficiency and performance can be achieved simultaneously PARAID-0 with 2 gears has already shown a 15% reduction in power with < 1% performance loss Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 55 55 Questions Google keywords Conquest file system Power-Aware RAID Andy Wang FSU Motivation Conquest Conquest-2 Power-Aware RAID Conclusion Slide 56 56 Gear-Shifting Details