hylog: a high performance approach to managing disk layout wenguang wang yanping zhao rick bunt...
TRANSCRIPT
![Page 1: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/1.jpg)
HyLog: A High Performance Approach to Managing Disk
Layout
Wenguang Wang Yanping Zhao Rick Bunt
Department of Computer ScienceUniversity of Saskatchewan
Saskatoon, Canada
USENIX FAST 2004April 1, 2004 Wenguang Wang
![Page 2: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/2.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 2
Background
• The write performance of a storage system is impacted by– the disk characteristics
• Disk positioning time• Transfer bandwidth
– the strategy for writing• Overwrite• LFS (Log-structured File System)
![Page 3: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/3.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 3
Overwrite
• Idea: new data are overwritten on top of old data
• Problems: lots of time lost in disk arm positioning in workloads with small writes scattered over the disk
![Page 4: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/4.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 4
LFS
• Idea: new data are accumulated and written to new disk locations in large sequential transfers
• Assumptions of the disk characteristics: large sequential transfers are more efficient than small block transfers
![Page 5: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/5.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 5
LFS (cont.)
• Advantages– good write performance– no small write penalty on RAID-5– fast recovery– easy to support snapshot and versioning
(WAFL)
• Problems: segment cleaning is expensive– For a year 1991 disk, TPC-B workload, and 50%
disk space utilization, cleaning overhead reduces overall system throughput by 33% (Seltzer et al. USENIX’95)
![Page 6: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/6.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 6
Motivation
• Observation: disk sequential transfer bandwidth has improved 10x more than positioning time
• Question: how are Overwrite and LFS affected by this trend?
DEC RZ26(year 1991)
Cheetah X15 36LP
(year 2003)Diff.
Positioning time
15ms 5.6ms 2.7x
Transfer B/w 2.3MB/s 61MB/s 27x
![Page 7: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/7.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 7
Objective
• Revisit the performance of LFS under modern and future disks
• Evaluate the performance of LFS under disk arrays and concurrent users
• Attempt to perform better than LFS and Overwrite
![Page 8: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/8.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 8
Outline
• Background, Motivation, and Objective
• The analysis of LFS and Overwrite• The design of HyLog• Experimental methodology and
results• Conclusions and future work
![Page 9: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/9.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 9
Experimental Parameters
• Three SCSI disks– DEC RZ26 (year 1991)– Quantum atlas10k (year 1999)– Cheetah X15 36LP (year 2003)
• Page size: 8KB• Workload: uniformly distributed
random update (TPC-B)
![Page 10: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/10.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 10
• In Overwrite:
• In LFS:
• T1 > T2
• Segment I/O Efficiency = T1 / T2
Modeling Write Performance
Time to write N pages is T1
Time to write a segment containing N pages is T2
![Page 11: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/11.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 11
A Simple Scenario
• Assume the segments to be cleaned are always 80% utilized (cleaning space utilization = 80%)
• LFS requires 5 seg. reads and 4 seg. writes to reclaim a free segment
• LFS requires 10 seg. I/Os (9 seg. for cleaning, 1 seg. for new data) to write a segment
• If Segment I/O Efficiency > 10, LFS is still faster than Overwrite!
![Page 12: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/12.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 12
Segment I/O Efficiency
0
5
10
15
20
25
30
35
40
45
0 500 1000 1500 2000 2500 3000 3500 4000
Seg
men
t I/O
Eff
icie
ncy
Segment Size (KB)
Cheetah X15 36LP (year 2003)
Quantum atlas10k, (year 1999)
DEC RZ26, (year 1991)
33
![Page 13: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/13.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 13
0
5
10
15
20
25
30
35
0 0.2 0.4 0.6 0.8 1
Wri
te C
ost
Cleaning Space Utilization
Overwrite vs. LFS
Overwrite
LFScleaning
LFS hole-plugging
1999 disk
0.88
![Page 14: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/14.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 14
Overwrite vs. LFS
• The crossing point where LFS has the same performance as Overwrite
Year of DiskCleaning
Space Utilization
Disk Space Utilization
1991 0.52 0.74
1999 0.88 0.94
2003 0.94 0.97
![Page 15: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/15.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 15
Disk Access Characteristics
• In most workloads, most writes are to a small number of pages (hot pages)
• Impact of skewness on LFS performance– Most of the cleaning cost comes from
cold pages– Most of the good write performance
comes from accumulating the writes to hot pages
![Page 16: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/16.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 16
HyLog (Hybrid Log-structured
Approach)• Separates the disk into two
partitions: hot partition and cold partition
• Uses log-structured approach to manage the hot partition
• Uses overwrite to manage the cold partition
![Page 17: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/17.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 17
0
5
10
15
20
25
30
35
0 0.2 0.4 0.6 0.8 1
Proportion of Hot Pages
Disk Space Utilization 97%
0
5
10
15
20
25
30
35
0 0.2 0.4 0.6 0.8 1
Proportion of Hot Pages
Disk Space Utilization 90%
Performance Potential of HyLog
Wri
te
Co
st
Disk: year 2003, workload: 80% references are in 20% pages
Overwrite
LFS
HyLog
Overwrite
LFS
HyLog
![Page 18: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/18.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 18
Design of HyLog• Key design issue: page separating
algorithm– Collects page write frequencies – Finds the hot page proportion to minimize
expected write cost– Determines the threshold of write
frequency from the desired hot page proportion
– Uses the threshold to distinguish hot pages from cold pages
![Page 19: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/19.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 19
Evaluation Methodology
• Trace driven simulation– Overwrite, LFS, WOLF, and HyLog are
implemented– TPC-C, Email, and OLTP traces– Year 1999, 2003, and 2008 disk models – No think time between requests
• Metrics– Throughput: # I/O requests finished per
second
![Page 20: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/20.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 20
HyLog adjusts the hot page proportion between 35-45%
Results – HyLog Page Separating Algorithm
0
2
4
6
8
10
12
14
0 0.2 0.4 0.6 0.8 1
Thro
ughp
ut (x
1000
)
Hot Page Proportion
Fixed Hot Page ProportionHyLog
TPC-C trace with 20 users and 4 disks, 98% disk space utilization
![Page 21: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/21.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 21
Results – Disk Space Utilization
TPC-C trace with 20 users and 4 disks
0
10
20
30
40
50
60
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
Thro
ughp
ut (x
1000
)
Disk Space Utilization
Overwrite (atlas10k - year'99 disk)HyLog (atlas10k - year'99 disk)
LFS (atlas10k - year'99 disk)
1999 disk
![Page 22: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/22.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 22
Results – Disk Type
TPC-C trace with 20 users and 4 disks
0
10
20
30
40
50
60
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
Thro
ughp
ut (x
1000
)
Disk Space Utilization
Overwrite (atlas10k - year'99 disk)HyLog (atlas10k - year'99 disk)
LFS (atlas10k - year'99 disk)
Overwrite (year’03 disk)
LFS/HyLog (year’03 disk)
1999 disk
2003 disk
![Page 23: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/23.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 23
Results – Disk Type (cont.)
TPC-C trace with 20 users and 4 disks
0
10
20
30
40
50
60
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
Thro
ughp
ut (x
1000
)
Disk Space Utilization
Overwrite (atlas10k - year'99 disk)HyLog (atlas10k - year'99 disk)
LFS (atlas10k - year'99 disk)
Overwrite (year’03 disk)
LFS/HyLog (year’03 disk)
Overwrite (year’08 disk)
LFS/HyLog (year’08 disk)
1999 disk
2003 disk
2008 disk
![Page 24: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/24.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 24
Results – Number of Users
TPC-C trace, disk space utilization 98%, year 1999 disk
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
5 10 15 20 25 30
Nor
mal
ized
Thr
ough
put
Number of Users
HyLog (8 disks)LFS (8 disks)
![Page 25: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/25.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 25
Results – Number of Disks
TPC-C trace, disk space utilization 98%, year 1999 disk
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2 4 6 8 10 12 14
Nor
mal
ized
Thr
ough
put
Number of Disks
HyLog (7 users)LFS (7 users)
![Page 26: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/26.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 26
Results – RAID-5
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
Overwrite LFS WOLF HyLog Overwrite LFS WOLF HyLog
Thr
oug
hp
ut
RAID-0RAID-5
Disk Space Utilization 98%Disk Space Utilization 80%
TPC-C trace, year 1999 disk, 8-disk RAID-0, 9-disk RAID-5
![Page 27: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/27.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 27
Results – Other Traces
0
1
2
3
4
5
6
Overwrite LFS WOLF HyLog Overwrite LFS WOLF HyLog
Nor
mal
ized
Thr
ough
put
Disk Space Utilization=90%
Disk Space Utilization=98%
OLTP Email
Year 1999 disk
![Page 28: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/28.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 28
Conclusions
• On modern and future disks, LFS significantly outperforms Overwrite unless the disk space utilization is very high
• HyLog performs comparably to the best of Overwrite, LFS, and WOLF
![Page 29: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/29.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 29
Future Work
• Add fast recovery support in HyLog– All meta-data are considered as hot
pages
• Stabilize NetBSD LFS implementation and measure its performance
• Implement and evaluate HyLog in NetBSD
![Page 30: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/30.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 30
![Page 31: HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt Department of Computer Science University of Saskatchewan](https://reader035.vdocument.in/reader035/viewer/2022062619/5518a12d550346991f8b48ac/html5/thumbnails/31.jpg)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 31
Nor
mal
ized
Thr
ough
put
5 10 15 20 25 30
2 4
6 8
10 12
14
0.4 0.6 0.8
1 1.2 1.4 1.6 1.8
Results – # Users and Disks
TPC-C trace, disk space utilization 98%, year 1999 disk
HyLogLFS
Number of Users
Number of Disks