journaling versus soft updates: asynchronous meta-data protection in file systems
DESCRIPTION
JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS. Margo I. Seltzer, Harvard Gregory R. Ganger, CMU M. Kirk McKusick Keith A. Smith, Harvard Craig A. N. Soules, CMU Christopher A. Stein, Harvard. INTRODUCTION. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/1.jpg)
JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA
PROTECTION IN FILE SYSTEMS
Margo I. Seltzer, Harvard Gregory R. Ganger, CMUM. Kirk McKusickKeith A. Smith, HarvardCraig A. N. Soules, CMUChristopher A. Stein, Harvard
![Page 2: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/2.jpg)
INTRODUCTION
• Paper discusses two most popular approaches for improving the performance of metadata operations and recovery:– Journaling – Soft Updates
• Journaling systems record metadata operations on an auxiliary log (Hagmann)
• Soft Updates uses ordered writes(Ganger & Patt)
![Page 3: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/3.jpg)
Metadata Operations
• Metadata operations modify the structure of the file system– Creating, deleting, or renaming
files, directories, or special files• Data must be written to disk in such a way that
the file system can be recovered to a consistent state after a system crash
![Page 4: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/4.jpg)
Metadata Integrity
• FFS uses synchronous writes to guarantee the integrity of metadata– Any operation modifying multiple pieces of
metadata will write its data to disk in a specific order
– These writes will be blocking• Guarantees integrity and durability of metadata
updates
![Page 5: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/5.jpg)
Deleting a file (I)
abc
def
ghi
i-node-1
i-node-2
i-node-3
Assume we want to delete file “def”
![Page 6: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/6.jpg)
Deleting a file (II)
abc
def
ghi
i-node-1
i-node-3
Cannot delete i-node before directory entry “def”
?
![Page 7: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/7.jpg)
Deleting a file (III)
• Correct sequence is1. Write to disk directory block containing
deleted directory entry “def”2. Write to disk i-node block containing
deleted i-node• Leaves the file system in a consistent state
![Page 8: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/8.jpg)
Creating a file (I)
abc
ghi
i-node-1
i-node-3
Assume we want to create new file “tuv”
![Page 9: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/9.jpg)
Creating a file (II)
abc
ghi
tuv
i-node-1
i-node-3
Cannot write directory entry “tuv” before i-node
?
![Page 10: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/10.jpg)
Creating a file (III)
• Correct sequence is1. Write to disk i-node block containing new
i-node2. Write to disk directory block containing
new directory entry• Leaves the file system in a consistent state
![Page 11: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/11.jpg)
Synchronous Updates
• Used by FFS to guarantee consistency of metadata:– All metadata updates are done through
blocking writes
• Increases the cost of metadata updates• Can significantly impact the performance of
whole file system
![Page 12: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/12.jpg)
SOFT UPDATES
• Use delayed writes (write back)• Maintain dependency information about
cached pieces of metadata:This i-node block must be updated before/after this directory entry
• Guarantee that metadata blocks are written to disk in the required order
![Page 13: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/13.jpg)
First Problem
• Synchronous writes guaranteed that metadata operations were durable once the system call returned
• Soft Updates guarantee that file system will recover into a consistent state but not necessarily the most recent one– Some updates could be lost
![Page 14: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/14.jpg)
Second Problem
• Cyclical dependencies:– Same directory block contains entries to be
created and entries to be deleted– These entries point to i-nodes in the same
block
![Page 15: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/15.jpg)
Example (I)
We want to delete file “def” and create new file “xyz”
i-node-2 def
--- ----------
NEW xyz
NEW i-node-3
Block A Block B
![Page 16: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/16.jpg)
Example (II)
• Cannot write block A before block B:– Block A contains a new directory entry
pointing to block B• Cannot write block B before block A:
– Block A contains a deleted directory entry pointing to block B
![Page 17: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/17.jpg)
The Solution (I)
• Roll back metadata in one of the blocks to an earlier, safe state
(Safe state does not contain new directory entry)
def
--- Block A’
![Page 18: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/18.jpg)
The Solution (II)
• Write first block with metadata that were rolled back (block A’ of example)
• Write blocks that can be written after first block has been written (block B of example)
• Roll forward block that was rolled back• Write that block• Breaks the cyclical dependency but must now
write twice block A
![Page 19: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/19.jpg)
def
First, block A’
The solution (III)
---
----------
NEW i-node-3
Then, block B
NEW xyz
def
Last, block A
![Page 20: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/20.jpg)
JOURNALING (I)
• Journaling systems maintain an auxiliary log that records all meta-data operations
• Write-ahead logging ensures that the log is written to disk before any blocks containing data modified by the corresponding operations.– After a crash, can replay the log to bring the
file system to a consistent state
![Page 21: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/21.jpg)
JOURNALING (II)
• Log writes are performed in addition to the regular writes
• Journaling systems incur log write overhead but– Log writes can be performed efficiently
because they are sequential– Metadata blocks do not need to be written
back after each update
![Page 22: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/22.jpg)
JOURNALING (III)
• Journaling systems can provide– same durability semantics as FFS if log is forced
to disk after each meta-data operation– the laxer semantics of Soft Updates if log writes
are buffered until entire buffers are full• Will discuss two implementations
– LFS-File– LFS-wafs
![Page 23: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/23.jpg)
LFS-File (I)
• Maintains a circular log in a pre-allocated file in the FFS (about 1% of file system size)
• Buffer manager uses a write-ahead logging protocol to ensure proper synchronization between regular file data and the log
![Page 24: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/24.jpg)
LFS-File (II)
• Buffer header of each modified block in cache identifies the first and last log entries describing an update to the block
• System uses – First item to decide which log entries can be
purged from log– Second item to ensure that all relevant log entries
are written to disk before the block is flushed from the cache
![Page 25: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/25.jpg)
Example
17
Log entries
Recycled
18
Written to Disk
19 20 22 23 24 25
Not YetWritten
Buffer blockFirst = 19Last = 23
Log entries 19 and above cannot be recycled until block is saved
Block cannot be saved until all log entries up to 23 are saved
4KB ofData
![Page 26: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/26.jpg)
LFS-File (III)
• LFFS-file maintains its log asynchronously– Maintains file system integrity, but does not
guarantee durability of updates
![Page 27: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/27.jpg)
LFS-wafs(I)
• Implements its log in an auxiliary file system:Write Ahead File System (WAFS)– Can be mounted and unmounted– Can append data– Can return data by sequential or keyed reads
• Keys for keyed reads are log-sequence-numbers (LSNs) that correspond to logical offsets in the log
![Page 28: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/28.jpg)
LFS-wafs(II)
• Log is implemented as a circular buffer within the physical space allocated to the file system.
• Buffer header of each modified block in cache contains LSNs of first and last log entries describing an update to the block
• LFFS-wafs uses the same checkpointing scheme and the same write-ahead logging protocol as LFFS-file
![Page 29: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/29.jpg)
LFS-wafs(III)
• Major advantage of WAFS is additional flexibility:– Can put WAFS on separate disk drive to avoid
I/O contention– Can even put it in NVRAM
• LFS-wafs normally uses synchronous writes – Metadata operations are persistent upon return
from the system call– Same durability semantics as FFS
![Page 30: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/30.jpg)
LFFS Recovery
• Superblock has address of last checkpoint– LFFS-file has frequent checkpoints– LFFS-wafs much less frequent checkpoints
• First recover the log• Read then the log from logical end (backward pass)
and undo all aborted operations• Do forward pass and reapply all updates that have
not yet been written to disk
![Page 31: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/31.jpg)
OTHER APPROACHES (I)
• Using non-volatile cache (Network Appliances) – Ultimate solution: can keep data in cache forever– Additional cost of NVRAM
• Simulating NVRAM with– Uninterruptible power supplies – Hardware-protected RAM (Rio): cache is
marked read-only most of the time
![Page 32: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/32.jpg)
OTHER APPROACHES (II)
• Log-structured file systems– Not always possible to write all related meta-
data in a single disk transfer– Sprite-LFS adds small log entries to the
beginning of segments– BSD-LFS make segments temporary until all
metadata necessary to ensure the recoverability of the file system are on disk.
![Page 33: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/33.jpg)
SYSTEM COMPARISON• Compared performances of
– Standard FFS– FFS mounted with the async option– FFS mounted with Soft Updates– FFS augmented with a file log using
asynchronous log writes– FFS augmented with a WAFS log using
• Synchronous /asynchronous log writes • WAFS log on same/different drive
![Page 34: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/34.jpg)
Microbenchmark
Best
Worst
???: Can explain it
![Page 35: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/35.jpg)
Comments
• FFS-async performs best• Original FFS, LFFS-wafs-2sync and
LFFS-wafs-1sync perform worst– Synchronous log updates are costly
• LFFS-file outperforms LFFS-wafs-2async and LFFS-wafs-1async– LFFS-file uses bigger block clusters
for log writes
![Page 36: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/36.jpg)
Ssh benchmark
![Page 37: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/37.jpg)
Netnews benchmark
![Page 38: JOURNALING VERSUS SOFT UPDATES: ASYNCHRONOUS META-DATA PROTECTION IN FILE SYSTEMS](https://reader036.vdocument.in/reader036/viewer/2022062518/56814547550346895db21589/html5/thumbnails/38.jpg)
CONCLUSIONS
• Journaling alone is not sufficient to “solve” the meta-data update problem– Cannot realize its full potential when
synchronous semantics are required• When that condition is relaxed, journaling and
Soft Updates perform comparably in most cases