04/05/2004 CSCI 315 Operating Systems Design 1

File System Implementation

04/05/2004 CSCI 315 Operating Systems Design 2

Acyclic-Graph DirectoriesHave shared subdirectories and files.

links: soft (symbolic)

hard

Unix: ln (read man page);

need to keep a reference count on each file or directory.

04/05/2004 CSCI 315 Operating Systems Design 3

Acyclic-Graph Directories (Cont.)

• Different names (aliasing) for the same file or directory.

• If dict deletes list dangling pointer.Solutions:– Backpointers, so we can delete all pointers.

Variable size records a problem.– Backpointers using a daisy chain

organization.– Entry-hold-count solution.

04/05/2004 CSCI 315 Operating Systems Design 4

General Graph Directory

04/05/2004 CSCI 315 Operating Systems Design 5

General Graph Directory (Cont.)

• How do we guarantee no cycles?– Allow only links to file not subdirectories.– Garbage collection.– Every time a new link is added use a cycle

detection algorithm to determine whether it is OK.

04/05/2004 CSCI 315 Operating Systems Design 6

File System Mounting• A file system (partition) must be mounted before it can be

accessed. Mounting allows one to attach the file system on one device to the file system on another device.

• A unmounted file system needs to be attached to a mount point before it can be accessed.

existing unmounted

04/05/2004 CSCI 315 Operating Systems Design 7

File Sharing

• Sharing of files on multi-user systems is desirable.

• Sharing may be done through a protection scheme.

• On distributed systems, files may be shared across a network.

• Network File System (NFS) is a common distributed file-sharing method.

04/05/2004 CSCI 315 Operating Systems Design 8

Protection• File owner/creator should be able to control:

– what can be done,– by whom.

• Types of access:– Read,– Write,– Execute,– Append,– Delete,– List.

Discretionary Access Control (DAC)

04/05/2004 CSCI 315 Operating Systems Design 9

Protection• Mandatory Access Control (MAC):

– System policy: files tied to access levels = (public, restricted, confidential, classified, top-secret).

– Process also has access level: can read from and write to all files at same level, can only read from files below, can only write to files above.

• Role-Based Access Control (RBAC):– System policy: defines “roles” (generalization of the Unix idea

of groups). – Roles are associated with access rules to sets of files and

devices. – A process can change roles (in a pre-defined set of possibilities)

during execution.

04/05/2004 CSCI 315 Operating Systems Design 10

Access Lists and Groups• Mode of access: read, write, execute• Three classes of users

RWXa) owner access 7 1 1 1

RWXb) group access 6 1 1 0

RWXc) public access 1 0 0 1

• Ask manager to create a group (unique name), say G, and add some users to the group.

• For a particular file (say game) or subdirectory, define an appropriate access.

owner group public

chmod 761 game

Associate a group with a file: chgrp G game

04/05/2004 CSCI 315 Operating Systems Design 11

File-System Structure

• File structure:– Logical storage unit,– Collection of related information.

• File system resides on secondary storage (disks).

• File system is organized into layers.

• File control block – storage structure consisting of information about a file.

04/05/2004 CSCI 315 Operating Systems Design 12

Layered File System

04/05/2004 CSCI 315 Operating Systems Design 13

A Typical File Control Block

04/05/2004 CSCI 315 Operating Systems Design 14

In-Memory File System Structures

file open

file read

04/05/2004 CSCI 315 Operating Systems Design 15

Virtual File Systems

• Virtual File Systems (VFS) provide an object-oriented way of implementing file systems.

• VFS allows the same system call interface (the API) to be used for different types of file systems.

• The API is to the VFS interface, rather than any specific type of file system.

04/05/2004 CSCI 315 Operating Systems Design 16

Schematic View of Virtual File System

ext3 FAT 32 NFS

same API forall file systemtypes

04/05/2004 CSCI 315 Operating Systems Design 17

Directory Implementation

• Linear list of file names with pointer to the data blocks:– simple to program, but…– time-consuming to execute.

• Hash Table:– decreases directory search time,– collisions – situations where two file names hash to

the same location,– fixed size.

The directory is a symbol table that maps file names to pointers that lead to the blocks comprising a file.

04/05/2004 CSCI 315 Operating Systems Design 18

Allocation Methods

An allocation method refers to how disk blocks are allocated for files:

• Contiguous allocation,

• Linked allocation,

• Indexed allocation.

04/05/2004 CSCI 315 Operating Systems Design 19

Contiguous Allocation• Each file occupies a set of contiguous blocks on

the disk.

• Simple – only starting location (block #) and length (number of blocks) are required.

• Random access.

• Wasteful of space (dynamic storage-allocation problem).

• Files cannot grow.

04/05/2004 CSCI 315 Operating Systems Design 20

Contiguous Allocation

• Mapping from logical to physical.

LA/512

Q

R

– Block to be accessed = ! + starting address– Displacement into block = R

04/05/2004 CSCI 315 Operating Systems Design 21

Contiguous Allocation of Disk Space

04/05/2004 CSCI 315 Operating Systems Design 22

Extent-Based Systems• Many newer file systems (i.e. Veritas File

System) use a modified contiguous allocation scheme.

• Extent-based file systems allocate disk blocks in extents.

• An extent is a contiguous set of block. Extents are allocated for each file. A file consists of one or more extents.

04/05/2004 CSCI 315 Operating Systems Design 23

Linked Allocation

Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk.

pointerblock =

04/05/2004 CSCI 315 Operating Systems Design 24

Linked Allocation (Cont.)• Simple – need only starting address• Free-space management system – no waste of space • No random access• Mapping

Block to be accessed is the Qth block in the linked chain of blocks representing the file.Displacement into block = R + 1

File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2.

LA/511Q

R

04/05/2004 CSCI 315 Operating Systems Design 25

Linked Allocation

04/05/2004 CSCI 315 Operating Systems Design 26

File-Allocation Table

04/05/2004 CSCI 315 Operating Systems Design 27

Indexed Allocation

• Brings all pointers together into the index block.• Logical view.

index table

04/05/2004 CSCI 315 Operating Systems Design 28

Example of Indexed Allocation

04/05/2004 CSCI 315 Operating Systems Design 29

Indexed Allocation (Cont.)• Need index table• Random access• Dynamic access without external fragmentation, but

have overhead of index block.• Mapping from logical to physical in a file of maximum

size of 256K words and block size of 512 words. We need only 1 block for index table.

LA/512Q

R

Q = displacement into index tableR = displacement into block

04/05/2004 CSCI 315 Operating Systems Design 30

Indexed Allocation – Mapping (Cont.)

• Mapping from logical to physical in a file of unbounded length (block size of 512 words).

• Linked scheme – Link blocks of index table (no limit on size).

LA / (512 x 511)Q1

R1

Q1 = block of index tableR1 is used as follows:

R1 / 512Q2

R2

Q2 = displacement into block of index tableR2 displacement into block of file:

04/05/2004 CSCI 315 Operating Systems Design 31

Indexed Allocation – Mapping (Cont.)

• Two-level index (maximum file size is 5123)

LA / (512 x 512)Q1

R1

Q1 = displacement into outer-indexR1 is used as follows:

R1 / 512Q2

R2

Q2 = displacement into block of index tableR2 displacement into block of file:

04/05/2004 CSCI 315 Operating Systems Design 32

Indexed Allocation – Mapping (Cont.)

outer-index

index table file

04/05/2004 CSCI 315 Operating Systems Design 33

Combined Scheme: UNIX (4K bytes per block)

04/05/2004 CSCI 315 Operating Systems Design 34

Free-Space Management

• Bit vector (n blocks)

…

0 1 2 n-1

bit[i] = 0 block[i] free

1 block[i] occupied

Block number calculation

(number of bits per word) *(number of 0-value words) +offset of first 1 bit

04/05/2004 CSCI 315 Operating Systems Design 35

Free-Space Management (Cont.)

• Bit map requires extra space. Example:

block size = 212 bytes

disk size = 230 bytes (1 gigabyte)

n = 230/212 = 218 bits (or 32K bytes)• Easy to get contiguous files • Linked list (free list)

– Cannot get contiguous space easily– No waste of space

• Grouping • Counting

04/05/2004 CSCI 315 Operating Systems Design 36

Free-Space Management (Cont.)

• Need to protect:– Pointer to free list– Bit map

• Must be kept on disk• Copy in memory and disk may differ.• Cannot allow for block[i] to have a situation where bit[i]

= 1 in memory and bit[i] = 0 on disk.

– Solution:• Set bit[i] = 1 in disk.• Allocate block[i]• Set bit[i] = 1 in memory

04/05/2004 CSCI 315 Operating Systems Design 37

Linked Free Space List on Disk

04/05/2004 CSCI 315 Operating Systems Design 38

Efficiency and Performance• Efficiency dependent on:

– disk allocation and directory algorithms– types of data kept in file’s directory entry

• Performance– disk cache – separate section of main memory for

frequently used blocks– free-behind and read-ahead – techniques to optimize

sequential access– improve PC performance by dedicating section of

memory as virtual disk, or RAM disk.

04/05/2004 CSCI 315 Operating Systems Design 39

Various Disk-Caching Locations

04/05/2004 CSCI 315 Operating Systems Design 40

Page Cache

• A page cache caches pages rather than disk blocks using virtual memory techniques.

• Memory-mapped I/O uses a page cache.

• Routine I/O through the file system uses the buffer (disk) cache.

• This leads to the following figure.

04/05/2004 CSCI 315 Operating Systems Design 41

I/O Without a Unified Buffer Cache

04/05/2004 CSCI 315 Operating Systems Design 42

Unified Buffer Cache

• A unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O.

04/05/2004 CSCI 315 Operating Systems Design 43

I/O Using a Unified Buffer Cache

04/05/2004 CSCI 315 Operating Systems Design 44

Recovery

• Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies.

• Use system programs to back up data from disk to another storage device (floppy disk, magnetic tape).

• Recover lost file or disk by restoring data from backup.

04/05/2004 CSCI 315 Operating Systems Design 45

Log Structured File Systems• Log structured (or journaling) file systems record each update to

the file system as a transaction.

• All transactions are written to a log. A transaction is considered committed once it is written to the log. However, the file system may not yet be updated.

• The transactions in the log are asynchronously written to the file system. When the file system is modified, the transaction is removed from the log.

• If the file system crashes, all remaining transactions in the log must still be performed.