linux virtual file system
DESCRIPTION
Linux Virtual File System. Robert Ledford Leif Wickland CS518 Fall 2004 I/O, I/O, It's off to disk I go-o-o, A bit or byte to read or write, I/O, I/O, I/O. Overview. What is a file system? Historical view of file systems Another layer of indirection Do I have to? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/1.jpg)
Linux Virtual File System
Robert LedfordLeif Wickland
CS518 Fall 2004
I/O, I/O, It's off to disk I go-o-o, A bit or byte to read or write, I/O, I/O, I/O...
![Page 2: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/2.jpg)
Overview
• What is a file system?
• Historical view of file systems
• Another layer of indirection
• Do I have to?
• File systems have layers
• How is it done?
• Sign me up
![Page 3: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/3.jpg)
Overview
• What is a file system?
• Historical view of file systems
• Another layer of indirection
• Do I have to?
• File systems have layers
• How is it done?
• Sign me up
![Page 4: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/4.jpg)
What is a file system?
• Speaking broadly, a file system is the logical means for an operating system to store and retrieve data on the computers hard disks, be they local drives, network-available volumes, or exported shares in a storage area network (SAN)
![Page 5: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/5.jpg)
What is a file system?
• There is some ambiguity in the term “file system”. The term can be used to mean any of the following:- The type of a file system refers to a specific implementation such as ext2, reiserfs or nfs, each implementation contains the methods and data structures that an operating system uses to keep track of files on a disk or partition
![Page 6: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/6.jpg)
What is a file system?
- An instance of a file system refers to a file system type residing at a location such as /dev/hda4
- Additionally a file system can refer to the methods and data structures that an operating system uses to keep track of files on a disk or partition
![Page 7: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/7.jpg)
What is a file system?
• Linux keeps regular files and directories on block devices such as disks
• A Linux installation may have several physical disk units, each containing one or more file system types
• Partitioning a disk into several file system instances makes it easier for administrators to manage the data stored there
![Page 8: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/8.jpg)
What is a file system?
Overheadview
sector track
cylinder
Disk blocks are composed of one or more contiguous sectors
The same track on each platter in a disk makes a cylinder; partitions are groups of contiguous cylinders
![Page 9: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/9.jpg)
What is a file system?
• Why have multiple partitions?
• Encapsulate your data:
- Since file system corruption is local to a partition, you stand to lose only some of your data if an accident occurs
![Page 10: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/10.jpg)
What is a file system?
• Increase disk space efficiency:- You can format partitions with varying block sizes, depending on your usage- If your data is in a large number of small files (less than 1k) and your partition uses 4k sized blocks, you are wasting 3k for every file- In general, you waste on average one half of a block for every file, so matching block size to the average size of your files is important if you have many files
![Page 11: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/11.jpg)
What is a file system?
• Limit data growth:
- Runaway processes or maniacal users can consume so much disk space that the operating system no longer has room on the hard drive for its bookkeeping operations
- This can lead to disaster. By segregating space, you ensure that things other than the operating system die when allocated disk space is exhausted
![Page 12: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/12.jpg)
What is a file system?
• Partitioning tools and utilities:- fdiskrledford@leonard > sudo fdisk -l /dev/had
Disk /dev/hda: 255 heads, 63 sectors, 4863 cylindersUnits = cylinders of 16065 * 512 bytesDevice Boot Start End Blocks Id System/dev/hda1 * 1 26 208813+ 83 Linux/dev/hda2 27 1070 8385930 83 Linux/dev/hda3 1071 1853 6289447+ 83 Linux/dev/hda4 1854 4863 24177825 f Win95 Ext'd (LBA)/dev/hda5 1854 2375 4192933+ 83 Linux/dev/hda6 2376 2897 4192933+ 83 Linux/dev/hda7 2898 3028 1052226 82 Linux swap/dev/hda8 3029 4863 14739606 83 Linux
- parted: GNU partition editor
![Page 13: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/13.jpg)
What is a file system?
• Review Questions - Where does Linux keep regular file types?
• On block devices such as disks.
- On average, how much of a block is wasted for every file?
• On average ½ of a block is wasted for every file.
![Page 14: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/14.jpg)
What is a file system?
• Review:• File system instances reside on partitions• Partitioning is a means to divide a single
hard drive into many logical drives• A partition is a contiguous set of blocks on
a drive that are treated as an independent disk
• A partition table is an index that relates sections of the hard drive to partitions
![Page 15: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/15.jpg)
What is a file system?
Entire Disk
Partition Table Disk PartitionsMaster BootRecord
Boot Block Super Block Inode List Data Blocks
A Possible File System Instance Layout
![Page 16: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/16.jpg)
What is a file system?
• The central structural concepts of a file system type are:- Boot Block
- Super Block
- Inode List
- Data Block
What is a file system?
Boot Block Super Block Inode List Data Blocks
![Page 17: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/17.jpg)
What is a file system?
• Boot Block: - Occupies the beginning of a file system
- Typically residing at the first sector, it may also contain the bootstrap code that is read into the machine at boot time
- Although only one boot block is required to boot the system, every file system may contain a boot block
Boot Block
![Page 18: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/18.jpg)
What is a file system?
• Super Block: - Describes the state of a file system
- How large it is
- How many files it can store
- Where to find free space in the file system
- Additional data that assists the file management system with operating on the file system
Boot Block Super Block
![Page 19: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/19.jpg)
What is a file system?
• Super Block:- Duplicate copies of the super block may reside through out the file system in case the super block is corrupted
Boot Block Super Block
![Page 20: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/20.jpg)
What is a file system?
• Inode List: - An inode is the internal representation of a file
contains the description of the disk layout of the file data
- file owner
- permissions
- The inode list contains all of the inodes present in an instance of a file system
Boot Block Super Block Inode List
![Page 21: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/21.jpg)
What is a file system?
• Data Blocks: - Contain the file data in the file system
- Additional administrative data
- An allocated data block can belong to one and only one file in the file system
Boot Block Super Block Inode List Data Blocks
![Page 22: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/22.jpg)
What is a file system?
• On a Linux system, a user or user program sees a single file hierarchy, rooted at /
• Every file and directory can trace its origins on a tree back to the root directory
![Page 23: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/23.jpg)
What is a file system?
/
bin boot dev etc home lib mnt proc root sbin tmp usr var
![Page 24: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/24.jpg)
What is a file system?
• Review Questions - Where is the boot strap code located
• In the Boot Block.
- What contains information about a file’s owner and the file’s permissions?
• The inode.
- What is the index that relates sections of the hard drive to partitions?
• The Partition Table.
![Page 25: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/25.jpg)
What is a file system?
• A file system implements the basic operations to manipulate files and directories
• These basic operations include:- Opening of files
- Closing of files
- Creation of directories
- Listing of the contents of directories
- Removal of files from a directory
![Page 26: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/26.jpg)
What is a file system?
• The kernel deals on a logical level with file systems rather than with disks
• The separate file systems that the system may use are not accessed by device identifiers
• Instead they are combined into a single hierarchical tree structure that represents the file systems as one whole single entity
![Page 27: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/27.jpg)
What is a file system?
• So what is a file system?
- A file system is a set of abstract data types that are implemented for the storage, hierarchical organization, manipulation, navigation, access, and retrieval of data
![Page 28: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/28.jpg)
Overview
• What is a file system?
• Historical view of file systems
• File systems have layers
• Do I have to?
• Another layer of indirection
• How is it done?
• Sign me up
![Page 29: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/29.jpg)
Historical view of file systems
• Managing storage has long been a prominent role for operating systems.– This role was so important to the MS-DOS OS
that it was named after that function. – DOS stands for Disk Operating System.– Created in 1980.
![Page 30: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/30.jpg)
Historical view of file systems
• The file system hasn’t changed much since the 1960s.– A research paper was presented in 1965
describing “A General-Purpose File System For Secondary Storage.”
• Laid out the notion of a hierarchal file system much as is used today.
![Page 31: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/31.jpg)
Historical view of file systems
• File system features in 1965 paper– Files– Directories– Links– Access permissions– Create, access and modify times– Path nomenclature: Directory:directory:file – All devices mount into a unified hierarchy– Backing up
• Everybody knew it was a good idea back then, too.
![Page 32: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/32.jpg)
Historical view of file systems
• This type of file system was implemented in Multics.
• Unix was created as an “emasculated” version of Multics.– Project started as gaming system in 1969.
• The designers of Unix had worked on Multics and brought to Unix a Multics-style file system.
![Page 33: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/33.jpg)
Historical view of file systems
• chdir: change directory• chmod: change permission• chown: change owner• chroot: change root• close: close a file• creat: create a file• dup: copy file descriptor• link: add a file reference• lseek: set open file cursor
• mknod: make a special file• mount: graft in a file system• open: open a file system• pipe: create a pipe• read: read from a file• stat: get file status• umount: opposite of mount• unlink: delete file• write: write to a file
File System API in Unix System V, c. 1983
![Page 34: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/34.jpg)
Historical view of file systems
• Tannenbaum wrote Minix, a pedantic version of Unix, in 1987.– Of course, it included a Unix-style file API.
![Page 35: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/35.jpg)
Historical view of file systems
• Linus Torvalds introduced Linux in 1991.– He developed Linux on Minix.– Consequently it was convenient for the OSes
to share a file system and file API.– Thus, Linux inherited the same style file
system as presented in the 1965 paper.– Today Linux supports a superset of the file
system features available in Unix System V.
![Page 36: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/36.jpg)
Historical view of file systems
• Review Questions– In what year was the paper released that
described the file system design that is the ancestor of Linux’ file system?
• 1965– Yes, that’s about 40 years ago
![Page 37: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/37.jpg)
Overview
• What is a file system?
• Historical view of file systems
• Another layer of indirection
• Do I have to?
• File systems have layers
• How is it done?
• Sign me up
![Page 38: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/38.jpg)
Another layer of indirection
• Multics, Unix, Minux, and Linux originally supported only one file system.– They only understood one type of layout on
disk for directories and files.– Because of its origins, Linux initially supported
just the Minix file system.• Limited to small partitions and short filenames
– However, it wasn’t long before people wanted more from their file systems
![Page 39: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/39.jpg)
Another layer of indirection
• The problem:– Linux was
implemented like this.
User Program
Minix FS Interface
Hard Drive
User Program
Minix FSOther FS
Hard Drive BHard Drive A
– To add support for another file system in a similar manner was unsavory and didn’t scale.
• User program must call a separate API for each type of file system.
![Page 40: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/40.jpg)
Another layer of indirection
• “Any problem in computer science can be solved with another layer of indirection.”– David Wheeler (chief programmer for the
EDSAC project in the early 1950s)
![Page 41: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/41.jpg)
Virtual File System
Another layer of indirection
• The solution was to add a layer of indirection to the file system stack.
• In Linux this layer is called the virtual file system (VFS).
• User programs access any file system through a consistent API.
• All File Systems implement an API which is called by the VFS.
User Program
Minix FSOther FS
Hard Drive BHard Drive A
![Page 42: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/42.jpg)
Another layer of indirection
• The VFS is– Another layer of indirection– A file system- and device-agnostic layer of the
operating system– A consistent API for user applications to
access storage independent of the underlying device or type of file system
![Page 43: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/43.jpg)
Task 1 Task 2 Task n…user space
kernel space
VIRTUAL FILE SYSTEM
minix ext2 msdos proc
device driverfor hard disk
device driver for floppy disk
Buffer Cache
softwarehardware
Hard Disk Floppy Disk
Linux Kernel
Robbed from http://www.cs.usfca.edu/~cruse/cs326/lesson22.ppt
![Page 44: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/44.jpg)
Overview
• What is a file system?
• Historical view of file systems
• Another layer of indirection
• Do I have to?
• File systems have layers
• How is it done?
• Sign me up
![Page 45: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/45.jpg)
Do I have to?
• Is a VFS worth doing?– What do you think?
![Page 46: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/46.jpg)
Do I have to?
• Cons– Harms system
performance• Another layer of
indirection
– Adds to the size of the system because more code must be written
– A conceptually simpler system
• Pros– Enables using multiple
file systems • Facilitates research
– Makes the computer more useful
• My Linux box has ext2, ext3, FAT32, and NTFS partitions mounted
– Facilitates code reuse– Simplifies
implementation
![Page 47: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/47.jpg)
Do I have to?
• Is a VFS worth doing?– Ultimately, the answer is yes for general
purpose operating systems.– All modern commercial operating systems do.– What would you do if you had to design an
OS’ file system? Would you use a VFS?
![Page 48: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/48.jpg)
Overview
• What is a file system?
• Historical view of file systems
• Another layer of indirection
• Do I have to?
• File systems have layers
• How is it done?
• Sign me up
![Page 49: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/49.jpg)
File systems have layers
• Like onions and ogres…
• In this context “file system” means the software stack that extends from the user application to the hardware.
![Page 50: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/50.jpg)
File systems have layers
• The file system of Unix System V has three layers– File/Directory API– Inodes– Buffers
File/Directory API
Buffers
Inodes
Storage Device
User Program
File system
![Page 51: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/51.jpg)
File systems have layers
• Unix System V File/Directory API– Functions like open()
and read()• Called by user
programs
– Directories are implemented as files
• Contain children’s– Name
– Inode Number
File/Directory API
Buffers
Inodes
Storage Device
User Program
File system
![Page 52: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/52.jpg)
File systems have layers
• Unix System V Inodes– Allocate disk blocks for
files.– Record file attributes
• Owner• Access Permissions• File Size• Type
– File– Directory– Special
• Not File Names
File/Directory API
Buffers
Inodes
Storage Device
User Program
File system
If file names aren’t stored in inodes,where are they stored?File names are stored in the parentdirectory entry.
![Page 53: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/53.jpg)
File systems have layers
• Unix System V Inodes, continued– Stored on disk– Cached in memory
• With added details– Device the inode is
from– The inode number– If the file is a mount
point– Much more…
– Many pathnames may point to a single inode
File/Directory API
Buffers
Inodes
Storage Device
User Program
File system
![Page 54: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/54.jpg)
File systems have layers
• Implementation of an inodeFile Attributes
Direct 0Direct 1Direct 2Direct 3Direct 4Direct 5Direct 6
Direct 8Direct 7
Direct 9Single IndirectDouble IndirectTriple Indirect
Direct Block
Direct Block
Direct Block
.
.
.
Indirect Block
Inode
Inode
Inode
Indirect BlockIndirect Blocks
Inodes Indirect2 Blocks
Inodes Inodes Indirect3 Blocks
![Page 55: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/55.jpg)
File systems have layers
• Notes on previous diagram– All the direct pointers of an inode are used
before using an indirect pointer– All of the slots in a single indirect inode are
consumed before starting to use double indirect inodes
• Likewise for double indirect
![Page 56: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/56.jpg)
File systems have layers
• Unix System V Buffers– In memory copy of
contents of a disk block
• Same size
– Mechanism through which caching is achieved
• Read ahead• Delayed write
File/Directory API
Buffers
Inodes
Storage Device
User Program
File system
![Page 57: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/57.jpg)
File systems have layers
• Review Questions– Where is the type of a directory entry stored?
• In the inode the directory entry points to.
– Where is the name of an inode stored?• In the directory entry which points to it.
– Can more than one file point to a given inode?• Yes. Many files may point to the same inode.
– Can more than inode point to a disk block?• No. Inodes point to zero or more blocks, but a block may be
referenced by zero or one inode.
– How is a directory different that a file?• A directory is a type of file, as indicated by the inode, which
contains a listing of the directory’s children
![Page 58: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/58.jpg)
File systems have layers
• How do you get from a file name to a file’s contents?
– Recursive procedure1. Start with inode of current or root directory
• If path begins with ‘/’ use root; otherwise use current• Inodes for both are cached for the process
2. Get disk block(s) pointed to by inode for directory3. Search directory listing for next part of path. 4. If found, get the inode pointed to by entry5. If inode says child is a directory, go to 2. If child
is a file, get disk blocks(s) inode points to.
![Page 59: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/59.jpg)
• Resolving a file path to a file’s contents– Consider how we’d
resolve the path /foo/foobarred/found to that file’s contents in this example directory tree.
– See following slides for an example
File systems have layers
/
foo bar.bashrc
foobarred.ssh
lost found
![Page 60: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/60.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Contents#
Step through the following slides to see the process
of resolving the file name /foo/foobarred/found
to its contents
![Page 61: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/61.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Inode for root directory is known to be in slot 0.
Contents#
![Page 62: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/62.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Root directory inode points to block 0
Contents#
![Page 63: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/63.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Root directory listing says that inode 1 is for child
foo.
Contents#
![Page 64: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/64.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Foo’s inode says that it’s a directory and points to
block 7
Contents#
![Page 65: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/65.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Foo’s directory listing says that inode 2 is for child foobarred
Contents#
![Page 66: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/66.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Foobarred’s inode says that it is a directory and its
listing is in block 2
Contents#
![Page 67: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/67.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Directory listing of foobarred says that
child found has inode 7.
Contents#
![Page 68: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/68.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
Inode says that found is a file and 1
is its first block
Contents#
![Page 69: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/69.jpg)
File systems have layers
• Resolving a file path to a file’s contentsInodes
01234567
foo: 1; .bashrc:5; bar: 4
lost: 6; found: 7
I’m found
All my secrets would go in here…
foobarred: 2; .ssh: 3
I’m lost
-empty-
I get into bars with my aliases
0
1
2
3
4
5
6
7
Disk Data Blocks
# Type Disk BlockDir 0Dir 7Dir 2File 3Dir 5File 4File 6File 1
The content of /foo/foobarred/found
is “I’m found”
Contents#
![Page 70: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/70.jpg)
File systems have layers
• Review Questions– What are the layers of the System V file
system?• File/Directory API• Inodes• Buffers
![Page 71: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/71.jpg)
File systems have layers
• Problems in System V file system implementation– Searching a directory listing for a child is time
consuming• Directory listing is unsorted
– Allows entries to be inserted and removed cheaply– Makes searching expensive– Requires that a linear be performed; can’t use binary– Implies that time to find entry increases linearly with the
number of directory entries
![Page 72: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/72.jpg)
File systems have layers
• Problems in System V file system implementation– The listing for the directory must be read from
disk at each step in a path• Can cause the disk head to jump around
– For example, if you want to read the file /foo/foobarred/found, you have to read and search the three directory listing along the way
• Detrimental to performance
![Page 73: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/73.jpg)
File systems have layers
• The Linux solution to these shortcomings– Add another layer of indirection– Layer is called the dcache
• Short for directory cache• Caches the contents of directory listings
– The dcache is composed of dentries• Short for directory entry
– A dentry is a cached association from a path name to an inode
• Also caches relationships to other dentries
![Page 74: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/74.jpg)
File systems have layers
• With the addition of the dcache, what does the Linux file system software stack look like, compared to the System V file system software stack?– Glad you asked– Next slide, please
![Page 75: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/75.jpg)
Dcache
File/Directory API
Inodes
Real FS Implementation
User Program
Storage Device
Buffers
File systems have layers
• Since version 2.1, the Linux virtual file system has had three layers– File/Directory API– Dcache– Inodes
• Buffers are not part of VFS
• Dcache doesn’t exist in System V
Virtual file system
Layers of the Linux kernel FS
![Page 76: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/76.jpg)
File systems have layers
• Modern Linux File/Directory API– Implements a superset
of the System V API– User programs interact
with the File Directory API through path names or integer file descriptors
• A file descriptors is an index into an array of pointers to file structures
– More on that later
Dcache
File/Directory API
Inodes
Real FS Implementation
User Program
Storage Device
Buffers
Virtual file system
Layers of the Linux kernel FS
![Page 77: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/77.jpg)
File systems have layers
• Modern Linux Dcache– Improves performance– Composed of dentries
• Caches the inode that a path name points to
• Caches relationships to other dentries
– Parent– Siblings– Children
• Has a hash value for the name of the file it represents
– Speeds up string comparisons
Dcache
File/Directory API
Inodes
Real FS Implementation
User Program
Storage Device
Buffers
Virtual file system
Layers of the Linux kernel FS
![Page 78: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/78.jpg)
File systems have layers
• Modern Linux Inodes– Much like System V
inodes– Contain pointers to file
system implementation specific operations
Dcache
File/Directory API
Inodes
Real FS Implementation
User Program
Storage Device
Buffers
Virtual file system
Layers of the Linux kernel FS
![Page 79: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/79.jpg)
File systems have layers
• Modern Linux Real File System Implementation– Can be compiled into
the kernel or can be loaded as a kernel module
Dcache
File/Directory API
Inodes
Real FS Implementation
User Program
Storage Device
Buffers
Virtual file system
Layers of the Linux kernel FS
![Page 80: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/80.jpg)
File systems have layers
• Modern Linux File Buffers– Integrated with the
virtual memory cache since 1999
– See virtual memory presentation for more information
Dcache
File/Directory API
Inodes
Real FS Implementation
User Program
Storage Device
Buffers
Virtual file system
Layers of the Linux kernel FS
![Page 81: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/81.jpg)
File systems have layers
• Review Questions– What’s the purpose of a dentry?
• A dentry exists to cache a directory entry in order to improve performance
– What are some of the things a dentry links to?• A dentry links to
– Its parent– A list of its children– A list of its siblings– Its file’s in memory inode
![Page 82: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/82.jpg)
Overview
• What is a file system?
• Historical view of file systems
• Another layer of indirection
• Do I have to?
• File systems have layers
• How is it done?
• Sign me up
![Page 83: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/83.jpg)
How is it done?
• Down to the nitty-gritty code details– There’s nothing to fear here
![Page 84: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/84.jpg)
How is it done?
• There are quite a few C structures that the kernel employs to keep track of open files– struct task_struct (View sched.h code)
• Details about a user process• Has a pointer to a files_struct• During a syscall, the kernel has a pointer the
current process’ task_struct
– struct files_struct (View file.h code)• Tracks the open files for a process• Has an array of pointers to file struct instances.
– When a user program opens a file, it’s given an integer index into this array
![Page 85: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/85.jpg)
How is it done?
• File related kernel structures, continued– struct file (View fs.h code)
• The kernel maintains an array of these with an element for each open file system-wide
• Has a pointer to dentry; plus file owner, read/write cursor position, and more
– struct dentry (View dcache.h code)• Points to the inode for the file
– struct inode (View fs.h code)• Stores its block number and the device it’s from• Points to operations in the FS implementation which can
read the disk
![Page 86: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/86.jpg)
How is it done?
• The following diagram shows the relationship of the aforementioned structures
![Page 87: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/87.jpg)
task_struct struct
files_struct pointer
Kernel’s array of all file structs
other stuff
Other stuffDentry pointer
Dcache
User Program
File handle (integer value)
In memory inode cache
A Dentry
A Dentry
Other stuffInode pointer
A file struct
A file struct
inode struct
inode struct
inode struct
Storage Device
Disk Inode
Disk Inode
Data Blocks
Inside the Linux Kernel
FS instance
files_struct struct
other stuff
file struct ptr array
file struct ptr
file struct ptrfile struct ptr
file struct ptr
![Page 88: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/88.jpg)
How is it done?
• Review Questions– When a user program calls open() to open a file, a
non-negative return value indicates success. What does the function of that non-negative number?
• The return value is the index into the array of file structure pointers in the files_struct structure.
– What is the cardinality between user processes and files_struct instances?
• There is a one-to-one relationship between files_struct instances and user processes.
– Why?• Because there are one-to-one relationships between user
processes and task_struct instances and between task_struct instances and files_struct instances.
![Page 89: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/89.jpg)
How is it done?
• Review Questions– If two user processes open the same file, are
two or one file struct instances created?• Trick question
– Normally, two instances are created– However, if a process opened a file and then forked to
create another process, parent and child have the file open and both share one file struct instance
![Page 90: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/90.jpg)
How is it done?
• Great. Now we see how the data relates• But we’d like to see some action• Consider the following C program#include <unistd.h> void main() { const int seekFromStart = 0; const int rdWrCreate = 00102; char ch = 'A'; int fd = open("fsSample.txt", rdWrCreate); write(fd, &ch, 1); lseek(fd, 0, seekFromStart); read(fd, &ch, 1); close(fd); printf("The character read was: %c\n", ch); }
• What actually happens when those file system API functions are called?
• Links in diagrams go to Linux 2.6 source
![Page 91: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/91.jpg)
getname()sys_open()
get_unused_fd()
How is it done?
System call layerMake a copy of the path name string in kernel space
Reserve an unused element in the array of file struct pointers in the process’ files_struct instance (diagram)
filp_open()The workhorse: see the next slide for more detail.Will return a file struct instance for the opened file.
fd_install()putname()
open()
User program Takes the path name of the file to open as an
argument
![Page 92: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/92.jpg)
How is it done?
Get the dentry for that directory.Call link_path_walk() which performs file name resolution. See next slide for more.
Get a dentry for the file we’re opening. Call real FS implementation to populate structure if not cached.
vfs_create()
If the file doesn’t exist, call the real FS implementation to create it.
may_open()Check that the user has permission to open the file.
path_lookup()
Determine if the path is relative to the current directory, the root directory, or a process specific root.
__lookup_hash()sys_open()
filp_open()
System call layer
open_namei()dentry_open()
open()
User program
![Page 93: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/93.jpg)
sys_open()
How is it done?
open()
System call layer
filp_open()
open_namei()
While there are segments left in the path name
If the segment is “..”
Fail if user does not have permission to inode
path_lookup()
link_path_walk()
Get a dentry for child of inode named by segment
Set inode to the parent of inode, allowing for crossing mount points or inode being root dir
Set inode to the inode for directory that path_lookup() determined the path name was relative to
Parse the next “/” separated segment in path name
If the dentry for the segment isn’t cachedAsk FS implementation for dentry and its inode
If the dentry points to a symbolic linkGet the dentry and inode for the item pointed to
If the dentry is for a mount pointGet the dentry and inode for mounted root dir
Set inode to dentry’s inode
Purpose: Resolve the path name to a dentryUser program
Return the dentry that owns inode
![Page 94: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/94.jpg)
How is it done?
Allocate a file struct instance
Returned a dentry for the path namesys_open()
filp_open()
System call layer
open_namei()dentry_open()
open()
User program
Populate that file struct from the dentry
![Page 95: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/95.jpg)
getname()sys_open()
get_unused_fd()
How is it done?
System call layer
filp_open()
Returned a file struct instance for the opened file
fd_install()putname()
Set the previously reserved file struct pointer to the file struct instance returned by filp_open()
Deallocate the kernel copy of the filename
open()
User program
![Page 96: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/96.jpg)
sys_open()
How is it done?
System call layer
Returns the index where fd_install() put the new file struct pointer. Value is called the file descriptor, or FD.
open()
User program
![Page 97: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/97.jpg)
fget_light()sys_read()
file_pos_read()
How is it done?
System call layerGet the file struct instance for the FD
vfs_read()file_pos_write()fput_light()
read()
User program Takes the open file’s FD as an argument
Copy the read/write cursor location from file struct
Call the real FS implementation to read a specified number of bytes from the file
Increment the read/write cursor by the number of bytes read
Release the file struct instance for the FD
![Page 98: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/98.jpg)
fget_light()sys_lseek()
How is it done?
System call layer Get the FD’s file struct instance
vfs_llseek()fput_light()
lseek()
User program Takes the open file’s FD as an argument
default_llseek()
Release the FD’s file struct instance
Get the read/write cursor location from file structUpdate the cursor according to the argumentsSave the cursor location back into the file struct
![Page 99: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/99.jpg)
fget_light()sys_write()
file_pos_read()
How is it done?
System call layerGet the file struct instance for the FD
vfs_write()file_pos_write()fput_light()
write()
User program Takes the open file’s FD as an argument
Copy the read/write cursor location from file struct
Call the real FS implementation to write the specified bytes at the write cursor location
Increment the read/write cursor by the number of bytes written
Release the file struct instance for the FD
![Page 100: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/100.jpg)
filp_close()
sys_close()
How is it done?
System call layer
Saves the value of FD so that an unused slot can be found quickly when next one is needed
Calls the real FS implementation to flush any unwritten data to storage
close()
User programTakes the FD of the file to close as an argument
Frees FD’s slot in the array of file struct pointers
If this is the last reference to the file struct instance
fput()
Allow the real FS implementation to free resources Deallocate the dentry for the file
![Page 101: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/101.jpg)
Overview
• What is a file system?
• Historical view of file systems
• Another layer of indirection
• Do I have to?
• File systems have layers
• How is it done?
• Sign me up
![Page 102: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/102.jpg)
Sign me up.
• So how do the Linux kernel and the Virtual File System find out about a particular file system implementation?
• (e.g., ext2, nfs, reiserfs)
![Page 103: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/103.jpg)
Sign me up.
• When you perform a Linux installation you are prompted to see if you want each of the supported file systems in your build
• When the kernel is actually built, the file system startup code contains calls to the initialization routines of all of the built in file systems
![Page 104: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/104.jpg)
Sign me up.
• Linux file systems may also be built as modules and, in this case, they may be demand loaded as they are needed or loaded by hand using insmod
• Whenever a file system module is loaded it registers itself with the kernel and unregisters itself when it is unloaded
![Page 105: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/105.jpg)
Sign me up.
• Each specific file system's initialization routine registers itself with the Virtual File System and is represented by a:
file_system_type
data structure which contains the name of the file system and a pointer to its VFS Super Block read routine
![Page 106: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/106.jpg)
Sign me up.
• When a file system is registered the file_system_type data structure is populated with data specific to the file system
• And the file_system_type structure is put into a list, pointed at by the file_systems pointer
![Page 107: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/107.jpg)
Sign me up.
• The kernel utilizes the file_systems list to check if a specific file system has been registered
• And to assist with the mapping of the Virtual File System’s operations to a specific implementation of a file system’s operations by using the VFS Super Block read routine
![Page 108: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/108.jpg)
Sign me up.
file_systems
name (ext2)
fs_flags
*read_super()
*next
name (reiserfs)
fs_flags
*read_super()
*next
name (nfs)
fs_flags
*read_super()
*next
file_system_type file_system_type file_system_type
Registered File Systems Example
file_systems pointer
list of file_system_type
structures
![Page 109: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/109.jpg)
Sign me up.
• Where is the file_system_type structure declared?
• What does it look like?
linux/fs.h
struct file_system_type {const char *name;int fs_flags;struct super_block *(*read_super) (struct super_block *, void *,int);struct file_system_type * next;
};
![Page 110: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/110.jpg)
Sign me up.
• Some details of the file_system_type structure:- name:
-The name of the file system type, such as ext2, nfs, reiserfs etc.- This field is used as a key and it is not possible to register a file system that is already in use- Function find_filesystem() utilizes the
name field to check if a file system has already been registered
![Page 111: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/111.jpg)
Sign me up.
- fs_flags:
- A number of flags which record features of the file system
- If the fs_flag is set to FS_REQUIRES_DEV then a block device must be given when mounting the file system
- Not all file systems need a device to hold them. The /proc file system, for example, does not require a block device
![Page 112: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/112.jpg)
Sign me up.
- fs_flags: (continued)- Aside: The command:$cat /proc/filesystemsdisplays file system information from the file_systems list of file_system_structures- In particular, it is possible to see if a file system requires a device or not by noting
the presence or lack of the “nodev” in front of a file system listing
![Page 113: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/113.jpg)
Sign me up.
- fs_flags: (continued)
ledford@esus ~ $ cat /proc/filesystemsnodev rootfsnodev bdevnodev procnodev sockfsnodev tmpfsnodev shmnodev pipefsnodev binfmt_miscext3ext2nodev ramfsiso9660nodev nfsnodev smbfsnodev autofsreiserfsnodev devptsxfs
![Page 114: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/114.jpg)
Sign me up.
- next:
- A pointer used for chaining all the file_system_type structures together
file_systems
name (ext2)
fs_flags
*read_super()
*next
name (reiserfs)
fs_flags
*read_super()
*next
name (nfs)
fs_flags
*read_super()
*next
file_system_type file_system_type file_system_type
![Page 115: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/115.jpg)
Sign me up.
- read_super
- This routine is called by the VFS when an instance of the file system is mounted
struct super_block *(*read_super) (struct super_block *, void *, int);
- What is going on here?
- A VFS super_block structure is being populated by the function call read_super with data specific to a particular file system
![Page 116: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/116.jpg)
Sign me up.struct super_block *(*read_super) (struct super_block *, void *, int);
- The void* pointer points to data that has been passed down from the mount system call- The trailing int signifies whether or not
read_super should be silent about errors- This is only set when mounting the root file system- Several file systems may be tried when attempting to mount the root
file system so avoiding unsightly errors is desired
![Page 117: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/117.jpg)
Sign me up.
• So what gets called when we register a file system?
• Linux finds out about new file system types by calls to:
register_filesystem()
• And forgets about them by the calls to its counterpart:
unregister_filesystem()
![Page 118: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/118.jpg)
Sign me up.
The formal declarations are:
#include <linux/fs.h>
…
int register_filesystem(struct file_system_type * fs);
…
int unregister_filesystem(struct file_system_type * fs);
![Page 119: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/119.jpg)
Sign me up.
• register_filesystem()linux/fs/super.c…int register_filesystem(struct file_system_type * fs){ struct file_system_type ** tmp;
if (!fs) return -EINVAL; if (fs->next) return -EBUSY; tmp = &file_systems; while (*tmp) { if (strcmp((*tmp)->name, fs->name) == 0) return -EBUSY; tmp = &(*tmp)->next; } *tmp = fs; return 0;}
• So what’s happening here?- Essentially register_filesystem() takes a, file system specific, populated file_system_type structure as a parameter performs some checks and, if successful, adds the file_system_type structure to the file_systems list
![Page 120: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/120.jpg)
Sign me up.
• unregister_filesystem()linux/fs/super.c…int unregister_filesystem(struct file_system_type * fs){#ifdef CONFIG_MODULES struct file_system_type ** tmp;
tmp = &file_systems; while (*tmp) { if (fs == *tmp) { *tmp = fs->next; fs->next = NULL; return 0; } tmp = &(*tmp)->next; }#endif return -EINVAL;}
• So what’s happening here?- unregister_filesystem() takes a, file system specific, populated file_system_type structure as a parameter and removes the file_system_type structure from the file_systems list if present
![Page 121: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/121.jpg)
Sign me up.
• How is the file_system_type structure populated with file system specific data?
- Each file system implementation defines a file_system_type structure with data specific to that file system
![Page 122: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/122.jpg)
Sign me up.
• Example:
- The ext2 file_system_type structurelinux/fs/ext2/super.c
…
static struct file_system_type ext2_fs_type = {
"ext2",
FS_REQUIRES_DEV /* | FS_IBASKET */, /* ibaskets have unresolved bugs */
ext2_read_super,
NULL
};
- Here we can see the fields: name, fs_flags, read_super and next being populated
![Page 123: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/123.jpg)
Sign me up.
• How is the register_filesystem() called from a specific file system?
- Each file system implementation defines an init function that calls register_filesystem()
![Page 124: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/124.jpg)
Sign me up.
• Example:
- The ext2 init_ext2_fs() functionlinux/fs/ext2/super.c
…
static int __init init_ext2_fs(void)
{
return register_filesystem(&ext2_fs_type);
}
- This is pretty self-explanatory
![Page 125: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/125.jpg)
Sign me up.
• How is the unregister_filesystem() called from a specific file system?
- Each file system implementation defines an exit function that calls unregister_filesystem()
![Page 126: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/126.jpg)
Sign me up.
• Example:
- The ext2 exit_ext2_fs() functionlinux/fs/ext2/super.c
…
static void __exit exit_ext2_fs(void)
{
unregister_filesystem(&ext2_fs_type);
}
- Again, This is self-explanatory
![Page 127: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/127.jpg)
Sign me up.
• So how and where are the calls to the init and exit functions made?
linux/fs/ext2/super.c…module_init(init_ext2_fs)module_exit(exit_ext2_fs)
- Calls to module_init() and module_exit() begin the, file system specific, registration and unregistration processes
![Page 128: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/128.jpg)
Sign me up.
• But how are module_init() and module_exit() called?- module_init() is called when the module is loaded, if built as a module, with a call to insmod
- Or it is called at the same time as all of the init calls are made during the kernel boot
process- module_exit() is called when the module is unloaded, if built as a module, with a call to rmmod
![Page 129: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/129.jpg)
Sign me up.
• So how does this all fit together to register a file system?
![Page 130: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/130.jpg)
Sign me up.module_init(init_ext2_fs)
init_ext2_fs(void)
register_filesystem(&ext2_fs_type)
file_systems
name (ext2)
fs_flags
*read_super()
*next
file_system_type
Called at boot time or with
insmod
name (ext2)
fs_flags
*read_super()
*next
file_system_typeTakes a file_system_type
structure specific to ext2 as a parameter
Registering the ext2 file system
Populates the file_systems list with the
file_systems_type structure for ext2
![Page 131: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/131.jpg)
Sign me up.
• So how does this all fit together to unregister a file system?
![Page 132: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/132.jpg)
Sign me up.Sign me up.module_exit(exit_foo_fs)
exit_foo_fs(void)
unregister_filesystem(&foo_fs_type)
file_systems
name (foo)
fs_flags
*read_super()
*next
file_system_type
Called with rmmod
name (foo)
fs_flags
*read_super()
*next
file_system_typeTakes a file_system_type
structure specific to foo as a parameter
Unregistering the foo file system
Remove from the file_systems list the file_systems_type structure for foo
![Page 133: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/133.jpg)
module_exit(exit_foo_fs)
exit_foo_fs(void)
unregister_filesystem(&foo_fs_type)
file_systems
name (foo)
fs_flags
*read_super()
*next
file_system_type
Called with rmmod
Unregistering the foo file system
Remove from the file_systems list the file_systems_type structure for foo
Sign me up.
![Page 134: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/134.jpg)
Sign me up.Sign me up.Sign me up.module_exit(exit_foo_fs)
exit_foo_fs(void)
unregister_filesystem(&foo_fs_type)
file_systems
name (ext2)
fs_flags
*read_super()
*next
file_system_type
Called with rmmod
Unregistering the foo file system
Remove from the file_systems list the file_systems_type structure for foo
Update the file_systems pointer to point at what
next was pointing at in the file_system_type structure for foo
In this case it is ext2
![Page 135: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/135.jpg)
Sign me up.
• Whoptie doo, Basil.. what does it all mean?- For the Virtual File System layer to work with a specific file system implementation it must have some knowledge of the file system- This knowledge is acquired in part by registering the file system- Once this is done the Virtual File System can map calls to files, on a specific file system, through the correct structures and perform the requested operations
![Page 136: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/136.jpg)
Sign me up.
• What’s next?
- After the file system has been registered we must mount it in order to use it
- A file system is mounted at boot or with the use of the mount command
- To unmount a file system the umount command is used
![Page 137: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/137.jpg)
Sign me up.
• So what does mounting do?
- When a file system is mounted the file_systems list is searched to see if the file system has been registered
![Page 138: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/138.jpg)
Sign me up.
file_systems
name (nfs)
fs_flags
*read_super()
*next
name (reiserfs)
fs_flags
*read_super()
*next
name (ext2)
fs_flags
*read_super()
*next
file_system_type file_system_type file_system_type
Looking for ext2 in the file_systems list
Is name ext2? Is name ext2? Is name ext2?
NO NO YES
![Page 139: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/139.jpg)
Sign me up.
- If the file system is found in the list then the read_super() function in the file_system_type structure for the particular file system is called
- The call to read super occurs in fs/super.c do_mount()
name (ext2)
fs_flags
*read_super()
*next
file_system_type
![Page 140: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/140.jpg)
Sign me up.
• Review read_super()- Like we saw earlier the read_super() function
populates a VFS super_block with data from a particular file system’s super_block
![Page 141: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/141.jpg)
Sign me up.
• So what the heck is a super_block?- A super_block is a structure that maintains information about a particular file system- The Virtual File System has a super_block structure that is populated with data from a specific file system’s implementation of the super_block- Is this confusing?
![Page 142: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/142.jpg)
Sign me up.
data 1
data 2
data 3
data 4
super_block
data 1
data 2
data 3
data 4
ext2_super_block
… …
data n data n
VFS ext2
![Page 143: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/143.jpg)
Sign me up.
• A little more in depth– The call to the VFS read_super() makes a call
to a specific file systems version of read_super()
– The file systems version of read_super() populates a VFS super_block structure with data from it’s version of the super_block and returns the populated VFS super_block back to the VFS read_super()
• Confused again?
![Page 144: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/144.jpg)
Sign me up.read_super()
ext2_read_super()
data 1
data 2
super_block
data 1
data 2
ext2_super_block
… …data n data 2
VFS version
ext2 version
Returns populated suber_blockback to read_super
Populates
super_block
![Page 145: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/145.jpg)
Sign me up.
• So why go through all this trouble?– In order for the VFS layer to operate on the
data or inodes residing on a specific file system it needs to know what the file system specific data/inode operations are
– Part of the VFS super_block structure is a pointer to a VFS super_operations structure
– This super_operations structure is populated during read_super with the operations specific to a file system
![Page 146: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/146.jpg)
Sign me up.
• Links to the source for the above operations:– VFS read_super()– ext2_read_super()– VFS super_block– ext2_super_block– VFS super_operations– ext2_sops // The ext2 version of the VFS
super_operations stucture
![Page 147: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/147.jpg)
Sign me up.
• What’s next?– Now that we’ve populated a VFS super_block
with file system specific data we need to maintain some lists in the VFS layer so we know what file systems have been mounted
– In read_super there is a call to insert_super()– insert_super places the VFS super_block
generated by read_super into the super_blocks list
![Page 148: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/148.jpg)
Sign me up.
fs/super.c
static void insert_super(struct super_block *s, structfile_system_type *type)
{
s->s_type = type;
list_add(&s->s_list, super_blocks.prev);
list_add(&s->s_instances, &type->fs_supers);
spin_unlock(&sb_lock);
get_filesystem(type);
}
![Page 149: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/149.jpg)
Sign me up.read_super()
ext2_read_super()
VFS version
ext2 version
Populates VFS super_block
Returns to read_super()
read_super()
insert_super()
Places super_block in list
super_blocks
data 1
data 2
super_block
…data n
super_block is added to the super_blocks list
![Page 150: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/150.jpg)
Sign me up.
• In addition to the super_blocks list another list, vfsmntlist, is also maintained with the currently mounted file systems- vfsmntlist is a list of vfsmount structures- The vfsmount list is populated by calls toadd_vfsmnt()- The function do_mount() calls add_vfsmount() after the call to read_super()- Within the vfsmount structure is a pointer to the VFS super_block for the mounted file system
![Page 151: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/151.jpg)
Sign me up.Sign me up.do_mount()
add_vfsmnt()
Adds a vfsmount structureto the vfsmntlist
Returns to do_mount()
read_super()
vfsmntlist
data 1
data 2
vfsmount
…
data n
vfsmount structure is added to vfsmntlist
do_mount()
![Page 152: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/152.jpg)
Sign me up.
• In addition to the pointer to the VFS super_block of the file system, the vfsmount structure contains: – The device number of the block device
holding the file system– And the directory where this file system is
mounted
![Page 153: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/153.jpg)
Sign me up.
• Review• In order to use a file system:
– It must be registered with VFS• Registration builds a VFS super_block structure from the
specific file systems super_block• And places a file_system_type structure entry in the
file_systems list– It must be mounted with VFS
• Mounting adds an entry into the super_blocks and the vfsmntlist lists
• By utilizing the information in these data structures VFS is able to resolve operations to specific file system implementations
![Page 154: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/154.jpg)
Overview
• What is a file system?
• Historical view of file systems
• Another layer of indirection
• Do I have to?
• File systems have layers
• How is it done?
• Sign me up
![Page 155: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/155.jpg)
Resources
• http://www.tldp.org/LDP/lki/lki-3.html
• http://mm.iit.uni-miskolc.hu/Data/texts/Linux/SAG/node74.html
• http://www.enterprisestorageforum.com/technology/features/article.php/2026611
• http://www.multicians.org/fjcc4.html
![Page 156: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/156.jpg)
Resources
• http://www.computerhope.com/history/unix.htm
• http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/intpatt-ch03.asp
• http://e2fsprogs.sourceforge.net/ext2intro.html
• http://www.cs.usfca.edu/~cruse/cs326/lesson22.ppt
![Page 157: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/157.jpg)
Resources
• http://bama.ua.edu/~dunna001/journeyman/html/x323.htm#AEN390
• http://ldp.rtin.bz/LDP/lki/lki-3.html
• http://people.netfilter.org/~rusty/unreliable-guides/kernel-hacking/routines-init-again.html
• http://www.cse.unsw.edu.au/~neilb/oss/linux-commentary/vfs-3.html
![Page 158: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/158.jpg)
Resources
• http://www.cs.wits.ac.za/~adi/courses/linuxadmin/content/module2doc.html
• http://www.tldp.org/LDP/tlk/fs/filesystem.html
• http://www.faqs.org/docs/kernel_2_4/lki-3.html
• http://www.tldp.org/HOWTO/Partition/partition-4.html#AEN487
![Page 159: Linux Virtual File System](https://reader035.vdocument.in/reader035/viewer/2022062409/568150e3550346895dbf00a9/html5/thumbnails/159.jpg)
Resources
• http://www.science.unitn.it/~fiorella/guidelinux/tlk/node94.html
• The design of the UNIX Operating System, Maurice J. Bach
• Linux File Systems, Moshe Bar