Project 3: File System Project 3: File System DesignDesign

Project 3: File System Project 3: File System DesignDesign

COS318COS318

Fall 2002Fall 2002

Last Time• Web Server

• Extensive use of a file system on server machine without actually worrying about how it was implemented.

request

response

server clientu/

vivek/public_html/

index.html cos318_02/

Outline• Introduction• Design requirement• Helper functions• Implementation of file system calls• Hints• Reference

Introduction• Why do we need a file system?

– Abstraction– Protected access– Security

• Simple Unix-like file system– i-node– directory tree– links(symbolic/physical)

What does my file server need to

accomplish• A New Client Program with• New Methods:

– GET <dir name | file name>• Fetch a file or dir listing and show them

– CREATE <dir name>• Create a dir in server disk

– DELETE <file name | dir name>• Delete a file or dir in server disk• populated directories should not be deleted

– PUT <file name> <local file name> <startpos> <endpos>• Create a new file in server disk, or append to an exsiting file with contents

from local file

[Notice: name uses absolute path]

More on file server functionality

• Use our client program to create a populated directory tree inside of server disk, and then use your web server to serve this content.

• More test programs:– Randomly requests– Maybe not just be PUT or GET: after some file

deletions, things will become more interesting.– Coming soon…

So what do you have• A Virtual Disk: a data structure

– Disk_name: dev_318– Disk_pointer: used to access the disk(in fact a

file descriptor)– Sector_size: 512 bytes – Capacity: M, G …

• Rather than write to a real disk, you'll be building a disk inside of a large file

• All development and debugging are in user space

Helper Functions: not tuxedo

• GetExistingDisk(char *pathname)• CreateNewDisk(void)• CloseDisk(void)• ReadSector(int sectorNum, char *buf) • WriteSector(int sectorNum, char *buf)• Notice:

– You are allowed to access disk only with those function calls;

Helper Functions(cont.)• GetExistingDisk(char *pathname)

– returns a pointer to previously created disk in directory pathname, or exits if no such disk

• CreateNewDisk(void)– returns a pointer to a new disk

• CloseDisk(void)– closes the virtual disk(dev_318)

Helper Functions(cont.)• ReadSector(int sectorNum, char *buf)

– reads 512 bytes from disk

• WriteSector(int sectorNum, char *buf)– writes 512 bytes from disk

• Notice:– If any other user space function calls can facilitate

your work, feel free to use them.

• Using file system functions to build disk is relatively easy compared to using really disk to build file system

What do I need to do• You should build on the code you wrote

for the previous assignments– Good thing: we have a reference server

• The first step is to understand the default file system provided by the base code(helper functions)

• You'll have to understand at least one new header with one of the new methods.

What do I need to do(cont.)

• You'll have to write versions of various system calls that you used in Project 1– mkfs, open, close, read, and write– Additionally, you'll also have to implement

versions of stat, mkdir, and a few others, also function calls to getting file metadata and directory information in order to properly build directory listings

• No need to handle all of the various flags and parameters, but should implement enough to get the project working correctly

One example-open()• For example, the open() system call can

be used to try to test the existence of a file in addition to creating it for writing

• O_CREATE: the file creation flag– To properly handle “PUT" method

• Not always true: need other flags– “GET” a non-existent file

mkfs• File system creation

– Dos: format– Linux: mkfs/mke2fs/mkfs.msdos– Solaris: newfs

• Structure of a typical file system(unix-like)– Superblock

• Describes layout of i-Nodes, bitmaps, blocks, root directory, etc.– i-Nodes

• Describes a file/dir – Link counter, list of blocks, size, etc. (owner, permission, …)

– Block Allocation Bitmap– Data blocks

mkfs(cont.)• Creates the file system

– Set up the Superblock– Flag all i-Nodes as type free (f)– Zero out the Alloc Bitmap– Create the root directory

• Flag an i-Node as type directory (D)• Store it’s number in Superblock• Alloc one block in the bitmap• Initialize the ‘.’ and ‘..’ directory entries and set

the others as being INVALID

BootblockKernelProcs

SuperBlock

i-NodesBlockAlloc

Bitmap*Data Blocks

SuperBlock• Describes the layout of the file system

– signature – A magic number– size – Total size of fs in blocks– root_inode – The root directory– inode_start – 1st block for i-Nodes– inode_blocks – Total i-Nodes blocks– bitmap_start – 1st block of bitmap– bitmap_blocks – Total bitmap blocks– alloc_start – 1st block for data– num_blocks – Total data blocks– free_blocks – Count of unallocated blocks

Bitmap• Bitmap

– Indicates which data blocks are free, and which are used

– Maps the data blocks only (not i-Nodes)

I-node• Describes a file/directory

– type – File, directory– opens – Count of open file handles– links – Count of links to this i-Node– size – Total byte count – direct[10] – List of data blocks– indirect[3] – List of indirect blocks

• Advantages– Fast access to small files– Supports very large files– Support sparse files

Hints• Break this project into two components -

expansion of the main server itself, and implementation of the underlying disk

• Within the file system:– Start with the creation of files and try to get

that working properly; – Then build directories on  top of that

mechanism;– Finally work on directory listing support.

• For files: direct blocks first

More Hints• Keep the high level interface as much

as possible– By having wrappers that can either call the

real call or your replacement, you'll be able to do testing quickly and efficiently

• Standard interface:– Stat(const char *pathname, struct stat *buf)

stat• Returns statistics for given a filename

– inodeNo – The file’s i-Node number– Type – File or directory– Opens – The open count– Size – The byte count to EOF– numBlocks – Number of allocated blocks

Reference• Check the manual page of the

function calls– What they do– What’s their interface– What’s the most useful parameters

Wrap Up• Fun again? Probably…. • Not too difficult.• But a little bit more coding.• Questions?