Weenix Intro & VFS

cs167 - Spring 2019

What is Weenix?

● A Unix-like OS written for the course by twd● Supports process management and device drivers● One thread per process, uniprocessor, non-preemptible kernel

○ This means threads switch when they yield the processor○ Happens more frequently than you’d think (due to driver interrupts)

● Up to you to implement VFS and an on-disk filesystem (S5FS, but that’s after VFS)

● Much more code than uthreads or mthreads in total○ We give a lot of support code and tools to navigate it!

Getting & Building weenix

$ git clone /course/cs1670/asgn/stencil/vfs/weenix <your_directory>1

README

$ make # `make -j 4` takes advantage of multiprocessors

$ ./weenix -n # fire up weenix

1 If you know git and use an internet repo, make sure you use a private repo. Otherwise, you’ll be violating the collab policy. If you don’t know git, never fear! There will be a quick git intro at the end of this help session.

Weenix Documentation

● README ← Read this

● Documentation is in doc/latex ← Read parts of this

○ In particular: Introduction, Appendices B & C, VFS & S5FS chapters

○ Need to run make first in the doc/latex directory to build the pdf

● The documentation is your friend and your guide

Tools: nyi

$ make nyi

● Run this to get a list of functions you still need to implement for the current

project

● Current project is specified in Config.mk (We do this for you; VFS=1)

Tools: GDB

YOU

GDB

No matter what your history is with GDB, please learn to love it. Printf debugging can only go so far. Come to hours if you want to learn GDB. There are also online resources available such as the CS0330 GDB guide that provide further details on how to use specific commands.

Tools: GDB

$ ./weenix -d gdb

● Learn how to use breakpoints and inspect data

● Read README.gdb in the root weenix directory on special “kernel” gdb commands○ e.g. kernel dbg <mode>

Tools: cscope

● cscope will make your lives MUCH easier when navigating large C codebases like Weenix○ go to definition, callers, callees, file, matching text, etc…

$ make cscope # Build the cscope database with make cscope

$ cscope # run cscope from the kernel/ directory

● cscope plugins exist for common text editors○ Or just use it in a separate terminal window - cscope will let you drop

into the text editor of your choice in order to make quick edits● You only need to rebuild the database if stuff has changed a lot

Tools: cscope

Tab key switches between top and bottom

C-n & C-p move up and down inside a section

(number keys work too)

C-d exits

Debugging

Debugging: Logs

● Use dbg() macro to log information

○ dbg(DBG_VFS, “Something bad happened %d\n”, val)

○ First arg is a debug mode - there are lots of different debug modes

○ Indispensable for reference-count debugging

● Enable/disable debug printouts by setting DBG variable in Config.mk

○ See the comments in Config.mk for information about how to enable

specific debug modes

○ It will be very useful to turn some of these on so you can see the full

output of vfstest()

Colors correspond to debug modes

Debugging: Logs

Debugging: Panics

● Happen when Weenix enters an illegal state

● Halts the processor (i.e. Weenix crashes)

● panic(“YOU DID BAD THING NUMBER %d\n”, 666)

Debugging: Asserts

● Use the KASSERT macro liberally: starts and ends of functions

● Causes kernel panic when assertion fails

KASSERT(NULL != input);

KASSERT(NULL != input && “a nice message for myself so I know which KASSERT this is”) //strings are logical-true in C

● KASSERT whenever something must be true for your kernel to function correctly

VFS

Virtual File System

● Interface (API) between high-level kernel and various possible filesystems (e.g. ramfs, s5fs, procfs*, ext4*, zfs*, ntfs*)

● Partially defines operations like open(), close(), read(), write(), mkdir(), mknod(), etc., which call upon the underlying to do some (a lot) of the work

* not actually part of Weenix

VFS: Where it Fits in

Virtual Memory

Userland

VFS

System Calls

ProcessManagement

TTY

TerminalDriver

DiskDriver

S5FSScheduler

open,read,write,lseek,mount,getcwd fork,exec,wait

OtherDrivers

yield mmap,brk

TLB,Page Tables,Page Frames

VFS: Important Data Structures

● vnode_t - represents a node (file/directory/etc.) in the filesystem tree

○ Each one is initialized by the file system

○ One for each directory and file (and link, etc.)

○ Contains operations (methods) for this object (function pointers to

read/write implementations for the underlying filesystem

○ Contains attributes such as type of vnode, pointer to inode (file system

implementation’s counterpart to vnode), and reference count

VFS: Important Data Structures

● fs_t - represents the file system itself

○ Includes info like the root vnode of the filesystem (“/”), FS ops (like reading vnodes, i.e., initializing a vnode from the underlying file system)

○ Only one mounted filesystem is required - and the mounting is hard-coded (don’t need to worry about it)

VFS: Vnode Memory management

We can't store vnodes for every file in the whole filesystem all day.

● Only need to be around when file is open or in-use● Vnodes are (re)constructed by the filesystem by reading the corresponding

data off the disk, or other underlying device.● To minimize space usage, we can keep a reference count and free the vnode

when it drops to zero.● vget(vnum), vput(vnode), vref(vnode) all change the ref count.● If refcount goes to zero prematurely, the vnode data becomes garbage, so

be careful.● If the refcount is nonzero when it shouldn’t be, you waste memory (and

Weenix panics on shutdown)

VFS: What You Have to Do

● Filesystem setup (mounting) ← done for you ● ramfs ← done for you● Name operations (vnode lookup)

○ lookup(), dir_namev(), open_namev()● System calls (lots to do, but simpler and similar to each other)

○ open,close,read,write,dup,mknod,mkdir, rmdir, unlink, link, “rename,” chdir, getdent, lseek, stat

○ Error-check like it’s your job ! (Which it is)● Devices: special_file_[read,write] is also your job

VFS: lookup()lookup(): given a vnode for a directory and a filename in that directory, return the vnode for that particular file if it exists

● Mostly delegates to filesystem-specific directory lookup routine

● But don’t forget to check for errors

VFS: dir_namev()Paths have a directory-part and a base-part

● /usr/bin/ls: /usr/bin = directory, ls = base● use basename on department filesystem to get a feel for this. (e.g., try

basename /bin/bash)● dir_namev(): given a full path, return the path for the base part, and the

vnode for the directory part● First, parse the path into parts● Continue calling lookup() down the tree until you get to the end

○ Check your ref-counting!

VFS: dir_namev() Example

Path = “/usr/bin/ls”

● First lookup() “usr” in the root directory...● ...then lookup “bin” in the resulting vnode...● ...then return that vnode, and return “ls” for the name...● ...and make sure you didn't mess up the vnode reference counts in the

process.● Make sure you can handle the following cases (use the department

filesystem to see what should happen when you use these paths with various commands and system calls!):○ /////////////////////○ ////course/////////////cs169//////

VFS: open_namev()open_namev(): given a pathname and the flags from open(2), return the vnode for the full path (creating it, if O_CREAT is specified and the file does not exist)

● Uses dir_namev(), lookup(), and possibly the create() vnode operation

VFS: Testing

Since you don’t have a functional filesystem like s5fs yet, we give you ramfs to use when testing VFS

● Weenix must halt cleanly - i.e. your refcounts must be correct

● Write lots of your own test code● Use our vfstest() (but it is not exhaustive!)

○ kshell command: “vfstest”○ Make sure the relevant debug printouts are on, so you can see

vfstest()’s full output

VFS: Getting Started

● Double check that “VFS = 1” in Config.mk● Go!!!!!!

○ lookup(), dir_namev(), and open_namev() are a good place to start

○ Then move on to system calls that use them!○ Use the manual (“man”) pages to see when particular errors are

appropriate for a given syscall.● Your s5fs (next project) will build on this project, so you will want to make it

very robust.

VFS: Handing In

Run /course/cs1670/bin/cs1670_handin vfs from the Weenix root directory