Intro to DFS assignment

Announcements

• DFS (PA4) Part 1 will be out over the weekend • Part 1 Due 10/30 11:59 pm • Part 2 12/7 11:59 pm • Part 3 12/14 11:59 pm • Part 4 (optional extra credit) 12/23 11:59pm

You build a DFS!

1. Lock server, at-most-once RPC semantics 2. Implement extent server; create/lookup/readdir FUSE ops 3. Implement read/write/open/setattr 4. Implement mkdir/unlink, integrate locks! (optional)

Programming Assignment: YFS

• Yet-another File System. :) • Simpler version of what we just talked about: only one extent server (you don’t have to implement Petal; single lock server)

Part 1: lock server

• Implementing the lock server • Provide mutual exclusion to ‘lock resources’ • Using pthread mutexes, condition variables!

• Implementing at-most-once RPC semantics

Providing mutual exclusion over the net • In the assignment: lock is a 64-bit number • Client might: •  acquire(lock_a); do work; release(lock_a); • You must create the lock if it doesn’t yet exist on the server

Example tester

% ./lock_tester 3772 simple lock client acquire a release a acquire a release a acquire a acquire b release b release a test2: client 0 acquire a release a test2: client 2 acquire a release a ...

What’s provided

• Simple RPC framework, skeleton for lockserver code • What it does: handles setting up sockets, marshalling/unmarshalling basic data types, sending over TCP (whew) • What it doesn’t: does not keep track of RPC request state; duplicate RPCs are invoked twice! :(

Example RPC call

Your job for lock server

Lossy

• RPC_LOSSY: drops, duplicates, delays. • Run lock_tester with RPC_LOSSY=0

• Now try running it with RPC_LOSSY=5

•  Simulates network failure •  it should hang or error. FAIL!

Why?

• No at-most-once RPC semantics •  implemented...yet.

•  If reply dropped, duplicate sent: acquire(a), acquire(a)..............................

Strawman

• Remember every RPC call • Client sends a unique RPC identifier

• Remember every RPC reply • To avoid invoking the actual function

• What’s the problem with this approach?

Sliding window RPCs

• Assume the following sent by client: marshall m1 << clt nonce // client id << svr nonce // server id << proc // procedure number (bind, acquire, etc). << myxid // unique request id for this RPC << xid_rep_window.front() // Last out-of-order RPC reply received << req.str(); // Data • But we need some additional info from the client

At the end

% export RPC_LOSSY=5 % ./lock_server 3772 & % ./lock_tester 3772 ... all tests passed % ./rpctest simple_tests ... all tests passed

Helper functions

• Method_thread

Code reading

Part 2

• Extent server •  FUSE! • Semantics of filesystem calls

Programming Assignment: YFS

• Part 1: lock server • Part 2: extent server

The extent server

• Don’t have to deal with storing data on disk: you will store contents in memory • map<inode, std::string>

• All yfs_client operations synchronize with same extent server

Getattr (given)

Your job

• Extend extent_server to support •  put(extentid, string, ...) •  get(extentid, string&) •  remove(extentid, ...) • See extent_protocol.h and extent_smain.cc •  Later you will likely add more! • Must properly deal with ctime/atime/mtime

Data formats at extent server

• Directories maintain a mapping of filenames to inode numbers • if root (inode 1) has two files “file1”, “file2”: get(1) might return •  file1:3983293923 •  file2:3384927553

• You can choose how you store this information as a string

Metadata time mgmt

• atime: access time •  Updated whenever file contents accessed •  Set to 0 on file creation

• mtime: modification time •  Updated when file contents modified

•  ctime: change time •  Updated whenever metadata modified

FUSE

• http://fuse.sourceforge.net/

Mapping FUSE functions

Example

•  Fuse.cc • void •  fuseserver_getattr(

CREATE/MKNOD

• Generate inode number (rand()) • This is called on files: make sure inode has MSB set to 1 • Directories have MSB set to 0

• Create the file at the extent server • Add the name to inode mapping to parent info at extent server

Lookup

• Given: filename, parent inode • Look through parent directory list •  Find inode that maps from filename • Getattr on that inode

• Fill in return structure (fuse_entry_param)

Readdir

• Given: inode number for a directory • Get filename:inode mapping for that dir •  For each filename:inode pair call dirbuf_add(...) function provided

• You will be able to create empty files in the root directory and do an ls on the root • You’re not yet storing files...

Testing

• ./start.sh • ./test-lab-2.pl • ./stop.sh • test-lab-2.pl creates a bunch of files, tests whether the files the script created are there

Parts 3 and 4

• Part 3: You implement open, read, write, setattr • Part 4: You implement mkdir, unlink, and locking