e lving rpc fo sconsistency semantics: nfs vs afs n n fs n s tateless server n c lie n t ch e cks p...
TRANSCRIPT
Evolving RPC for Active StorageEvolving RPC for Active Storage
Muthian Sivathanu
Andrea C. Arpaci-DusseauRem
zi H. Arpaci-Dusseau
University of Wisconsin-M
adison
Architecture of the futuren
Everything is “active”n
Cheaper, faster processing powern
Example: “sm
art” disksn
Everything is “distributed”n
Network between processors and devicesn
Example: Network-attached storage
nNeed: Effective software paradigm
sn
Leverage power of active components
nBut rem
ains easy to use
Software systems of the future
nNeed: Tools to build “active”, “distributed” system
sn
Pragmatic: Easy for system
developers to usen
Powerful: Exploit active nature of systems
nActive system
s permit extensibility
nDownload code to device
nTailor to needs of applications/system
nSim
plicity, maintainability
•Provide prim
itives, allow clients to compose interface
nTraditional “distributed” system
s built w/ RPCn
Simple, easy-to-use com
munication paradigm
nBut not designed for “active” world
nBuild better distributed system
s w/ “active” components
Scriptable RPCn
SRPC: Paradigm for extensible distributed system
sn
Pragmatic: RPC-like developm
ent processn
Powerful: Exploit active components easily
nCase study: Active storagen
High Performance
•Efficient “com
position” of primitives
nRapid addition of new functionality
•Powerful: Advanced consistency sem
antics over NFS•
Simple: Substantial functionality in < 20 lines of code
nSim
plicity in design•
Obviate distributed locking, crash recovery
nCom
pelling paradigm for future system
s
Outline
nM
otivationn
Scriptable RPCn
Case Study: Active Storagen
Performance
nFunctionality
nSim
plicityn
Summ
ary
Scriptable RPC (SRPC)n
Evolve Remote Procedure Call (RPC)
nAugm
ent RPC interface of “server” with ascripting capability
ClientScript
Limited execution
context
Active Disk
Script Interpreter
Result
ß Prototype uses Tcl as the scripting language
SRPC : Key issuesn
Migration path
nEfficient execution of scripts
nSafety
Migration path
nM
ake transition to new paradigm less
intrusiven
Code to embed scripting into server
automatically generated
nExisting unm
odified clients co-exist withscripting clients
nDevelopm
ent process exactly the same
Development: RPC
Interfacedefinition
InterfaceIm
plementation
IDL
Compiler
Stubs
Native
Compiler
Distributed Service
Development: SRPC
Interfacedefinition
InterfaceIm
plementation
IDL
Compiler
Stubs
Native
Compiler
Distributed Service
Tcl Wrappers
Interpreter
Efficient execution of scriptsn
Hide script interpretation overheadn
Script cachingn
Exploit efficient Tcl bytecode representationn
Concurrencyn
Multiple interpreters run sim
ultaneouslyn
“Fast” standard library of primitives
nIm
plemented in C
Safetyn
Guard against m
isbehaving client scriptsn
Limited execution environm
ent: SafeTcln
Even while loops can be turned offn
Runtime type-checking
nPrevent illegal m
emory references
nAutom
atic tracking of locksn
Safe concurrent execution
Outline
nM
otivationn
Scriptable RPCn
Case Study: Active Storagen
Performance
nFunctionality
nSim
plicityn
Summ
ary
Case Study: Active storagen
Utilize CPU power at disks for client-specific processing
nPrevious approachesn
Demonstrate perform
ance benefitsn
But, require radically new architectures•
No migration path for existing services
nLim
ited class of applications•
Parallel database primitives
Evaluation environment
nPlatformn
P-III 550 MHz m
achines, 1GB m
em, 100 M
b/s netn
Linux kernel v2.2.19n
Case studies enhance ScFS using SRPC
Application
VFS
ScFS
ActiveD
iskN
FS - like protocolLinux
Kernel
Clien
t
ScFS: Performance enhancem
entsn
Combine dependent sequence of
operations into single scriptn
Reduction in network round-trips needed fora logical operation
•Benefit sensitive to network delay
•Significant savings over dialup, wide-area
•Even across overloaded “fast” networks
nReduction in total network traffic
nHelps overcom
e limitations in interface
Pathname lookup
Read (dir page “/”)
ClientD
isk
RPC
Lookup“/foo”
Pathname lookup
Find inodenum
ber
Read (dir page “/”)
Page Data
ClientD
isk
RPC
abc
21d
ef39
bar
52foo
40
Lookup“/foo”
Pathname lookup
Find inodenum
berPage D
ata
GetAttr (inode 40)
ClientD
isk
RPC
Lookup“/foo”
Read (dir page “/”)
Pathname lookup
Find inodenum
berPage D
ata
GetAttr (inode)
Attributes
ClientD
isk
RPC
Lookup“/foo”
Read (dir page “/”)
Pathname lookup
Find inodenum
berPage D
ata
GetAttr (inode)
Attributes
ClientD
iskClient
Disk
ExecScript (Read-G
etAttr)
Read page, find
inode number, get
attributes
RPC
SRPC
Lookup“/foo”
Lookup“/foo”
Read (dir page “/”)
Pathname lookup
Find inodenum
berPage D
ata
GetAttr (inode)
Attributes
ClientD
iskClient
Disk
ExecScript (Read-G
etAttr)
Read page, find
inode number, get
attributes
Attributes
RPC
SRPC
Lookup“/foo”
Lookup“/foo”
Read (dir page “/”)
Pathname lookup: Benefits
Pathname lookup: Benefits
Pathname lookup: Benefits
Pathname lookup: Benefits
Performance: Sum
mary
nExam
ples only illustrativen
Other “com
positions” possible too!n
Micro-benchm
arksn
Benefit due to reduced network roundtripsn
Macro-benchm
arksn
Postmark: 54%
less network trafficn
TPC-B: 96% less network traffic
nFacilitates working around m
inimal interfaces
Outline
nM
otivationn
Scriptable RPCn
Case Study: Active Storagen
Performance
nFunctionality
nSim
plicityn
Summ
ary
ScFS: Functionality enhancements
nIm
plement enhanced virtual protocols
over physical protocolsn
State can be added to stateless protocolsn
System provides prim
itives•
Clients compose them
into desired functionalityn
Examples
nAFS consistency sem
antics over NFSn
Sprite consistency semantics over NFS
Consistency semantics: NFS vs AFS
nNFSn
Stateless servern
Client checks periodically for updatesn
AFSn
Write-on-close sem
anticsn
Server tracks clients caching a file•
Notifies clients when modified file written
nRequires server-side state, participation
•Cannot im
plement using existing paradigm
s
Scripted AFS consistency
Client A
Client B
Scripted AFS consistency
Client A
Client B
ExecScript (AFS_OPEN
)
Open
Scripted AFS consistency
Client A
Client B
File Data
A
ExecScript (AFS_OPEN
)
Callback list
Cached file
Scripted AFS consistency
Client A
Client B
ExecScript (AFS_OPEN
)
Open
ACallback list
Cached file
Scripted AFS consistency
Client A
Client BFile D
ata
ABExecScript (AFS_O
PEN)
Callback list
Cached file
Scripted AFS consistency
Client A
Client BExecScript
(AFS_CLOSE_D
IRTY)
AB
File Close
Callback list
Cached file
Scripted AFS consistency
Client A
Client B
B
INVALID
ATE_CACHE
ExecScript
(AFS_CLOSE_D
IRTY)Callback list
Cached file
Functionality: Summ
aryn
SRPC: Powerfuln
Possible to add complex functionality
nEven those requiring augm
enting server staten
SRPC: Simple
nAFS consistency
•2 scripts, < 10 lines each
nSprite consistency
•3 scripts, < 20 lines each
nSim
ple base system, com
pact scripts to extend it
ScFS: Simplicity enhancem
ents
nAbility to group operations at servern
Simplifies im
plementation of atom
ic sets ofoperations
nO
ften, obviates need for distributed locks,distributed crash recovery
nExam
ple - concurrent directory updates
Concurrent directory updates
Client AClient B
abc
21d
ef39
Concurrent directory updates
Client AClient B
READab
c21
def
39
abc
21d
ef39
Create(“/foo”)
Concurrent directory updates
Client AClient B
READ
abc
21d
ef39
abc
21d
ef39
Create(“/foo”)Create(“/bar”)
abc
21d
ef39
Concurrent directory updates
Client AClient B
abc
21d
ef39
abc
21d
ef39
foo41
abc
21d
ef39
bar
52
Concurrent directory updates
Client AClient B
WRITE
abc
21d
ef39
foo41
abc
21d
ef39
foo41
abc
21d
ef39
bar
52
Concurrent directory updates
Client AClient B
WRITE
abc
21d
ef39
bar
52
abc
21d
ef39
foo41
abc
21d
ef39
bar
52
Concurrent directory updatesn
Non-scriptingn
Distributed locking, distributed crash recovery•
Clients acquire locks before read-modify-write
•Recover from
client failures while holding locksn
SRPCn
Script acquires in-mem
ory lock at servern
Just enforce mutual exclusion within single
address space
Summ
aryn
SRPCn
High Performance, rapid extensibility, sim
plicityn
Makes effective use of “active” architecture
nAll scripts less than 20 lines of coden
Some im
plement non-trivial functionality
nFewer lines of code =>n
Fewer bugs, more robust system
sn
Ease of building systems with active com
ponentsn
Don’t have a complex system
catering to allclient requirem
entsn
Provide primitives, enhance with com
pact scripts
Wisconsin N
etwork D
isks Group
http://ww
w.cs.w
isc.edu/wind
Questions ?