Internet Backplane Protocol:Storage in the Network

James S. Plank, Micah Beck,

Wael Elwasif, Terry Moore,

Martin Swany, Rich Wolski

University of Tennessee

The End-to-End Model:

There is no state managementin the Internet.

The Main Issue:

No State Management:A Good Thing

• It scales.

• It has a clean failure model.

• Resources used in a transient manner.

It has worked in the past. It works now.It will work in the future.

No State Management:A Bad Thing

• Performance suffers.

• End-to-end retransmissions required.

• Locality cannot be exploited.

• Data movement cannot be managed.

• Balkanization of roll-your-own state management techniques.

IBP: The Internet Backplane Protocol

• Managing and using state in the network.

• Inserting storage in the network so that:– Applications may use it advantageously.– Storage owners do not lose control of their

resources.

• Deep-sixing the End-to-End model!

Provides low-level primitives and software for:

What is in This Talk

• The Software Structure of IBP and its API.

• IBP Applications and Projects

• Deployment: The Logistical Backbone

Our Overarching Buzzword:

Logistical Networking

Allows an application to manage the trajectory of data in space and time as

it travels across the network.

The Byte Array:IBP’s Unit of Storage

• You can think of it as a buffer.

• You can think of it as a “file”.

• Append-only semantics.

• Transience built in.

The IBP Client API

• Can be used by anyone who can talk to the server.

• Seven procedure calls in three categories:– Allocation (1)– Data transfer (5)– Management (1)

Client API: Allocation

• IBP_allocate(char *host, int maxsize, IBP_attributes attr)

• Returns a trio of capabilities.– Read / Write / Manage– ASCII Strings (obfuscated)

• No user-defined file names:– Big flat name space.– No registration required to pass capabilities.

Allocation Attributes

• Permanent or Time-Limited

• Volatile or Stable

• Read/Write semantics:– Byte Array– Pipe– Circular Queue

Client API: Data Transfer

• IBP_store(IBP_cap wr_cap, char *data, int size)• IBP_read(IBP_cap rd_cap, char *buf, int size,

int offset)

• IBP_remote_store(IBP_cap wr_cap, char *host, int port, int size)

• IBP_deliver(IBP_cap rd_cap, char *host, int port, int size, int offset)

• IBP_copy(IBP_cap source, IBP_cap target, int size, int offset)

2-party:

3-party:

IBP Client API: Management

• IBP_manage(IBP_cap man_cap, int cmd, IBP_status info)

• Allows for resizing byte arrays.

• Allows for extending/shortening the time limit on time-limited allocations.

• Manages reference counts on the read/write capabilities.

• Some simple state probing.

IBP Servers

• Daemons that serve local disk or memory

• Root access not required.

• Can specify sliding time limits or revokability.

• Encourages resource sharing.

Logistical Application Strategies

The Goal:

To improve application performance, resource utilization or application

functionality as a result of logistical networking.

Strategy #1: Keep data close to the sender

• Network Weather Service sensor data collection (Wolski).

• Checkpoint servers (Plank, Agbaria).

Sender ReceiverIBP Network

Strategy #2: Place data close to the receiver

• IBP Mail (Elwasif).

• Speculative HTTP Transfer (Beck).

• NWS Client Data Acquisition (Wolski).

Sender ReceiverIBPNetwork

Strategy #3: Utilize transient storage throughout

• “Logistical Session Layer” (Swany).

Sender ReceiverIBPIBP IBP

Strategy #4: Optimized Producer/ConsumerC

lien

t S1

S2

Network

Cli

ent S1

S2

Network

IBP

Add Logistical Scheduling to NetSolve(The LoCI Team)

The Logistical Backbone (L-Bone)

• Goal: To provide ready storage for experi-mental logistical networking applications.

• Composed of dedicated and lent resources running IBP.

• Currently: – 5 I2-DSI machines + few machines at Tennessee

To Conclude

• Logistical Networking: Managing data movement and locality on the network.

• IBP: A key piece of software infrastructure for Logistical Networking.

• IBP Applications: Demonstrate the effective-ness of IBP and Logistical Networking.

• L-Bone: Experimental public test facility.

You Can Play Too

• IBP code available– http://www.cs.utk.edu/~plank/IBP

• L-Bone participation– http://www.icl.utk.edu/ibp

• Working IBP Applications– IBP Mail (talk later today)– NWS client data acquisition– Checkpointing in NetSolve

Internet Backplane Protocol:Storage in the Network

James S. Plank, Micah Beck,

Wael Elwasif, Terry Moore,

Martin Swany, Rich Wolski

University of Tennessee