OceanStore/TapestryToward Global-Scale, Self-Repairing,

Secure and Persistent Storage

Anthony D. JosephJohn Kubiatowicz

Sahara Retreat, January 2003

OceanStore:2Sahara Retreat, January 2003 ©2003 UC Berkeley

OceanStore Context: Ubiquitous Computing

• Computing everywhere:– Desktop, Laptop, Palmtop– Cars, Cellphones– Shoes? Clothing? Walls?

• Connectivity everywhere:– Rapid growth of bandwidth in the interior of the net– Broadband to the home and office– Wireless technologies such as CMDA, Satelite, laser

• Where is persistent data????

OceanStore:3Sahara Retreat, January 2003 ©2003 UC Berkeley

Utility-based Infrastructure

Pac Bell

Sprint

IBMAT&T

CanadianOceanStore

IBM

• Data service provided by storage federation• Cross-administrative domain • Pay for Service

OceanStore:4Sahara Retreat, January 2003 ©2003 UC Berkeley

OceanStore Data Model• Versioned Objects

– Every update generates a new version– Can always go back in time (Time Travel)

• Each Version is Read-Only– Can have permanent name– Much easier to repair

• An Object is a signed mapping between permanent name and latest version– Write access control/integrity involves managing

these mappings

Comet Analogy updates

versions

OceanStore:5Sahara Retreat, January 2003 ©2003 UC Berkeley

The Path of an OceanStore Update

Second-TierCaches

Multicasttrees

Inner-RingServers

Clients

OceanStore:6Sahara Retreat, January 2003 ©2003 UC Berkeley

Archival Disseminationof Fragments

OceanStore:7Sahara Retreat, January 2003 ©2003 UC Berkeley

Tapestry

OceanStore:8Sahara Retreat, January 2003 ©2003 UC Berkeley

Enabling Technology: DOLR(Decentralized Object Location and

Routing)

GUID1

DOLR

GUID1GUID2

OceanStore:9Sahara Retreat, January 2003 ©2003 UC Berkeley

4

2

3

3

3

2

2

1

2

4

1

2

3

3

1

34

1

1

4 3

2

4

NodeID0xEF34

NodeID0xEF31NodeID

0xEFBA

NodeID0x0921

NodeID0xE932

NodeID0xEF37

NodeID0xE324

NodeID0xEF97

NodeID0xEF32

NodeID0xFF37

NodeID0xE555

NodeID0xE530

NodeID0xEF44

NodeID0x0999

NodeID0x099F

NodeID0xE399

NodeID0xEF40

NodeID0xEF34

Basic Tapestry MeshIncremental Prefix-based Routing

OceanStore:10Sahara Retreat, January 2003 ©2003 UC Berkeley

Use of Tapestry MeshRandomization and

Locality

OceanStore:11Sahara Retreat, January 2003 ©2003 UC Berkeley

Stability under Faults• Instability is the common case….!

– Small half-life for P2P apps (1 hour????)– Congestion, flash crowds, misconfiguration,

faults

• Must Use DOLR under instability!– The right thing must just happen

• Tapestry is natural framework to exploit redundant elements and connections– Multiple Roots, Links, etc.– Easy to reconstruct routing and location

information– Stable, repairable layer

• Thermodynamic analogies: – Heat Capacity of DOLR network– Entropy of Links (decay of underlying order)

OceanStore:12Sahara Retreat, January 2003 ©2003 UC Berkeley

Single Node Tapestry

Transport Protocols

Network Link Management

Application Interface / Upcall API

OceanStoreApplication-LevelMulticast

OtherApplications

RouterRouting Table

&Object Pointer DB

Dynamic Node

Management

OceanStore:13Sahara Retreat, January 2003 ©2003 UC Berkeley

It’s Alive! • Planet Lab global network

– 98 machines at 42 institutions, in North America, Europe, Australia (~ 60 machines utilized)

– 1.26Ghz PIII (1GB RAM), 1.8Ghz PIV (2GB RAM)– North American machines (2/3) on Internet2

• Tapestry Java deployment– 6-7 nodes on each physical machine– IBM Java JDK 1.30– Node virtualization inside JVM and SEDA– Scheduling between virtual nodes increases

latency

OceanStore:14Sahara Retreat, January 2003 ©2003 UC Berkeley

Segments of OceanStorewithin Sahara Retreat

• Today (5:30pm – 6:00pm)– OceanStore Long-Term Archival Storage

• Tomorrow morning session (8:30am-10:00am)– Tapestry status and deployment information

OceanStore:15Sahara Retreat, January 2003 ©2003 UC Berkeley

For more info:http://oceanstore.org

• OceanStore vision paper for ASPLOS 2000“OceanStore: An Architecture for Global-Scale

Persistent Storage”

• OceanStore Prototype (FAST 2003):“Pond: the OceanStore Prototype”

• Tapestry algorithms paper (SPAA 2002):“Distributed Object Location in a Dynamic Network”

• Upcoming Tapestry Deployment Paper (JSAC)“Tapestry: a Global-Scale Overlay for Rapid Service Deployment”

• Probabilistic Routing (INFOCOM 2002):“Probabilistic Location and Routing”

• Upcoming CACM paper (not until February):– “Extracting Guarantees from Chaos”