1

The Design of a RobustPeer-to-Peer System

Gisik Kwon

Dept. of Computer Science and Engineering

Arizona State University

Reference: SIGOPS European Workshop 25/09/2002

Rodrigo Rodrigues(MIT)

Arizona State University2

Scalable Computing Lab.

Motivation

What if we wanted to store the state of the application in a P2P system?

E.g., mail service, archiving Need a robust P2P storage system Current algorithms do not provide reliable

service:– No admission control– Trust configuration information– Vulnerable to malicious routing– Tolerance to Byzantine faults

Arizona State University3

Scalable Computing Lab.

Architecture

ConfigurationService P2P Nodes

Configuration Services(CS)– Selected statically(a subset of P2P nodes) or dynamically

P2P nodes – set of servers, not clients

Common to equip with– Co-processor, read-only disk, watch-dog timer, Cryptographic co-

processor

Arizona State University4

Scalable Computing Lab.

Configuration Service

Four main functions:– Admission control– Node monitoring– Deciding on a new configuration– Propagating configuration information

CS nodes carry BFT protocol [CL99]

Arizona State University5

Scalable Computing Lab.

Admission Control

Prevents single user controlling large fraction of the ID space

Maintains the list of node– <ID, IP, public key>

Hard to do in volunteer-based system We assume servers Nodes have public / private keys

Arizona State University6

Scalable Computing Lab.

Node Monitoring

Detection fault node– Fail-stop failure: ping protocol– Byzantine failure: hard!

proactively recovered frequently -> restart from the correct code in read-only disk -> copy state from other nodes

All CS do its own monitoring for all P2P nodes

Arizona State University7

Scalable Computing Lab.

Configuration Information

What to propagate?– Incomplete config is a limitation of p2p– Disseminate entire config– Transmit using diffs

When to propagate?– Server machines: not very often (e.g., every hour)– Configs include start and expiration times

Periodically deciding the new config– Using non-deterministic choices validation

Send eviction request to other CS -> validation

Arizona State University8

Scalable Computing Lab.

Architecture – P2P Nodes

Lookup layer– Periodically receive the latest config from CS– Notify to storage layer

Storage layer– Uses Byzantine quorums to store data– Data placement algorithm assigns 3 f + 1 replicas to each data

item

Lookup

Storage

Application

Lookup

Storage

Lookup

Storage

Client Server Server

Arizona State University9

Scalable Computing Lab.

Correctness Condition

“At any moment, any group of 3 f +1 replicas of a data item contains no more than f faulty replicas”

Reconfiguration must be frequent enough to preserve this invariant

Arizona State University10

Scalable Computing Lab.

State Transfer

During reconfiguration, set of replicas may change

Replica finds out of new configuration:– Know old configuration– Pulls data items from old replicas (as seen before)