1 first and second generation of peer-to-peer systems my group or university title of the lecture...

1First and Second Generation

of Peer-to-Peer SystemsMy group or university

Title of the lectureTitle of the lecture

Chapter 5:

First and Second Generation First and Second Generation of Peer-to-Peer Systemsof Peer-to-Peer Systems

Peer-to-Peer Systems and Applications, Springer LNCS 3485

*Original slides provided by Rüdiger Schollmeier and Jörg Eberspächer (Technische Universität München)

Somename University Department for File-Sharing and Indexing



X. Overview

1. General Characteristics of Early Peer-to-Peer Systems

2. Centralized Peer-to-Peer Networks1. Basic Characteristics2. Signaling Characteristics3. Discussion

3. Pure Peer-to-Peer Networks1. Basic Characteristics2. Signaling Characteristics3. Discussion

4. Hybrid Peer-to-Peer Networks1. Basic Characteristics2. Signaling Characteristics3. Discussion



X. Overview







X.1 General Characteristics of 1st And 2nd Gen. P2P

1st and 2nd Generation P2P systems are overlay architectures, with the following characteristics:

TCP/IP based Decentralized and self organizing (with possible centralized elements) Content:

Distributed “randomly” on the network, with several replicas content and its descriptions are not structured (stays at the nodes which bring it

into the network) Content transfer:

Out of band, i.e. on separate connections and not via signaling connections Mostly via HTTP

Employ distributed shared resources (computing power and data storage) Generally two kinds of requests:

Content requests: to find content in the overlay Keep-alive requests: stay connected in the overlay

Initially developed for file-sharing Various realizations exist



X.1 The Architectures of 1st and 2nd Gen. P2P

Client-Server Peer-to-Peer

1. Server is the central entity and only provider of service and content. Network managed by the Server

2. Server as the higher performance system.

3. Clients as the lower performance system

Example: WWW

1. Resources are shared between the peers

2. Resources can be accessed directly from other peers

3. Peer is provider and requestor (Servent concept)

Unstructured P2P Structured P2P

Centralized P2P Pure P2P Hybrid P2P DHT-Based

1. All features of Peer-to-Peer included

2. Central entity is necessary to provide the service

3. Central entity is some kind of index/group database

Example: Napster


2. Any terminal entity can be removed without loss of functionality

No central entities

Examples: Gnutella 0.4, Freenet



dynamic central entities

Example: Gnutella 0.6, JXTA



No central entities

4. Connections in the overlay are “fixed”

Examples: Chord, CAN

1st Gen. 2nd Gen.



X.1 Basic Topology Characteristics

Peer:

Node actively participating in the overlay (PC running Kazaa)

content provider+content requestor+router in the overlay network

Identifiable by a General Unique ID (hash-value of “unique” or random ID)

Overlay Topology: “virtual” signaling network established via TCP connections between the peers

Characteristics of the overlay topology:

completely independent from physical network, because of the abstraction layer TCP/IP

May include hierarchies (rendezvous peers in JXTA) (hub network)

May include centralized elements (lookup server in Napster) (star network)

May be a completely randomized network (Gnutella 0.4) which can be modeled with random graphs (randomly meshed network)

Separate addressing scheme



X.1 Basic Routing Behavior

Request messages: Include a hop-counter, a GUID and a TTL (Time-to-Live) in the header

TTL determines along how many hops a message may be forwarded

Are flooded in the overlay network

Every node forwards every incoming message to all neighbors except the neighbor, it received the message from

Exceptions: see below

Request messages terminate, if

Same message-type with same GUID is received more than once (loop!!)

Hop-counter=TTL

Response messages: Include a hop-counter, a GUID and a TTL (Time-to-Live) in the header

GUID is the same as of the initializing request message

Are routed back on the same way to the requestor, the request message was transmitted to the responding peer

every peer has to store the GUID of each request for a certain amount of time

No flooding to save resources



X.1 Basic Bootstrapping

Mostly not part of the protocol specification

Necessary to know at least one active participant of the network

Otherwise no participation at the overlay possible for a new node

Address (TCP) of an active node can be retrieved by different means: Bootstrap cache: Try to establish one after another a connection to a node seen in a previous

session

Bootstrap server:

Connect to a “well known host”, which almost always participates

Ask a bootstrap server to provide a valid address of at least one active node

Realizations:

FIFO of all node-addresses which recently used this bootstrap (a node which just connected is assumed to be still active)

Random pick of addresses which recently connected via this server to the overlay (+ no loops, -may be outdated)

Broadcast on the IP layer

Use multicast channels

Use IP broadcasting (-limited to local network)



X. Overview







X.2 Definition of centralized P2P

All peers are connected to central entity Peers establish connections between each other on demand to

exchange user data (e.g. mp3 compressed data) Central entity is necessary to provide the service Central entity is some kind of index/group database Central entity is lookup/routing table



X.2 Basic Characteristics of centralized P2P

Bootstrapping: Bootstrap-server = central serverCentral entity can be established as a server farm, but

one single entry point = single point of failure (SPOF)All signaling connections are directed to central entityPeer central entity: P2P protocol, e.g. Napster

protocol To find content To log on to the overlay To register To update the routing tables To update shared content information

Peer Peer: HTTP To exchange content/data



X.2 Topology of Centralized P2P

ServentConnection between 2 servents (TCP)

Connection between router & serventConnection between routers (Core)



X.2 Example: Napster

Program for sharing files over the Internet A “disruptive” application/technology? Brief History:

May 1999: Shawn Fanning (freshman, Northeastern University) founds

Napster Online music service

December 1999: First Lawsuit

March 2000: University of Wisconsin reports that 25% of its IP traffic is

Napster traffic

December 2000: estimated 60 million users

February 2001: US Circuit Court of appeals: napster knew users violating

copyright laws

Shut down of the service



X.2 Napster: How Does It Work

CentralNapsterIndexserver

Data Transfer

NapsterHost

NapsterHost

NapsterHost

NapsterHost

Application-level, client-server protocol over point-to-point TCP Partcipants:

Napster Hosts/peers Client Service

Login Data-requests Download-requests

P2P Service Data-transfer

Napster Indexserver Pure Server

Five steps: Connect to Napster Server Upload your list of files (push) to server Query Indexserver with a list of keywords to search the full list with Select “best” of correct answers Connect to providing host/peer



X.2 Napster: Wrap-Up



X.2 Further P2P Systems based on a centralized Structure

Audiogalaxy

WinMX

BiTorrent



X.2 Summary of Napster Siganling

NapsterPeer (Req)

NapsterServer

NapsterPeer (Prov)

Login: [0x24|0x02|…]

Login Ack: [0x00|0x03|…]

HTTP: GET[Filename]

OK[data]

Notif: [0x46|0x64|…]

Notif: [0x46|0x64|…]

Notif: [0x46|0x64|…]

Search: [0x7E|0xC8|…]

Response: [0xC4|0xC9|…]

Response: [0xC4|0xC9|…]

Sample message sequence chart for one Napster server with one requesting and one providing peer



X.2 Discussion

Disadvantages Single Point of Failure easily attackable

Bottleneck

Potential of congestion

Central server in control of all peers

Advantages Fast and complete lookup (one hop lookup)

Central managing/trust authority

No keep alive necessary, beyond content updates

Application areas VoIP (SIP, H.323)

Auctioning (Ebay)



X. Overview







X.3 Definition of Pure P2P

Any terminal entity can be removed without loss of functionality No central entities employed in the overlay Peers establish connections between each other randomly

To route request and response messages

To insert request messages into the overlay



X.3 Basic Characteristics of Pure P2P

Bootstrapping: Via bootstrap-server (host list from a web server) Via peer-cache (from previous sessions) Via well-known host No registration

Routing: Completely decentralized Reactive protocol: routes to content providers are only established on

demand, no content announcements Requests: flooding (limited by TTL and GUID) Responses: routed (Backward routing with help of GUID)

Signaling connections (stable, as long as neighbors do not change): Based on TCP Keep-alive Content search

Content transfer connections (temporary): Based on HTTP Out of band transmission



X.3 Topology of Pure P2P

ServentConnection between 2 servents (TCP)

Connection between router & serventConnection between routers (Core)



X.3 Example: Gnutella 0.4

Program for sharing files over the InternetFocus: decentralized method of searching for filesA “disruptive” application/technology?Brief History:

March 2000: open source release by by Justin Frankel and Tom Pepper of Nullsoft, a division of AOL, and almost immediately withdrawn

Spring 2001: further developments to improve scalability Gnutella 0.6 (Hybrid P2P)

Since then: available in a lot of implementations (Limewire, bearshare,…) Developed further on (privacy, scalability, performance,…)



X.3 The Gnutella Network

Measurements taken at the LKN in May 2002



X.3 Gnutella: How Does It Work

Application-level, peer-to-peer protocol over point-to-point TCPPartcipants:

Gnutella peers/servents Router Service

Flood incoming requests (regard TTL!)

Keep alive content

Route responses for other peers(regard GUID of message)

Keep alive (PING/PONG) Content (QUERY/QUERYHIT)

Data-requests Download-requests

Lookup Service Initialize Data requests Initialize keep alive requests

“Server”-Service Serve Data-requests (HTTP)

Five steps: Connect to at least one active peer (address received from bootstrap) Explore your neighborhood (PING/PONG) Submit Query with a list of keywords to your neighbors (they forward it) Select “best” of correct answers (which we receive after a while) Connect to providing host/peer

G

G

G

G

G

G

G

TCP connection

G

Peer/Servent

G

G

G

G



X.3 Gnutella Routing

• Flooding: Received PINGS and QUERIES must be forwarded to all connected Gnodes

• PINGS or QUERYS with the same FUNCTION ID and GNODE ID as previous messages are destroyed (avoid loops)

•Save Origin of received PINGs and QUERIEs

•Decrease TTL by 1

•If TTL equals 0, kill the message

• PONG and QUERY HIT are forwarded to the origin of the according PING or QUERY

• Basic Routing Principle: „Enhanced“ Flooding



X.3 Gnutella Connection Setup

GNODE ID: 2000IP: 002




2123

2625

17

2224

17Gnutella Connect

18Gnutella OK

19PING 20PONG/IP:004


27PONG/IP:001


28PONG/IP:003

25PING

26PING

18

19202728

Gnode 2000 establishes a connection to 4000



X.3 Gnutella Wrap-Up



X.3 Further P2P Systems Based on Pure P2P

Freenet Focus on anonymity and privacy

Content is transferred in chunks over the signaling network

Gnunet



X.3 Summary of the Signaling in Gnutella 0.4

1

7

3

2

4

5

6

8

Gnu-ConGnu-Con

Peer7 Peer3 Peer1 Peer5 Peer2 Peer4 Peer6

Gnu-ConOKOK

OK

PINGPING

PINGPINGPING PING

PING

PONG

PINGPING

PING

PONG

PONG

PONG

Peer8

PING

PONG

PONG

PONG

PONG

PONGPONG

PONG

PONG

PING

Sample Gnutella 0.4 network:

Sample message sequence chart according to the sample network:



X.3 Discussion

Disadvantages High signaling traffic, because of decentralization Modem nodes may become bottlenecks Overlay topology not optimal, as

no complete view available, no coordinator

If not adapted to physical structure delay and total network load increases Zigzag routes loops

Advantages No single point of failure Can be adapted to physical network Can provide anonymity Can be adapted to special interest groups

Application areas File-sharing Context based routing (see chapter about mobility)



X. Overview







X.4 Definition of Hybrid P2P

Main characteristic, compared to pure P2P: Introduction of another dynamic hierarchical layer

Hub based network Reduces the signaling load without reducing the reliability Election process to select an assign Superpeers Superpeers: high degree (degree>>20, depending on network

size) Leafnodes: connected to one or more Superpeers (degree<7)

Superpeer

leafnode



X.4 Model of Hybrid P2P Networks

1.4

11.4

, 1 7

1 0.05, 1,

0.05, 20

0,

: 2.8var 3.55

d

c d d

p dc dp d with c

cc d

in any other case

average dd

Degree distribution:

According sample graph:

Major componen

t

Separate sub

networks

Hub connections

(2nd hierarchy)Superpeer

leafnode



X.4 Basic Characteristics of Hybrid P2P

Bootstrapping: Via bootstrap-server (host list from a web server) Via peer-cache (from previous sessions) Via well-known host Registration of each leafnode at the Superpeer it connects to, i.e. it announces its shared files to

the Superpeer Routing:

Partly decentralized Leafnodes send request to a Superpeer Superpeer distributes this request in the Superpeer layer If a Superpeer has information about a matching file shared by one of its leafnodes, it sends this information

back to the requesting leafnode (backward routing) Hybrid protocol (reactive and proactive): routes to content providers are only established on

demand; content announcements from leafnodes to their Superpeers Requests: flooding (limited by TTL and GUID) in the Superpeer layer Responses: routed (Backward routing with help of GUID)

Signaling connections (stable, as long as neighbors do not change): Based on TCP Keep-alive Content search

Content transfer connections (temporary): Based on HTTP Out of band transmission (directly between leafnodes)



X.4 Topology of Hybrid P2P

43

39

7

100

3

118116

18

39

118

7

116

3, 43

18100

Abstract network structure of a part of the Gnutella network (222 nodes Geographical view given by Figure on the right, measured on 01.08.2002

Geographical view of a part of the Gnutella network (222 nodes); The numbers depict the node numbers from the abstract view ( Figure on the left, measured on 01.08.2002)

• Virtual network not matched to physical network. See path from node 118 to node 18.• Superpeer (hub) structure clearly visible in abstract view

• Virtual network not matched to physical network. See path from node 118 to node 18.• Superpeer (hub) structure clearly visible in abstract view



X.4 Example: Gnutella 0.6

Program for sharing files over the InternetFocus:

decentralized method of searching for files

Higher signaling efficiency than Pure P2P

Same reliability (no single point of failure)

Basis of most file-sharing applications (not BitTorrent)Brief History:

Spring 2001: resulted from Gnutella 0.4 by further developments to

improve scalability Gnutella 0.6 (Hybrid P2P)

Since then: available in a lot of implementations (Limewire, bearshare,…)

Developed further on (privacy, scalability, performance,…)



X.4 Gnutella 0.6 Network Organization

New connection/network setup Upon connection to the network via a Superpeer, each node is a

leafnode

It announces its shared content to the Superpeer it connected to

Superpeer thus updates its routing tables

Election mechanism decides which node becomes a Superpeer or a

leafnode (depending on capabilities (storage, processing power)

network connection, the uptime of a node,…), if Too many nodes are connected to one Superpeer

A Superpeer leaves the network

To less nodes are connected to a Superpeer



X.4 Gnutella 0.6 Routing

Content requests: Leafnode sends request to Superpeer Superpeer looks up in its routing tables whether content is offered by one of its leafnode. In this

case the request is forwarded to this node. Additionally the Superpeer increases the hopcounter and forwards this request to the Superpeers it

is connected to. To enable backward routing, the peer has to store the GUID of the message connected to the

information from which peer it received the request in the previous hop If a Superpeer receives such a request from another Superpeer, this request is handled the same

way, as if it would have received it from one of its leafnodes After the hopcounter of the request reaches the TTL-value it is not forwarded any further (prevent

circles) Content responses:

If a leafnode receives a request, it double-checks whether it shares the file (should be the case, as long as the routing tables of the Superpeer are correct)

In case of success, the leafnode sends a content reply back to the requesting peer, by sending it back to that node (Superpeer) it received the message from (backward routing)

Hop by hop the message can thus be routed back to the requesting node Content exchange:

Directly between the leafnodes, via HTTP connections



X.4 Gnutella 0.6: How Does It Work



X.4 Further P2P systems based on hybrid P2P

EdonkeyKazaa/FastTrackEmuleOpenNap…



X.4 Summary of the Signaling in Gnutella 0.6

Sample Gnutella 0.6 network:

Sample message sequence chart according to the sample network:

4L1 L3

S2

S3

S1

L2

L5

L4

L6

L7

Gnu-Con

L2 L3 L1 S1 S3 S2 L7

OK

PONG

L6 L5 L4

RTU

PING

PONGPONG

PING

PINGPONG

PONG

PINGPING

QUERYQUERY

QUERY

QUERYQUERY

QUERYQUERY

QUERY

QUHIT

QUHIT QUHITQUHIT

QUHITQUHIT

QUHITQUHIT



X.4 Discussion

Disadvantages Still High signaling traffic, because of decentralization No definitive statement possible if content is not available or not found Modem nodes may become bottlenecks Overlay topology not optimal, as

no complete view available, no coordinator

If not adapted to physical structure delay and total network load increases Zigzag routes Loops

Can not be adapted to physical network completely because of hub structure Asymmetric load (Superpeers have to bear a significantly higher load)

Advantages No single point of failure Can provide anonymity Can be adapted to special interest groups

Application areas File-sharing Context based routing (see chapter about mobility)



X. Summary

P2P technologies offer an innovative overlay infrastructure for decentralized and distributed systems

Due to the distributed nature, the signaling load is very high, but it can be reduced with introduction of hierarchies, compression and geo-sensitive protocols

Advantages: Simple basic principle Enhanced reliability Redundancy (high replication rate) Unsusceptible against Denial of Service attacks (DOS) No single point of failure No central instances/administration Direct and instantaneous communication possible Large variety of applications possible



X.3 Model of Pure P2P Networks

Major componen

t

Separate sub

networks

11.4 , 0 7,

0,

: 2.2var 1.63

d

p dc d dp d with c

cin any other case

average dd

Degree distribution:

According Sample Graph:



X.2 Napster Message Structure

<Payload Length>2byte

<Function>2Byte

HEADER 4byte PAYLOAD

General Header Structure:

Describes the message type (e.g. login, search,…)

Describes parameters of the message (e.g. IDs, keywords,…)



X.2 Napster: Initialization

<Nick> <Password>

<Port>

1: LOGIN (Function:0x02)

NapsterHost

IP: 001Nick: LKN

<Client-Info> <Link-type>

LOGIN(0x02)

lkn 54332 6699 „nap v0.8“ 9

LOGIN ACK(0x03)

2: LOGIN ACK (Function: 0x03)

„<Filename>“

<MD5>

3: NOTIFICATION OF SHARED FILE (0x64)

<Size> <Bitrate>

<Freq> <Time>

NOTIFICATION(0x64)

„band - song.mp3“ 3f3a3... 5674544

128 44100 342


Client/Server Service



X.2 Napster: File Request Procedure

[FILENAME CONTAINS „Search Criteria“]

[LINESPEED <Compare> <Link-Type>]

1: SEARCH (Function: 0xC8)

[BITRATE <Compare> “<Bitrate>”]

SEARCH(0xC8)

FILENAME CONTAINS „song“ MAX_RESULTS 100

LINESPEED „AT LEAST“ 6 BITRATE „AT LEAST“ „128“

FREQ „EQUAL TO“ „44100“

[FREQ <Compare> “<Freq>”]

[MAX_RESULT <Max>]

NapsterHost

IP: 002Nick: MIT

2: SEARCH RESPONSE (Function: 0xC9)

„<Filename>“ <MD5> <Size> <Bitrate> <Freq>

<Time> <Nick> <IP> <Link-Type>




X.3 Gnutella Message Structure


Describes the message type (e.g. login, search,…)

Describes parameters of the message (e.g. IDs, keywords,…)


GnodeID16 Bytes

Function1 Byte

MESSAGEHEADER: 23Byte

TTL1 Byte

Hops1 Byte

Payload Length4 Bytes

• GnodeID: unique 128bit Id of any Hosts

• TTL(Time-To-Live): number of servents, a message may pass before it is killed

• Hops: number of servents a message already passed



X.3 Gnutella Messages

Port2 Bytes

IP Address4 Bytes

PING (Function:0x00)

Nb. of shared Files4 Bytes

Nb. of Kbytes shared4 Bytes

No Payload

PONG (Function:0x01)

Minimum Speed2 Bytes

Search Criterian Bytes

QUERY (Function:0x80)

Nb. of Hits1 Byte

Port2 Bytes

GnodeID16 Bytes

Result Setn Bytes

QUERY HIT (Function:0x81)

Speed1 Byte

File Index4 Bytes

File Namen Bytes

IP Address4 Bytes



X.4 Gnutella 0.6 Messages

Content requests and responses QUERY (defined as in Gnutella 0.4) QUERY_HIT (defined as in Gnutella 0.4)

Keep alive: PING (defined as in Gnutella 0.4) PONG (defined as in Gnutella 0.4)

Announcement of shared content: ROUTE_TABLE_UPDATE (0x30), Reset variant (0x0): to clear the routing table and to

set a new routing table for one leafnode

ROUTE_TABLE_UPDATE (0x30), Patch variant(0x1): to update and set a new routing table with a certain number of entries (e.g. new shared files)

0 1 4 5 Variant Table_Length Infinity

0 1 2 3 4 5 n+4 Variant Seq_No Seq_Size Compressor Entry_Bits DATA

1 first and second generation of peer-to-peer systems my group or university title of the lecture...

Documents

peer systems peer

generation of peer

peer networks

centralized peer

features of peer

pure peer

hybrid peer

peer systems title