stream spider distributed music streaming service with sliced music files from different originating...
TRANSCRIPT
Stream SpiderDistributed Music Streaming Service with Sliced
Music Files from Different Originating Hosts
Justin Steffy and Michael D. Elder
Outline
• Service Operation• Design
– Peer to Peer Client and Service– Availability and Fault Tolerance– Consistency and Correctness
• Implementation– Foundations: Derby, Jetty, Eclipse RCP– Protocols (HTTP/XML)
• Demonstration
System Operation
• Register with service to find a set of peers with music to share
• Search for particular files on other peers
• Play music while streaming contents from multiple peers
Design
• Peer to Peer protocol– Peers can search and request files directly– Peers also service search and file requests– Two levels of peer
• Highly transient peers– Weak consistency– Sporadic availability
• Maintain state of peers on the network
Peers
• Types of Peers– Service Peer
• Executes and responds to file requests• Requests and services music streams• Maintains small “neighbor set” to service requests
– Bootstrap Peer: • Receives initial registration for new Peers• Replicates available registrations• Maintain consistency with other Bootstrap peers• Can also act as a Service Peer
Diagram: Registration
Register A List of Peers
Register C
Register B
Peer B
Peer A
Peer C Bootstrap Peer
List of Peers including A
List of Peers including A & B
Diagram: File Search
Bootstrap Peer
Have files matching A?
List of Files
Peer A
Peer C
Peer B searches for songs that match A
Diagram: Music Streaming
Bootstrap Peer
Request Segment
Octet Stream of data
Peer A
Peer C
Request SegmentOctet Stream of data
Peer B
Availability and Fault Tolerance
• Availability does not depend on any one bootstrap peer– Replicate state among many peers– Loss of a bootstrap causes minimal effect
• Peers tolerate losses in neighbor set– When list gets low, request more peers– Losing a peer while streaming is handled
Important State
• Set of available peers (Shared) – Bootstrap Peers synchronize Peer locations– Service Peers request more peers as their
neighbor set leaves or fails– Peer availability is volatile state, maintained
relative to each Peer
• Spliced Music Streams – Recovery when an identified peer dies while
servicing a stream
Consistency and Correctness
• Bootstrap Servers enforce Weak Consistency when sharing available Peers
• Correctness Criteria– Peers should be able to register with one of
the active Bootstrap Peers– Peers should receive search results from
active peers in its neighbor set– Peers should receive music splices upon
request in order to stream music
Implementation
• Java and Eclipse– Eclipse UI– Database– Servlets– JUnit
• HTTP and XML basis for protocol– Object Models
Foundations
• Derby 10.1– Embedded Java Database– Maintains local state at each Peer
• Jetty 6.1– Embedded Java 2 EE Servlet Container– Act as protocol endpoints
• Eclipse Rich Client Platform (RCP)– Provides for rapid development of application architecture and
UI• Eclipse Modeling Framework (EMF)
– Provides an object model abstraction and XML serialization• JLayer 1.0
– MP3 playback• Milestone development with unit tests (~20)
Protocols
• Based on HTTP and XML• Registration
– Accepts a requested number of peers– Answers set of available peers
• Search– Accepts a filename – Answers set of files that match a portion of the input
• Fragment– Accepts a file id and fragment range– Answers the portion of the file
Registration
• Request – HTTP Query String<Hostname, Port, Number of peers>
• Response - XML<PeerList>
<Peer1 …/><Peer2 …/>…
</PeerList>
Search
• Request – HTTP Query String<Filename>
• Response - XML<FileList>
<File1 …/><File2 …/>…
</FileList>
Fragment Request
• Request – HTTP Query String<FileID, Start, Length>
• Response – Octet Stream– Beginning at start for length bytes
Consistency Protocols
• Status – Determine peer availability– Send status request to a peer– Peer returns a current load value
• Synchronization (Bootstrap only)– Keep list of other bootstrap peers– In round robin style, send all newest peers
seen at this peer (HTTP Post)– Return the new peers seen
Bootstrap Synch
POST with XML Peer List
Return an XML Peer List
•Get last used time of Peer B
•Find all newer peers
•Construct XML and POST to Peer B
•Receive the response and add peers to our list
•Update the last used time of B
Peer A Peer B
•Receive the Peer List
•Add peers to our list
•Find last used time of Peer A
•Gather all peers newer than time
•Send a Peer List
•Update the last used time of A
Key API
• Net API – Contains all server and state– Database Manager – Handles peer state– Bootstrap Manager – Synchs peer state– Peer Manager – Keep alive with other peers– Protocol Requests – Encapsulate requests
• Servlets – Started and forgot by Net API– One per request type– Run independently of rest of system
Architecture
Core
UINet
Diagram: Metamodel
Diagram: Database
Diagram: Playback
Diagram: Net API
Diagram: Servlets
Diagram: Requests
Diagram: UI
Demonstration
• Normal Operation
• Failure of bootstrap
• Failure of service peer