P2P: Grid-Oriented Distribution Systems

KCCMG Fall Impact 2005

Lorin Olsen, Sprint Nextel

History of File Sharing

• 1980’s: Corporate file sharing (anyone remember IBM MVS/BDT???) and initial IP-based file transfer protocols for DoD use

• 1990’s: FTP used for point-to-point file transfers• 1999: Shawn Fanning releases Napster• March 2000: Justin Frankel releases Gnutella;

"See? AOL can bring you good things!"• March 2001: Niklas Zennström and Janus Friis

introduce KaZaA• Defcon 2001 / CodeCon 2002: Bram Cohen

shows off BitTorrent

Client-Server File Sharing

Napster• You started the Napster software on

your machine. Your machine became a small server able to make files available to other Napster users.

• Your machine connected to Napster's central servers. It told the central servers which files were available on your machine. So the Napster central servers had a complete list of every shared song available on every hard disk connected to Napster at that time.

• You typed in a query for a song. Let's say you were looking for the song "Roxanne" by The Police. Napster's central servers listed all of the machines storing that song.

• You picked a version of the song from the list.

• Your machine connected to the user's machine that had that song, and downloaded the song directly from that machine.

“Napster’s Architecture, http://computer.howstuffworks.com/file-sharing1.htm

P2P Philosophy

“Many hands make light work” - Proverb

“Single servants are less powerful then a single server but the collective of many servants is more powerful then any single server” - Daniel Stephen Rule.

P2P Evolution

“For the people that aim to stop P2P, they have turned a centralized system like Napster – easily controlled, easily monitored – into a fully decentralized system in the form of Kazaa, as well as a fragmented ecosystem of thousands of centralized servers through BitTorrent. This was probably a bad decision.”

Monkey Methods Research Group

P2P Architecture

P2P Dark Side

“Too many cooks spoil the broth” - Proverb

“Arggghhhhh” – The sound Lorin makes when trying to coordinate the efforts of his four children

Gnutella

• Distributed query + lightweight content distribution

• Clients– BearShare– Gnucleus– LimeWire– Morpheus– WinMX– XoloX

BitTorrent• You open a Web page and click on a link for

the file you want.

• BitTorrent client software communicates with a tracker to find other computers running BitTorrent that have the complete file.

• The tracker identifies the swarm.

• The tracker helps the client software trade pieces of the file you want with other computers in the swarm.

• If you continue to run the BitTorrent client software after your download is complete, others can receive .torrent files from your computer; your future download rates improve because you are ranked higher in the "tit-for-tat" system.

“What BitTorrent Does”; How Stuff Works; http://computer.howstuffworks.com/bittorrent3.htm

BitTorrent Architecture

Thompson, Clive; “The BitTorent Effect”; Wired Magazine; January 2005

• A single source file within a group of BitTorrent users, called a swarm, spreads around pieces of a film or videogame or TV show so that everyone has a chunk to share.

• After the initial downloading, those pieces are then uploaded to other needy users in the swarm.

• Before long, the swarm has shared all the pieces, and everyone has their own complete source.

BitTorrent Terms• Peer: A peer is one instance of a BitTorrent client running on a computer on the

Internet that you connect to and transfer data. Usually a peer does not have the complete file, but only parts of it.

• Seed: A seed is a peer that has a complete copy of the torrent and still offers it for upload. The more seeds there are, the better the chances are for completion of the file.

• Leech A leech is a peer that does not have a complete copy of the torrent yet. When downloading is complete, it may stay around and seed the file as a seed so that others can complete their download. The term leech is also used for peers that have very poor upload/download ratios or leave the swarm immediately after their downloads are complete. The leeches usually contribute a majority of the bandwidth in a swarm.

• Swarm: Together, all peers sharing a torrent are called a swarm. Six leeches and two seeds makes a swarm of eight.

• Tracker: A tracker is a broker service that mediates contacts between peers. The tracker is not directly involved in the data transfer and does not have a copy of the file.

The Role of the Tracker

Azureus: A Client in Action

File Distribution

BitTorrent Traffic Patterns

• BitTorrent has taken an enormous share of Internet bandwidth

• BitTorrent usage increases during “off-prime” hours (5PM-6AM)

Legal Uses of BitTorrent

• The demo of the flight sim X-Plane is offered via BitTorrent, as well as the World of Warcraft beta and its patches.

• PlaneShift is a free open-source MMORPG, which uses BitTorrent for its primary method of distribution.

• The fan-film Star Wars: Revelations is distributing two DVD images as well as the film by itself via BitTorrent.

• The NetBSD operating system version 1.6.2 and later as well as most major Linux distributions use BitTorrent as an alternative way of distributing ISO images of their releases.

“BitTorrent”; http://en.wikipedia.org/wiki/Bittorrent

References

Cohen, Bram; “Incentives Build Robustness in BitTorrent”; 5/22/2003

Ewing, J & Lowry,T (2004); “It Seemed Like A Good Idea”; Business Week, New York; Aug 2, 2004, Iss. 3894; pg. 60

Pouwelse, Johan; “The BitTorrent P2P file-sharing system”; 12/18/2004; http://www.theregister.co.uk/2004/12/18/bittorrent_measurements_analysis/

Zihui Ge, Daniel R. Figueiredo, Sharad Jaiswal, Jim Kurose, Don Towsley; “Modeling Peer-Peer File Sharing Systems”; IEEE Infocom 2003;