BitTorrent Framework

Background

• BT is a very popular peer to peer protocol with many implementations:– http://en.wikipedia.org/wiki/

Comparison_of_BitTorrent_clients

• The first one was created in Python by Bram Cohen.

Workshop goal

• Exploit BitTorrent protocol to give less for more.

• Real world motivation– Places with low upload (Africa, China)– Bad character

How do I test my idea

• Create my own (or modified) BitTorrent client.

• Run together:– “The World”: N “standard” BitTorrent Clients.

• Specific upload rate• Various download stages

– My Client

• See how much time my client ran, and what was its upload / download ratio.

Execution example

run-all.sh

Tracker

BTBTBT

BTBT

BTBT

BT

BTBT

BT

How does it work?

• Run tracker

• Run the “world”

• Run your client (if given)

How does it work?

• Both tracker and the “world” run on the same computer

• we have – pc-hanoch-w1– pc-hanoch-w2– Please reserve time in:

• http://0368350023.wikispaces.com/• Or from course www - "Scheduling computers for

tests."

The main script - run_all.sh

• Location:– /users/courses/hanochworkshop/bittorrent

• Run_all.sh:– Parameters:

• Start / stop• My client – optional

run_all.sh operation

• Start:– Create download directory– Run the “world”

• Each world client starts with different “saved” amount.

– Run “my client” – if given– All uploads rates are restricted.

• Stop– Kill all python threads

To run client separately:

• BitTornado-My-Client/btdownloadheadless.py – --saveas /vol/scratch/bt-download/picture.tst.bmp – --max_upload_rate 70 – --security 0 – ./picture.pc-hanoch-w[1,2].bmp.torrent

• Not necessarily on the same machine as the world.

What implementation we use

• The clients are a modified version based on an up to date version of Bram Python Client.

• Modifications:– “World” clients

• do not print as it would only make a mess• reincarnate after the finish to make a real world feel.

– “My Client” template – kills all clients when finished– All clients – print total time

Python

• Very common script language

• Intuitive and readable

• References:– python.org :http://docs.python.org/reference/ – http://wiki.python.org/moin/BeginnersGuide/NonProgrammers– http://wiki.python.org/moin/BeginnersGuide/Programmers

• The book "Think like a Computer Scientist in Python" has a very good reputation:

My Client

• We supply a default “My Client” which is like the “standard” one.

• If you want you may replace it with any other implementation, with certain reservations:– Remember we compare to our client.– It does not count to download a faster client than our

client and simply say it is faster.– Client should either limit it’s upload rate, or be able to

work with trickle.

Getting started

• Run the environment– ssh pc-hanoch-w[1,2]– cd /users/courses/hanochworkshop/bittorrent– run_all.sh

• start • [BitTornado-My-Client/btdownloadheadless.py]

– Note tracker prints a message for each GET.

• You can connect to the tracker with a browser at:– http://pc-hanoch-w1:6970/

• Do ls -l /vol/scratch/bt-download/ to see downloaded files

What is it doing?

• Run bttrack.

• Copy From FreshFiles a set of “downloaded” files to download directory.

• Run X seeds

• Run a python client per “downloaded” file.

• Run my client if given

For debug: run client separately

• ./run_all.sh start to start the world• BitTornado-My-Client/btdownloadheadless.py

– --saveas /vol/scratch/bt-download/picture.tst.bmp – --max_upload_rate 70 – --security 0 – ./picture.pc-hanoch-w[1,2].bmp.torrent

• Start debug. • Use your favorite debug environment. • IDLE is standard source level debugger.

Directory Structure

• BitTornado-My-Client – – high level application

• BitTornado-My-Client/BitTornado – – services

• BitTornado-My-Client/BitTornado/BT1 –– Actual work

Client Operation

• btdownloadheadless.py – First to run.– Performs initializations.

• get_response – parse the .torrent• Infohash – the ID of the .torrent• startEngine – Start connection with peers• startRerequester - Start connection with tracker

– Run rawserver.listen_forever from RawServer.py.

– listen_forever is: • Polling port • executing func() from a task queue.

Receiving messages

• From listen_forever loop it goes to:– BT1/Connecter.py/got_message

• CHOKE • UNCHOKE• INTERESTED• NOT_INTERESTED• BITFIELD• REQUEST• CANCEL• PIECE• HAVE

Influencing decisions• Can be done in in one of the callbacks

which spring from Connecter.py / got_message:

• Downloader.py– HAVE->got_have->send_interested

• maybe check connection before sending?

BT tracker

• Send GET request in Rerequester.py / announce

• Receive GET data in thread in Rerequester.py / _rerequest

Influencing decisions

• In Encrypter.py / start_connection or _start_connection_from_queue

For instance try different max_connections.

_rerequest_single – check timout for scheduling.

Policy

• Choker.py/_round_robin – decide who to choke / optimistically un choke / send_have to.

• Downloader.py / _request_more.

• PiecePicker.py / next - rarest piece, etc.

• got_unchoke->_request_more->next…