where in the world is carmen bitdiego? and who is she, anyways…
DESCRIPTION
Where in the World is Carmen BitDiego? And who is she, anyways…. Alexandru IOSUP [email protected] u delft.nl. The 12th annual ASCI Computing Workshop. Introduction (1 of 3). Peer-2-Peer File sharing Everybody has the same rights. P2P average everybody ? Who? Where? When? How? Why? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/1.jpg)
1
Where in the World is Carmen BitDiego? And who is she, anyways…
Alexandru [email protected]
The 12th annual ASCI Computing Workshop
![Page 2: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/2.jpg)
2
Introduction (1 of 3) Peer-2-Peer
File sharingEverybody has the same rights. P2P average everybody?
• Who?• Where?• When?• How?• Why?
Tons of studies over the past 5 years• Saroiu’02, Yazti’02, Yzal’04, Pouwelse’04• We go for something else! (tbs)
![Page 3: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/3.jpg)
3
Introduction (2 of 3) BitTorrent
Most used P2P network today (53% traffic)Attributes
• 2nd gen. P2P network – no centralized servers; optimizes transfer speed; favors high-bandwidth users; files are split in chunks
• Peers – Trackers – Web sites• Tit-for-tat sharing mechanism – everybody gives some;
except when they don’t…• no search at peer level• Owners are called seeds, we are called leeches
So much to know: I want my BitTorrent today!
![Page 4: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/4.jpg)
4
Enters Carmen… Carmen SanDiego
Famous spyLocation: unknownLikes: to hideClues to where
she is: history, complicated hints
Never caught
Carmen BitDiegoFamous P2P networkLocation: unknownLikes: who knows?Clues to where she is:
some history, lightweight hints
Caught (?)• NO Multi-files studies• NO Country-per-file• NO Organizations• NO, NO, NO…
Who is this Carmen, anyways…
![Page 5: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/5.jpg)
5
Introduction (3 of 3) We track Carmen BitDiego
Tracked data attributes• Users got 204,454,719,497,935B (ok, 204,5TB)• 40,000,000 contacts• 200,000 unique users (*)• 120 files• 9 specific media types
• The first aliased media view
• 7 unique views
We got her now! Or is it…
![Page 6: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/6.jpg)
6
Mission statement We want to know about Carmen BitDiego
Where she goes• Continent, country, city, organization
When she goes• Time-patterns per country• Time-patterns in seeds/leeches ratio• How many file chunks at any time?
With whom she hangs out• Special users? Super-peers, collector peers
Is she a good companion?• How many users get what they want?
We’re getting to this info in no time…
![Page 7: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/7.jpg)
7
Outline of the presentation Intro Enters Carmen… Mission statement Our data looks like this… Methods, or how to catch her Results, or how we caught her Conclusions
(done)
(done)
(done)
(we are here)
(coming up next)
![Page 8: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/8.jpg)
8
Our data looks like this… We track 120 files
120 trace files• Time stamp, IP, port, # of chunks = record = 1 observation
12 big traces (+500,000 observations/trace)• December 2003 – January 2004
108 small traces• March 2004
3 global categories All, Big, Small
9 special categories Movies, Games,
Music, Applications Alias media
Same contents, different names
• Same language• Different language
![Page 9: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/9.jpg)
9
Outline of the presentation Intro Enters Carmen… Mission statement Our data looks like this… Methods, or how to catch her Results, or how we caught her Conclusions
(we are here)
(coming up next)
(done)
(done)
(done)
(done)
![Page 10: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/10.jpg)
10
Methods, or how to catch her We want to know about Carmen BitDiego
Where she goes• Un-DNS(*): continent (1), country (2), city (3), organization (4)
When she goes (5)• Parse and correlate Time-patterns per country• Parse and correlate Time-patterns in seeds/leeches ratio• Parse and correlate How many file chunks at any time?
With whom she hangs out (6)• Super-peers = nodes that own more than one complete file• Collector peers = nodes that try to get more than one file
Is she a good companion? (7)• How many users get what they want?
* Thanks MaxMind (GeoIP lib, database) and WebLog Expert (databases)
![Page 11: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/11.jpg)
11
Outline of the presentation Intro Enters Carmen… Mission statement Our data looks like this… Methods, or how to catch her Results, or how we caught her Conclusions
(we are here)
(coming up next)
(done)
(done)
(done)
(done)
(done)
WARNING! We show only a selection of our results!
![Page 12: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/12.jpg)
12
Results, or how we caught her Where she goes
continent
Europe is now the biggest BitTorrent consumer (not NA)
Tit-for-tat discourages low-bandwidth users!
![Page 13: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/13.jpg)
13
Results, or how we caught her Where she goes
continent
Not the same distribution for different sets of files!
Europe is now the biggest BitTorrent consumer (not NA)
Tit-for-tat discourages low-bandwidth users!
Coarse media locality property Asia > North America (themed game)
![Page 14: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/14.jpg)
14
Results, or how we caught her Where she goes
country
US still the biggest overall BitTorrent consumer – continent view can be misleading!
NL is only 6th!
![Page 15: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/15.jpg)
15US still the biggest overall BitTorrent consumer – continent view can be misleading!
Where she goes country
Not the same distribution for different sets of files!
Localized versions of the files attract
local users!
Themed files attract very specific audiences! What about a marketing
study based on BitTorrent file ranks?
Fine media locality!Countries have habits!
Results, or how we caught her
Hong Kong, Chile: soccer management sim
Israel: action movie
Japan: animes
The Nederlands 6th Romania ~50th
![Page 16: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/16.jpg)
16
Results, or how we caught her Where she goes
city
Oldenburg, Eschborn, Herndon … Internet nodes placed outside major cities –
cannot use this to track real users!
30% unknown – not reliable!
Dispersed locations
![Page 17: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/17.jpg)
17
Results, or how we caught her Where she goes
organization
Not the same distribution for different sets of files!
We’d like to thank:The Walt Disney Company,
Sony Corporation, SANYO Electric Software Co. Ltd.,
and Merrill Lynchfor actively supporting BitTorrent!
1 ISP covers +60% users
10 ISPs cover <50% users
ISP caching policy different for different files and communities!
Academic institutions < 10% users!
![Page 18: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/18.jpg)
18
Results, or how we caught her When she goes
Time-patterns per country
8:30AM, 1PM, 6-9PM, 12-1AMmostly at work,
during slow hours?
Europe guides the time-patterns!
![Page 19: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/19.jpg)
19 0
2e+006
4e+006
6e+006
8e+006
1e+007
1.2e+007
1.4e+007
BitTorrent no. of chunks in the network, group Bigchunks
Sat,03/12/06
00:00
Sat,03/12/13
00:00
Sat,03/12/20
00:00
Sat,03/12/27
00:00
Sat,04/01/03
00:00
Sat,04/01/10
00:00
Sat,04/01/17
00:00
Results, or how we caught her When she goes
How many file chunks at any time?
The network is not robust all the time – attacks at these precise moments
could be fatal!
Causes:- trackers down- users interest down- others
![Page 20: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/20.jpg)
20
Results, or how we caught her When she goes
Time-patterns per no. of chunks/seeders/leeches ratio
0
5000
10000
15000
20000
25000
Sat,03/12/0600:00
Sat,03/12/1300:00
Sat,03/12/20
00:00
Sat,03/12/2700:00
Sat,04/01/0300:00
Sat,04/01/1000:00
Sat,04/01/1700:00
BitTorrent no. of chunks/users/seeds ratio, group Big
chunks (/10 3)users (/10)
seeds
users:seeds ~ 10:1leeches:seeds ~ 9:1
chunks:seeds ~ 1000:1
![Page 21: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/21.jpg)
21
Results, or how we caught her With whom she hangs out
Super-peers = nodes that own more than one complete file Collector peers = nodes that try to get more than one file
1
10
100
1000
10000
100000
1e+006
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Num
ber
of U
sers
(log
)
Number of Files
BitTorrent superpeers/collectors, groups Big/Small
Collectors, group BigSuperpeers, group Big
Collectors, group SmallSuperpeers, group Small
# users / # files decreases exponentially!
Group Small:Collectors (n files) ~
2x Superpeers (n files)
![Page 22: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/22.jpg)
22
Results, or how we caught her Is she a good companion?
1
10
100
1000
10000
100000
1e+006
0 100 200 300 400 500 600 700 800 900 1000 1100 1200
Num
ber
of U
sers
(log
)
Number of Points
Group Movie - Popular (Aliased Media)Group Small
BitTorrent users/points distribution, groups Small/Aliased Media
1 Point = 1% of any file
Group Small users download whole files!
Aliased Media results in exponential drop!people drop after getting 1/many
Group Small Avg. (any) ~ 81 points Avg. (1 file) ~ 113 points
Aliased Media Avg. (any) ~ 52 points Avg. (1 file) ~ 109 points
113
81
81
81
Users download 1 file then disconnect!
YES!
![Page 23: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/23.jpg)
23
Outline of the presentation Intro Enters Carmen… Mission statement Our data looks like this… Methods, or how to catch her Results, or how we caught her Conclusions
(done)
(done)
(done)
(we are here)
(done)
(done)
(done)
![Page 24: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/24.jpg)
24
Conclusions Carmen BitDiego
Famous P2P network Location: known Likes: established (study per specific file groups)
Clues to where she is: complete hints• Multi-files study• Continents, Country, Cities, Organizations, global and per-file• Time-patterns in the users/seeds/leeches behavior (also country)• Super-nodes / collector nodes analysis
Carmen BitDiego almost caught!• Trivial and Non-trivial locality properties• Alias media hints
Need a full study w/ these methods to catch her!
![Page 25: Where in the World is Carmen BitDiego? And who is she, anyways…](https://reader035.vdocument.in/reader035/viewer/2022062814/56816790550346895ddcc139/html5/thumbnails/25.jpg)
25
Thank you…
Questions? Remarks? Observations? All welcome!
Alexandru IOSUPTU Delft
[email protected]://www.pds.ewi.tudelft.nl/~iosup/index.html
I would like to thank Johan Pouwelse and Pawel Garbacki for all their help in creating this study. Thank you, Johan! Thank you, Pawel!
Their previous work: http://www.theregister.co.uk/2004/12/18/bittorrent_measurements_analysis/