design and implementation of http-gnutella gateway
DESCRIPTION
Design and Implementation of HTTP-Gnutella Gateway. Baoning Wu (baw4) Wei Zhang (wez5) CSE Department Lehigh University. Motivation. Peer-to-peer networking is a hot topic. Can P2P nodes search and get files from Web sites? Can one P2P network search and get files from other P2P networks? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/1.jpg)
Design and Design and Implementation of Implementation of
HTTP-Gnutella GatewayHTTP-Gnutella Gateway
Baoning Wu (baw4)Baoning Wu (baw4)Wei Zhang (wez5)Wei Zhang (wez5)
CSE DepartmentCSE DepartmentLehigh UniversityLehigh University
![Page 2: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/2.jpg)
MotivationMotivation Peer-to-peer networking is a hot topic.Peer-to-peer networking is a hot topic. Can P2P nodes search and get files Can P2P nodes search and get files
from Web sites?from Web sites? Can one P2P network search and get Can one P2P network search and get
files from other P2P networks?files from other P2P networks?
In our project, we have built a special In our project, we have built a special gateway between Gnutella and Web gateway between Gnutella and Web sites.sites.
![Page 3: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/3.jpg)
Related WorkRelated Work David McNab has launched Freenet search David McNab has launched Freenet search
engine.engine. Asiayeah is a Gnutella search engine.Asiayeah is a Gnutella search engine. Filedonkey.com is an Edonkey search Filedonkey.com is an Edonkey search
engine.engine. Kalepa Networks , Inc is doing work about Kalepa Networks , Inc is doing work about
connecting different P2P systems.connecting different P2P systems.
Our work is kind of reverse to all above Our work is kind of reverse to all above works.works.
![Page 4: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/4.jpg)
Mechanism of Gnutella Mechanism of Gnutella SearchingSearching
Node A sends a query to its neighbor B; Node A sends a query to its neighbor B; Node B boardcasts the query to its neighors C, D; Node B boardcasts the query to its neighors C, D; Node C has the objects node A needs and then returns a Node C has the objects node A needs and then returns a
query hit message to node B; query hit message to node B; Node B forwards the query hit message by consulting Node B forwards the query hit message by consulting
the local states.the local states.
![Page 5: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/5.jpg)
Architecture of HTTP-Architecture of HTTP-Gnutella GatewayGnutella Gateway
![Page 6: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/6.jpg)
Mechanism of the Mechanism of the gatewaygateway
1.1. Node A broadcasts a query message directly or Node A broadcasts a query message directly or indirectly to the HTTP-Gnutella gateway; indirectly to the HTTP-Gnutella gateway;
2.2. The HTTP-Gnutella gateway forwards the translated The HTTP-Gnutella gateway forwards the translated query message to search engine; query message to search engine;
3.3. The search engine returns a bunch of query results The search engine returns a bunch of query results to the gateway; to the gateway;
4.4. The gateway translates the results into Gnutella The gateway translates the results into Gnutella formats and then forwards them to node A;formats and then forwards them to node A;
5.5. If node A initializes a download requests to the If node A initializes a download requests to the gateway, the gateway will translate the Gnutella gateway, the gateway will translate the Gnutella request into a well-formatted HTTP request to the request into a well-formatted HTTP request to the Web server; Web server;
6.6. The gateway fetches the data from the Web server; The gateway fetches the data from the Web server; 7.7. The gateway forwards the data from the Web server The gateway forwards the data from the Web server
to node A.to node A.
![Page 7: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/7.jpg)
Handle Query MessagesHandle Query Messages
We still use the original Gnutella We still use the original Gnutella mechanism to judge whether to mechanism to judge whether to forward the message or not.forward the message or not.
The gateway captures all of queries The gateway captures all of queries with hops# < 5 and sends them to with hops# < 5 and sends them to search engine.search engine.
![Page 8: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/8.jpg)
Search Engine APISearch Engine API
Google search engine API has a limit Google search engine API has a limit of up to 1,000 requests per day.of up to 1,000 requests per day.
Search engine API consists of three Search engine API consists of three main functions:main functions: Query conversionQuery conversion Extraction of URLsExtraction of URLs Measurement of content sizeMeasurement of content size
![Page 9: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/9.jpg)
Generate Query Hit Generate Query Hit MessagesMessages
Two considerations:Two considerations: Let Gnutella nodes contact Web servers Let Gnutella nodes contact Web servers
directlydirectly Let the gateway work as a proxyLet the gateway work as a proxy
The gateway fills its own IP address The gateway fills its own IP address and a specific port number (currently and a specific port number (currently 9999) in the query hit messages.9999) in the query hit messages.
File names are URLs of Web objects.File names are URLs of Web objects.
![Page 10: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/10.jpg)
Downloading ServiceDownloading Service Translate Gnutella download request Translate Gnutella download request
into a well-formatted HTTP request. into a well-formatted HTTP request. e.g.e.g.
GET /get/1234/http://www.foo.com/foo.mp3 HTTP/1.1GET /get/1234/http://www.foo.com/foo.mp3 HTTP/1.1
User-Agent: GnutellaUser-Agent: Gnutella
Host: 123.123.123.123:6346Host: 123.123.123.123:6346=>=>
GET http://www.foo.com/foo.mp3 HTTP/1.1GET http://www.foo.com/foo.mp3 HTTP/1.1
User-Agent: GnutellaUser-Agent: Gnutella
Host: www.foo.comHost: www.foo.com
It should handle Gnutella handshakes It should handle Gnutella handshakes properly.properly.
It also records the bytes transferred.It also records the bytes transferred.
![Page 11: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/11.jpg)
Problems & SolutionsProblems & Solutions
Irregular handshakesIrregular handshakes We handle all possibilitesWe handle all possibilites
File sizeFile size We use HTTP HEAD request to get file We use HTTP HEAD request to get file
sizesize Broken Pipe signalBroken Pipe signal
We use forked processWe use forked process
![Page 12: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/12.jpg)
Experiment ResultsExperiment Results
OutlineOutline Basic verification and validationBasic verification and validation Log file formatLog file format Results #1 to #4Results #1 to #4
![Page 13: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/13.jpg)
Basic Verification & Basic Verification & ValidationValidation
Run our special gateway on machine 1 Run our special gateway on machine 1 and run a normal gtk-gnutalla client on and run a normal gtk-gnutalla client on machine 2. After machine 2 connects to machine 2. After machine 2 connects to machine 1, we use machine 2 to send machine 1, we use machine 2 to send query messages and downloading query messages and downloading request to machine 1.request to machine 1.
For downloaded files from machine 1, For downloaded files from machine 1, we use wget to get the same file from we use wget to get the same file from web server directly and use diff to test web server directly and use diff to test if they are identical. if they are identical.
![Page 14: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/14.jpg)
Log File FormatLog File Format
Log 1Log 1 Time stamp, MUID, IP address, Type, Time stamp, MUID, IP address, Type,
QueryQuery
Log 2Log 2 Time stamp, IP address, URL, Size, Time stamp, IP address, URL, Size,
Code, SuccessCode, Success
![Page 15: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/15.jpg)
Results #1Results #1
No. of Query messages: 319,245No. of Query messages: 319,245 No. of Query Hit messages: 930,860No. of Query Hit messages: 930,860 No. of served requests: 113,391No. of served requests: 113,391 Average Response Time: 16.33 Average Response Time: 16.33
secondsseconds
![Page 16: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/16.jpg)
Result #2Result #2
1 2 3 4 5 6 7 8 90
10000
20000
30000
40000
50000
60000
70000
80000
90000
100000
number of responses
nu
mb
er
of r
eq
ue
sts
![Page 17: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/17.jpg)
Result #3Result #3 No. of Downloading requests: 952No. of Downloading requests: 952 No. of Different IP addresses: 67No. of Different IP addresses: 67 No. of served Requests: 945No. of served Requests: 945 No. of sucessfully served requests: 740No. of sucessfully served requests: 740 Total size transfered: 244,227,881 bytesTotal size transfered: 244,227,881 bytes Average response time: 3.15 secondsAverage response time: 3.15 seconds Average total download time: 15.92 Average total download time: 15.92
seconds seconds
![Page 18: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/18.jpg)
Result #4Result #4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200
10
20
30
40
50
60
70
80
90
100
110
120
different sites
num
ber
of d
ownl
oade
d fil
es
![Page 19: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/19.jpg)
Future WorkFuture Work
Support a variety of file types and Support a variety of file types and measure their popularitymeasure their popularity
Build a gateway to connect different Build a gateway to connect different P2P systemsP2P systems
Deployment of such gatewaysDeployment of such gateways
![Page 20: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/20.jpg)
ConclusionConclusion
An HTTP-Gnutella gateway was built An HTTP-Gnutella gateway was built and worked for the Gnutella users.and worked for the Gnutella users.
Only 5 days, the gateway transferred Only 5 days, the gateway transferred about 244MB data from the Web sites about 244MB data from the Web sites to the Gnutella nodes.to the Gnutella nodes.
The systems achieved all goals of our The systems achieved all goals of our design.design.
![Page 21: Design and Implementation of HTTP-Gnutella Gateway](https://reader036.vdocument.in/reader036/viewer/2022062517/5681336a550346895d9a80fe/html5/thumbnails/21.jpg)
Question?Question?