department of computer science internet performance measurements using firefox extensions scot l....
TRANSCRIPT
Department of Computer Science
Internet Performance Measurements using Firefox Extensions
Scot L. DeDeo Professor Craig Wills
Department of Computer Science
Motivation
There exists an API within the Firefox web browser that allows for interacting with its downloading and processing of websites
Can this functionality be exploited to allow us not just to improve the web surfing experience, but to understand it under realistic conditions?
Department of Computer Science
Outline Introduction to Firefox Extensions
Description Capabilities
Page Stats Extension Overview Approach and paths taken Demo
Summary
Department of Computer Science
Introduction Firefox Extensions are
add-ons that integrate directly into the Firefox Web browser.
Function through open source API that allows for manipulating both the content being received by the web browser as well as the graphical outlook.
Department of Computer Science
Capabilities Graphical add-ons (Use the XML User Interface Language – XUL)
Allows for changing the existing Firefox overlay, adding new buttons, menus, and windows.
E.g. Google toolbar allows for a toolbar to be inserted directly into Firefox.
Middleware modifications (Cross Platform Component Object Module – XPCOM Interfaces - Accessible through JavaScript or C++)
Allows for changing networking properties, modifying / blocking data requests / responses, etc.
E.g. Fasterfox allows you to tweak many network and rendering settings such as simultaneous connections, pipelining, cache, DNS cache, and initial paint delay.
E.g. AdBlock and AdBlock plus allow for use of a pre-formed list or to customize a list of sites to block downloads from.
Department of Computer Science
Capabilities Combination of Graphical add-ons and Middleware modifications
can yield new applications that exist within Firefox. E.g. Gmail Space – Allows users to take advantage of Gmail’s 2 GB quota by
providing a graphical interface to easily transfer files.
Fixing bugs in other sites
Allows for changing sites html, style sheets, and inserting JavaScript. E.g. Style sheets can be modified to change the background color and text of a site
(change to black on white instead of yellow on white).
Department of Computer Science
Website Request / Response Breakdown
1. Location Changes – Request for container (main) page goes out.
2. Response for container (main) page is received and processed.
3. Additional requests are spawned in response to the processed page: CSS, JavaScript, images, etc.
4. Responses come in and are processed and may / may not spawn additional requests.
Container Page
CSS
JavaScript
Jpeg
Spawns
JavaScript
Spawns
Browser location set to:
http://www.scotd.com/index.php
http://www.scotd.com/index.php?id=all
http://www.scotd.com/scot.css
http://www.scotd.com/background/valley2k.jpg
Spawns
Department of Computer Science
Outline Introduction to Firefox Extensions
Description Capabilities
Page Stats Extension Overview Approach and paths taken Demo
Summary
Department of Computer Science
Page Stats Extension Measures the HTTP requests associated with the load of a webpage.
Page Measurements Include:# of Request Request submit time (msec)Finish Time (msec) HTTP Response CodeSize (bytes) content typeURL
Page Summary Includes:URL total page size (bytes) total time (msec) total # of requests # dead requests
Capabilities: Control input and output with a graphical interface Output to a log file Run in a batch mode and visit a list of sites
Department of Computer Science
Page Stats Extension Usage:
1. Graphical Interface found under the tools menu in Firefox - “Site Measurements”
• Ability to enable / disable measurement listeners, log file, and select a list of sites to visit.
• Also allows for saving and clearing the text on the measurement window.
• Provides a summary of all sites visited.
Department of Computer Science
Page Stats Extension Usage:
2. Batch mode – Invoked by a configuration file specified on the command line (firefox pagestatsconfig=“/home/sdedeo/config.txt”).Parameters:
measurement: onlog: /home/sdedeo/log.txtsites: /home/sdedeo/sites.txtexitOnFinish: truerequestTimeout: 8000
pageTimeout: 45000
Allows for easily measuring pages in bulk
Pages are still fully rendered
Department of Computer Science
Approach and paths taken Began in January 2006 based on work completed by
Paul Timmins Included initial work of setting up an extension with ideas on how
to make measurements
Initial version based on an XPCOM component called nsiWebProgress which reports to the browser: State, Progress, Location, Status, and Security changes
Department of Computer Science
Approach and paths taken First Approach Issues
nsiWebProgress progress notification only occurs on text based files (E.g. html, CSS, JavaScript). Images and other static files are omitted.
No way to measure the time the request is submitted Occasionally the progress notification gives an unknown length of a file If a site was visited more then once, caching effects skew measurements
Second Approach – nsiHTTPChannel Intercepts requests browser sends – “http-on-modify-request” Intercepts responses before processing – “http-on-examine-response” Clears system cache between sites in order to maintain consistent
measurements
Department of Computer Science
Approach and paths taken Second Approach (Continued)
nsiHTTPChannel provides no state notifications Solved using nsiWebProgress state change to detect state
stopped / finished Probe to insure all requests that have gone out have come back
(nsiWebProgress state stop returned before all images received)
Timeout Mechanism Two Timeouts:
Max time between receiving responses – defaults to 8 seconds Max time for a page – defaults to 45 seconds
Department of Computer Science
Approach and paths taken
Sites with Issues: http://www.cnn.com routinely sends requests for images in which no
response comes. These are JavaScript invoked images on mouseovers This problem is solved by the request timeout and the requests are labeled
as ‘dead’
http://my.netscape.com uses a JavaScript based redirect: ‘document.location = http://my.netscape.com/index2.psp’ which when used in batch mode was causing the extension to move onto the next site and process two sites simultaneously
This was solved by giving a one milliseconds delay before switching sites after a site’s results were processed
http://www.dallasnews.com sends an image banner which receives an update once every two seconds
This was solved with the max site timeout
Department of Computer Science
Demo
Launch Firefox
Department of Computer Science
Outline Introduction to Firefox Extensions
Description Capabilities
Page Stats Extension Overview Approach and paths taken Demo
Summary
Department of Computer Science
Summary
Firefox extensions are a powerful tool that allow for: Measuring and manipulating how a page is downloaded and
processed Adding additional features, both graphical and background
Building Extensions: Firefox 1.5 enables much easier testing of extensions XPCOM API is still in the beta stages and certain libraries must
be used with caution since they change Documentation available is still minimal, but the development
community is growing at a fast pace
Department of Computer Science
Summary
Can this functionality be exploited to allow us not just to improve the web surfing experience, but to understand it under realistic conditions?
The pagestats extension shows one way we can create quantitative measurements for a website
Pagestats can be used in conjunction with other extensions, such as adblock and fasterfox, to compute a measurement of effectiveness
Department of Computer Science
Questions / Comments
Scot L. DeDeoProfessor Craig Wills
Page Stats Extension:http://www.wpi.edu/~sdedeo/pagestats
Related Sites:https://addons.mozilla.org/firefox/extensions/http://www.xulplanet.com (XUL and XPCOM API Reference)