measurement in the internet measurement in the internet paul barford university of wisconsin -...
TRANSCRIPT
![Page 1: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/1.jpg)
Measurement in the Internet Measurement in the Internet
Paul Barford
University of Wisconsin - Madison
Spring, 2001
![Page 2: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/2.jpg)
WhyWhy measure the Internet? measure the Internet?
The size, growth, complexity and diversity of the Internet make it impossible to understand, manage, protect or provision without measurement.
Prior measurement studies have taught us a great deal– Self-similarity of packet traffic– Ubiquity of power laws
Businesses use measurements to provision, manage and operate
Future developments will require more and/or different measurements
![Page 3: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/3.jpg)
WhatWhat can be measured in the can be measured in the Internet?Internet?
Structure– Topology, routing, CDN’s, wireless, etc.
Traffic– Transport, end-to-end performance, etc.
Users and Applications– WWW, (x,my)Napster, Peer-to-Peer,Streaming, security, etc.
Failures– In all areas
Nefarious behavior– Pattern attacks, port scans
![Page 4: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/4.jpg)
WhereWhere are measurements made in are measurements made in the Internet?the Internet?
For some measurements, this is obvious– Web logs
The goal for other measurements is to be “representative”– Various “Internet weather reports”
Placement of measurement nodes is not a well understood problem– More is better??
![Page 5: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/5.jpg)
HowHow are Internet measurements are Internet measurements made?made?
Passive methods– Application monitors (logs), packet monitors
Active methods– Probes, application simulation
SurveysSignificant infrastructure is always requiredAll methods present difficulties
![Page 6: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/6.jpg)
WhenWhen must Internet must Internet measurements be made?measurements be made?
Diurnal traffic cycleTime scales depend on “what” and “how”Passive measurements are typically continuous
– Can generate huge data sets– Many people will not allow access to their logs
Active measurements are typically discrete– Important characteristics can be missed– Probes can be filtered and/or detected
![Page 7: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/7.jpg)
WhoWho is doing Internet is doing Internet Measurements?Measurements?
Businesses do a great deal of measurement– What measurements are they taking and what do they
do with their data?Instrumentation for measurement-based research
is relatively new– Developments over the past 12 years have been slow– 10’s of current studies (see CAIDA and SLAC pages
for lists of these)Most studies are not coordinated and relatively
narrowly focused
![Page 8: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/8.jpg)
In the past…In the past…
Bellcore Ethernet packet traces – Leland et al.– High time resolution LAN data collected over 4 year
period beginning in 1989– Thorough analysis showed self-similar properties
NPD study – Paxson– Characterized routing and packet behavior in wide area– First installation of large measurement infrastructure
![Page 9: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/9.jpg)
Internet topology: Skitter ProjectInternet topology: Skitter Project
CAIDAInternet topology measurement infrastructures
– Traceroute studies focused on router/link discovery– Skitter: 17 sources, 54,000 destinations world wide– Data from Skitter is publically available
www.caida.org
![Page 10: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/10.jpg)
Skitter example – BGP pathsSkitter example – BGP paths
![Page 11: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/11.jpg)
SurveyorSurveyor
Advanced Network SystemsInfrastructure for measuring one-way Internet
latency, loss and routes between hosts– 61 sites world wide– GPS enabled– Closed platform (until recently)
www.advanced.org
![Page 12: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/12.jpg)
Surveyor example - delaySurveyor example - delay
![Page 13: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/13.jpg)
Wide Area Web Measurements Wide Area Web Measurements (WAWM)(WAWM)
Barford and Crovella (Wisconsin & Boston Univ.)Application level measurement infrastructure
– 11 clients (national and international)– GPS enabled
Combined passive and active measurementsApplication of critical path analysis to TCP
transactions www.cs.wisc.edu/~pb
![Page 14: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/14.jpg)
Example of WAWM resultsExample of WAWM results
File transfer delay for 500KB file between Denver and Boston
![Page 15: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/15.jpg)
National Internet Measurement National Internet Measurement Infrastructure (NIMI)Infrastructure (NIMI)
Paxson (ACIRI), Adams and Mathis (PSC)Secure management platform for wide area
measurements– Designed for general probe installation
Distributed client infrastructure– Principally academic sites (currently 41 world wide)
A number of projects are running on NIMIwww.ncne.nlanr.net
![Page 16: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/16.jpg)
Short Term Challenges of Short Term Challenges of Internet MeasurementInternet Measurement
Developing methods for gathering more precise data– Current tools are frequently quite poor
Developing methods for analyzing measurement data more thoroughly– What we know about how things work is limited
Developing methods for understanding causes and effects across multiple domains
![Page 17: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/17.jpg)
The future – Global Internet The future – Global Internet Measurement (GIM)Measurement (GIM)
Ubiquitous measurement capability– Embedded into the design of the Internet– Emphasis on extensible API’s for measurement– Analysis capability must be built in
Measurement quality and soundness Data formats that enable aggregation that reflects higher level behaviors
Many difficulties– Management, security, privacy, heterogeneity, deployment, etc.
See Performance Evaluation vol. 864 www.elsevier.com/locate/peva
![Page 18: Measurement in the Internet Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001](https://reader036.vdocument.in/reader036/viewer/2022082821/5697c0021a28abf838cc35df/html5/thumbnails/18.jpg)
ConclusionConclusion
Measurements are necessary for understanding Internet structure and behavior– Rule #1: Expect surprises!
Internet measurements are difficult– Rule #2: Expect the data to be “dirty”
Measurements are necessary for populating and validating any reasonable Internet models– Rule #3: Garbage-in-garbage out
Current measurement infrastructures can provide a great deal of data but fall short of the GIM goal