internet measurement initiatives in the wisconsin advanced internet lab
Post on 20-Jan-2016
34 Views
Preview:
DESCRIPTION
TRANSCRIPT
Internet Measurement Initiatives in the Wisconsin Advanced Internet Lab
Paul Barford
Computer Science Department
University of Wisconsin – Madison
Spring, 2003
pb@cs.wisc.edu 2
Talk Objectives
• Motivate and describe Wisconsin Advanced Internet Lab (WAIL)– Internal lab environment– External lab environment
• Provide some detail on three current projects– Anomaly detection and characterization– Distributed intrusion monitoring– Understanding packet loss
pb@cs.wisc.edu 3
Motivation for New Tools• Any area of scientific research is limited by the tools
available for experimental study– “If your only tool is a hammer then everything looks
like a nail”• 2001 NRC report: “network research community is in
danger of ossification due to strictures of experimental systems”– Challenge: “Capturing a day in the life of the Internet”
• New experimental tools can open up areas of research that have not previously been accessible
pb@cs.wisc.edu 4
An Internet Instance Lab • A hands-on test environment designed to recreate
paths and conditions identical to those in the Internet from end-to-end-through-core– Requires large amount of routing and end host equipment
• Network and host equipment able to recreate (not emulate) a wide range of services, configurations and traffic conditions– Complete instrumentation of end-to-end paths
– Deployment of disruptive prototypes
pb@cs.wisc.edu 5
Key Challenges
• Design• Configurations and management• Traffic generation• Propagation delay• Validation
pb@cs.wisc.edu 6
The Wisconsin Advanced Internet Lab
• Our realization of an IIL• Developed over past 18 months by UW/Cisco team• Supported by $3.5M equipment grant from Cisco and
UW matching funds– Used to purchase over 75 pieces of networking equipment
• Phase 1 nearing completion => Abilene recreation• Other partners: EMC, Spirent, Intel, Fujitsu, Sun• Research initiatives in many areas…
pb@cs.wisc.edu 7
External Environment• Essential complement to internal environment• Existing infrastructure
– DOMINO systems (1 class A + 2 class B’s + Dshield)
– Surveyor + WAWM systems (~70 nodes)• New database and front end by summer ‘03
• Partnerships and other available systems– Condor/Grid Infrastructures
• Passive flow measurements– FlowScan data from UW, Internet2, others…
pb@cs.wisc.edu 8
Project 1: Detecting Anomalies in IP Flows
• Motivation: Anomaly detection remains difficult• Objective: Improve understanding of traffic anomalies• Approach: Multiresolution analysis of data set that
includes IP flow, SNMP and an anomaly catalog• Method: Integrated Measurement Analysis Platform for
Internet Traffic (IMAPIT)• Results: Identify anomaly characteristics using wavelets
and develop new method for exposing short-lived events
pb@cs.wisc.edu 9
Our Data Sets• Consider anomalies in IP flow and SNMP data
– Collected at UW border router (Juniper M10)– Archive of ~6 months worth of data (packets, bytes, flows)– Includes catalog of anomalies (after-the-fact analysis)
• Group observed anomalies into four categories– Network anomalies (41)
• Steep drop offs in service followed by quick return to normal behavior– Flash crowd anomalies (4)
• Steep increase in service followed by slow return to normal behavior– Attack anomalies (46)
• Steep increase in flows in one direction followed by quick return to normal behavior– Measurement anomalies (18)
• Short-lived anomalies which are not network anomalies or attacks
pb@cs.wisc.edu 10
Multiresolution Analysis• Wavelets provide a means for describing time series
data that considers both frequency and time– Powerful means for characterizing data with sharp spikes
and discontinuities
– Using wavelets can be quite tricky
• We use tools developed at UW which together make up IMAPIT– FlowScan software
– The IDR Framenet software
pb@cs.wisc.edu 11
Ambient IP Flow Traffic
pb@cs.wisc.edu 12
Flow Traffic During DoS Attacks
pb@cs.wisc.edu 13
Deviation Score for Three Anomalies
pb@cs.wisc.edu 14
Project 2: Coordinated Intrusion Detection
• Motivation: Intrusion detection is a moving target• Objective: Coordinate intrusion monitoring between
multiple sites around the Internet• Approach: Share data from firewalls, NIDS and tarpits
(on unused IP space)• Method: Distributed Overlay for Monitoring Internet
Outbreaks (DOMINO)• Results: Blacklists can be rapidly generated, false
positives can be substantially lowered, new outbreaks can be easily identified
pb@cs.wisc.edu 15
DOMINO: A new approach to DNIDS
• Partnership with dshield.org– 1600 firewall and NIDS logs
• Tarpits– Active monitor of unused IP space– 1 class A (this week), 2 class B’s
• A protocol for node participation, data sharing and alert clustering– Chord-based overlay network– Extension of Intrusion Detection Message Exchange
Format– Various clustering methods
pb@cs.wisc.edu 16
Marginal Utility of Adding Nodes
pb@cs.wisc.edu 17
SQL-Sapphire Analysis
pb@cs.wisc.edu 18
Project 3: Understanding Packet Loss
• Motivation: Many of the most basic aspects of packet loss are not understood– Where, when, how long, how often?
• Focus: Developing a comprehensive understanding of packet loss in the Internet
• Approach: Combine understanding of protocols and queue behavior to create a probe train which can accurately measure delay and loss.
• Implications: End-to-end tools for pin-pointing loss, better transport protocols, better network management for congestion
pb@cs.wisc.edu 19
Active versus Passive Loss Measures
• Hypothesis: Active measures of loss are correlated with passive measures of loss
• Assessment in Abilene– SNMP loss measures on all backbone routers– Active probes via Ping/Zing in Surveyor nodes at
10Hz, 20Hz and 100Hz– Tests in full mesh over one month period
pb@cs.wisc.edu 20
Result: Active <> Passive
pb@cs.wisc.edu 21
Summary
• Both internal lab building initiatives and external measurement initiatives in WAIL
• Internal facilities are intended to be open
• We are seeking partnerships in external measurement projects.– DOMINO in particular
top related