trajectory sampling for direct traffic observation matthias grossglauser joint work with nick...
Post on 19-Dec-2015
212 views
TRANSCRIPT
![Page 1: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/1.jpg)
Trajectory Sampling forDirect Traffic Observation
Matthias Grossglauser
joint work with Nick Duffield
AT&T Labs – Research
![Page 2: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/2.jpg)
Traffic Engineering
overload!
Two large flows
![Page 3: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/3.jpg)
Traffic Engineering
overload!
New egress pointfor first flow
Multi-homed customer
![Page 4: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/4.jpg)
Traffic Engineering
overload!
OSPF shortest path splitting
![Page 5: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/5.jpg)
Traffic Engineering• Goal: domain-wide control & management to
– Satisfy performance goals– Use resources efficiently
• Knobs:– Configuration & topology: provisioning, capacity
planning– Routing: OSPF weights, MPLS tunnels, BGP
policies,…– Traffic classification (diffserv), admission control,…
• Measurements are key: closed control loop– Characterize demand: what’s coming in?– Observe network state: how is the network
reacting? (low-level adaptivity!)– Check performance: what’s the customer’s QoS?
![Page 6: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/6.jpg)
Traffic Matrix vs. Path Matrix
• Traffic matrix– # bytes from ingress i to egress j
• Path matrix– Spatial flow of traffic through domain– # bytes for every path from i to j
![Page 7: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/7.jpg)
flow 1 flow 2 flow 3 flow 4
Flow Measurement
• IP flow abstraction– Set of packets with “same” src and dest IP
addresses– Packets that are “close” together in time (a
few seconds)
• Cisco NetFlow – Router maintains a cache of statistics about
active flows– Router exports a measurement record for
each flow
![Page 8: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/8.jpg)
Inferring the Path Matrix from the Traffic Matrix
![Page 9: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/9.jpg)
Network State Uncertainty
• Hard to get an up-to-date snapshot of…• …routing
– Large state space– Vendor-specific implementation– Deliberate randomness– Multicast
• …element states– Links, cards, protocols,…
• …element performance– Packet loss, delay at links
![Page 10: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/10.jpg)
missing “down” alarms spurious down
noise
missing alarms
![Page 11: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/11.jpg)
Direct Traffic Observation• Goal: direct observation
– No network model & state estimation
• Basic idea:– Sample packets at each link– Sampling decision based on hash over packet
content– Consistent sampling trajectories– Labels based on second hash function
• Exploit entropy in packet content to obtain statistically representative set of trajectories
![Page 12: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/12.jpg)
Sampling and Labeling
• Fields of interest collected only once• Multicast: trajectory is a tree
![Page 13: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/13.jpg)
Fields Included in Hashes
![Page 14: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/14.jpg)
Collisions: Identical Packets
![Page 15: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/15.jpg)
Sampling and Labeling Hashes
• x: subset of packet bits, represented as binary number
• Sampling hash– h(x) = x mod A– Sample if h(x) < r– r/A: thinning factor
• Labeling hash– g(x) = x mod M
• Make appropriate choice of A, M– predictable patterns should “mix” well
![Page 16: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/16.jpg)
Pseudo-Random Sampling• Goal: infer metrics of interest from
trajectory samples– E.g., what fraction of traffic of
customer x on a link y?
• Question: is sample set statistically representative?– Obvious for “really random” sampling– Distribution of a field in the sampled
subset = real distribution?– In other words: does the complement
of the field provide enough entropy?
![Page 17: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/17.jpg)
Quality of Deterministic Sampling
• Experiment: statistical test to check if sampled and full distributions are close– Chi-square statistic to verify independence
hypothesis– Hypothesis: sampled distribution consistent
with full distribution
– Confidence level C(T) for hypothesis, where C is cdf of with I-1 degrees of freedom
2
jn j bin in samples # :
nnnn
mmmm
mmmm
I
I
I
...
...
...
21
111211
000201
1
0 1
2)(
i
I
j ij
ijij
m
mmT
jmij bin in d(un)sample packets # :
![Page 18: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/18.jpg)
Chi-square Test on Source AddressIf , then accept hypothesis 1)(TC
![Page 19: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/19.jpg)
Bitwise Independence• 2x2 contingency table formed by
– sampling decision– l-th bit of packet
![Page 20: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/20.jpg)
Optimal Sampling
• Fix amount of measurement traffic c per time period
• Problem:– n: number of samples in sampling period– M: alphabet size, m=log2(M) bits/label– nm: total amount of measurement traffic [bits]– Goal: maximize # unique labels, subject to nm<c
• Result:– optimal alphabet size M*=c log(2)– optimal number of samples n*=M*/log(M*)– example: c=1Mb/period
![Page 21: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/21.jpg)
Label Collisions and Trajectory Ambiguity
![Page 22: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/22.jpg)
Ambiguity cont.
• Rule for acyclic subgraphs + unicast packets:– unambiguous if each connected component of the subgraph
is• (a) a source tree• (b) a sink tree without loss
![Page 23: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/23.jpg)
InferenceExperiment
• Experiment: infer from trajectory samples– Estimate fraction of traffic from customer– Source address customer– Source address sampling + label
• Fraction of customer traffic on backbone link:
b on labels unique #cb, on common labels unique #
ˆ
![Page 24: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/24.jpg)
Estimated Fraction (c=1000bit)
![Page 25: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/25.jpg)
Estimated Fraction (c=10kbit)
![Page 26: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/26.jpg)
Sampling Device
MPLS: simple additional logic to look “behind” label stack
![Page 27: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/27.jpg)
Sampling Device Implementation
• Interface vs. processing speed– OC-192: 10 Gbps– State of the art DSP:
• Proc: 600M MACs x 32 bit: 20 Gbps• I/O: 300MHz x 256 bit: 70 Gbps
– Moore’s law vs. interface speed growth
• Vendor interest: cisco, juniper, avici
![Page 28: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/28.jpg)
![Page 29: Trajectory Sampling for Direct Traffic Observation Matthias Grossglauser joint work with Nick Duffield AT&T Labs – Research](https://reader038.vdocument.in/reader038/viewer/2022110322/56649d365503460f94a0e9d5/html5/thumbnails/29.jpg)
Summary• Advantages
– Trajectory sampling estimates path matrix…and other metrics: loss, link delay
– Direct observation: no routing model + network state estimation
– No router state– Multicast (source tree), DDoS (sink tree)– Control over measurement overhead– Small measurement delay
• Disadvantages– Requires support on linecards
• Open questions & research problems– Collection, storage, querying (in progress)– Management interface