multiscale traffic processing techniques for network inference and control
DESCRIPTION
Multiscale Traffic Processing Techniques for Network Inference and Control. R. Baraniuk R. Nowak E. Knightly R. Riedi V. Ribeiro S. Sarvotham A. Keshavarz. NMS PI meeting San Diego May 2003. SPiN .Rice.edu: S ignal P rocessing i n N etworking. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/1.jpg)
Multiscale Traffic Processing Techniques for Network Inference and Control
R. Baraniuk R. Nowak E. Knightly R. Riedi
V. Ribeiro S. Sarvotham A. Keshavarz
NMS PI meeting San Diego May 2003
SPiN.Rice.edu: Signal Processing in Networking
![Page 2: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/2.jpg)
Rice University, SPiN Group spin.rice.edu
Chirp Probing
always-on, non-intrusivebandwidth estimation
on-line decisionanomaly detection
Effort 1
![Page 3: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/3.jpg)
Rice University, SPiN Group spin.rice.edu
Milestones pathChirpMay 2003 milestone: • C code testing + distribution [completed]
– Allen McIntosh (Telcordia): testing, useful comments,– SPAWAR (hosted Phuong Nguyen at Rice),– CAIDA (kc Claffy, Margaret Murray),– RPI (Shivkumar).
– Demo: on-line estimation of bandwidth
Towards Nov 2003 milestones: • Integration of pathChirp
– GaTech pdns (Riley/Fujimoto) [completed]– UIUC, JavaSim (Hou) [in progress]
• Validation in controllable environment [enabled, to be done]
Towards May 2004 milestones: [basis provided, work to be done]• Final tool and theory for bursty traffic over multiple hops
![Page 4: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/4.jpg)
Rice University, SPiN Group spin.rice.edu
Efficient probing: PathChirp• Traditional probing paradigm:
– Produce (light) congestion – PacketPair:
• Sample the traffic– Pathload: flood at variable rate
• intolerable level of congestion
– TOPP: • PacketPairs at variable spacing
• New: – PathChirp:
• Variable rate within a train of probes
• More efficient, light
![Page 5: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/5.jpg)
Rice University, SPiN Group spin.rice.edu
pathChirp Developments
pathChirp• …a real world tool • …with improved performance
– Increased queuing delay correlates with cross traffic on network path
– Last excursion in chirp link capacity
– Weighted averaged of onset of excursions available link resources
Departure pattern
Queuing against departure
Methodology
![Page 6: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/6.jpg)
Rice University, SPiN Group spin.rice.edu
pathChirp DevelopmentspathChirp• …a real world tool • …with improved performance
– Queuing delay cross traffic – Final excursion link capacity– Averaged excursions available resources
• …converges in a handful of RTTs
Departure pattern
Queuing against departure
Methodology
Number of chirps
12 chirps
Real world experiments
Estimation against true x-trafficInternet experiment
![Page 7: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/7.jpg)
Rice University, SPiN Group spin.rice.edu
PathChirp Performance
pathChirp• performs comparably to
– PathLoad– PacketPair– TOPP
• …at smaller probing rate• …more robust to bursty traffic• Best paper at PAM2003• Ongoing work:
– Exploit dispersion informationcaptured in excursions to become robust against multiple hops
pathChirp vs TOPP square error
PathLoad converged after 6.7 Mb
.5 Mb 1 Mb
![Page 8: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/8.jpg)
Rice University, SPiN Group spin.rice.edu
Connection-level Analysis and Modeling of Network Traffic
understanding the cause of burstscontrol and improve performancedetect changes of network state
Effort 2
![Page 9: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/9.jpg)
Rice University, SPiN Group spin.rice.edu
Milestones Alpha-Beta
May 2003 milestone [completed]: • C++ version of decomposition and analysis module
Towards Nov 2003 milestone: • Verification of alpha-beta hypothesis in wider range of
topologies, protocols, applications [Analysis module ready; collection and analysis to be done]
• Collaborations with Telcordia and SLAC [initiated]• Collaborations with CAIDA [pursuing]
December 2004 milestones [to be done]:– Integration into simulators, verification in large simulation– Applications: alpha-bottleneck aware AQM, Admission control
![Page 10: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/10.jpg)
Rice University, SPiN Group spin.rice.edu
Non-Gaussianity and Dominance
Connection level separation:– remove packets of the ONE strongest connection– Leaves “Gaussian” residual traffic
Traffic components:– Alpha connections: high rate (> ½ bandwidth)– Beta connections: all the rest
Overall traffic Residual traffic1 Strongest connection
= +Mean
99%
![Page 11: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/11.jpg)
Rice University, SPiN Group spin.rice.edu
Simple Connection Taxonomy
Bursts arise from largetransfers over fast links.
RTTcwnd
bandwidthRTTcwnd
bandwidth
![Page 12: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/12.jpg)
Rice University, SPiN Group spin.rice.edu
CWND or RTT?
Correlation coefficient=0.68
Short RTT correlates directly with high rate and bursts.
103
104
105
106
10-1
100
101
102
peak-rate (Bps)
1/R
TT
(1/
s)
Correlation coefficient=0.01
103
104
105
106
102
103
104
105
peak-rate (Bps)
cwnd
(B
)
Colorado State University trace, 300,000 packets
cwnd 1/RTT
ratepeak cwnd
1/RTTratepeak
Beta Alpha Beta Alpha
![Page 13: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/13.jpg)
Rice University, SPiN Group spin.rice.edu
Impact: Performance• Beta Traffic rules the small Queues• Alpha Traffic causes the large Queue-sizes
(despite small Window Size)
Alpha connections
Queue-size overlapped with Alpha PeaksTotal
traffic
![Page 14: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/14.jpg)
Rice University, SPiN Group spin.rice.edu
Two models for alpha traffic
Impact of alpha burst in two scenarios:• Flow control at end hosts
– TCP advertised window
• Congestion control at router– TCP congestion window
![Page 15: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/15.jpg)
Modeling Alpha Traffic• ON/OFF model revisited:
High variability in connection rates (RTTs)
Low rate = beta High rate = alpha
fractional Gaussian noise stable Levy noise
+
=
+
+
=
![Page 16: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/16.jpg)
Rice University, SPiN Group spin.rice.edu
Alpha-Beta Model of Traffic
• Model assumptions:– Total traffic = Alpha component + Beta component– Alpha and Beta are independent– Beta=fractional Brownian motion
• Alpha traffic: two scenarios– Flow control through thin or busy end-hosts
• ON-OFF Burst model
– Congestion control allowing large CWND• Self-Similar Burst model
• Methods of analysis– Self-similar traffic– Queue De-multiplexing– Variable service rate
![Page 17: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/17.jpg)
Self-similar Burst Model• Alpha component = self-similar stable
– (limit of a few ON-OFF sources in the limit of fast time)
• This models heavy-tailed bursts – (heavy tailed files)
• TCP control: alpha CWND arbitrarily large – (short RTT, future TCP mutants)
• Analysis via De-Multiplexing:– Optimal setup of two individual Queues to come closest to
aggregate Queue
De-Multiplexing:Equal critical time-scales
Q-tail ParetoDue to Levy noise
Beta (top) + Alpha
![Page 18: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/18.jpg)
Rice University, SPiN Group spin.rice.edu
ON-OFF Burst Model• Alpha traffic = High rate ON-OFF source (truncated)• This models bi-modal bandwidth distribution• TCP: bottleneck is at the receiver (flow control
through advertised window)• Current state of measured traffic• Analysis: de-multiplexing and variable rate queue
Beta (top) + Alpha Variable Service Rate Queue-tail Weibull (unaffected) unless
• rate of alpha traffic larger than capacity – average beta arrival • and duration of alpha ON period heavy tailed
![Page 19: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/19.jpg)
Rice University, SPiN Group spin.rice.edu
Alpha traffic: Influence of TCP • All Alpha connections show
– Unusually small advertised window– Drastic drop in advertised window (sometimes to zero)– …which correlates with burst arrival
Flow controlled, Weibull Q-tails
![Page 20: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/20.jpg)
Rice University, SPiN Group spin.rice.edu
Separation on Connection Level
• Alpha connections: dominant. Properties:– Definition: Peak rate > mean arrival rate + 1 std dev– Few, light load– Responsible for violent bursts, large queuing delays– Typically short RTT– Typically FLOW-CONTROLLED (limited at receiver)
• Beta connections: Residual traffic– Main load– Gaussian, LRD– Typically limited at bottleneck link
• Future of empowered hosts and transfer protocols: – Higher peaks, larger bursts, longer queues
![Page 21: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/21.jpg)
Rice University, SPiN Group spin.rice.edu
Future work• : Network/user-driven traffic model
– Correlations between network and user– Through simulation and measurements assess impact of
protocols, applications, clientele, end-host server– Performance parameters from network and user specifications
• pathChirp – Model based estimation meeting challenges of bursty traffic – Through simulation validate realism (multihop, bursty traffic)– Anomaly detection through chirp-web
Current Collaborations & Tech Transfer–IP-tunneling, coordinated measurements (Telcordia)–Integration of PathChirp into network simulators (GaTech, UIUC)–Ready for integration into SPAWAR–Demystify self-similarity (UC Riverside)
![Page 22: Multiscale Traffic Processing Techniques for Network Inference and Control](https://reader036.vdocument.in/reader036/viewer/2022062409/5681466b550346895db38fd0/html5/thumbnails/22.jpg)
Rice University, SPiN Group spin.rice.edu
spin.rice.edu