Download - Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations
![Page 1: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/1.jpg)
Joint Data Streaming and Sampling Techniques for Detection of Super
Sources and DestinationsQi (George) Zhao, Abhishek Kumar, Jun (Jim) Xu
College of Computing, Georgia Institute of TechnologyInternet Measurement Conference 2005
Speaker: Yongming Chen on Dec. 07, 2006
![Page 2: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/2.jpg)
Outline
Introduction Motivation AlgorithmsEvaluation Conclusion
![Page 3: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/3.jpg)
Outline
Introduction Motivation AlgorithmsEvaluation Conclusion
![Page 4: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/4.jpg)
Introduction
detect super sources/destinations at high link speeds (10 to 40 Gbps) in real-time super sources: source with a large fan-out fan-out: number of distinct destinations
data streaming: sequentially process every packet passing through a better alternative to sampling and monitoring
of high-speed links NO connection with multimedia streaming!
![Page 5: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/5.jpg)
Outline
Introduction Motivation AlgorithmsEvaluation Conclusion
![Page 6: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/6.jpg)
Motivation
detecting super sources and destinations is useful network monitoring and security hot-spot or flash crowds detection
traditional per-flow schemes cannot scale to high-speed links information lost in packet sampling FlowScan: maintain per-flow state with hash table
network data streaming with a small but well-organized data structure
![Page 7: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/7.jpg)
Outline
Introduction Motivation Algorithms
Simple Scheme Advanced Scheme
Evaluation Conclusion
![Page 8: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/8.jpg)
Simple Scheme
traditional hash-based flow sampling reduce the amount of incoming traffic,
e.g., a 10M pkt/s link can be processed in 400ns with 25% sampling rate
this approach will fail with traffic burst
filtering after sampling
![Page 9: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/9.jpg)
Simple Scheme
![Page 10: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/10.jpg)
Simple Scheme
at most one packet from each sampled flow need to be processed
need a umin to reduce estimation error typically set to w/2
extremely low storage complexity only 128KB SRAM is needed for OC-192 links
with 25% flow sampling
![Page 11: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/11.jpg)
Advanced Scheme
Separation of counting and identity gathering: streaming module encodes fan-out information sampling module captures candidate source
![Page 12: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/12.jpg)
On-line Streaming Module
![Page 13: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/13.jpg)
Estimation Module
Fs : fan-out of source s
Ai, i=1 to k : the column indices we can obtain by hashing s with h1 to hk
Ti : the set of packets hashed into Ai
UTi : # ‘0’ bits in Ai
DTi : a fairly accurate estimator of |Ti|
![Page 14: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/14.jpg)
Outline
Introduction Motivation AlgorithmsEvaluation Conclusion
![Page 15: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/15.jpg)
Settings
packet header traces IPKS+ and IPKS- : OC192c USC : Los Nettos tracing facility at USC UNC : 1Gbps
flow label (<src_ip, src_port>, <dst_ip>) (<src_ip>, <dst_ip, dst_port>)
![Page 16: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/16.jpg)
Simple Scheme
![Page 17: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/17.jpg)
Advanced Scheme
![Page 18: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/18.jpg)
Outline
Introduction Motivation AlgorithmsEvaluation Conclusion
![Page 19: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/19.jpg)
Conclusion
combine the power of data streaming and sampling to perform efficient and accurate detection
but no comparison with other approaches in evaluation
![Page 20: Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations](https://reader035.vdocument.in/reader035/viewer/2022062315/56815706550346895dc4ac74/html5/thumbnails/20.jpg)
Reference
Q. Zhao, A. Kumar, J. Xu. Joint Data Streaming and Sampling Techniques for Detection of Super Sources and Destinations. Internet Measurement Conference 2005.
Q. Zhao, A. Kumar, and J. Xu. Joint data streaming an sampling techniques for detection of super sources and destinations. In Technical Report, July 2005.
FlowScan: http://www.usenix.org/events/lisa2000/full_papers/plonka/plonka_html/