![Page 2: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/2.jpg)
Polly Huang, NTU EE 2
Dynamics Papers
• Hongsuda Tangmunarunkit, Ramesh Govindan, and Scott Shenker. Internet path inflation due to policy routing. In Proceedings of the SPIE ITCom, pages 188-195, Denver, CO, USA, August 2001. SPIE
• Lixin Gao. On inferring automonous system relationships in the internet. ACM/IEEE Transactions on Networking, 9(6):733-745, December 2001
• Vern Paxson. End-to-end internet packet dynamics. ACM/IEEE Transactions on Networking, 7(3):277-292, June 1999
• Craig Labovitz, G. Robert Malan, Farnam Jahanian. Internet Routing Instability. ACM/IEEE Transactions on Networking, 6(5):515-528, October 1998
![Page 3: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/3.jpg)
Polly Huang, NTU EE 3
Doing Your Own Analysis
• Having a problem
• Need to simulate or to test
• Define experiments– Base scenarios– Scaling factors– Metrics of investigation
![Page 4: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/4.jpg)
Polly Huang, NTU EE 4
Base Scenarios
• The source models– To generate traffic
• The topology models– To generate the network
• Then?
![Page 5: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/5.jpg)
Polly Huang, NTU EE 5
Internet Dynamics
• How traffic flow across the network– Routing– Shortest path?
• How failures occur– Packets dropped– Routes failed– i.i.d?
Policy routing
Packet/Route dynamics
![Page 6: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/6.jpg)
Identifying Internet Dynamics
Routing Policy
Packet Dynamics
Routing Dynamics
![Page 7: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/7.jpg)
To the best of our knowledge, we could now generate:
AS-level topology
Hierarchical router-level topology
![Page 8: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/8.jpg)
Polly Huang, NTU EE 8
The Problem
• Does it matter what routing computation we use?
• Equivalent of – Can I just do shortest path computation?
![Page 9: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/9.jpg)
Polly Huang, NTU EE 9
Topology with Policy
• Internet Path Inflation Due to Policy Routing
• Hongsuda Tangmunarunkit, Ramesh Govindan, Scott Shenker
• In Proceedings of the SPIE ITCom, pages 188-195, Denver, CO, USA, August 2001. SPIE
![Page 10: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/10.jpg)
Polly Huang, NTU EE 10
Paper of Choice
• Methodological value– A simple ‘re-examine’ type of study– To strengthen technical value of prior work
• Technical value– Actual paths are not the shortest due to routing policy.– The routing policy is business-driven and can be quite
hard to obtain. – Shown in this paper, for simulation study concerning
large-scale route path characteristics, a simple shortest-AS policy routing may be sufficient.
![Page 11: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/11.jpg)
Polly Huang, NTU EE 11
shortest
Inter-AS Routing
AS 1
AS 3AS 2
AS 4
AS 5
source destination
![Page 12: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/12.jpg)
Polly Huang, NTU EE 12
Hierarchical Routing
Inter-AS shortest
sourcedestination
Intra-AS shortest
![Page 13: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/13.jpg)
Polly Huang, NTU EE 13
Flat Routing
sourcedestination
shortest
![Page 14: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/14.jpg)
5:3
Hierarchical Routing is not optimal
Or
Routes are inflated
![Page 15: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/15.jpg)
How sub-optimal?
![Page 16: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/16.jpg)
Polly Huang, NTU EE 16
Prior Work
• Based on – An actual router-level graph– An actual AS-level graph at the same time– Overlay the AS-level graph on the router-level graph
• Compute– For each source-destination pair– Shortest path using hierarchical routing– Shortest path using flat routing
• Compare route length – In number of router hops
![Page 17: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/17.jpg)
Polly Huang, NTU EE 17
Prior Conclusions
• 80% of the paths are inflated
• 20% of the paths are inflated > 50%
• There exists a better detour for 50% of the source-destination pairs– There exists an intermediate node i such that Le
ngth(s-i-d) < Length(s-d)
![Page 18: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/18.jpg)
Polly Huang, NTU EE 18
This Work
• To address 2 shortcomings– There’s now a newer router-level graph– There’s now a more sophisticated policy model
• Paper #4
• Inter-AS routing is not quite ‘shortest-AS routing’
![Page 19: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/19.jpg)
Polly Huang, NTU EE 19
Newer vs. Older Graph
• Inflation difference not the same– Difference is larger in the newer graph– Due to the newer graph being larger
• Inflation ratio remains the same
![Page 20: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/20.jpg)
Polly Huang, NTU EE 20
Shortest-AS vs. Policy-AS Routing
• Shortest-AS– Simplified model
– Every AS is equal
• Policy-AS– Realistic model
– Not all ASs are the same• Some are provider ASs
• Some are customer ASs
• Customer ASs do not transit traffic
![Page 21: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/21.jpg)
Polly Huang, NTU EE 21
Consider TANET CHT
CHT
NTU
TANET
UUNET
Through NTU?
Through UUNET?
Provider
Customer
![Page 22: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/22.jpg)
Polly Huang, NTU EE 22
Routing with Constraints
• Routes could be– Going up – Going down– Going up and then down
• Routes can never be– Going down and then up
![Page 23: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/23.jpg)
Polly Huang, NTU EE 23
Inferring the Constraints
• On Inferring Autonomous System Relationships in the Internet
• Lixin Gao
• ACM/IEEE Transactions on Networking, 9(6):733-745, December 2001
![Page 24: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/24.jpg)
Polly Huang, NTU EE 24
Not All ASs the Same
• 2 types of ASs– Customer– Provider
• 3 types of Relationships– Customer-provider– Provider-provider
• Peer-peer
• Sibling-sibling
![Page 25: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/25.jpg)
Polly Huang, NTU EE 25
Customer-Provider
• Formal definition– A provider transits for its customer
– A customer does no transit for its provider
• Informal– Provider: I’ll take any traffic
– Customer: I’ll take only the traffic to me (or my customers)
![Page 26: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/26.jpg)
Polly Huang, NTU EE 26
Peer-Peer
• Formal Definition– A provider does not transit for another provider
• Informal– I’ll take only the traffic to me (or my customers)
– You’ll take only the traffic to you (or your customers)
![Page 27: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/27.jpg)
Polly Huang, NTU EE 27
Sibling-Sibling
• Formal Definition– A provider transits for another provider
• Informal– I’ll take any traffic
– You’ll take any traffic
![Page 28: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/28.jpg)
Polly Huang, NTU EE 28
Never “Going Down and then Up”
• A provider-customer link can be followed by only– Provider-customer link
– (Or sibling-sibling link)
• A peer-peer link can be followed by only– Provider-customer link
– (Or sibling-sibling link)
![Page 29: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/29.jpg)
Polly Huang, NTU EE 29
Heuristics
• Compute out-degrees
• For each AS path in routing tables– 1st AS with the max degree the root of hierarchy– From the root, drawing providercustomer
relationship down 2 ends of the AS path
![Page 30: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/30.jpg)
Polly Huang, NTU EE 30
Determining Siblings
• After gone through all AS paths
• Any AS pair being both provider and customer to each other are siblings
![Page 31: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/31.jpg)
Polly Huang, NTU EE 31
Determining Peers
• Do another pass on the AS paths in routing tables
• For each AS path– Top AS who does not have sibling relationships
with the neighboring ASs– Could have peering relationship with the higher
out-degree neighbor – Given the Top AS and the higher out-degree ne
ighbor are comparable in out-degree
![Page 32: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/32.jpg)
Polly Huang, NTU EE 32
Back to Path Inflation
• Draw the customer-provider, peer-peer, and sibling-sibling relationships on the overlay AS graph
• Compute the best routes under the ‘never going down and then up’ constraint
• Compare the inflation difference and ratio again with these running at the inter-AS level– Shortest – Policy
![Page 33: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/33.jpg)
Polly Huang, NTU EE 33
Shortest vs. Policy Routing
• Pretty much the same both in terms of – Inflation difference
– Inflation ratio
![Page 34: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/34.jpg)
Polly Huang, NTU EE 34
Therefore
• The observations from the prior work holds– With a newer graph– With the more realistic inter-AS policy routing
![Page 35: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/35.jpg)
Now forget path inflation
How far away is the shortest to the policy inter-AS routing?
![Page 36: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/36.jpg)
Polly Huang, NTU EE 36
Shortest vs. Policy
• In AS hops– 95% paths have the same length– Policy routes always longer
• In router hops– 84% paths have the same length– Some policy routes longer, some shorter
![Page 37: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/37.jpg)
95% and 84% are pretty good numbers
Therefore shortest path at the inter-AS level might be OK…
![Page 38: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/38.jpg)
Polly Huang, NTU EE 38
To Answer the Question
• Can we simply do shortest path computation?– A likely yes for AS-level graph– A firm no for hierarchical graph
• Must separate inter-AS shortest and intra-AS shortest
![Page 39: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/39.jpg)
Questions?
![Page 40: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/40.jpg)
Identifying Internet Dynamics
Routing Policy
Packet Dynamics
Routing Dynamics
![Page 41: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/41.jpg)
It’s never a perfect world…
![Page 42: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/42.jpg)
Polly Huang, NTU EE 42
The Problem
• But how perfect is the Internet?
• The Internet– A network of computers with stored information
– Some valuable, some relevant
– You participate by putting information up or getting information down
– From time to time, you can’t quite do some of these things you want to do
![Page 43: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/43.jpg)
Why is that?
![Page 44: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/44.jpg)
At the philosophical level…
Humans are so bound to failures.And the Internet is human-made.
![Page 45: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/45.jpg)
But, Seriously…
Consider loading a Web page
![Page 46: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/46.jpg)
Polly Huang, NTU EE 46
Web Surfing Failures
• The ‘window’ waving forever?
• An error message saying network not reachable
• An error message saying the server too busy
• An error message saying the server is down
• Anything else?
![Page 47: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/47.jpg)
Polly Huang, NTU EE 47
Network Specific Failures
• The ‘window’ waving forever?
• An error message saying network not reachable
• An error message saying the server too busy
• An error message saying the server is down
• Anything else?
![Page 48: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/48.jpg)
Polly Huang, NTU EE 48
The Causes
• The ‘window’ waving forever– Congestion in the network
– Buffer overflow
– Packet drops
• An error message saying network not reachable– Network outage
– Broken cables, Frozen routers
– Route re-computation
– Route instability
![Page 49: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/49.jpg)
Polly Huang, NTU EE 49
Back to the Problem
• But how perfect is the Internet?
• Equivalent of– Packets can be dropped
• How frequent• How much
– Routes may be unstable• How frequent• For how long
![Page 50: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/50.jpg)
Polly Huang, NTU EE 50
Significance
• Knowing the characteristics of packet drops and route instability helps – Design for fault-tolerance– Test for fault-tolerance
![Page 51: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/51.jpg)
There are tons of formal/informal study on the dynamics…
Let’s take a look at a couple that are classical
![Page 52: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/52.jpg)
Polly Huang, NTU EE 52
Packet Dynamics
• End-to-End Internet Packet Dynamics
• Vern Paxson
• ACM/IEEE Transactions on Networking, 7(3):277-292, June 1999
![Page 53: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/53.jpg)
Polly Huang, NTU EE 53
Emphasis in Reverse Order
• Real subject of study– Packet loss– Packet delay
• Necessary assessment– The unexpected– Bandwidth estimation
![Page 54: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/54.jpg)
Polly Huang, NTU EE 54
Measurement
• Instrumentation– 35 sites, 9 countries– Education, research, provider, company
• 2 runs– N1: Dec 1994– N2: Nov-Dec 1995– 21 sites in common
![Page 55: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/55.jpg)
Polly Huang, NTU EE 55
Measurement Methodology
• Each site running NPD – A daemon program– Sender side sends 100KB TCP transfer
• Sender and receiver sides both – tcpdump the packets
• Noteworthy– Measurement occurred in Poisson arrival
• Unbiased to time of measurement
– N2 used big max window size• Prevent window size to limit the TCP connection throughput
![Page 56: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/56.jpg)
Polly Huang, NTU EE 56
Packet Loss
• Overall loss rate:– N1 2.7%, N2 5.2%– N2 higher, because of big max window?
• I.e. Pumping more data into the network therefore more loss?
• Big max window in N2 is not a factor– By separating data and ack loss– Assumption: ack traffic in a half lower rate
• Won’t stress the network
– Ack loss: N1 2.88%, N2 5.14%– Data loss: N1 2.65%, N2 5.28%
![Page 57: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/57.jpg)
Polly Huang, NTU EE 57
Quiescent vs. Busy
• Definition– Quiescent: connections without ack drops– Busy: otherwise
• About 50% of the connections are quiescent
• For connections are busy– Loss rate: N1 5.7%, N2 9.2%
![Page 58: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/58.jpg)
Polly Huang, NTU EE 58
More Numbers
• Geographical effect
• Time of the day effect
![Page 59: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/59.jpg)
Polly Huang, NTU EE 59
Towards a Markov Chain Model
• For hours long– No-loss connection now indicates further no-loss conne
ction in the future
– Lossy connection now indicates further lossy connections in the future
• For minutes long– The rate remains similar
pn
No loss Loss
pl1-pn
1-pl
![Page 60: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/60.jpg)
Polly Huang, NTU EE 60
Another Classification
• Data– Loaded data: packets experiencing queueing delay due t
o own connection
– Unloaded data: packets not experiencing queueing delay due to own connection
– Bottleneck bandwidth measurement is needed here to determine whether a packet is loaded or not
• Ack– Simply acks
![Page 61: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/61.jpg)
Polly Huang, NTU EE 61
3 Major Observations
• Although loss rate very high (47%, 65%, 68%), all connections complete in 10 minutes
• Loss of data and ack not correlated• Cumulative distribution of per connection loss rate
– Exponential for data
– Not so exponential for ack
– Adaptive sampling contributing to the exponential observation?
![Page 62: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/62.jpg)
Polly Huang, NTU EE 62
More on the Markov Chain Model
• The loss rate Pu – The rate of loss
• The conditional loss rate Pc– The rate of loss when the previous packet is lost
• Contrary to the earlier work– Losses are busty– Duration shows pareto upper tail – (Polly: maybe more log-normal)
![Page 63: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/63.jpg)
Polly Huang, NTU EE 63
You might ask…pl ,pn?
pn
No loss Loss
pl1-pn
1-pl
![Page 64: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/64.jpg)
Polly Huang, NTU EE 64
Values for the pl’s
N1 N2
Loaded data 49% 50%
Unloaded data 20% 25%
Ack 25% 31%
![Page 65: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/65.jpg)
Polly Huang, NTU EE 65
Possible Invariant
• Conditional loss rate
• For the value remains relatively close over the 1 year period
• More up-to-date data to verifying this?
• The loss burst size log normal?
• Both interested research questions
![Page 66: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/66.jpg)
Polly Huang, NTU EE 66
Packet Delay
• Looking at one-way transit times (OTT)• There’s model for OTT distribution
– Shifted gamma– Parameters changes with regards to time and
path…
• Internet path are asymmetric– OTT one way often not equal OTT the other
way
![Page 67: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/67.jpg)
Polly Huang, NTU EE 67
Timing Compression
• Ack compressions are small events
• So not really pose threads on– Ack clocking– Rate estimation based control
• Data compression very rare– For outlier filtering
![Page 68: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/68.jpg)
Polly Huang, NTU EE 68
Queueing Delay
• Variance of OTT over different time scales– For each time scale – Divide the packets arrival into intervals of – For all 2 neighboring intervals l, r
• ml the median of OTT in interval l
• mr the median of OTT in interval r
• Calculate (ml-mr)
• Variance of OTT over is median of all (ml-mr)
![Page 69: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/69.jpg)
Polly Huang, NTU EE 69
Finding the Dominant Scale
• Looking for ’s whose queueing variance are large– Where control most needed
• For example, if those ’s re smaller than RTT– Then TCP doesn’t need to bother adapting to q
ueueing fluctuations
![Page 70: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/70.jpg)
Polly Huang, NTU EE 70
Oh Well
• Queueing delay variations occur– Dominantly on 0.1-1 sec scales– But non-negligibly on larger scales
![Page 71: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/71.jpg)
Polly Huang, NTU EE 71
Share of Bandwidth
• Pretty much uniformly distributed
![Page 72: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/72.jpg)
Polly Huang, NTU EE 72
Conclusions on Analysis
• Common assumptions violated– In-order packet delivery– FIFO queueing– Independent loss– Single congestion time scale– Path asymmetry
• Behavior– Very wide range, not one typical
![Page 73: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/73.jpg)
Polly Huang, NTU EE 73
Conclusions on Design
• Measurement methodology– TCP-based measurement shown viable– Sender-side only inferior
• TCP implementation– Sufficiently conservative
![Page 74: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/74.jpg)
The Pathologies
The strange stuff
![Page 75: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/75.jpg)
Polly Huang, NTU EE 75
Packet Re-Ordering
• Varying widely and too few samples• Therefore, deriving only a rule of thumb
– The Internet paths sometimes experience bad reordering
– Mainly due to route flapping
– Occasionally this funny case of router implementation• Buffering packets while processing a route update
• Sending these packets interleaving with the post-update arrivals
![Page 76: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/76.jpg)
Polly Huang, NTU EE 76
Orthogonal to TCP SACK
• Receiver end modification– 20 msec wait before sending duplicate acknowledgeme
nt
– Waiting for re-ordered packets therefore lower false duplicate acknowledge
– Dup acks should be indication of losses
• Sender end motification– Fast retransmission after 2 duplicate acknowledgements
– Reactive fast retransmission, higher throughput
![Page 77: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/77.jpg)
Polly Huang, NTU EE 77
Packet Replication
• Very strange, can’t quite explain– A pair of acks duped 9 times, arriving 32 msec apart
– A data packet duped 23 times, arriving in burst• False-configured bridge?
• Observation– Most of these site specific
– But small number of dups spread between other sites
– Senders dup packets too
![Page 78: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/78.jpg)
Polly Huang, NTU EE 78
Packet Corruption
• Checksum good?
• Problem– The traces contain only the header data– Pure ack OK, the header = the packet– Data not OK, the header <> the packet
• Use an corruption inferring algorithm in tcpanaly
![Page 79: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/79.jpg)
Polly Huang, NTU EE 79
Corruption Rate
• 1 corruption out of 5000 data packets• 1 corruption out of 300,000 pure acks
• Possible reasons of the difference– Header compression– Packet size– Inferring tool discrepancy– Other router/link level implementation artifacts
![Page 80: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/80.jpg)
Polly Huang, NTU EE 80
Implication
• 16-bit checksum no longer sufficient– A corrupted packet has a one 216th chance to have the s
ame checksum as the non-corrupted packet– I.e., one out of the 216 corrupted packet can’t be detecte
d by the checksum
• Since 1 out of 5000 data packets is corrupted– 1 out of 5000 * 216 (300 M) packets can’t be identified a
s corrupted by the TCP 16-bit checksum– Consider one Gbps link and packet size 1Kb 1M Pps– 3 seconds per falsely received corrupted packet
![Page 81: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/81.jpg)
Polly Huang, NTU EE 81
Estimating Bottleneck Bandwidth
• The packet pair technique– Send 2 packets back to back (or close enough)
• Inter-packet time, T2-T1, very small
– When then go across the bottleneck• Serving packet 1 while packet 2 will be queued
• Packet 2 immediately follow packet 1
– Packets will be stretched • Internet-packet time, T2-T1 , now the transmission time of
packet 1
– Estimated bandwidth = (Size of packet 1)/(T2-T1 )
![Page 82: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/82.jpg)
Polly Huang, NTU EE 82
This Won’t Work
• Bottleneck bandwidth higher than sending rate
• Out-of-order delivery
• Clock resolution
• Changes in the bottleneck bandwidth
• Multi bottlenecks
![Page 83: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/83.jpg)
Polly Huang, NTU EE 83
PBM
• Instead of sending a pair
• Send a bunch
• More robust again the multi bottleneck problem
![Page 84: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/84.jpg)
Questions?
![Page 85: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/85.jpg)
Identifying Internet Dynamics
Routing Policy
Packet Dynamics
Routing Dynamics
![Page 86: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/86.jpg)
Polly Huang, NTU EE 86
Route Instability
• Internet Routing Instability
• Craig Labovitz, G. Robert Malan, Farnam Jahanian
• ACM/IEEE Transactions on Networking, 6(5):515-528, October 1998
![Page 87: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/87.jpg)
Polly Huang, NTU EE 87
BGP Specific• BGP is an important part of the Internet
– Connecting the domains– Widespread– Known in prior work that route failure could result in
• Packet loss• Longer network delay• Network outage (Time to globally converge to local change)
• A closer look at the BGP dynamics– How much route updates are sent– How frequent are they sent– How useful are these updates
![Page 88: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/88.jpg)
Polly Huang, NTU EE 88
BGP (In a Slide)
• The routing protocol running among the border routers– Path Vector– Think DV– Exchange not just next hop, but entire path
• Dynamics– In case of link/router recovery
• Exchange from the recovering point the route announcements
– In case of link/router down• Exchange from the closed point the route withdraws
– Route updates• Including route announcements/withdraws
![Page 89: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/89.jpg)
Polly Huang, NTU EE 89
Data Collection
• Monitoring exchange of route updates– Over 9 month period– 5 public exchange points in the core
• Exchange point– Connecting points of ASs– Public exchange: of the US government– Private exchange: of the commercial providers
![Page 90: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/90.jpg)
Polly Huang, NTU EE 90
Terminology
• AS– You all know
– In the path of the path vector exchanged by BGP• AS-PATH
• Prefix– Basically network address
– The source/destination of the route entries in BGP• 140.119.154/24
• 140.119/16
![Page 91: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/91.jpg)
Polly Huang, NTU EE 91
Classification of Problems
• Forward instability– Legitimate topological changes affecting paths
• Routing policy fluctuation– Changes in routing policy but not affecting
forwarding paths
• Pathological updates– Redundant information not affecting routing
nor forwarding
![Page 92: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/92.jpg)
Polly Huang, NTU EE 92
Forwarding Instability
• WADiff– A route is explicitly withdrawn– Replaced with an alternative route– As it becomes unreachable– The alternative route is different in AS-PATH or next-hop
• AADiff– A route is implicitly withdrawn– Replaced with an alternative route– As it becomes unreachable or a preferred alternative route
becomes available
![Page 93: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/93.jpg)
Polly Huang, NTU EE 93
In the Middle• WADup
– A route is explicitly withdrawn– Then re-announced as reachable– Could be
• Pathological• Forwarding instability: transient topological change
• AADup– A route is implicitly withdrawn– Replaced with a duplicate of the original route
• Same AS-PATH and next-hop
– Could be • Pathological• Policy fluctuation: differ in other policy attributes
![Page 94: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/94.jpg)
Polly Huang, NTU EE 94
Pathological
• WWDup– Repeated withdraws for a prefix no longer reac
hable– Pathological
![Page 95: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/95.jpg)
Polly Huang, NTU EE 95
Observations – The Majority
• Pathological updates (redundant)– Minimum effect on
• Route quality
• Router processing load
– Some not agree– Adding significant amount of traffic
• 300 updates/second could crash a high-end router
![Page 96: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/96.jpg)
Polly Huang, NTU EE 96
Observation - Instability
• Forwarding instability– 3-10% WADiff– 5-20% AADiff– 10-50% WADup
• Policy fluctuation– AADup quite high – But most probably pathological
• Need this– The Internet routing works become of these necessary a
nd frequent updates
![Page 97: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/97.jpg)
Polly Huang, NTU EE 97
Observation – Distribution
• No spacial correlation– Correlates to router implementation instead
• Temporal– Time the the date effect, date of the week effect
– Therefore correlates to network congestion
• Periodicity– 30, 60 second period
– For self-sync, mis-configuration, BGP is soft-state based, etc
![Page 98: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/98.jpg)
Basically, not saying much…
But for the background
And ease of reading
![Page 99: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/99.jpg)
Questions?
![Page 100: Network Simulation and Testing Polly Huang EE NTU phuang phuang@cc.ee.ntu.edu.tw](https://reader033.vdocument.in/reader033/viewer/2022061614/56649ee85503460f94bfa285/html5/thumbnails/100.jpg)
Polly Huang, NTU EE 100
What Should You Do?
• Routing policy– Intra-AS: shortest path– Inter-AS: shortest path (95%, 84% OK)– Better model in progress…
• Packet losses– 2-state markov chain model
• pl: some info• pn: no info…
• Routing instability: outage time– The paper #2 of the original paper set (OSPF vs. DV)