measuring isp topologies with rocketfuel · 2018. 1. 29. · • complete city-level...
TRANSCRIPT
![Page 1: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/1.jpg)
Measuring ISP topologies with Rocketfuel
Ratul MahajanNeil Spring
David Wetherall
University of Washington
![Page 2: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/2.jpg)
ratul // ipam // 02 2
T
T
T
What are we after?
u High fidelity router-level map of ISPs• Routers§ backbone§ gateway§ directly connected customers and peers
• Links between these routers• Understanding the structure of backbone and PoPs
u ISP vs AS• customer networks not included• dialup and broadband access networks not included
![Page 3: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/3.jpg)
ratul // ipam // 02 3
T
T
T
Why measure?
u Network topologies closely guardedu Inaccurate or incomplete official versions
u Want higher fidelity than current mapping efforts• not just backbone• we found 7 times more routers and links than Skitter by
focusing on one ISP at a time
![Page 4: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/4.jpg)
ratul // ipam // 02 4
T
T
T
But, what is it good for?
u Realistic topologies for design and simulation• e.g., studying intra-domain routing protocols
u Generating synthetic topologies• Synthetic ISP maps + Synthetic AS-level map = Synthetic Internet map
u Understanding design and engineering principles behind these networks
u Understanding how ISPs connect to customers and other ISPs
![Page 5: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/5.jpg)
ratul // ipam // 02 5
T
T
T
Methodology
u Traceroute • known limitations• but pretty much the only tool• need multiple vantage points
T
T
T
u Public traceroute servers• 1000s of servers out there
§ we used 300 servers that provided 800 vantage points
• no deployment required
![Page 6: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/6.jpg)
ratul // ipam // 02 6
T
T
T
Challenges
u Software engineering• understanding the language spoken by each server• system architecture
u Reducing measurements• traceroute servers are a public resource• network admins do not like getting too many probe packets• to make it even possible
u Identifying ISP routersu Alias resolutionu Geographical and role information of routers
![Page 7: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/7.jpg)
ratul // ipam // 02 7
T
T
TReducing measurements (making it possible)
u Goal: reduce the number of traces as much as possible with as little loss of detail as possible
1. Identify traces that are likely to transit the ISP• BGP directed probing: leverage information in the BGP routing
tables
2. Identify likely duplicate traces • Path reductions: exploit properties of IP routing
Another helpful trick: find a live host within a prefix to speed up the traceroute
![Page 8: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/8.jpg)
ratul // ipam // 02 8
T
T
T
BGP directed probing
Using the BGP tables identify traces that are likely to transit the ISP
TT T
1. Traces from anywhere to dependent prefixes
Dependentprefix
Insider vantage point
T
2. Traces from insidersto anywhere
TTT
3. Traces from upstream ASes that use the ISP to reach some prefix
Upstream AS-1
Dst prefix P1
![Page 9: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/9.jpg)
ratul // ipam // 02 9
T
T
T
Path reductions
u Interested only in part of the trace that traverses the ISPu Avoid duplicate tracesu Exploit properties of IP routing to determine if paths
through the ISP is likely to be identical
1. Ingress reduction2. Egress reduction3. Next-hop AS reduction
![Page 10: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/10.jpg)
ratul // ipam // 02 10
T
T
T
Ingress reduction
u Path taken by a packet in the network is dependent only on the destination
u Shared ingress points
T T
Ingress reduction: when two traces to the same destination enter the ISP at the same point, only one of them is required
Also helps to load balance across traceroute servers
![Page 11: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/11.jpg)
ratul // ipam // 02 11
T
T
T
Egress reduction
u Dependent subprefixes attach to the ISP at a unique point
u Some egresses would be sharedu Requires egress discovery
• finding possibly multiple egresses for a dependent prefix
• done locally
Egress reduction: when two traces from the same ingress to destinations that attach to the ISP at the same point, only one of them is required
T
![Page 12: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/12.jpg)
ratul // ipam // 02 12
T
T
T
Next-hop AS reduction
u “Hot potato” routing• traces from same ingress to
destinations behind the same next-hop AS exit the ISP at the same point
u Next-hop AS information present in the BGP table
Next-hop AS reduction: when two traces from the same ingress cross over to the same next-hop AS, only one of them is required
T
![Page 13: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/13.jpg)
ratul // ipam // 02 13
T
T
TMeasurement reduction summary
u BGP directed probing• every vantage point to dependent prefixes• insiders to all prefixes• upstream ASes to downstream prefixes
u Path reductions (composable)• ingress reduction• egress reduction• next-hop AS reduction
u Helped reduce the number of measurements by three orders of magnitude
![Page 14: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/14.jpg)
ratul // ipam // 02 14
T
T
T
Challenges
u Software engineering• send request to each server in a language it understands• understanding their replies
u Reducing measurements• traceroute servers are a public resource• network admins do not like getting too many probe packets• to make it even possible
u Identifying ISP routersu Alias resolutionu Geographical and role information of routers
![Page 15: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/15.jpg)
ratul // ipam // 02 15
T
T
T
Identifying ISP routers
What part of the trace belongs to the ISP?
u Use DNS names• all Verio routers are *.verio.net• prune out cable modems, DSL and dial-up modem pools• customer networks have different DNS names
u Use BGP tables in absence of names• address space advertized by the ISP
![Page 16: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/16.jpg)
ratul // ipam // 02 16
T
T
T
Alias resolution
u Resolving multiple IP addresses of the same router
u Return IP address• the return address for packets
generated by the router is that of the outgoing interface
IP0
IP1
IPx
to IP0
to IP1from IPx
from IPx
u IP Identifier• closely spaced responses from
the same router will have close IP ID
• finds 3 times more aliases than return IP address approach
IP0
IP1
IPx
to IP0
to IP1
ID = X + ∆
ID = X
![Page 17: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/17.jpg)
ratul // ipam // 02 17
T
T
T
Location and role discovery
u Where is this router located?• use DNS names§ sl-bb11-nyc-3-0.sprintlink.net is a Sprint router in New
York City
• use connectivity information§ if a router connects only to routers in Seattle, it probably is in Seattle
u What role does this router play in the topology?• only backbone routers connect to other cities • use DNS names§ sl-gw2-sea-3-1.sprintlink.net is a Sprint gateway router
![Page 18: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/18.jpg)
ratul // ipam // 02 18
T
T
T
Rocketfuel architecture
![Page 19: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/19.jpg)
ratul // ipam // 02 19
T
T
T
Summary statistics
ASN Name Routers Links PoPs
1221 Telstra (Aus) 4,440 4996 54
1239 Sprint 11,889 15,263 25
1755 Ebone (Eur) 438 1,192 26
2914 Verio 7,574 19,175 103
3257 Tiscali (Eur) 618 839 52
3356 Level3 2,064 8,669 44
3967 Exodus 688 2,166 22
4755 VSNL (India) 664 484 8
6461 Abovenet 843 2,667 22
7018 AT&T 13,993 18,083 109
![Page 20: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/20.jpg)
ratul // ipam // 02 20
T
T
T
Validation
u Talk to ISPs• complete city-level interconnection (backbone); PoPs “looked fine”• spurious links (tools lie) L
u Comparison with peering in BGP data • number of router-level peerings observed• >90% for most ISPs; worst case was Sprint with >70%
u Comparison with Skitter• we have 7 times more routers and link
u Conclusions • our maps are fairly complete• very useful for qualitative inferences • our maps are not authoritative
![Page 21: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/21.jpg)
ratul // ipam // 02 21
T
T
T
AT&T (AS-7018)
Background image courtesy JHU, applied physics labs
![Page 22: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/22.jpg)
ratul // ipam // 02 22
T
T
T
Sprint (AS-1239)
Background image courtesy JHU, applied physics labs
![Page 23: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/23.jpg)
ratul // ipam // 02 23
T
T
T
Level3 (AS-3356)
Background image courtesy JHU, applied physics labs
![Page 24: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/24.jpg)
ratul // ipam // 02 24
T
T
T
Telstra (AS-1221)
Background image courtesy National Geographic
![Page 25: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/25.jpg)
ratul // ipam // 02 25
T
T
T
Backbone structure
u Different across ISPs• AT&T and Telstra - hubs and spokes• Sprint - fewer, bigger PoPs• Level3 - circuits
![Page 26: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/26.jpg)
ratul // ipam // 02 26
T
T
T
PoP structure
Sprint PoP in Springfield, MA
u Backbone routers within a PoP are densely connected
u Gateway routers connect to two backbone routers
u Customers connect to gateway routers
u High variability in PoP sizes• more than two orders of
magnitude• [not a power law]
![Page 27: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/27.jpg)
ratul // ipam // 02 27
T
T
T
Ongoing and future work
u Releasing maps and data to the communityu Analysis
• peering strucuture
u Alias resolutionu Map annotations
• latency, bandwidth, link weights
u Synthetic generation
![Page 28: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/28.jpg)
ratul // ipam // 02 28
T
T
T
Conclusions
u 1000s of public traceroute servers can be used for high-fidelity map collection• measurement reduction is the main challenge• routing tables and properties of IP routing can be exploited• DNS names are very useful for studying various aspects of collected
maps
u ISPs have different backbone structureu PoPs have similar designs; two orders of magnitude difference in PoP
sizes within an ISP
u More information: {ratul,nspring,djw}@cs.washington.edu
![Page 29: Measuring ISP topologies with Rocketfuel · 2018. 1. 29. · • complete city-level interconnection (backbone);PoPs “looked fine” • spurious links (tools lie) L uComparison](https://reader036.vdocument.in/reader036/viewer/2022071218/605389bb6ef804794e149d0c/html5/thumbnails/29.jpg)
ratul // ipam // 02 29
T
T
T
Acknowledgements
u Mapped ISPs, especially those who validated our maps
u Public traceroute servers, RouteViews
u Steve Bellovin, Randy Bush, Christophe Diot and Ramesh Govindan
u CAIDA - visualization tools and Skitter data
u Venkat and Lakshmi - GeoTrack city codes