mapping the internet and intranets

1 of 75Mapping the Internet and Intranets

75 slides

Mapping the Internet and

IntranetsBill Cheswick

[email protected]

http://www.cheswick.com

mailto:[email protected]


Motivations• Intranets are out of

control– Always have been

• Highlands “day after” scenario

• Panix DOS attacks– a way to trace

anonymous packets back!

• Internet tomography

• Curiosity about size and growth of the Internet

• Same tools are useful for understanding any large network, including intranets


Related Work• See Martin Dodge’s cyber geography page

• MIDS - John Quarterman

• CAIDA - kc claffy

• Mercator

• “Measuring ISP topologies with rocketfuel” - 2002– Spring, Mahajan, Wetherall

• Enter “internet map” in your search engine

http://portal.acm.org/results.cfm?query=%22P208053%22%20%3CIN%3E%20author&querydisp=%22Neil%20%20Spring%22%20%3CIN%3E%20author&coll=portal&dl=ACM&CFID=11111111&CFTOKEN=2222222

http://portal.acm.org/results.cfm?query=%22P347085%22%20%3CIN%3E%20author&querydisp=%22Ratul%20%20Mahajan%22%20%3CIN%3E%20author&coll=portal&dl=ACM&CFID=11111111&CFTOKEN=2222222

http://portal.acm.org/results.cfm?query=%22P64337%22%20%3CIN%3E%20author&querydisp=%22David%20%20Wetherall%22%20%3CIN%3E%20author&coll=portal&dl=ACM&CFID=11111111&CFTOKEN=2222222


The Goals• Long term reliable

collection of Internet and Lucent connectivity information– without annoying

too many people

• Attempt some simple visualizations of the data

– movie of Internet growth!

• Develop tools to probe intranets

• Probe the distant corners of the Internet


Methods - data collection• Single reliable host connected at the

company perimeter

• Daily full scan of Lucent

• Daily partial scan of Internet, monthly full scan

• One line of text per network scanned– Unix tools


Methods - network scanning• Obtain master network list

– network lists from Merit, RIPE, APNIC, etc.– BGP data or routing data from customers– hand-assembled list of Yugoslavia/Bosnia

• Run a traceroute-style scan towards each network

• Stop on error, completion, no data– Keep the natives happy


TTL probes• Used by traceroute and other tools

• Probes toward each target network with increasing TTL

• Probes are ICMP, UDP, TCP to port 80, 25, 139, etc.

• Some people block UDP, others ICMP


TTL probes

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

RouterApplication level

TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3 Hop 4


Send a packet with a TTL of 1…

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware


TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3 Hop 4


…and we get the death notice from the first hop

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware


TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3 Hop 4


Send a packet with a TTL of 2…

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware


TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3 Hop 4


… and so on …

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware


TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3 Hop 4


Advantages• We don’t need access (I.e. SNMP) to the

routers

• It’s very fast

• Standard Internet tool: it doesn’t break things

• Insignificant load on the routers

• Not likely to show up on IDS reports

• We can probe with many packet types


Limitations• Outgoing paths only

• Level 3 (IP) only– ATM networks appear as a single node– This distorts graphical analysis

• Not all routers respond

• Many routers limited to one response per second


Limitations• View is from scanning host only

• Takes a while to collect alternating paths

• Gentle mapping means missed endpoints

• Imputes non-existent links


The data can go either way

A

E F

D

B C


But our test packets only go part of the way

A

E F

D

B C


We record the hop…

A

E F

D

B C


The next probe happens to go the other way

A

E F

D

B C


…and we record the other hop…

A

E F

D

B C


We’ve imputed a link that doesn’t exist

A

E F

D

B C


Data collection complaints• Australian parliament was the first to

complain• List of whiners (25 nets)• Military noticed immediately

– Steve Northcutt– arrangements/warnings to DISA and CERT

• These complaints are mostly a thing of the past– Internet background radiation

predominates


Visualization goals• make a map

– show interesting features– debug our database and collection

methods– hard to fold up

• geography doesn’t matter

• use colors to show further meaning


Infovis state-of-the-art in 1998• 800 nodes was a huge graph

• We had 100,000 nodes

• Use spring-force simulation with lots of empirical tweaks

• Each layout needed 20 hours of Pentium time

75 slides

Visualization of the layout algorithm

Laying out the Internet graph

75 slides

Visualization of the layout algorithmLaying out an intranet


A simplified map• Minimum distance spanning tree uses 80%

of the data

• Much easier visualization

• Most of the links still valid

• Redundancy is in the middle


Colored byAS number


Map Coloring• distance from test host

• IP address– shows communities

• Geographical (by TLD)

• ISPs

• future– timing, firewalls, LSRR blocks


Colored by IP address!


Colored by geography


Colored by ISP


Colored by distancefrom scanning host


US militaryreached by ICMP ping


US military networksreached by UDP


History of the Project• Started in August 1998 at Bell Labs

• April-June 1999: Yugoslavia mapping

• July 2000: first customer intranet scanned

• Sept. 2000: spun off Lumeta from Lucent/Bell Labs

75 slides

YugoslaviaAn unclassified peek at a new

battlefield

75 slides

Intranets: the rest of the Internet


The Pretty GoodWall of China


This wasSupposedTo be aVPN

75 slides

Anything large enough to be called

an “intranet” isout of control


Case studies: corp. networksSome intranet statistics

Min MaxIntranet sizes (devices) 7,900 365,000Corporate address space 81,000 745,000,000% devices in unknown address space 0.01% 20.86%

% routers responding to "public" 0.14% 75.50%% routers responding to other 0.00% 52.00%

Outbound host leaks on network 0 176,000% devices with outbound ICMP leaks 0% 79%% devices with outbound UDP leaks 0% 82%

Inbound UDP host leaks 0 5,800% devices with inbound ICMP leaks 0% 11%% devices with inbound UDP leaks 0% 12%% hosts running Windows 36% 84%

75 slides

Leak DetectionLumeta’s “special sauce”


Leak Detection

Internet intranet

Mapping hostA

Test hostB

mittD

C

• A sends packet to B, with spoofed return address of D

• If B can, it will reply to D with a response, possibly through a different interface


Outbound Leak Detection

Internet intranet

Mapping hostA

Test hostB

mittD

C

• Packet must be crafted so the response won’t be permitted through the firewall

• A variety of packet types and responses are used

• Either inside or outside address may be discovered

• Packet is labeled so we know where it came from


Inbound Leak Detection

Internet intranet

Mapping hostA

Test hostB

mittD

C

• This direction is usually more important

• It all depends on the site policy…

• …so many leaks might be just fine.


Inbound Leak Detection

Internet intranet

Mapping hostA

Test hostB

mittD

C


Some Lumeta lessons• Reporting is the really hard part

– Converting data to information• “Tell me how we compare to other clients”• Offering a service was good practice, for a

while• The clients want a device• We have >70 Fortune-200 companies and

government agencies as clients• Need-to-have vs. want-to-have


Honeyd – network emulation• Anti-hacking tools by Niels Provos at

citi.umich.edu

• Can respond as one or more hosts

• I am configuring it to look like an entire client’s network

• Useful for testing and debugging

• Product?

75 slides

Open questions and future work


How do you analyze a large graph over time?

• Five years of Internet data, mostly unanalyzed

• Alternate paths to a target country

• Sample insight: “Poland was off the Internet yesterday”

• Placement of monitoring tools?

• Compute a display differences between two complex graphs


Visualizations• These graphs are too big for a piece of paper

• Various approaches available, but none really satisfactory

• Build visualization graph as the data comes in, and as the network evolves

75 slides

Mapping the Internet and

IntranetsBill Cheswick

[email protected]

http://www.cheswick.com

mailto:[email protected]

mapping the internet and intranets

Documents