hiseasnet data layer
DESCRIPTION
HiSeasNet Data Layer. What happens after the modems are talking through the satellite. Overview. Big picture The subnet collection Tunnels (GRE and IPsec ) Cisco router use in HiSeasNet ( not an IOS primer) Troubleshooting Advanced topics. The Big Picture. Legend: - PowerPoint PPT PresentationTRANSCRIPT
HiSeasNet Data LayerHiSeasNet Data Layer
What happens after the modems are talking through
the satellite
What happens after the modems are talking through
the satellite
1SIO Aug '09
OverviewOverview
Big picture The subnet collection Tunnels (GRE and IPsec) Cisco router use in HiSeasNet
(not an IOS primer) Troubleshooting Advanced topics
Big picture The subnet collection Tunnels (GRE and IPsec) Cisco router use in HiSeasNet
(not an IOS primer) Troubleshooting Advanced topics
2SIO Aug '09
Legend:
Purple is Satellite RF
Teal is Synchronous Serial
Orange is “foreign” IP space
Blue is local IP space
The Big Picture
3SIO Aug '09
Internet Protocol Fundamentals
Internet Protocol Fundamentals
Overview/review of concepts and how they apply to
HiSeasNet
Overview/review of concepts and how they apply to
HiSeasNet
4SIO Aug '09
Fundamental conceptsFundamental concepts The Internet Protocol (IP) is a set of rules that are followed
when computers talk on the Internet, regardless of content. IP is a layer of networking above modems where packets
are relayed from one host to another until they get to their destination.
The IP layer is in “Layer 3” from the OSI model The act of accepting a packet, looking at the header, and
sending it closer to its destination is called routing and is performed by devices called routers.
Routers are like people: Have many interfaces that handle information Must think a little to determine where that info goes
Switches do brainless transactions on lower level packets (Layer 2), not routing.
The Internet Protocol (IP) is a set of rules that are followed when computers talk on the Internet, regardless of content.
IP is a layer of networking above modems where packets are relayed from one host to another until they get to their destination.
The IP layer is in “Layer 3” from the OSI model The act of accepting a packet, looking at the header, and
sending it closer to its destination is called routing and is performed by devices called routers.
Routers are like people: Have many interfaces that handle information Must think a little to determine where that info goes
Switches do brainless transactions on lower level packets (Layer 2), not routing.
5SIO Aug '09
Typical network layoutTypical network layout
A packet must always go from ship to earth station to institution’s campus, then maybe Internet. All are physical places/interfaces, so…
At least 3 hops between subnets for a packet to take to get to the Internet on at least 3 routers (not switches, so we have to route in layer 3)
3 IP subnets with routes in between those hops HiSeasNet subnet on the ship (“DMZ”) Ship/shore point-to-point (“Sat P2P”) Earth station to home institution P2P (“Tunnel P2P”)
Not just a block of address assignments, but full-fledged, subnetted IP blocks that everyone in the subnet can agree on
A packet must always go from ship to earth station to institution’s campus, then maybe Internet. All are physical places/interfaces, so…
At least 3 hops between subnets for a packet to take to get to the Internet on at least 3 routers (not switches, so we have to route in layer 3)
3 IP subnets with routes in between those hops HiSeasNet subnet on the ship (“DMZ”) Ship/shore point-to-point (“Sat P2P”) Earth station to home institution P2P (“Tunnel P2P”)
Not just a block of address assignments, but full-fledged, subnetted IP blocks that everyone in the subnet can agree on
6SIO Aug '09
IP subnet reviewIP subnet review IP Networks are like…Castles? Hamster habitats?
Multi-track conferences? Carrier pigeons? All IP networks are broken into “subnets” that
define what addresses are considered local (who the neighbors are that can be reached with just a broadcast packet) to an address
Subnets have a “network” part and a “host” part. The network part is defined by the subnet mask,
notated by the number of bits (ie “/24”) from left to right that indicate the network (bits must be 1s).
The remaining bits are the host part or decimal value of octets (the 0s on the right).
For example: Binary: 11111111 11111111 11111111 00000000 Decimal: 255 255 255 0
IP Networks are like…Castles? Hamster habitats? Multi-track conferences? Carrier pigeons?
All IP networks are broken into “subnets” that define what addresses are considered local (who the neighbors are that can be reached with just a broadcast packet) to an address
Subnets have a “network” part and a “host” part. The network part is defined by the subnet mask,
notated by the number of bits (ie “/24”) from left to right that indicate the network (bits must be 1s).
The remaining bits are the host part or decimal value of octets (the 0s on the right).
For example: Binary: 11111111 11111111 11111111 00000000 Decimal: 255 255 255 0
7SIO Aug '09
IP subnet review cont’dIP subnet review cont’d Each subnet must have a network address (the
bottom most address in the block) and a broadcast address (the top most address in the block).
Remaining addresses in the block can be used for the hosts are in the network. If those hosts want to talk outside that subnet (usually the
case), there must be a router/gateway that ferries traffic to another network via another interface (serial, ethernet, etc.)
Subnets can occur only at certain places in an address range…where network addresses can be on bit divisions (depends on the size of the subnet)
RFC 1918 defines “private” subnet ranges to be 192.168.0.0/16, 172.16.0.0/12, and 10.0.0.0/8
Each subnet must have a network address (the bottom most address in the block) and a broadcast address (the top most address in the block).
Remaining addresses in the block can be used for the hosts are in the network. If those hosts want to talk outside that subnet (usually the
case), there must be a router/gateway that ferries traffic to another network via another interface (serial, ethernet, etc.)
Subnets can occur only at certain places in an address range…where network addresses can be on bit divisions (depends on the size of the subnet)
RFC 1918 defines “private” subnet ranges to be 192.168.0.0/16, 172.16.0.0/12, and 10.0.0.0/8
8SIO Aug '09
Class C IP subnet exampleClass C IP subnet example
The typical “Class C” address block Subnet mask of 255.255.255.0 (aka “/24”) 11111111 11111111 11111111 00000000 The last 8 host bits can vary giving 255 host
addresses, but 2 (network and broadcast) are used in definition of subnet
Network address could be, for example, 192.168.0.0 Broadcast address is then 192.168.0.255 Usable addresses are 192.168.0.1 through
192.168.0.254
The typical “Class C” address block Subnet mask of 255.255.255.0 (aka “/24”) 11111111 11111111 11111111 00000000 The last 8 host bits can vary giving 255 host
addresses, but 2 (network and broadcast) are used in definition of subnet
Network address could be, for example, 192.168.0.0 Broadcast address is then 192.168.0.255 Usable addresses are 192.168.0.1 through
192.168.0.254
9SIO Aug '09
HiSeasNet DMZ subnet example
HiSeasNet DMZ subnet example
The typical “DMZ” block in HiSeasNet Subnet mask of 255.255.255.240 (aka “/28”) 11111111 11111111 11111111 11110000 The last 4 host bits can vary giving 16 host addresses,
but 2 (network and broadcast addresses) are used in definition of subnet
Network address could be, for example, 172.16.1.16 Broadcast address is then 172.16.1.31 14 usable addresses are 172.16.1.17 through
172.16.1.30 One address gets used by the router/gateway (usually
the first one, 172.16.1.17 in this case), so really only 13 free addresses if you want to actually use HiSeasNet
The typical “DMZ” block in HiSeasNet Subnet mask of 255.255.255.240 (aka “/28”) 11111111 11111111 11111111 11110000 The last 4 host bits can vary giving 16 host addresses,
but 2 (network and broadcast addresses) are used in definition of subnet
Network address could be, for example, 172.16.1.16 Broadcast address is then 172.16.1.31 14 usable addresses are 172.16.1.17 through
172.16.1.30 One address gets used by the router/gateway (usually
the first one, 172.16.1.17 in this case), so really only 13 free addresses if you want to actually use HiSeasNet
10SIO Aug '09
172.16.1.x/28 network possibilities
172.16.1.x/28 network possibilities
Confused about valid network address locations?
Use a table. Generate one by hand, or cheat and use an IP calculator.
I like the one at www.subnetmask.info
11SIO Aug '09
HostsNetworkfrom to
BroadcastAddress
172.16.1.0 172.16.1.1 172.16.1.14 172.16.1.15172.16.1.16 172.16.1.17 172.16.1.30 172.16.1.31172.16.1.32 172.16.1.33 172.16.1.46 172.16.1.47172.16.1.48 172.16.1.49 172.16.1.62 172.16.1.63172.16.1.64 172.16.1.65 172.16.1.78 172.16.1.79172.16.1.80 172.16.1.81 172.16.1.94 172.16.1.95172.16.1.96 172.16.1.97 172.16.1.110 172.16.1.111172.16.1.112 172.16.1.113 172.16.1.126 172.16.1.127172.16.1.128 172.16.1.129 172.16.1.142 172.16.1.143172.16.1.144 172.16.1.145 172.16.1.158 172.16.1.159172.16.1.160 172.16.1.161 172.16.1.174 172.16.1.175172.16.1.176 172.16.1.177 172.16.1.190 172.16.1.191172.16.1.192 172.16.1.193 172.16.1.206 172.16.1.207172.16.1.208 172.16.1.209 172.16.1.222 172.16.1.223172.16.1.224 172.16.1.225 172.16.1.238 172.16.1.239172.16.1.240 172.16.1.241 172.16.1.254 172.16.1.255
172.16.1.x/29 network possibilities
172.16.1.x/29 network possibilities
HostsNetworkfrom to
BroadcastAddress
172.16.1.0 172.16.1.1 172.16.1.6 172.16.1.7172.16.1.8 172.16.1.9 172.16.1.14 172.16.1.15172.16.1.16 172.16.1.17 172.16.1.22 172.16.1.23172.16.1.24 172.16.1.25 172.16.1.30 172.16.1.31172.16.1.32 172.16.1.33 172.16.1.38 172.16.1.39172.16.1.40 172.16.1.41 172.16.1.46 172.16.1.47172.16.1.48 172.16.1.49 172.16.1.54 172.16.1.55172.16.1.56 172.16.1.57 172.16.1.62 172.16.1.63172.16.1.64 172.16.1.65 172.16.1.70 172.16.1.71172.16.1.72 172.16.1.73 172.16.1.78 172.16.1.79172.16.1.80 172.16.1.81 172.16.1.86 172.16.1.87172.16.1.88 172.16.1.89 172.16.1.94 172.16.1.95172.16.1.96 172.16.1.97 172.16.1.102 172.16.1.103172.16.1.104 172.16.1.105 172.16.1.110 172.16.1.111172.16.1.112 172.16.1.113 172.16.1.118 172.16.1.119172.16.1.120 172.16.1.121 172.16.1.126 172.16.1.127172.16.1.128 172.16.1.129 172.16.1.134 172.16.1.135172.16.1.136 172.16.1.137 172.16.1.142 172.16.1.143172.16.1.144 172.16.1.145 172.16.1.150 172.16.1.151172.16.1.152 172.16.1.153 172.16.1.158 172.16.1.159172.16.1.160 172.16.1.161 172.16.1.166 172.16.1.167172.16.1.168 172.16.1.169 172.16.1.174 172.16.1.175172.16.1.176 172.16.1.177 172.16.1.182 172.16.1.183172.16.1.184 172.16.1.185 172.16.1.190 172.16.1.191172.16.1.192 172.16.1.193 172.16.1.198 172.16.1.199172.16.1.200 172.16.1.201 172.16.1.206 172.16.1.207172.16.1.208 172.16.1.209 172.16.1.214 172.16.1.215172.16.1.216 172.16.1.217 172.16.1.222 172.16.1.223172.16.1.224 172.16.1.225 172.16.1.230 172.16.1.231172.16.1.232 172.16.1.233 172.16.1.238 172.16.1.239172.16.1.240 172.16.1.241 172.16.1.246 172.16.1.247172.16.1.248 172.16.1.249 172.16.1.254 172.16.1.255
Confused about valid network address locations?
Use a table. Generate one by hand, or cheat and use an IP calculator.
I like the one at www.subnetmask.info
HiSeasNet P2P subnet example
HiSeasNet P2P subnet example
The point-to-point “transit network” Subnet mask of 255.255.255.252 (aka “/30”) 11111111 11111111 11111111 11111100 The last 2 host bits can vary giving 4 host addresses, but 2
(network and broadcast addresses) are used in definition of subnet Network address could be, for example, 192.168.68.4 Broadcast address is then 192.168.68.7 Usable addresses are 192.168.68.5 and 192.168.68.6
Allows us to route between two routers. They each get an address in the transit network and have a very small place where just they can talk to each other. Only room for two in this network. Each is the other’s gateway.
Used between ship and earth station Used between earth station and home institution (wrapped in
a tunnel)
The point-to-point “transit network” Subnet mask of 255.255.255.252 (aka “/30”) 11111111 11111111 11111111 11111100 The last 2 host bits can vary giving 4 host addresses, but 2
(network and broadcast addresses) are used in definition of subnet Network address could be, for example, 192.168.68.4 Broadcast address is then 192.168.68.7 Usable addresses are 192.168.68.5 and 192.168.68.6
Allows us to route between two routers. They each get an address in the transit network and have a very small place where just they can talk to each other. Only room for two in this network. Each is the other’s gateway.
Used between ship and earth station Used between earth station and home institution (wrapped in
a tunnel)
13SIO Aug '09
HiSeasNet subnet collection
HiSeasNet subnet collection
DMZ (subnet before firewall) is usually a /29 (5 usable addresses + router/gateway) or, better yet, a /28 subnet (13 usable addresses + router/gateway)
Sat P2P (between ship and earth station) is a /30 subnet
Tunnel P2P (between earth station and home institution) is a /30 subnet. Physical end points are internet hosts, logical addresses in tunnel are /30.
All three of these subnets come from home institution address space be it public or private. This is the key to making the ship appear as though it is part of the home institution network!
DMZ (subnet before firewall) is usually a /29 (5 usable addresses + router/gateway) or, better yet, a /28 subnet (13 usable addresses + router/gateway)
Sat P2P (between ship and earth station) is a /30 subnet
Tunnel P2P (between earth station and home institution) is a /30 subnet. Physical end points are internet hosts, logical addresses in tunnel are /30.
All three of these subnets come from home institution address space be it public or private. This is the key to making the ship appear as though it is part of the home institution network!
14SIO Aug '09
IP TunnelsIP Tunnels
Overview of GRE and IPsec and how those protocols
apply to HiSeasNet
Overview of GRE and IPsec and how those protocols
apply to HiSeasNet
15SIO Aug '09
Logical Oceanus networking
Logical Oceanus networking
SIO Aug '09 16
Ship Net128.128.252.33
Internet
Campus Net128.128.x.x
RandomInternet Host
Actual Oceanus networking
Actual Oceanus networking
SIO Aug '09 17
Ship Net128.128.252.33
Internet
Campus Net128.128.x.x
RandomInternet Host
UCSDNetwork
What is an IP tunnel?What is an IP tunnel? Analogy: Campus mail system
You send a internal memo to a colleague on another campus, addressed by building and/or mail code.
That message is bundled with other mail headed to that same campus destination and may travel in a USPS package.
When it arrives, your colleague gets the message without the intermediate USPS address on it and it appears to have simply come through your internal mail system.
Another analogy: Container shipment May travel any way the forwarder wants, but your toolbox
is still in the container when it gets to you in a foreign port.
A way of designating a collection of packets as part of a higher level virtual link (think wormhole)
Done through additional identifying headers on packets. Only endpoints of tunnels look inside packets.
The effect might be to: Shorten a long path logically Use foreign IP space in a network Encrypt a link Use other protocols (ie IPX) through IP
Examples include GRE, IPsec, L2TP, SSL, PPTP
Analogy: Campus mail system You send a internal memo to a colleague on another
campus, addressed by building and/or mail code. That message is bundled with other mail headed to that
same campus destination and may travel in a USPS package.
When it arrives, your colleague gets the message without the intermediate USPS address on it and it appears to have simply come through your internal mail system.
Another analogy: Container shipment May travel any way the forwarder wants, but your toolbox
is still in the container when it gets to you in a foreign port.
A way of designating a collection of packets as part of a higher level virtual link (think wormhole)
Done through additional identifying headers on packets. Only endpoints of tunnels look inside packets.
The effect might be to: Shorten a long path logically Use foreign IP space in a network Encrypt a link Use other protocols (ie IPX) through IP
Examples include GRE, IPsec, L2TP, SSL, PPTP 18SIO Aug '09
GRE tunnelsGRE tunnels Generic Routing Encapsulation (GRE) standard defined in RFC
1701 Not TCP or UDP, but GRE…it rides on IP directly Very simple, efficient, easy to configure, well supported in
most firewalls and routers Has no authentication or encryption by default…just for
packaging strange packets in normal links Configured as a simple tunnel interface on a Cisco 24 byte header is added, so the new payload in a standard
1500 byte packet is now 1476 bytes. Data coming into tunnel is 24 bytes too big and needs to be
repackaged into smaller packets on shore. Sometimes repackaging is done on the ship. Either way, it is less efficient, adds to router CPU load, but works fine
for slow links.
Generic Routing Encapsulation (GRE) standard defined in RFC 1701
Not TCP or UDP, but GRE…it rides on IP directly Very simple, efficient, easy to configure, well supported in
most firewalls and routers Has no authentication or encryption by default…just for
packaging strange packets in normal links Configured as a simple tunnel interface on a Cisco 24 byte header is added, so the new payload in a standard
1500 byte packet is now 1476 bytes. Data coming into tunnel is 24 bytes too big and needs to be
repackaged into smaller packets on shore. Sometimes repackaging is done on the ship. Either way, it is less efficient, adds to router CPU load, but works fine
for slow links.
19SIO Aug '09
GRE tunnel packetsGRE tunnel packets
IP Header
To: campus front doorFrom: ship front door
20 bytes
GRE Header
Protocol: IP4 bytes
IP Header
To: Campus internal mail serverFrom: Ship internal mail server
Payload
(web page or something)
•Normal Internet packet, as sent by computer on ship and seen inside campus•Addresses may be private
•Added by Earth Station•Travels through Internet•Removed by Campus
The GRE header just indicates that it is carrying another packet of some sort that should be pulled out
20SIO Aug '09
IPsec tunnelsIPsec tunnels “IP Security” standard defined in RFC 2401 Can act like a tunnel, but not necessarily. They are still
valid IP packets that contain a complete IP packet in their payload.
Look like any other IP packet to routers Implemented in a host networking stack or in a router or
switch Work may be offloaded to a hardware accelerator Encrypts and authenticates between ends Bridges a network, so no need for tunnel subnet Trickier to configure on end points (filter-based) Very standard and supported by many routers/firewalls
now
“IP Security” standard defined in RFC 2401 Can act like a tunnel, but not necessarily. They are still
valid IP packets that contain a complete IP packet in their payload.
Look like any other IP packet to routers Implemented in a host networking stack or in a router or
switch Work may be offloaded to a hardware accelerator Encrypts and authenticates between ends Bridges a network, so no need for tunnel subnet Trickier to configure on end points (filter-based) Very standard and supported by many routers/firewalls
now
21SIO Aug '09
Tunnels in HiSeasNetTunnels in HiSeasNet Primarily used for shortening routes between
earth station and institution and carrying that institutions IP addresses to the ship.
We prefer to use GRE tunnels for efficiency and ease of setup. They also separate traffic a little better with clear endpoints and no bridging.
Some institutions want security over the shore links (may be internal networks they are passing to the ship or no GRE support), so we have run IPsec links for them.
We can run IPsec over GRE, but we don’t in HiSeasNet anywhere
Primarily used for shortening routes between earth station and institution and carrying that institutions IP addresses to the ship.
We prefer to use GRE tunnels for efficiency and ease of setup. They also separate traffic a little better with clear endpoints and no bridging.
Some institutions want security over the shore links (may be internal networks they are passing to the ship or no GRE support), so we have run IPsec links for them.
We can run IPsec over GRE, but we don’t in HiSeasNet anywhere
22SIO Aug '09
Why bother with tunnels?Why bother with tunnels?
So why even bother with tunnels and subnets when a single IP address could be offered to a ship and set up NAT like a cable/DSL modem? Not all ships want to NAT (especially bigger ones) Would require a VPN on the ship to see private
networks at the home institution Doesn’t allow incoming connections very easily UCSD networking will not offer permanent IP space
to folks that don’t sign up for UCSD networking policies
So why even bother with tunnels and subnets when a single IP address could be offered to a ship and set up NAT like a cable/DSL modem? Not all ships want to NAT (especially bigger ones) Would require a VPN on the ship to see private
networks at the home institution Doesn’t allow incoming connections very easily UCSD networking will not offer permanent IP space
to folks that don’t sign up for UCSD networking policies
23SIO Aug '09
Cisco RoutersCisco Routers
Not an IOS review, but a larger picture of how Ciscos
are used in HiSeasNet
Not an IOS review, but a larger picture of how Ciscos
are used in HiSeasNet
24SIO Aug '09
Why use Cisco routers?Why use Cisco routers? There is a lot going on at the IP layer
IP packets on the ships need to come off via synchronous serial satellite modems
IP packets at the earth station need to go to the home institution through the internet
We need IP routing to be done in a simple, efficient, reliable, hands-off way.
Cisco Systems, Inc. makes routers that support synchronous serial interfaces that work with Comtech modems. These routers can reliably handle all the strange shore routing that HiSeasNet does with minimal interaction.
Doesn’t have to be a Cisco box, but it is a known, solid solution that we have experience with.
There is a lot going on at the IP layer IP packets on the ships need to come off via synchronous
serial satellite modems IP packets at the earth station need to go to the home
institution through the internet We need IP routing to be done in a simple, efficient,
reliable, hands-off way. Cisco Systems, Inc. makes routers that support
synchronous serial interfaces that work with Comtech modems. These routers can reliably handle all the strange shore routing that HiSeasNet does with minimal interaction.
Doesn’t have to be a Cisco box, but it is a known, solid solution that we have experience with.
25SIO Aug '09
Cisco gear on shoreCisco gear on shore Ship and earth station require synchronous serial
interfaces (WIC-1T or WIC-2T module board) and accompanying EIA-530 cable (CAB-530MT or CAB-SS-530MT depending on the serial board)
Earth station uses a pair of 2821 routers to handle more serial connections and more routing capacity. Should handle at least 5Mbit okay, but may need to upgrade for larger data rate events.
Earth station routers split for Ku-band vs. C-band Home institutions terminate GRE or IPsec links on
campus Cisco routers, switches, PIX firewalls, or even Netscreen firewalls…anything that supports tunnel protocol and is in the right network place on the campus network.
Ship and earth station require synchronous serial interfaces (WIC-1T or WIC-2T module board) and accompanying EIA-530 cable (CAB-530MT or CAB-SS-530MT depending on the serial board)
Earth station uses a pair of 2821 routers to handle more serial connections and more routing capacity. Should handle at least 5Mbit okay, but may need to upgrade for larger data rate events.
Earth station routers split for Ku-band vs. C-band Home institutions terminate GRE or IPsec links on
campus Cisco routers, switches, PIX firewalls, or even Netscreen firewalls…anything that supports tunnel protocol and is in the right network place on the campus network.
26SIO Aug '09
Cisco out at seaCisco out at sea Doesn’t take much at the ship…usually a fairly low end
router (2600 series or newer 2800 series…usually a 2811) with a switch attached for the DMZ.
Basic ship Cisco config is a serial-to-ethernet box with minor routing and BGP announcement, plus a few other options on some ships.
Some ships handle more in their router configs (firewalling, failover interfaces for shore connections, links to SWAP, etc.). May require additional hardware modules.
Do you need a backup router? Possibly. Had a small Cisco die in the first month of HiSeasNet.
No one is terminating a HiSeasNet link at a Linux box, but it is theoretically possible.
Doesn’t take much at the ship…usually a fairly low end router (2600 series or newer 2800 series…usually a 2811) with a switch attached for the DMZ.
Basic ship Cisco config is a serial-to-ethernet box with minor routing and BGP announcement, plus a few other options on some ships.
Some ships handle more in their router configs (firewalling, failover interfaces for shore connections, links to SWAP, etc.). May require additional hardware modules.
Do you need a backup router? Possibly. Had a small Cisco die in the first month of HiSeasNet.
No one is terminating a HiSeasNet link at a Linux box, but it is theoretically possible.
27SIO Aug '09
Cisco configurationCisco configuration Routers keep a configuration file in NVRAM. The initial loading of this file can be tricky The configuration can be modified on the router, then
saved to the startup file Routers have user accounts and one or two layers of
passwords Routers are usually named “rv-shipname-gw” Generally a configuration gets set and is left alone
Any fiddling you want to do is at your own risk Firmware should be updated from time to time when
security problems are announced. HiSeasNet does not buy or maintain routers on ships, but
we will help spec or configure where we can. Maintenance and upgrades are up to the ship.
Routers keep a configuration file in NVRAM. The initial loading of this file can be tricky The configuration can be modified on the router, then
saved to the startup file Routers have user accounts and one or two layers of
passwords Routers are usually named “rv-shipname-gw” Generally a configuration gets set and is left alone
Any fiddling you want to do is at your own risk Firmware should be updated from time to time when
security problems are announced. HiSeasNet does not buy or maintain routers on ships, but
we will help spec or configure where we can. Maintenance and upgrades are up to the ship.
28SIO Aug '09
Basic Ship Config
Basic Ship Confighostname rv-endeavor-gw
!boot-start-markerboot system flash:c2800nm-entservicesk9-mz.123-14.T7.binboot system flash:c2800nm-entservicesk9-mz.123-8.T9.binboot-end-marker!enable secret 5 $1$/nBM$yIEQ7v/blahblahblah!no ip dhcp use vrf connectedip dhcp excluded-address 131.128.217.225 131.128.217.239!ip dhcp pool dmz network 131.128.217.224 255.255.255.224 default-router 131.128.217.225 !interface FastEthernet0/0 description Ships Network ip address 131.128.217.225 255.255.255.224 no ip mroute-cache duplex auto speed auto no cdp enable
interface Serial0/0/0 description "Serial line to satellite modem" ip address 131.128.19.225 255.255.255.252 ip access-group 131 in no ip redirects no ip unreachables no ip proxy-arp ip accounting output-packets ip mtu 1400 no ip mroute-cache no keepalive fair-queue no cdp enable!router bgp 64521 no synchronization bgp log-neighbor-changes network 131.128.0.0 redistribute connected neighbor 131.128.19.226 remote-as 64521 neighbor 131.128.19.226 description ucsd-sdsc-roadnet-2611 neighbor 131.128.19.226 next-hop-self neighbor 131.128.19.226 weight 30000 neighbor 131.128.19.226 distribute-list 28 out no auto-summary!ip route 0.0.0.0 0.0.0.0 Serial0/0/0 205access-list 28 permit 131.128.217.224 0.0.0.31access-list 131 permit ip any 131.128.217.224 0.0.0.31access-list 131 permit ip any 131.128.19.224 0.0.0.3access-list 131 deny ip any any log
29SIO Aug '09
More Cisco ResourcesMore Cisco Resources Internetwork Operating System (IOS) is the
operating system Cisco routers run. It has a steep learning curve and lots of ways to get into trouble.
If you feel the need to fiddle with your router: You are on your own. Get a good reference
Cisco IOS Cookbook (ISBN: 0596527225) Cisco: The Complete Reference (ISBN: 0072192801) Anything else that seems fit to your interest/skill level
Really handy to know how to ping from router It can be handy to know how to update firmware…
but you will probably mess it up the first time or two.
Internetwork Operating System (IOS) is the operating system Cisco routers run. It has a steep learning curve and lots of ways to get into trouble.
If you feel the need to fiddle with your router: You are on your own. Get a good reference
Cisco IOS Cookbook (ISBN: 0596527225) Cisco: The Complete Reference (ISBN: 0072192801) Anything else that seems fit to your interest/skill level
Really handy to know how to ping from router It can be handy to know how to update firmware…
but you will probably mess it up the first time or two.
30SIO Aug '09
TroubleshootingTroubleshooting
Ways to go about troubleshooting the data layer
of HiSeasNet
Ways to go about troubleshooting the data layer
of HiSeasNet
31SIO Aug '09
Problem solved!Problem solved! The vast majority of routing problems are on shore
If you think you found a routing problem: Contact HiSeasNet tech team and ask us to look into our
routing on shore. We probably know about it already, but sometimes our
routers spontaneously hang or otherwise go quiet. Sometimes the problem is an outage at the home
institution somewhere between the internet and the tunnel to the earth station (often both)
Those problems on the ship may be: Problems getting to the HiSeasNet router Due to a router change after installation (accidental,
intentional mistake, unplugged cable, reboot, etc.) Cabling – unplugged ports are turned off by the router!
The vast majority of routing problems are on shore If you think you found a routing problem:
Contact HiSeasNet tech team and ask us to look into our routing on shore.
We probably know about it already, but sometimes our routers spontaneously hang or otherwise go quiet.
Sometimes the problem is an outage at the home institution somewhere between the internet and the tunnel to the earth station (often both)
Those problems on the ship may be: Problems getting to the HiSeasNet router Due to a router change after installation (accidental,
intentional mistake, unplugged cable, reboot, etc.) Cabling – unplugged ports are turned off by the router!
32SIO Aug '09
Troubleshooting overviewTroubleshooting overview The data link is only in question if:
The antenna is successfully tracking the bird AND The modem is locked up happily and is transmitting and
receiving AND The router is powered up and all cable connections are correct
(must have a link light on the ethernet port!) …but no packets are going across the link Important to note that packets must go in both
directions for the data path to be established Tools to use are “ping”, “traceroute”, sometimes
“telnet”, and a packet sniffer if you know how Clueless? Follow the packet path on the next slide, and
tell us what you find (substitute your own addresses, though)
The data link is only in question if: The antenna is successfully tracking the bird AND The modem is locked up happily and is transmitting and
receiving AND The router is powered up and all cable connections are correct
(must have a link light on the ethernet port!) …but no packets are going across the link Important to note that packets must go in both
directions for the data path to be established Tools to use are “ping”, “traceroute”, sometimes
“telnet”, and a packet sniffer if you know how Clueless? Follow the packet path on the next slide, and
tell us what you find (substitute your own addresses, though)
33SIO Aug '09
Warning!Warning!
Not all ships have the same router setup. HiSeasNet
flexibility causes some things to be different for how institutions connect in.
Know how you connect for best troubleshooting results.
Not all ships have the same router setup. HiSeasNet
flexibility causes some things to be different for how institutions connect in.
Know how you connect for best troubleshooting results. 34SIO Aug '09
Oceanus exampleOceanus example
35SIO Aug '09
Ship DMZ.32 Net.33 Gateway.34-.46 Hosts.47 Broadcast
Sat P2P.212 Net.213 Ship.214 Shore.215 Broadcast
Tunnel P2P.216 Net.217 WHOI.218 SIO.219 Broadcast
Example packet path from the ship
Example packet path from the ship
See diagram on the previous slide Packet leaving workstation in Oceanus’s DMZ
Workstation in DMZ [/28] (128.128.252.34) Into DMZ side of router (128.128.252.33) Out serial side of ship router in Sat P2P [/30]
(128.128.252.213) In shore side of Sat P2P (128.128.252.214) Out earth station side of tunnel [/30] (128.128.252.218)
Ignore Internet hops from 137.110.255.81 to 128.128.252.194 here…its all in a tunnel.
In institution side of tunnel (128.128.252.217) Inside WHOI network…can stay here, go to Internet, or be
ignored. If packets go to Internet, they must return through
reverse path (ie via WHOI)
See diagram on the previous slide Packet leaving workstation in Oceanus’s DMZ
Workstation in DMZ [/28] (128.128.252.34) Into DMZ side of router (128.128.252.33) Out serial side of ship router in Sat P2P [/30]
(128.128.252.213) In shore side of Sat P2P (128.128.252.214) Out earth station side of tunnel [/30] (128.128.252.218)
Ignore Internet hops from 137.110.255.81 to 128.128.252.194 here…its all in a tunnel.
In institution side of tunnel (128.128.252.217) Inside WHOI network…can stay here, go to Internet, or be
ignored. If packets go to Internet, they must return through
reverse path (ie via WHOI)
36SIO Aug '09
PingPing The DMZ side (“inside”) of the router on the ship
should respond to pings from the Internet if the network is public. That is the end-to-end test.
Pings should be about 800ms round-trip on a clean link. When the link is congested, it could be many thousands of ms.
Since congested links give lots of delay, use a ping command that handles a large delay. Windows ping command needs a -w (?) option, Cisco needs a “timeout” option.
If you ping from the router, be sure to set your source correctly if you have many interfaces.
Far ends of tunnels don’t ping so well.
The DMZ side (“inside”) of the router on the ship should respond to pings from the Internet if the network is public. That is the end-to-end test.
Pings should be about 800ms round-trip on a clean link. When the link is congested, it could be many thousands of ms.
Since congested links give lots of delay, use a ping command that handles a large delay. Windows ping command needs a -w (?) option, Cisco needs a “timeout” option.
If you ping from the router, be sure to set your source correctly if you have many interfaces.
Far ends of tunnels don’t ping so well.
37SIO Aug '09
Ping testingPing testing If end-to-end ping doesn’t work, try a closer
destination, possibly from the router (if you are ok with IOS): “ping <dest> timeout 10 source serial0/0/0” <dest> can be the other address in the Sat P2P
network Dest can be other hosts, possibly along the way,
possibly on the Internet. Looks like:
If end-to-end ping doesn’t work, try a closer destination, possibly from the router (if you are ok with IOS): “ping <dest> timeout 10 source serial0/0/0” <dest> can be the other address in the Sat P2P
network Dest can be other hosts, possibly along the way,
possibly on the Internet. Looks like:
ucsd-sdsc-roadnet-gw#ping knorr timeout 4 source serial0/0/0 Type escape sequence to abort.Sending 5, 100-byte ICMP Echos to 128.128.252.209, timeout is 4 seconds:Packet sent with a source address of 137.110.255.93 !!!!!Success rate is 100 percent (5/5), round-trip min/avg/max = 620/620/624 ms
foley@epicenter 4> ping -s 128.128.252.17PING 128.128.252.17: 56 data bytes64 bytes from hsnkr.whoi.edu (128.128.252.17): icmp_seq=0. time=871. ms64 bytes from hsnkr.whoi.edu (128.128.252.17): icmp_seq=1. time=795. ms64 bytes from hsnkr.whoi.edu (128.128.252.17): icmp_seq=2. time=795. ms^C
Cisco ->
Unix ->
38SIO Aug '09
TracerouteTraceroute
Is supposed to show all hops between two hosts
Does not work well through NAT connections May not work due to firewalling at campus Source is important…sourcing from router’s
ethernet interface is always helpful Will take a very long time from the ship due
to satellite delay
Is supposed to show all hops between two hosts
Does not work well through NAT connections May not work due to firewalling at campus Source is important…sourcing from router’s
ethernet interface is always helpful Will take a very long time from the ship due
to satellite delay
39SIO Aug '09
Traceroute output from router
Traceroute output from router Traceroute to
generic host on UCSD campus from Pelican’s router
Looks like: 1 is shore side of Sat
P2P 2 is institution side of
tunnel 3 is inside institution 4 to15 is Internet 16 is UCSD front door 17 is UCSD campus 18 is end node
Traceroute to generic host on UCSD campus from Pelican’s router
Looks like: 1 is shore side of Sat
P2P 2 is institution side of
tunnel 3 is inside institution 4 to15 is Internet 16 is UCSD front door 17 is UCSD campus 18 is end node
rv-pelican-gw#traceroute 132.239.4.66 source 204.196.250.129
Type escape sequence to abort.Tracing the route to 132.239.4.66
1 192.168.2.6 572 msec 572 msec 576 msec 2 192.168.2.1 628 msec 628 msec 628 msec 3 162.75.221.73 632 msec 632 msec 636 msec 4 64.200.121.22 640 msec 640 msec 644 msec 5 64.200.121.21 648 msec 648 msec 648 msec 6 64.200.210.53 644 msec 648 msec 648 msec 7 64.200.210.65 648 msec 648 msec 648 msec 8 64.200.249.130 656 msec 648 msec 648 msec 9 4.68.110.13 648 msec 648 msec 648 msec 10 4.68.19.126 656 msec 660 msec 648 msec 11 4.69.136.157 656 msec 656 msec 652 msec 12 4.69.132.77 684 msec 680 msec 684 msec 13 4.69.137.18 692 msec 4.69.137.22 740 msec 4.69.137.26 692 msec 14 4.68.20.4 680 msec 4.68.20.68 680 msec 4.68.20.132 716 msec 15 4.78.194.82 680 msec 680 msec 684 msec 16 137.164.24.210 684 msec 684 msec 688 msec 17 132.239.255.145 712 msec 684 msec 684 msec 18 132.239.4.66 684 msec 684 msec 684 msec
40SIO Aug '09
Packet sniffingPacket sniffing Probably overkill for simple problems…most
HiSeasNet issues are just about connectivity Looks at what packets are flowing across an
interface Great tool for seeing what is really going on, but
may not be easy to use Often more than you want to know Not standard on all computers Check out snoop, tcpdump, WireShark (formerly
Ethereal), snort, or something else Traces can be very helpful for subtle problems (like
MTU issues, worms, link saturation, etc.)
Probably overkill for simple problems…most HiSeasNet issues are just about connectivity
Looks at what packets are flowing across an interface
Great tool for seeing what is really going on, but may not be easy to use
Often more than you want to know Not standard on all computers Check out snoop, tcpdump, WireShark (formerly
Ethereal), snort, or something else Traces can be very helpful for subtle problems (like
MTU issues, worms, link saturation, etc.)
41SIO Aug '09
Other tools that might help
Other tools that might help mtr
Like traceroute, but Seems to get through some firewalls and make more sense
http://www.bitwizard.nl/mtr/ Long-term network monitoring from
shore or ship MRTG or Cricket Smokeping Nagios Intermapper
mtr Like traceroute, but Seems to get through
some firewalls and make more sense http://www.bitwizard.nl/mtr/
Long-term network monitoring from shore or ship MRTG or Cricket Smokeping Nagios Intermapper
42SIO Aug '09
Advanced topicsAdvanced topicsA few extra things to think about after it is all working
Starts with answers, ends with questions
A few extra things to think about after it is all working
Starts with answers, ends with questions
43SIO Aug '09
Common carrier routingCommon carrier routing
We automatically share the shore-to-ship link via Cisco fair queuing
Allows for bursting when other ships are quiet All shore-to-ship routes for a satellite go
through one transmitting modem Ship routers receive all traffic for all ships in
that footprint on the serial port, but filter out just theirs
Modems on shore are largely receive-only Shore routing is a bit tricky to split traffic in on
one modem, but sent on another
We automatically share the shore-to-ship link via Cisco fair queuing
Allows for bursting when other ships are quiet All shore-to-ship routes for a satellite go
through one transmitting modem Ship routers receive all traffic for all ships in
that footprint on the serial port, but filter out just theirs
Modems on shore are largely receive-only Shore routing is a bit tricky to split traffic in on
one modem, but sent on another
44SIO Aug '09
The MTU problemThe MTU problem Maximum Transmission Unit (MTU) is the largest
amount of data that can be sent in one bundle through an interface Default for Ethernet is 1500 Smaller if tunnel overhead is added
Bigger MTU means fewer headers and more efficiency. Smaller MTU means less efficient.
All routers along the way must agree on this or be willing to adjust to meet something else
Hard to agree since many routers block agreement protocols for security reasons.
In HiSeasNet, we want smaller packets for better VoIP, but shared outroute causes problems. Every institution chokes on different values.
Maximum Transmission Unit (MTU) is the largest amount of data that can be sent in one bundle through an interface Default for Ethernet is 1500 Smaller if tunnel overhead is added
Bigger MTU means fewer headers and more efficiency. Smaller MTU means less efficient.
All routers along the way must agree on this or be willing to adjust to meet something else
Hard to agree since many routers block agreement protocols for security reasons.
In HiSeasNet, we want smaller packets for better VoIP, but shared outroute causes problems. Every institution chokes on different values.
45SIO Aug '09
Quality of Service (QoS)Quality of Service (QoS) Is it a delay problem or a jitter problem? Wouldn’t it be great to increase the priority of
certain packets? Yes: VoIP connections might be smoother, important
traffic could be faster, etc. No: It really doesn’t do much good if the packets are small
We do this for VoIP boxes, and have found that such small packets get stuck behind large ones if they are just a little too late. (If you setup VoIP and want an IP included here, let us
know and we will add it to the list) Solution: Smaller MTUs…but not so good in
HiSeasNet as discussed earlier
Is it a delay problem or a jitter problem? Wouldn’t it be great to increase the priority of
certain packets? Yes: VoIP connections might be smoother, important
traffic could be faster, etc. No: It really doesn’t do much good if the packets are small
We do this for VoIP boxes, and have found that such small packets get stuck behind large ones if they are just a little too late. (If you setup VoIP and want an IP included here, let us
know and we will add it to the list) Solution: Smaller MTUs…but not so good in
HiSeasNet as discussed earlier
46SIO Aug '09
Accelerator capabilitiesAccelerator capabilities Goes in-line between router and DMZ Creates a routing tunnel between ship
accelerator and one at the earth station Invisible to campus, Internet, and DMZ Over that tunnel, all packets can be
inspected and: Compressed if not already compressed Cached so duplicate packets don’t get sent
completely multiple times Fragmented so that all packets are broken
and small real-time data can flow better Given dedicated amounts (minimums or
maximums) of bandwidth per application (QoS)
Some also do web and DNS caching for ships that don’t already do this
Goes in-line between router and DMZ Creates a routing tunnel between ship
accelerator and one at the earth station Invisible to campus, Internet, and DMZ Over that tunnel, all packets can be
inspected and: Compressed if not already compressed Cached so duplicate packets don’t get sent
completely multiple times Fragmented so that all packets are broken
and small real-time data can flow better Given dedicated amounts (minimums or
maximums) of bandwidth per application (QoS)
Some also do web and DNS caching for ships that don’t already do this
SIO Aug '09 47
Accelerators in HiSeasNetAccelerators in HiSeasNet We have been testing on Revelle and
Atlantis in 2009. Improvements of shore-to-ship
bandwidth about 20% and 70% ship-to-shore. Depends on traffic patterns.
Funded for fleet roll out soon Will get things setup on shore, and have
boxes shipped to institutions for plug-in on ships.
Should be racking, cabling, and setting IP address…rest is configured from shore.
Work with us over time to get it tuned to your ship needs
We have been testing on Revelle and Atlantis in 2009.
Improvements of shore-to-ship bandwidth about 20% and 70% ship-to-shore. Depends on traffic patterns.
Funded for fleet roll out soon Will get things setup on shore, and have
boxes shipped to institutions for plug-in on ships.
Should be racking, cabling, and setting IP address…rest is configured from shore.
Work with us over time to get it tuned to your ship needs
SIO Aug '09 48
Services offeredServices offered
What services are you willing to offer your users? VoIP (Skype, Vonage, campus PBX, etc.), instant
messaging, video conferencing Web, FTP, mail, rsync SSH, VPNs Streaming data Software updates, license key servers, etc.
How will these services be maintained and advertised to the user?
Will they fit into your network layout?
What services are you willing to offer your users? VoIP (Skype, Vonage, campus PBX, etc.), instant
messaging, video conferencing Web, FTP, mail, rsync SSH, VPNs Streaming data Software updates, license key servers, etc.
How will these services be maintained and advertised to the user?
Will they fit into your network layout?
49SIO Aug '09
SecuritySecurity This comes in many flavors and may include:
Privacy of communications Data can be encrypted (ship-to-shore? shore-to-shore?)
Permission to use network services Are you in a public or private network? Access to resources can be restricted
Viruses and spam getting onto the ship Wasting bandwidth (denial of service)
What threats are you afraid of? What risks are you willing to take? HiSeasNet can be flexible, but define your
policy first, then look for the technical solution
This comes in many flavors and may include: Privacy of communications
Data can be encrypted (ship-to-shore? shore-to-shore?) Permission to use network services
Are you in a public or private network? Access to resources can be restricted
Viruses and spam getting onto the ship Wasting bandwidth (denial of service)
What threats are you afraid of? What risks are you willing to take? HiSeasNet can be flexible, but define your
policy first, then look for the technical solution
50SIO Aug '09
PolicyPolicy Who gets access to HiSeasNet? What priorities are there for HiSeasNet use? Where can HiSeasNet be used on a ship? What sort of guest access is available? How is bandwidth used efficiently? Can shore staff change configurations on the ship? Can the ship be contacted from shore, or should
connections only be one-way? What VoIP services are offered and to who?
Discussion about who is doing what in the fleet?
Who gets access to HiSeasNet? What priorities are there for HiSeasNet use? Where can HiSeasNet be used on a ship? What sort of guest access is available? How is bandwidth used efficiently? Can shore staff change configurations on the ship? Can the ship be contacted from shore, or should
connections only be one-way? What VoIP services are offered and to who?
Discussion about who is doing what in the fleet?
51SIO Aug '09
MonitoringMonitoring
How does one monitor a network where links are expected to go out?
How does one monitor a network with status that is not under his control or knowledge?
How does one monitor through firewalls into private IP space?
Links go down, stations go offline, and sirens do not go off at the earth station. It is hard to tell what is a problem and what is normal behavior.
How does one monitor a network where links are expected to go out?
How does one monitor a network with status that is not under his control or knowledge?
How does one monitor through firewalls into private IP space?
Links go down, stations go offline, and sirens do not go off at the earth station. It is hard to tell what is a problem and what is normal behavior.
53SIO Aug '09