cisco net workers - deploying interior gateway protocols (2007)
TRANSCRIPT
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 1
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 2
Deploying Interior Gateway Protocols
TECRST-2021
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 3
Deploying Interior Gateway Protocols
Design Theory
Working with Addressing and Summarization
Working with Hierarchy
Working with Topologies
Working with Redistribution
Transitioning Routing Protocols
BGP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 4
Design Theory
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 5
Design Theory
Design Goals
Resiliency
Simplicity
Functional Separation
Hiding Reachability
Hiding Topology
Virtualization
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 6
―… a reliable network delivers virtually every packet accepted by the network, to the right destination, within a reasonable amount of time…‖
Optimal Routing Design
Cisco Press®
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 6
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 7
Design Goals
Networks deliver packets!
A network is judged on its ability to support applications
All the other elements of network design support this single goal
The three primary goals:
Resiliency (Reliability)
Simplicity
Functional Separation
DeliverPackets
Adjust to Real World Changes
Device Failure Business Changes
High AvailabilityRedundancy Scaling
Reduced Downtime
Fast TroubleshootingFast Recovery
Simplicity
Functional Separation
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 8
Design Goals
Another view of network design is to determine why networks fail
Device failure, resolved through:
Resiliency
High availability techniques
Functional Separation
Negative feedback loops, resolved through:
Simplicity
Functional Separation
The Same Goals!
Resiliency
Simplicity
Functional Separation
NetworkFailure
High AvailabilityRedundancy
Reduced Downtime
Fast TroubleshootingFast Recovery
Simplicity
Functional Separation
Device Failure Feedback Loops
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 9
Notes on OPEX
Operational Expenses are directly tied to:
Day to day costs of running the network
The costs of downtime
Do these network design principles impact operational expenses?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 10
Notes on OPEX
Resiliency
Manages the costs of downtime
Simplicity
Manages the costs of monitoring and changing the network
Manages the costs of downtime
Functional Separation
Manages the costs of monitoring and changing the network
Manages the costs of downtime
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 11
Design Goals
Provides alternate paths to route around failures
Resiliency
Easier to grasp and troubleshoot
Simplify configurations, reducing human error
Downtime includes troubleshooting time
Simplicity
Enables simplified configurations
Allows complexity in one part of the network to be hidden from other parts of the network
Divide and conquer
FunctionalSeparation
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 12
Resiliency
Resiliency is the ability of the network to adjust to changing conditions
Two dimensions
How many packets inserted at the edge of the network do not make it to their destination?
How long is it between unplanned network failures, and how long does it take to fix the network when it‘s broken?
In general: Avoid Brittleness!
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 13
Resiliency
What Are You Planning For?
Yes NoSevere Weather with Local Power Failure?
No YesFootball Playoffs?
Yes YesBeginning of School?
Yes NoSpring Break?
The Worst Case or the Common Case?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 14
Resiliency
It‘s Important to Understand:
Mean Time Between Failures (MTBF)
How long the device or system runs before failing
Mean Time To Repair (MTTR)
How long it takes to repair the device or system after a failure
Uptime, or Reliability
How many ―9‘s‖
Total Time/(MTBF+MTTR)
Statistical Analysis
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 15
Resiliency
Break failure domains apart
A single failure impacts less of the network
Improves Troubleshooting
Troubleshooting is split and test
Splitting the failure domain presplits the troubleshooting domains
Decreases MTTR
Functional Separation
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 16
Resiliency
The simplest path to increased resiliency is adding redundancy...
Not so fast!
Resiliency must be balanced against simplicity and functional separation
Redundancy doesn‘t always add resiliency
A
10.1.1.0/24
B
Redundancy
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 17
Resiliency
There are other resilient techniques besides redundancy
High availability
Fast convergence
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 18
―Could I explain this at 2AM to a TAC Engineer who lives halfway across the world?‖
The 2AM Rule of Thumb
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 18
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 19
Simplicity
Simplicity Encompasses:
Network Design
Covered throughout the remainder of this presentation
Management Simplicity
Configuration Simplicity
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 20
Simplicity
Choose the simplest configuration that will do the job
Choose the easier configuration to change in the future
Choose the configuration that contains the intent
Examples
Use prefix lists for route filtering, rather than access lists
Use tags for filtering redistributed routes, rather than building a long list of networks
Configuration Simplicity
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 21
Simplicity
OSPF Network
Install new router...
Examine configuration of hub router
Examine configuration of existing spoke router
Configure new router
Connect to network
Network breaks!
Why?
hub_router#show run
....
interface s0/0
ip address 10.1.1.100 255.255.255.0
....
spoke_router#show run
....
interface s0/0
ip address 10.1.1.200 255.255.255.0
....
new_router#show run
....
interface s0/0
ip address 10.1.1.80 255.255.255.0
....
Configuration Simplicity
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 22
Simplicity
Why were the interface IP addresses set up this way?
The interface isn‘t a point-to-point, so it must be a multipoint
The DR must be the hub router...
What ensures this? The interface IP addresses!
This is not obvious!
A specific control is buried under a normal looking configuration
hub_router#show run
....
interface s0/0
ip address 10.1.1.100 255.255.255.0
....
spoke_router#show run
....
interface s0/0
ip address 10.1.1.200 255.255.255.0
....
new_router#show run
....
interface s0/0
ip address 10.1.1.80 255.255.255.0
....
Configuration Simplicity
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 23
Simplicity
What if we use the OSPF interface priority, instead?
The reason for the configuration directly relates to what the configuration does
This makes network maintenance simpler
Rules of thumb:
Apply the most obvious configuration possible
Apply the configuration as close to the point of control as possible
hub_router#show run
....
interface s0/0
ip address 10.1.1.100 255.255.255.0
ip ospf priority 240
....
spoke_router#show run
....
interface s0/0
ip address 10.1.1.200 255.255.255.0
ip ospf priority 0
....
new_router#show run
....
interface s0/0
ip address 10.1.1.80 255.255.255.0
ip ospf priority 0
....
Configuration Simplicity
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 24
Functional Separation
Allows us to hide information
Allows us to break the network into multiple failure domains
The amount of separation between the failure domains depends on the the strength of the separation
Watch out for fate sharing (should cover this later in the presentation)
Two Types:
Hierarchy
Virtualization
Can be mixed/blended
Many grey areas between these
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 25
Functional Separation
Going back to our design goals
Redundancy (Resiliency)
Breaking the network up into smaller pieces allows us to design, understand, and troubleshoot smaller pieces
This adds to the resiliency of the network
Simplicity
Breaking the network up into smaller pieces allows us to break a single large problem into a number of smaller, simpler problems
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 26
Functional Separation
What Do We Gain by Hiding Information?
Improved Stability
Improved Convergence
A tradeoff
Some types of information hiding cost more, in processing time, than the cost of computing across the information in the first place
Essentially, try to hide the right amounts of information in the right places...
Apparent Simplicity
A tradeoff
Sometimes, the cost of overall complexity is higher than the offsets in increased simplicity in one specific area or topology
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 27
Functional Separation
Topological
Divide the network along topological ―choke points‖
Aggregate reachability information
Aggregate topology information
Aggregate traffic flows
Distribution
Access
Core
Aggregation
Two Directions
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 28
Functional Separation
Logical
Divide the network into multiple topologies
Divide topology information between topologies
Leak minimal information between topologies
The most common implementation
Split ―outside routes‖ from ―next hop routes‖
Advertise in two different routing protocols, an EGP and an IGP
Two Directions
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 29
Hiding Reachability
IP addressing is built around the concept of summarizing reachability information
A doesn‘t advertise each of the host addresses attached to its interface, but rather a range of addresses, or a network address
19
2.1
68
.1.0
/29
.1.2
.3
.5
.4
.6
A
192.168.1.1192.168.1.2192.168.1.3192.168.1.4192.168.1.5192.168.1.6
192.168.1.0/29
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 30
Hiding Reachability
In the same way, summarizing multiple networks into one advertisement just increases the scope of reachable hosts
192.168.1.0/29 and 192.168.1.8/29 can be aggregated (summarized) to one advertisement, 192.168.1.0/28
To routers and devices beyond the summarization point, all the hosts from 192.168.1.0 through 192.168.1.15 are reachable through A 1
92
.16
8.1
.8/2
9
.1.2
.3
.5
.4
.6
19
2.1
68
.1.0
/29
.1.2
.3
.5
.4
.6
19
2.1
68
.1.0
/28
A
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 31
(192.168).00000001.00000000
(192.168).00000010.00000000
(192.168).00000011.00000000
(192.168).00000000.00000000
24
bits
22
bits
28 destinations
210 destinations
Hiding Reachability
Seen from the binary perspective, as you make the prefix length shorter, you move the network/host separation line to the left
As you move the red line to the left, you encompass more reachable destinations in the same advertisement, but you have fewer advertisements
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 32
Hiding Reachability
192.168.1.0/24, 192.168.2.0/24, and 192.168.3.0/24 can be advertised as 192.168.0.0/22
Rather than three networks, each with 255 addresses (253 hosts), A advertises a single network, with 1024 addresses
192.168.1.0/24
192.168.2.0/24
192.168.3.0/24
253 hosts
192.168.0.0/22
1 network
1024 addresses
3 networks
255 addresses each
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 33
192.168.1.0/24
192.168.2.0/24
192.168.3.0/24
A
B
C
D
192.168.0.0/22
Hiding Reachability
Address summarization also hides changes in the network
Even if the link between A and C fails, A can still advertise the 192.168.0.0/22 address space (as long as 192.168.2.0/24 isn‘t reachable via some other path)
Routers beyond A don‘t need to know about the reachability or topology change
Summary doesn‘t
change!
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 34
Hiding Reachability
One way of looking at hierarchical design is to determine the difference summarization makes statistically
If we know the rate at which prefixes change state within a network, we can predict how many state changes any given router will need to adjust to in a given time period
For instance suppose we know the average prefix will change once every month. What impact will this have on a large network?
Assessing the Impact
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 35
1000 routes
1000 routes
1000 routes
1000 routes
4000+100 routes
400+100 routes
Hiding Reachability
1000 routes each failing once/month means 4100/30 = 136.7 state changes per day in the core of this network
Summarizing each 1000 route area into 100 routes reduces the core to 500, rather than 4100, routes
Summarization hides individual route changes, so we are down to 100/30 = 3.3 state changes per day
Assessing the Impact
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 36
Hiding Topology
Topology information describes how devices are interconnected in the network
While topology information is useful, we‘d like to hide this information at some point in the network
Hiding topology information reduces the amount of data routers need to process when converging
C
A
B
10.1.1.0/24
D
A is connected to BA is connected to CB is connected to DC is connected to DD is connected to10.1.1.0/24
C is connected to 10.1.2.0/24
B is connected to10.1.2.0/24
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 37
Hiding Topology
Hiding topology information also hides information about changes in the topology
C advertises reachability to 10.1.1.0/24
If the F to G link fails, C can still reach 10.1.1.0/24 (although the metric might change)
If B can still use C to reach 10.1.1.0/24, does B need to know about the F to G link failure?
No!
C
A B
10.1.1.0/24
G
D
E F
C can reach
10.1.1.0/24, and
I‘m connected to
C!
Hide
topology
here
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 38
Virtualization
Virtualization is placing two apparently separate resources on top of a single resource
If every application stream over every IP pair over every logical subnet had its own physical path, there would be no virtualization
Virtualization is an extremely powerful tool
It allows multiple logical topologies to reside on a single underlying topology or network
Red
DWDM over fiber
Blue
100
802.1q VLANs
101
Silver
Virtual Topologies
Gold
xxx
TCP/IP Sessions
yyy
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 39
Virtualization
Virtualization always introduces fate sharing
If an underlying topology, or network, fails, all overlaying topologies fail as well
This is fate sharing
Fate sharing makes virtualization complex to design and troubleshoot
The more ―global‖ the virtualization, the more added complexity
Red
DWDM over fiber
Blue
100
802.1q VLANs
101
Silver
Virtual Topologies
Gold
xxx
TCP/IP Sessions
yyy
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 40
Virtualization
Control Plane Only
EGP (BGP) over IGP (EIGRP, OSPF, or IS-IS)
Separates control plane information into internal and external
Fairly simple to implement and deploy
Data Plane Only
L3 Tunneling (most implementations), including L3VPNs
Multiple forwarding tables with a single routing protocol database (or instance)
Moderately simple to implement and deploy
L2VPNs Multiple virtual Layer 2 networks on top of a single IP network
Multiple routing and forwarding tables
Moderately simple to implement and deploy
Virtual Networks Such as MTR
Multiple virtual topologies on a single IP infrastructure
Multiple routing and forwarding tables
Difficult to implement and deploy
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 41
Working with Addressing and Summarization
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 42
Addressing and Summarization
Address Allocation
Summary Metrics
Aggregation Issues
Aggregation Techniques
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 43
Address Allocation
A hierarchical topology isn‘t enough to hide reachability information—the way the addressing is laid out in the network is also critical
There are several possible methods you can use to assign addresses within a network
Allocating addresses as they are requested is a common method
This only creates summarization points if you happen to get address allocation requests that coincide with the topology of the network
10.1.2.0/24
10.1.1.0/24
10.1.3.0/24
Can‘t
summarize hereI asked
second!
I asked
first!
I asked
third!
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 44
Address Allocation
Assigning addresses based on the political structure of the organization is another method
10.1.x.x is marketing
10.2.x.x is sales
This only creates summarization points if the political structure of the corporation follows the logical topology of the network 10.1.1.0/24
10.1.2.0/24
10.1.3.0/24
10.2.3.0/24
10.2.2.0/24
10.2.1.0/24
Can‘t
summarize
here
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 45
10.2.1.0/24
10.2.2.0/24
10.1.2.0/24
Address Allocation
Assigning address by the geographic location of the device or network is also common
10.1.0.0/16 is Nevada
10.2.0.0/16 is California
This only creates summarization points if the topological and geographical layouts of the network coincide, which isn‘t always the case
10.1.1.0/24
Can‘t
summarize here
California
Nevada
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 46
Address Allocation
Addressing needs to follow the network topology to create summarization points
Any scheme will create summarization points as long address allocation happens to follow the network topology
But, it‘s best just to use topological addressing from the start
Creates summarization points
Allows flexibility in moving sections of a network from one place to another (moving connections to network regions)
10.1.2.0/24
10.1.1.0/2410.1.3.0/24
10.1.1.0/24
10.1.2.0/24
10.1.3.0/24
10.2.3.0/2410.2.2.0/24
10.2.1.0/24
10.2.1.0/24
10.2.2.0/24
10.1.2.0/2410.1.1.0/24
California
Nevada
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 47
Address Allocation
Start with a very large address space
Summarization always wastes address; this is a natural consequence of hiding reachablity
You could use private address space
It might be possible to gain huge summarizable address spaces by deploying IPv6 in the future
Try to balance between
Conserving address space
Providing room to grow without breaking summarization
10.1.2.0/24
10.2.2.0/24
10.3.2.0/24
10.3.1.0/24
10.2.1.0/24
10.1.1.0/24
10.1.0.0/16
10.2.0.0/16
10.3.0.0/16
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 48
Address Allocation
Several techniques can be used to conserve address space, where needed
Use /31‘s on point-to-point links to conserve address space
Avoid IP unnumbered, for management reasons—you can‘t reach the remote device if the remote link fails
Don‘t be frightened of odd length masks where it makes sense
10.1.2.0/24
10.2.2.0/24
10.3.2.0/24
10.3.1.0/24
10.2.1.0/24
10.1.1.0/24
10.1.0.0/16
10.2.0.0/16
10.3.0.0/16
/31 on point-to-point
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 49
Summary Metrics
In all interior gateway protocols, the summary metric is dependant on the metrics of the components
The metric of the highest or lowest cost component route is chosen as the summary metric
A
BC
10
.1.0
.0/2
4
Co
st 1
0
10
.1.1
.0/2
4
Co
st 2
0
10
.2.0
.0/2
4
Co
st 1
0
10
.2.1
.0/2
4
Co
st 2
0
10.1.0.0/23
Cost 20
10.2.0.0/23
Cost 20
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 50
10.1.0.0/23
Cost 20
10.2.0.0/23
Cost 20
Summary Metrics
If the component the metric was taken from flaps, the summary flaps as well!
You‘re using the summary to hide reachability information, but it‘s passing metric information through, and the routers beyond the summary are still working to keep up with the changes
A
BC
10
.1.0
.0/2
4
Co
st 1
0
10
.1.1
.0/2
4
Co
st 2
0
10
.2.0
.0/2
4
Co
st 1
0
10
.2.1
.0/2
4
Co
st 2
0
10.1.0.0/23
Cost 10
10.2.0.0/23
Cost 20
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 51
Summary Metrics
EIGRP takes its summary metric from the component route with the smallest metric
OSPF takes its summary cost from the component route with the smallest metric
If no compatible rfc1583 is configured, in which case the cost from the component with the largest cost is used
IS-IS takes its summary cost from the component route with the largest cost
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 52
Summary Metrics
Use a loopback interface to force the metric to remain constant
Create a loopback interface within the summary address range with a higher or lower metric than any other component
The summary will use the metric of the loopback, which doesn‘t ever go down
A static route to null0 on the summarizing router can also be used
You can sometimes use a route map to force the summary‘s metric to always be the same
A
B
10
.1.0
.0/2
4
Co
st 1
0
10
.1.1
.0/2
4
Co
st 2
0
10.1.0.0/23
Cost 10
loopback 0
ip address 10.1.1.1 255.255.255.255
ip ospf cost 10
10.1.0.0/23
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 53
Aggregation Issues
B and C are advertising 10.1.0.0/23 to A with a metric of 30
A has two routes to 10.1.0.0/23
B with a cost of 30
C with a cost of 40
A forwards traffic to 10.1.1.1 to B (40), although this is not the optimal route to reach 10.1.1.0/24 (30)
Summarization hides information, so the best path may not always be chosen
C
10 20
10.1.0.0/24 10.1.1.0/24
A
B
D E20
1010
10
.1.0
.0/2
3 (3
0)
10
.1.0
.0/2
3 (3
0)
Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 54
Aggregation Issues
When summarizing down the hierarchy in OSPF, we can use manual summaries instead of stub areas
Always prefer to summarize more information rather than less C
10
20
10.1.0.0/24 10.1.1.0/24
A B
D
E
20
1010
Are
a b
ord
er
10
.1.0
.0/2
3
area 1 range 10.1.0.0 255.255.254.0
10
.1.0
.0/2
31
0.1
.1.0
/24
area 1 range 10.1.0.0 255.255.254.0
area 1 range 10.1.1.0 255.255.255.0
no discard-route
Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 55
Aggregation Issues
It‘s also possible to use LSA type 3 filtering to solve this problem
Permit only a default plus some number of longer prefix routes to allow optimal routing to those destinations C
10
20
10.1.0.0/24 10.1.1.0/24
A B
D
E
20
1010
Are
a b
ord
er
0.0
.0.0
/0
ip prefix-list AREA_1_OUT seq 10 permit 0.0.0.0
!
router ospf 1000
area 1 filter-list prefix AREA_1_OUT out
0.0.0.0/0
0.0
.0.0
/01
0.1
.1.0
/24
ip prefix-list AREA_1_OUT seq 10 permit 0.0.0.0/0
ip prefix-list AREA_1_OUT seq 20 permit 10.1.1.0/24
!
router ospf 1000
area 1 filter-list prefix AREA_1_OUT out
Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 56
Aggregation Issues
IS-IS automatically summarizes down the hierarchy
You can use route leaking to leak more specific routes when optimal routing towards the core is important
C
10
20
10.1.0.0/24 10.1.1.0/24
A B
D
E
20
1010
L1
/L2
bo
rde
r
0.0
.0.0
/0
0.0
.0.0
/01
0.1
.1.0
/24
access-list 100 permit ip 10.1.1.0 0.0.0.255
!
router isis
redistribute isis ip level-2 into level-1 distribute-list 100
metric-style wide
Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 58
Aggregation Issues
EIGRP always requires either summarization or filtering to reduce routing information from the core towards the edge
There are several techniques we can use to summarize routing information towards the edge and allow more specific information to leak to prevent suboptimal routing
As with all the other protocols, you need to carefully weigh the gains in network stability and scaling against the gains from optimal routing!
C
10
20
10.1.0.0/24 10.1.1.0/24
A B
D
E
20
1010
Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 59
Aggregation Issues
Rather than summarizing, redistributed static routes paired with distribute lists can be used
C
10
20
10.1.0.0/24 10.1.1.0/24
A B
D
E
20
1010
10
.1.0
.0/2
31
0.1
.1.0
/24
ip route 10.1.0.0 255.255.254.0 null0
!
access-list 10 permit 10.1.0.0 0.0.1.255
access-list 10 permit 10.1.1.0 0.0.0.255
!
router eigrp 100
redistribute static
default-metric 1000 1 255 1 1500
distribute-list 10 out serial 0/0
10
.1.0
.0/2
3
ip route 10.1.0.0 255.255.254.0 null0
!
access-list 10 permit 10.1.0.0 0.0.1.255
!
router eigrp 100
redistribute static
default-metric 1000 1 255 1 1500
distribute-list 10 out serial 0/0
Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 60
Aggregation Issues
Another option is to create a pair of summaries containing the more and less specific routes
EIGRP also allows leaking more specifics past a summary C
10
20
10.1.0.0/24 10.1.1.0/24
A B
D
E
20
1010
0.0
.0.0
/01
0.1
.1.0
/24
interface serial 0/0
ip summary-address 10.1.1.0 255.255.255.0 250
ip summary-address 0.0.0.0 0.0.0.00
.0.0
.0/0
interface serial 0/0
ip summary-address 10.1.1.0 255.255.255.0 250
ip summary-address 0.0.0.0 0.0.0.0
Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 61
Aggregation Issues
Routers B and C are summarizing 10.1.0.0/24 and 10.1.1.0/24 into a single advertisement, 10.1.0.0/23, towards A
Routers B and C are also advertising a default route only towards each other through 10.1.0.0/24 and 10.1.1.0/24
10.1.0.0/23 10.1.0.0/23
10.1.0.0/24
10.1.1.0/24
A
B C0.0.0.0/0
Distance Vector Summary Black Holes
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 62
Aggregation Issues
If Router B loses its link to 10.1.0.0/24, what happens?
Router B isn‘t learning about 10.1.0.0/24 through C, since C is only advertising a default route—so B no longer knows how to get there
The routes advertised by B and C to A look the same before and after the failure
10.1.0.0/24
10.1.1.0/24
A
B C
10.1.0.0/23 10.1.0.0/23
0.0.0.0/0
10.1.1.0/24 isn‘t learned from A
Distance Vector Summary Black Holes
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 63
Aggregation Issues
A could still forward traffic destined to 10.1.0.1 to B
We have a summarization black hole
If A is load sharing per packet, every other packet will be dropped
If A is load sharing per session, then some hosts will be able to reach destinations on 10.1.0.0/24, and others won‘t
10.1.0.0/24
10.1.1.0/24
A
B C
10.1.0.1
10.1.0.0/23 10.1.0.0/23
Distance Vector Summary Black Holes
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 64N
o
Su
mm
ari
za
tion
Aggregation Issues
One way to solve this problem is to always have at least one unsummarized link between the summarizing routers
The summarizing routers always have someplace to send the traffic if they lose connectivity to the link
Another option is not to summarize both up the hierarchy and down the hierarchy
This reduces network scaling!
10.1.0.0/23 10.1.0.0/23
10.1.0.0/24
10.1.1.0/24
A
B C0.0.0.0/0
Don‘t summarize up and down
Distance Vector Summary Black Holes
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 65
Aggregation Issues
Routers E and F are not intended to transit traffic between C and D
Routers C and D issue summaries containing 10.1.1.0/24
Router A chooses D as its best path to the summary
The link from Router D to Router E fails
How can we prevent Router D from using the link through F to reach 10.1.1.0/24?
A
C D
B
10.1.1.0/24
E
F
10.1.2.0/24
10.1.1.0/24
10.1.0.0/16
Link State Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 66
Aggregation Issues
Place a link between C and D within the same area as E and F
The link cost between C and D should be lower than the link cost through F, causing D to route through this new link
New link
A
C D
B
10.1.1.0/24
E
F
10.1.2.0/24
10.1.1.0/24
10.1.0.0/16
Link State Summary Suboptimal Routing
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 67
Aggregation Techniques
In this network, it appears almost impossible to summarize at any point because of the addressing
Summarize anyway!
Router B can advertise 10.1.0.0/22
Routes which don‘t fall within this summary range will be leaked through to Router A
10.1.1.0/24
10.2.1.0/24
10.1.2.0/24
10.2.4.0/24
10.1.3.0/24
10.2.2.0/24
10.1.4.0/24
10.2.3.0/24
10.1.5.0/24
10.2.4.0/24
10
.1.1
.0/2
4
10
.2.1
.0/2
4
10
.1.2
.0/2
4
10
.2.4
.0/2
4
10
.2.2
.0/2
4
10
.1.4
.0/2
4
10
.2.3
.0/2
4
10
.1.5
.0/2
4
10
.1.3
.0/2
4
10
.2.5
.0/2
4
A
BC
Leaking More Specifics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 68
Aggregation Techniques
Summarizing to 10.1.0.0/22 on Router B will reduce the number of routes at Router A by two
10.1.0.0/22
10.2.1.0/24
10.2.4.0/24
10.2.2.0/24
10.1.4.0/24
10.2.3.0/24
10.1.5.0/24
10.2.5.0/24
10
.1.1
.0/2
4
10
.2.1
.0/2
4
10
.1.2
.0/2
4
10
.2.4
.0/2
4
10
.2.2
.0/2
4
10
.1.4
.0/2
4
10
.2.3
.0/2
4
10
.1.5
.0/2
4
10
.1.3
.0/2
4
10
.2.5
.0/2
4
A
BC
Leaking More Specifics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 69
Aggregation Techniques
We can do the same thing with the 10.2.0.0 networks on Router C, with 10.2.0.0/21, dropping the number of routes on Router A by two more
The more specific information is still leaked through the summary, so routing still works
10.1.0.0/22
10.2.1.0/24
10.2.4.0/24
10.2.0.0/21
10.1.4.0/24
10.1.5.0/24
10
.1.1
.0/2
4
10
.2.1
.0/2
4
10
.1.2
.0/2
4
10
.2.4
.0/2
4
10
.2.2
.0/2
4
10
.1.4
.0/2
4
10
.2.3
.0/2
4
10
.1.5
.0/2
4
10
.1.3
.0/2
4
10
.2.5
.0/2
4
A
BC
Leaking More Specifics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 70
Aggregation Techniques
If one of the networks behind Router B fails, traffic for that network will be forwarded to Router C
At C, it will be discarded because of the NULL0 route automatically created with the summary
The only danger here is that the link from A to C may be overwhelmed with the extra traffic
10.1.0.0/22
10.2.1.0/24
10.2.4.0/24
10.2.0.0/21
10.1.4.0/24
10.1.5.0/24
10
.1.1
.0/2
4
10
.2.1
.0/2
4
10
.1.2
.0/2
4
10
.2.4
.0/2
4
10
.2.2
.0/2
4
10
.1.4
.0/2
4
10
.2.3
.0/2
4
10
.1.5
.0/2
4
10
.1.3
.0/2
4
10
.2.5
.0/2
4
A
BC
Packets dropped to null 0
Leaking More Specifics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 71
Aggregation Techniques
It‘s also useful to leak more specifics along with (or through) an aggregate
C should receive as few routes as possible
But still optimally route to 10.1.1.0/24 and 10.1.2.0/24 dynamically
There are several ways to accomplish this
Redistributed static routes and route filters
Overlapping Aggregates
Route Leaking (EIGRP)
10.1.0.0/16
10
.1.0
.0/1
6
10
.1.0
.0/1
6
A B
C
10.1.1.0/24 10.1.2.0/24
Leaking More Specifics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 72
Aggregation Techniques
router eigrp 100
redistribute static route-map aggroutes
default-metric 1000 1 255 1 1500
distribute-list 20 out serial0/0
!
ip route 10.1.0.0 255.255.0.0 null0
!
route-map agg-routes permit 10
match ip address 10
match interface serial 0/0
!
access-list 10 permit 10.1.0.0 0.0.255.255
access-list 20 permit 10.1.1.0 0.0.255.255
router eigrp 100
redistribute static route-map aggroutes
default-metric 1000 1 255 1 1500
distribute-list 20 out serial0/0
!
ip route 10.1.0.0 255.255.0.0 null0
!
route-map agg-routes permit 10
match ip address 10
match interface serial 0/0
!
access-list 10 permit 10.1.0.0 0.0.255.255
access-list 20 permit 10.1.2.0 0.0.255.255
10.1.0.0/16
10
.1.0
.0/1
6
10
.1.0
.0/1
6
A B
C
10
.1.1
.0/2
4
10
.1.2
.0/2
4
10.1.1.0/24 10.1.2.0/24
Leaking More Specifics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 73
Aggregation Techniques
EIGRP allows overlapping summaries
Set the administrative distance on the longer prefix so it‘s not installed...
interface serial 0/0
....
ip summary-address eigrp 1 10.1.0.0 255.255.0.0
ip summary-address eigrp 1 10.1.1.0 255.255.255.0 255
Interface serial 0/0
....
ip summary-address eigrp 1 10.1.0.0 255.255.0.0
ip summary-address eigrp 1 10.1.2.0 255.255.255.0 255
10.1.1.0/24 10.1.2.0/24
10.1.0.0/16
10
.1.0
.0/1
6
10
.1.0
.0/1
6
A B
C
10
.1.1
.0/2
4
10
.1.2
.0/2
4
Leaking More Specifics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 74
Aggregation Techniques
EIGRP can leak more specific routes through a summary, as well
CSCed01736, 12.3(11.01)T
route-map LeakList permit 10
match ip address 1
!
access-list 1 permit 10.1.2.0
!
interface Serial0/0
ip summary-address eigrp 1
10.1.0.0 255.255.0.0 leak-map LeakList
10.1.1.0/24 10.1.2.0/24
10.1.0.0/16
10
.1.0
.0/1
6
10
.1.0
.0/1
6
A B
C
10
.1.1
.0/2
4
10
.1.2
.0/2
4
route-map LeakList permit 10
match ip address 1
!
access-list 1 permit 10.1.1.0
!
interface Serial0/0
ip summary-address eigrp 1
10.1.0.0 255.255.0.0 leak-map LeakList
Leaking More Specifics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 75
Aggregation Techniques
We can also get some gains by trying to do less, and using smaller summary blocks
Router B can advertise 10.1.2.0/23, saving one route
Router C can advertise 10.1.4.0/23
Router C can advertise 10.2.2.0/23
The gains might seem small, but with enough work, they can build up into significant savings
10.1.1.0/24
10.2.1.0/24
10.1.2.0/23
10.2.4.0/24
10.2.2.0/23
10.1.4.0/23
10.2.4.0/24
10
.1.1
.0/2
4
10
.2.1
.0/2
4
10
.1.2
.0/2
4
10
.2.4
.0/2
4
10
.2.2
.0/2
4
10
.1.4
.0/2
4
10
.2.3
.0/2
4
10
.1.5
.0/2
4
10
.1.3
.0/2
4
10
.2.5
.0/2
4
A
BC
Smaller Aggregates
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 76
10
.1.1
.0/2
4
10
.2.1
.0/2
4
10
.1.2
.0/2
4
10
.2.4
.0/2
4
10
.2.2
.0/2
4
10
.1.4
.0/2
4
10
.2.3
.0/2
4
10
.1.5
.0/2
4
10
.1.3
.0/2
4
10
.2.5
.0/2
4
A
BC
Aggregation Techniques
We can combine the larger summaries with the smaller summaries to have the most impact
These are two very effective tools if used together, with a little planning
10.1.0.0/22
10.2.1.0/24
10.2.4.0/24
10.2.0.0/21
10.1.4.0/23
Smaller Aggregates
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 77
Aggregation Techniques
Balance this sort of optimization with the maintenance work it produces in the network
Leaking routes through summaries means checking what adding a new route will do to the summaries and the routing
Summarizing on small blocks means considering the summaries when moving a set of addresses
10
.1.1
.0/2
4
10
.2.1
.0/2
4
10
.1.2
.0/2
4
10
.2.4
.0/2
4
10
.2.2
.0/2
4
10
.1.4
.0/2
4
10
.2.3
.0/2
4
10
.1.5
.0/2
4
10
.1.3
.0/2
4
10
.2.5
.0/2
4
A
BC
10.1.0.0/22
10.2.1.0/24
10.2.4.0/24
10.2.0.0/21
10.1.4.0/23
Smaller Aggregates
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 78
Hiding Topology
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 79
Hiding Topology
Topology information is naturally hidden in distance vector protocols, beyond the next hop
C and D only advertise that they can reach 10.1.1.0/24, not that they are connected to D, which is then connected to 10.1.1.0/24
C
A
B
10.1.1.0/24
DI can reach 10.1.1.0/24
I can reach 10.1.1.0/24
I can reach 10.1.1.0/24
I can reach 10.1.1.0/24
Distance Vector
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 80
Hiding Topology
Distance vector protocols can still have too much topology information
Multiple parallel links can slow down convergence because of overwhelming topology information
General EIGRP rule of thumb: There should be no more paths in the topology table than are allowed to be installed in the routing table
(show ip eigrp topology all vs.maximum paths)
A
10.1.1.0/24
B
Distance Vector
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 81
Hiding Topology
In link state protocols, routers flood information about the state of their links to all other routers, carrying topology information to all the routers in the network
All the routers receiving the flooded link state information are said to be in the same flooding domain
We summarize topology information into reachability information at a flooding domain border
C
A B
10.1.1.0/24
G
D
E F
Border
Connected to E, F, and 10.1.1.0/24
Connected to D and G
Connected to D and G
Connected to C, E, and F
Link State Flooding Domains
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 82
Hiding Topology
OSPF
Flooding Domain == Area
Flooding Domain Border == Area Border Router
Link State Summary == Type 3
Contains only reachability and cost information, no topology
External == Type 5
Contains only reachability and cost information, no topology
Autonomous System Border == Type 4
How to reach a router injecting reachability information from outside OSPF (type 5‘s)
C
A
B
D
10.1.2.0/24
10.1.1.0/24
10.1.2.0/24 external AA BB AB 10.1.1.0/24B CC B
10.1.2.0/24 external AA C10.1.1.0/24 C
Redistributed
Are
a
bo
rde
r
Area 0
10
.1.3
.0/2
4
10.1.3.0/24 C
Link State Flooding Domains
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 83
10
.1.3
.0/2
4
Hiding Topology
Decoding OSPF Stub Areas
―Stub‖ == no link state summaries (type 3)
―Totally‖ == no external information (type 4 or 5)
―Not so‖ == Externals injected as type 7‘s and translated at the border
Stub area receives external routing information from outside the area only (no redistribution within the area)
C
A
B
D
Are
a
bo
rde
r
10.1.2.0/24
10.1.1.0/24
10.1.2.0/24 external AA Cdefault C
Redistributed
Stub Area
No information about 10.1.1.0/24
10.1.2.0/24 external AA BB AB 10.1.1.0/24B CC B
Area 0
10.1.3.0/24 C
Link State Flooding Domains
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 84
10
.1.3
.0/2
4
Hiding Topology
Totally stubby areas receive no information about reachability to external or internal destinations
In a ―Not So Stubby Area (NSSA),‖ or a ―Totally Not So Stubby Area (Totally NSSA),‖ D could originate information about destinations external to OSPF
You should use stub areas by default
Supply minimal information where possible
Consider suboptimal routing when necessary
C
A
B
D
Are
a
bo
rde
r
10.1.2.0/24
10.1.1.0/24
Default C
Redistributed
Totally Stub
No information about 10.1.1.0/24 or 10.1.2.0/24
10.1.2.0/24 external AA BB AB 10.1.1.0/24B CC B
Area 0
10.1.3.0/24 C
Link State Flooding Domains
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 85
Hiding Topology
1000 routers: 90 to 100 ms
2000 routers: 130 to 140 ms
3000 routers: 195 to 205 ms
4000 routers: 285 to 300 ms
350
300
200
150
100
50
250
Milliseco
nd
s
5000
10000
15000
20000
25000
Assessing the Impact
Considering SPF run time for a link state protocol, convergence times vary around the number of routers and the number of routes:
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 86
Hiding Topology
Changing the number of routes can make up to a 10 millisecond difference in SPF run time
Changing the number of routers can make up to a 200 millisecond difference in SPF run time
The number of routers is the primary determinant in SPF run time
350
300
200
150
100
50
250
Milliseco
nd
s
5000
10000
15000
20000
25000
10 ms
200 ms
Assessing the Impact
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 87
Hiding Topology
This isn‘t always the case
The primary cost in convergence is route installation time
Varies platform to platform, and Cisco IOS® to Cisco IOS
350
300
200
150
100
50
250
Milliseco
nd
s
5000
10000
15000
20000
25000
10 ms
200 ms
Assessing the Impact
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 88
Working with Hierarchy
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 89
Working with Hierarchy
Hierarchical Design
Two Layer Hierarchy
Three Layer Hierarchy
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 90
Hierarchical Design
Zones (or Nodes)
A topologically defined part of the network
Attached to other parts of the network through choke points
Choke Points
Places where zones or nodes are connected together
Zones
Choke Points
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 91
Hierarchical Design
Each zone represents a failure domain
Choke points provide:
A place to aggregate reachability information
A place to aggregate topology information
A place to aggregate traffic flows
A place to apply traffic policy
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 92
Hierarchical Design
There are two basic designs:
Two layer
Three layer
Which one is right for a specific network?
Rule of Thumb:
Balance simplicity, optimal routing, and functional separation
How Many Layers?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 93
Hierarchical Design
Geography
Networks contained in smaller spaces lend themselves to two layers
Networks with more ―reach‖ lend themselves to three layers
Topology Depth
The maximum number of hops from one edge to another
The greater the depth, the more layering will help the design
How Many Layers?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 94
Hierarchical Design
Topology Design
The more complex the design, the more splitting the network up into zones will help the design
Policy Implementation
Traffic engineering tends to prefer two layer designs
Resource restriction policies tend to prefer three layer designs
How Many Layers?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 95
Hierarchical Design
Moving the boundary between two pieces of the network may create a choke point which didn‘t exist before
With the logical boundary point behind the lower routers, based on the divisional structure, there‘s no place to summarize
10
.1.0
.0/2
4
10
.1.2
.0/2
4
10
.2.0
.0/2
4
10
.2.2
.0/2
4
10
.1.1
.0/2
4
10
.1.3
.0/2
4
10
.2.1
.0/2
4
10
.2.3
.0/2
4
Sales
Marketing
Logistics
Engineering
No
summarization Logical
boundary
points
Creating Choke Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 96
Hierarchical Design
The logical network structure no longer follows the corporate departments
We now have a point at which we can summarize routes!
Logical
boundary
point10.1.0.0/22
10.2.0.0/22
10
.1.0
.0/2
4
10
.1.2
.0/2
4
10
.2.0
.0/2
4
10
.2.2
.0/2
4
10
.1.1
.0/2
4
10
.1.3
.0/2
4
10
.2.1
.0/2
4
10
.2.3
.0/2
4
Sales
Marketing
Logistics
Engineering
Creating Choke Points
What happens if we move the logical boundary point up one layer?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 97
Hierarchical Design
In this case, moving the logical boundary point down one layer can be used to improve summarization
With EIGRP, it‘s just a matter of configuring summaries in the best possible place
With OSPF and IS-IS, some restructuring of the area or routing domain borders may be needed to change where summarization takes place
Logical
boundary
point
10
.1.0
.0/2
4
10
.1.2
.0/2
4
10
.1.1
.0/2
4
10
.1.3
.0/2
4
10
.2.1
.0/2
4
10
.2.3
.0/2
4
10
.2.0
.0/2
4
10
.2.2
.0/2
4
Creating Choke Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 98
Hierarchical Design
Sometimes, you need to change the topology to build a choke point
A full mesh is just a hierarchical network in disguise!
Creating Choke Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 99
Hierarchical Design
Separating complexity from complexity through choke points amplifies the benefits of hierarchy
Sometimes, logical or physical topology changes are needed to separate complexity from complexity
Creating Choke Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 100
Two Layer Hierarchy
The core gets traffic from one topological area of the network to another
High Speed Switching is the focus
Within the core, avoid
Policy (the more complex the more to avoid it) within the core
Reachability and topology aggregation
Core
Aggregation
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 101
Two Layer Hierarchy
Core routers should summarize routing information towards the aggregation layer
Typically, the fewer number of routes advertised towards the edge, the better
Routing policy may also be implemented at the core edge
How many and what routes will be accepted from each aggregation area, etc.
Core
Aggregation Summary
Policy
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 102
Two Layer Hierarchy
The aggregation layer provides user attachment points
Information about the edge should be hidden from the core using summarization and topology hiding techniques
Core
Aggregation Summarize
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 103
Two Layer Hierarchy
Policy should be placed at the edge of the network
Traffic acceptance (based on load and traffic type)
Filtering unwanted traffic
Security policy
Layer 2 and Layer 3 filters apply at the edge
Core
Aggregation
Policy
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 104
Two Layer Hierarchy
A moderate number of routers are attached to the network
The network doesn‘t have a large wide area component
Distances are small, and all links are similar in speed
Core
Basic Concepts
Small and medium scale campus networks are often modeled as two layer networks
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 105
Customers
Two Layer Hierarchy
ISP networks are often modeled on a two layer hierarchy as well
The core is often mesh or a set of rings, with each POP modeled as a ring or a two layer hierarchy
Topology information is summarized between the POPs and the network core
Address summarization is generally from the core towards the POPs
Core
POP
POP
POP
POP
POP
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 106
Two Layer Hierarchy
In an EIGRP network, the hierarchy is created through summarization, rather than through some protocol defined boundary
There are no ―areas‖ or other ways of dividing a network built into EIGRP itself, since topology information is hidden at each hop in the network anyway
EIGRP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 107
Two Layer Hierarchy
Summarization from the edge towards the core hides details about the user access points from the core
Summarization towards the core can cause routing black holes, however
Summarization
EIGRP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 108
Two Layer Hierarchy
Summarization from the core towards the edge can hide details about the core from the edge routers, as well
This type of summarization can cause suboptimal routing, however
Summarization
EIGRP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 109
Two Layer Hierarchy
OSPF creates edges through areas, using Area Border Routers (ABRs)
Typically, with a two level hierarchy, the ABRs are at the edge of the core
The core is area 0
Area Border
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 110
Two Layer Hierarchy
Summarization is configured at the ABR, on the edge of the edge/aggregation areas and the core
Summarization can also be configured to reduce the amount of reachability information carried into the areas
Area 0
Su
mm
ari
za
tion
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 111
Two Layer Hierarchy
To remove virtually all reachability information into the areas, declare them totally stub or not so totally stub areas
Use totally stub areas when there is a single area border, or when suboptimal routing of traffic exiting the area isn‘t an issue
Use stub areas when there is more than one area border, and optimal routing of traffic leaving the area is important
Area 0
network .... area 1 stub
network .... area 2 stub nosummary
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 112
Three Layer Hierarchy
The core gets traffic from one topological area of the network to another: High Speed Switching
Within the core, avoid
Policy (the more complex the more to avoid it) within the core
Aggregation
Core
Distribution
Access
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 113
Three Layer Hierarchy
Address summarization and aggregation occur at the distribution layer
Address Summarization
Within the distribution layer
At the edge of the distribution layer and the core
At the edge of the distribution layer and the access layer
At both edges of the distribution layer
Traffic Aggregation
High to low speed link transitions
Core
Distribution
Access
Summary
Tra
ffic
ag
gre
ga
tio
n
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 114
Three Layer Hierarchy
The distribution layer is where most of the policy in a three layer network should reside
Routing Policy
Routes accepted from the access layer
Routes will be passed from the core into the access layer
Traffic Engineering
Directing traffic into the best core entry point
Access layer failover
Traffic filters
This should take all the policy load off the network core
Core
Distribution
Access
Policy
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 115
Three Layer Hierarchy
Summarization should be avoided between distribution layer routers!
This can cause a lot of odd and hard to troubleshoot problems within the network
Focus summarization and policy up and down the layers, rather than along the layers
Core
Distribution
Access
No
su
mm
ari
za
tio
n!
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 116
Three Layer Hierarchy
The access layer provides ports for the users to plug in to
Traffic filtering and packet policies are implemented here
Traffic acceptance (based on load and traffic type)
Filtering unwanted traffic at Layer 2 and Layer 3
Security policy
Core
Distribution
Access
Policy
Basic Concepts
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 117
Three Layer Hierarchy
Deeper hierarchy doesn‘t change EIGRP‘s fundamental design concepts
The distribution layer should be the blocking point for EIGRP queries
Provide minimal information toward the core
Provide minimal information toward the access
Access layer routers should be considered for configuration as EIGRP stubs
We discuss EIGRP stubs more in hub and spoke topology considerations
Distribution
Access
Core
Summarize
Consider stubs
EIGRP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 118
Three Layer Hierarchy
For OSPF, the question is whether to place the area borders in the distribution layer, or in the core
The answer to this question is, as always, ―it depends‖
There are two rules of thumb we can work with, though:
Separate complexity from complexity
Place area borders to reduce suboptimal routing and to increase summarization
Distribution
Access
Core
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 119
Three Layer Hierarchy
Complex areas include
Full mesh topologies
Large scale hub and spoke
Highly redundant topologies
Try to separate complex topologies from one another with an area border
You can vary the location of the area borders placing them in the distribution or access layers, depending on the network design
Highly parallel data center
Full mesh core
Large scale hub and spoke
Highly redundant campus
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 120
OSPF Two Layer Hierarchy
To remove virtually all reachability information into the areas, declare them totally stub or not so totally stub areas
Use totally stub areas when there is a single area border, or when suboptimal routing of traffic exiting the area isn‘t an issue
Use stub areas when there is more than one area border, and optimal routing of traffic leaving the area is important
network .... area 1 stub
network .... area 2 stub nosummary
Highly parallel data center
Full mesh core
Large scale hub and spoke
Highly redundant campus
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 121
Working with Topologies
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 122
Working with Topologies
Link State Point-to-Point Broadcast
Controlling Physical Parallelism
Hub and Spoke
Full Mesh
Link State Border Connections
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 123
Link State Point-to-Point Broadcast
Normally, if a set of routers are connected over a broadcast link, each router would form a neighbor relationship with every other router on the link
This can cause a large amount of flooding over the single broadcast network
To reduce flooding and apparent network complexity, link state protocols elect one router to control flooding
OSPF: Designated Router
IS-IS: Designated Intermediate System
A
B C
D E
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 124
Link State Point-to-Point Broadcast
To reduce flooding:
In OSPF, a router that receives new information floods it to the DR, which then refloods it to the other connected routers
In IS-IS, the first router to receive new information floods it, and the DIS coordinates database synchronization between the routers
To reduce apparent complexity:
Each connected router advertises a link to the DR/DIS
The DR/DIS advertises a 0 cost link to each connected router
This converts the full mesh to a set of point-to-point links
A
B
D
C
E
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 125
Link State Point-to-Point Broadcast
If there are only two routers on the broadcast link the DR/DIS adds complexity, rather than removing it
Point-to-point high speed Ethernet segments used in campus environments, data centers, etc.
What could be advertised as a point-to-point is actually advertised as two point-to-points to the DR/DIS
We could reduce the apparent complexity, again, by treating the link as a point-to-point link, rather than as a broadcast link
A
B
D
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 126
Link State Point-to-Point Broadcast
draft-ietf-isis-igp-p2p-over-lan describes a method for OSPF and IS-IS to treat a broadcast link with only two devices attached as a point-to-point link
Implemented in IS-IS with CSCdu51410, using the isis network interface command
Implemented in OSPF as well, using the ip ospf network interface command
A
B
D
A
B
D
interface FastEthernet 0
isis network point-to-point
ip ospf network point-to-point
....
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 127
Controlling Physical Parallelism
More redundancy is better, right?
Not always...
There are 64 paths between these two hosts, 26
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 128
Controlling Physical Parallelism
There Are Several Reasons for Redundancy in a Network:
To provide multiple attachment points for servers and hosts in case of a link or device failure
To provide alternate links through the network in case of link or device failure
To provide optimal routing to services
To provide load sharing in heavily utilized areas
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 129
Controlling Physical Parallelism
It‘s common to build networks with back-to-back routers for redundancy
The routing protocol sees each of these links as a possible transit path, so each link adds another set of paths the routing protocol must consider when calculating the best path
You want to route to these links, not through them
RP Transit
Paths
HSRP Peers
Server Farm Example
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 130
router ospf 100
passive-interface fastethernet 0/0
passive-interface fastethernet 0/1
passive-interface fastethernet 0/2
passive-interface fastethernet 0/3
....
router ospf 100
passive-interface default
no passive-interface fastethernet 1/0
....
-or-
Controlling Physical Parallelism
The solution to this is passive-interface
Configuring an interface as passive in EIGRP, OSPF, or IS-IS will cause it not to form neighbor relationships across the link
These networks will still be advertised as reachable destinations, but they will never be advertised as transit links
Server Farm Example
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 131
Controlling Physical Parallelism
It‘s common to build out alternate links in a network
Adds network resiliency
Can provide optimal routing to resources
Adds additional bandwidth in congested areas of the network
The second link also adds moderate complexity, and more information, into the network
Backup path
Optimal routing
Additional bandwidth
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 132
Controlling Physical Parallelism
Adding a third link almost always approaches the point of diminishing returns, and adds much more network complexity
When considering adding more redundancy, always balance the increased resiliency against the added complexity
Increased network convergence times
Increased management effort
Increased troubleshooting times
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 133
2.5
0 10000
Seconds
Routes
Feasible successor
Controlling Physical Parallelism
The impact of greater levels of redundancy on convergence times can be seen in routing protocol scalability testing
Using EIGRP, with a single backup path, it takes about 1.3 seconds for a router with 10000 routes to converge when the best path fails
Best path
fails
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 134
Controlling Physical Parallelism
Adding the third path increases convergence time to 2 seconds
Adding the fourth path increases convergence time to 2.25 seconds
2.5
0 10000
Seconds
Routes
Best path
fails
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 135
Controlling Physical Parallelism
High availability studies also show the impact of adding the third link is not all that great
Adding a second link will increase reliability significantly
Adding a third link approaches the point of diminishing returns
Combined with the impact of slower convergence times, higher management costs, and slower troubleshooting, the total downtime in a network may actually increase with the addition of large amounts of redundancy
99.50
99.60
99.70
99.80
99.90
100.00
1 link 2 links 3 links 4 links
Relia
bili
ty
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 136
Controlling Physical Parallelism
Try to hide this complexity from other parts of the network, if possible
Summarize just the parallel links into a single advertisement at both sides if you‘re using a distance vector protocol
Summary
Summary
If you‘re adding more links to increase the available bandwidth in a specific place in the network
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 137
Controlling Physical Parallelism
Layer 2 bundling (such MLPPP or EtherChannel®) may be useful to reduce the Layer 3 complexity when using multiple links to build required bandwidth
But be careful of issues with processor utilization due to bundling overhead, troubleshooting complexity, etc.
Link bundle
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 138
Controlling Physical Parallelism
Consider using High Availability (HA) techniques to reduce overlapping redundancy
Stateful Switchover/ NonStop Forwarding with redundant hardware in the same box may be able to replace redundant connections to network connected devices
Single high
availability device
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 139
Controlling Physical Parallelism
Balance between complexity and resiliency
Hide the additional complexity created by redundant links where possible
Summarization
Link bundling (but balance against overhead)
Consider High Availability techniques to reduce heavy redundancy for resiliency
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 140
Hub and Spoke
Hub and spoke networks are often built over point-to-multipoint networks
If the hub is configured to treat the entire point-to-multipoint network as a single interface, it can transmit multicast and broadcast packets which are received by all spoke routers
Layer 3 on the hub router will not notice a single circuit failure
Packets transmitted
here are received by
all spokes
Packets transmitted
here are received
only by the hub router
interface s0/0
ip address 10.1.1.1 255.255.255.0
Basic Design
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 141
Hub and Spoke
The hub router can also be configured to treat each spoke‘s circuit as an individual point-to-point circuit on a subinterface
If end-to-end signaling is in use, a failed circuit will cause the subinterface to fail
Packets
transmitted
here are received
by one spoke
Packets transmitted
here are received
only by the hub router
interface s0/0.1 point-to-point
ip address 10.1.1.0 255.255.255.254
....
interface s0/0.2 point-to-point
ip address 10.1.1.2 255.255.255.254
....
interface s0/0.3 point-to-point
ip address 10.1.1.4 255.255.255.254
interface s0.1 point-to-point
ip address 10.1.1.x 255.255.255.254
....
Basic Design
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 142
Hub and Spoke
You can mitigate the single point of failure in the routers using high availability techniques
Highly
available
Basic Design
In single homed hub and spoke networks, the hub router, spoke routers, and the links themselves are all single points of failure
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 143
Hub and Spoke
Summarize towards the core
Number the remote links out of the same address space as the remote networks, if possible
Use /31‘s to conserve address space for point-to-points
Send the remotes a default only
If you can‘t address the links out of the summary address space, then use a distribute list to filter them from being advertised back into the core of the network
0.0.0.0/0
Summary
only
192.168.1.0/24
192.168.2.0/24
192.168.2.0/24
access-list 10 deny 192.168.0.0 0.0.0.255
access-list 10 permit any
....
router eigrp 100
distribute-list 10 out
Basic Design
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 144
Hub and Spoke
All the same principles apply to dual homed hub and spoke networks
Summarize or filter the links to the remotes
Use /31‘s on point-to-points to conserve address space
Provide as little information as possible to the remotes
Something more than a default route may be required to provide optimal routing
Avoid Summary Black Holes!
0.0.0.0/0
Summary
only
192.168.1.0/24
192.168.2.0/24
192.168.2.0/24
Basic Design
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 145
Hub and Spoke
How do we limit the amount of information passed down to the remote sites?
You can summarize at A and B towards the remote routers
The summary will generate a local route with an administrative distance of five
The external default route learned from D will have an administrative distance of 170
What happens?
Internet
EIGRP
A B
C
DExternal
default
route
D* 0.0.0.0/0 is a summary, 00:08:41, Null0
ip summary-address eigrp 1
0.0.0.0 0.0.0.0
Basic Design: Administrative Distance
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 146
Hub and Spoke
If two routing protocols provide a route to the same destination, how do we choose between them?
Their metrics are not comparable
An administrative distance is added to each route learned based on the protocol installing the route
Static routes can be configured with a distance
This can create a floating static
The route will not be used unless the dynamic protocols have no route to that destination
router#show ip eigrp topology
P 10.0.1.0/24, 1 successors, FD is 2681856
via 10.1.1.1 (2681856/2169856)
router(config)#ip route 10.0.1.0
255.255.255.0 null0
router(config)#ip route 10.0.1.0
255.255.255.0 null0 200
distance 90
distance 1
distance 200
The static
route wins
The EIGRP
route wins
Basic Design: Administrative Distance
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 147
Basic Hub and Spoke Design
The route generated by the summary is called a discard route
What would happen if this route isn‘t created?
Configure two routers back to back with overlapping summaries
Generate a packet towards 10.1.2.1 from either router
At A, the best path is through 10.1.0.0/16 to B
At B, the best path is through 10.0.0.0/8 to A
Routing Loop
10
.0.0
.0/8
10
.1.0
.0/1
6
ip summary-address eigrp 1 10.0.0.0
255.0.0.0
ip summary-address eigrp 1 10.1.0.0
255.255.0.0
10.1.1.0/24
10.2.1.0/24
A
B
10.1.2.1
Basic Design: The Discard Route
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 148
Hub and Spoke
In this case, the locally generated discard route wins
The route learned from D will not be installed in the local table
Hosts behind C will not be able to reach destinations on the Internet
There are ways to prevent this discard route from being installed, but we need to be careful with the design
Routing Loops
Routing Black Holes
There is enough rope here to hang yourself!
D* 0.0.0.0/0 is a summary, 00:08:41, Null0
ip summary-address eigrp 1
0.0.0.0 0.0.0.0
Internet
EIGRP
A B
C
DExternal
default
route
Basic Design: The Discard Route
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 149
Internet
EIGRP
A B
C
DExternal
default
route
Hub and Spoke
To remove the discard route
In EIGRP, add an administrative distance after the ip summary address command
In OSPF, use the command no discard-route under the routing process
What happens if A loses its path to D?
C will now prefer the internal learned through A over the external learned trough B
We have a black hole
ip summary-address eigrp 1 0.0.0.0
0.0.0.0 200D* 0.0.0.0/0 [170/409600] via <A>
[170/409600] via <A>
D* 0.0.0.0/0 [90/409600] via <A>
[90/409600] via <A>
Basic Design: The Discard Route
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 150
Hub and Spoke
You can also use floating static routes at the two hub routers and redistribute them into the routing protocol
Distribute list 10 only allows the default route to be advertised to the remotes
Distribute list 20 prevents a default route from being leaked back into the core
This has the same problem if a single link back towards the core and the injected external route both fail
There are other situations under which this also fails
A
C
B
access-list 10 permit host 0.0.0.0
access-list 20 deny host 0.0.0.0
access-list 20 permit any
....
ip route 0.0.0.0 0.0.0.0 null0 250
....
router eigrp 100
redistribute static
distribute-list 10 out <remote 1>
distribute-list 10 out <remote 2>
distribute-list 10 out <remote 3>
distribute-list 20 out <core>
Basic Design: Summary Black Hole
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 151
Full routing information
Basic Hub and Spoke Design
One solution is to have a link between the summarizing routers across which they share full routing information
Conditional advertisement of routing information is another possible solution
OSPF can conditionally generate a default route
EIGRP has conditional advertisement as a planned feature
Internet
EIGRP
A B
C
DExternal
default
route
Basic Design: Summary Black Hole
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 152
Hub and Spoke
EIGRP can run over either a multipoint interface at the hub router or point-to-point subinterfaces
A single multipoint interface is easier to configure but it can be harder to troubleshoot
Use summarization at the hub routers to reduce information into the network core
Provide as little information to the remotes as possible
Declare the remote routers as stubs
0.0.0.0/0
Summary
only
192.168.1.0/24
192.168.2.0/24
192.168.2.0/24
Single multipoint
or several
point-to-points
router eigrp 100
eigrp stub connected
....
EIGRP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 153
Hub and Spoke
Multiple Interfaces
Processor/Process Scalability is the primary limiting factor
Same Interface
Queue Congestion/Drops bottleneck is the primary limiting factor
Theoretical Limitations
EIGRP has a limitation of 2000 peers per interface, currently
EIGRP Scaling
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 154
Hub and Spoke
The blue line shows the rate at which the convergence time increases as EIGRP neighbors are added to hub routers and does not pass 500
The red line shows the convergence time if the neighbors added are all configured as EIGRP stub routers and scales to over 1000 peers
Measure initial bring up convergence until all neighbors are established and queues empty
Dual Homed Remotes, NPE-G1 with 1G RAM, 3000 prefixes advertised to each spoke
2
5
9
0 500 1000 1500
Number of Neighbors
Tim
e (
min
ute
s)
Test performed with 12.3(14)T1
Non-Stub
EIGRP Stub
EIGRP Scaling
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 155
Hub and Spoke
The blue line with the steep slope shows the rate at which the failover convergence time increases as EIGRP neighbors are added to a single hub router
The red line shows the failover convergence time if the neighbors added are all configured as EIGRP stub routers and is extremely linear in behavior
Primary Hub failed, time measured for EIGRP to complete failover convergence
Dual Homed Remotes, NPE-G1 with 1G RAM, 3000 prefixes advertised to each spoke
0
1
60
0 200 400 600 800 1000 1200 1400 1600
Number of Neighbors
Tim
e (
min
ute
s)
Test performed with 12.3(14)T115
EIGRP Stub
Non-Stub
EIGRP Scaling
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 156
Hub and Spoke
Most EIGRP Neighbors Seen
800 deployed in live, working networks
1400 is the largest number ever tested in a lab environment
Key Strategy for achieving scalability is design!
Stub for EIGRP hub and spoke environments is a must
Minimize advertisements to spokes
EIGRP Scaling
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 157
Hub and Spoke
B and D don‘t receive C‘s packets, so they think A has the highest IP address, and elect A as DR
C elects itself as DR
Flooding will fail miserably in this situation
―A is DR‖ ―C is DR‖ ―A is DR‖
―C is DR‖
A
B C D
interface s0/0
ip address 10.1.1. 255.255.255.0
ip ospf priority 200
....interface s0
ip ospf priority 0
....
OSPF
OSPF can treat a multipoint link as a broadcast network, but we need to be careful about designated router (DR) issues
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 158
A
B C D
Hub and Spoke
Set the OSPF DR priorities so the hub router is always elected DR
Set the spokes to 0 so they don‘t participate in DR election
The remote sites won‘t be able to reach each other without some special considerations, either
Maps pointing each remote‘s address to A‘s circuit can solve this
―A is DR‖ ―C is DR‖ ―A is DR‖
―C is DR‖
interface s0/0
ip address 10.1.1. 255.255.255.0
ip ospf priority 200
....interface s0
ip ospf priority 0
....
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 159
Hub and Spoke
OSPF can treat a multipoint link as a non-broadcast network
Each spoke router must be manually configured as a neighbor
In a large hub and spoke environment, this would be very difficult to maintain
The remote sites can‘t reach each other using this method
Circuit maps pointing each remote to each other remote can be used to resolve this
interface s0/0
ip ospf network non-broadcast
....
router ospf 100
neighbor 10.1.1.2
neighbor 10.1.1.3
neighbor 10.1.1.4
interface s0
ip ospf network non-broadcast
....
router ospf 100
neighbor 10.1.1.1
A
B C D
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 160
Hub and Spoke
You can also configure the serial interface at the hub router as a point-to-multipoint type
All the remotes are in a single IP subnet
OSPF treats each remote as a separate point-to-point link for flooding
OSPF will advertise a host route to the IP address of each spoke router to provide connectivity
10.1.1.2/32
10.1.1.3/32
10.1.1.4/32
...
interface s0/0
ip address 10.1.1.1 255.255.255.0
ip ospf network point-to-multipoint
interface s0
ip address 10.1.1.x 255.255.255.0
ip ospf network point-to-point
A
B C D
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 161
Hub and Spoke
OSPF can also use point-to-point subinterfaces, treating each one as a separate point-to-point link
These uses more address space, and requires more administration on the router
Use /31 addresses for these point to point links
interface s0/0.1 point-to-point
ip address 10.1.1.0 255.255.255.254
....
interface s0/0.2 point-to-point
ip address 10.1.1.2 255.255.255.254
....
interface s0/0.3 point-to-point
ip address 10.1.1.4 255.255.255.254
interface s0.1 point-to-point
ip address 10.1.1.x 255.255.255.254
....
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 162
Hub and Spoke
Network Type Advantages Disadvantages
Single interface at the hub treated as an OSPF broadcast network
ip ospf network-type
broadcast
Single IP subnet
Fewer nodes in the SPF tree
Manual configuration of each spoke with the correct OSPF priority
Remote-to-remote connectivity difficult
Single interface at the hub treated as an OSPF nonbroadcast network
ip ospf network-type
nonbroadcast
Single IP subnet
Fewer nodes in the SPF tree
Manual configuration of the hub and spokes with correct unicast neighbors
Remote-to-remote connectivity difficult
Single interface at the hub treated as an OSPF point-to-multipoint network
Single IP subnet
No configuration per spoke
Additional host routes inserted in the OSPF database and routing table
Individual point-to-point interface at the hub for each spoke
ip ospf network-type point-
to-point
Can take advantage of end-to-end signaling for down state
Lost IP address space
More routes in the OSPF database and routing table
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 163
Hub and Spoke
If possible, make them totally stubby
If there is redistribution at the spokes, make the area totally not-so-stubby
Area 1
router ospf 100
area 1 stub no-summary
....
router ospf 100
redistribute rip metric 10
....
router ospf 100
area 1 nssa no-summary
....
OSPF
The areas the spokes are placed in should always be the ―most stubby‖ you can get away with
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 164
Hub and Spoke
If you need to leak some routing information from area 0 into the spoke areas, use type 3 LSA filtering at the border to remove as much information as possible
OSPF Hub and Spoke Areas, currently in development, would allow an area where the spoke routers only receive the default route
Area 1
ip prefix-list 10 permit 10.1.1.0/24 ge 25
ip prefix-list 10 deny all
....
router ospf 100
area 1 filter-list prefix-list 10 in
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 165
Hub and Spoke
Once you‘ve determined how to configure the hub‘s interface, you need to decide how to divide the remote sites among flooding domains
If the hub and spoke section of the network is small, and fits well within some other area structure, then the entire hub and spoke can be placed in this single flooding domain
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 166
Hub and Spoke
Remember each spoke router receives all the topology information from all the other spoke routers
OSPF
If the hub and spoke is large enough, you‘ll want to split it off as its own flooding domain
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 167
Hub and Spoke
Low speed links and large numbers of spoke may require multiple flooding domains
Balance the number of flooding domains on the hub against the number of spokes in each flooding domain
The link speeds and the amount of information being passed through the network determine the right balance
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 168
Hub and Spoke
Dual homed remotes make the division of flooding domains significantly more difficult
If all the spoke routers will fit, put both the hubs and all the spokes in a single flooding domain
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 169
Two links, one in each flooding domain
Hub and Spoke
You should build links between the hub routers within each flooding domain in some way to prevent routing black holes
Put two links between the area borders, one in each area or flooding domain
OSPF
If all the spokes will not fit into a single flooding domain, split the hub and spoke up into multiple areas or flooding domains
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 170
Hub and Spoke
The blue line shows the rate at which the startup convergence time increases as OSPF neighbors are added to the hub routers and peaks at the 700 router mark
The red line starts and ends below the green line showing the startup convergence time if the neighbors added are all configured as OSPF neighbors are added to a Totally Stubby area
Measure initial bring up convergence until all neighbors are established and queues empty, SPF completes
Dual Homed Remotes, NPE-G1 with 1G RAM, 800 prefixes advertised to each spoke
0
50
100
150
200
250
300
350
400
0 100 200 300 400 500 600 700 800
Number of Spokes
Co
nv
erg
en
ce T
ime (
seco
nd
s)
Test performed with 12.3(14)T1
Single Area
Totally Stubby Area
OSPF Scaling
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 171
Hub and Spoke
The blue line, ending above the red line, shows the rate at which the failover convergence time increases as OSPF neighbors are added to a single hub router
The red line shows the failover convergence time if the neighbors added are all configured as OSPF neighbors are added to a Totally Stubby area
Primary Hub failed, time measured for OSPF to complete failover convergence
Dual Homed Remotes, NPE-G1 with 1G RAM, 800 prefixes advertised to each spoke
0
5
10
15
20
25
30
35
40
45
50
0 100 200 300 400 500 600 700 800
Number of Spokes
Co
nv
erg
en
ce T
ime (
seco
nd
s)
Test performed with 12.3(14)T1
Single Area
Totally Stubby Area
OSPF Scaling
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 172
Hub and Spoke
Most OSPF Neighbors Seen
200 Deployed in live, working networks
600 is the largest number ever tested in a lab environment
Key Strategy for achieving scalability is design!
Minimize advertisements to spokes
Area placement is the key to summarization, filtering, etc.
Use the most stubby area possible
OSPF Scaling
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 173
Full Mesh
Full mesh topologies are complex:
2 routers == 1 link
3 routers == 3 links
4 routers == 6 links
5 routers == 10 links
6 routers == 15 links
...
Adjacencies == nodes(nodes-1)/2
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 174
Full Mesh
60 node TEST network
1770 links
NPE-G1, NPE-400s
All devices on same physical Ethernet (via a switch), full mesh created with GRE Tunnels
Three tests performed
Initial convergence, measured from interface bring up
Flap a transit link, such that a routing adjacency will reset
Flap a stub network, to measure prefix propagation
This test does not consider stability, only convergence!
Scaling Tests
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 175
Full Mesh
EIGRP OSPF Default Timers OSPF Tuned Timers
Initial Convergence 1:13 1:13 1:18
Link Flap 0:51 0:43 0:41
Prefix Flap 0:15 0:09 0:03
Scaling Tests
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 176
Full Mesh
Flooding routing information through a full mesh topology is also complicated
Each router will, with optimal timing, receive at least one copy of every new piece of information from each neighbor on the full mesh
There are several techniques you can use to reduce the amount of flooding in a full mesh
Mesh groups reduce the flooding in a full mesh network
Mesh groups are manually configured ―designated routers‖
New information
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 177
interface serial x
ip ospf database-filter all out
....
Full Mesh
Pick one or two routers to flood into the mesh, and block flooding on the remainder
This will reduce the number of times information is flooded over a full mesh topology
New information
OSPF
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 178
Full Mesh
Routes must be advertised between every pair of peers in the mesh so each router has the correct next hop and routing information
Number the links so they can be summarized to a single advertisement at the edge
Number the links so the link information can be filtered out at the edge
Summarize
EIGRP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 179
Full Mesh
Consider High Availability ring topologies, such as SRP, SONET rings, and others as an alternative to full mesh high speed networks in POPs and other enclosed networks
This can provide resiliency against a single failure in the network, and simplify the topology from the perspective of routing dramatically
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 180
Link State Border Connections
Be careful with links between border routers in OSPF and IS-IS
Traffic prefers to stay within the flooding domain no matter what the actual link costs are
To reach A, we will take the higher cost link if the border link is in the backbone
To reach B, we will take the higher cost link if the border link is in the area or L1 domain
This is because we are removing topology information at the border, and always trust routes with more explicit topology information
100
10 10
A
B100
10.1.1.0/24, cost 10
100
10 10
A
B100
10.1.1.0/24, cost 10
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 181
Link State Border Connections
In OSPF, we have to decide which traffic we want to route optimally
The ability to place a single link in two areas is under consideration within the OSPF working group
In IS-IS, we can place the link in both the L1 and L2 routing domains, and optimally route both ways
100
10 10
A
B100
10.1.1.0/24, cost 10
100
10 10
A
B100
10.1.1.0/24, cost 10
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 182
Working with Redistribution
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 183
Working with Redistribution
Alternatives to Redistribution
Single Point of Redistribution
Multiple Points of Redistribution
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 184
Alternatives to Redistribution
When connecting to an outside network, creating static routes at the edge, and redistribute those, instead of redistributing live routing information
This prevents misconfigurations and rapid topology changes in the other network from impacting you
It also prevents someone from injecting false information to attack your routing system
BigShoes, Inc
10.1.0.0/16
MediumSocks, LTD
10.2.0.0/16
Redistribute
EIGRP to OSPF
Redistribute
OSPF to EIGRP
ip route 10.1.0.0 255.255.255.0 s0/0
!
router ospf 100
redistribute state metric 10
ip route 10.2.0.0 255.255.255.0 s0/0
!
router eigrp 100
redistribute state metric 1000 1 255 1 1500
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 185
Alternatives to Redistribution
Even if you must have live routing data, don‘t redistribute between IGPs to connect to an outside network; this opens serious security holes in routing
Instead, use eBGP, so you can do policy based filtering on the routes you‘re receiving
BigShoes, Inc
MediumSocks, LTD
Redistribute
EIGRP to OSPF
Redistribute
OSPF to EIGRP
eBGP
AS65000
AS65001
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 186
Alternatives to Redistribution
Use redistribution when permanently merging two networks into a single administrative domain
Use redistribution as a transition strategy when switching routing protocols
Use redistribution to split off a section of the network for security, experimental, or administrative reasons
BigShoes, Inc
MediumSocks, LTD
BigShoes, Inc
MediumSocks, LTD
Redistribute
EIGRP to OSPF
Redistribute
OSPF to EIGRP
Socks&Shoes, Corp
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 187
Single Point of Redistribution
Single points of redistribution are simple to manage and control
There is little or no chance of routing loops or other problems with single points of redistribution
They are also single points of failure; consider using high availability methods to reduce the risk
EIG
RP
OS
PF
router ospf 100
redistribute eigrp 100 metric 10
....
!
router eigrp 100
redistribute ospf 100 metric 1000 1 255 1 1500
....
Single point of failure
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 188
Multiple Points of Redistribution
Multiple points of redistribution resolve the single point of failure
The cost is dramatically increased network complexity and the possibility of permanent routing loops E
IGR
P
OS
PF
router ospf 100
redistribute eigrp 100 metric 10
....
!
router eigrp 100
redistribute ospf 100 metric 1000 1 255 1 1500
....
router ospf 100
redistribute eigrp 100 metric 10
....
!
router eigrp 100
redistribute ospf 100 metric 1000 1 255 1 1500
....
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 189
Multiple Points of Redistribution
A route is injected into EIGRP as an external; this route is redistributed through B into OSPF
The route is transmitted to A through OSPF, and redistributed into EIGRP
The metric is set manually in redistribution at A to something lower than the original external injected into EIGRP
B prefers this route, building a routing loop
A
EIG
RP
OS
PF
BMetric 10 Metric 2816000
10.1.1.0/24
Metric 2560256
Metric
2688000
Metric 25 Metric 2560256
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 190
Multiple Points of Redistribution
There Are Three Ways to Prevent This Routing Loop:
Only redistributing live routing information in one direction
Filtering routes based on the network advertised to prevent feedback
Filtering routes using routing tags to prevent feedback
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 191
Multiple Points of Redistribution
If live routing data is only needed in one direction (normally, this is true), redistribute a static in one direction, and between protocols in the other direction
ip route 10.2.1.0 255.255.255.0 serial 0/0
....
router ospf 100
redistribute eigrp 100 metric 10
....
router eigrp 100
redistribute static 100 metric 1000 1 255 1 1500
....
A
EIG
RP
OS
PF
B
10.1.1.0/2410.1.2.0/24
Single Redistribution Direction
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 192
Multiple Points of Redistribution
To filter based on prefixes, configure access lists which match the address ranges used by each section of the network
Use these access lists to filter routes redistributed between protocols
access-list 10 permit 10.1.0.0 0.0.255.255
access-list 20 permit 10.2.0.0 0.0.255.255
....
router ospf 100
redistribute eigrp 100 metric 10 distribute-list 10
....
router eigrp 100
redistribute ospf 100 metric 1000 1 255 1 1500 distribute-list 20
....
10.1.1.0/2410.1.2.0/24
A
EIG
RP
OS
PF
B
Filtering Based on Prefixes
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 193
Multiple Points of Redistribution
EIGRP and OSPF can set tags on their external routes
Set the tag when redistributing between the protocols; deny tagged routes at the redistribution point
route-map usetags deny 10
match tag 1000
route-map usetags permit 20
set tag 1000
....
router ospf 100
redistribute eigrp 100 metric 10 route-map usetags
....
router eigrp 100
redistribute ospf 100 metric 1000 1 255 1 1500 route-map usetags
....
10.1.1.0/2410.1.2.0/24
A
EIG
RP
OS
PF
B
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 194
Transitioning Protocols
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 195
Transitioning Routing Protocols
Basics
Cutover At Once
Splitting the Problem
Using Redistribution
Using Administrative Distance
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 196
Transitioning Routing Protocols
There is a quick and easy way to transition from one protocol another without any network downtime
Perhaps—If you discover it, let me know
It‘s impossible to transition from one routing protocol to another in a really large network
It‘s almost always difficult, but never impossible
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 197
Transitioning Routing Protocols
It‘s never worth the trouble of switching routing protocols
That depends...
Would the cost benefits outweigh the transition costs?
Differentials in overall equipment costs in the future
Convergence speeds on specific network topologies
Other factors
You sometimes don‘t have a choice, such as when merging two networks
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 198
Transitioning Routing Protocols
―We want faster convergence...‖
Generally convergence is a matter of design, rather than protocol
―Our network design is hub and spoke, so it fits better for EIGRP...‖
Can‘t argue with this one…
―We want a standards based protocol…‖
What, so you can install some ―other‖ vendor‘s equipment? Are you insane????
―We‘re all studying for our CCIEs, and need exposure to other protocols…‖
Basics
What reasons have we heard in the field for switching routing protocols?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 199
Cutover at Once
Start from one end of the network
Telnet to the other end hop by hop, removing routing at each step
Apply new routing protocol at the router farthest away
Back out, applying new routing protocol hop by hop
Telnet
Telnet
Telnet
Removerouting
Removerouting
Configure routing
Configure routing
Removerouting
Configure routing
Removerouting
Configure routing
Don‘t count on routed reachability while you are switching the routing protocol
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 200
Cutover at Once
If the network destabilizes, take a break
At each router, wait until the network has converged before moving to the next router
When you configure routing on a given router, wait until the routing protocol is quiescent
For instance, for EIGRP, look at show ip eigrp neighbors, and wait until
the Q Count is 0 on all interfaces
This technique should be used whether converting manually or when using a script
Telnet
Telnet
Telnet
Removerouting
Removerouting
Removerouting
Removerouting
Configure routing
Configure routing
Configure routing
Configure routing
Wait for convergence
Wait for convergence
Wait for convergence
Wait for convergence
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 201
Cutover at Once
If the process stalls or fails, each device should be left completely in a known state
There should be no chance of partial configurations
Only one of three states should be possible
The old routing protocol is completely configured
No routing is configured
The new routing protocol is completely configured
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 202
Cutover at Once
Create each new routing configuration in a locally accessible file
Remember not to count on reaching a server
At each router
Open file with new routing protocol configuration commands
no router xxxx to remove
the old routing protocol
config t
(Copy/paste)
copy run start
reload
router-b(config)#no router xxxx
A
B
router-a#telnet <b>
host#telnet <a>
host#<edit config b>
router xxxx
network xx.xx.xx.xx
....
router-b(config)#router xxxx
router-b(config-rtr)#....
<Copy>
<Paste>
router-b(config-rtr)#exit
router-b#copy run start
router-b#reload
(Text editor)
router xxxx
network xx.xx.xx.xx
....
Make the Process Atomic
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 203
Cutover at Once
Create each new routing configuration
Copy the configuration into a file on the local flash of each device
To convert
Telnet to each router
Remove routing
no router xxxx
Copy the local file from flash to the startup configuration
copy <file> run
Reload the router
You can also copy the new configuration directly to the startup configuration and reload, rather than to the running configuration
router-b(config)#no router xxxx
A
B
router-a#telnet <b>
host#telnet <a>
router-b(config)#exit
router-b#copy slot0:newconfig run
router-b#copy run start
router-b#reload
....
router-b>
<ctrl>+<shift>+6
x
router-a#copy slot0:newconfig run
router-a#copy run start
router-a#reload
Make the Process Atomic
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 204
Cutover at Once
Failure Point Result
Before the old routing protocol is removedRouter can be reached through old routing protocol or direct connections (interface addresses are not removed)
After the old routing protocol is removedRouter can be reached through direct connections
After the new routing protocol is configured
Router can be reached through new routing protocol or direct connections
After the new routing protocol is configured and saved, and router is reloaded
Router can be reached through new routing protocol or direct connections
Make the Process Atomic
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 205
Splitting the Problem
In really large networks, you might have to split the problem into pieces
Consider the network as a set of smaller networks, and convert each part separately
Where can you split a network?
Hierarchical division points
Aggregation points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 206
Splitting the Problem
In a two layer hierarchy, the only real choice is to split the network along the core/aggregation divide
Each ―lobe‖ within the aggregation layer can be converted separately
The network core can be converted separately
core
aggregation
Hierarchical Division Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 207
Splitting the Problem
In a three layer hierarchy, the split points are going to depend on the size of each ―lobe‖ and ―layer‖ in the network
Each access layer ―lobe‖ can be converted separately
The core can be converted as one unit
The distribution layer can either be converted with the core, with the access layer, or separately, in ―lobes,‖ etc.
core
distribution
access
Hierarchical Division Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 208
Splitting the Problem
Which part should you convert first?
Start at the edge and work in?
Start at the core and work out?
This question applies to both
Converting individual pieces of the network
The order in which to convert network pieces
Ed
ge
in?
Co
re O
ut?
Hierarchical Division Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 209
Splitting the Problem
Typically, it‘s easier to work from the edge in…
This tends to work with aggregation and network design, rather than against it
Provides a set of ―lower risk‖ areas to work in, and perfect techniques
But… in some cases, core out might be easier
I‘ve just never seen a network where it is…
Ed
ge
in?
Co
re O
ut?
Hierarchical Division Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 210
Splitting the Problem
Another good place to divide the network is at aggregation points
This will often be along hierarchical boundaries, anyway…
If you choose different aggregates in the new protocol, both protocols can run at the same time, along the edges
This allows you to convert one section of the network at a time
Aggregation Points
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 211
Using Redistribution
Once you‘ve split the network up into pieces to convert, how do you actually convert each piece, and still have a working network?
One ―easy‖ answer is redistribution…
A
B
C
router ospf 100
network 0.0.0.0 0.0.0.0 area 0
router ospf 100
network 0.0.0.0 0.0.0.0 area 0
area 0 range 10.1.0.0 0.0.255.255
....
router eigrp 100
network 0.0.0.0
router eigrp 100
network 0.0.0.0
Redistribute here?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 212
Using Redistribution
Redistribution is probably one of the most ―counted on‖ tools to convert from one routing protocol to another
But, it‘s a lot like playing with fire…
You can cook a really nice omelet, or you can get really burnt!
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 213
Using Redistribution
In a simple network design, it‘s easy to move redistribution around as you convert
How many networks have simple linear topologies like this one, though?
New protocol
Old protocol
Old protocol
New protocol
Old protocol
Redistribution
New protocol
Old protocol
Redistribution
Old protocol
New Protocol
New protocolNew protocol
Old protocol
Old protocol
New protocol
Old protocol
Redistribution
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 214
Using Redistribution
The more complex a network‘s topology is, the more places redistribution is required to convert from one protocol to another
More points of redistribution means:
More complexity in moving the protocol conversion over at each step
More chances for human error in configurations
More complex problems if the network fails during conversion
etc.
New Protocol
Redistribution
Old Protocol
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 215
10.1.1.0/24
Using Administrative Distance
Create the new protocol on all the routers
Set the administrative distance so the new protocol never wins
Take the old protocol off
router ospf 100
network 0.0.0.0 0.0.0.0 area 0
distance 200
no router rip
B
C
D
A
Distance Vector to Link State
When converting from a Distance Vector to Link State protocol…
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 216
Using Administrative Distance
Add OSPF to all routers
At B, remove RIP
Does this work?
A has a route to 10.1.1.0/24 through OSPF
B has a route to 10.1.1.0/24 through OSPF
C has a route to 10.1.1.0/24 through RIP
D has a connected route to 10.1.1.0/24
This works…
10.1.1.0/24
router ospf 100
network 0.0.0.0 0.0.0.0 area 0
distance 200
no router rip
B
C
D
A
Distance Vector to Link State
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 217
Using Administrative Distance
Things can get harder around aggregation points and area borders
Suboptimal routing is the rule rather than the exception
In some cases, suboptimal routing can become extreme
Area 0
Area 1Totally Stub
RIP
ip summary-address 10.1.0.0 255.255.0.0 10.1.1.0/24
Only path to 10.1.2.1 is through C
Best path to 10.1.2.1 is through B
Throw traffic to 10.1.2.1 away through the discard
route
Distance Vector to Link State
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 218
Using Administrative Distance
Add EIGRP to all routers
Remove RIP from B
D advertises 10.1.1.0/24 through RIP and EIGRP
C receives 10.1.1.0/24 in both RIP and EIGRP, but doesn‘t advertise it through EIGRP because the RIP route is installed in the routing table
B has no route to 10.1.1.0/24
This doesn‘t work!
10.1.1.0/24
router eigrp 100
network 0.0.0.0 0.0.0.0
distance eigrp 190 200
no router rip
B
C
D
A
Distance Vector to Distance Vector
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 219
Using Administrative Distance
Note B is the first router that doesn‘t have a route to 10.1.1.0/24
This technique won‘t work in the general case, then, but it is useful in some cases, even with distance vector protocols
10.1.1.0/24
router eigrp 100
network 0.0.0.0 0.0.0.0
distance eigrp 190 200
no router rip
B
C
D
A
B is the first router without a route to 10.1.1.0/24
Distance Vector to Distance Vector
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 220
Using Administrative Distance
You can use the administrative distance to your advantage when using a ―cutover at once‖ technique
Rather than removing the old routing protocol at each step, then installing the new one…
Configure the new routing protocol at each router, making certain the new protocol doesn‘t take routing over
To convert the network, walk through the each router, changing one of the two protocol‘s administrative distance to make the new protocol win, and the old protocol lose
Telnet
Telnet
Telnet
Removerouting
Removerouting
Removerouting
Removerouting
Configure routing
Configure routing
Configure routing
Configure routing
Wait for convergence
Wait for convergence
Wait for convergence
Wait for convergence
Combined with a Cutover
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 221
Using Administrative Distance
If you use this technique…
Watch for unpredictable routing as you‘re converting, especially if you‘re converting from a distance vector protocol to a link state protocol
Be careful not to rely on routing to modify routing
Never count on a routed path to reach a router that you‘re working on
Always telnet hop by hop when converting
Don‘t be too hasty to back out, if things start looking wrong
Troubleshoot the problem
Make certain it doesn‘t relate to both protocols running at the same time
Combined with a Cutover
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 222
Using Administrative Distance
For protocols that rely on the administrative distance to sort routes…
EIGRP
BGP
Do not reverse the administrative distance of their routes
Don‘t make external EIGRP routes preferred over internal EIGRP routes
This is a certain path to routing loops and major network failures
Warning
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 223
BGP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 224
BGP
BGP Basics
Route Reflectors
BGP Cores
Outside Connections
BGP/IGP Interaction
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 225
BGP Basics
Interior Gateway Protocols:
Automatic discovery
Generally trust your IGP neighbors
Routes go to all IGP neighbors
Exterior Gateway Protocols
Specifically configured peers
Connecting with outside networks
Set administrative boundaries
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 226
BGP Basics
Autonomous System: A network(s) sharing the same routing policy
Possibly multiple IGPs
Usually under single administrative control
Contiguous internal connectivity
Numbering range from 1 to 65,535—globally unique—―AS Number‖
Private range: 64512–65535
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 227
BGP Basics
Learns multiple paths via internal and external BGP speakers
Picks THE bestpath, installs it in the IP forwarding table, forwards to EBGP neighbors (not IBGP)
Policies applied by influencing the bestpath selection
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 228
BGP Basics
Summary of Peering Operation:
TCP connection established (port 179)
Both peers attempt to connect—there is an algorithm to resolve ―connection collisions‖
Exchange messages to open and confirm the connection parameters
Initial exchange of entire table
Incremental updates after initial exchange
Keepalive messages exchanged when there no updates
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 229
BGP Basics
External (eBGP) connections are to BGP peers in other autonomous systems
Internal (iBGP) peers are to BGP peers in the same autonomous system
A
B
BGP core
IGP Area
C
eBGP
sessionrouter bgp 65000
neighbor 10.1.1.1 remote-as 65000
router bgp 65000
neighbor 10.1.1.2 remote-as 65000
neighbor 10.2.2.1 remote-as 65001
router bgp 65001
neighbor 10.2.2.2 remote-as 65001
iBGP
session
Peering
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 230
BGP Basics
When B learns a route from C through eBGP, it sets the next hop towards the destination to C
When it advertises this route to A, through iBGP, it does not reset the next hop
A need to learn how to reach C through some other method than BGP
An IGP needs to underlie BGP
eBGP
session
A
B
BGP AS
BGP AS
C
Next hop is
set to C
Next hop
remains C
A needs to learn
how to reach C
Peering
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 231
BGP Basics
Routes learned from eBGP peers are readvertised to iBGP peers
Routes learned from iBGP peers are not readvertised to other iBGP peers
iBGP peers have to be fully meshed, or some other technique needs to be used to distribute iBGP routes through an autonomous system
A
B
BGP AS
C
C
eBGP
session
iBGP
session
iBGP
session
Learn eBGP routes
Readvertise
eBGP routes to
iBGP peers
Don‘t readvertise
iBGP routes to
iBGP peers
iBGP session
Peering
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 232
Route Reflectors
B receives 10.1.1.0/24 with an AS Path of {65000,65001}
C receives 10.1.1.0/24 with an AS Path of {65001,65000}
D receives 10.1.1.0/24 with an AS Path of {65001,65000}
B receives the same route with the same attributes, setting up a loop!
eB
GP
AS65001
10.1.1.0/24
10.1.1.0/24
10.1.1.0/24
AS65000A
B
C
D10.1.1.0/24
Basics
We know that iBGP doesn‘t guarantee loop free routing through an AS
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 233
Route Reflectors
What we need is an AS Path to prevent loops within the AS!
RFC2796, BGP Route Reflection, defines two BGP attributes to provide loop detection within an AS
Originator ID
Set to the ID of the router injecting the route into the AS
Cluster List
Each route reflector the route passes through adds their ID to this list
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 234
Route Reflectors
B receives 10.1.1.0/24 with an AS Path of {65000,65001}
C receives 10.1.1.0/24 with an AS Path of {65001,65000}, but adds A‘s Router ID as the Originator ID
C also starts a Cluster List, and adds its own local Router ID into the list
eB
GP 10.1.1.0/24
10.1.1.0/24
AS65000A
B
CD
neighbor <B> route-reflector-client
neighbor <D> route-reflector-client
neighbor <B> route-reflector-client
neighbor <C> route-reflector-client
AS65001
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 235
Route Reflectors
D receives 10.1.1.0/24 with an AS Path of {65001,65000} and an Originator ID of A
D adds its own router ID to the Cluster list
Before sending the route to A, D compares the Originator ID and the Cluster ID list to see if A‘s router ID matches any ID on either one
D finds A‘s ID as the Originator ID, so it doesn‘t send the route to A
eB
GP 10.1.1.0/24
10.1.1.0/24
10.1.1.0/24
10.1.1.0/24
AS65000A
B
C
AS65001
D
neighbor <B> route-reflector-client
neighbor <C> route-reflector-client
neighbor <B> route-reflector-client
neighbor <D> route-reflector-client
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 236
Route Reflectors
eB
GP
AS65000A
B
C
AS65001
D
neighbor <B> route-reflector-client
neighbor <C> route-reflector-client
neighbor <B> route-reflector-client
neighbor <D> route-reflector-client
10.1.1.0/24AS Path: {65001, 65000}
10.1.1.0/24AS Path: {65001, 65000}Originator ID: ACluster List: {C}
10.1.1.0/24AS Path: {65001, 65000}
Originator ID: ACluster List: {C,D}
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 237
Route Reflectors
A route reflector is an iBGP speaker that reflects routes learned from iBGP peers to other iBGP peers
Route reflectors add the Originator ID and the Cluster List to routes they reflect
Route reflectors are designated by configuring some of their iBGP peers as route reflector clients
Route reflectors
neighbor <X> route-reflector-client
neighbor <X> route-reflector-client
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 238
Route Reflectors
A route reflector clientis just an iBGP speaker
There is no special configuration for a route reflector client
Route reflectors
neighbor <X> route-reflector-client
neighbor <X> route-reflector-client
Route reflector client
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 239
Route Reflectors
A cluster is a route reflector and its clients
Route reflector clusters may overlap
Route reflectors
neighbor <X> route-reflector-client
neighbor <X> route-reflector-client
Route reflector client
Cluster
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 240
Route Reflectors
A non-client is any route reflector iBGP peer that is not a route reflector client
Each route reflector is also a non-client of each other route reflector in this network
Route reflectors must be fully iBGP meshed with non-clients
Route reflectors
neighbor <X> route-reflector-client
neighbor <X> route-reflector-client
Route reflector client
Cluster
Non-client
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 241
Route Reflectors
When reflecting a route, a route reflector always:
Creates a Cluster List if one doesn‘t exist
Adds its router ID (or the configured cluster ID) to the Cluster List
If no Cluster List exists, adds the router ID of the peer it received the route from as the Originator ID
When sending a route, a route reflector always follows normal BGP processing rules
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 242
Route Reflectors
Send the route to all clients
Send the route to all non-clients
eBGP peer Non-client
Client
Client
SendSend
Send
Basics
If a route reflector receives a route from an eBGP peer:
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 243
Route Reflectors
Reflect the route to all clients
Reflect the route to all non-clients
Send the route to all eBGP peers
eBGP peer Non-client
Client
Client
Reflect
Reflect
Send
Basics
If a route reflector receives a route from a client:
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 244
Route Reflectors
Reflect the route to all clients
Send the route to all eBGP peers
eBGP peer Non-client
Client
Client
Reflect
Send
Reflect
Basics
If a route reflector receives a route from a non-client:
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 245
Route Reflectors
A advertises 10.1.1.0/24 to B
B sends 10.1.1.0/24 to D
D sends 10.1.1.0/24 to E
E reflects 10.1.1.0/24 to C
D chooses the path through B (via C)
C chooses the path through E (via D)
We have a permanent routing loop!
A
B
D
E
C
eB
GP
B is a client of D
C is a client of E
1
2
3
4
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 246
Route Reflectors
Always configure the reflector topology to follow the physical topology
No route reflector client should ever peer through a route reflector the client isn‘t peered to
C (a client) should not be peered to E (a reflector) through D (a reflector) without being peering to D as well as E
In this case, making C a client of D would resolve the loop
A
B
D
E
C
eB
GP
B is a client of D
C is a client of E
1
2
3
4
Basics
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 247
Route Reflectors
All of the route reflectors will need to be fully meshed
Reflectors still follow the normal rules of iBGP route propagation between themselves
This full iBGP mesh between reflectors can still contain so many routers that it presents a scaling problem
ClusterCluster
Full iBGP mesh between reflectors
Hierarchical Route Reflectors
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 248
Route Reflectors
To resolve this, route reflectors can be deployed in a hierarchy
A single router can be a reflector client and a reflector
ClusterCluster
Cluster
Client and reflector
Hierarchical Route Reflectors
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 249
Hierarchal Route Reflectors
An unlimited number of tiers that can be used
The edges of route reflector tiers are a natural place to reduce the amount of routing information being carried in the lower tiers
The same topology rule applies: The reflector topology must follow the physical topology to prevent loops and black holes
Suboptimal routing can actually be worse, and harder to figure out
Hierarchical Route Reflectors
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 250
Route Reflectors
Use the divide con conquer approach to convert from a full iBGP mesh to route reflectors
Divide network into multiple clusters, using the physical topology as a guide to the logical divisions
Pick out one router to act as the reflector in each cluster, making certain reflection follows the physical topology
Remove redundant iBGP sessions as you configure reflectors in each cluster
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 251
Route Reflectors
If you‘re going to use hierarchal route reflectors, do the outer edge first, leaving the core full mesh iBGP until the outer edge is done
Continue using a single IGP—the next-hop is unmodified by reflectors unless set via an explicit route-map
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 252
Route Reflectors
A client may peer with more than one reflector, in different clusters
A client that peers to only one reflector has a single point of failure
Clients should peer to at least two reflectors to provide redundancy
How many reflectors should a single route reflector be peered to?
Should redundant reflectors be in the same cluster or should they be in separate clusters?
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 253
Route Reflectors
How many route reflectors should a single client be peered to?
Two considerations are important:
Network configuration and management
Router memory and processing requirements
If A is the client of only one reflector, it only receives one copy of the route to 10.1.1.0/24
B
A
E
10.1.1.0/24neighbor <a> route-reflector-client
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 254
Route Reflectors
E new route reflector A becomes a client of adds more configuration and management
Each new route reflector A becomes a client of adds another path to 10.1.1.0/24
This increases the amount of memory A requires to operate, and also increases A‘s processing requirements
B C D
A
E
10.1.1.0/24neighbor <a> route-reflector-client
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 255
Route Reflectors
Each new client B, C, and D are peered to also increased their processing requirements
At some point, the additional reflectors will stop adding to the resilience of the network, and make management and memory requirements similar to a full iBGP mesh
B C D
A
E
10.1.1.0/24neighbor <a> route-reflector-client
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 256
Route Reflectors
Some redundancy is needed
Too much burns memory on RRCs because the client learns the same information from each RR
Also burns memory on the RRs because they learn multiple paths for each route introduced by a RRC
Two or three reflectors peer cluster should be plenty
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 257
Route Reflectors
Assume A and B have the same route reflector clients configured
These two reflectors are redundant
Should they be configured with the same cluster ID or different cluster IDs?
neighbor <c> route-reflector-client
neighbor <d> route-reflector-client
neighbor <c> route-reflector-client
neighbor <d> route-reflector-client
A B
C D
E
10.1.1.0/24
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 258
Route Reflectors
Assume A and B are using the same cluster ID, 10.10.10.10
E advertises 192.168.1.0/24 to D
D sends this route to its reflector, B
B adds a Cluster List and the Originator ID, and reflects the route to A and C
When A receives this route, it notes its local cluster ID is already in the Cluster List (since A and B have the same cluster ID), and rejects the route
A B
C D
E
192.168.1.0/24
192.168.1.0/24
192.1
68.1
.0/2
4
192.168.1.0/24Cluster: 10.10.10.10
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 259
Route Reflectors
If the A to D link fails, A won‘t have any path to 192.168.1.0/24, since it is rejecting the route from B
If the B to C link fails, C won‘t have any path to 192.168.1.0/24, since A is rejecting the route from B, and won‘t reflect it to C
This configuration only protects against some link failures, not all of them
A B
C D
E
192.168.1.0/24
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 260
Route Reflectors
One way to resolve this problem is to configure the iBGP sessions between the router‘s loopbacks, rather than their physical interfaces
If the A to B link fails, the A to B iBGP session stays up (through C), so A maintains connectivity to 192.168.1.0/24
If the B to C link fails, the B to C iBGP session stays up (through A), so C maintains connectivity to 192.168.1.0/24
A B
C D
E
192.168.1.0/24
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 261
Route Reflectors
Another option is to configure A and B with different cluster IDs
Now, when A receives B‘s reflected route, it will keep the route, since the cluster ID in the Cluster List doesn‘t match its own cluster ID
A will run the BGP bestpath algorithm, and advertise either its path through B or its path through D to C
A B
C D
E
192.168.1.0/24
192.168.1.0/24
192.1
68.1
.0/2
4
192.168.1.0/24Cluster: 10.10.10.10
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 262
Route Reflectors
If the A to D link fails, A will still have the path through B to reach 192.168.1.0/24
If the B to C link fails, C will still have the path through A to reach 192.168.1.0/24
This provides full redundancy
A B
C D
E
192.168.1.0/24
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 263
Route Reflectors
A now also has two routes to 192.168.1.0/24, one through D, and one through B
Each additional path A must hold and process adds additional memory and processor overhead
This solution is less scalable than A and B being configured with the same cluster ID
A B
C D
E
192.168.1.0/24
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 264
Route Reflectors
RedundancyAdministrative
FactorsAttribute
Combinations
Reflector Memory
Consumption
Same Cluster ID
100% with sessions between
loopbacks
Easy to identify network
regions based on cluster ID
MediumOne path from
each client
Different Cluster ID 100%
Easy to identify reflection chain
based on Cluster List
High
One path from each client and one path from each reflector
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 265
BGP Cores
When the network becomes ―too large‖ for an interior gateway protocol to manage
When the core of the network becomes an ―internal service provider,‖ connecting several large, independent networks with separate support staffs, policies, and (possibly) interior gateway protocols
Why Use a BGP Core?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 266
BGP Cores
How do you know your network is too big for a single interior gateway protocol domain or instance to handle?
When the network fails on a regular basis
When the network never converges (constant churn)
The upper limit on most interior gateway protocols is about 5,000 to 10,000 routes
The more complex the network is in terms of available alternate paths, the fewer routes the IGP will be able to manage
Why Use a BGP Core?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 267
BGP Cores
But…
If you have deployed the scaling techniques we‘ve talked about, you shouldn‘t hit these limits until the network is truly gigantic!
BGP cores deployed for scaling are generally a sign the network design needs to be rethought
In some cases, however, the network design is just what it is, and we have to do what we can to make it work
Why Use a BGP Core?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 268
BGP Cores
Some networks are not networks, but rather internetworks
An internetwork is made up of multiple smaller networks, each one under separate administrative control
An interior gateway protocol may work as a ―core protocol,‖ as long as the network isn‘t too large, and the administrators all work together well
OSPF Core
Finance
(EIGRP)
HQ (RIP)
Redistribute here
Why Use a BGP Core?
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 269
BGP Cores
Each administration team can better control routing information flow
A major failure in one part of the network is less likely to impact the core or other sections of the network
Less finger pointing means a smoother running, more stable network
OSPF Core
Finance
(EIGRP)
HQ (RIP)
Redistribute here
BGP Core
Why Use a BGP Core?
It‘s better to use a policy based protocol in the core, however:
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 270
BGP Cores
Consider administrative division points
Divide up complex areas of the network as much as possible
Consider physical and topological choke points
Consider places where you could summarize, if at all possible
Complex
topological areas
BGP core
Deployment
Determine where the boundaries of the core should be
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 271
BGP Cores
Don‘t ever redistribute all the routing data from BGP into the IGP at the edge; routes should be injected in a very controlled manner
If possible, inject just the default into the IGP
To provide optimal routing, you can inject summaries into the IGP as well, but this should be limited to one or two routes
BGP core
0.0.0.0/0
10.1.0.0/16
10.2.0.0/16
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 272
BGP Cores
There are several possible ways to manage getting routes into the IGP from the BGP core
The primary factor is in whether the filtering should be done by the administrators of the IGP areas, or the BGP core
Don‘t pass full routes to the IGP area routers unless you want the filtering done by the IGP area administrators
eBGP
session
A
B
BGP core
IGP Area
Redistribute eBGP
learned routes into the
IGP
Generate or permit a default
and other routes towards
the IGP area edge
Pass the entire BGP table to
the IGP area edge
Redistribute filtered
eBGP learned routes
into the IGP
Generate a default
and other routes
into the IGP
Pass no routes to the
IGP area edge
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 273
BGP Cores
If the core doesn‘t have a default, you can generate a default on the edge
router
router bgp <AS number>
neighbor 10.1.1.1 default-information originate
If the core has a default you can pass on through the edge, but you want to make
certain there is always a default route
supplied to the IGP areas
ip route 0.0.0.0 0.0.0.0 null0 200
!
access-list 10 permit host 0.0.0.0
!
route-map 0-only permit 10
match ip address 10
!
router bgp <AS number>
neighbor 10.1.1.1 distribute-list 10 out
redistribute static route-map 0-only
neighbor 10.1.1.1 default-information originate
If the core has a default and you want it to be dynamically provided to the IGP
areas
access-list 10 permit host 0.0.0.0
!
router bgp <AS number>
redistribute eigrp 100 metric 10
neighbor 10.1.1.1 distribute-list 10 out
neighbor 10.1.1.1 default-information originate
A
B
BGP core
IGP Area
Redistribute all eBGP learned routes into the IGP here
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 274
BGP Cores
Pass the more specific into the IGP area into
the core using a
distribute-list to filter out all the other routes
access-list 10 permit host 0.0.0.0
access-list 10 permit host 10.1.0.0
!
router bgp <AS number>
neighbor 10.1.1.1 distribute-list 10 out
Generate it using a summary (but
remember to watch out
for summary black holes)
!
router bgp <AS number>
aggregate-address 10.1.0.0 255.255.0.0 summary-only
A
B
BGP core
IGP Area
Redistribute all eBGP learned routes into the IGP here
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 275
BGP Cores
If the IGP area edge router is receiving full routing information, filtering redistribution into the IGP is required
A
B
BGP core
IGP Area
Full BGP routing
information
access-list 10 permit host 0.0.0.0
access-list 10 permit host 10.1.0.0
!
route-map localin permit 10
match ip address 10
!
router eigrp 100
redistribute bgp <AS number> route-map localin
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 276
BGP Cores
If the core edge router isn‘t providing any routing information to the IGP area edge, a locally generated default can be created
A
B
BGP core
IGP Area
No routing
information
OSPF router ospf 100
default-information originate always
EIGRP ip route 0.0.0.0 0.0.0.0 null0 200
!
router eigrp 100
redistribute static metric ....
IS-IS router isis
default-information originate
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 277
BGP Cores
Filter or summarize from the IGP areas into the core; be careful of routing black holes
Be very careful with complex filtering techniques at the edge; consider maintenance requirements carefully
BGP core
Summarize
and filter
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 278
BGP Cores
Filter the default route and any routes learned from BGP when redistributing into BGP at the IGP area edge
Filtering routing information using a list of specific
prefixes
access-list 10 deny host 0.0.0.0
access-list 10 deny host 10.1.0.0
access-list 10 permit any
!
route-map nolocalout permit 10
match ip address 10
!
router bgp <AS number>
redistribute ospf 100 route-map nolocalout
Tagging routes into the IGP, and filtering on the tags
redistributing from the IGP
access-list 10 deny host 0.0.0.0
!
route-map tagfilter deny 10
match tag 100
match ip address 10
route-map tagfilter permit 20
set tag 100
!
router bgp <AS number>
redistribute OSPF route-map tagfilter metric 10
!
router ospf 100
redistribute bgp <AS number> route-map tagfilter
A
B
BGP core
IGP Area
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 279
BGP Cores
What autonomous system numbers should you use when deploying a BGP core?
It depends on whether or not the BGP core is going to be tied into the network‘s connectivity to the outside networks, including the Internet
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 280
BGP Cores
If the BGP running in the core is not going to touch, in any way, connections to outside networks, use private AS numbers throughout, even for the network core
BGP core
Internet
Partner
DMZ
Routes generated at the
edge, rather than passed
through from the core
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 281
BGP Cores
If routes are passed through the BGP core, a public AS number can be used for the core
The IGP areas can be assigned private AS numbers
Advertisements from the IGP areas can be filtered at the edge towards the outside networks
The routing information can be aggregated at the edge
BGP core
Internet
Partner
Routing
information
passes through
the core
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 282
BGP Cores
If each IGP area is considered a network under separate administrative control, the BGP core can become a ―mini service provider,‖ offering various services to the ―client networks,‖ even though they are all within the same large organization
For instance, one such service would be the provisioning of MPLS VPN tunnels through the core between IGP areas and outside partners, or between IGP areas
BGP core
Internet
Partner
MPLS VPN to a
partner
MPLS VPN
between IGP
areas
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 283
BGP Cores
The BGP core could also provide quality of service forwarding, using QPPB to transport quality of service information to the edges of the core
Communities carried in BGP, along with access lists and AS path lists, can be used to classify packets on the edges of the BGP core
This classification is then used to modify the way packets are forwarded through the network
BGP core
Internet
Partner
Routes from partner
marked for QOS
service level
Packets marked based on BGP
transported QOS information
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 284
BGP Cores
The BGP core could be used as a basis for providing high quality connectivity to the Internet (and partners)
Optimized Exit Routing (OER) can determine the best path to given destinations, and steer traffic along that path
For more information, attend the Optimized Edge Routing (OER) presentation
BGP core
Internet
Internet
OER steers
traffic along the
best exit point
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 285
BGP Cores
MPLS VPNs through a BGP Core
http://www.cisco.com/en/US/tech/tk436/tk428/technologies_configuration_example09186a00800a6c11.shtml
RST-1601, Introduction to MPLS VPNs
RST-2602, Deploying MPLS VPNs
RST-3605, Troubleshooting MPLS VPNs
Quality of Service BGP Propagation
http://www.cisco.com/en/US/partner/products/hw/routers/ps133/products_configuration_guide_chapter09186a008007df4f.html#1015477
Optimized Exit Routing
RST-4311, Advances in Routing Protocols
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 286
BGP Cores
A has two paths to 10.0.0.0/8 with the same metric down to the router ID
It will mark one of them as the best path, and send all traffic along the link to that exit point
iBGP multipath allows A to load share between these two paths
AS65000
AS65001
10.0.0.0/8
A
B C
D E
router-a#sh ip bgp 10.0.0.0
65001
192.168.1.1 from 192.168.1.1
(192.168.1.1)
Origin IGP, metric 0, localpref 100,
valid, internal,
65001
192.168.2.2 from 192.168.2.2
(192.168.2.2)
Origin IGP, metric 0, localpref 100,
valid, internal, best
All traffic
sent through
C to
10.0.0.0/8
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 287
BGP Cores
Flag multiple iBGP paths as ‗multipath‘
Each path must have a unique NEXT_HOP
Number of multipaths configured are inserted in the routing table
maximum-paths ibgp <1–6>
Only the bestpath is advertised to A‘s BGP peers
http://www.cisco.com/en/US/partner/products/sw/iosswrel/ps1839/products_feature_guide09186a0080087b00.html
http://www.cisco.com/cgi-bin/Support/Bugtool/ onebug.pl?bugid=CSCdp72929
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 288
BGP Cores
AS65000
AS65001
10.0.0.0/8
A
B C
D E
router bgp 65000
maximum paths ibgp 2
....
router-a#sh ip bgp 10.0.0.0
65001
192.168.1.1 from 192.168.1.1 (192.168.1.1)
Origin IGP, metric 0, localpref 100, valid,
internal, multipath
65001
192.168.2.2 from 192.168.2.2 (192.168.2.2)
Origin IGP, metric 0, localpref 100, valid,
internal, multipath, best
....
router-s#sh ip route 10.0.0.0
Routing entry for 10.0.0.0/8
* 192.168.1.1 , from 192.168.1.1 , 00:00:09 ago
Route metric is 0, traffic share count is 1
AS Hops 1
192.168.2.2 , from 192.168.2.2 , 00:00:09 ago
Route metric is 0, traffic share count is 1
AS Hops 1
Traffic is
load shared
across
both links
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 289
AS65000
AS65001
A
D
C
AS65002
10.0.0/8
AS65003
B
BGP Cores
If two paths are learned from different autonomous systems, it‘s impossible to load share between them
router-a#sh ip bgp 10.0.0.0
65001 65002
192.168.1.1 from 192.168.1.1 (192.168.1.1)
Origin IGP, metric 0, localpref 100, valid, internal,
65003 65002
192.168.2.2 from 192.168.2.2 (192.168.2.2)
Origin IGP, metric 0, localpref 100, valid, internal, best
Cannot load share
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 290
BGP Cores
Even when two paths are learned from the same AS through eBGP, BGP won‘t load share between them by default
But we could get load sharing by building a single multihop session between B and C
AS65000
AS65001
A
D
C
AS65002
10.0.0/8
AS65003
B
Only one route is
installed in the
routing table
Deployment
router-a#sh ip bgp 10.0.0.0
65001 65002
192.168.1.1 from 192.168.1.1 (192.168.1.1)
Origin IGP, metric 0, localpref 100, valid, internal,
65001 65002
192.168.2.2 from 192.168.2.2 (192.168.2.2)
Origin IGP, metric 0, localpref 100, valid, internal, best
router-s#sh ip route 10.0.0.0
Routing entry for 10.0.0.0/8
* 192.168.1.1 , from 192.168.1.1 , 00:00:09 ago
Route metric is 0, traffic share count is 1
AS Hops 2
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 291
BGP Cores
router bgp 65000
neighbor 192.168.2.1 remote-as 65001
neighbor 192.168.2.1 ebgp-multihop 2
neighbor 192.168.2.1 update-source
192.168.1.1
!
ip route 192.168.2.1 255.255.255.255 10.1.1.1
ip route 192.168.2.1 255.255.255.255 10.1.2.1
router-a#sh ip bgp 10.0.0.0
65001
192.168.2.1 from 192.168.2.1 (192.168.1.1)
Origin IGP, valid, internal, best
router-s#sh ip route 10.0.0.0
Routing entry for 10.0.0.0/8
192.168.2.1, from 192.168.2.1, 00:00:09 ago
Route metric is 0, traffic share count is 1
AS Hops 1
router-a#show ip route 192.168.2.1
Routing entry for 192.168.2.1/32
* 10.1.1.1 from 0.0.0.0, 00:00:00 ago
Route metric is 0, traffic share count is
1
10.1.2.1 from 0.0.0.0, 00:00:00 ago
Route metric is 0, traffic share count is
1
A
B
AS65000
AS65001
192.168.1.1
192.168.2.1
eBGP
10
.1.1
.1
10
.1.2
.1
The eBGP session is set
up as a multihop session
between the loopbacks
There are multiple paths
between the loopbacks
There‘s only one path to
10.0.0.0/8, but there are
multiple paths to the next
hop; A load shares
between the two possible
paths to the next hop
Deployment
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 292
Outside Connections
If the BGP running in the core is not going to touch, in any way, connections to outside networks, use private AS numbers throughout, even for the network core
BGP core
Internet
Partner
DMZ
Routes generated at the
edge, rather than passed
through from the core
Advertising Routes Outside
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 293
Outside Connections
If routes are passed through the BGP core, a public AS number can be used for the core
The IGP areas can be assigned private AS numbers
Advertisements from the IGP areas can be filtered at the edge towards the outside networks
The routing information can be aggregated at the edge
BGP core
Internet
Partner
Routing
information
passes
through the
core
Advertising Routes Outside
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 294
Outside Connections
! permit anything in 10.1.4.0/20 to partner 1
ip prefix-list pl-ptner1 permit 10.1.40.0/20 ge 21
!
! permit anything from private as 65005 to partner 1
ip as-path access-list 100 permit ^.*_65005$
!
! route map putting partner 1’s filters together
route-map rm-ptner1 permit 10
match ip address prefix-list pl-ptner1
route-map rm-ptner1 permit 20
match as-path 100
route-map rm-ptner1 deny 30
!
! other filters as needed for other partners
!
router bgp <public as number>
! aggregate public address space to the internet
aggregate-address 192.168.40.0 255.255.248.0 summary-only
neighbor <internet> remote-as <isp as>
! build peering with partner 1 and put filters on
neighbor <partner1> remote-as <partner as>
neighbor <partner1> route-map rm-ptner1 out
Internet
Partner 1
BGP
Core
Advertising Routes Outside
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 295
Outside Connections
You can also use communities to express filtering from the IGP areas into outside networks
Communities are opaque ―route tags‖ which can carry policy on a per prefix basis in BGP
This could be combined with aggregation, as well, for public address space advertised into the Internet
Internet
Partner
BGP
Core
10.1.1.1
10.2.2.2
Apply communities
marking routes to be
filtered
Filter outbound to
partners based on
communities; aggregate
towards the Internet
Advertising Routes Outside
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 296
route-map to-ptner1 permit 10
match community 10
set community NO_EXPORT
route-map to-ptner1 deny 20
!
route-map to-ptner2 permit 10
match community 20
set community NO_EXPORT
route-map to-ptner2 deny 20
!
router bgp 65000
neighbor <partner 1> route-map to-ptner1 out
neighbor <partner 2> route-map to-ptner2 out
Internet
Partner 1
BGP
Core
! routes to advertise to partner 1
access-list 10 permit 10.2.8.0/24
! routes to advertise to partner 2
access-list 20 permit 10.2.9.0/24
!
route-map tocore permit 10
match ip address 10
set community 1000
route-map tocore permit 20
match ip address 20
set community 2000
!
router bgp 65004
neighbor <bgp core> route-map tocore out
! routes to advertise to partner 1
access-list 10 permit 10.1.1.0/24
!
route-map tocore permit 10
match ip address 10
set community 1000
!
router bgp 65005
neighbor <bgp core> route-map tocore out
Outside ConnectionsAdvertising Routes Outside
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 297
Outside Connections
Make use of the NO_EXPORT community to prevent routes from leaking out of the BGP core
Make use of the NO_EXPORT community to prevent routes from leaking out from partner networks to their peers
In the future, more interesting filtering capabilities will be built on BGP communities
NOPEER community for BGP route scope controlhttp://www.ietf.org/rfc/rfc3765.txt
Controlling the redistribution of BGP routeshttp://www.ietf.org/proceedings/03mar/I-D/draft-ietf-ptomaine-bgp-redistribution-02.txt
Advertising Routes Outside
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 298
Outside Connections
Should you run BGP at all to connect to the Internet?
If you are connecting in a single place, no
Distribute a default into your network, and allow the ISP to originate the routes to your networks at their edge
If you are dual homed to the same ISP in the same physical location, there‘s no reason to run BGP
ISP
Enterprise
192.168.1.0/24
A
B
C
192.168.1.0/24
0.0.0.0/0
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 299
Outside Connections
If you are dual homed to the same SP in two different locations, you may want to accept at least partial routes at both locations, and use the MED to route optimally
If you always want to take the closest exit point out of your network, however, you don‘t need to run BGP
Enterprise
192.168.1.0/24
London Raleigh
ISP A
AS65000
London New York
Optimal
path to
London
Closest
exit path
to
London
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 300
Outside Connections
If you are dual homed to two ISPs, you should run BGP to advertise routing information to both of them
This doesn‘t mean you should accept the full routing table from both service providers, however
You can still originate a local default route into your network, and accept no routes from either SP
0.0.0.0/0
ISP A
AS65000
Enterprise
192.168.1.0/24
AISP B
AS65001
192.168.1.0/24
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 301
Outside Connections
Why would you accept partial routes?
So you can optimally route to destinations connected to one of the ISP‘s you‘re peering to, while allowing traffic to more distant destinations to flow along default routes
Typically, you will accept all of the routes originated by each ISP, and possibly the routes of each of their directly connected customers
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 302
ISP A
AS65000
Enterprise
AS65002
ISP B
AS65001
ip as-path access-list 100 permit ^65000(_[1-9]*)\1*$
ip as-path access-list 110 permit ^65001(_[1-9]*)\1*$
!
router bgp 65002
neighbor <ISP A> remote-as 65000
neighbor <ISP A> filter-list 100 in
neighbor <ISP B> remote-as 65001
neighbor <ISP B> filter-list 110 in
Match any AS path starting
with AS65000
Match any single AS
number repeated any
number of times
Outside ConnectionsInternet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 303
Outside Connections
You can also ask the ISP to filter the routes they are sending at the edge of their network, which reduces the load on your edge router
ISP A
AS65000
Enterprise
AS65002
ISP B
AS65001
Filter for connected
customer and
originated routes
Accept all
advertised
routes
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 304
Outside Connections
You could ask the ISP to configure Outbound Route Filtering, which allows you to configure the filters, but the ISP router actually filters the routes
http://www.cisco.com/en/US/partner/products/sw/iosswrel/ps1839/products_feature_guide09186a0080087c26.html
http://www.ietf.org/internet-drafts/draft-ietf-idr-route-filter-10.txt
This only works for prefix based filters, not for AS Path filters right now
http://www.ietf.org/internet-drafts/draft-ietf-idr-aspath-orf-06.txt
AS Path ORF support is planned in Cisco IOS
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 305
Outside Connections
ISP advertises some route to AS65002, which then readvertises the route to ISP B
ISP B chooses the path through AS65002 as the best path, directing all traffic for that destination through the customer‘s network
The customer network has become a transit
ISP A
AS65000
Enterprise
AS65002
ISP B
AS65001
172.18.1.0/24
Best path for
172.18.1.0/24
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 306
Outside Connections
How can you prevent this from happening?
One common way is to count on lack of synchronization to prevent routes from being readvertised
Don‘t count on synchronization; at some point it will be off by default!
Filtering these routes is simple; a single line AS path access list will do the right thing
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 307
ISP A
AS65000
Enterprise
AS65002
ISP B
AS65001ip as-path access-list 100 permit ^$
!
router bgp 65002
neighbor <ISP A> remote-as 65000
neighbor <ISP A> filter-list 100 out
neighbor <ISP B> remote-as 65001
neighbor <ISP B> filter-list 100 out
Outside ConnectionsInternet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 308
ISP A
AS65000
Enterprise
AS65002
ISP B
AS65001
Outside Connections
You dual home to gain diversity in your routing path:
If a links fails due to backhoe fade, you still have a connection to the outside
If an ISPs fails, you still have a connection to the outside
What if the two physical links run through the same conduit?
What if both ISPs use the same upstream?
ISP C
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 309
Outside Connections
The Problem:
Logical Diversity isn‘t the same as physical diversity
Diversity of any type at one point doesn‘t guarantee diversity throughout; things may recombine at some point
The Solution:
When dual homing, try to dual home from and to physically diverse points
If dual homing from the same physical location, consider using a single provider, and putting physical diversity in the contract
Try to ensure that your providers aren‘t dependant on each other, or on a common point behind them
Internet Connection Considerations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 310
BGP/IGP Interaction
G advertises 10.1.1.0/24 to F through eBGP; F readvertises it to B through iBGP
B checks its local routing table, and finds that G is reachable, so it installs the route, and advertises 10.1.1.0/24 to A through eBGP
A
B
E
F
D
G
10.1.1.0/24
BGP AS
eBGP
eBGP
iBGP
10.1.1.0/24 via B
10.1.1.0/24 via G
G is reachable via D
BGP Synchronization
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 311
BGP/IGP Interaction
A receives a packet for 10.1.1.1, and forwards it to B
B examines its routing table, and finds the next hop is G, a recursive route, and find the next hop of the recursive route is D, so it forwards the packet to D
D, since it‘s not running BGP at all, has no route to 10.1.1.0/24, so it drops the packet!
A
B
E
F
D
G
10.1.1.0/24
BGP AS
eBGP
eBGP
iBGP
10.1.1.0/24 via B
10.1.1.0/24 via G
G is reachable via D
No route to
10.1.1.0/24!
BGP Synchronization
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 312
BGP/IGP Interaction
Synchronization solves this by forcing the IGP and BGP routing tables to match before a route can be advertised to a peer
B would not advertise 10.1.1.0/24 to A if the route isn‘t reachable via some path other than BGP
Unless you want 150,000 routes in your IGP, this isn‘t very useful
A
B
E
F
D
G
10.1.1.0/24
BGP AS
eBGP
eBGP
iBGP
No IGP route to
10.1.1.0/24; don‘t
advertise to
eBGP peers
BGP Synchronization
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 313
BGP/IGP Interaction
The more general solution is to run BGP on D an E, and disable synchronization
This requires running full mesh iBGP on B, D, E, and F, or running route reflectors in the core
A
B
E
F
D
G
10.1.1.0/24
BGP AS
eBGP
eBGP
Full mesh iBGP
BGP Synchronization
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 314
BGP/IGP Interaction
Conveys relative preference of entry points
Lowest MED is best—Default is no MED==0
Comparable only if paths are from same AS
Non-transitive—Do not pass MED from one AS to another
route-map: set metricset metric-type internal
AS 1
AS 6
AS 5
AS 2
AS 3
AS 4
CITY B
CITY A CITY A
BGP/IGP Interaction
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 315
B
AS 1
AS 6
AS 2
AS
A
Configuration:
router bgp 1
neighbor x.x.x.x remote-as 2
neighbor x.x.x.x route-map set_MED out
!
route-map set_MED permit 10
match as-path 2
set metric-type internal
BGP/IGP InteractionSet MED to IGP Metric
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 316
C
10
10
BGP/IGP Interaction
E is learning 10.1.1.0/24 through iBGP from D with a next hop of A
E examines the path to A, and finds an IGP route through D to A; it installs this route in the routing table
C is now inserted into the circuit; after a few seconds, the IGP has converged, and E now chooses C as the best path to A
A
B
D
E
10.1.1.0/24
eBGP
Full iBGP
mesh
20
20
Original
best path
to A
C starts and
provides a
better path to
A
Wait for BGP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 317
C
10
10
BGP/IGP Interaction
However, BGP takes much longer to converge if C is accepting full routes (about 150,000 routes) from A; at least five minutes
When E forwards packets to C for 10.1.1.1, C hasn‘t finished building its BGP tables, so it doesn‘t know how to reach this destination
C drops the packets
A
B
D
E
10.1.1.0/24
eBGP
full iBGP
mesh
20
20C has no path
to 10.1.1.0/24
Wait for BGP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 318
C
10
10
BGP/IGP Interaction
Instead, once the IGP has converged, C signals its IGP neighbors that they should not route this direction
The IGP remains in this state until BGP notifies the IGP it has converged
E will continue using D as its best path to A, even though a better one is available, until BGP converges on C
A
B
D
E
10.1.1.0/24
eBGP
Full iBGP
mesh
20
20
Don‘t use me yet!
Wait for BGP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 319
BGP/IGP Interaction
OSPF uses max-metric router-lsa on-startup wait-for-bgp to configure this feature
Available in 12.2T
http://www.cisco.com/en/US/partner/products/sw/iosswrel/ps1839/products_feature_guide09186a0080087c09.html
IS-IS uses set-overload-bit on-startup wait-for-bgp to configure this feature
Available in 11.3
http://www.cisco.com/en/US/tech/tk365/tk381/technologies_tech_note09186a00800a4bb1.shtml
Wait for BGP
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 320
Summary
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 321
Other References
ASIN: 1578701651 ISBN: 0201657732 ISBN: 1587051877
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 322
Other References
ISBN: 1587050323 ISBN: 1578702208 ISBN: 0201657724
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 323
Other References
ISBN: 0321127005 ISBN: 1587051095 ISBN: 0201379511
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 324
Recommended Reading
Continue your Networkers at Cisco Live learning experience with further reading from Cisco Press
Check the Recommended Reading flyer for suggested books
Available Onsite at the Cisco Company Store
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 325
Q and A
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 326
Complete Your Online Session Evaluation
Win fabulous prizes; give us your feedback
Receive ten Passport Points for each session evaluation you complete
Go to the Internet stations located throughout the Convention Center to complete your session evaluation
Winners will be announced daily at the Internet stations
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential
TECRST-2021
13881_06_2007_c1 327