intra-domain te via igp metric tuning who i am andrew lange exodus, a cable & wireless service...
TRANSCRIPT
![Page 1: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/1.jpg)
![Page 2: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/2.jpg)
Intra-Domain TE via IGP Metric Tuning
![Page 3: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/3.jpg)
Who I Am
Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect [email protected]/
[email protected] Successfully navigated the
straights of Chapter 11, between Scylla & Charybdis...and somehow ended up in Britain.
![Page 4: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/4.jpg)
What this IS
This IS an introduction to the wonderful world of using flow information to tune your IGP metrics.
![Page 5: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/5.jpg)
What this is NOT
This is NOT an end-all be-all guide to how to optimize your IGP metrics.
![Page 6: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/6.jpg)
Problem We're Trying to Solve How can we maximally* utilize our
network infrastructure, without adding the complications associated with MPLS?
Can this even be done? Well, of course it can, or this
presentation would be remarkably short.
Why? The more we can get out of our
network the more cost effective it becomes, and the happier the finance people get. Plus, it's cool.
![Page 7: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/7.jpg)
*Maximally
Maximally is a thorny term. Long story short: Optimum network flows can be
represented as shortest paths with respect to a set of positive link weights (Wang, Wang & Zhang).
With current IGP's, determining optimum is NP-Hard, BUT, very close (within 3%) approximations can be made (Fortz & Thorup).
![Page 8: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/8.jpg)
Scholarship Abounds
The first concrete way of doing this that we ran across was in Fortz & Thorup's paper Internet Traffic Engineering by Optimizing OSPF Weights.
This literature tends to be quite recent (1999 and newer).
How easy it is to determine the optimal values very much depends on what your network and flows look like.
![Page 9: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/9.jpg)
What is Required
Accurate flow data between each set of backbone nodes.
An optimization routine to apply to the flow data.
![Page 10: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/10.jpg)
Getting the Flow Data
![Page 11: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/11.jpg)
Getting the Data - Tool Selection Using Netflow is the only way to
gather a traffic matrix without using an overlay design.
Looked at a variety of options, including building our own, and settled on the Ixia (nee Caimis) product.
![Page 12: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/12.jpg)
Getting the Data - Our Issues Vendor HW/SW combinations are not
always supported with the netflow feature set.
Full deployment of the tools is pending operational deployment of the right code base.
Sampling rate needs to be grossed up.
![Page 13: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/13.jpg)
Getting the Data - Configuration How have we configured our
collectors? Data is collected on the interfaces
inbound into the backbone routers from the datacenter.
Flow data is sampled at 1:100. Collectors peer with the backbone
routers as route-reflector clients. Collectors gather, among other
things, BGP NextHop information.
![Page 14: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/14.jpg)
Node Overview
BBR
BACKBONE
PeersBBR
IBRTransit Fabric
DCRDCR
Customer Service Routers
![Page 15: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/15.jpg)
Munging the Data- Basics
How do we process the collected data? Data is summarized daily. To assemble the flow matrix, data
is aggregated across the interfaces and the routers for a given site. There are some problems with this.
![Page 16: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/16.jpg)
Munging the Data - Problems Data is an average, peak utilization is
not available. This is probably ok for this application, since average and peak tend to follow the same proportions. But we're working on getting peak to compare the results using that data.
Assumes both routers function as one (Nodal Aggregation). This is useful to simplify things as we first work out the models, but we will need to get more detailed as the models are refined.
![Page 17: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/17.jpg)
Munging the Data - Aggregating the Flows Aggregated daily summaries are post
processed with a script that correlates BGP NextHop with destination datacenter and combines the flows destined for that datacenter.
Currently does not gross up flow size to compensate for sampling.
![Page 18: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/18.jpg)
Flow Matrix - Example (sntc08)IDC Bits/secondSNTC04 21,696WHKN01 31,859AUST01 2,295SNTC01 257,722ELSG01 31,298SNTC03 21,492SNTC05 164,455Etc...
![Page 19: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/19.jpg)
Building a Model
![Page 20: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/20.jpg)
Offline vs. Online
We have chosen to pursue offline metric optimization. Online, or dynamic, metric
optimization imposes a whole other set of requirements, such as speed of the optimization model, and lot of trust.
At least at this point, our target for intra-domain TE is in the medium/long term timeframe. If we are running our network so hot that we have to reoptimize multiple times a day, then we need more bandwidth.
![Page 21: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/21.jpg)
Modeling Assumptions
Model assumes that when flows grow the proportions remain the same.
Model does not take flow splitting (ECMP) into account currently. Except ECMP between two adjacent nodes, which is represented by increasing the size of the link between them in the model.
![Page 22: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/22.jpg)
What follows is an Example 10 nodes, 15 links (30 arcs). 10 demand sets. Real backbone network would be
more complicated, but findings still hold.
![Page 23: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/23.jpg)
Example Data
Because we were not able to poll the full matrix of data from the network, the data we're using for this example is extrapolated from the flow data we do have. It is only approximate.
![Page 24: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/24.jpg)
Example Network Diagram
SEA
LAX
SVA
CHI
DLS
ATL
BOS
WDC
NYC
MIA
![Page 25: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/25.jpg)
Example Network Info
All links are OC48. There are no nodal constraints (i.e.
Routers are assumed to be able to push line rate.)
![Page 26: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/26.jpg)
Base Demands
Flow Amount (Mbps)sva- - >sea 57.63sva- - >nyc 595.51sva- - >wdc 461.04nyc- - >sva 479.78nyc- - >wdc 500.64wdc- - >sva 285.2wdc- - >nyc 384.4sea- - >mia 2.54sea- - >sva 58.42mia- - >sea 2.19
![Page 27: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/27.jpg)
Current Metrics
Current Metrics are agnostic to flow information (based on RTT between nodes).
Under current loads the example network is nicely overprovisioned.
We're going to focus on how much more load we can put on this network before any link exceeds 80% utilization (to allow for microbursts, etc.) This is 1990 Mbps for an OC48. We are going to do this by increasing the values of the Base Demands.
![Page 28: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/28.jpg)
Shortest Paths - Current MetricsFlow Pathmia- - >sea mia- - >atl- - >dls- - >chi- - >seanyc- - >sva nyc- - >wdc- - >svanyc- - >wdc nyc- - >wdcsea- - >mia sea- - >chi- - >dls- - >atl- - >miasea- - >sva sea- - >svasva- - >nyc sva- - >wdc- - >nycsva- - >sea sva- - >seasva- - >wdc sva- - >wdcwdc- - >nyc wdc- - >nycwdc- - >sva wdc- - >sva
![Page 29: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/29.jpg)
Link Loading - Current MetricsLink Current Loadatl- - >dls 2.19 4.18
atl- - >mia 2.54 4.78
chi- - >dls 2.54 4.78chi- - >sea 2.19 4.18
dls- - >atl 2.54 4.78dls- - >chi 2.19 4.18
mia- - >atl 2.19 4.18nyc- - >wdc 980.42 1843.19
sea- - >chi 2.54 4.78sea- - >sva 58.42 109.83
sva- - >sea 57.63 108.34sva- - >wdc 1056.55 1986.32
wdc- - >nyc 979.91 1842.23wdc- - >sva 764.98 1438.17
Max Load (+88%)
![Page 30: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/30.jpg)
Abracadabra!
Sample run of one of the models: ampl: model fixtwo-int.mod; data
cap-3.3.dat; solve; CPLEX 7.1.0: optimal integer
solution; objective 12909.92 31 MIP simplex iterations 0 branch-and-bound nodes
![Page 31: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/31.jpg)
A Bit Behind the Curtain
Using AMPL/CPLEX to define the models. This consists of a model file specifying
the model's: Objective (e.g. Minimize overall network load).
Constraints (e.g. Do not exceed capacity on links.)
And a data file, which specifies: What the network looks like. What the demands are.
![Page 32: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/32.jpg)
Shortest Paths - New MetricsFlow Pathmia- - >sea mia- - >nyc- - >chi- - >seanyc- - >sva nyc- - >chi- - >svanyc- - >wdc nyc- - >wdcsea- - >mia sea- - >chi- - >nyc- - >miasea- - >sva sea- - >svasva- - >nyc sva- - >chi- - >nycsva- - >sea sva- - >seasva- - >wdc sva- - >wdcwdc- - >nyc wdc- - >nycwdc- - >sva wdc- - >sva
![Page 33: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/33.jpg)
Link Loading - New Metrics
Link Current Loadchi- - >nyc 598.05 1973.56
chi- - >sea 2.19 7.23
chi- - >sva 479.78 1583.27mia- - >nyc 2.19 7.23
nyc- - >chi 481.97 1590.5nyc- - >mia 2.54 8.38
nyc- - >wdc 500.64 1652.11sea- - >chi 2.54 8.38
sea- - >sva 58.42 192.79sva- - >chi 595.51 1965.18
sva- - >sea 57.63 190.18sva- - >wdc 461.04 1521.43
wdc- - >nyc 384.4 1268.52wdc- - >sva 285.2 941.16
Max Load (+330%)
![Page 34: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/34.jpg)
Resources and Thanks
![Page 35: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/35.jpg)
Optimization Resources - Papers Sample Papers:
Internet Traffic Engineering by Optimizing OSPF Weights, Fortz & Thorup.
Internet Traffic Engineering without Full Mesh Overlaying, Wang, Wang & Zhang.
Dynamic Optimization of OSPF Weights using Online Simulation, Ye, et. al.
![Page 36: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/36.jpg)
Optimization Resources - Tools Mathematical Programming Tools
AMPL/CPLEX (www.ampl.com) OPL/CPLEX
(http://www.ilog.com/products/oplstudio/)
![Page 37: Intra-Domain TE via IGP Metric Tuning Who I Am Andrew Lange Exodus, a Cable & Wireless Service Principal Network Architect andrewl@exodus.net/andrewl@cw.net](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e395503460f94b2aa7f/html5/thumbnails/37.jpg)
Special Thanks To
Dr. Irv Lustig, for invaluable help with the modeling languages.