don’t mind the gap · 2016. 8. 26. · don’t mind the gap: bridging network-wide objectives and...
TRANSCRIPT
Don’t Mind the Gap: Bridging Network-wide Objectives and Device-level Configurations
Ryan Beckett (Princeton, MSR) Ratul Mahajan (MSR) Todd Millstein (UCLA) Jitu Padhye (MSR) David Walker (Princeton)
2
50-80% of outages from configuration changes
-Juniper 2008
~60% of network downtime is caused by human error
-Yankee group 2002
Configuring Networks is Error-Prone
3
YouTube/Pakistan incident: Could something similar whackyour site?
Configuring BGP properly is key to avoidance, 'Net registry officialsays
By Carolyn Duffy Marsan Network World | Mar 10, 2008 1:00 AM PT
In light of Pakistan Telecom/YouTube incident, Internet registry official explains how you can avoid havingyour web site victimized by such an attack.
When Pakistan Telecom blocked YouTube's traffic one Sunday evening in February, the ISP created aninternational incident that wreaked havoc on the popular video site for more than two hours.
RIPE NCC, the European registry for Internet addresses, has conducted an analysis of what happenedduring Pakistan Telecom's hijacking of YouTube's traffic and the steps that YouTube took to stop theattack.
We posed some questions to RIPE NCC's Chief Scientist Daniel Karrenberg about the YouTube incident.Here's what he had to say:
How frequently do hijacking incidents like the Pakistan Telecom/YouTube incident happen?
Misconfigurations of iBGP (internal BGP, the protocol used between the routers in the sameAutonomous System) happen regularly and are usually the result of an error. One suchmisconfiguration caused the Pakistan Telecom/YouTube incident. It appears that the PakistanTelecom/YouTube incident was not an "attack" as some have labeled it, but a configuration error. (SeeColumnist Johna Till Johnson's take on the topic.)
What is significant about the YouTube incident?
Sign In | Register
�
Configuring Networks is Error-Prone
4
Configuring Networks is Error-Prone
YouTube/Pakistan incident: Could something similar whackyour site?
Configuring BGP properly is key to avoidance, 'Net registry officialsays
By Carolyn Duffy Marsan Network World | Mar 10, 2008 1:00 AM PT
In light of Pakistan Telecom/YouTube incident, Internet registry official explains how you can avoid havingyour web site victimized by such an attack.
When Pakistan Telecom blocked YouTube's traffic one Sunday evening in February, the ISP created aninternational incident that wreaked havoc on the popular video site for more than two hours.
RIPE NCC, the European registry for Internet addresses, has conducted an analysis of what happenedduring Pakistan Telecom's hijacking of YouTube's traffic and the steps that YouTube took to stop theattack.
We posed some questions to RIPE NCC's Chief Scientist Daniel Karrenberg about the YouTube incident.Here's what he had to say:
How frequently do hijacking incidents like the Pakistan Telecom/YouTube incident happen?
Misconfigurations of iBGP (internal BGP, the protocol used between the routers in the sameAutonomous System) happen regularly and are usually the result of an error. One suchmisconfiguration caused the Pakistan Telecom/YouTube incident. It appears that the PakistanTelecom/YouTube incident was not an "attack" as some have labeled it, but a configuration error. (SeeColumnist Johna Till Johnson's take on the topic.)
What is significant about the YouTube incident?
Sign In | Register
�
←
5
Configuring Networks is Error-Prone
YouTube/Pakistan incident: Could something similar whackyour site?
Configuring BGP properly is key to avoidance, 'Net registry officialsays
By Carolyn Duffy Marsan Network World | Mar 10, 2008 1:00 AM PT
In light of Pakistan Telecom/YouTube incident, Internet registry official explains how you can avoid havingyour web site victimized by such an attack.
When Pakistan Telecom blocked YouTube's traffic one Sunday evening in February, the ISP created aninternational incident that wreaked havoc on the popular video site for more than two hours.
RIPE NCC, the European registry for Internet addresses, has conducted an analysis of what happenedduring Pakistan Telecom's hijacking of YouTube's traffic and the steps that YouTube took to stop theattack.
We posed some questions to RIPE NCC's Chief Scientist Daniel Karrenberg about the YouTube incident.Here's what he had to say:
How frequently do hijacking incidents like the Pakistan Telecom/YouTube incident happen?
Misconfigurations of iBGP (internal BGP, the protocol used between the routers in the sameAutonomous System) happen regularly and are usually the result of an error. One suchmisconfiguration caused the Pakistan Telecom/YouTube incident. It appears that the PakistanTelecom/YouTube incident was not an "attack" as some have labeled it, but a configuration error. (SeeColumnist Johna Till Johnson's take on the topic.)
What is significant about the YouTube incident?
Sign In | Register
�
←
2/5/2016 China routing snafu briefly mangles interweb • The Register
http://www.theregister.co.uk/2010/04/09/china_bgp_interweb_snafu/ 1/4
More like this
China Network SecurityChina routing snafu briefly mangles interwebCockup, not conspiracy
Bad routing information sourced from China has disrupted the internet for the second time in a fortnight.
Global BGP (Border Gateway Routing) lookup tables sucked in data from a small ISP called IDC ChinaTelecommunication, apparently accidentally broadcast by stateowned carrier ChinaTelecommunications, IDG reports. ISPs including AT&T, France Telcom, Level3, Deutsche Telekom,Qwest and Telefonica accepted illthought out traffic routes as a result of the incident.
BGP is a core routing protocol which maps options for the best available routes for traffic to flow acrossthe net. Several routing options are normally included. The China BGP incident is the internet routingequivalent of TomTom publishing routes via Shanghai for motorists looking for alternative routesbetween London and Paris.
IDC China Telecommunication published illconceived routes for between 32,000 and 37,000 networks about 10 per cent of the net instead of the normal 40 or so routes, and this information was taken asviable routing options by many service providers for about 20 minutes early on Thursday morning (UStime) after China Telecommunications republished it and before the mixup was resolved. Routers inAsia would have been more likely to adopt the false routes as potentially viable, but effects of theincident were recorded all over the world.
BGPmon.net, a BGP monitoring service, has a detailed technical writeup of the snafu, which itdescribed as a prefix hijack, here.
Although it seems they [IDC China Telecommunication] have leaked a whole table, onlyabout 10 per cent of these prefixes propagated outside of the Chinese network. Theseinclude prefixes for popular websites such as dell.com, cnn.com, www.amazon.de,www.rapidshare.com and www.geocities.jp.A large number of networks impacted this morning were actually Chinese networks. Theseinclude some popular Chinese website such as www.joy.cn , www.pconline.com.cn ,www.huanqiu.com, www.tianya.cn and www.chinaz.com
A cockup is suspected, rather than a conspiracy, at least by BGPmon.net.
Given the large number of prefixes and short interval I don’t believe this is an intentionalhijack. Most likely it’s because of configuration issue, i.e. fat fingers. But again, this is justspeculation.
The practical consequences of the screwup are still being assessed but it could have resulted indropped connections or, worse, traffic routed through unknown systems in China. The mess providesone of the clearest illustrations of the security shortcomings of BGP, a somewhat obscure butnonetheless important network protocol.
The China BGP global routing represents a rare but not unprecedented mixup in global internet trafficmanagement. For example, just two weeks ago bad routing data resulted in the redirection of Chileaninternet traffic through a DNS (Domain Name System) server in China, as explained in a detailed postmortem by internet monitoring firm Renesys here. Bad BGP routing information from Pakistan caused
,
Networks Broadband
9 Apr 2010 at 12:24, John LeydenAbout the FT
The surest investmentyou’ll make this year.
Subscribe & save 33%
0:15
The surest investmentyou'll make this yearThe FT's comprehensive coverage ofglobal business provides the insightand analysis you need to stay onestep ahead in 2016 and beyond.
Viewing
Most read
German Chancellor fireshydrogen plasma with thepush of a button
Who would code a selfdestruct feature into theirown web browser? Oh,hello, Apple
Who wants a quadcore4.2GHz, 64GB, 5TB SSD
RAID 10 … laptop?
5 0
DATA CENTER SOFTWARE NETWORKS SECURITY INFRASTRUCTURE DEVOPS BUSINESS HARDWARE SCIENCE BOOTNOTES FORUMS
Log in Sign up Cash'n'Carrion Whitepapers The Channel The Next Platform
6
Configuring Networks is Error-Prone
YouTube/Pakistan incident: Could something similar whackyour site?
Configuring BGP properly is key to avoidance, 'Net registry officialsays
By Carolyn Duffy Marsan Network World | Mar 10, 2008 1:00 AM PT
In light of Pakistan Telecom/YouTube incident, Internet registry official explains how you can avoid havingyour web site victimized by such an attack.
When Pakistan Telecom blocked YouTube's traffic one Sunday evening in February, the ISP created aninternational incident that wreaked havoc on the popular video site for more than two hours.
RIPE NCC, the European registry for Internet addresses, has conducted an analysis of what happenedduring Pakistan Telecom's hijacking of YouTube's traffic and the steps that YouTube took to stop theattack.
We posed some questions to RIPE NCC's Chief Scientist Daniel Karrenberg about the YouTube incident.Here's what he had to say:
How frequently do hijacking incidents like the Pakistan Telecom/YouTube incident happen?
Misconfigurations of iBGP (internal BGP, the protocol used between the routers in the sameAutonomous System) happen regularly and are usually the result of an error. One suchmisconfiguration caused the Pakistan Telecom/YouTube incident. It appears that the PakistanTelecom/YouTube incident was not an "attack" as some have labeled it, but a configuration error. (SeeColumnist Johna Till Johnson's take on the topic.)
What is significant about the YouTube incident?
Sign In | Register
�
←
2/5/2016 China routing snafu briefly mangles interweb • The Register
http://www.theregister.co.uk/2010/04/09/china_bgp_interweb_snafu/ 1/4
More like this
China Network SecurityChina routing snafu briefly mangles interwebCockup, not conspiracy
Bad routing information sourced from China has disrupted the internet for the second time in a fortnight.
Global BGP (Border Gateway Routing) lookup tables sucked in data from a small ISP called IDC ChinaTelecommunication, apparently accidentally broadcast by stateowned carrier ChinaTelecommunications, IDG reports. ISPs including AT&T, France Telcom, Level3, Deutsche Telekom,Qwest and Telefonica accepted illthought out traffic routes as a result of the incident.
BGP is a core routing protocol which maps options for the best available routes for traffic to flow acrossthe net. Several routing options are normally included. The China BGP incident is the internet routingequivalent of TomTom publishing routes via Shanghai for motorists looking for alternative routesbetween London and Paris.
IDC China Telecommunication published illconceived routes for between 32,000 and 37,000 networks about 10 per cent of the net instead of the normal 40 or so routes, and this information was taken asviable routing options by many service providers for about 20 minutes early on Thursday morning (UStime) after China Telecommunications republished it and before the mixup was resolved. Routers inAsia would have been more likely to adopt the false routes as potentially viable, but effects of theincident were recorded all over the world.
BGPmon.net, a BGP monitoring service, has a detailed technical writeup of the snafu, which itdescribed as a prefix hijack, here.
Although it seems they [IDC China Telecommunication] have leaked a whole table, onlyabout 10 per cent of these prefixes propagated outside of the Chinese network. Theseinclude prefixes for popular websites such as dell.com, cnn.com, www.amazon.de,www.rapidshare.com and www.geocities.jp.A large number of networks impacted this morning were actually Chinese networks. Theseinclude some popular Chinese website such as www.joy.cn , www.pconline.com.cn ,www.huanqiu.com, www.tianya.cn and www.chinaz.com
A cockup is suspected, rather than a conspiracy, at least by BGPmon.net.
Given the large number of prefixes and short interval I don’t believe this is an intentionalhijack. Most likely it’s because of configuration issue, i.e. fat fingers. But again, this is justspeculation.
The practical consequences of the screwup are still being assessed but it could have resulted indropped connections or, worse, traffic routed through unknown systems in China. The mess providesone of the clearest illustrations of the security shortcomings of BGP, a somewhat obscure butnonetheless important network protocol.
The China BGP global routing represents a rare but not unprecedented mixup in global internet trafficmanagement. For example, just two weeks ago bad routing data resulted in the redirection of Chileaninternet traffic through a DNS (Domain Name System) server in China, as explained in a detailed postmortem by internet monitoring firm Renesys here. Bad BGP routing information from Pakistan caused
,
Networks Broadband
9 Apr 2010 at 12:24, John LeydenAbout the FT
The surest investmentyou’ll make this year.
Subscribe & save 33%
0:15
The surest investmentyou'll make this yearThe FT's comprehensive coverage ofglobal business provides the insightand analysis you need to stay onestep ahead in 2016 and beyond.
Viewing
Most read
German Chancellor fireshydrogen plasma with thepush of a button
Who would code a selfdestruct feature into theirown web browser? Oh,hello, Apple
Who wants a quadcore4.2GHz, 64GB, 5TB SSD
RAID 10 … laptop?
5 0
DATA CENTER SOFTWARE NETWORKS SECURITY INFRASTRUCTURE DEVOPS BUSINESS HARDWARE SCIENCE BOOTNOTES FORUMS
Log in Sign up Cash'n'Carrion Whitepapers The Channel The Next Platform
2/5/2016 Internet-Wide Catastrophe—Last Year - Dyn Research | The New Home Of Renesys
http://research.dyn.com/2005/12/internetwide-nearcatastrophela/ 1/5
± ¬ Search �
2 DECEMBER 24, 2005 ç COMMENTS (0) - VIEWS: 3038
1 ENGINEERING TODD UNDERWOOD
Internet‐WideCatastrophe—Last Year
One year ago today TTNet in Turkey (AS9121) pretended to be the entireInternet. And unfortunately for the rest of the Internet, many largenetwork providers believed them (or at least believed them in part). Asfar as anyone knows, it was a mistake, not a malicious act. But theconsequences were far from benign: for several hours a large number ofInternet users were unable to reach a large number of Internet sites.Twelve months later we can take a look at what happened, and whetherwe’ve learned much in the intervening time.
Early Christmas Eve morning 2004, TTNet (AS9121) started announcing
The NewThreat:Targeted
Internet Traffic Misdirection
NOVEMBER 19, 2013
EgyptLeaves theInternet
JANUARY 27, 2011
InternetTouchesHalf MillionRoutes:Outages
Possible Next Week
AUGUST 13, 2014
Authors ArchivesPopular
HOME TOPICS PRESENTATIONS ABOUT OUTAGES DYNCONTENTHUB
« Previous Story Next Story »
Fundamental Tradeoff?
7
Distributed Centralized
Distributed
Centralized
ControlMechanism
Configuration
Fundamental Tradeoff?
8
Distributed Centralized
Distributed
Centralized
ControlMechanism
OSPF
RIP
BGP
ScalabilityRobustnessComplexity
Configuration
Fundamental Tradeoff?
9
Distributed Centralized
Distributed
Centralized
ControlMechanism
OSPF
RIP
BGP
ScalabilityRobustnessComplexity
Configuration
100,000s oflines of config
Fundamental Tradeoff?
10
Distributed Centralized
Distributed
Centralized
ControlMechanism
OSPF
RIP
BGP
SDNScalabilityRobustnessComplexity
ScalabilityRobustnessComplexity
Configuration
Fundamental Tradeoff?
11
Ideal
Distributed Centralized
Distributed
Centralized
ControlMechanism
OSPF
RIP
BGP
SDNScalabilityRobustnessComplexity
ScalabilityRobustnessComplexity
Configuration
Propane Overview
12
Propane
Topology
BGP ConfigsCompiler
0,0
2,1
1,1 0,1
Propane System
13
AS 1AS 2
AS 3
Uniform abstractions for intra- and inter-domain routing
I) Language for expressing network-wide objectives with:
Path constraints and preferences in case of failures
14
Propane System
2) Compiler for a purely distributed implementation
Policy
15
Propane System
Generate BGP configs for each router
Compiler guarantees policy-compliance for all failures
2) Compiler for a purely distributed implementation
BGP
BGP
BGP
BGP
Example: A DC network with traditional configs
16
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
Goals• Local prefixes reachable only internally
• Global prefixes reachable externally
• Aggregate global prefixes as GP
• Prefer leaving through Peer1 over Peer2
• Prevent transit traffic between peers
Configuration Attempt
• Don’t export from G, H to external
• Aggregate externally as GP
Goals• Local prefixes reachable only internally
• Global prefixes reachable externally
• Aggregate global prefixes as GP
• Prefer leaving through Peer1 over Peer2
• Prevent transit traffic between peers
Example: A DC network with traditional configs
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
17
GP
18
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
GP
Configuration Attempt
• Don’t export from G, H to external
• Aggregate externally as GP
Example: A DC network with traditional configs
Goals• Local prefixes reachable only internally
• Global prefixes reachable externally
• Aggregate global prefixes as GP
• Prefer leaving through Peer1 over Peer2
• Prevent transit traffic between peers
19
Configuration Attempt
• Don’t export from G, H to external
• Aggregate externally as GP
• X,Y block routes through each other
Example: A DC network with traditional configs
Goals• Local prefixes reachable only internally
• Global prefixes reachable externally
• Aggregate global prefixes as GP
• Prefer leaving through Peer1 over Peer2
• Prevent transit traffic between peers
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
20
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
GP GP
Configuration Attempt
• Don’t export from G, H to external
• Aggregate externally as GP
• X,Y block routes through each other
Example: A DC network with traditional configs
Goals• Local prefixes reachable only internally
• Global prefixes reachable externally
• Aggregate global prefixes as GP
• Prefer leaving through Peer1 over Peer2
• Prevent transit traffic between peers
21
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
Goals• Local prefixes reachable only internally
• Global prefixes reachable externally
• Aggregate global prefixes as GP
• Prefer leaving through Peer1 over Peer2
• Prevent transit traffic between peers
Example: A DC network with traditional configs
GP1 traffic dropped GP
Configuration Attempt
• Don’t export from G, H to external
• Aggregate externally as GP
• X,Y block routes through each other
22Aggregation-Induced Black Hole!
Example: A DC network with traditional configs
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
GP
Configuration Attempt
• Don’t export from G, H to external
• Aggregate externally as GP
• X,Y block routes through each other
Goals• Local prefixes reachable only internally
• Global prefixes reachable externally
• Aggregate global prefixes as GP
• Prefer leaving through Peer1 over Peer2
• Prevent transit traffic between peers
GP1 traffic dropped
23
Goals• Local prefixes reachable only internally
• Global prefixes reachable externally
• Aggregate global prefixes as GP
• Prefer leaving through Peer1 over Peer2
• Prevent transit traffic between peers
Example: A DC network with traditional configs
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
24
Example: A DC network with Propane
define Destination = {GP1 => end(A) GP2 => end(B) LP1 => end(E) LP2 => end(F) true => exit(Peer1 >> Peer2)}
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
25
define Destination = {GP1 => end(A) GP2 => end(B) LP1 => end(E) LP2 => end(F) true => exit(Peer1 >> Peer2)}
define Locality = {LP1 | LP2 => internal}
Example: A DC network with Propane
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
26
define Destination = {GP1 => end(A) GP2 => end(B) LP1 => end(E) LP2 => end(F) true => exit(Peer1 >> Peer2)}
define Locality = {LP1 | LP2 => internal}
define transit(X,Y) = enter(X|Y) and exit(X|Y)
Example: A DC network with Propane
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
27
define Destination = {GP1 => end(A) GP2 => end(B) LP1 => end(E) LP2 => end(F) true => exit(Peer1 >> Peer2)}
define Locality = {LP1 | LP2 => internal}
define transit(X,Y) = enter(X|Y) and exit(X|Y)
define NoTransit = {true => !transit(Peer1,Peer2)}
Example: A DC network with Propane
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
28
define Destination = {GP1 => end(A) GP2 => end(B) LP1 => end(E) LP2 => end(F) true => exit(Peer1 >> Peer2)}
define Locality = {LP1 | LP2 => internal}
define transit(X,Y) = enter(X|Y) and exit(X|Y)
define NoTransit = {true => !transit(Peer1,Peer2)}
define Main = Destination & Locality & NoTransit & agg(GP, in -> out)
Example: A DC network with Propane
GlobalPrefixes
LocalPrefixes
GP1 GP2 LP1 LP2
Peer2
YX
DC
A B E F
HG
Peer1
Y
C D
A B
G H
E F
Compilation
Propane Compiler
Topology
BGP Configs
29
Compilation
Propane Compiler
Topology
BGP Configs
30
0 1 2
0 1
State Machines
Constraints on Policy
Compilation
Propane Compiler
Topology
BGP Configs
31
0,0
2,1
1,1 0,10 1 2
0 1
Product GraphState Machines
Jointly analyzewith topology
Compilation: A simple Example
32
Y
A
C
D
EB
X
end(Y) & (path(A,C,D) >> any)
Compilation: A simple Example
33
Y
A
C
D
EB
X
Convert to Regex
XACDY >> (Σ*)Y
end(Y) & (path(A,C,D) >> any)
Reversed Automata from Policies
34
Policy:
1. XACDY
2. (Σ*)Y
Y
A
C
D
EB
X
More preferred paths
Less preferred paths
Reversed Automata from Policies
35
0 1 2 3 4 5Y D C A X
0Y
Y
A
C
D
EB
X
Σ
1
Policy:
1. XACDY
2. (Σ*)Y
Reversed Automata from Policies
36
0 1 2 3 4 5Y D C A X
Reversed automatatracks BGP message flow
Y
A
C
D
EB
X
0Y
Σ
1
Policy:
1. XACDY
2. (Σ*)Y
Constructing the Product Graph (PG)
37
0 1 2 3 4 5Y D C A X
0Y
Σ
1
{1,2}
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2}
(C, -,1)
(A, -,1)
YA
C
D
EB
X
Constructing the Product Graph (PG)
38
0 1 2 3 4 5Y D C A X
0Y
Σ
1
(X,5,1)
TopologyLocation
AutomataStatesY
A
C
D
EB
X
Constructing the Product Graph (PG)
39
0 1 2 3 4 5Y D C A X
0Y
Σ
1
(X,5,1)
TopologyLocation
AutomataStatesY
A
C
D
EB
X
{1,2}
Path preferences
Constructing the Product Graph (PG)
40
0 1 2 3 4 5Y D C A X
0Y
Σ
1
(Y,1,1)
(-,0,0)
YA
C
D
EB
X
Constructing the Product Graph (PG)
41
0 1 2 3 4 5Y D C A X
0Y
Σ
1
(D,2,1)
(Y,1,1)
(-,0,0)
YA
C
D
EB
X
Constructing the Product Graph (PG)
42
{1,2}
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2}
(C, -,1)
(A, -,1)
Graph capturing all possible policy-compliant paths through the topology
YA
C
D
EB
X
Constructing the Product Graph (PG)
43
0 1 2 3 4 5Y D C A X
0Y
Σ
1
{1,2}
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2}
(C, -,1)
(A, -,1)
Preferences
YA
C
D
EB
X
Constructing the Product Graph (PG)
44
0 1 2 3 4 5Y D C A X
0Y
Σ
1
{1,2}
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2}
(C, -,1)
(A, -,1)
Preferences
YA
C
D
EB
X
Constructing the Product Graph (PG)
45
0 1 2 3 4 5Y D C A X
0Y
Σ
1
{1,2}
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2}
(C, -,1)
(A, -,1)
Y D C A XAccept: Preferences
YA
C
D
EB
X
Constructing the Product Graph (PG)
46
0 1 2 3 4 5Y D C A X
0Y
Σ
1
{1,2}
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2}
(C, -,1)
(A, -,1)
YA
C
D
EB
X
Compilation to BGP:
47
Idea 1: Restrict advertisements to edges• Encode state in a BGP community tag• Incoming edges — import filters• Outgoing edges — export filters
Let BGP find some allowed path dynamically
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
Compilation to BGP:
48
D allows import matching regex(Y)
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
YA
C
D
EB
X
Compilation to BGP:
49
D exports to C with tag (2,1)
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
YA
C
D
EB
X
Compilation to BGP:
50
C allows import from D with tag (2,1)
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
YA
C
D
EB
X
Compilation to BGP:
51
C exports to A,B,D,E with tag (3,1)
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
(-,0,0)
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
YA
C
D
EB
X
Compilation to BGP:
52
Idea 2: Find preferences
Let BGP find the best path dynamically
• Direct BGP towards best path• Under all combinations of failures
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
start
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
Compilation to BGP:
53
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
start
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
Router Cmatch peer = D …
match peer = E …
Compilation to BGP:
54
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
start
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
??????
Router Cmatch peer = D …
match peer = E …local-pref ← ???
local-pref ← ???
Compilation to BGP:
55
(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
start
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
Efficient algorithm to assign preferences that forces BGP to find the best paths for all possible failures
See the paper for details!
Router Cmatch peer = D …
match peer = E …local-pref ← ???
local-pref ← ???
Compilation to BGP:
56
Less preferredimport
More preferredimport
Implementation: Local preference(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
start
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
Compilation to BGP:
57
Less preferredimport
More preferredimport
Implementation: MED/Prepending(D,2,1)
(C,3,1)
(A,4,1)
(X, -, 1) (X,5,1)
(Y,1,1)
start
(D, -,1)
(E, -,1)
(B, -,1)
{2} {1,2}
(C, -,1)
(A, -,1)
Compilation: A simple Example
58
end(Y) & (path(A,C,D) >> any)
AS 100
Y
A
C
D
EB
X
lp←100
lp←101
MED←80
MED←81
filter(Y)
filter(Y)
Implementation
Written in 7000 lines of F#
Generates Quagga configurations
A number of other analyses & features
59
Benchmarks
60
Configurations from a large cloud provider
Policy described in English documents
Datacenter and Backbone policies
Policy Size
61
Without prefix/peer definitions
Datacenter policy: 31 lines of Propane
Backbone policy: 43 lines of Propane
Conventional BGP configurations are 1000s of lines
Compilation Time
62
Data center (< 9 min) Backbone (< 3 min)
Compile for each prefix equivalence class
Compile for each equivalence class in parallel
8 core, 3.6 GHz Intel Xeon processor
Configuration Size
63
Reuse community values across peers
Merge import/export behaviors across peers
Optimizations yield 50-100x decrease in config size
Avoid using community tags when unambiguous
Optimizations
Results
Configurations ~1000-10000 lines per router
64
Propane: Summary
Centralized network programmability
Distributed implementation via BGP
Uniform abstractions for Inter- and Intra-domain routing
High-level language
Compiler
Static analysis guarantees policy compliance for all failures
Scales to reasonably sized network topologies
Core policy in 30-50 lines of Propane vs. 1000s
Constraints specify preferred paths and backups in case of failure