Fast Control Plane Analysis

Using an Abstract Representation

Aditya Akella

Aaron Gember-Jacobson, Raajay Viswanathan and Ratul Mahajan

UW-Madison and Microsoft

1

Controlplane

Control plane is…

• Essential

• Complex

2

RoutingprocessRoutingprocess

Dataplane

Forwarding tableTo: A

Routingprocess

→ configuration errors may cause security/availability problems

→ errors may not be immediately apparent

Routingtable

Routingtable

Routingtable

Always traverse middlebox

Important functional invariants

3

Always blocked Always isolated

Always equivalent paths

Challenge: Invariants violated under some (combinations of) failures

Analyze current data plane [HSA,

Veriflow]

4

Forwarding Table’

Forwarding Table’

Forwarding Table’

Forwarding Table’’

Forwarding Table’’

Forwarding Table’’

Forwarding Table’’’

Forwarding Table’’’

Forwarding Table’’’

Generate data planes [Batfish]

→ time consuming→ cannot verify invariants always hold

Blocked, isolated, waypoints, equivalence …

• Properties of paths, not paths themselves

• Data centers, enterprises use a limitedset of control plane constructs

Higher-level abstractionFast analysis

Proactive Verification

Abstract Representation forControl planes (ARC)

5

• Encodes the network’s forwarding behavior under all possible infrastructure faults

Control plane configuration Abstract representation

B I C O

B O

D I

D O3C I

0

3

0 0

Dst:TSrc:U

0 0

Dst:U Src:T

0 0

1

1

1 1

C O D I

D OC I

0 0

3

3

…

• Encodes the network’s forwarding behavior under all possible infrastructure faults

• Proactive verification boils down to checking simple graph-level properties fast

• Ignore which protocols used and how

B C

OSPFTD

U

1

31

T

T

T

Key requirements of ARC1) Sound & Complete: each digraph contains

every feasible path and no infeasible paths verification of invariants

2) Precise: assign edge weights such that the min-costpath matches the real path counter-examples, equivalence testing

6

B I C O

B O

D I

D OC I

Dst:TSrc:U

Dst:U Src:T

0 0

3

0

3

0 0

1

1

1 1

C O D I

D OC I

0 0

0 0

3

3

• Why weighted digraphs?

• How to ensure soundness, completeness, precision?

7

Routing protocols used today

• Commonality: cost-based path selection algorithm

• Differences:

• Also must account for:

– Traffic class specificity

– Route redistribution

– Route selection based on administrative distance

8

BGP AS1 AS2OSPF Router1 Router2

4 IVDijsktra’s algorithm Path length

& preferenceAD=110 AD=20

Challenge: determining the structure and edge weights of the graphs

granularity & currency

Extended topology graphs (ETGs)• One per traffic class

• Vertices: routing processes

• Edges: flow of data enabled by exchange of routing information

9

Z

B

X

BGP1

OSPF3

T

S

Y

1

2

A

3

SRC:S

DST:T

A.1I

A.1O B.1I

B.1O Z.1I

Z.1O

Z.3I

Z.3O

Y.3O

Y.3I

X.3I

X.3O

Edge-weights based on configured costs and

administrative distances1

1

1

1

0 0 0

000

0.4

0.4

0.6

0.6

Sound and complete(for OSPF, BGP, redistr…)

ETG edge weights• Inter-device: OSPF weights;

unit cost per hop for BGP (each router is an AS)

• Intra-device: redistribution only: no cost within process; fixed-cost between processes

10

Z

B

X

BGP1

OSPF2

T

S

Y

1

2

A

3

A.1I

A.1O B.1I

B.1O Z.1I

Z.1O

Z.3I

Z.3O

Y.3O

Y.3I

X.3I

X.3O

2

2 3

3

1

1

1

1

0 0 0

000

+ scaling

1

0.2

0.2

0.3

0.3

Shortestpath = 5

Gap = 1

2

Shortestpath = 1

Longestpath = 0.5

SRC:S

DST:T

≤

Precise(for DAG

redistribution, AD graphs)

ARC properties

Construct Sound & Complete

Precise

OSPF Single area

RIP

eBGP Select by AS path length, local pref.

Static Routes

ACLs

Route filters

Route selection (based on Administrative Distance)

No redistribution OR redistribution costs congruent with ADs

Route redistribution Acyclic & costs congruent with ADs

11

Sound and complete for 100% Precise for 96%

Verification

• Always traverse middlebox:1) remove all edges with middleboxes2) Src and Dst in same connected component?

12

DO

DI

EO

EI

FO

FI

GO

GI

CO

CI

BO

BIDST:S

SRC:U

E

F G

CB

D US

OSPF

Verification

• Always reachable with < k link failures:max-flow on unit-weight ETG ≥ k?

13

Max-flow = 3

DO

DI

EO

EI

FO

FI

GO

GI

CO

CI

BO

BIDST:S

SRC:U

E

F G

CB

D US

OSPF

3 edge-disjoint paths

∞1

VerificationInvariant Graph property Required ARC

Properties

Always blockedSeparate connectedcomponents

Sound & Complete

Always reachable with < k failures

Max flow ≥ k Sound & Complete

Always traverse waypoint (chain)

Separate connected components

Sound & Complete

Always isolated No common edges Sound & Complete

Equivalence Same structure & weightsSound, Complete, & Precise

14

Precision required to produce counter-examples

Implementation and evaluation

• Implemented in Java using Batfish (parsing) and JGraphT (graph algorithms)https://bitbucket.org/uw-madison-networking-research/arc

• Configurations from 314 datacenter networks operated by alarge online service provider

• 4-core 2.8GHz CPU24GB RAM

15

Evaluation: time to generate ARC

16

Fast (< 10 sec) even for large

networks

Most time is spent parsing

Evaluation: verification time

17

Always blocked Always reachablewith < k failures

Always isolated

< 500 ms(Batfish: 694 days!)

Up to 16 min< 1 sec

Verification time is proportional to the number of traffic classes; easily parallelized

Next steps

• Precision under fewer assumptions

• Generality of ARCs

• Other uses…

18

Next steps: automated repair

19

Configurations ARC

Repairs

Challenge: finding a minimal repair(e.g., many ACLs vs. remove BGP neighbor)

without side-effects

1) Transform ETGs to have desired attributes (e.g., src and dst→ always blocked)

2) Translate to configchanges (e.g., remove edge → add ACL)

Controller

Next steps: Transition to SDN

20

Configurations ARC

Controller uses ETGs to drive forwarding plane configurations

Minimize controller involvement, churn?

Different underlying network topology?

Next steps: synthesis

• Operators require fine-grained control over routing: waypoints, isolation, traffic engineering– Intents configurations

• Distributed routing based on shortest path –very difficult to program!

• One approach: input data planes resilient ARCs configs

21

Synthesis

22

S1

S2

S3

S4

S5 S6

1

1

1

1

11

3

3

• Edge weights• Input path to dst must be

the shortest path• Uniqueness of shortest path

• Route filtering• Disable edges for a destination

to ensure path is shortest

• Backup paths• Weights such that backup path

is chosen during link failures

Summary

• Presented an abstract representation for control planes

– Fast and simple verification under arbitrary failures

– Verification is based on graph-level properties

– Up to 5 orders of magnitude speed-up

• Useful for repair, transition, synthesis, …

23

Try it!https://bitbucket.org/uw-madison-

networking-research/arc

Backup

24

Evaluation: verification time

25

Always blockedusing ARC

Always blockedusing Batfish

< 500 ms > 694 days!

Verification with ARC is 3 to 5 orders of magnitude faster!

Verification

• Always blocked: Src and Dst in same connected component?

26

B C

TD

U

1

31

T

T

TOSPF

S

CI

CO

DO

DI

DST:S SRC:T

CI

CO

DO

DI

DST:U SRC:T

T S

T U

?

?

Fast Control Plane Analysis

Using an Abstract Representation

Aditya Akella

Aaron Gember-Jacobson, Raajay Viswanathan and Ratul Mahajan

UW-Madison and Microsoft

28

Fast Control Plane Analysis

Using an Abstract Representation

Aditya Akella

29