control architectures for multi-layer networking

Control Architectures for Multi-layer Networking: Distributed, centralized,

or something in between?

Ori Gerstel, CTO

Sedona Systems

24-Mar-2015 1OFC15 tutorial - Sedona Systems

What is SDN (in 1 slide)• A control architecture that is based on centralized

control instead of distributed control• Why?

• Simpler design/code than a distributed solution• Opportunity to commoditize the hardware• Open for 3rd party app development• More optimal global decisions

24-Mar-2015 2

CP CP CP

Distributed architecture (PMO) Centralized architecture

SDN

Data Data Data Data Data Data

OFC15 tutorial - Sedona Systems

What is SDN (more realistic definition)

• A control architecture that is based on some centralized control instead of fully distributed control

• Why?• Existing gear will live in the network for many years• Some functions are better implemented in distributed fashion• The risk of moving to a disruptive new architecture is too high

24-Mar-2015 3

Hybrid architecture

SDN


CP CP CPData Data Data

Different forms of control distribution (outline)

• Network control architecture

• Internal controller architecture

• Multi-layer/vendor architecture


Sweet-spot for distributed and centralized control

Distributed control sweet-spot

• Real-time: optimized for restoration

• Localized changes

• Good enough for customers with basic

agility needs

• Highly survivable – even during large

scale disasters

• Harder to extend

Centralized control sweet-spot

• Slower: good for optimization

• Global network changes

• Adds value for customers expecting

advanced features

• Less available – unless significant

complexity is added

• Ease of extending the code

A combination of both can provide an optimal solution24-Mar-2015 5

Hybrid distributed and centralized control – not our invention…

Centralized control

Distributed control

24-Mar-2015 6

http://www.google.co.il/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=KWW5I97I30CQzM&tbnid=X7JG2UZLE21NTM:&ved=0CAUQjRw&url=http://users.rcn.com/jkimball.ma.ultranet/BiologyPages/P/PNS.html&ei=cdd0UeudKMLnOryMgKAD&psig=AFQjCNE76VYopMQB2z0Kp2wSoXmfAy75RQ&ust=1366697878713160

http://www.google.co.il/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=KWW5I97I30CQzM&tbnid=X7JG2UZLE21NTM:&ved=0CAUQjRw&url=http://users.rcn.com/jkimball.ma.ultranet/BiologyPages/P/PNS.html&ei=cdd0UeudKMLnOryMgKAD&psig=AFQjCNE76VYopMQB2z0Kp2wSoXmfAy75RQ&ust=1366697878713160

R1 R3

O1

O2

O3

SDN

controller

3. GMPLS-UNI creates the

optical circuit leveraging

WSON

3

2. SDN controller triggers

creation of new link by

provisioning the router with

policies

2

Hybrid control example

1a

1. SDN controller learns

optical network from (a)

Optical layer, (b) IP layer

1b

24-Mar-2015 7

4. Upon failure GMPLS

acts according to the

policies

Hybrid control – a spectrum of architectures

Distributed model

Central path compute (PCE) model

Policy provisioning model

Centralized model

Hybrid control models

24-Mar-2015 9

Initiation Policy decision

Path selection

Path setup

Distributed model

Central path compute model


Centralized model

• Initiation: who decides to set up the path?

• Policy decision: who decides about the constraints for the path?

• Path selection: who decides how to route the path?

• Path setup: who configures the nodes along the path – and (typically) who restores it?


• Setup initiated by the head-end of the path

• It decides on the constraints (min latency? max diversity?...)

• The head-end computes the path based on local info (or this could be a collaborative effort with other nodes along the path)

• The path is set up via distributed signaling (RSVP)

• The network retains autonomous actions (e.g., restoration)

24-Mar-2015 10


Path selection

Path setup

Distributed model D D D D

Central path compute model


Centralized model


• Setup initiated by the head-end of the path

• It decides on the constraints (e.g., based on user request)

• The head-end asks for a path from centralized path computation (PCE)

• PCE may provide an strict path (C) or a loose path (C+D)


• The network retains autonomous actions (e.g., restoration)24-Mar-2015 11


Path selection

Path setup


Central path compute model D D C [+D] D


Centralized model


• Setup initiated by the central controller

• It decides on the constraints and provisions the head-end with one or more policies

• The head-end computes the path based on the policies


• The network retains autonomous actions (e.g., restoration)

24-Mar-2015 12


Path selection

Path setup



Policy provisioning model C C D D

Centralized model

R1 R3

O1

O2

O3

SDN

controller

3. GMPLS-UNI creates the

optical circuit leveraging

WSON

3

2. SDN controller triggers

creation of new link by

provisioning the router with

policies

2

Hybrid control example (recap)

1a

1. SDN controller learns

optical network from (a)

Optical layer, (b) IP layer

1b

24-Mar-2015 13

4. Upon failure GMPLS

acts according to the

policies


• Setup initiated by the SDN controller

• It decides on the constraints

• It computes the path based on global info

• It sets up the path by provisioning each node separately

• Most (all) network control functions are centralized

24-Mar-2015 14


Path selection

Path setup



Policy provisioning model C C D D

Centralized model C C C C

Carrier grade requirements

• Typical carrier-grade req.s for the controller:1. It must survive HW/SW failures

2. It may have to keep working during SW upgrades

3. It may have to scale to a large number of transactions

• (Some of these req.s are not mandatory if the controller is not involvedin real-time operation)

Distributed systems address these req.s

• A failure of one copy does not affect other ones

• Upgrade of one copy allows other copies to continue working

• The workload can be distributed amongst multiple copies

Distributed systems address these req.s

• Typical solution: use distributed database to keep all copies in sync Controller

Adapter

NE

Adapter

NE

Apps

Network DB

Controller

Adapter

NE

Adapter

NE

Apps

Controller

Adapter

NE

Adapter

NE

Apps

Controller

Adapter

NE

Adapter

NE

Apps

Distributed network DB

Are we back to complex distributed control?

• One of the main promises of SDN was to simplify SW development by not having to deal with the complexity of keeping a distributed system in sync

• Are we reintroducing this complexity?

• Answer: no• Distributed state management is handled by the distributed

database in a unified fashion – no need to reinvent the wheel for every protocol

• The application works the same as in a centralized system

• All changes are under transactions, limiting possible inconsistencies

Example for ML control: Router bypass

• Based on traffic conditions, skip intermediate routers and connect directly routers that were not connected before

• Result: cost savings and better adaptability to changing traffic conditions

24-Mar-2015 21


• Consider a network and 2 end-to-end traffic flows

• What happens when a new bypass is added?

• Case 1: routing metric too low• The new link will attract

unexpected traffic

• Result: congestion

24-Mar-2015 22


• Case 2: routing metric too high• The new link will not attract

sufficient traffic

• Result: wasted resources

• Case 3: routing metric in the right range • The new link may not attract

sufficient traffic due to load balancing

• Result: (less) wasted resources

24-Mar-2015 23

Example for ML control: Router bypass• Should the purple link be

added to bypass router N3?

• Not simple to answer – it depends…• On the end to end IP traffic

• On the routing metric assigned to the link

• On use of IP load balancing

• On share risks between the new link and existing ones

• On protection capacity elsewhere

24-Mar-2015 24

Example for ML control:conclusions

• Whenever IP topology changes network simulation must be done• To ensure the new topology can support the traffic in an

efficient manner and under all failure scenarios

• This requires collection of a lot of global data

• Therefore must be done via a centralized controller

• It also requires a controller smart enough to accurately simulate IP layer behavior

24-Mar-2015 25

Monolithic SDN architecture

Monolithic SDN controller

ML App

26

• Monolithic approach: • Single controller controlling

multiple layers and vendors

• Concerns:• Complex to manage

• Complex to troubleshoot

• Complex to upgrade

• Recreating some of the issues SDN was meant to solve

24-Mar-2015

L3 NEsL0/1 NEs


Monolithic SDN architecture

Monolithic SDN controller

ML App

27

• More concerns:• Optical feasibility “must” be

computed by the optical vendor

• Simulation of IP layer behavior “must” be computed by a tool that understands its complexity

• No single vendor can provide all of this

24-Mar-2015

L3 NEsL0/1 NEs


Modular Hierarchical SDN architecture

• IETF proposal for rest API between controllers

• May work well for single vendor at each layer

• No “vendor lock-in” no vendor at the top of the pyramid

• (Still leaving the question of open: who will develop ML apps?)

L3 SDN controller

L3 NEs

L0/1 SDN controller

L0/1 NEs


• Very complex with several vendors and controllers (MxNproblem)

• Each API change affects many systems

• How are race conditions avoided?

• Does every vendor create their own ML apps?

• Very complex testing

L3 SDN controller

L3 NEs

L0/1 SDN controller

L0/1 NEs

L3 SDN controller

L0/1 SDN controller

L3 SDN controller

L0/1 SDN controller


Orchestrator

L3 SDN controller

L3 NEs

L0/1 SDN controller

L0/1 NEs

ML App

30

• Modular approach:• Specialized SDN controller for

each layer• Separate SDN orchestrator

which relies on single layer controllers for single layer services

• More open and flexible solution

• Leaves room for vendor expertise and innovation

24-Mar-2015 OFC15 tutorial - Sedona Systems


Orchestrator

L3 SDN controller

L3 NEs

Vendor 2 controller

Vendor 2 NEs

ML App

31

• In fact, in some layers (E.g., DWDM), a controller per vendoris likely• We have failed to

agree on a common way to model transmission impariments

24-Mar-2015 OFC15 tutorial - Sedona Systems

Vendor 1 controller

Vendor 1 NEs

Summary

• Both distributed and centralized control systems have valuable attributes for network control

• It is possible to keep some functions distributed while centralizing others

• The entire controller may become distributed to meet carrier grade req.s

• Multi-vendor systems are likely to be more distributed to allow for vendor innovation

• Typical network control is likely to be hybrid for the above reasons

Thank you!


control architectures for multi-layer networking

Documents