Understanding NFV Management and Orchestration

Alberto Diez April 2015

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

2

Executive Summary

Networks Functions Virtualization (NFV) has been one of the hot topics of 2015s Mobile

World Congress. After a year of proof of concepts and trials Mobile Network Operators

(MNOs) are ready to move forward with this architectural evolution.

MNOs see clear advantages in the application of NFV; they not only get rid of proprietary

purpose-specific hardware but they also expect to introduce a more flexible and economical

network which permits them to deploy new services with agility.

Adding to these advantages, NFV has been pointed out as a solution to overcome net

neutrality regulations limitations to provide higher value services. If MNOs cannot treat

differently services provided over the Internet they can, with NFV, create several virtual

purpose specific networks separated from the neutral Internet.

Equipment vendors, on the other hand, are also pushing for NFV. Not all vendors embrace

NFV with the same intensity; the louder advocates are the ones that sold their solutions over

standard hardware from 3rd parties. These vendors see NFV as a mean to reduce their low

margin hardware business and focus in their higher value propositions.

The introduction of NFV comes with costs to the MNOs. To have a positive business case it

is key to count with the operational efficiencies of managing a virtualized network. Those

benefits are realized by the Management and Orchestration (MANO) layer in the NFV

architecture.

MANO reduces operators development and operations costs and frees teams employed in

repetitive low value tasks for more valuable roles like optimizing the network and analyzing

the data provided by the management tools.

The relevance of MANO for the business case has triggered fierce competition in the MANO

tools market with a wide range of participants coming from Open Source, telco equipment

vendors, OSS vendors, IT cloud companies and SDN solution sellers.

Software Defined Networking (SDN) is directly related to MANO since controlling the virtual

network centrally is necessary for agility and operational efficiencies. In a broader sense the

NFV network is a Software Defined Network.

MANO is today a field of experimentation in which many interests converge. Interestingly

Open Source projects and a wide inclusive community with multiple stakeholders are

cooperating towards a satisfactory solution that will meet MNOs needs and foster industry

innovation. This is bringing about a new ecosystem and new business paradigms close to

those known in the Linux distributions.

NFV/SDN and MANO is one of the most stimulating fields in the telco industry today but also

somewhat confusing. This short report clarifies the current status for all those interested in

NFV/SDN MANO and its evolution.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

3

Table of Contents

Motivation........................................................................................................................... 4

NFV MANO in Context ........................................................................................................ 5

Service provider motivations to NFV ................................................................................ 6

NFV Management and Orchestration............................................................................... 7

What role does SDN play? .............................................................................................. 9

The Policy Control digression .........................................................................................10

Open Source is the new standard ...................................................................................12

Outlook ..........................................................................................................................13

Case Study: Scaling out a Media Resource Function..........................................................14

Company Landscape .........................................................................................................16

Standards and Protocols....................................................................................................19

Open Source Projects........................................................................................................22

Bibliography and References .............................................................................................24

About the Author

Alberto Diez is an unaffiliated professional who has been working with standard telco architectures since 2007. He started his career in Fraunhofer FOKUS where he conceived and prepared the launch of the OpenEPC project. Later he worked in the industry with Nokia Siemens Networks and in Siemens CVC with their carrier grade products as their AAA server and PCRF, as solution architect of the former and product manager of the latter. Current topics of interest include: new services and architectures for mobile operators, NFV/SDN and WebRTC. Contact via email: alberto@posteo.net

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

4

Motivation

NFV/SDN is revealing itself as a major shift in the telecommunications landscape. Since mid

2013 the succession of initiatives, press releases and product announcements has been

uninterrupted.

In 2015 focus has shifted to NFV Management and Orchestration (MANO); arguably

because MANO is key to the NFV business case and of primary significance for service

providers. It was not until end of 2014 that ETSI finally detailed the high-level specification of

MANO components. In the mean time, conflicting messages and biased visions contributed

to a strained environment difficult to navigate.

Equipment vendors and service providers have understood MANOs relevance and need to

position themselves. The time elapsed while standardization was ongoing has been too long

and actors coming from the IT sector, in which virtualization and cloud has long been norm,

have adapted their products and offerings to address telco industry too. Additionally Open

Source projects and initiatives have emerged and are being embraced substituting , to a

certain extent, the role of standard bodies and creating new paradigms.

In this context, MANO constitutes still a battlefield in which different wars between traditional

actors and emerging players are being held. Battles spread in the different layers defined for

MANO: infrastructure, application and services. Additionally, there are silent confrontations

between standard bodies active in the areas of Enterprise IT, Internet standards and Telco.

The battle in infrastructure management has been settled with OpenStack established as de

facto standard and catalyzing the establishment of an ecosystem similar to the Linux

distributions business. It is remarkable that VMWare has acknowledged this battle for its own

products as lost and released its own OpenStack flavor1, just like RedHat or HP do.

On top of infrastructure orchestration, application lifecycle management is required.

Enterprise IT vendors have well established solutions for application lifecycle management

provided as wrappers to OpenStack but lack support for some traditional telco concepts and

in practice they will require significant development and operations costs. Telco application

vendors prefer to view application lifecycle as completely separated from infrastructure and

are positioning themselves in the lifecycle management with separate products, which does

not contribute by itself to operational efficiencies in a normal multi-vendor environment.

The highest layer of MANO: service orchestration, requires a deep knowledge of

applications, infrastructure and networking. Is in this area where most operational gains

could be achieved but offerings are still not mature and risks are high.

Rivals in MANO are traditional telco vendors, OSS/BSS vendors and IT software vendors;

each with their biased interpretations of concepts and requirements.

1 VMWare Press Release “VMware vSphere 6 - The Foundation for Hybrid Cloud”, February 2nd 2015, San Francisco, California

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

5

NFV MANO in Context

Network Functions Virtualization (NFV) is an initiative from ETSI that started in 2013 with the

aim to standardize the application of virtualization techniques for telco deployments just as it

had been applied in the IT and datacenters.

Reduced to its basic principles NFV decouples software from hardware. NFV is a step further

in the trend, in telecommunications sector, moving from proprietary vendor specific hardware

to commercial hardware deployments of telco functions and now hardware independent

cloud deployments. In theory with NFV telecommunications equipment vendors no longer

need to provide servers for their functions instead they can just list their requirements in

generic processing performance, memory, storage, networking etc.

ETSI has standardized a relatively complex architecture with six blocks as depicted in the

figure.

Figure 1 ETSI NFV Architecture

In the left side of the architecture, the lowest layer includes the NFV Infrastructure (NFVI),

that is, processing, storage and networking physical resources. The lowest layer represents

the datacenters which with their virtualization layer (hypervisor) expose their capabilities to

the upper layer. The Virtual Network Functions (VNF) are deployed on top of the NFVI layer.

Each VNF represents an entity as it could be a DNS, Firewall, or a mobile core network

component like the MME or PDN-Gw. Each VNF may include an Element Manager (EM).

The top layer encompasses the Operational and Business Support Systems (OSS/BSS).

On the right side three layers of entities are grouped in the Management and Orchestration

(MANO) block. These three layers represent a control plane for infrastructure, applications

and services.

MANO is capturing much of the attention of NFV because during the early stages ETSI did

not concretely specify it giving room for different interpretations. Additionally two antagonist

approaches for the Virtual Infrastructure Manager (VIM) were available. Third, and probably

most importantly, MANO is instrumental for achieving the benefits promised to service

providers by NFV evangelists.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

6

Service provider motivations to NFV

NFV is a paradigm change that affects all stakeholders in the telecommunications industry. It

promises paramount benefits in the long term but it requires significant investments in the

short term.

Within equipment vendors two approaches have been observable in the last two years: the

ones that have embraced it from first day and actively promote it and the ones that tried to

ignore it first and are trying to catch up now when it is unavoidable. The formers are the ones

that have been selling their equipment on top of COTS hardware from third parties and are

willing to stop re-selling 3rd party hardware, with low margins, to concentrate in selling their

own software, virtualized. The latter vendors probably have more difficulties to adapt

products to NFV because they have been using purpose specific hardware. Most of the

larger vendors have a mixed situation; some products in their portfolio can be evolved to

NFV easily and others require serious re-architecting and may never be NFV capable at all.

Most interesting is the motivation of service providers to NFV. It has to be noted that except

for small operators NFV does not reduce hardware required to run a network and it just

permits to homogenize deployments.

Infonetics Research did a survey in March 20142 in which they asked a large group of service

providers if they would deploy NFV and SDN and why. The results are depicted in Figure 2.

Figure 2 Infonetics research service providers survey results (March 2014)

It is remarkable that Service Providers consider Quick Revenue as second most important

motivation for NFV/SDN deployment, most likely this category referred to the agility that NFV

deployments bring in which an operator can reduce the time-to-market significantly for a new

service rollout thanks to hardware independence and automatization of processes. It may

also refer to the possibility of new use cases associated with NFV/SDN, but those will

unlikely generate any quick revenues.

The remaining motivations are basically hardware homogenization and operational

efficiencies (grouping scaling up/down as an operational efficiency because it was also

possible without NFV it is just dramatically more efficient with NFV).

2 Infonetics Research: SDN and NFV Strategies: Global Service Provider Survey. March 2014

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

7

If quick revenue is more about quick than about revenue and all other motivations are

operational efficiencies it is clear that in order to have a positive business case for NFV/SDN

it is necessary to achieve significant operational efficiencies.

This leads to the question: how is it possible to achieve the maximum operational efficiencies

of NFV/SDN? The answer is, of course, through NFV Management and Orchestration.

NFV Management and Orchestration

NFV MANO has been part of the ETSI architecture since the beginning but it has taken long

until its role has been completely described and details specified. The architecture presents

three elements: the Virtualized Infrastructure Manager (VIM), the VNF Manager and the

Orchestrator.

Figure 3 MANO layer

The role of the VIM includes the management and allocation of virtual resources. It

participates in all procedures related to adding, removing or modifying resources for the

different functions. Resources refer to computing resources, storage or networking

resources.

The VIM had been the first attention focus for the Service Providers. Two competing

strategies available: VMWare or OpenStack. Already in the last half of 2014 it was clear that

OpenStack was the only possible strategy what has ultimately being ratified by VMWare

launching its own distribution of OpenStack in February 20153.

The debate shifted from which VIM to use, to whether it was anything else necessary for

MANO than just OpenStack. The reason for that is that OpenStack as a project

encompasses many different components4, the most famous are Nova and Neutron the

3 VMWare Press Release “VMware vSphere 6 - The Foundation for Hybrid Cloud”, February 2nd 2015, San Francisco, California 4 https://www.openstack.org/software/roadmap/ and http://docs.openstack.org/

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

8

compute and networking managers which expose all operations on resources via APIs.

OpenStack also provides additional components like a control Dashboard and including

Heat, a component for orchestration.

The reality is that while OpenStack constitutes a close to ideal VIM for data center use

cases, it is not suitable for the tasks of the higher layers of MANO due to its infrastructure

focus and the particularities of the telecommunications equipment and networks architecture

requirements. Ultimately, OpenStack provides components and APIs that could be all what

very skilled and trained development and operations teams need for MANO. It would require

developing plugins on top of the existing features and using other configuration management

tools of the like of Puppet and Chef. Such approach does not automatically achieve the

highest of the efficiencies available with the upper layers of MANO.

The VNF Manager cares about the lifecycle of the application. Basically instantiation, initial

configuration, scaling up/down and in/out, updates, termination. There are currently two

trends in the market: generic VNF Managers delivered together with Orchestrators or VNF

Managers provided by the VNF vendors. Since applications in telco world are very complex

with several components types and different scaling rules etc. it seems convenient to have a

VNF Manager from the application vendor. Vendors of several applications will share the

same VNF Manager for all their applications. The VNF Manager must provide APIs fo r

higher level orchestration since a service provider will have many VNF Managers deployed,

unless it goes for a single vendor deployment which is unlikely and would limit the

advantages of NFV.

On top of the application lifecycle management sits the Orchestrator which according to

ETSI cares about end to end service including VNF configuration, provisioning, scaling and

networking setup. Its broad scope makes it the most complex element and undoubtedly

where most operational efficiencies rely, therefore the most interesting element in the MANO

framework.

Performing end to end service management requires the Orchestrator to have a deep

knowledge of the network, the VNF catalogue available, the different instances and the

resources available.

Additionally, the Orchestrator must be able to interact with different VNF Managers for which

there is no standard interface; therefore the Orchestrator requires a very flexible engine and

multi protocol support. To a certain extent, the Orchestrator is meant to be a central point of

control and configuration. It has some common areas with Configuration Management but it

goes much further. Orchestration is dealing continually with change, supporting run-time

automation via models/templates.

The three layers MANO framework are key for the NFV business case. The lowest layer, the

VIM, is taken by OpenStack. VNF Managers are provided by applications vendors. Therefore

it is in the Orchestrator’s layer where new and not so new companies are trying to establish

their offerings. It is an interesting field because it shares some common features with

traditional OSS/BSS systems making it attractive for those vendors to participate. In the

same way providers of IT Cloud solutions are also interested in orchestration. If that was not

enough, service orchestration interacts with SDN control and SDN players have come into

the picture.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

9

What role does SDN play?

Software Defined Networking (SDN) refers to the decoupling of packet forwarding from its

control. Initially it was a paradigm shift for routers and switches and almost since the

beginning it was identified with a protocol: OpenFlow.

OpenFlow permits a control element to instruct routers and switches how to forward packets

dynamically by setting forwarding rules. OpenFlow and SDN has been extensively

researched in universities and applied to datacenters. SDN brings with it significant

operational efficiencies when operating large networks.

SDN is not limited to OpenFlow since other management approaches using other protocols

like NETCONF and YANG or REST APIs also belong to SDN.

NFV decoupling hardware from software is much like SDN decoupling user plane fr om

control plane. In particular where SDN requires a flexible SDN Controller to achieve its

highest potentials, NFV requires the MANO framework and in particular the Orchestrator to

excel. Both the SDN Controller and NFV MANO provide a translation between OSS and

applications requirements, expressed in a generic abstract way, into detail control of the

infrastructure resources available.

Moreover, NFV MANO requires SDN. That is, since orchestration requires provisioning,

configuring and automating the processes for end to end service it requires control over the

network and forwarding planes. Since MANO is on the spot, it has been more likely to see

NFV/SDN as combined terms. ETSI has acknowledged this overlap by including annexes

about SDN5.

Figure 4 SDN provides remote control of forwarding tables

The confluence of NFV and SDN has triggered that several vendors of traditional networking

equipment have launched very interesting offerings in the orchestration area. It has to be

noted that SDN had been in the picture longer than NFV and the SDN controller already

provided template driven automatization requiring deep knowledge of the network and its

configuration capabilities.

A established trend is to refer to SDx or Software Defined Everything and exposure of APIs

in all components to permit programmatic remote configuration and manageability. In this

5 ETSI GS NFV-SWA 001 V1.1.1 - Network Functions Virtualisation (NFV); Virtual Network Functions Architecture

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

10

sense, SDx and NFV Orchestration roadmaps are convergent and constitute also a reason to

refer to NFV/SDN as a combined paradigm.

Another aspect in which NFV and SDN are complementary is that of forwarding graphs.

Forwarding graphs, as described in ETSI NFV specifications, represent a chain of VNFs

which implement a given service. Ultimately it is a matter of network control to configure the

data flow to execute a forwarding graph. There has been much noise about forwarding

graphs for mobile operators since it is foreseen as an enabler for agile new use cases

implementation, combining existing VNFs to provide different service experiences.

Figure 5 Service Functions Chaining and Forwarding Graphs

The Policy Control digression

Although more a SDN topic than an NFV one, selecting and modifying the forwarding graph

has been often promoted as a possibility brought by NFV very relevant for service providers.

An example use case would be selecting that a given data flow has to go through a firewall,

media or protocol optimizer or similar functionality, while other data flow goes through a

different path. These use cases are known as Service Functions Chaining (SFC).

It is not relevant whether the different functions are physical or virtual functions. The

importance for the use case is the ability to select the path, or forwarding graph, dynamically

based on the different criteria, in particular real-time, personalized and business intelligence.

According to ETSI NFV standards it is the role of the Orchestrator to establish forwarding

graphs and manage the infrastructure that provides the end to end services. SFC use cases

are most valuable when the SFC decision is taken in real-time based on subscriber profile or

application aware. Currently such information sources are not directly available to the

Orchestrator; therefore the whole use case must reside elsewhere in the network: the policy

controller.

The policy controller, PCRF in 3GPP terminology, takes decisions about how to treat

connectivity of the subscribers based on the subscriber profile, network information and

application awareness. It is an element already available in most mobile networks and

increasing importance due to its role for VoLTE.

As part of the decisions the PCRF takes it is possible to provide indications to the Gateways

regarding SFC. Those indications can be in the form of predefined rules, which can be

translated at the gateways by different forwarding tags and forwarding paths selection; IETF

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

11

is developing standards that address this scenario6. An alternative development is that

PCRFs communicates its decisions to an Orchestrator / SDN Controller that would create the

paths in real-time.

Therefore there is little overlap between the PCRF functionality, SDN or NFV. The

relationship between PCRF and NFV is that it has been identified as one of the first

components to be deployable virtualized due to the nature of its functionality and the fact that

it does not handle user plane traffic. The relationship between PCRF and SDN is stronger;

the PCRF may use SDN methods or communicate with the SDN Controller to realize the

SFC use cases. Some adventurous PCRF vendors will add SDN capabilities to their PCRF in

the future but it is still too early for such approaches to have real commercial relevance.

Policy in a broader sense is of course part of NFV MANO and SDN. ETSI specification does

mention the Orchestrator as checking policy conformance of requests and issuer but it is not

the policy control that mobile operators know from 3GPP, instead more an authorization and

access management related entitlements control combined with a rules engine.

Policy and rules engine based decisions are relevant to orchestration since in many

scenarios, for example automatic elasticity, the Orchestrator has to take decisions on which

components to scale and how. While current approach foresees static templates the next

step will be more dynamic rule based decisions based on OSS/BSS inputs. A long-term step

will be incorporating other sources of context, which are more dynamic in nature.

A much more disruptive development where policy and NFV/SDN encounter is that of “intent

based networking”7. Intent based networking is best explained as an application of the

Android mobile operating system framework concept of Intent to the NFV/SDN world.

In Android it is possible to provide services by combining applications much like operators

would like to do with SFC, in a very dynamic and agile way. An Android application calls the

framework when it has an event that could be handled by another application. The calling

application does not know if there is another application to handle the event but if there is the

framework will know, since the called application must have registered this capability before.

The calling application provides the framework the type of intent and a context for the second

application. For example, if the user wants to share a contact it goes to the address book and

selects the contact and the option of sharing. That creates an intent to the framework of

“sending” with the contact as its context. The framework will offer the user the different

applications that can send something like the different email clients, MMS client and all

available messaging applications.

Current research is aiming to provide a common information model and interface for

communications of intents. Applicability of intents is confined today in the lowest

infrastructure layer of MANO and SDN Control (i.e. OpenStack / OpenDayLight) but only

complexity and market priorities are preventing intents to climb up the layer and situate itself

in the highest layer of orchestration.

Whether intent based networking will find a pragmatic application in mobile operators ’

network is still unknown but if it does it will combine with SFC and may require cooperation

from NFV MANO / SDN Controller and the subscriber and application aware policy controller.

6 RFC 7498 Problem Statement for Service Function Chaining, April 2015 7 https://www.opennetworking.org/?p=1633&option=com_wordpress&Itemid=471

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

12

Open Source is the new standard

An interesting development has been the role that Open Source software projects have

played in extending the impact of SDN and NFV and the fainting role of traditional

telecommunications standard bodies.

The trend started with SDN which comes from a research, university and datacenter

background and that for a while has not been perceived as key for telco. SDN found in

OpenFlow its standard bearer. OpenFlow has been specified and promoted by ONF, out of

the mainstream industry bodies like IETF. SDN has spread thank to a myriad of Open

Source initiatives for switching software etc., and when it had already gained traction in the

industry the Linux Foundation has been instrumental in organizing OpenDaylight , with

support from vendors, as the SDN Controller for OpenFlow and others.

NFV has followed a distinct but convergent path. Standardized within the traditional bodies in

ETSI, it has seen a significant push thanks to Open Source projects developed for non-telco

specific purposes like KVM, and very predominantly, OpenStack. NFV MANO has been a

confusing standardization area for long and the creation of the OPNFV Open Source project

under the leadership of the Linux Foundation has catalyzed the market interest in MANO.

Traditional standard bodies are not neglected but industry has seen that the Open Source

model and institutions like the Linux Foundation, with a strong emphasis in inclusive

cooperation of multiple stakeholders, are a better channel to bring the new architectural and

technological improvements that NFV/SDN require.

The wide spread of Open Source projects comes with some new effects that not all or actors

in the telco industry will embrace in the same way. In the technical side it has to be seen how

it affects innovation in the long-term and what will the results be. Key questions can be, for

example around OPNFV. The work of OPNFV will with no doubt result in a concrete and

detailed interpretation of the NFV MANO standard and a selection of some of the options

open in the ETSI specification. Will this selection become a de facto standard? Will there be

room for different interpretations? Will the OpenStack APIs be considered standardized and

all VIM implementations should support them in order to guarantee interoperability?

Furthermore, Open Source creates new challenges in the commercial and business areas.

Open Source requires a different business model and not all companies are prepared for it.

Interestingly, if what is happening around Open Source in NFV/SDN would be completely

internalized in the industry it might reduce the role of patents and intellectual property rights.

This will upset many stakeholders but on the other hand will probably foster innovation in a

software industry running virtualized over standard hardware. It will shrink entry barriers and

open the telco industry for breakthrough innovations coming from outside the well-known

reduced vendor ecosystem.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

13

Outlook

NFV MANO and particularly the highest layer service orchestration is still a field of much

experimentation. Both technology and business-wise there are surely surprising

developments upcoming.

Looking at technology a necessary development is addressing hybrid networks, that is, those

networks which contain both virtual and physical functions. Service providers will have for

long time hybrid networks and the Orchestrator can create operational efficiencies in both.

While of course physical networks limit the applicability of the Orchestrator (e.g. scaling) they

can still be managed and configured.

An imminent step is the availability of container technology as a different flavor of

virtualization. If today infrastructure virtualization manages principally ESXi and KVM based

Virtual Machines, soon it will also manage different types of containers. Containers bring the

advantage of less overhead and more efficient use of resources (depending of the scenario)

with the cost of more complexity in the Management and Orchestration layer.

An already emerging reality is the fierce competition that NFV MANO products will face. It is

one of the key topics of the year but also, as described before, fundamental for the whole

NFV business case. All possible actors want to play a role in NFV MANO. And all actors,

mean all possible actors, from OSS vendors who see that orchestration may deprecate some

of their offerings to IT Cloud and Enterprise computing stacks companies which see a

possible new customer base where higher margins are possible. Until today, most interesting

offers have come from SDN Controllers vendors that had re-branded their products and

added features to cater the NFV Orchestration market too. They had been in a significant

advantageous position due to the overlap of NFV MANO and SDN, particularly the common

requirements and shared concepts permit them start the orchestration adventure with a well

suited architecture on top of which build the product.

Competition is not only in technologies, features, business models and approaches, but also

ideological. Due to the lack of concreteness in the standard significant evangelization is

ongoing and in many cases vendor-biased views need to be balanced with independent and

objective approaches.

Influenced by this, another trend is the substitution of traditional telco standards by Open

Source projects and open initiatives. The irruption of the Linux Foundation with the OPNFV

initiative backed up by major players has made obvious a shift that had long time ago started

in which Open Source has gained relevance in telco sector.

As a corollary to the role of Linux Foundation and the ubiquitous presence of Open Source in

the telco industry, there is an emerging Linuxization of business models in the vendor

ecosystem. This does not refer to traditional Linux distributions vendors occupying a more

prominent role in the NFV landscape, which is also happening, but to a shift from the

traditional software licensing to a Linux distribution business model in which the higher

portion of the incomes come from complementary activities and services like trainings,

support etc. Just like in the Linux distribution world some vendors are taking Open Source

projects, e.g. OpenStack, modifying or substituting some components and re-packaging

them for telco customers.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

14

Case Study: Scaling out a Media Resource Function

ETSI includes in its specifications examples in the annexes, those examples often illustrate

all possible options supported which potentially creates more confusion than clarity. This

section provides a simple example of scaling out a Media Resource Function (MRF) in an

NFV environment.

The MRF is one of the elements showcased by ETSI in its examples. An MRF is a

component required for IMS deployments which processes media e.g. transcoding,

conference mixing. The MRF showcased by ETSI has two components: the MRB which is a

signaling broker and the MRF C+P which combines both control and processing.

For the example the assumed generic standard deployment of a virtualized IMS including an

MRF is depicted in Figure 6. The Element Manager (EM) of the MRB and MRF C+P VNFs

are not shown for simplicity but could be deployed separated or shared with several other

functions. The figure shows the PCSCF and SCSCF sharing the same EM while the HSS,

maybe from another vendor, is integrated with a distinct EM. The scaling out operation

consists in adding more capacity to the MRF dynamically and can be triggered, for example,

when the network detects a peak in the IMS traffic or overload situation in current resources.

Figure 6 Example architecture for scaling a MRF

The following steps implement the scale out operation:

1. Either an EM or the NFVI communicate performance data to the OSS/BSS. For example

the SCSCF communicates that the number of call establishment per second has surpassed

a threshold or the MRFB communicates the number of transactions or CPU usage or any

other performance measurement to the OSS/BSS layer.

2. The OSS/BSS system notifies the Orchestrator about the event. In this step the split of

intelligence is vendor specific. The OSS/BSS may be intelligent enough to decide how to

react to the event occurring and ask the Orchestrator to perform a specific action or instead

notify the event and let the Orchestrator decide. The later approach is more favorable of

complex Orchestrators.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

15

3. The Orchestrator analyzes the event using information from its repositories that model the

services and VNFs, current Instances running and resources consumed but also rules

describing how to react in such circumstances. For example the Orchestrator can decide,

based on rules, that due to the increase in SCSCF call establishments it has to scale out the

MRF C+P component but not the SCSCF or MRB or any other components which handle

only signaling.

4. Two options possible here depending on the intelligence split between Orchestrator and

VNF Manager. The Orchestrator can communicate the VNF Manager responsible for the

MRF C+P component that it needs to scale out the system. The Orchestrator could also

communicate first with the VIM for reserving resources and then the VNF Manager indicating

which resources to use for the scaling out operation. The relevance of these two options is

that when there are several VIMs managed by the Orchestrator the VNF Manager may not

be able to decide which to use and the Orchestrator, which has a knowledge of end to end

service can take a better decision in this regard.

5. The VNF Manager requests the VIM what resources to reserve using templates from its

VNF catalog. The resources are communicated as a set of computing, storage, memory

requirements and affinity and anti-affinity rules.

6. VIM reserves the resources for the new virtual machine.

7. VNF Manager instantiates the new MRF C+P component in the newly available resource.

This includes both installation and initial configuration.

8. Either VNF Manager or Orchestrator can tell the VIM about network resources to allocate

for the new component. Traditionally it will be a role of the Orchestrator if it is a forwarding

graph associated with the end to end service. It would be the role of the VNF Manager when

it refers to the internal VNF networks, for example when MRF C and MRF P are deployed in

different virtual machines.

9. As part of the scaling out of the MRF it is necessary to change the configuration of the

MRB to start using the newly available system (unless it has auto-discovery features). If what

had been scaled out would be an MRB it would necessary to change the configuration of a

totally different component, i.e. the SCSCF or AS, and this task would be performed by the

Orchestrator since that component could be managed by a different VNF Manager. Since the

MRB and MRF C+P probably belong to the same vendor and are managed by the same VNF

Manager it is possible that with the message in step 4 in which the Orchestrator requests

scaling out the MRF C+P the VNF Manager updates the MRB too. If not the Orchestrator

would have to trigger an update in configuration in a different message.

10. The VNF Manager of the MRB updates the configuration to start using the newly

instantiated MRF C+P.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

16

Company Landscape

This section describes a reduced set of companies actively promoting products in the NFV

MANO area. It does not provide a comprehensive list, the information included is highly

subjective and based in presentations, webinars, whitepapers and demonstrations during

MWC 2015 by the listed companies. For a more comprehensive list of companies active in

the NFV spectrum refer to the SDxCentral NFV report from April 20158.

Other companies active in the NFV MANO area are: Huawei, NEC/NetCracker, Nakina,

Overture.

Canonical

Canonical is the company behind the Ubuntu Linux distribution. Compared to other Linux

distribution companies Canonical has a very broad focus and customer base and i t does not

have a large footprint in the telco industry. Still due to their presence in the IT and Enterprise

sector they have a competitive OpenStack distribution and have added a flexible

orchestration product called Juju.

Juju is a flexible engine and GUI that uses recipes called charms to perform lifecycle and

configuration management activities automatically. It runs on top of the NFVI (e.g.

OpenStack or AWS) and only with Ubuntu based applications, which is a great limitation of

an otherwise very interesting proposal that seems to be more relevant to the IT/Enterprise

world than to the telco.

Canonical is actively partnering with telco equipment vendors to provide a more interesting

catalog of telco charms for their products.

Ciena

Ciena is a company with presence in the packet networking and optical communications

business. As others in that business they started early with SDN perceiving that it will

significantly affect their strategy. In the same way they had evolved their SDN portfolio to an

NFV/SDN portfolio with particular focus in NFV Orchestration.

Ciena offers a paradigm shift providing mostly enterprises with a platform that permits

deployment of VNFs as a service with associated payment per consumption, similar to

generic cloud offerings. It is an interesting offering for some verticals or smaller operators but

maybe does not fit current needs of the larger Tier 1s.

Cisco tail-f

Cisco is well positioned in the orchestration market due to its acquisition of tail-f. tail-f was a

Swedish company which gained success with SDN control. tail-f strategy towards SDN

favors NETCONF and Yang against other protocols. Yang is used in tail-f NCS/NSO

Orchestrator as modeling language and NETCONF as preferred transport.

The main offering is a central Orchestrator as a runtime engine with a real-time database

including data models for networks, devices and services. Data structures are the key

element for configurability in their solution and are described using Yang.

8 https://www.sdxcentral.com/reports/nfv-2015/

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

17

After the acquisition, Cisco is still promoting actively tail-f NSO as NFV Orchestration

solution. The product provides a model-driven and transactional higher level Orchestrator.

Cisco has announced a generic VNF manager as part of the solution which is an interesting

development since while major VNF vendors are looking to provide their own VNF Managers

many other smaller functions in the operator network may require a programmable generic

VNF Manager.

Though the product supports multiple protocols, Cisco still evangelizes for the broader

application of NETCONF and YANG. As an interesting move, they have released as Open

Source the basic version of ConfD which is their management service which other equipment

vendors can use for their products.

Cyan

Cyan is another company coming from the optical communications background which

stepped early in the SDN trend and has evolved to provide a well-suited higher level

Orchestrator. It combines SDN control and APIs for a complete offering that includes Planet

Orchestrate, Planet Operate, Planet View, Planet Inventory and Planet Design.

Cyan’s solution claims to be multi-vendor and provides both southbound and northbound

APIs. In the southbound interfaces it supports NETCONF/YANG, OpenFLow, SNMP and

others.

While the product has a more SDN orientation it is not a bad starting point and its adding

capabilities in the NFV direction; such efforts have been shown during the Mobile World

Congress, where it was presented as the Orchestrator used in a joint test with Telefonica,

Brocade, Intel and RedHat9 in which Cyan provided the higher level service orchestration

and VNF Management capability and Telefonica the VIM with its OpenMANO solution.

Overall Cyan Blue Planet NFV Orchestrator is a competitive solution which has already been

selected by CenturyLink as its reference Service Orchestrator10 for enterprise business.

HP

HP has acknowledged NFV as key for their business. It, of course, affects their servers

business but their involvement in OPNFV and new products illustrates a full commitment to

NFV.

HP has launched Helion11, their own OpenStack distribution providing their support.

Additionally HP offers NFV Director as their Orchestrator, coming from their strong presence

in their OSS/BSS area.

During the Mobile World Congress 2015 HP and Telefonica issued a press release12 in which

they announced working together for the Telefonica UNICA architecture. The most relevant

part of the announcement is the commitment of Telefonica for NFV Directo r as their

Orchestrator. In the press-release they adequately refer that NFV Director “implements

orchestration functionality in the emerging model from ETSI critical to NFV operation”. While

9 http://pressoffice.telefonica.com/documentos/EndtoEndNFVArchitectureFinal_1.pdf 10 http://www.cyaninc.com/company/news-room/press-releases/2015/centurylink-to-offer-nfv-enhanced-services-to-enterprise-and-smb-customers-with-cyans-blue-planet-#.VTIKIJOuzLU 11 http://www8.hp.com/us/en/hp-news/press-release.html?id=1668354#.VTH99pOuxHA 12 http://www8.hp.com/us/en/hp-news/press-release.html?id=1923363#.VTH_yJOuxHA

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

18

it is completely correct and relevant to acknowledge that orchestration is critical for NFV

operation (and for the NFV business case); the accent should be in the fact that the

“emerging model” is by no mean a fixed standard and NFV Director is another vendor

interpretation of what it may be. In particular the HP product offer, as showcased during the

MWC, is a re-branding of HP OSS/BSS offerings with little adaptation for NFV.

RedHat

RedHat is not only one of the older Linux distribution companies but also a well connected

partner in the telco equipment vendor landscape. Their experience with the Linux distribution

business model and their established partnerships in the telco world has positioned RedHat

in an advantageous position for the NFV MANO race.

RedHat focuses in the VIM component with its own OpenStack distribution in which they

enhance OpenStack with their own volume storage component and with RedHat CloudForms

and The Foreman for Management.

One of the main value that RedHat delivers is their pre-integration of their platform with a

wide set of partners and which particularly in the MANO layer translates into almost all the

main actors for VNF Lifecycle Managers and service orchestration.

VMWare

VMWare was since the beginning very well positioned for NFV since it was the best known

brand associated with virtualization. As a traditional proprietary software vendor they are

also well positioned to be the first suffering the Linuxification of the business and the violent

irruption of Open Source as the new standard.

Their hypervisor ESXi is very relevant and well established although increasingly operators

favor its Open Source competitor KVM. Their venture with Cisco and EMC for creating a

virtualization specific platform: vBlocks had not gained significant adoption and VMWare

stepped out of it.

In a remarkable twist of events, VMWare announced in February 2015 their own OpenStack

distribution changing their approach. The question for the operator no longer is if OpenStack

or VMWare technologies for their VIM but instead which version of OpenStack. VMWare

taking this step is an effective recognition that OpenStack is the VIM further on and positions

VMWare to better compete in the NFV MANO market.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

19

Standards and Protocols

ETSI

The European Telecommunications Standards Institute (ETSI) has been the standard body

that has formalized NFV by holding the space for the discussions and publishing the

specifications in which service providers and vendors describe its requirements and

architecture.

Since the first activities in early 2013 it has been clear that Management and Orchestration

was of paramount importance for realizing the benefits of NFV, still it took ETSI long to

produce their NFV MANO specification due to the complexity.

ETSI does not have a mobile network focus and has defined some targeted use cases that

include mobile, fixed network, content delivery networks etc. The more diverse input is

resulting in a generic and inclusive definition of NFV than what other bodies would have

produced.

OASIS

OASIS is a consortium standardizing information data formats and languages. It is much

more relevant in the enterprise and IT world than in the telco environment. In NFV it is

relevant since OASIS had been working in standards for cloud computing already before the

NFV wave and as a result the TOSCA standard is relevant in the NFV MANO environment.

ETSI MANO specifications include TOSCA as an option for representation of deployment of

NFV applications and services.

IETF

IETF is, of course, a very relevant standardization body but did not seem particularly well

positioned in the NFV/SDN context where other standardization bodies have taken a place

that traditionally IETF would occupy. Particularly, OpenFlow has established itself as a

synonymous of the protocol for SDN and it comes from the ONF, an outsider.

The predominance of OpenFlow for SDN is not being questioned but IETF protocol

NETCONF and modeling language YANG are now, more than before, also perceived as key

for SDN, in particular since YANG is used in OpenDaylight. YANG is also winning relevance

in the NFV MANO area, since ETSI proposes YANG as a candidate language for modeling

its information elements, where it also competes with TOSCA from OASIS.

With the Service Functions Chaining (SFC)13 track, IETF is also becoming relevant in one of

the most predominant use cases of SDN for MNOs in the short-term. The SFC approach is

not substitutive of OpenFlow but it requires a control protocol for transmitting configuration to

the network elements which in the SFC construct could be the standard 3GPP Policy Control

protocols, very much inline with the position of telco equipment vendors; OpenFlow for a

more SDN view or NETCONF and YANG in a possible fully IETF SDN scenario.

So while SFC does not substitute OpenFlow it re-allocates IETF at the center of SDN and

positions OpenFlow as one possible option, but not necessarily the most adequate for all use

cases.

13 https://datatracker.ietf.org/wg/sfc/documents/

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

20

ONF

The Open Networking Foundation is active since 2011 and has managed to establish

OpenFlow as a synonym of SDN. ONF is working actively in advancing OpenFlow

specifications and adding features and use cases. They also organize plug-fests and

certifications for OpenFlow devices.

With the current maturity achieved by OpenFlow, the acknowledgement that SDN value

exceeds that of OpenFlow and the attention attracted by NFV; ONF is shifting its focus in

promoting a more advanced, open, multi-player and inclusive SDN.

OpenFlow

OpenFlow is a protocol for controlling the forwarding table of network elements: routers and

switches. OpenFlow is specified by the Open Networking Foundation. Beside its reduced

initial scope it has been often considered as the key element of SDN. Many vendors of

switches and networking elements support OpenFlow today and the protocol is adding

features to broaden its applicability with extensions to forwarding tables manipulation.

OpenFlow relevance for NFV MANO is in the overlap between NFV and SDN, that is when

the SDN Controller is part of the MANO elements. OpenDayLight is the Open Source SDN

controller of preference and provides OpenFlow support as its main southbound interface.

SFC

Service Functions Chaining (SFC) is a recent working group from IETF creating new protocol

extensions to cover one of the most prominent SDN use cases.

SFC defines an architecture and protocol extension (NSH Network Service Header) to create

network overlays that decouple the service path from the network topology. It requires a

network control element to provision the NSH and its meaning in all the elements in the path.

SFC defines the transport protocol extensions and analyzes its applicability in both the Data

Center environment and the Mobile Network Operator architecture but does not define the

control protocol used for provisioning and configuration of the elements which can be

OpenFlow, NETCONF/YANG or PCRF steered.

TOSCA

OASIS TOSCA is a format for describing applications deployment templates. An application

deployment template is a data structure that represents the deployment of a VNF and its

basic lifecycle operations, like install, reinstall, start, stop restart etc. It also provides means

to describe initial configuration.

TOSCA is based on YAML and can also describe a service as a set of VNFs and the

forwarding graph necessary for it.

TOSCA is applicable in the three layers of MANO as a standard way to communicate

application and service deployment data with resource abstraction and independence.

TOSCA has many competitors in the interface between VNF Manager and VIM like HOT

format (from OpenStack), Charms (from Canonical Juju) and CloudFormation (Amazon Web

Services). Also YANG is being perceived as a possible competitor to TOSCA.

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

21

TOSCA limits its applicability to deployment of VNFs and services realized of VNFs and

there is a debate whether YANG is better suited for NFV MANO since it supports hybrid

environments and runtime configurability which is key in NFV/SDN. Ultimately it is possible to

use both: TOSCA for the deployment templates and NETCONF/YANG for runtime

configuration and orchestration14.

YANG

YANG is the IETF choice for declarative modeling language. It is human readable, extensible

and can represent both configuration and state. YANG can be used to describe VNF or

services deployment templates as well as runtime configuration needs.

YANG has won success in the networking space enabling network programmability and

SDN. While it is applicable for both physical and virtual appliances, YANG is interesting in

NFV Orchestration as a modeling language for templates and state. It is a key asset of an

intelligent orchestrator to have a powerful modeling language in order to keep complexity

under control.

NETCONF

NETCONF is a management protocol from IETF that uses a simple remote procedure call

based mechanism to configure remote devices. It can be used for configuration

management, performance management etc. In the long-term NETCONF is meant to replace

SNMP.

NETCONF provides a well defined protocol for management APIs and remote

programmability of network devices. It provides transactions support, feature discovery, data

model independence and discovery etc. It is IETF ’s answer to the spread of REST based

APIs in network element management APIs. While REST is easy to understand, it does not

provide a formal protocol and many required features are built on top in proprietary ways in

each implementation.

In NFV Orchestration, NETCONF is only relevant as the most convenient transport for YANG

although it is only one of the possible transports.

14 http://www.tail-f.com/wordpress/wp-content/uploads/2015/02/HR-Cisco-ALU-TOSCA-YANG-WP-2-17-15.pdf

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

22

Open Source Projects

OPNFV

OPNFV is a community project of the Linux Foundation created with the purpose of

advancing NFV and creating a reference implementation of ETSI NFV architecture.

OPNFV foundation follows the emergence of OpenStack as the VIM for NFV and the

increasing role of OpenDaylight project as SDN Controller. Observed from the outside, there

was a need for collaboration for those two projects and the many other Open Source

initiatives (e.g. vSwitch, libvirt, OVS). The interest of telco operators in bringing NFV

triggered the creation of OPNFV.

Operators first understood that to realize the service deployment agility that NFV promised

with a positive business case they depend fully in the MANO and control layer; and only then

they realized that their carrier-grade requirements where not being considered by the key

projects in those areas. OPNFV role is to be the forum where telco service providers bring in

their requirements for OpenStack and OpenDaylight.

Topics like pre-integration, interoperability and testing have been in the scope of mobile

operators; but other participants also benefit from the establishment of test-beds and it is

foreseen that the overall community will benefit from OPNFV as an open platform for

innovation.

Initial focus is the VIM in MANO layer and its interfaces. OPNFV does foresee commercial

distributions and implementations and contributions back to ETSI15.

OpenStack

OpenStack is a public and private cloud manager tool that is increasingly recognized as the

Virtual Infrastructure Manager for NFV. OpenStack is free and Open Source developed with

the key openness principles that characterize the free software community. It is widely used

in IT cloud environments.

OpenStack architecture comprises compute, networking and storage control elements and

supporting shared services like a dashboard, authentication and identity management

component and an orchestration component (Heat) etc. All OpenStack elements expose

REST APIs for automated programmability.

Its usage in telco industry is being fostered by the OPNFV project and the emergence of

OpenStack distributions supported and pre-integrated; much like Linux distributions have

contributed to the spread of Linux.

OpenStack plays an undisputable role in NFV MANO since even VMWare, who would have

been a competitor, has included its own OpenStack distribution as part of its VIM solution.

15 https://www.opnfv.org/sites/opnfv/files/pages/files/opnfv_whitepaper_103014.pdf

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

23

OpenDaylight

OpenDaylight is a Linux Foundation project for an open source SDN Controller. It provides a

flexible control engine with REST and OSGi northbound interfaces and permits several

southbound plugins beside the different versions of OpenFlow. OpenDaylight provides

means for network topology management etc.

OpenDaylight uses YANG extensively internally and while it is very rich in it support of

versions and features of OpenFlow it also supports NETCONF as southbound interface.

Since the last version OpenDaylight can be tightly integrated with OpenStack providing

management of the network resources within the Virtual Infrastructure Manager.

Brocade16 has already released a commercial product based in OpenDaylight and combines

it with its professional services offers and technical support.

OpenDaylight project plans support for SFC as part of its core control capabilities, although

currently it is not complete.

OpenMANO

OpenMANO is project released by Telefonica R&D department that comprises a VIM and an

Orchestrator. It has been used by Telefonica in a proof-of-concept17 announced and

showcased at the Mobile World Congress.

OpenMANO, as of today, is a very basic implementation not suitable for commercial

deployment but as with Open Source projects the situation may change quickly. It may turn

interesting if they focus in the orchestration capabilities and give up the VIM component

which is well covered by OPNFV, OpenStack and OpenDaylight.

16 http://www.brocade.com/downloads/documents/data_sheets/product_data_sheets/brocade-vyatta-controller-ds.pdf 17 http://www.ietf.org/proceedings/92/slides/slides-92-nfvrg-7.pdf

Alberto Diez April 2015 Understanding NFV Management

and Orchestration

24

Bibliography and References

Most relevant sources of information about MANO used:

[1] ETSI NFV portal http://www.etsi.org/technologies-clusters/technologies/nfv

[2] SDx Central NFV Report 2015 https://www.sdxcentral.com/reports/nfv-2015/

[3] OPNFV https://www.opnfv.org/news-resources/publications-collateral

[4] OpenStack https://www.openstack.org/

[5] OpenDaylight http://www.opendaylight.org/

[6] IETF SFC https://datatracker.ietf.org/wg/sfc/documents/

[7] OASIS TOSCA https://www.oasis-open.org/committees/tosca/

[8] IETF RFC 6020 - YANG https://tools.ietf.org/html/rfc6020

[9] IETF RFC 6241 – NETCONF https://tools.ietf.org/html/rfc6241

Acronyms

MANO – Management and Orchestration

MNO – Mobile Network Operator

NFV – Network Functions Virtualization

PCRF – Policy and Charging Rules Functions

SDN – Software Defined Network

SFC – Service Functions Chaining

VNF – Virtual Network Function