cognet d7.7 contribution to standards report vfinal · 2017. 8. 21. · d 7.7 – contribution to...

D7.7 – Contribution to Standards Report

Document Number D 7.7

Status Final

Work Package WP 7

Deliverable Type Report

Date of Delivery 30/June/2016

Responsible Unit TID

Contributors TID: Diego R. López, Antonio Pastor-‐Perales

Keywords Standardization, SDO, Open source, IETF, ETSI, ONF, OpenStack, OPNFV, OSM

Dissemination level PU

D 7.7 – Contribution to Standards Report

CogNet Version final of 15

Acronyms and Definitions

Acronym Defined as

ANIMA Autonomic Networking Integrated Model and Approach

ETSI European Telecommunications Standards Institute

IETF Internet Engineering Task Force

IRTF Internet Research Task Force

ISG Industry Specification Group

NMLRG Network Machine Learning Research Group

NMRG Network Management Research Group

ONF Open Networking Foundation

OPNFV Open Platform for NFV

OSM Open Source MANO

SDO Standard Development Organization



Executive Summary

This document presents the results of the CogNet project in standardization

activities. This includes a discussion of the feasible targets for standardizing

machine-‐learning techniques in network management, how they have been

evolving during the project lifetime, and how the different original targets

described in D7.6 have been considered, and the nature and reach of the

different contributions.

Evolving from the initial intention of standardizing machine-‐learning techniques

for network management, the project contributions have focused on

standardizing interfaces and facilitating integration for open data-‐centric

management. The contributions have been structured along three main lines, as

described in the document:

• The applicability of the CogNet open data-‐centric management

principles and results.

• Open data sources and formats.

• Open action streams for policy-‐based management.



Table of Contents 1. Introduction ............................................................................................................................ 5

2. The Evolution of Standardization Targets ................................................................................ 6

3. Applying Principles and Results ............................................................................................... 9

4. Open Data Streams ............................................................................................................... 11

5. Open Action Streams ............................................................................................................ 12

6. References ............................................................................................................................ 14



1. Introduction

Standardization activities were identified as one of the key ways of achieving a high impact of the results and of demonstrating the technical progress of the project, especially given that the application of Machine Learning techniques to network infrastructures was becoming a realistic approach for actual network operation, beyond the initial steps of exploratory research within industry and academia. The project has always understood standardization in the widest possible sense, being aware of the new standardization mechanisms offered by open source projects, beyond the use of open source software as the base for project development and the further distribution of project results.

As the project evolved, and the interest in cognitive techniques grew among the standardization organizations, the CogNet team became more active in this process, though it became progressively clear that a traditional standardization path for the application of cognitive techniques to network management was unfeasible, and therefore the project re-‐focused its goals. CogNet has concentrated its efforts on standardizing interfaces and facilitating integration for open data-‐centric management, without pursuing a rigid architectural approach, in an inclusive strategy able to facilitate the application of data-‐intensive evidence to network management.

The rest of the document discusses the rationale for this approach, and how it evolved with the direct action of the CogNet partners in different SDOs, and how the contributions of the project have been structured along three main lines that define the structure of the following sections of the document:

• The application of the CogNet open architecture, based on a double closed control loop of data-‐centric management interfaces.

• The application of open data sources and formats in the data streams of this double closed control loop.

• The use and evolution of standards for policy-‐based management in the action streams of this double closed control loop.



2. The Evolution of Standardization Targets

At the beginning of the project, the application of machine learning techniques to network management was still perceived mostly as a research activity by the different standards communities, and several activities at the research level, specifically in the ETSI NFV ISG and the IRTF were launched. In particular, the start of a prospective research group within the IRTF, the Network Machine Learning Research Group (NMLRG), was a clear indicator of this interest. The CogNet partners contributed actively in these initial stages, building awareness on the CogNet activities, reporting results (especially in what relates to the architecture that was being consolidated by those days), and identifying future collaborations in which CogNet could provide feedback to the bodies dealing with some of the technologies at the CogNet basis.

But from these initial contributions, and the direct exchange with the participating communities, it became clear that a traditional standards path approach to apply cognitive methods to network management, including the definition of an architecture or reference framework, and the identification and specification of “cognitive modules” in such an architecture, was not the right approach, especially in the light of the concerns about architectural and protocol ossification of networks, and given the lack of traction of previous related attempts like GANA [1].

Cognitive management methods can be considered a particular case of general data-‐driven management techniques, able to support a wide variety of autonomous and semi-‐autonomous behaviours by consuming operational and situational data, applying analysis mechanisms to them, and taking action according to this analysis. These correspond to the well-‐known closed control loop common to Control Theory [2], and whether the analysis phase is performed by means of elements that are fixed or able to evolve by self-‐tuning is something that is immaterial in terms of the basic architectural principles around the closed control loop.

Figure 2-‐1: A canonical closed control loop, well established in general control theory



This is one of the core findings within the NMLRG that led to its quick closing, subsuming some of its main goals into the existing NMRG (on Network Management) and the ANIMA IETF WG (focused on protocols and data models to express autonomous behaviour). And it was this that made the CogNet partners refocus the project standardization contributions towards other targets, better suited to the relevant results CogNet could bring. A new, related proposal appeared during this period, the ETSI ENI ISG [3], that seemed to be addressing the same original goals as CogNet. The partners present in ETSI (and especially in the NFV ISG) participated actively in the discussions about the goals and methods of this group, expressing the conclusions of previous experiences, and suggesting the proponents to join forces in the directions described in this document. The ENI proponents decided to only pay partial attention to this appeal, and they followed up with the creation of the group, with a set of terms of reference that does not clearly define what we think are attainable and realistic goals, and therefore without the participation of any of the CogNet partners. Nevertheless, the project continues monitoring the evolution of the ENI ISG, to identify opportunities for convergence.

The most salient characteristic of the CogNet architecture is the creation of a double closed control loop with common interfaces in each one. One control loop is the “classical” one, where CogNet proposes to apply cognitive control to an integrated NFV/SDN Software Network architecture, which provides the data to be fed into the cognitive control module. The other control loop is the machine-‐learning one, where different algorithms and modules can be applied in order to shape (and control) the cognitive controller of the first loop, using a set of data derived from the same sources as the classical loop. The cognitive controller, at the top of one loop and the bottom of the other, is the nexus between them.

Figure 2-‐2: The basic CogNet architecture, showing the double closed-‐control loop

CogNet Solution

Network Management Existing Solutions

NFV/SDN-based Environment

Data Collector

Policy Engine

DataStream

ScoresPolicies

Data Stream

DataStream

Rea

l Tim

e En

gine

Cog

Net

Sm

art E

ngin

e

Cog

Net

D

ata

Col

lect

orC

ogN

et

Polic

y En

gine

Batc

h En

gine



While, as said above, it does not seem reasonable to try to standardize such an architecture, as it would become an over-‐specification that would either constrain further evolutions or directly ignored, there are two important aspects related to the data and action flows in the double control loop that require the application (and further feedback and contribution) of standard approaches: open mechanisms and formats for publishing, routing, and consuming data streams, and open protocols and models to express actions. And, in addition to these two aspects, the application of the CogNet architecture is able to provide further insights in the requirements and use of cognitive methods in particular application domains, suitable for contribution to standardization activities in the broadest sense mentioned above.



3. Applying Principles and Results

The CogNet architecture is directly based on the SDN/NFV integration originally proposed by the ETSI NFV ISG in its EVE005 document [4] that was produced in coordination with ONF analysis in [5]. The CogNet team has confirmed the applicability of the EVE005 findings and recommendations, and provided additional feedback to the EVE (Evolution and Ecosystem) WG within ETSI NFV. This feedback has been consolidated in the recent contributions on slice orchestration and slice isolation within the new work that the EVE WG has started around the applicability of NFV orchestration principles to network slicing [6]. The evolution of the ONF towards a reduced standardization activity and a much stronger focus on open-‐source development within the ONOS platform [7] has discouraged further contributions from the CogNet partners to the ONF regarding this issue.

Figure 3-‐1: Mapping between the CogNet architecture and EVE005 SDN/NFV integration

Beyond the application statements on the architectural principles on which CogNet is based, and following the philosophy described in the previous sections about avoiding over-‐specification and focusing contributions on practical aspects of network management, the team has sought direct collaboration on the application of the CogNet results to standardization and open-‐source initiatives, in particular:

• The application of CogNet cognitive models able to identify different root causes for VNF performance degradation, contributed to OPNFV and OpenStack through the Vitrage initiative. Within OPNFV, this is related to the Doctor project, a fault management and maintenance framework for high availability of network services on top of virtualized infrastructure [8].

• The integration of CogNet-‐based cognitive capabilities for NFV orchestration as part of the OPNFV Orchestra proposal, focused on the integration of the open source Open Baton platform, with existing OPNFV projects for specific scenarios and use cases [9].

• The future OSS/BSS initiative within the TMForum [10], that acknowledges the need of business and operational processes driven by data analytics, as the service levels and flexibility required to compete cannot be achieved with offline manual analysis and

MANO Block

Resources and Services Managment

NFVi

Tenant Controller

EM(s)NFVO

VNF Manager(s)

Virtualized Infrastructure Manager(s)

VNF(s)

Virtualized Infrastructure

Virtualization Layer

Physical Hardware Resources

Compute Storage Network

OSS/BSS/VTN

Servcie, VNF and interface descriptions

SDN Controller

LCSE

Proxy

Policy Engine

Policy Repository

Optimizer(Optimisation)

FunctionsGeneration

Policy Distribution

Policy Adaptor

Policy Publisher

Functions/Policies

Policies

Policies Generation

(Semi-automated)

Policies Recommender

Recommen-dations

Policies

DataStream

Data Collector

(Near) Real-time Processing Engine

Batch Processing Engine

Data Pre-processing

User inputs

Data Normalisation, Transformation

Adaptors

Feature Selection and Extraction

Distributed File System

Measurements (Network and Infrastructure)

Automated Model Selection

Dataset Analysis

Comparative Analysis

ModelSelection

Data Storage

Frozen Data

StreamGenerator

(Semi) Supervised & Unsupervised ML

Distortion Evaluation

Model Training

Online Learning and Scoring or Score using model from Batch Layer

Model Training

Pull Model ModelScoring

Model Scoring

Data Cleaning, Filtering, Feature

Extraction

Hardware AnalysisUser

inputs

User inputs

E2E inputs

Action Parameters

Thresholds &Events Values

Event Instance Name (Events Values)

Target Parameters



adjustment. A first contribution in this direction has been made through the Open Source MANO community [11]



4. Open Data Streams

The data streams in the CogNet double closed-‐loop architecture have been implemented in the common infrastructure using open-‐source elements, that allows to provide feedback from CogNet to the relevant upstream projects: OpenStack Monasca, Apache Kafka, and the relevant monitoring activities in OPNFV and OSM. At the time of this writing, no other contributions than application statements are available for these projects, but the consolidation of the common infrastructure and the demonstrators running on it may well bring opportunities for more significant feedback to the upstream projects.

The CogNet project has struggled with the lack of usable data sets since it started, and developed the concept of an open environment for the generation of significant synthetic datasets, suitable to be applied to the different use cases in the project, and generalized to further application of data-‐driven network management.

Figure 4-‐1: Synthetic dataset generation at CogNet’s “Mouseworld”

The concept has been introduced at the recent ETSI Summit on 5G Network Infrastructure [12], and brought into several other research projects within the H2020 programme. To produce the datasets, the team has analysed several open formats and open source tools for the generation of network flow information, and finally selected tstat [13]. We are currently working in specific extensions to tstat, intended to improve the quality of the training synthetic datasets, that will be contributed to the tool development community.

Apps&clients CSP&Internet

Network

WebServer

CloudFileProvider

Videostream

Illegal

Cloud

Video

Browser

Illegal

VNFs

Browsing

CloudVideo

Illegal

OSSManageabletraffic

??

App1

App2

Unknown

OSS

Manageabletraffic

App1

App2

…..

MLAlgorithms

MouseWorld

HoneyPot

Attacks



5. Open Action Streams

One of the key challenges for CogNet was the selection and application of an open and extensible mechanism to specify the control actions. Almost in parallel with the start of the project, the IETF launched its SUPA WG [14], focused on the definition of information and data models to communicate network management policies. After some initial discussions with the main proponents and chairs of the WG, CogNet decided to embrace SUPA, and that is the way in which control actions (of whatever nature, from the identification of an event to the request of concrete activity to one of the control and orchestration elements) are communicated within the CogNet action streams. The project has monitored and contributed to the SUPA effort since this decision was taken, collaborating in the refinement of the information and data models, and the consolidation of the SUPA solution with the introduction to the applicability of SUPA to support the data-‐driven network management control loop. The work of the SUPA WG is going to conclude likely by the end of 2017, that brings an interesting parallelism between the lifespan of CogNet and the standards activity the project has been most directly involved with.

Figure 5-‐1: The SUPA policy framework

The project team has followed, reviewed, and contributed to several standardization activities connected with some of the project use cases, defining network management semantics for these areas. In particular:

• The IETF I2NSF WG [15], dedicated to specify a model for the operation and management of network security functions.



• The ITEF I2RS WG [16], committed to define a programmatic interface to network routing systems.

• The ONF and OIF activities around the definition of a Transport API (TAPI) [17][18], a unified approach to manage optical networks.

The CogNet action streams essentially transmit policy statements to the policy optimizers and proxies in charge of requesting the control actions at all layers. This implies the integration of policy-‐based management and orchestration in the current frameworks proposed for implementing the Software Network approach. The project has taken the opportunity of the growing interest on policy-‐based management in the ETSI NFV ISG and contributed to the document IFA023 (“Report on Policy Management in MANO”) [19], aimed at developing the use cases of applying policy framework in the NFV MANO functionality, and analyzing the potential impacts of policy management on the MANO architecture and work flows. The IFA023 document is about to be finally approved and published by ETSI NFV at the time of this writing. In addition, the SEC WG has started the recently approved SEC017 work-‐item [20], focused on identifying potential use cases of NFV security policies design, and to identify the types of information to be included in security policies for those use cases. The CogNet team has participated in the discussions to charter the new work-‐item, and plans to contribute the CogNet experience in security applications, even beyond the project lifetime.



6. References [1] ETSI AFI, “Autonomic network engineering for the self-‐managing Future Internet (AFI); Generic

Autonomic Network Architecture (An Architectural Reference Model for Autonomic Networking, Cognitive Networking and Self-‐Management)” http://www.etsi.org/deliver/etsi_gs/AFI/001_099/002/01.01.01_60/gs_afi002v010101p.pdf

[2] M. Barr, “Introduction to Closed-‐Loop Control”, Embedded Systems Programming http://www.embedded.com/story/OEG20020726S0044

[3] ETSI ENI ISG, “Experiential Networked Intelligence” https://portal.etsi.org/Portals/0/TBpages/ENI/Docs/ISG_ENI_presenatation.pdf

[4] ETSI NFV ISG, “Network Functions Virtualisation (NFV); Ecosystem; Report on SDN Usage in NFV Architectural Framework” http://www.etsi.org/deliver/etsi_gs/NFV-‐EVE/001_099/005/01.01.01_60/gs_NFV-‐EVE005v010101p.pdf

[5] ONF, TR-‐518: “Relationship of SDN and NFV” https://www.opennetworking.org/images/stories/downloads/sdn-‐resources/technical-‐reports/onf2015.310_Architectural_comparison.08-‐2.pdf

[6] ETSI NFV ISG, “Details of DGR/NFV-‐EVE012 Work Item” https://portal.etsi.org/webapp/WorkProgram/Report_WorkItem.asp?WKI_ID=51391

[7] ONF, “Open Networking Foundation and ON.Lab to Merge to Accelerate Adoption of SDN” https://www.opennetworking.org/news-‐and-‐events/press-‐releases/3194-‐open-‐networking-‐foundation-‐and-‐on-‐lab-‐to-‐merge-‐to-‐accelerate-‐adoption-‐of-‐sdn

[8] OPNFV, “Doctor Project: Fault Management” https://www.opnfv.org/community/projects/doctor

[9] OPNFV, “Orchestra Project” https://wiki.opnfv.org/display/PROJ/Orchestra

[10] TMForum, “Building the OSS/BSS of the future” https://www.tmforum.org/future-‐ossbss/

[11] D. Lopez, “Open Source MANO. The open-‐source approach to build a reference MANO stack” http://www.tmforumlive.org/wp-‐content/uploads/2016/05/1440-‐LOPEZ.pdf

[12] ETSI Summit on 5G Network Infrastructure http://www.etsi.org/news-‐events/past-‐events/1148-‐etsi-‐summit-‐on-‐5g-‐network-‐infrastructure

[13] “TStat – TCP Statistic and Analysis Tool” http://www.tstat.polito.it

[14] IETF, “Simplify Use of Policy Abstractions (SUPA)” https://datatracker.ietf.org/wg/supa/about/



[15] IETF, “Interface to Network Security Functions (I2NSF)” https://datatracker.ietf.org/wg/i2nsf/about/

[16] IETF, “Interface to the Routing System (I2RS)” https://datatracker.ietf.org/wg/i2rs/about/

[17] ONF, TR-‐527: “Functional requirements for Transport API” https://www.opennetworking.org/images/stories/downloads/sdn-‐resources/technical-‐reports/TR-‐527_TAPI_Functional_Requirements.pdf

[18] L. Org, “OIF SDN Transport API NFV Proof of Concept” http://www.oiforum.com/wp-‐content/uploads/L123-‐NFV-‐WC-‐OIF-‐NFV-‐PoC.pdf

[19] ETSI NFV ISG, “Network Function Virtualisation (NFV); Management and Orchestration; Report on Policy Management in MANO” https://docbox.etsi.org/ISG/NFV/Open/Drafts/IFA023_Policy_Mgmt_in_MANO_report/NFV-‐IFA023v080.zip

[20] ETSI NFV ISG, “Details of DGR/NFV-‐SEC017 Work Item” https://portal.etsi.org/webapp/WorkProgram/Report_WorkItem.asp?WKI_ID=52906

cognet d7.7 contribution to standards report vfinal · 2017. 8. 21. · d 7.7 – contribution to...

Documents