how to use machine learning for testing and …€¢ dynamic networking requires sophisticated...

1 © Nokia 2016 Public 2017

How to use Machine Learning for Testing and Implementing Optical Networks

Jesse Simsarian, Steve Plote, Marina Thottan, and Peter J. Winzer October 4, 2017

2 © Nokia 2016

1.  Introduction •  Machine learning applications and algorithms •  Network operating system •  Network abstractions

2.  Sensors •  OSNR •  Fiber polarization sensing

3.  Algorithms •  QoT estimation optimization for margin reduction

4.  Actions •  Error-aware rerouting - SLA to set error tolerance

5.  Challenges

Outline

Public 2017

3 © Nokia 2016

•  Ubiquitous cloud services will drive bandwidth and dynamism •  Computing resources instantiated in minutes •  Comparable flexibility presently not possible in IP/Optical networks

•  Today: Largely quasi-statically re-configurable components •  Flexgrid ROADMs, flexible bitrate transponders, tunable lasers, FlexEthernet, …

•  Dynamic networking requires sophisticated Network Operating System (OS) •  Manage and allocate resources for optimal network capacity, security and reliability

•  Need to be able to physically sense and control the optical network

•  Advanced machine learning algorithms on top of the Network OS •  Dynamic Networking = Sensors + Abstraction + Algorithms + Actuators

•  Communications industry efforts: •  Optical Internetworking Forum (OIF): SDN for transport networks

•  Open Daylight controller (ODL)

•  Open Network OS (ONOS): Open-source NetOS from ON.Lab, Linux Foundation

Introduction

Public 2017

4 © Nokia 2016

Foundations of Machine Learning Applications

Public

Data Center 1

Data Center 2

Flexible, tunable network

Programmable network operating system (netOS)

Machine learning applications IP/Optical grooming

Optical margin

reduction

Optical network defragmentation

sensors actuators

sensors + abstraction + algorithms + actuators algorithms

2017

OSNR Sensor

abstraction

Polarization Sensor

5 © Nokia 2016

T. Mitchell, Machine Learning, McGraw Hill (1997) “A computer program is said to learn

•  from experience E •  with respect to some class of tasks T •  and performance measure P

if its performance at tasks in T, as measured by P, improves with experience E.”

Algorithms Machine Learning

Public

Artificial Neural Networks

Numerical Optimization

https://www.youtube.com/watch?v=qv6UVOQ0F44

2017

MarI/O: Artificial Neural Network

http://www.benfrederickson.com/numerical-optimization/

W2

W3

WN

W1

x > T ?

I1

I2

I3

IN

I1

I2

Ip

…

W11

Wp1

W12

Wp2

W1q

Wpq

…

O1

O2

Oq

…

Σ >

> Σ

Σ >

Inputs Outputs

O

x T

O

6 © Nokia 2016

Dynamic and Flexible Networks

Public 2017

Data Center 1

Data Center 2

Data Center 3

IP Layer

Optical Layer

Flex Ethernet client interface

Bandwidth variable transponders and channels

Flexgrid ROADMs

Disturbance /Impairment

7 © Nokia 2016

WDM Transmission Capacity and Flexible Optical Transponders

Get the Most Capacity out of Your WDM System

5

10

15

2 100 1,000 10,000

Transmission distance [km]

Spe

ctra

l effi

cien

cy [b

/s/H

z]

25

50

75

10

C-b

and

capa

city

[Tb/

s]

PSE

2S (3rd gen. coherent, 2016)

Public

Bell Labs 1 Tb/s Field Trial

400G

250G

200G

100G

2017

8 © Nokia 2016

NetOS: The Network Brain

The network brain is a cognitive global network control plane

Network state

Application needs Network resources capacity

latency

security

reliability

coordinate

schedule

control

2017

9 © Nokia 2016

NetUNIX NetOS Functions

Network Config.

Dynamic intents

Network Resource Manager

Network Events

Network Topology

Performance Data

Intent Framework

Prototype NetUNIX system designed and demonstrated

Continuous testing fram

ework

2017

10 © Nokia 2016

•  Advanced algorithms on an vendor-independent, abstract network representation •  What information is needed? How many data points are needed? •  Joint IP/Optical network grooming and optimization - IP/Optical network topology is needed •  Optical margin reduction - optical network OSNR data is needed

•  The Internet Engineering Task Force (IETF) •  YANG data modeling language to model configuration of network elements

•  OpenROADM (AT&T, Ciena, Fujitsu, Nokia, SK Telecom…) •  Defines interoperability specifications for ROADMs, transponders, pluggable optics •  Includes YANG data models for devices, network, services

•  OpenConfig (Google, AT&T, Microsoft, BT, Facebook…) •  YANG data models for optical transport: terminal optics, wavelength router, optical amplifier

•  OpenFlow 1.4 (Open Networking Foundation) •  Common southbound interface definition for multi-vendor network element control

•  NetGraph - mathematically-rigorous, expresses multilayer network complexity •  Does not replace other efforts, adds explicit layering and mathematical rigor to network model

Public

Data Models, Network Abstraction

2017

11 © Nokia 2016 Public

NetGraph Primitives Underlying Mathematical Model

+

2

Southbound(discover, monitor, program)

Distributed Core(network state management)

Northbound(service intent framework)

Application(e.g., multi-layer network optimization)

a) b)

Multi-Vendor IP/MPLS/Optical Network Elements

primitiveslink

(e.g. fiber)adapter

layer 1

layer 2

1 3

connect

paths

bind

s t s ts t

link

NetGraph

S. Fortune, “Equivalence and generalization in a layered network model”, J. Comput. Syst. Sci., vol. 81, no, 8, pp. 1698–1714, 2015.

2017

Primitives Link

(e.g. fiber) Adapter

Layer 1

Layer 2

Connect

Path

Bind Link

12 © Nokia 2016

NetGraph abstract network representation maps to network resources

Implementation: Adaptation to Network Elements

Public

PXCsOF Switches

DeviceMgr

LinkMgr

FlowMgr

REST/GRPC

1830PSSs

Telnet/SSH

WiFi APs

NokiaNE Proxy

REST

OpenFlow

Device/Link/Host/Flow Providers

HostMgr

NetGraphMgr

Intent Mgr

ResourceMgr

DeviceDrivers

Network-‐level granularity

Device-‐level granularity

Device-‐specificadaptors

Protocols

Event bus

SAM-‐ SOAP

2017

Populate network elements into NetUNIX Gather statistics

Control network elements TCP

REST

Optical Transport

13 © Nokia 2016

1.  Introduction •  Machine learning

•  Network operating system

•  Network abstractions

2.  Sensors •  OSNR

•  Fiber polarization sensing 3.  Algorithms

•  QoT estimation optimization for margin reduction 4.  Actions

•  Error-aware rerouting - SLA to set error tolerance

5.  Challenges

Outline

Public 2017

14 © Nokia 2016

•  Single photodiode eye diagram machine learning analysis: •  J. Thrane, J. Wass, M. Piels, J. C. M. Diniz, R. Jones, and D. Zibar, “Machine learning techniques for optical performance monitoring from directly

detected PDM-QAM signals,” Journal of Lightwave Technol., vol. 35, no. 4, 2017.

OSNR Sensing with an Artificial Neural Network

Public

Artificial neural network used to estimate OSNR from eye diagram

2017

Measurable Input Output Unknown transformation

Target OSNR Function of eye diagram variance and weights

15 © Nokia 2016

Fiber as a Sensor

Optical fiber network becomes a massively-distributed motion detector

2017 Public

16 © Nokia 2016

OSC for Polarization Sensing – Span Monitoring

OSC SFP

OSC SFP

Direction 1

Direction 2

OA

OA

PBS

Fiber

Fiber

D1

D2

Amplifier Cards

OF

OF

OSC SFP

OF

OF

+-

Processing

+-

OSC data channel

Polarization monitor signal

2017 Public

OSC OSC OSC

… Rx

OA OA

PBS

PDTIA

1 mm

PD

TIA

V

H

17 © Nokia 2016

Experiment

Pol.Analyzer

CW Laser

D1

D2PBS

Scope

Hammer

Test Fiber7 cm

Pol.Synth.

Adaptif A3300

Ch. 1

Ch. 2

Trig.

2017 Public

( )( ) scopeArmS SR

VVVVPR ⎟

⎟⎠

⎞⎜⎜⎝

⎛

+

−Δ= −

21

2112, 5.0sin2

3 Results for Comparison

( ) analyzerfullA SRSSSPR 23

22

21

1, 5.0sin2 Δ+Δ+Δ= −

( )( ) analyzerArmA SR

PPPPPR ⎟

⎟⎠

⎞⎜⎜⎝

⎛

+

−Δ= −

21

2112, 5.0sin2

Full analyzer measurement

“2-Arm” analyzer calculation

2-Arm scope measurement

18 © Nokia 2016

Results Summary

2017 Public

58 Fiber Impacts

Figure of Merit

22.058.0

22.062.0

)max()max(

2,

2,

,

2

±=

±=

=

ArmS

ArmA

fullA

Arm

F

F

PRPRF

90% > 0.25

J. E. Simsarian and P. J. Winzer, “Shake Before Break: Per-Span Fiber Sensing with In-Line Polarization Monitoring,” in Proc. OFC 2017, paper M2E.6, 2017.

19 © Nokia 2016

F. Boitier, V. Lemaire, J. Pesic, L. Chavarría, P. Layec, S. Bigo, E. Dutisseuil, “Proactive fiber damage detection in real-time coherent receiver,” ECOC 2017.

Coherent Transponder Polarization Sensing – Path Monitoring

Public

Machine Learning Event Classification

2017

controller

Event classification: Y = f(X)

21 © Nokia 2016

Why and When do We Need to “Learn from Experience” (As opposed to “knowing what we are doing”)

•  Good models of the underlying physics are generally best •  Subsystem: Chromatic dispersion compensation in a coherent DSP •  System: Predicting physical-layer performance through adequate models •  Network: Rule-based (x-layer) routing and wavelength assignment

•  But: How good / comprehensive is the respective model ? •  Hybrid approach: physical model + parameter tuning with machine learning

- A dB in system planning accuracy is as valuable as a dB in coding gain!

FEC threshold

Q fa

ctor

Predicted performance

Wavelength

Actual performance

Margin

Baseline planning tool

FEC threshold

Q fa

ctor

Predicted performance

Actual performance

Margin

Wavelength

Improved planning tool

FEC threshold

Q fa

ctor

Actual performance

Margin

Wavelength

“Supervised learning” (TRX feedback)

2017

22 © Nokia 2016

Optical System Margin Reduction

Public

E. Seve, J. Pesic, C. Delezoide and Y. Pointurier, “Learning process for reducing uncertainties on network parameters and design margins,” in Proc. OFC 2017, paper W4F.6.

2017

Learning process to reduce uncertainties Lifecycle

Re-training

Learning Algorithm

SDN data base

24 © Nokia 2016

SDN Testbed at Bell Labs

Public

•  Optical switching inside data center

•  PXC MEMS switch

Photonic Crossconnect (PXC)

TOR Switches & Servers

AirFrame POD AirFrame Rack

Executive Briefing Center

IP Transformation Center (IPTC)

•  Error-aware networking

•  Optical Transport

•  Service Routers

•  SDN Switches

Service Routers

Optical Transport NetUNIX

Fiber Connectivity

2017

25 © Nokia 2016

Error-Aware Rerouting

Public

Host 1Host 2

Switch 1

Switch 2

Switch 3

Transport 2

Transport 3

Host 3Host 4

Transport 1

Host 1Host 2

Switch 1

Switch 2

Switch 3

Transport 2

Transport 3

Host 3Host 4

Transport 1

Optical Transport

SDN Switch

2017

Regular intent Error-aware intent

26 © Nokia 2016

•  Stability and reliability •  Event classification probability

•  How much data is “enough” as a basis for machine learning? •  What data is needed at which layer of the network? (abstraction)

•  Machine learning algorithms local to network element for sensor data

•  Reliability and operational implications •  Human operator confirmation of network operations

•  Business structures - IP and optical businesses

•  What are operational barriers to more efficient and dynamic network operation?

•  Outages and accountability – applications on top of NetOS

Challenges and Discussion Points for Network Operators

Public 2017

*J. Thrane, J. Wass, M. Piels, J. C. M. Diniz, R. Jones, and D. Zibar, “Machine learning techniques for optical performance monitoring from directly detected PDM-QAM signals,” Journal of Lightwave Technol., vol. 35, no. 4, 2017.

Classification of Modulation format

27 © Nokia 2016 Public 2017

•  Cloud services as drivers of bandwidth and network dynamism •  Dynamic networks = Sensors + Abstraction + Algorithms + Actuators •  Machine learning algorithms to identify network events, reduce optical system margins, and optimize

the network •  Built on programmable NetOS and flexible network infrastructure

•  Different machine learning methods ranging from “black box” artificial neural networks, to event classification, to hybrid methods using a physical model of the system and machine learning

•  NetUNIX prototype NetOS •  Improve sensing of the physical layer to detect impairment or possible disruption

•  Abstract representations of the network layers with NetGraph

•  Applications •  Impairment aware IP/optical re-routing

•  Optical channel monitoring

Conclusions

how to use machine learning for testing and …€¢ dynamic networking requires sophisticated...

Documents