METRO-HAUL: METRO High bandwidth, 5G Application-aware optical network, with edge storage, compute and low Latency

H2020-ICT-2016-2 Metro-Haul Grant No. 761727http://metro-haul.eu

The Impact of the Optical Network on 5G –the Metro-Haul Project

Andrew Lord Andrew.lord@bt.com

ONDM, Athens 13th May, 2019

Contributions from many in Metro-Haul including:

Nicola Calabretta and team at TuE on novel WSS

Antonio D’Errico and Filippo Cugini and teams at Ericsson and CNIT on novel WSS

Pablo Pavon and team at UPCT on 5G network modelling using Net2Plan

Alex Stavdas and team at OLC on novel optical transport metro architectures

Talk flow

2

Metro-Haul overview

The key question – what kind of optical layer will be needed for 5G?

Lots of bandwidth to handle all applications?

And what should the nodes look like?

Lots of compute and storage at local nodes to provide low latency applications?

How will we measure all of this?

KPIs

What technologies shall we use?

Focus on some of the optical solutions and discussion

Ultimately we could do anything

With enough money, power, space…. Metro-Haul techno-economic modelling using Net2Plan

How can we prove our new technology?

Metro-Haul planned demos

Conclusions

The Metro-Haul Solution

METRO TRANSPORT

AccessMetroEdge

Metro CoreEdge Photonic

Core

Metro CoreEdge

Metro Node

Metro Node

Metro Node

Metro scope

10-20 km

0.1-0.2ms

Control / Orchestration

5G Access

50-200 km

0.5-2ms

100-1000 km

1-10ms

Access Metro Edge Node (AMEN) – multiple ubiquitous access technologies, cloud enabled (storage, compute)

Metro Transport Network – metro node: pure transport

Metro Core Edge Node (MCEN) – Larger cloud capabilities

Metro Control Plane – full orchestration

5G PPP - https://5g-ppp.eu/ - Metro-Haul Golden Nuggets

4

5G-PPP EU body overseeing all the 5G projects

Golden Nuggets are available from every project to capture the essence of the innovations from each one

GN1 - High Capacity & Flexible Metro Optical Network with Edge ComputingDynamic data plane with intelligent control plane involving multiple network segments and layers, spanning multiple geographicalData Centre (DC) locations and addressing resource heterogeneity including, notably, the optical transport. Without this data & control plane architectures, network resources supporting future 5G services would require enormous over provisioning, of both optical transport capacity across metro and core networks, and edge Data-Centre resources such as compute and storage.

GN2 - Real-Time Performance Monitoring & AnalyticsTelemetry/monitoring framework which provides a global, real-time view of the E2E network performance. This new technology enables services configuration and reliable operation. It provides pro-active actions on early detection of issues. Machine-Learning within the decision engine allows this new Metro-Haul technology to continually learn and improve as real network data is collected. It includes state-of-the-art advanced planning, placement and re-optimization/re-configuration tools, enabling holistic (joint) optimization across heterogeneous resources.

GN3 - Open Multi-Layer Disaggregated NetworkSystematic and unified approach based on model driven development for the SDN control of multilayer disaggregated and open transport networks, while allowing flexibility in deployment choices, extensibility for the integration of new technologies and agility

in migration processes without vendor lock-in.

5G-PPP KPIs

5

KPIs – Key Performance Indicators

Designed to measure the performance of the E2E solution

Designed to ensure the wide range of potential 5G vertical applications will work

Metro-Haul has converted 5G-PPP KPIs into quantities pertinent to the optical layer

KPI CategoryTarget

1Optical Point-to-Point

connection set-up time≤ 1 min

2Metro-Haul E2E Point-

to-Point connection set-up time

≤ 2 min

3Set-up time of network

service slice across Metro-Haul

≤ 1 hr

4Capacity of Metro-Haul

Controller

Control of 10 – 100 nodes (AMENs/MCENs, i.e., Open

Disaggregated ROADMs)

Optical layer time to set up or reconfigure services handling 5G applications enabled by

SDN-based management framework – includes control plane latency, optical node

reconfiguration delay, network instantiation time.

Set-up time between two VNF elements as part of a service slice. Includes packet over

optical pt-pt connection.

Time to set up a network slice as a set of interconnected VNFs

Maximum number of Netconf devices that a single SDN optical controller can support.

5G-PPP KPIs (2)

6

KPI CategoryTarget

5Fault / degradation

detection timeTo be defined

6Capacity of Metro-Haul

infrastructure

100x more 5G capacity supported over the same optical fibre infrastructure

7New optical components

/ systemsTo be defined

8

CapEx ReductionTo be defined

9

Energy ConsumtionTo be defined

Time between instant fault happens (e.g. some threshold is violated) until it is detected

Number of service instances that can be supported. Combines optical connections and

AMEN capacities (throughput, storage, compute)

New components defined in Metro-Haul

Relative cost reduction compared to baseline network to support a pre-defined set of

verticals

Reduction of energy using new node technologies such as PIC, filterless and dynamic

service infrastructure (service set-up / tear down)

Metro-Haul is an optical project – what are the optical issues?

7

Optics in the metro will need to serve thousands of 5G wireless endpoints with high bit rates (10G + )

Unlike the core network (where there is only one), this is a large distribution across all regions of a country

Distances are less than the core but cost is critical

We can’t just use core transport technologies

Core coherent transmission is far too expensive.

Core optical switching (advanced WSS based ROADMs) are far too expensive.

Fully integrated one-vendor solutions lock the solution in and this is also far too expensive as well as restrictive

But we DO need optical switching. We DO need optical networking (e.g. to pick up multiple wireless base stations on a single horse-shoe

On the other hand – distances are limited so performance is not so challenging

Under consideration:

Ericsson and TuE both developing low cost integrated WSS components

Filterless solutions also considered – e.g. DuFiNet using PON technology

Whitebox mentality – requiring SDN-based architectures to allow control and monitoring of components to allow an E2E multivendor / multitechnology dynamic solution

Tx1

Tx2

Tx3CAPS 25-50 Gb/s

PM-QPSK100 Gb/s

NRZ 25-50 Gb/s

Photonic Integrated White Box

OPEN ROADM

AgentOpen Config Agent

Agent

Agent

Agent

NETCONF

AgentOpen Config Agent

• Photonic Integrated Whitebox

• Silicon photonics chips will enable huge cost reduction

• Performance doesn’t have to match LCoS-based WSS

• 200 mm wafer realization

Rx

Several modulation formats used

Dispersion tolerant CAPS3

transmitter

Photonic adaptable

dispersion

compensator

100 Gb/s Commercial Apparatus

Open Config Agent

Open Config Agent

Open Config Agent

Ericsson and CNIT Innovation

Cell size: 4 mm x 4,6 mm

InGaAsP/InP SOAs: 950 μm gate SOA, 2 mm pre-amplifier

AWG: Channel Spacing = 2,4 nm, Free Spectral Range = 8 x 2,4 nm

9

TuE Photonic integrated 8 WDM Add/drop switch

Input

PassPre-amplifier

SOA

Booster

SOA

WSS

module

DropCh1

Ch8:

Spectral characterization of the switch

10

On/Off switching ratio > 35 dB

On-chip gain estimated >11 dB

1.00E-11

1.00E-10

1.00E-09

1.00E-08

1.00E-07

1.00E-06

1.00E-05

1.00E-04

1.00E-03

-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0

BER

Received power [dBm]

B2B CH 1 CH 2 CH 3 CH 4

. . . .

Access-Metro

Edge Node

#1

Access-Metro

Edge Node

#K

Upstream

Path

Downstream

Path

3dB coupler

mux/demux

Rx’s

DR

OP

AD

D

BM-Tx

DR

OP Rx’s

AD

D

BM-Tx

Metro Core Edge Node

* Uzunidis et al DuFiNet: A Flexible, High-Capacity and Cost-Effective Solution for Metropolitan Area Networks; ECOC 2018

DuFiNet - a filterless dual-bus Metro architecture using PON technology

Standard protocols are used

(NetConf, OpenFlow)

no extensions introduced

The LTs and ONUs, are

represented as legacy

OF/Netconf - L2 switches

Openflow for flow

management over the

PON

Broadband Forum’s

switch YANG model for

configuring queues

Overall, the Metro abstraction

was realised using YANG

modeling

Control Plane

Techno-economics – what is so hard about it?

12

Has to take a wide range of 5G scenarios with multiple KPI definitions

Has to take a geography (multiple operators in the project)

Deploy resources – optical, compute, storage to deliver 5G

With constraints –

Cost, Power, Space

Critical questions –

Do we simply over provision optical fat pipes? What is the cost penalty in doing this? Do we have the space / power in exchanges to allow this?

How do we handle latency? Do we simply over provision the local exchanges? What is the cost, space, power penalty for doing this?

Three main scenarios to be compared

Over provisioned everything – no Metro-Haul tech needed

Static DC optimisation – Slow CP management only

Dynamic DC optimisation – fully dynamic slice assignment changing during a day

All of this needs a highly advanced simulation tool

Pretty sure this is better

This will be even better – but by how

much and is it worth the hassle?

Methodology for techno-ecnonomic analysis

Reference Topology

Traffic Model

Excel Template

NIWNFV over IP over

WDM Net2Plan Library

Scenario Definition Software frameworkReusable modules /

multiple partners

Algorithms

Aut. reports

Benchmark tests scripts

• NIW library developed within MH (open-source)

• Based on abstract model of an IP over WDM network with IT resources in the nodes.

• Simplifies development of Net2Plan algorithms, automatic reports etc. for these networks.

• Simplifies import/export from defined Excel-based template.

• Publicly available: shipped with Net2Plan (www.net2plan.com)

Methodology for techno-economic analysis. Workflow example

Dense-urbanMost L < 20 kmSome 20 ≤ L < 50 km

SuburbanSome 20 ≤ L < 50 kmMost 50 ≤ L < 100 kmSome 100 ≤ L < 200 km

Aggregation node

Core node

Core node BB

L: length of horseshoe

UrbanMost 20 ≤ L < 50 kmSome 50 ≤ L < 100 km • Three reference topologies (Telecom Italia)

• Diameter <200 km, [52, 102, 159] nodes• Anonymized in node position

• With traffic information in agreement with Deliverable 2.3• Background traffic: P2P, regular, cacheable (video)• Traffic from verticals: VNF & traffic requisites from key

vertical services considered (D2.3).

Reference Topology &

Traffic

Capacity Planning

alg.

IP BOM

DC BOM

WDM BOM

Cost model

Energy model

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 20 22 24

T1 (Business)

T1 (Residential)

Time of day

Nor

mal

ized

tra

ffic Multi-hour

tests

Multi-period

tests

Methodology for techno-economic analysis. Workflow example

• JOINT capacity planning of IT & network resources• Different algs from different partners for

different problem variants / approaches• ML algorithms in resource allocation in dynamic

traffic environments• Code reuse: Algorithms can leverage on proven

routines or other algorithms

Reference Topology &

Traffic

Capacity Planning

alg.

IP BOM

DC BOM

WDM BOM

Cost model

Energy model

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 20 22 24

T1 (Business)

T1 (Residential)

Time of day

Nor

mal

ized

tra

ffic Multi-hour

tests

Multi-period

tests

RUN

Methodology for techno-economic analysis . Workflow example

Reference Topology &

Traffic

Capacity Planning

alg.

IP BOM

DC BOM

WDM BOM

Cost model

Energy model

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 20 22 24

T1 (Business)

T1 (Residential)

Time of day

Nor

mal

ized

tra

ffic Multi-hour

tests

Multi-period

tests

Automatic scripts creating Bill-of-Materials reports• IT: BOMs for different DC architectures• IP & WDM: BOMs for different capabilities of

optical equipment

Methodology for techno-economic analysis . Workflow example

Datacenter ComponentsDescription (text)

Compute Nodes

Examples: HP Proliant DL 360, Huawei 1288-v5 or

2288H-V5, Dell R740 or R940

Small

Intel Xeon Gold 6134 with 8 cores, 64 GB RAM, 600GB

HDD

Medium

Intel Xeon Gold 6140 with 16 cores, 128 GB RAM, 1.2TB

HDD

Large

Intel Xeon Platinum 8160 with 2 CPUs x 24 cores, 128 GB

RAM, 3.6 TB HDD

ExtraLarge

Intel Xeon Platinum 8160 with 4 CPUs x 24 cores, 192 GB

RAM, 3.6 TB HDD

Storage

Examples: Samsung SSD, iXSystems TrueNAS X10 2U,

HPE Nimble Storage HF40/60, Dell EMC Unity 300

SSD Small 4 TB

SSD Large 8 TB

NAS Small 20 TB

NAS Medium 60 TB

NAS Large 120 TB

NAS ExtraLarge 400 TB

Other specialized hardware

BRAS

HW Firewall eg Fortinet FortiGate 3000D

Load Balancer

eg HPE OpenCall Load Balancer, Radware Alteon. F5Big-

IP, etc

Reference Topology &

Traffic

Capacity Planning

alg.

IP BOM

DC BOM

WDM BOM

Cost model

Energy model

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 20 22 24

T1 (Business)

T1 (Residential)

Time of day

Nor

mal

ized

tra

ffic Multi-hour

tests

Multi-period

tests

MH Cost model & energy consumption model• A cost model is being produced, updating and

expanding efforts in previous Eus• Cost model includes energy consumption figures• Automatic reports are being developped to

produce cost & energy figures from the BOMs & networks.

Methodology for techno-economic analysis . Workflow example

Reference Topology &

Traffic

Capacity Planning

alg.

IP BOM

DC BOM

WDM BOM

Cost model

Energy model

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 20 22 24

T1 (Business)

T1 (Residential)

Time of day

Nor

mal

ized

tra

ffic Multi-hour

tests

Multi-period

tests

Different tests are being conducted. Scripts for bulk definition of the multiple tests can be also shared.• Multi-hour: Exploit knowledge of well known traffic profile variation along day.• Multi-period: Studies considering traffic growth. Future-proof network evolution.

Results coming soon... example question to address: how the operational benefit of having the MH Dynamic Control Plane, that on-demand allocates IP flows and VNFs, translates into CAPEX savings whan dimensioning the IT+IP+WDM metro resources?

Connection with on-demand resource allocation

19

RES

T/A

PI NFV-O

WIM Connector

OaaS(N2P Server)

DataREST/API

Oaa

S C

lien

t

OSS

RES

T/A

PI

SDN-C(Packet)SDN-C

(Packet)SDN-C

(Optical)SDN-C

(Optical)

PoP (VIM) 1 PoP (VIM) 2

NetworkInterface

NetworkInterface

VNF3VNF1 VNF2

OXC OXC

OXC

VLAN-A VLAN-B

Packet Switch Packet Switch

Parent-C OaaS Client

NIW-based routines used in techno-ec analysis can be reused in dynamic allocation algorithms in MH demos

• Optimization-as-a-Service: planning tool making on-line joint VNF & IP & WDM resource allocation decisions

• Net2Plan-OaaS module in demos, uses NIW-based algorithms for dynamic allocation.

End of project demo #1 – Crowdsourcing application

20

End of project demo #2 – video security

21

Conclusions

22

Metro-Haul nearly 2 years in

Most of the base technologies in place

Control Plane

Monitoring

Optical layer options

Disaggregation options

Techno-economic modelling with KPI constraints

Final phase demonstrations

Control Plane demonstration at EUCNC in Valencia in June

Two full project demos

BUT –

Although operators will almost certainly need more cost effective optical transport solutions in the metor

The case for disaggregation isn’t proven. Operators may go different ways depending on appetite for innovation

Local caching / compute for latency reasons will be v expensive – do those use cases really exist?

Dynamic slice handling – still VERY ambitious