mozart - cisco.com · mozart : cloud services platform for service providers network function...

Mozart : Cloud Services platform For Service Providers Network Function Virtualization

Rex Fernando, Lead Architect, Distinguished Engineer, CVG

Lakshmi Sharma, Director Engineering, CVG

Cisco and/or its affiliates. All rights reserved.

Key SP challenges and Path Forward

TRADITIONAL

SP

Traffic

Growth

Network

Cost and

Complexity

Time to

Market

Competitiv

e Pressure

Slow

Innovation

Lean

SP

Rapid

Innovation


TRANSFORMATION TO CARRIER AS A SERVICE

NFv + SDN +

Physical

E2E Service

Orchestration

Self

Service

Carrier

Class

Carrier

As A

Service

MOZART


MOZART Defined

… an open, standards-based, modular architecture and

platform for services orchestration

… manages the physical & virtual network elements,

both Cisco and 3rd party, as well as the compute &

storage infrastructure to deliver carrier-class services

… that range from VPC to NFV

Cisco Confidential 6 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

MOZART- End to End Architecture for SP Cloud Services

Physical Network SP METRO/ ACCESS SP Core

SP DC

Cisco Confidential 7 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

MOZART- End to End Architecture for SP Cloud Services

Physical Network SP METRO/ ACCESS SP Core

SP DC

End-to-End Carrier Class Service Orchestration

Virtual Network, Compute &

Storage

vCPE, vEPC SDN Controller CSR, ASA, XRvr, BNG, cDVR

Service Catalog

Routing / VPN Mobility Video/ Content Security Virtual Private

Cloud Managed Services

Transport

Service Catalog “Business Intent”

catalogs

Cross Domain Orchestration

Network Orchestration - DC SDN

- WAN Orchestration & Optimization

Service Provisioning (Physical & Virtual

Services) - Provision WAN services

- Provision NFV in DC

Compute & Storage Control - Elastic Services Control

- Service Lifecycle management

Orchestration Engine

“Execution” configuration,

workflow, automation, provisioning C

arr

ier

Cla

ss R

eliability

and H

igh A

vailability


SP-WAN

Customer

VPC

SP-WAN

Customer

SaaS Cloud

Transit-NFv

SP-WAN

Customer

Legacy Services

R-a-a-S

vBRAS

Mozart - “A flexible multi-tenanted cloud services orchestration platform for the virtualized data center”

NETWORK AND APPLICATION CONNECTIVITY MODELS

WHAT CAN BE VIRTUALIZED?

3-tier Apps, Web Servers, DB Servers, Hadoop Clusters,

Distributed Storage

DPI, Firewall, NAT, Load Balancers,

WaaS, GI-LAN Applications

IPSec Gateways, SSL VPNs,

vEPC Applications

DNS, Routing, BRAS, NTP

ENTERPRISE APPS

TRANSIT NFV

TERMINATE NFV

NETWORK SERVICES


Transit NFv Examples

• Provide internet connectivity for VPN customers and apply NAT and Firewall policies per customer.

• VNF = CSR/ASA1kv/3rd party per customer VRF instance

• Provide remote branch of an enterprise with ability to access headquarters over a secure tunnel using IPSec


• Map IPSec tunnel to a enterprise VRF

• Provide connectivity between 2 different enterprise VPNs. Apply firewall policies and translate addresses.


• Use case based on many tier 1 SP customer requirements


Model Driven Service Specification

A M LS

App Zone Ext Zone Mgd Zone Segment Plug Logical Service

RAN Network Enterprise

Network

Service Chain

EPC

(SGW, PGW – SI, DI)

S1 S2 S3

S4 S5

Gi-LAN Service

M

L

S

L

S

L

S

L

S L

S

vEPC +

Gi-LAN

L

S

L

S

Managed

Services

Web DB

Firewall Service

L

S A L

S A VPC

Vrf=Coke

Vrf=Pepsi

Vrf=Cisco

Vrf=Amazon


vFW

Public Zone

(DMZ) Protected FE Zone 1 Zone 2 Zone 3

S

u

b

-

Z

o

n

e

W

S

u

b

-

Z

o

n

e

X

S

u

b

-

Z

o

n

e

Y

S

u

b

-

Z

o

n

e

Z

Front-end Zones

L3 VPN

Internet

Back-end Zones

vLB vLB vLB

vIPSec GW

Virtualized Compute and DC overlay

Data Center Evolution

Public Internet

L3 VPN

Server VLANS

Firewall Outside VLAN

Firewall Inside, ACE outside VLAN

AS 65522

R

P

R

P

AS 109

ASA-

VPN Front-end VRF

Back-end VRF

Shared/public VLAN

ASA-

FW

WAN Edge

(NGN PE)

Aggregation

Service-

Core

Aggregation

Compute

ASA-

VPN

ASA-

FW

eMBGP

VRF

VRF

eBGP + static redist.

VRF *

*

*

*

*

*

*

*

*

* Tennant L3

Edge (VRF-CE)

L3 VPN Edge

(DC-PE)

L3 VPN Edge

(NGN-PE)

Legacy DC Mozart DC

Agility (Create/Delete), Scale, Flexible Topologies, BYOD, Elasticity,

Utility Based Pricing


Architectural Goals

Mozart Architecture

Open Systems Design

(Inclusive)

Customer Experience

(Agility, Opex)

Deployments (Variables)

NG-SP-DC Requirements (Constants)

LARGE NUMBER OF SERVERS

AND VMs

MULTI-TENANCY

HIGH BISECTIONAL BANDWIDTH

OPTIMAL L2 and L3

FORWARDING

NETWORK

VIRTUALIZATION

SLA ASSURANCE

AVAILABILITY AND

RELIABILITY

SEAMLESS DCI

INTEGRATION

NETWORK SERVICES

ELASTIC SCALING

WORKLOAD MOBILITY

MULTIPLE CONNECTIVITY

OPTIONS

PHYSICAL DEVICE INTEGRATION


Architectural Goals

Mozart Architecture

Open Systems Design

(Inclusive)

Customer Experience

(Agility, Opex)



THIRD PARTY SWITCHES, SERVERS AND

DCI

DIFFERENT PHYSICAL UNDERLAY

CONNECTIVITY

OPENSTACK SUPPORT

CEPH & NETAPP SUPPORT

BARE METAL INTEGRATION

3RD PARTY NFVS

L2 VPN, L3VPN, INTERNET FOR WAN

ACCESS

ENCAPSULATIONS (VXLAN, MPLS-O-GRE,

L2TPV3)

SERVICE TOPOLOGIES

TENANT APPLICATIONS

IDENTITY MANAGEMENT

ADMINISTRATIVE

SEPARATION

Easy integration into existing

OSS systems

BROWN FIELD DEPLOYMENTS


Architectural Goals

Mozart Architecture

Open Systems Design

(Inclusive)

Customer Experience

(Agility, Opex)



YANG MODELS

REST, HTTP, RESTCONF

BGP, IGP

ETHERNET/IP

MPLS-OVER-GRE, VXLAN, MPLS-OVER-

UDP, L2TPV3

L3VPN & L2VPN INTEGRATION

OVF, QCOW2, VMDK

LINUX/UBUNTU

OPENSTACK/KVM

NAGIOS

MYSQL

GANGLIA

PUPPET & COBBLER

JUNIPER AND ALU DCI

PLUGGABLE DHCP AND DNS

AGENTLESS NFv

3RD PARTY SERVERS – IPMI/PXE

3rd party NICS (DPDK)


Architectural Goals

Mozart Architecture

Open Systems Design

(Inclusive)

Customer Experience

(Agility, Opex)



AUTOMATED INSTALLER

AUTOMATED UPDATES

ISSU

SCALE OUT

MODULAR POD

APPLICATION CENTRIC PODS

ZERO TOUCH PROVISIONING OF SERVERS AND

DCI

CENTRAL TRACING OF SYSTEM EVENTS

APPLICATION ORIENTED SERVICE

PROVISIONING

CONTROL HA

PHYSICAL NODE HA

DCI HA

GEO-REDUNDANCY FOR SERVICE

TOPOLOGIES

AUTOMATIC RESTART OF FAILED

PROCESSES

OVERLAY NETWORK OAM


Mozart POD – Reference Design


Components of Cisco Mozart Solution

Virtual Systems Operations Center (vSOC) Extensible Service Orchestrator

vSOC

Virtual PE Forwarder (vPEF) – Light weight forwarding element per Server

vPEF

vASA, CSR 1000v , SSI, DI (StarOS), 3rd Party

NfV Services

ASR9k, Nexus 7k, 3rd party - Physical PE (DC WAN Gateway)

DC WAN Gateway


Components of Mozart

Tenant 1

VM 1

Tenant 2

VM 1

vPEF

VRF2

VRF1

Data Center

Network IP NGN

MPLS-

VPN VRF1

VRF2

VRF3

DC gateway

Provider Network

Tenant 1

VNF 1

Tenant 2

NNF 1

vPEF

Tenant 1

VNF 2

Tenant 2

VNF 3

vPEF

VRF2 VRF1 VRF2 VRF1

Tunnels MPLSoGRE,

L2TPv3, VXLAN etc

BGP

CLI, XML, NC/YANG (Future)

Elastic network

services

YANG-API

VM Control & Elasticity

Management

IP Address Management

DHCP

DNS

Routing Control

DC

Edge

Service Routing

Performance / Fault

Monitoring

Services Configuration

Cluster Controller

vSOC Infrastructure (Orchestration, Event Notification & Messaging)

REST APIs

YANG-API

CLI/XML BGP

Service Policy Manager & DB

Service Catalog


Mozart Forwarding in a Server vPE-F

• Light weight software forwarding plane

• Provides highly optimized forwarding in x86 environment

• Runs inside a VM in each server

• Contains a unique forwarding context per tenant

• Provides per-tenant L3, L2 and PBR forwarding

• Support for IPv4, IPv6 address families

• Provides multiple tunnel encaps (MPLS-over-GRE, L2TPv3 P2P L2 transport, VXLAN in future)

• Provides DHCP relay function

• Programmed by vSOC using YangAPI (tenancy and service chaining)

• Zero-memory-copy forwarding between VMs using shared memory as a para-virtualized NIC

User Space

Kernel Space 10GE NIC

(underlay)

vPEf VM

Zero-Copy Paravirtualized NIC

VM A VM B Data Plane

Control

Plane

vPEf-Agent Op

en

sta

ck

1GE NIC

(Ctl Net)


Server-2

Serv

er-

3

Three Tier Fat Tree

Single Tier Access

Aggregation Type Single Tier Full-Mesh

Computer Cluster

Folded Clos

Data Center Fabric – The Underlay Network

• Many Options for building the underlay

• Provides Fast Reliable Network Connectivity

• Should support P2P and P2MP Capabilities

• Hardware optimized for cost and efficiency

Server-1 Server-3

Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

Data Center

DCI

Cisco and/or its affiliates. All rights reserved. Server-1 Server-3

Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

L2 Segment and Forwarding

• Each vPE-f has VRF L2 tables

• vPE-f populated with MAC entries

• VMs see each other in an L2 segment

• MT traffic encapsulated in single transport tunnel

• Only a small class of applications need strict L2 connectivity

Three Tier Fat Tree

Single Tier Access


Computer Cluster

Folded Clos

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

VM1 GE-DB

VM2 GE-WEB

L2

L2

MPLS-over-GRE (or)

VXLAN (or)

L2TPv3 Tunnels

Data Center

Virtual Topology


Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

L2 Segment and DCI


• vPE-f populated with MAC entries

• VMs see each other in an L2 segment


• Only a small class of applications need strict L2 connectivity

Three Tier Fat Tree

Single Tier Access


Computer Cluster

Folded Clos

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

VM1 GE-DB

VM2 GE-WEB Virtual Topology

L2

L2 L2 Segment Attached to L3VPNs or L2VPNs

Data Center


Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

IP Forwarding


• vPE-f populated with L3 /32 or /128 entries

• vPE-f is first hop router/DHCP Relay

• VMs can reach each other in L3 network


Three Tier Fat Tree

Single Tier Access


Computer Cluster

Folded Clos

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

VM1 GE-DB


L3

L3

MPLS-over-GRE (or)

VXLAN Tunnels

V3

GE-FW

L3

Data Center


Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

Network Function Virtualization • Network Services could be bump-in-the wire services or

termination services

• Using L2/L3 entries in tables an arbitrary

services topology can be created

• No hair-pinning of traffic as it moves from service to service

• Control Plane responsible for computation of paths and

optimal routing of traffic

• Bring-your-own-Service or choose from Cisco service catalog

Three Tier Fat Tree

Single Tier Access


Computer Cluster

Folded Clos

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

VM1 GE-DB


L3

L3

MPLS-over-GRE (or)

VXLAN Tunnels

VM3

GE-FW

L3 VM

WALMART

VM1 GE-FW

vPE-f L2/L3 VRF FIB

L3


Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

L3VPN, L2VPN & Internet Access

• DCI can be either by injecting /32 or aggregates in SP-WAN

MP-BGP

• All VMs default route to DCI for unknown destinations

Three Tier Fat Tree

Single Tier Access


Computer Cluster

Folded Clos

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

VM1 GE-DB


L3

L3

MPLS-over-GRE (or)

VXLAN Tunnels

VM3

GE-FW

VM WALMAR

T

VM1 GE-FW

vPE-f L2/L3 VRF FIB

L3

L3


Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

Mozart Service Chains • Network Services can be daisy chained

• No restriction on the number of services in a chain

• Services can be dynamically inserted in the chain

Three Tier Fat Tree

Single Tier Access


Computer Cluster

Folded Clos

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

VM1 GE-DB


L3

L3

MPLS-over-GRE (or)

VXLAN Tunnels

VM3

GE-FW

VM WALMAR

T

VM1 GE-FW

vPE-f L2/L3 VRF FIB

L3

L3

VM WALMAR

T

VM1 GE-NAT

vPE-f L2/L3 VRF FIB

L3

NAT

Data Center


NFv Horizontal Scale, Stateful Load Balancing, Elasticity & Flow Stickiness

Mozart-f

L2/L

3

VR

F F

IB M

ozart

-f

L2/L

3

VR

F F

IB

FW-1

FW-2

FW-3

FW-4


Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

Multi-Tenancy, Varied Topologies

Three Tier Fat Tree

Single Tier Access


Computer Cluster

Folded Clos

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

L3

L3

VM WALMAR

T

VM1 GE-FW

vPE-f L2/L3 VRF FIB

L3

L3

VM WALMAR

T

VM1 GE-NAT

vPE-f L2/L3 VRF FIB

L3

VM1 GE-DB

VM2 GE-WEB

VM3

GE-FW

NAT

NAT


Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

Mozart Control Components

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

L3

L3

VM WALMAR

T

VM1 GE-FW

vPE-f L2/L3 VRF FIB

L3

L3

VM WALMAR

T

VM1 GE-NAT

vPE-f L2/L3 VRF FIB

L3

Service Routing DCI

Routing

Virtual Systems Operations Center (vSOC)

DHCP DNS

VM Management

(ESC) Address Management Routing Control

Policy

Database Admin Interface Monitoring

BSS/OSS

Integration

Services

Configuration

VM Orchestrator

vSOC-Cluster

Controller

vSOC Infrastructure (Orchestration, Event Notification/Messaging)

Network Service Specification

(Multi-Tenanted)

Restful-API Interfaces 1

2

Service VM management

3 DH

CP

Dis

co

ve

r/O

ffe

r

4

Se

rvic

e C

on

fig

ura

tio

n

6

Se

rvic

e R

ou

te

Pro

vis

ion

ing

5 BGP Routing

7


Server-4

Server-2

SP WAN (L3VPN,

L2VPN,

IPv4/v6,

Internet)

DCI

Server-2

Serv

er-

3

Mozart High Availability

VM WALMAR

T

VM1 GE-WEB

vPE-f L2/L3 VRF FIB

VM WALMAR

T

VM1 GE-DB

vPE-f L2/L3 VRF FIB

L3

L3

VM WALMAR

T

VM1 GE-FW

vPE-f L2/L3 VRF FIB

L3

L3

VM WALMAR

T

VM1 GE-NAT

vPE-f L2/L3 VRF FIB

L3

• vSOC is not required in steady state

• vSOC supports Active-Standby HA

Service

Routing

DCI

Routing

vSOC - ACTIVE

DHC

P DNS

VM Management


Policy

Database

Admin

Interface Monitoring

BSS/OSS

Integration

Services

Configuration

vSOC-

Cluster

Controller


Service

Routing

DCI

Routing

vSOC - BACKUP

DHC

P DNS

VM Management


Policy

Database

Admin

Interface Monitoring

BSS/OSS

Integration

Services

Configuration

vSOC-

Cluster

Controller



TENANT-1

VMs Proxy

ARP DHCP

Relay

Host - Linux or EXSi

Tenant1

Context

Tenant2

Context

VM2,IP2, MAC2

VM3,IP3, MAC3 VM3,IP3, MAC3

VM2,IP2, MAC2

MPLS-over-

GRE/VXLAN/L2TPv3

Traffic

Mozart Forwarding Element

(vPE-f) vPE

Control Agent Mozart-

Forwarding DP

Thin driver for ESXi and KVM

Mozart Forwarder • Industry’s first user space forwarder

• Multi-threaded, 10G in single core

• IPv6 Support

• ISSU, Fault Isolated

• Highly Portable

• 64-bit, endian clean

TENANT-1

VMs

DPDK

Drivers


VPE-Forwarder Deployment Modes

• The patch panel provides a virtual point-to-point connection from the tenant VMs to the vPE-f dataplane

• Patch panel is a L2 switch running as host kernel module configured for Point-to-point connectivity without Mac learning

• VM deployment model: easy portable, high performance


Provisioning the Mozart System MODEL BASED – DECLARATIVE SERVICE DEFINITION

REST BASED API

Define CPU,

Memory,

Network Interfaces,

Horizontal Scale Factor,

Elasticity,

Disk Storage,

Storage Persistency Requirements,

Service Configuration

Network Zones,

Zone Connectivity,

External Zones,

Managed Zones,

Transit NFv Appliances,

Terminate NFv Appliances,

Service Topology Definition,

Service Chain Definition,

Multi-Path Requirements

Tenant Identifier,

Tenant Specific VPN Identifier,

L3VPN & L2VPN Extended Communities

Organization Definition,

Global Tenant Specific IP Address Pools

TENANT

&

ORGANIZATION

DEFINITION

NETWORK

&

TOPOLOGY

DEFINITION

COMPUTE

&

STORAGE

DEFINITION


Provisioning the Mozart System – Object Model

Universe

Tenants (coke, pepsi,…)

Organization (finance, engineering…)

Applications (FW service, Web Farm..)…)

External Zone Application

Zone External Zone

L3VPN

Customer

Coke

Customer-ID

VRF

Enterprise

Network

Connection

Parameters Service Chain

WEB Server

Image,

CPU,

Memory,

Elasticity Firewall NAT

Service


MOZART MODELS EXAMPLE: MOBILITY (vEPC + Gi-LAN)

Enterprise

CDN Charging Analytics

Analytics NAT

Analytics TCP/Http

optimization

VPN

NAT

Video Optimization

Tele

matics C

lassifie

r

BW mngmt

BW mngmt

vE

PC

Cla

ssifie

r

External Zone Application Zone External Zone

RAN Network

Connection

Parameters

Enterprise

Network

Connection

Parameters

Service Chain

EPC

Applications

S1 S2 S3

S4 S5

Service


MOZART SERVICE BUILDER SCREENSHOT: DESIGNING A SERVICE


Demo..

38


Mozart PODs

Mozart PODs

Scale

Modular Construction

Fault Containment

Application Centricity

High Availability

and

Admin Separation

Data Center

Metro Data Center Metro Data Center

Legacy CPE

Virtualized CPE

Satellite POD Satellite POD

Satellite POD (# Servers = 1)


Mozart PODs

Mozart PODs

Scale

Modular Construction

Fault Containment

Application Centricity

High Availability

and

Admin Separation

BGP

BGP BGP BGP

Data Center


OPTICA

L

MPLS

TE

BGP

ROUTE

S

IP OPTICAL

NETWORK

CUSTOMER NETWORK

MAESTRO (Cross Domain Orchestration)

EMS WAN

OPTIMIZER

CLOUD

SERVICES

OSS SERVICE PORTAL REST APIs for End-to-End Design and

Performance Monitoring

Customer requesting

service

Provision network

connectivity in

WAN

Multi-layer visibility

and SLA

assurance

Provision

Cloud/NFV

services

Data Center 1 Data Center 2

Mozart SLA Aware End-to-End Service Provisioning

SP WAN/MAN


OPTICA

L

MPLS

TE

BGP

ROUTE

S

IP OPTICAL

NETWORK

CUSTOMER NETWORK

MAESTRO (Cross Domain Orchestration)

EMS WAN

OPTIMIZER

CLOUD

SERVICES

Customer requests

service

Provision

Cloud/NFV

services

Data Center 1 Data Center 2

Mozart Geo-Redundancy

SP WAN/MAN

PRIMARY BACKUP

mozart - cisco.com · mozart : cloud services platform for service providers network function...

Documents