network disaggregation · packet forwarding chip (asic) ocp switch hardware (facebook wedge, im...

NetworkDisaggregation

RonaldvanderPolSURFnet

RoutzSharePlus,6February2017,Diemen

Overview

• Whatisnetworkdisaggregation?• OpenComputeProject- Networking• demo• OpenFlow,PipelinesandSilicon• TableTypePattterns (TTPs)andP4• demo• Summary


Vertically integratedClosed, proprietary

Slow innovationSmall industry

SpecializedOperatingSystem

SpecializedHardware

AppAppAppAppAppAppAppAppAppAppApp

SpecializedApplications

HorizontalOpen interfacesRapid innovation

Huge industry

Microprocessor

Open Interface

Linux MacOS

Windows(OS) or or

Open Interface

(slidebyNickMcKeown,StanfordUniversity)

Vertically integratedClosed, proprietary

Slow innovation

AppAppAppAppAppAppAppAppAppAppApp

HorizontalOpen interfacesRapid innovation

ControlPlane

ControlPlane

ControlPlane or or

Open Interface

SpecializedControlPlane

SpecializedHardware

SpecializedFeatures

MerchantSwitching Chips

Open Interface

(slidebyNickMcKeown,StanfordUniversity)

NetworkDisaggregation

• Bestofbreedinhardwareandsoftware• OpenAPIs• OpenHardware• User/operatorincontrol

– Lessdependentonvendorroadmaps– Implementandexperimentwithnewprotocols


BenefitsofDisaggregation

• From– Closedvendorproprietaryall-in-oneblackboxes

• Vendordecideswhichfeaturestosupport• Vendordecideswhenthesefeaturebecomeavailable• Dependentoninnovationstrengthofvendor

• TO– Whitelabelswitcheswithdisaggregatedfirmware

• Choosebestvendorforhardware• Choosebestvendorforfirmware• Useopensourcefirmware(communitycontroloverfeaturesandinnovation)


Computingvs Networking

ClosedSystems

OpenHardwareOpenSoftware

1970 1980 1990 2000 2010

ClosedSystems

closed

open

ClosedhardwareOpenFlow APIOpenSourceApplicationsPortableapplications

OpenHardwareOpenFirmware

ClosedhardwareWorkstations+UNIXUNIXSystemCallAPIStartofOpenSourceSoftwarePortableapplications

SlidebyR.vanderPol

OpenComputeProject- Networking

• ONIE(switchbootloader)• SAI(SwitchAbstractionInterface)• OpenNetworkLinux(ONL)• SoftwareforOpenNetworkingintheCloud(SONiC)


ONIEBootloader


OpenNetworkLinux


ADDITIONAL / FUTURE APPLICATIONS HERE

ONL: THREE+ FORWARDING AGENTS; 20+ PLATFORMS

8 © 2015 BIG SWITCH NETWORKS, INC.

CPU (PowerPC, x86)

Miscellaneous Hardware (Fans, LED controllers, SFP sensors, Power…)

Packet Forwarding Chip (ASIC)

OCP Switch Hardware (Facebook Wedge, IM Niagara, Accton 6712, 7512, Dell S6000-ON, Quanta LY6, ...)

Broadcom SDK (others coming soon)

OF-DPA

Platform Specific ASIC Drivers

Open NSL

SAI Interface

ONL Linux Kernel (Includes extra drivers: I2C, MUX, management Ethernet, …)

Open Network Linux Platform Abstraction Layer (Platform specific drivers including Optics)

Installer (ONIE)

FORWARDING AGENT PROGRAMMING APIS

Indigo OpenFlow

Agent

Facebook FBOSS

HA

RD

WA

RE

PLA

TFO

RM

Platform Driver

Microsoft SONiC

SONiC

SWITCH(HOST)


OpennetworkingHardware

BaseOSandinstaller

H/WProgramminginterface

NetworkSoftwareComponentsNoOCPSolutionAvailable

SwitchAbstractionInterfaceSAI

OCPHardware

OpenNetworkLinuxONL

OpenNetworkInstallEnvironment

ONIE

SoftwareforOpenNetworkingintheCloudSONiC

CumulusNetworks

• CommercialL2/L3• HardwareCompatibilityList:

– Dell,EdgeCore,Facebook,HPE,Mellanox,Penguin,QCT,Supermicro,Agema


Pica8

• PicOS commercialL2/L3andOpenFlow• HardwareCompatibilityList:

– EdgeCore,HPE,Inventec,InterfaceMasters,Pica8,Dell,Penguin,QCT


OpenNetworkLinux

• OpenSourceL2/L3andOpenFlow• HardwareCompatibilityList:

– QuantaMesh,Accton/EdgeCore,DNI/Agema,Dell,InterfaceMasters,Mellanox


OpenSwitch

• OpenSourceL2/L3andOpenFlow• Switchplugins:

– OVS– OpenNSL (Broadcom)– P4– XPliant– SAI– Marvell


OpenSwitch Architecture

Virtual Interfaces I2C/other drivers SDK driver

ops-powerd

ops-tempd

ops-fand

SDK

SDK specific plugin

SDK independent layer

OpenFlow

sflow

OVSDB

CLI, REST, Ansiblesystem daemonsL2/L3 daemons

RFC 7047

KERNEL


Demo


OpenFlow

• OpenFlow givesuser/operatordirectaccesstoflowforwardingtables

• OpenFlow providesMatch/Actionsemantics• Supportedonmanyhardwareswitches

– PureOpenFlow switches– Hybridswitches(conventionalswitchadd-on)

• Many(opensource)controllerplatforms• OpenFlow startedthenetworkdisaggregationefforts


SoC ASICbasedOpenFlow Switches

• ManybasedonBroadcomASICs(e.g.Trident)• Onlyasmallfixedamountoflookuptables

– TCAM(wildcardentries,ACLs)– MACForwardingDatabase– L3longestprefixmatchtable– L3hostroutes


OpenFlow 1.3MultipleTables

• Preventflowentryexplosion• Multi-tablepipeline

table0

table1

tablen

ExecuteAction

Set

packet in packet outIngressPort

ActionSet = {}

ActionSet

Packet +IngressPort +

metadata Packet

ActionSet


MappingofFlowTables

ExecuteAction

Set

packet in packet outIngress

Port

ActionSet = {}

ActionSet

Packet +IngressPort +

metadata Packet

ActionSet

ingressport

packet in IPtable

MACtable

ExecuteAction

Set

egressport

packet out

OpenFlow 1.3 Pipeline

Broadcom Pipeline (simplified)

table0

ingressTCAM

based on Pica8 documentation

table1

tablen


BroadcomTridentII

• ThereisverylittlepublictechnicalinformationbecauseofBroadcom’sNDA

• SeveralTCAMs,L2,L3,LPMtables• UnifiedForwardingTable(UFT)memorybankscanbeallocatedto:– L2entries– ARPentries– L3LPMentries– ExactmatchACLentries


BroadcomTridentIIUFTBANK SIZE

0 4Kx420bits

1 4K x420bits

2 16Kx420 bits

3 16Kx420 bits

4 16Kx420bits

5 16Kx420bits

6 1Kx420bits

7 1Kx420bits

8 1K x420bits

9 1Kx420bits


} DedicatedL2MAC(32Kx105bits)

} SharedEntries(256Kx105bits)

} DedicatedL3hostentries(16Kx105bits)

TridentIIUFTCombinationsMode L2 L3hosts LPM

0 288K 16K 0

1 224K 56K 0

2 160K 88K 0

3 96K 120K 0

4 32K 16K 128K(77K– IPv6)


LimitationsofSoCASICs

• Fixedsemanticstables(L2,L3,LPM,TCAM)• Fixedsizetables(orlimitedresizing)• Norecirculationofpackets(onepassthroughpipeline)


ASIC/OpenFlow LimitationExamples

• LimitationofSoC ASICs– OpenDaylight ServiceFunctionChaining(SFC)projectconfiguresmultipletables

– Theseendupin1TCAManddoesnotwork– Result:genericapplicationslikeODLSFCcannotbeused;applicationneedstobeadaptedtoASIC

• LimitationofOpenFlow– StilldependenceonSDOsandvendorsfornewencapsulations/protocols

– WewanttoexperimentwithNetworkServicesHeader(NSH),butnosupportinOpenFlow


ProgrammableNetworkSilicon• FPGAs(FieldProgrammableGateArrays)+TCAM+DDR– Corsa DP2200*)

• NetworkProcessors(NPUs)+TCAM+DDR– NoviFlow NS2128*)

• FlowProcessor– Netronome NFP-4000*)

• ProgrammableSwitchSilicon– CaviumXpliant,BarefootTofino

*)presentinSURFnet testbed


Corsa (FPGA/TCAM/DDR3)


CONFIDENTIAL 10

Network Hardware Virtualization

Multiple SDN applications controlling Virtual Switches

Overlay

Hardware Resource Pool

Single piece of hardware

OpenFlow SDN App L3 Routing App

OpenFlow Switch

Any OpenFlow match ANY RateANY PortAny Protocol

L3 Routes + ACLsANY RateANY PortAny Protocol

Underlay

Optimized L3 Router

NoviFlow NS2128

SerD

es

MAC

s

TOPPacket Processors &

Search EnginesTraffic

Manager

OAM

Internal Memory

Classification &Queuing

Internal TCAM

SDRAM DDR3 Controllers


Mellanox EZchip NP-5

NoviFlow PipelineConfiguration

• Setconfig pipeline<id><size><width><type>– <type>isexact(DDR)orwildcard(TCAM)– Default

• 28wildcard+28exacttables• 4096rows• 40bytewide


PipelineAbstractions

• Flexibleprogrammablepipelinesneedanabstractiontodescribethem

• Twopopularapproaches:– TableTypePatterns(TTP)– OpenFlow pipelines– P4(ProgrammingProtocol-IndependentPacketProcessors)

• Bothcanbeusedto– Lettheswitchadvertiseitssupportedpipeline(s)– Telltheswitchwhatpipelinetoconstruct

• P4ismuchmorepowerfulthanTTPs


TableTypePatterns(TTPs)

• ATTPisanabstractmodelthatdescribes(inJSONsyntax)theforwardingbehaviour– Descriptionofflowtables– Descriptionofvalidflow_mods,group_mods andmeter_mods

• SwitchandcontrollermaysupportmultipleTTPs

• Atstartup thereisanegotiationbetweenswitchandcontrolleraboutwhichTTPtouse


P4Language• P4:ProgrammingProtocol-IndependentPacketProcessors

• DomainSpecificLanguageforprogrammabledataplanes

• P4programà P4compilerà targetcode

• TargetcodeisloadedonP4switch• Consistsofpacketparserandlookuptables


P4Switch

INPUT

MatchAction

MatchAction

OUTPUT

ParseGraph

ControlProgram

Match+ActionTable Config

Ingress Egress

Switch Configuration

P4 Switch

PARSER

Queuesand/orBuffers

Source: The P4 Language SpecificationVersion 1.0.2


ExampleP4HeaderDefinitions

header_type ethernet_t {fields{dstAddr :48;srcAddr :48;etherType :16;

}}

header_type ipv4_t{fields{version:4;ihl :4;diffserv :8;totalLen :16;identification:16;flags :3;fragOffset :13;ttl :8;protocol:8;hdrChecksum :16;srcAddr:32;dstAddr:32;

}}


ExampleP4Parserparserstart{returnparse_ethernet;

}

parserparse_ethernet {extract(ethernet);returnselect(latest.etherType){ETHERTYPE_IPV4:parse_ipv4;default:ingress;

}}

parserparse_ipv4{extract(ipv4);returningress;

}


P4SupportedTableTypes• Exact:value==tableentry

– E.g.IPv4hostroute• Ternary:valueANDmask==tableentry

– Wildcard• LPM:LongestPrefixMatch

– Specialcaseofternary(1111….11110000.....0000)• Range:lowentry<=value<=highentry• Valid:tableentry={true,false}

– True:headerfieldisvalid– False:headerfieldisnotvalid


P4SupportedChecksumAlgorithms

• XOR16

• CSUM16

• CRC16

• CRC32

• Programmable_CRC– ArbitraryCRCpolynomial


AdditionalP4Features• Counters

– Type:bytesorpackets– Min-width– Saturating:stopcounting;defaultiswrap

• Meters

• Registers

• Resubmit(originalpacket+metadata)

• Recirculate(packetafteregressmodifications)


P4ControlFlow• If/else

• +,*,-,<<,>>,&,|,^

• ~,-

• OR,AND

• >,>=,==,<=,<,!=


WorkFlow

• WriteP4program,typicallythesesourcefiles:– foo.p4– headers.p4– parser.p4

• ConvertP4programtoJSONconfiguration

• LoadJSONconfigurationonP4switch


Demo


Summary

• OpenFlow startedthenetworkingdisaggregation

• Manycompanieshavejoinedthenetworkingdisaggregationefforts

• Manyopenhardwarevendors• SeveralcommercialNOS’savailable• SeveralopensourceNOS’savailable


[email protected]@rvdp.org

https://www.rvdp.org@rvdpdotorg


network disaggregation · packet forwarding chip (asic) ocp switch hardware (facebook wedge, im...

Documents