white box - listes.renater.fr
TRANSCRIPT
![Page 1: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/1.jpg)
White Box
16th November 2018
Journée Réseaux RégionauxXavier JEANNIN
Maxime Wisslé
Frédéric Loui
1
![Page 2: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/2.jpg)
• Disagregation trend
– Network Operating System and hardware
– Junos on ‘kind of generic hardware’ – OCX1100 (now end of life)
• The idea of having a NOS running on unmarked hardware emerge
– Even programmable hardware
– Reduce Total Cost of Ownership
• The architecture of data-center, regional network and telecom carrier
become very similar
• A large scope of use-cases
– Routing/switching: From Data-center to telecom carrier
– Advanced network feature: Telemetry, Analytics,
2
Introduction
![Page 3: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/3.jpg)
3
Proprietary design
• Business model
– Hardware design Proprietary
– Proprietary NOS (embedded)
– Hardware maintenance
– NOS maintenance
• Dependence on one vendor
![Page 4: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/4.jpg)
4
White-box Design
• Chipset are the same as proprietary devices
• Performances and features depend on forwarding chipset
(switching ASIC)
– Trident, Trident 2, Tomahawk, Qumran, Jericho
– Juniper/Cisco use the same chipset
– ...
• Network Operating System (NOS)
– Commercial and open source
![Page 5: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/5.jpg)
5
Network Operating SystemNOS commentaires
DELL
OS9 ou OS10 (Free BSD)
Commercial - Linux + Quagga + BGP EVPN (VXLAN ?)
Cumulus Networks Commercial - Unix + CLI Data center
IP Infusion
OcNOS
Commercial - Expérience opérateur
- Nombreuse fonctionnalités opérateur : MPLS
Pluribus Networks
NetVisor
Commercial - Solution SDN mais contrôleur embarqué dans chaque boite
- fonctionnalités opérateurs spécifiques : data analytics,
service chaining and VNF functions
Open Network Linux Open / Free - https://opennetlinux.org/
Barefoot networks
Sur chipset Tofino
Open et peut être
commercial
- certainement l’avenir
- A partir de chipset programmable en P4 : Telemetry, des
DDOS mitigateurs, des load balancers, …
Pica8
PicOS
Commercial Hybrid Networking: OpenFlow agent, with native L2 and L3
features
Big Switch Networks
SwitchLight
Commercial Solution SDN avec controleur non réparti
Canonical Snappy
Ubuntu Core
Commercial
Software for Open Networking in the
Cloud
SONIC
Open Microsoft and co-contributors to OCP
Orienté cloud pour AZUR
OpenSwitch Open Se focalise sur peu de fonctions mais très rapide
SnapRoute FlexSwitch Open
Open Network Foundation Atrium SDN
Distribution
Open - CORD use case
Open Compute Project
http://www.opencompute.org/
projects/networking/
Open Projet “Mère” qui porte :
- Campus Branch Wireless (CBW)
- Open Network Install Environment (ONIE)
- Open Network Linux (ONL)
- SONIC
- Switch Abstraction Interface (SAI)
![Page 6: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/6.jpg)
• Routing and switching
– Cloud and data center (SURFsara), Campus, GIX (London Internet
Exchange [LINX]), Telco Operator …
• Analytics, Telemetry - Pluribus
• Service chaining and Network functions - Pluribus
• SDN architecture – Big switch, …
• Data manipulation
– Security/filtering, deduplication, load balancing, packet slicing, time
stamping and protocol stripping.
• Optical white box ???
6
Scope of white box 1/2
![Page 7: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/7.jpg)
• Data Plane Programmable – P4 language
– Based on PISA architecture [Protocol-Independent Switch Architecture], FPGA, Open VSwitch
Tofino chipset barefoot network (https://www.barefootnetworks.com)
• Application
– Security: In-Network DDoSDetection
– Performance: Layer 4 Load Balancer
– Monitoring: Advanced Network Telemetry
– Analytics
7
Scope of white box 2/2
https://p4.org/
![Page 8: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/8.jpg)
8
![Page 9: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/9.jpg)
Are white boxes able to scale
and
can they make the job?
9
![Page 10: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/10.jpg)
10
White-box architecture
TCAMAdditional memory
![Page 11: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/11.jpg)
• Arista 7050X switch architecture
11
Chipset architecture
![Page 12: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/12.jpg)
12
Chipset characteristics
• Buffer and memory
– Packet queue length
• Route number limit in FIB
– For instance, Trident2 is limited to around 200 000 FIB routes
![Page 13: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/13.jpg)
• Traditionally not scalable
• Several core
• Scalable?
– Depend on #port
– Depend on network services
– Do not know the answer
• “Probably”, it should work with few ports (4 ports) ?
• Can address some use cases ?
13
Why not use a X86 server ?
NIC NIC NIC
![Page 14: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/14.jpg)
• Crucial to check the way the NOS is written
– Feature available
– Perfomance and Scalability• Manage route update, #routes, …
– Reliability
– Monolithic Vs several subsystems (deamon)
– Relationship between FIB and RIB
– Time sharing
– Sharing memory
– Watch dog
– Security – Routing Engine protection
14
NOS: points to consider
![Page 15: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/15.jpg)
• Linux-based Universal NOS Installer
• Combination of boot loader and light OS
• Pre-installed on White-Box
• NOS Install/Uninstall
• NOS image host
– DHCP/Web server
– Licence server
• Serial access required
15
ONIE : Open Network Install
Environment
![Page 16: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/16.jpg)
• Being more independent regarding the hardware choice;
• Reduce network devices the total cost ownership;
• Possibly benefit advanced features;
– Traffic Engineering, PCE/SDN, Telemetry, Analytics, load
balancer, security (DDOS mitigation) ...
• Lot of use cases for R&E community
– Campus, data center / Cloud, …
• Being able to benefit of specific to education and
research network features and services;
– LHCONE traffic
– …
• Data Plane Programmability for research
16
White-box stakes
![Page 17: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/17.jpg)
• The same services that NRENs provide to theirs end-users?
• Is this technology mature enough regarding NREN reliability requirement?
• What is missing in an open network operating system before going into
production?
– Manageability, Open network operating system security, Documentation,
Maintenance model
– …
• What are the potential first use cases for such technology?
– Campus, Metropolitan network;
– security feature (DDOS mitigation, …);
– Scientist experiment
– …
• What would be a transition model that could be put in place in order to go in
production?
• What is the total cost ownership for such technology?
17
Condition to adopt white-box model?
![Page 18: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/18.jpg)
• NREN backbone
• Regional network
• Campus network
• Science project
• Global Internet eXchange (GIX)
• Cloud Fabric
18
Use cases
![Page 19: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/19.jpg)
• PE for NREN Backbone
– 72 PoPs
– Inter-AS services
• Requirement
– Lot of network features
– VPN services (L2/L3)
– Advanced L2/L3 features
– CISCO (ASR 9000) and Juniper interoperability
(MX 2010) 19
NREN backbone
![Page 20: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/20.jpg)
• Our regional network is based on carrier Ethernet
• G 8032 ring, ERP, E-LINE, E-LAN, E-Tree, OAM
20
Access Network
/ Regional Network
![Page 21: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/21.jpg)
• Our education and research community used to run network campus, these campuses could be a good candidate for white box
21
Campus
![Page 22: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/22.jpg)
• Can be isolated in our backbone (VRF or VRF-light) and
forwarded by a set of white boxes
• The traffic can be heavy but the route number is quite small
(several thousands)
22
Science project
![Page 23: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/23.jpg)
• NREN use to run Internet eXchange Point
• Exchange point can be used for providing connectivity
between academic cloud and also with commercial cloud
23
Global Internet eXchange (GIX)
and distributed GIX
![Page 24: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/24.jpg)
• Both RENATER and research & education community
run data-center that can be for small to large size ;
• Technologies:– IP Fabric + EVPN/VXLAN or …
– Others ….
24
Cloud fabric
![Page 25: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/25.jpg)
• Objectives
– Check white-box usage according to different context
(uses cases)?
• Features and bugs
– Are all elements available for production (network operation)?
• Monitoring
• Management (Tacacs, Routing engine protection, …)
• Software support provided by NOS provider
• Documentation
– Business model• Licence model
• TCO
25
A first white-box project
![Page 26: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/26.jpg)
26
Testbed
![Page 27: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/27.jpg)
• Hardware : DELL S4048-ON 1
– 48 interfaces 10G
– 4 interface 40G
– Trident II
• Very polyvalent chipset
• Open Network Install Environment
– (ONIE) 2
– Network Operating System (NOS)
• first NOS tested MPLS capable
• Support of lot of NOS
– Netvisor
– Cumulus
– …
27
Test environment
S4048 DELL
• Hardware issue from Data center world • No small machine available
![Page 28: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/28.jpg)
• Management
– Authentication
• TACACS, Radius, …
– Monitoring
• sFLOW, Port mirroring, NTP, SMTP, DHCP
– …
• Routing
– VPN (L3VPN, L2VPN, ...)
– VLAN
– VXLAN
– IS-IS (Single-topology, no Multi-topology)
– BGP, MPLS, LDP
– Load balancing
– ….
• White_Box_PoC_result_table.docx
28
First results
![Page 29: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/29.jpg)
• This NOS can maybe used in specific use cases: GIX,
Campus, …
– Maturity issue but encouraging perspective
• Currently use it for our new GIX design in Paris
– Still have to solve the problem of the unicity of routing engine …
– …
29
Next step
![Page 30: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/30.jpg)
• Operation, security, documentation, maintenance model
are mandatory
• Transition model is the key point
– Innovate in order to overcome what is missing
– Find the appropriate use case
• To start, train your team and get experience
– Use experience in RE community
• Big data center already use white box
30
Conclusion 1/3
![Page 31: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/31.jpg)
• Do not expect better that you have
– Same will be a great result
– New usage/feature will be a breakthrough
• White box will not replace our Juniper or Cisco box in a
first stage
– Instead move specific services on white box: GIX, LHCONE, …
• Classical vendors will not stay without any reactions
– Price of legacy vendor box is already decreasing a lot
31
Conclusion 2/3
![Page 32: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/32.jpg)
• The network landscape is changing deeply so a long
term approach is necessary
– Linux does not replace Solaris in one year /
Rome was not built in a day
• Not reversible path
– For White Box
– For data plane programming
32
Conclusion 3/3
![Page 33: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/33.jpg)
• New hardware dedicated to Telco is
coming: Qumran, Jericho
– Internet route: Additional TCAM
– Buffer size increased
– Scalability
• Other use cases: telemetry, security
• Network programmability
– P4 / Bigfoot
– A perfect tools for computing research
33
Perspectives
![Page 34: White Box - listes.renater.fr](https://reader033.vdocument.in/reader033/viewer/2022052521/628c8ef4e5854f1455355b63/html5/thumbnails/34.jpg)
• GEANT project in GN4ph3 Jan. 2019 – Dec 2020
– White Box
• Study best NOS for Education and Research
• Maybe Open Source NOS
– Data Plane Programming
• First use case Telemetry/monitoring
• Looking for your use case that can be implement in pilot
production
– RENATER will certainly propose GIX and possibily study your
use cases
– GRNET will propose an IP fabric for Data Center with an overlay
network (VXLAN/EVPN or …) with Open Stack orchestration
34
Perspectives