osdc 2014 onie by nat morris
Post on 13-Sep-2014
1.006 views
DESCRIPTION
Nat Morris' deck during his ONIE presentation at OSCD 2014 in Berlin, Germany.TRANSCRIPT
![Page 1: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/1.jpg)
Open Network Install Environment
NETWAYS OSDC 2014, Berlin
Nat Morris
9th April 2014
![Page 2: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/2.jpg)
•Overview
•What Is ONIE?
•Lessons Learned
•ONIE Development
•Demo
Agenda
![Page 3: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/3.jpg)
IP Fabric Networking Landscape
Netw
ork
H
ard
ware
Netw
ork
O
S
Open Closed
cumulusnetworks.com
![Page 4: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/4.jpg)
The Expanding Landscape
hardware
operating system
appapp
hardware
operating system
app app
Single Vendor Blob
Multi-Vendor Ecosystem
app app
cumulusnetworks.com
![Page 5: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/5.jpg)
Understanding Characteristics of a Leaf Switch
cumulusnetworks.com
10/40 Gigabit spine uplink ports
Serial console port
Ethernet Out-of-Band
Management Port
1/10 Gigabit Ethernetnetwork access ports
* SFP+ ports can be grouped together into a single QSFP 40G port via reverse connecting breakout cable options* QSFP ports can be broken out into four SFP+ ports via copper or optical transceiver options
![Page 6: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/6.jpg)
Understanding Characteristics of a Spine Switch
cumulusnetworks.com
Serial console port
Ethernet Out-of-Band Management Port
* QSFP ports can be broken out into four SFP+ ports via copper or optical breakout cable options
40G / 10G Fabric ports
![Page 7: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/7.jpg)
Bare Metal Switch Provisioning
Similar approach to installing OS on server
BIOS + PXE = U-Boot + ONIE (Open Network Install Environment)
Supported hardware (HCL) preloaded with ONIE
ONIE available on GitHub• http://onie.github.io/onie/
bare metal server
operating system
app
app
app
BIOS and PXEbare metal
switch
operating system
app
app
app
U-Boot and ONIE
cumulusnetworks.com
![Page 8: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/8.jpg)
Choice
cumulusnetworks.com
![Page 9: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/9.jpg)
Choice
cumulusnetworks.com
![Page 10: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/10.jpg)
What Is ONIE?
Network OS Install Environment Provides an environment for network OS installer
discovery and execution Like a pre-installed kickstarter Defined by its behaviors Implemented using a modern Linux kernel and BusyBox
An Open Source Project within OCP http://www.onie.org/
Evolving …
cumulusnetworks.com
![Page 11: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/11.jpg)
Hardware Vendors
cumulusnetworks.com
![Page 12: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/12.jpg)
Operating System Vendors
cumulusnetworks.com
![Page 13: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/13.jpg)
ONIE: Anatomy of a Network Switch
( Management Interfaces ) ( Data Plane )
CPUSoC
DRAMBootFlash
MassStorage
SwitchingASIC
SerialConsole
EthernetMgmt Port
10GbPort
40GbPort…10Gb
Port40GbPort
…
PCIe
cumulusnetworks.com
![Page 14: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/14.jpg)
ONIE: Uses Management Interfaces
( Management Interfaces ) ( Data Plane )
CPUSoC
DRAMBootFlash
MassStorage
SwitchingASIC
SerialConsole
EthernetMgmt Port
10GbPort
40GbPort…10Gb
Port40GbPort
…
PCIe
cumulusnetworks.com
![Page 15: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/15.jpg)
ONIE: Bare Metal Install – First Time Boot Up
Boot Loader(HW Vendor Supplied)
ONIE(HW Vendor Supplied)
Installer(OS Vendor)
Boot Loader• Low Level boot loader, configures CPU complex• Loads and boots ONIE
ONIE• Linux Kernel with Busybox• Configures management Ethernet interface• Locates and executes an OS installer• Provides tools and environment for installer
OS Installer• Available from network or USB• Linux executable• Installs vendor OS into mass storage
Network OS(OS Vendor Supplied)
Fetches
Installs
cumulusnetworks.com
![Page 16: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/16.jpg)
ONIE: Subsequent Reboots – Vendor’s OS is Already Installed
Boot Loader(HW Vendor Supplied)
ONIE(HW Vendor Supplied)
Boot Loader• Low Level boot loader, configures CPU complex• Loads and boots OS vendor’s installed OS
Network OS• Configures Switching ASIC• Runs Network Protocols• Provides CLI
Network OS(OS Vendor Supplied)
ONIE• Still exists, but is not used• Available for uninstall / re-install operations
cumulusnetworks.com
![Page 17: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/17.jpg)
ONIE: Network OS Installer Discovery and Install Behavior
Configure Network Interface
Locate Installer
Run Installer
• Uses DHCPv4, DHCPv6• Configures Ethernet interface for IPv4 / IPv6• Configures DNS and hostname
• Determines the location of an installer executable• Examines local file systems, e.g. USB flash drives• Uses DHCP options, DNS Service Discovery, Multicast DNS
and IPv6 Neighbors
• Downloads installer via URL• Passes various environment variables to installer• Launches installer
cumulusnetworks.com
![Page 18: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/18.jpg)
Why Not Use PXE?
• IPv6 and HTTP out of the box• Use existing Linux device drivers. No need to write new ones• Integrated automation
cumulusnetworks.com
![Page 19: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/19.jpg)
ONIE: Other Behaviors
Provides a mechanism for a network OS to invoke the above behaviors.
Reinstall Remove currently installed OS and return to the “out of box” provisioning state
Uninstall Completely wipe out everything, except ONIE
Rescue Reboot box into ONIE for repair, debug and forensics
Update Install a new ONIE version
Diag Run HW Vendor’s diag (optional)
cumulusnetworks.com
![Page 20: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/20.jpg)
Lessons Learned
Hardware Platforms Diverse mix of CPU complex designs, even
within a single CPU family. HW Vendors need freedom to customize ONIE Need mechanism to run HW vendor diag
Recurring Themes TLV based EEPROM format widely adopted Common HW designs reduce development time
cumulusnetworks.com
![Page 21: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/21.jpg)
ONIE Ongoing Development
Support Multiple CPU Architectures PowerPC – Today x86 – VM Prototype available today Thinking about ARM, MIPS Maintain ONIE behaviors across architectures
Testing and Compliance Enhance and extend regression test suites Develop compliance test suites
cumulusnetworks.com
![Page 22: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/22.jpg)
ONIE Ongoing Development
New Features DHCPv6 DNS Service Discovery / Multicast DNS
discovery HW Vendor diagnostic
Releases Quarterly release cadence Ongoing maintenance Enhancements
cumulusnetworks.com
![Page 23: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/23.jpg)
ONIE x86 Strategy
Use existing BIOS from hardware vendors During manufacturing install GRUB2 and
ONIE-x86 on the mass storage block device NOS Installer adds partitions, installs software
and updates GRUB2 configuration.
cumulusnetworks.com
![Page 24: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/24.jpg)
Block Device Partitioning
Partition using GUID Partition Table (GPT) format GPT supported by Linux, GRUB, UEFI Supports dual-booting multiple operating systems
Partition # Name R/W Notes
1 GRUB Boot r/o Used by GRUB and GPT
2 ONIE BOOT r/o ONIE Kernel
3 ONIE CONFIG r/w Configuration
4+ For NOS use
cumulusnetworks.com
![Page 25: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/25.jpg)
GRUB Menu
cumulusnetworks.com
![Page 26: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/26.jpg)
NOS Installer Duties
Create new GPT partition(s) Create file systems on partition(s) Install NOS files into partition(s) Update ONIE-CONFIG using ONIE provided
CLIs onie-boot-entry-add onie-boot-entry-remove onie-boot-entry-show onie-boot-default onie-boot-update
cumulusnetworks.com
![Page 27: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/27.jpg)
Resources
ONIE Websites Main Page: http://www.onie.org/ Source Code: https://github.com/onie/onie/ Documentation: http://onie.github.io/onie/
cumulusnetworks.com
![Page 28: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/28.jpg)
© 2013 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.
Thank You!
![Page 29: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/29.jpg)
Back Up Slides – Cumulus Background
cumulusnetworks.com
![Page 30: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/30.jpg)
Pushing Changes Down
CPU, RAM, Flash, etc. Switch Silicon
Front Panel Ports
lldpd
Routing Tables
ARP Table
Devices
Bridge FDB Filter Tables
Bonds VLANs
Lin
ux
Kern
el
Virtual Kernel Ports
Bridging
mstpd
ACLRouting Suite
Quaggasnmpd
vconfig
iptable
ebtable
ip6tableiproute
2
VXLAN Bridges
Switch HAL
brctl
SwitchDriver
Use
r Space
Quagga daemon, Quagga.conf, and vtysh
CLI and /etc/network/interfaces
switchd
cumulusnetworks.com
![Page 31: OSDC 2014 ONIE by Nat Morris](https://reader033.vdocument.in/reader033/viewer/2022052503/5414c4678d7f724d6c8b46ce/html5/thumbnails/31.jpg)
Cumulus Linux Hardware Compatibility List
40G Portfolio Part number DescriptionMerchant Silicon
Cumulus Linux Release
Dell S6000 32 x 40G-QSFP+ Trident II 2.1 or later
Edge-Core AS6700-32X 32 x 40G-QSFP+ Trident II 2.0 or later
Penguin Computing Arctica 3200XL 32 x 40G-QSFP+ Trident II 2.0 or later
Quanta QCT T5032-LY6* 32 x 40G-QSFP+ Trident II 2.0 or later
10G Portfolio Part number DescriptionMerchant Silicon
Cumulus Linux Release
Agema AG-7448CU48 x 10G-SFP+ and 4 x 40G-QSFP+
Trident 1.5.0 or later
Dell S481048 x 10G-SFP+ and 4 x 40G-QSFP+
Trident 2.0.x or later**
Edge-Core AS5600-52X48 x 10G-SFP+ and 4 x 40G-QSFP+
Trident+ 1.5.0 or later
Edge-Core AS5710-54X48 x 10G-SFP+ and 6 x 40G-QSFP+
Trident II 2.1 or later**
Edge-Core AS5710-96X96 x 10G-SFP+ and 8 x 40G-QSFP+
Trident II 2.1 or later**
Penguin Computing Arctica 4804X48 x 10G-SFP+ and 4 x 40G-QSFP+
Trident+ 1.5.1 or later
Quanta QCT T-3048-LY2*48 x 10G-SFP+ and 4 x 40G-QSFP+
Trident+ 1.5.0 or later
Quanta QCT T5048-LY8*48 x 10G-SFP+ and 6 x 40G-QSFP+
Trident II 2.1 or later**
1G Portfolio Part number DescriptionMerchant Silicon
Cumulus Linux Release
Edge-Core AS4600-54T 48 x 1G-T and 4 x 10G-SFP+ Apollo2 2.0 or later
Penguin Computing Artica 4804i 48 x 1G-T and 4 x 10G-SFP+ Triumph2 1.5.1 or later
Quanta QCT T1048-LB9* 48 x 1G-T and 4 x 10G-SFP+ FireBolt3 1.5.0 or latercumulusnetworks.com