presented by scott hogg, cto gtri - meetupfiles.meetup.com/19296132/gtri - cisco nexus 9000...

© 2015 Global Technology Resources, Inc. All Rights Reserved. Contents may contain confidential information and are not to be copied.

Cisco Nexus 9000 NX-OS Network Programmability

Presented by Scott Hogg, CTO GTRICCIE #5133, CISSP #4610

January 7, 2016 – Denver NPUG


• Welcome to the Denver NPUG• The Need for Network Programmability

• Nexus 9000 Overview– Hardware and Software

• BASH Shell, package management, vi, scheduler, SMTP

• Python (on-box and off-box)

• NX-API• Web Interface Sandbox

• Guest Shell and Containers (Puppet, Chef, Ansible)

• Broadcom Shell• XMPP

• NETCONF

• Additional Resources

Agenda


• Network Programmability User Group (NPUG)• NPUG Started in Atlanta

– Nick Matthews (@nickpowpow) and Stefano Pirrello

– https://netprog.atlassian.net/wiki/display/NPUG/NPUG

• Now NPUG Chapters are popping up in many cities– Atlanta, Chicago, Denver, Houston, Kansas City, New York,

Philadelphia, RTP, San Francisco, St. Louis

– http://www.npug.net/locations/

• NPUG.net has many useful online resources– http://www.npug.net/

– http://www.npug.net/past-events/

– http://www.npug.net/how-tos/labathon/

Welcome to the Denver NPUG


• http://www.meetup.com/Denver-NPUG/

Denver NPUG MeetUp Site


• “The Times They are a-Changin’”, Bob Dylan

• Movement toward virtualization, multi-tenancy, cloud services, the third platform, and influenced by consumption economics

• “Why Software Is Eating The World”, by Marc Andreessen, (WSJ, August 20, 2011)

• DevOps isn’t just a popular digital-age portmanteau, its a movement of IT de-silo-izationthat is also coming to data-networking

Network Programmability


• Servers have transformed from bare-metal to virtualized OSs, and now applications are moving to containers.

• Storage systems now have dynamic features like automatic tiering, thin-provisioning, de-duplication, backups and replication.

• Networking hasn’t changed substantially in 15 years– Unfortunately, network devices are still manually configured one at a

time

– Network Admins have only managed to moved from Telnet to SSH

• SDN provides a global view of the network allowing applications and policies to control network forwarding behavior

• You might have reason to be fearful if your job primarily involves assigning Ethernet ports to VLANs

Benefits of Network Virtualization and SDN


We fear change - The World Needs Ditch Diggers, Too

“The erosion of IT's middle class”, by Scott Hogg

http://www.networkworld.com/article/2226369/cisco-subnet/the-erosion-of-it-s-middle-class.html


• Cisco has been working to transform with the industry and produce products that fit this new software-driven cloud-enabled world

• Cisco has produced a number of SDN controllers and SDN protocols– Cisco developed Open Network Environment Platform Kit (onePK)– Cisco eXtensible Network Controller (XNC) is the output from their efforts

with the Open Network Foundation (ONF) and OpenDaylight and OpenFlow– Cisco is a significant contributor to OpenStack (especially Neutron)– Cisco created Application Centric Infrastructure (ACI) and OpFlex– Cisco offers a SD-WAN solution with Intelligent WAN (IWAN)

• Cisco is opening up their traditionally closed architectures and software

• Cisco is changing their software licensing model, e.g. Cisco ONE

Cisco & the Market are Transforming


• Data center switches using merchant-plus silicon– 9396PX/9396TX/93128TX fixed Top-of-Rack switches– Nexus 9504/9508/9516 modular switches– 9500s have no midplane - line cards in front connects to each fabric module

in back, improved airflow– Fully redundant and hot-swappable components

• Up to 60 Tbps of non-blocking performance with <5-microsecond latency• 40Gbps BiDi optics and 100Gbps interfaces

• Fewer ASICs to reduce power consumption– 80 Plus Platinum in power supply efficiency– 3.85 Watts/10GbE port, 15.4 Watts/40GbE port

• http://www.cisco.com/go/nexus9000

• BRKARC-2222 - Cisco Nexus 9000 Architecture

Cisco Nexus 9000 Hardware


• Nexus 9000 switches can also run in one of two modes:

– 1) In NX-OS mode, they run the NX-OS software and operate much like any

other Nexus switch you know and love.

– 2) In Application Centric Infrastructure (ACI) mode, they use the Application

Policy Infrastructure Controller (APIC) to control their configuration and they

participate in an ACI data center fabric as either a spine or leaf switch.

• Nexus NX-OS on N9Ks uses 64-bit Linux 3.4.10 kernel

• Modular OS with fault containment and resiliency with single common

image file (same image for 9500s and 9300s)

– http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-

switches/white_paper_c11-622511.html

• Power-On Auto Provisioning (POAP) automates software update process

– https://github.com/datacenter/ignite

Cisco Nexus 9000 Software


• Cisco is opening up their NX-OS operating system and the object model that is used for configuration (Camden 7.0(3)I2(X))

• Exposing the internal 64-bit Linux v3.4 Kernel to the bash shell

• Can install native RPMs and 3rd party applications just like on a Linux server (built in support for YUM),

– NX-OS processes can be upgraded with “yum update”

• Open interfaces to software with adaptable SDK (Yocto 1.2)

• Fork of Nexus 3000/9000 (7.0(3)I2X)

• https://opennxos.cisco.com/

Open NX-OS


Nexus 9000 With the Openness of Linux

Source: Cisco BRKDEV-2003


• The BASH shell runs on-box and gives you access to the Linux command shell prompt.– N9K(config)# feature bash

– N9K# run bash

– bash-4.2$ sudo su root

– bash-4.2#

• When you are in the BASH shell you can run the VI (visual editor)– bash-4.2# vi /bootflash/my-script-filename.py

Bourne-Again SHell (BASH) #!/bin/bash

Bill Joy


GTRI Nexus 9000 Lab

CISCO NEXUS N9K-C9372TX

53 5451 5249 501 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48


53 5451 5249 501 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48


53 5451 5249 501 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48Nexus 9372PX #2

MGMT 10.16.0.232

Nexus 9372PX #1

MGMT 10.16.0.231

Nexus 9372PX #3

MGMT 10.16.0.233

Eth1/2 Eth1/3

Eth1/1 Eth1/1

Eth1/21 Eth1/22

UCS C220 M3 SDN Server

vSphere 6.0 10.20.50.201NAT (10.16.0.207)

VMNIC2 VLAN 280

10.20.80.201

Eth1/1

Lab Switch3560X VLAN 250

10.20.50.102 Gig0/8

VLAN 250

VLAN 250 10.20.50.131

VLAN 250 10.20.50.132 VLAN 250 10.20.50.133

Gig0/9

VLAN 250

VMNIC3 VLAN 290

10.20.90.201


• Nexus 9000 switches also have a scheduler that allows you to periodically run specific processes or scripts. – N9K(config)# feature scheduler

– N9K(config-job)# python bootflash:/scripts/abc.py

– N9K(config-job)# exit

– N9K(config)# conf t

– N9K(config)# scheduler schedule name run-abc-job

– N9K(config-schedule)# job name run-abc-job

– N9K(config-schedule)# time start now repeat 0:0:1

– Schedule starts from Tue Jan 5 15:13:11 2016

– N9K# show scheduler job

– N9K# show scheduler schedule

Job Scheduler from BASH


• You can direct the output of a command to SMTP

• “email” command

– show run | email [email protected]

smnp-mail.domain.com subject-show-run-

output

SMTP Destinations


• The Nexus 9000 switches also allow for Python v2.7.5 scripting natively on-box. You can run python in an interactive mode by simply typing “python” at the CLI prompt or the BASH shell.– N9K# python

– Python 2.7.5 (default, Oct 8 2013, 23:59:43)

– [GCC 4.6.3] on linux2

– Type "help", "copyright", "credits" or "license" for more information.

– >>> print “Hello World!”

– Hello World!

– >>> exit()

• Alternatively, you can create your own python scripts and run them in non-interactive mode as needed.

Python In The Box


• The python scripting environment has a standard library

that needs to be imported into scripts.

• Need to put “import cli” at the top of all python scripts.

• Three of the more useful functions in the python CLI module are:

– cli.cli() - executes a CLI command and returns the output

– cli.clid() - outputs the JSON syntax for the CLI command

– cli.clip() - takes the output of the CLI command and

echoes it to standard-out (stdout)

More Python


• Once you write your python script, you store the python file in the scripts directory on the bootflash: storage. Then you can execute your script from the CLI or BASH or GuestShell.

• The “bootflash:scripts” directory is the default script directory• N9K# python bootflash:/my-script-name.py

• There is also a way to compile your python scripts – N9K# python –m compileall my-python-script.py

• If you should need it, Cisco has written a “Troubleshooting Python API” page of the Troubleshooting guide that may give you some tips.– http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus900

0/sw/7-x/troubleshooting/guide/b_Cisco_Nexus_9000_Series_NX-OS_Troubleshooting_Guide_7x.html

Even More Python


• Git is a widely used version control system for software development– https://git-scm.com/ or https://desktop.github.com/– sudo apt-get install git

• GitHub is a Web-based Git repository hosting service

• Cisco has put together a set of sample scripts that you can learn from and build on.

• Cisco has many scripts and tools on GitHub and, provided you have git, you can retrieve them by cloning the repository with the following command.– git clone https://github.com/datacenter/who-moved-my-cli.git

Finding/Borrowing Scripts (Code Re-Use)


• Cisco has also published a set of Nexus 9000 scripts that you can retrieve using git.– git clone https://github.com/datacenter/nexus9000.git

• Other GitHub Repositories– https://github.com/datacenter– https://github.com/datacenter/nexus9000– https://github.com/datacenter/who-moved-my-cli– https://github.com/datacenter/nxos– https://github.com/datacenter/nxtoolkit– https://github.com/datacenter/open-nxos-getting-started– https://github.com/datacenter/opennxos– https://github.com/datacenter/nxtoolkit– https://github.com/cisco/cisco-nxapi

Where to get sample scripts for the N9K?


• Allows for Model Driven Programmability (MDP) simplified interaction and configuration of NX-OS mode switches

• Secure access to object model with authentication and encryption (avoids CLI command sequencing)

• Change-based notifications – apps can subscribe to events from network objects preventing redundant polling, screen scraping

• Available on NX-OS 7.1(2)+, 7.0(3)I2(1), NX-API REST SDK added

• N9K1(config)# feature nxapi

• https://opennxos.cisco.com/public/api/nxapi-rest/

NX-API REST Interface


• Web interface to the NX-API RESTful interface that helps quickly create Python code

• Open web browser and go to: https://<switch_IP_address>/ins• If you want to interact with the NX-API in sandbox mode, then you would

open your web browser to: http://<switch_IP_address>/

• You can enter CLI commands and then determine the scripting syntax based on CLI commands you are already familiar with.

• Quick way to learn the XML, JSON or python syntax to use in other scripts.

• Cisco also has a good page on “Troubleshooting NX-API” that may come in handy.– http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-

x/troubleshooting/guide/b_Cisco_Nexus_9000_Series_NX-OS_Troubleshooting_Guide_7x.html

Nexus 9000 INS Sandbox Web Interface


• Built-in embedded container environment that uses CentOS 7 Linux Container (LXC) to decouple/segregate execution space of DevOps services– Access to the network over Linux network interfaces.

– Access to Cisco Nexus 9000 bootflash & volatile tmpfs

– Access to Cisco Nexus 9000 CLI & NX-API REST interface

– The ability to install and run python scripts– The ability to install and run 32-bit & 64-bit Linux applications

• N9K1# guestshell enable

• N9K1# run guestshell

• N9K1# guestshell

• [guestshell@guestshell ~]$

• https://opennxos.cisco.com/public/api/guest_shell/

Guest Shell 2.0 & Containers


• Agent-Based, Pull-based– Agent on the device

periodically connects to master for config info

– Changes made on master are pulled down

– e.g. Puppet, Chef

• Agent-less, Push-Based– Config scripts are run on the

master– Scripts connect to the device

and execute the tasks– No timer, control lies with the

master– e.g. Ansible

Configuration Management Tools

• Configuration Management Tools:

• Enforce model compliance and eliminate config drift

• Provide audit and change logging

• Support concept of no-op runs

• Operations are Idempotent (e.g. 1 X 1 = 1)


• Integration of Puppet Enterprise 2015.2.x (now 2015.3), Puppet 4.x and Cisco Open NX-OS

• Native Puppet NX-OS agent and Cisco Puppet Module

• Uses declarative language to define desired state

• Simulate deployment

• Enforce system configuration into desired state

• Report on changes to configuration state

Puppet

Source: Cisco


• Open Source configuration management and systems orchestration with a kitchen theme

• Chef server provides local repository for cookbooks• Chef agent/client for NX-OS and IOS-XR

– https://opennxos.cisco.com/public/api/chef/

• Chef Supermarket is a repository for chef agents– https://supermarket.chef.io/cookbooks/cisco-cookbook

• Cisco Chef cookbook also on GitHub– https://github.com/cisco/cisco-network-chef-cookbook

Chef


• Ansible is an agentless DevOps tool that provides orchestration simplicity using human-readable YAML v1.2 files (www.yaml.org)

• Ansible playbooks are collections of plays, which are sets of tasks to be performed in sequence. A task is a granular work item.

• The Ansible server makes an SSH connection to the device(s) in the managed inventory and executes the task/play/playbook

• Third-party Ansible Modules on GitHub

– https://github.com/datacenter/nxos-ansible

• Ansible 2.0 (Q1CY16) will include complete support for Nexus switches

Ansible


• Nexus 9000 switches use Broadcom merchant silicon (Trident II ASIC) + Cisco custom silicon

• In the N9K Switch CLI there is a Broadcom Shell (bcm-shell) that allows you to access the Broadcom Network Forwarding Engines (NFE) with 64-bit Broadcom SDK client library.

• Allows you to access and read information from the T2 ASICs without any limitations– attach module <mod#>

– test hardware internal bcm-usd bcm-diag-shell

– bcm-shell module <mod#> [instance_number:command]

Broadcom Shell


• XMPP is an open standard protocol used to share near-real-time XML data (e.g. Jabber)

• XMPP is standardized by the IETF (RFC 6120, 6121, 6122) and XMPP.org

• XMPP can be used as a publish/subscribe method of signaling

• N9K NX-OS switches can use a Python XMPP module to subscribe to an XMPP/Jabber room/channel and receive signaling of configuration changes

• # yum install jabberd

Extensible Messaging and Presence Protocol (XMPP)


• NETCONF v1.1 is a configuration management protocol standardized by the IETF (RFC 6241)

• NETCONF communications use RPCs with XML formatted data, typically over SSH (RFC 6242)

• NETCONF requires secure connection-oriented communication, with authentication, and connection data integrity

• NETCONF separates operation from configuration and allows for configuration locking

• Nexus 9000 switches can be configured using NETCONF

• Python library for NETCONF clients– https://pypi.python.org/pypi/ncclient

NETCONF


• NX-OS with Splunk (Universal Forwarder) enables network operators to:– Gain visibility into their infrastructure– Track detailed network inventory– Track power usage and temperature– Authenticate and audit configuration changes– Collect performance data from network devices

• Tcollector runs on-box and gathers data from local collectors and pushes data to OpenTSDB

• Collectd is a daemon that collects system performance metrics and stores values in RRD files

• Ganglia is a distributed monitoring system for HPC clusters/grids, uses XML, XDR, and RRDtool

• Nagios remote plugin executor (NRPE) can monitor N9Ks through SSH or SSL tunnels

NX-OS Visibility


NX-OS and Splunk

Source: Cisco


• Cisco is opening up their software and allow for open source programmability and DevOps integration

• Cisco Nexus 9000 switches seem a lot like a Linux server: bash, vi, python, RPMs, yum, scheduler

• Cisco has provided many resource to help ease programmability: github repositories, help guides, examples, NX-API sandbox, NX-Toolkit

• Cisco Nexus 9000 switches use containers to allow for guest-shell, Puppet, Chef, Ansible integration

• Cisco Nexus 9000 switches are competitively priced, have high port density, high performance, low latency, and power efficiency

Summary


• Different Personalities of the Nexus 9000 in NX-OS Mode– https://cisco-

marketing.hosted.jivesoftware.com/community/technology/datacenter/data-center-networking/nexus-series-switches/blog/2015/06/06/different-personalities-of-the-nexus-9000-in-nx-os-mode

• Nexus NX-OS Programmability Guide– http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-

x/programmability/guide/b_Cisco_Nexus_9000_Series_NX-OS_Programmability_Guide_7x.html

• Cisco Nexus 9000 Series Switches: Integrate Programmability into Your Data Center– http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-

switches/white-paper-c11-733915.html

• Programmability and Automation with Cisco Open NX-OS – http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/nexus9000/sw/open_n

xos/programmability/guide/Programmability_Open_NX-OS.pdf

Additional Resources


• https://developer.cisco.com• https://developer.cisco.com/site/nx-api/• https://developer.cisco.com/site/devnet/sandbox/• https://learninglabs.cisco.com• https://opennxos.cisco.com• https://github.com/datacenter/nexus9000• Cisco Live 2015 San Diego Breakout Sessions

– BRKDCT-2459 - Programmability and Automation on Cisco Nexus Platforms

– BRKDEV-2003 - Programming the Network: Let's Get Started

– BRKSDN-1119 - Hitchhiker's Guide to Device APIs

– BRKDEV-2001 - DevOps in Programmable Network Environments

– PSODCT-2030 - How OpenNXOS enables more Open, Extensible, Modular and Flexibile Datacenters

Additional Resources (Cont.)


Thank you for your time!

Scott Hogg

303-949-4865 | [email protected] | @scotthogg

presented by scott hogg, cto gtri - meetupfiles.meetup.com/19296132/gtri - cisco nexus 9000...

Documents