nx-os in depthd2zmdbbm9feqrf.cloudfront.net/2011/anz/pdf/brkarc-3471.pdfnx-os origins & overview...

BRKARC-3471

NX-OS In Depth

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco PublicBRKARC-3471 2

Session Overview & Agenda

NX-OS Origins & Overview

NX-OS Modular Architecture

High-Availability Infrastructure

High-Availability Features & Capabilities

Command Line Interface

Operational & Management Features

Innovation

Licensing & Lifecycle


The Cisco Unified Fabric Family

Complete Data Centre class switching portfolio

Consistent Data Centre operating system across all platforms

Infrastructure scalability, transport flexibility and operational manageability

NX-OS Data Centre Operating System

Data Centre Network Manager (DCNM)

Nexus 2000Nexus 4000

Nexus 1000V

Nexus 7000 MDS 9000

Nexus 5000/5500


Cisco NX-OS Highlights

Feature Rich Operating SystemComprehensive L2 and L3 feature set

Modular, Multi-Threaded/ProcessorHighly scalable unprecedented uptime

Intelligent IOS-Like CLILittle or no retraining required

Zero Service DisruptionMaintenance ≠ Downtime

Virtualisation SupportIndustry first virtualised network OS, VN-Link

Layer 2 Multi-Pathing Resilient scalable Layer 2 domains

Unified Fabric FCoE, iSCSI, HPC

Advanced Management InfrastructureXML and Web Services

Designed to Meet the Operational Needs of the Data Centre


NX-OS: Designed for the Data Centre

NX-OSSAN-OS

IOS

CatOS

MDS 9000

Catalyst 6500

Nexus 7000/5x00/1000V


Comprehensive Data Centre Feature SetAvailable to all Platforms Layer 3 Distributed IPv4/IPv6 Hardware Forwarding OSPF, EIGRP, IS-IS, BGP, RIP, PBR PIM-SM, SSM/Bidir, MSDP, MP-BGP, IGMP/MLD 16-way ECMP (HSRP, GLBP, VRRP) + Object Tracking MPLS BFD

Virtualisation VRF-lite Virtual Device Contexts (VDCs)

High Availability In-Service Software Upgrade (ISSU) Non-Disruptive Stateful supervisor switchover (SSO) Stateful process restarts Graceful Process Restart

Operational Manageability GOLD, Smart Call Home, EEM w/ TCL NetFlow, NDE v5/v9, FNF CLI SPAN, ERSPAN Wireshark SNMP NETCONF/XML Configuration checkpoint & rollback

Layer 2 Distributed Hardware Based Layer 2 PVRST, MST STP Guards, Bridge Assurance, UDLD 802.1ad/LACP Portchannels Private VLANs Virtual Port Channel (vPC) Overlay Transport Virtualisation (OTV) Data Centre Bridging (DCB) Layer 2 Multipathing (FabricPath/TRILL)

Security RACLs, VACLs, PACLs Cisco TrustSec & LinkSec (CTS/802.1AE) CoPP & Rate Limiters DHCP snooping, DAI, IP source guard 802.1x & Port Security Storm control Unicast RPF check

Storage Area Networks FCoE FIP & FIP Snooping

Quality of Service Ingress/Egress queuing with WRED Marking Policies & Mutation Ingress/Egress ―1-rate 2-colour‖ & ―2-rate

3-colour‖ policing Colour-aware policing MQC CLI model


Comprehensive Data Centre Feature SetClosing in on IOS Parity Layer 3 Distributed IPv4/IPv6 Hardware Forwarding OSPF, EIGRP, IS-IS, BGP, RIP, PBR PIM-SM, SSM/Bidir, MSDP, MP-BGP, IGMP/MLD 16-way ECMP (HSRP, GLBP, VRRP) + Object Tracking MPLS BFD



Operational Manageability GOLD, Smart Call Home, EEM w/ TCL NetFlow, NDE v5/v9, FNF CLI SPAN, ERSPAN Wireshark SNMP (even more MIBs) NETCONF/XML Configuration checkpoint & rollback







Comprehensive Data Centre Feature SetInnovation for the Data Centre Layer 3 Distributed IPv4/IPv6 Hardware Forwarding OSPF, EIGRP, IS-IS, BGP, RIP, PBR PIM-SM, SSM/Bidir, MSDP, MP-BGP, IGMP/MLD 16-way ECMP (HSRP, GLBP, VRRP) + Object Tracking MPLS BFD



Operational Manageability GOLD, Smart Call Home, EEM w/ TCL NetFlow, NDE v5/v9, FNF CLI SPAN, ERSPAN Wireshark SNMP NETCONF/XML Configuration checkpoint & rollback







Nexus Certifications for NX-OS 5.1

IPv6 Ready Logo Phase I Certification Complete

FIPS Certification in Progress

Nexus 7010 has passed already

Nexus 7018: target completion date in Q1 CY2011

EAL4 Common Criteria in Progress

Target completion date in Q1 CY2011


NX-OS Non-Stop Forwarding

OS designed to leverage distributed hardware architecture

Fabric & forwarding engine removed from supervisor

Each I/O module has independent control-plane and forwarding hardware

Control-plane & data-plane separation

Fully distributed system for non-disruptive SSO & ISSU(also mostly true for Nexus 5x00)

Supervisor

(Control-Plane)

Fabrics

I/O Module

(Forwarding Engine)

EO

BC









Innovation




Kernel

Netstack

Management

Infrastructure

Hardware

Drivers

FeatureAPI

API

Feature Velocity

Faster Defect Resolution

Consistency

HA

Infrastructure

API


NX-OS Kernel

Stripped down Linux 2.6

kernel

Brings the benefits of Linux

Resilient Pre-emptive Multitasking

(~real-time)

Multi-threaded

Scalable Multi-CPU/Core support

Constant development and

enhancement Kernel

Netstack

Management

Infrastructure

Hardware

Drivers

FeatureAPI

API

HA

Infrastructure

API


NX-OS Platform Specific Portion

Chipset specific code

Provide Hardware

Abstraction Layer (HAL)

Ported per platform

Kernel

Netstack

Management

Infrastructure

Hardware

Drivers

FeatureAPI

API

HA

Infrastructure

API


NX-OS Netstack

Complete network stack

implemented in user

space

• L2 Packet Management/ARP

• IPv4/IPv6

• ICMPv4/ICMPv6

• TCP/UDP & Socket Library

Added Functionality

• Virtualisation (VDCs/VRFs)

• High-Availability (SSO)

Added system stability

Intellectual Property

Rights/Licensing

Kernel

Netstack

Management

Infrastructure

Hardware

Drivers

FeatureAPI

API

HA

Infrastructure

API


NX-OS Management Infrastructure

Provides CLI and

configuration interfaces

Provides SNMP agent

Provides NETCONF/XML

interface

Kernel

Netstack

Management

Infrastructure

Hardware

Drivers

FeatureAPI

API

HA

Infrastructure

API


NX-OS Feature/Service Granularity

Highly granular

implementations

Each service is an

individual memory

protected process

Including multiple instances

of particular service

Effective fault isolation

between services

Individually Monitored &

Managed Kernel

Netstack

Management

Infrastructure

Hardware

Drivers

FeatureAPI

API

HA

Infrastructure

API


NX-OS Feature/Service Granularity

Minimised failure domain

Streamlined deployment

Reduced attack surface

Improved bug triage

Kernel

Netstack

Manageme

nt

Infrastructur

e

Hardware

Drivers

HA Infra

UDLD SSH IGMP STP

HSRP

1OTV vPC

HSRP

2

OSPF

1

EIGR

PBGP

OSPF

2


NX-OS Conditional Features

Services (Protocols/Features) can be explicitly enabled/disabledN7K-1(config)# feature ?

bgp Enable/Disable Border Gateway Protocol (BGP)

dot1x Enable/Disable dot1x

eigrp Enable/Disable Enhanced Interior Gateway Routing Protocol

eou Enable/Disables feature l2nac(eou)

hsrp Enable/Disable hsrp (an example)

igmp Enable/Disable Internet Group Management Protocol (IGMP)

Disabling a service:•Releases associated resources

•Removes associated CLI

•Removes associated configuration


NX-OS High-Availability Infrastructure

Actually composed of 3

sub-services

System Manager

Message & Transaction

Service (MTS)

Persistent Storage Service

(PSS)

Kernel

Netstack

Management

Infrastructure

Hardware

Drivers

FeatureAPI

API

HA Infra

API


NX-OS Linecard Microcode

Microcode version of NX-

OS powers the linecards

Same foundations

Service processes on the

linecards are for

hardware and functional

support

Runs on linecard control-

plane CPU

Reinforces highly

distributed archtectureKernel

Netstack

Management

Infrastructure

Hardware Drivers

HA

Infrastructure

Port Manager

NetFlow Manager

FIB Manager


Nexus Unicast RoutingClient-Server Architecture

I/O Module

StaticRIP EIGRPIS-IS OSPF BGP

Unicast Routing Information Base (uRIB)

uFDM

uFDM & FIB Manager

FIB Hardware

•Manage

adjacencies/neighbours

•Add/Delete prefixes

•Route download

•Apply routing policy

•Select best-

nexthop(s) per prefix

•Program

hardware

forwarding engine

•Provides

common API to

Routing Protocols

Supervisor


NX-OS Platform Packaging and Delivery

Modular nature of NX-OS allows delivery of “permutations” based on

hardware capabilities

Kernel, core infrastructure code, and APIs remain consistent

Minimises development

Maximises code reuse & feature velocity

Kernel

Netstack

Managemen

t

Infrastructur

e

Hardware

Drivers

HA

Infrastructure

UDLD FCF FCOE STP

HSRP 1 OTV vPC HSRP 2

OSPF 1 EIGRP BGP OSPF 2

Kernel

Netstack

Managemen

t

Infrastructur

e

Hardware

Drivers

HA

Infrastructure

UDLD FCF FCOE STP



Kernel

Netstack

Managemen

t

Infrastructur

e

Hardware

Drivers

HA

Infrastructure

UDLD FCF FCOE STP



Nexus 7000 Nexus 5x00 Nexus 1000v


Single NX-OS System Image

NX-OS Platform Universal System ImageLicense-based Feature Management

Eliminate the guess work and decoder ring required to identify the right image to download & install

A+B

A+B

+C

A+C

A+B

+D

A+C

+D

A+D

Multiple SW Images NX-OS Base (A)

Enterpris

e License

(B)

Advance

d

License

(C)

Storage

Features

(D)

Future

License

Packages

Finding the right

image can be a

challenge!!!









Innovation



HA Infrastructure Component Relationship

PSS

Process

Process

Process

MTS

System

ManagerProcess


NX-OS Message & Transaction Service

Message relay system for IPC

communications

Provides reliable unicast & multicast delivery

Used for service-to-service and module-to-

module messaging

HA Infrastructure


MTS Unicast Message Delivery

Process A Process B Process C

MTS

Process A

Receive Queue

Process C

Message Buffer

Create Message:

src A, dst C1

Route Message:

src A, dst C2

Buffer Message:

src A, dst C3

Notify Process4

De-queue/Parse

Message5

ACK6

Notify Process7


MTS Multicast Message Delivery

Process A Process B Process C

MTS

Process A

Receive Queue

Process C

Message Buffer

Create Message:

src A, dst group 11

Route Message:

group 1 = dst B, dst C2

Buffer Message:

src A, dst B3

De-queue/Parse

Message5

ACK6

Notify Process7

Process B

Message Buffer

Buffer Message:

src A, dst C3

4 Notify ProcessNotify Process4

De-queue/Parse

Message5


Process Messaging Across Slots

Ethernet Out-of-Band Channel (EOBC)

Supervisor Linecard/ Supervisor

Process A

MTS

IPC Queue

Process Y

MTS

IPC Queue


NX-OS Persistent Storage Service

Lightweight key/value

database

Provides store options for

DRAM or NVRAM

API for services to store data

Used to maintain runtime

data/state

HA Infrastructure


NX-OS System Manager

Centre of service management and

fault recovery

Acts like *nix initd

Starts up configured features/services

Heartbeats received from services

HA Infrastructure


Hierarchical Fault Detection & Recovery

System Manager monitors services

• Exit codes for crashes

• Heartbeat for freeze-ups

• Can kill/restart child processes

Kernel monitors System Manager

Hardware monitors KernelSystem Manager

Feature

Kernel

Hardware

Feature Feature Feature


System Manager Fault Recovery Logic

Process

freezes/crashes

Process

restarted

recently?

Restart process

(Stateless)

Restart process

(Stateful)

Monitor service

process

Already

tried

stateless

restart?

Initiate SSO

Yes

No Yes

No









Innovation



Nexus High AvailabilityNX-OS – Stateful Process Restart

STP

OSPF

LACPSTP

OSPF

LACP

PSS

PSS = Persistent Storage Service

PSS provides reliable persistent storage to

the software components to „checkpoint‟

their internal state and data structures

enabling non-disruptive restart

No interaction with the

neighbour to recover state


NX-OS Stateful Process Restart

If a fault occurs in a process…

HA manager determines best recovery action (restart process, switchover to redundant supervisor)

Process restarts with no impact on data plane

Total recovery time: ~80ms

State is recovered, operation resumes

Restart process!

Kernel

Netstack

Management

Infrastructure

Hardware

Drivers

HA

Infrastructur

e

UD

LD

SS

H

IGM

P

ST

P

HS

RP

1

OT

V

vP

C

HS

RP

2

OS

PF

1

EIG

RP

BG

P

OS

PF

2

Control-Plane

Data-Plane

NX-OS services checkpoint their

runtime state to the PSS for recovery in

the event of a failure


Nexus 7000 High AvailabilityNX-OS – Graceful Restart & NSF

STP

OSPF

LACPSTP

OSPF

LACP

OSPF has already crashed once in last 4 min. Let‟s do a “Graceful Restart”

Graceful Restart requires

interaction with the neighbours

to recover


NX-OS Redundant Supervisor Model*

PSS

Proces

sProces

sProces

s

MTS

System

Manager

Proces

s

PSS

Proces

sProces

sProces

s

MTS

System

Manager

Proces

s

Active Supervisor Standby Supervisor

Ethernet Out-of-Band Channel (redundant 1GE)

*Applies to those system with dual supervisors


Active and Standby Supervisor Syncing

Services start in stdby mode

Active SUP

PSS

Service

System manager

MTS

Standby SUP

PSS

System manager

MTSStandby Online (all services gsync)

Service

Determine Active/Standby1

Request Initial States (gsync)2

Snapshot of Initial States3

Services Set Initial States4

Event-driven Syncing5

Initial State for

ServicesRuntime config

Runtime states/data


Stateful Supervisor Switchover

Active/Standby

Initial state synchronisation, subsequent event driven

sync keep active/standby in sync

Fast switchover time – State is already in place

Switchover initiated if:

repeated critical process restart failures

kernel failures

supervisor hardware failure


In-Service Software UpgradeN7K# install all kickstart bootdisk:5.0-kickstart system bootdisk:5.0-systemN7K#

N7K#

Sup 2Sup 1

Upgrade standby supervisor1

Reload standby supervisor2

Upgrade standby supervisor 4

Initiate SSO 3

Upgrade LCs in series6

Release

5.0Reload standby supervisor 5

Release

5.0

Release 5.0


Hitless ISSU on the Nexus 5x00

Difference in the detailed operation from Nexus 7k

Single supervisor vs. dual supervisor

Concept proven on MDS 91xx series

Enables hitless ISSU for N5k, its modules, and Nexus 2000s

Upgrades system, kickstart and BIOS images

During this time, control plane functions of the switch undergoing ISSU

are temporarily suspended, and configuration changes disallowed.

The control plane will be brought online again within 80 seconds to allow

protocol communications again.

Supports FEX Active/Active and Straight-Thru‟

Primary upgrades the FEX. It is the peer switch‟s responsibility to hold onto its state

until ISSU process is complete

From NX-OS 4.2(1)N1 Restrictions apply.


Nexus 5x00 ISSUPreconditions

•The ISSU process is executed through the installer, and

certain conditions must be satisfied before it can

proceed.Restriction on Configuration

changesRestriction on Topologies

& Topology Changes

•CLI and SNMP config change requests are denied during ISSU operations

•Module insertion not supported

•Network/Topology changes like STP, FC Fabric are not supported.

•Some management & FC services are unavailable.

See Docs for details









Innovation



NX-OS CLI Highlights

N7K(config)# int e1/1N7K(config-if)# ip address 192.168.0.1/23

Support for CIDR „slash‟ notation for IPv4/IPv6 masks

N7K(config)# show interface e1/1Ethernet1/1 is upHardware: 10/100/1000 Ethernet, address: 001b.54c1.5d44 (bia 001b.54c1.5d44)MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,reliability 255/255, txload 1/255, rxload 1/255

<snip>

Hierarchy Independent CLI allows „show‟ commands to be executed from exec-mode or config-mode

IOS-like but Improved

N7K# show cli history ?<CR> config-mode Display history of config commands onlyexec-mode Display history of exec commands onlythis-mode-only Display history from current mode onlyunformatted Display just the commands

N7K# show cli history config-mode12 05:20:34 int e1/113 05:20:42 where detail

Mode-aware CLI history

Show interface displays operational state + (reason)T-1# sh interf eth 2/3

Ethernet2/3 is down (linkNotConnected)

Hardware is Ethernet, address is 00:1b:21:06:32:71


Review Configuration with Flexibility

N7K# show running-config ntp

ntp server 171.68.10.80 use-vrf managementntp server 171.68.10.150 use-vrf managementntp source 172.26.244.101clock format 12-hoursclock format show-timezone

N7K# show running-config exclude aaa cert-enroll diagnostic ntp track acllog cfs eem radius vshdaclmgr cmp ipqos rpm callhome copp license security cdp dhcp monitor spanning-tree

Exclude features with lengthy configuration (e.g. ACL, QoS, etc.)

Compare between startup- and running-configurationN7K# copy running-config startup-config[########################################] 100%N7K# config terminalEnter configuration commands, one per line. End with CNTL/Z.N7K(config)# feature telnetN7K(config)# sh running-config diff *** Startup-config--- Running-config*** 1,11 ****feature lacp

--- 1,11 ----+ feature telnetfeature lacp

Identify the line number and difference between startup-config and running-config

Display feature-specific configuration


NX-OS CLI Output Piping

Variety of advanced pipe options for CLIdemolab-N7K-10# sh run | ?

egrep Egrep

grep Grep

less Stream Editor

no-more Turn-off pagination for command output

wc Count words, lines, characters

begin Begin with the line that matches

count Count number of lines

exclude Exclude lines that match

include Include lines that match

demolab-N7K-10# sh run | egrep ?

-A Print <num> lines of context after every matching line

-B Print <num> lines of context before every matching line

-c Print a total count of matching lines only

-i Ignore case difference when comparing strings

-n Print each match preceded by its line number

-v Print only lines that contain no matches for <expr>

-w Print only lines where the match is a complete word

-x Print only lines where the match is a whole line

WORD Search for the expression

demolab-N7K-10# sh run | egrep -A 2 -B 2 ospf

interface Ethernet2/22

ip address 10.2.22.1/24

ip router ospf 10 area 0


ip address 10.2.23.1/24



--

interface loopback0

ip address 10.255.255.1/32


router ospf 10

hostname demolab-N7K-10

demolab-N7K-10# sh run | in ospf | wc -l

4

demolab-N7K-10#

Supports multilevel piping

Powerful & Flexible output manipulation built-in


NX-OS running-config permutations

“show running-config” works as expected, but with many other enhancements

N7K# show running-config ?

<CR>

> Redirect it to a file

aaa Display aaa configuration

all Current operating configuration with defaults

am Display am information

arp Display arp information

bgp Display bgp information

callhome Display callhome configuration

cdp Display cdp configuration

cmp Display CMP information

copp show running config for copp

dhcp Display dhcp snoop configurations

diagnostic Display diagnostic information

diff Show the difference between running and startup configuration

dot1x Display dot1x configuration

eem Show the event manager running configuration

eigrp Display eigrp information

icmpv6 Display icmpv6 information

igmp Display igmp information

interface Interface configuration

ip Display ip information

ipqos show running config for ipqosmgr


NX-OS Interfaces Differences from IOS

No hidden interface configuration like that in IOS

Configuration is deleted when interface functional type is changed

User needs to (re)configure as needed for L2 or L3 modes

Interface Identifier Keywords

Port-channel replaces Etherchannel (no PAgP – just LACP)

All Ethernet interfaces are simply called “Ethernet”

No more FastEthernet, GigabitEthernet, TenGigabitEthernet

Show output formatting slightly different









Innovation



Embedded WireShark AnalyserEthAnalyzer Real-time, on-the-device protocol analyser provide

ultimate visibility into various traffic hitting CPU from remote locations

Control

ProcessorData

Traffic mgmt0Inband

Monitor traffic from

inband and mgmt0

interfaces to the

Control Processor

Extensive capture and

display options,

including to file (.pcap)

Capture rules/filters


NX-OS ConfigurationCheckpoint & Rollback

System and user generated

checkpoints

System checkpoint

automatically created when

any conditional features are

disabled

User-defined checkpoint can

be initiated from CLI

Rollback to any checkpoint

allows easy recovery

Facilitate change-management with configuration snapshots

Current Running

Configuration

System

Checkpoi

nt

New Running

Configuration

User-Defined

Checkpoint

Ro

llb

ack


Checkpoint & Configuration RollbackExamples

N7K(config)# no feature vpcN7K(config)# sh checkpoint summary System Checkpoint Summary-------------------------------------1) system-fm-vpc:Created by adminCreated at Fri, 16:51:40 06 Nov 2009Size is 24,567 bytesDescription: None

System-checkpoint created automatically upon feature removal

User-defined checkpoint with description simplifies configuration managementN7K# checkpoint 2009-11-06 description SQL DC ACL UpdateN7K# sh checkpoint summary User Checkpoint Summary-------------------------------------1) 2009-11-06:Created by adminCreated at Fri, 18:33:41 06 Nov 2009Size is 25,773 bytesDescription: SQL DC ACL Update

Flexible option for configuration rollbackN7K# rollback running-config checkpoint 2009-11-11 ?<CR> atomic Stop rollback and revert to original configuration

(default) best-effort Skip errors and proceed with rollbackstop-at-first-failure Stop rollback at the first errorverbose Show the execution log

Default name for system-checkpoint, ‘system-fm-

xxx’

Timestamp of checkpoint help

configuration management


foo

Layer 3

OSPF 300

OSPF Area 0

OSPF Hello 1s

NX-OS Port-ProfilesSimplify the configuration of groups of interfaces

Enables the application of common configuration

across groups of ports

A port-profile can inherit attributes from other port-

profiles (nested profiles)

A change to a port-profile automatically updates

configuration of all member ports

Any interface command available on a Nexus

interface can be a part of a port-profile

e.g. ACL, L3, VLAN, etc.

Configuration precedence/order:

Default config. < Port-profile < Manual config.

Speed/Duplex

100 Mbps

Full Duplex

QoS

Service Policy Input

E2/1 E7/9 E11/4

port-profile foo

speed 100

duplex full

service-policy input xyz


ip ospf hello-interval 1

Interface e2/1,e7/9,e11/4

port-profile foo


Provides flexible user access control based on a framework

Role-based Access Control (RBAC)

Security

Admin

L3

Admin

Network

Admin

VDC

Admin

User privileges defined by roles simplifies command authorisation

Rules are pre-classified based on feature groups and components

Manually defined roles are local but can be distributed via CFS protocol

Some common roles pre-defined for fast deployment


NX-OS RBAC provides more options for AAA

RBAC vs. Traditional AAA

Traditional

AAA RBAC

User Authorisation Based on Privilege Level

User Authorisation Based on Roles Assigned

Per-Command Authorisation

Feature-aware Command Authorisation

Centralised Accounting Log

Local Accounting Log

Distribution of RADIUS/TACACS+ Configuration


NX-OS XML Integration

<?xml version=”1.0”?><rpc message-id=”101”

xmlns=”urn:ietf:params:xml:ns:netconf:base:1.0”><edit-config>

<target><running/>

</target><config>

<xs:interfacexmlns:xs=“http://www.cisco.com/SANOS/1.0/interface”>

<xs:mgmt><xs:Naming>

<xs:intf>0</intf></xs:Naming><xs:ip>

<xs:address><xs:host>1.1.1.1</xs:host><xs:netmask>255.255.255.0</xs:netmask>

</xs:address></xs:ip>

</xs:mgmt></xs:interface>

</config></edit-config>

</rpc>

Remote management via NETCONF/XML

Pipe CLI command output to XML formattingN7K# show int e1/1 | xml<?xml version="1.0" encoding="ISO-8859-1"?><nf:rpc-reply xmlns:nf="urn:ietf:params:xml:ns:netconf:base:1.0"

xmlns="if_manager"><nf:data><show><interface>

<TABLE_interface><ROW_interface><interface>Ethernet1/1</interface><state>up</state><eth_hw_desc>10/100/1000 Ethernet</eth_hw_desc><eth_hw_addr>001b.54c1.5d44</eth_hw_addr><eth_bia_addr>001b.54c1.5d44</eth_bia_addr><eth_mtu>1500</eth_mtu>…

Eliminate/Simplify screen

scraping for output data

Human-readable format – easier

parsing

Future-proofing through open and

flexible standard protocol

NetConf over SSH for security

XML API allow easy integration

with 3rd-party NMS applications

http://www.cisco.com/SANOS/1.0/interface


Session Overview & Agenda NX-OS Origins & Overview






Innovation

Virtual Device Contexts

Virtual Port-Channels

FEX-Link

FabricPath

Overlay Transport Virtualisation

Fibre Channel over Ethernet



Various Degrees of Virtualisation

VRFs and VLANs

Logical separation of data-plane (and some

control-plane) functionality

Virtual Contexts (i.e. Firewalls, ACE, etc.)

Logical separation of configuration or

management and data-plane

Virtual Device Contexts

Logical separation of control-plane, data-plane,

management, resources, and system processes

Data/Control Plane

Data/Control Plane+

Management Plane

Data/Control Plane+

Management Plane+

Resources+

Operating Environment

Hypervisor Model


Virtual Device Contexts (VDCs)

VDC—Virtual Device Context

Flexible separation/distribution of

Software Components

Flexible separation/distribution of

Hardware Resources

Securely delineated

Administrative Contexts

VDCs are not…

The ability to run different OS levels on

the same box at the same time

based on a hypervisor model; there is

a single „infrastructure‟ layer that

handles h/w programming…

Infrastructure

Layer-2 Protocols Layer-3 Protocols

VLAN mgr

STP

OSPF

BGP

EIGRP

GLBP

HSRP

VRRP

UDLD

CDP

802.1XIGMP sn.

LACP PIMCTS SNMP

RIBRIB

Protocol Stack (IPv4 / IPv6 / L2)

Layer-2 Protocols Layer-3 Protocols

VLAN mgr

STP

OSPF

BGP

EIGRP

GLBP

HSRP

VRRP

UDLD

CDP

802.1XIGMP sn.

LACP PIMCTS SNMP

RIBRIB

Protocol Stack (IPv4 / IPv6 / L2)

Kernel

VDC A

VDC B

VDC A VDC B

VDC n


Virtual Device Contexts (VDCs)

Typical silo/stovepipe design

Production, Development, Test

Intranet, Internet, DMZ, Extranet

Application A, Application B, Application C

Customer A, Customer B, Customer C

Cluster A, Cluster B, Cluster C

Storage Replication, Secure Transaction DB

VDCs enable collapsing of physical

infrastructure into logical infrastructure

Preserves security, administration, and

organisational boundaries, & fault isolation

“The results clearly demonstrate that VDCs can be

effectively deployed as though they are physically

separate devices”

Source: NSS Labs

Physical network

islands are virtualised

onto common Data

Centre infrastructure

VDCExtranet

VDCProd

VDCDMZ

Ref: http://nsslabs.com/research/network-security/virtualization/cisco-nexus-7000-q2-2010.html

http://nsslabs.com/research/network-security/virtualization/cisco-nexus-7000-q2-2010.html












vPC on Nexus

Virtual Port Channel (vPC) Objectives

Provides a loop-free topology

Maximises bandwidth / lower over-subscription

Improved convergence & availability

logical equivalent


5 6 7 8

1 2 3 4

Up-to 32 Ports

vPC Topology Example―Back-to-Back‖

vPC member

Routed Interface

Host Port

Two layers of vPC peers

can be connected back-to-

back e.g. N7k to N5k

Opportunity for very high

bandwidth using an

evolutionary development

of STP

Up to 32-way port-channel

Nexus

7000

Nexus

5x00

Nexus

2000


FEX-Link: Extending the Fabric

• Nexus 7000/ 5x00 + FEX is like a “Virtual Chassis”

• Nexus 2000 FEX is a “Virtual Line Card” to its “parents”

• No Spanning Tree between the FEX and its “parent”

• No local switching on the FEX

• NX-OS Linecard code runs on the 2148/2248/2232

Fabric Extender (FEX)

Nexus 5x00 Nexus 7000


Access Layer with Nexus 2000

Physical view(Efficient cabling)

Top of Rack:

minimises cable runs in high-density deployments

Logical view(End of Row: efficient management)

Combines benefit of ToR

and EoR architecture

• Physically resides on the top of each server rack, Logically acts like an end of access row device

• Reduces cable runsMajority of physical cabling is within the rack, <2m cable

• Reduce management points in the network1,500 server HA network with 48-port ToR access switches: 34 management points1,500 server HA network with Nexus 7000 and Nexus 2000 : 2 management points1,500 server HA network with Nexus 5500 and Nexus 2000 : 4 management points

• Easier to ensure feature consistency across hundreds or thousands of server ports


Cisco FabricPathScaling and Simplifying Layer 2 Ethernet Networks

-All Links ActiveTraditional Spanning Tree Based Network

Up to 16 Agg

switches

Up to 32

access

switches

-Blocked Links

Cisco FabricPath Network

160+ Tbps

switching

capacity

Eliminate Spanning tree limitations

High resiliency, faster network re-convergence (~160mS)

Any VLAN Anywhere in the Fabric

Multi-pathing across all links, high cross-sectional bandwidth

Nexus 7000 today, with Nexus 55xx in H2 CY11


FabricPath in the NewsIndependent Test from Network World

http://www.networkworld.com/reviews/2010/102510-cisco-fabricpath-test.html

Testing Highlights

Impressive performance: six Nexus 7010 with 12,800 emulated hosts

No multicast performance penalty: Highly scaled Multicast environment

Fast Fabric failover: Zero to ~160 milliseconds

Data Centre Network Manager: Efficient Tool to configure and monitor FabricPath Networks









Virtual Machine MobilityOverlay transport Virtualisation

OTV

AED

AED

OTV

OTV

OTV

MAC X

MAC X

MAC X

MAC X

VM moves

MAC X

Local MAC = Blue

Remote MAC = Red

AED

West East

OTV

OTVWest

AED

OTV

OTV

MAC X

MAC XMAC

X

East

Site West see MAC X advertisement with a better

metric from site East and changes them to remote

MAC address.

MAC X

MAC

X


Mapping of FC Frames over

Ethernet

Enables FC to Run

on a Lossless

Ethernet Network

Fewer Cables

Both block I/O & Ethernet

traffic co-exist on same

cable

Fewer adapters needed

Overall less power

Interoperates with existing

SAN‟s

Management of SAN‟s

remains constant

No Gateway

FCoE Benefits

Fibre

Channel

Traffic

Ethernet

Unified FabricFibre Channel over Ethernet (FCoE)

Nexus 5x00 today, with Nexus 7000 and MDS following in H1 2011









Innovation



NX-OS Licensing

Grace Period

Enables features to be run for a certain

period without installing a license

Allows feature testing/trials without buying

a license (e.g. 120 days)

Exceptions exist

Periodic syslog, callhome and SNMP traps

warning when grace period nears expiry

Time-Bound Licenses

License with expiry date

Currently used in SAN-OS as an emergency when grace period is over and need time to buy license

Expiry date is absolute(expires at midnight UTC on expiry date)

Periodic syslog, callhome and SNMP traps warning when time bound license nears expiry

After expiry date feature will continue to run if grace period has not been exhausted

License PAK

(product activation key)

www.cisco.com

PAK +

chassis serial #

<xml...

licA ...>

license

file Licenses are enforced on the switch# show license host-id

License tied chassis serial # stored in dual redundant NVRAM modules on backplane

Licenses are issued in the form of a digitally signed text file

# install license bootflash:N7K-1234.lic

http://www.cisco.com/


License Delivery ProcessPhysical or eDelivery

~ 1.5 days minutes minutes

Order Entry

BookedReceive Order

Generate License Doc

Notify Customer

Download

eDelivery Process

Current Physical Delivery Process

~ 3 days several days 2 – 14 days

Order Entry

BookedSchedule

dDispatch Packout Ship Delivery


Base

Enterprise Advanced Enhanced L2 FCoE*

MPLS XLTransport

Services

NX-OS Software PackagingLicenses Overview

NEW in 5.1

Simplified Software ManagementFive NX-OS enforceable licenses enable full suite of functionalities for DC deployment

Less Costly Software UpgradesNX-OS feature upgrades can be done by enabling a new license key, reducing the need for truck-roll to remote locations

Enable Development of New Software Based Business Models “Pay-as-you grow”

Licenses are independent of each other, i.e. not cumulative

Transport Services evolution (tentative): L2VPN (EoMPLS and VPLS)

* Per Module-based license


NX-OS Release Sync Across Platforms

5.1(1) 5.25.0(2) 5.1(2)5.0(x)5.0(1) 5.1(3)

Q2CY2010 Q4CY2010Nexus 7000

Nexus 5000

Sync SyncSnapshotSync

Complete sync done at major releases Architectural changes

Major enhancements

Major new features

Partial sync done at minor releases Critical flaws/bugs

Minor new features

Minor enhancements

Platform specific interim releases Addresses platform specific bugs or enhancements

5.0(2)N1 5.0(x)N1 5.1(1)N1 5.1(1)N2 5.1(3)N3 5.2N1

Single PI train strategy on roadmap Features consistency across Nexus


N5K 5.0 N5K 5.x Future Release

N7K 5.0 N7K 5.1 Future releases

2HCY09 1HCY10 2HCY10 1HCY11 2HCY11 1HCY12 2HCY12

NX-OS 4.2

FCSNX-OS 4.2 EOS*

Maintenance every 3 months

Over 18-24 months

N5K 4.2 FCS N5K 4.2 EOS*Maintenance every 3 months

Over 18-24 months

NX-OS 5.2

FCS

NX-OS 5.2

EOS

Maintenance every 3 months

Over 18 months

N5K 5.2 FCS N5K 5.2 EOSMaintenance every 3 months

Over 18 months

NX-OS Software Life CycleLong-lived release candidates

*Actual 4.2 EoS is To Be Advised

Nexu

s 7

00

0N

exu

s 5

00

0

Short-lived release candidates

4.2(x) is the stable GD train for NX-OS on N7K

5.2 is candidate GD train for NX-OS on N7K and N5K


NX-OS Software ArchitectureSummary: Top Things to Remember

NX-OS has been built around High-Availability as a core principle

Continuous Innovation to enhance the Data Centre & the

emergence of Virtual Machines

Based on proven SAN-OS & a lineage of battle-hardened

features/protocol implementations

Practical experience reflected in operational support

NX-OS‟ highly-granular modularity provides streamlined

deployments & improved security

NX-OS is built to compartmentalise, scale (up or down), be

portable, and extendable


BRKARC-3471Recommended Reading

http://www.cisco.com/go/nexus

http://www.cisco.com/go/nexus


Q & A


Complete Your Online Session Evaluation

Complete your session evaluation:

Directly from your mobile device by visiting

www.ciscoliveaustralia.com/mobile and login by

entering your badge ID (located on the front of

your badge)

Visit one of the Cisco Live internet stations

located throughout the venue

Open a browser on your own computer to

access the Cisco Live onsite portal

http://www.ciscoliveaustralia.com/mobile

nx-os in depthd2zmdbbm9feqrf.cloudfront.net/2011/anz/pdf/brkarc-3471.pdfnx-os origins & overview...

Documents