unified computing system (ucs) 2.0 - new hardware...

Matthew Wronkowski – Escalation Engineer

Jose Martinez – Escalation Engineer

Unified Computing System (UCS) 2.0 - New Hardware & Software Features

© 2011 Cisco and/or its affiliates. All rights reserved. 2

Cisco Support Community – Expert Series Webcast  Today’s featured experts are Cisco Support

Escalation Engineers Jose Martinez and Matthew Wronkowski

 We also have Expert Panelists Dave McFarland & Mike Timm to help answering technical questions during the live session

2


Thank You for Joining Us Today Today’s presentation will include audience polling

questions

We encourage you to participate!


Thank You for Joining Us Today If you would like a copy of the presentation slides, click the

PDF link in the chat box on the right or go to

https://supportforums.cisco.com/community/netpro/data-center/unified-computing

Or, https://supportforums.cisco.com/docs/DOC-22551


Polling Question 1

a)  UCSM 2.0(1)

b)   UCSM 1.4(1) or 1.4(3)

c)  UCSM 1.3(1)

d)   Don’t have a UCS system at this time

e)  Other release

What Cisco UCS Manager (UCSM) software are you currently using?


Submit Your Questions Now Use the Q&A panel to submit your questions. Experts will start responding those

Matthew Wronkowski – Escalation Engineer

Jose Martinez – Escalation Engineer

Unified Computing System (UCS) 2.0 - New Hardware & Software Features


Agenda  UCS 2.0 Hardware Introduction

UCS 6248 Fabric Interconnect UCS 2208XP I/OM Port-channel support VIC 1280 adapter

 UCS 2.0 Software Introduction iSCSI boot support Disjointed L2 support VMDirectPath with vMotion RedHat KVM with VM-FEX HDD Health Status

UCS 2.0 Hardware Introduction UCS 6248UP Fabric Interconnect


UCS 6248UP Fabric Interconnect   2nd Generation of 1RU Fabric Interconnect

  PID: UCS-FI-6248UP

  Total of 48 unified ports. 32 base board + 16 GEM

  1/10GE and 1/2/4/8FC option

  Slider based configuration for ports

  New ASICs (Carmel / Sunnyvale) which improves many aspects of the switching capabilities:

•  Lower latency (2 us) •  Bigger TCAM size (4k) •  Higher number of active VLANs (4k) •  Higher number of IGMP entries (2k)

•  Better buffer allocation (higher number of buffer per port, number of unicast/multicast VoQ and egress queues among other)


UCS 6248UP Fabric Interconnect

Power Entry Power Entry Fan Module Fan Module

Out of Band Mgmt 10/100/1000

Console

Fabric Interconnect Cluster Connectivity

N + N Redundant FANs N + N Power Supplies

32 x Fixed ports: 1/10 GE or 1/2/4/8 FC Expansion Module (GEM)


UCS 6248UP Unified Ports

Use-cases

Native Fibre Channel

  Flexible LAN & storage convergence based on business needs

  Service can be adjusted based on the demand for specific traffic

FC Eth

Lossless Ethernet: 1/10GbE, FCoE, iSCSI, NAS

  Simplify switch purchase - remove ports ratio guess work

  Increase design flexibility

  Remove specific protocol bandwidth bottlenecks


UCS 6248UP Unified Ports

  Ports on the base card or the Unified Port GEM Module can either be Ethernet or FC

  Only a continuous set of ports can be configured as Ethernet or FC

  Alternating Ethernet or FC are not supported

  Ethernet Ports have to be the 1st set of ports

  Port type changes take effect after next reboot of switch for Base board ports or power-off/on of the GEM for GEM unified ports.

Base card – 32 Unified Ports GEM – 16 Unified Ports

Eth FC Eth FC


UCS 6248UP Unified Ports •  Each fabric is configured

independently

•  Click on Fabric Interconnect

•  Click on Configure Unified Ports

•  Reboot warning displayed

•  Require approx. 6 minutes

•  Change SFP!

UCS 2.0 Hardware Introduction UCS 2208XP I/OM


UCS 2208XP I/OM

•  2nd Generation of IOModule (FEX)

•  PID: UCS-IOM-2208XP

•  8x SFP+ uplinks for fiber or copper/twinax

•  Total bandwidth per chassis 160Gig (80Gig per fabric)

•  32x GE ports to the backplane

•  Access to blades can be discrete (pinned) or a port-channel connection

•  Uses Woodside ASIC (Similar to Nexus 2232)

•  Can operate with 1st Generation hardware

•  Non-disruptive upgrade path


UCS 2208XP I/OM Throughput Increase

1st Generation 2nd Generation

2208XP

80 Gig Per fabric


UCS 2208XP I/OM Discrete Mode

Slot 1 Slot 2

Slot 3 Slot 4 Slot 5 Slot 6

Slot 7 Slot 8

Slot 1 Slot 2

Slot 3 Slot 4 Slot 5 Slot 6

Slot 7 Slot 8

Double blade throughput using “Discrete” mode

10Gb per blade 20Gb per blade

1st Genera8on 2nd Genera8on

UCS 2.0 Hardware Introduction Port-Channel Support


Port-channel Support

  Up to 8 * 10 Gb aggregated bandwidth to each fabric in a chassis.   Only supported with combination of 6248 FI and 2208 IOM.   Port-channel is optional, default mode discrete (same as 1.4)   UCSM automatically configures fabric-port-channel based on Discovery

Policy and Hardware Capability   Per chassis connection policy   Discrete to port-channel is per side change to minimize disruption (no

explicit re-ack is required)   Better redundancy. Add/remove PO members will cause traffic re-

distribution   Supported no of links in a port-channel: 1,2,3,4,5,6,7 or 8   Load Balancing

•  Better traffic distribution among IOM uplinks and thus better link utilization •  Load balancing NOT user-configurable, implicitly managed by UCSM


Chassis Discovery Policies •  Global Discovery Policy

•  Per-Chassis Policy


UCS I/OM Mode Comparison 4 LINKS, DISCREET

slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 slot 8

F E X

8 LINKS, DISCREET

slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 slot 8

8 LINKS, PORT-CHANNEL

Available Bandwidth Per Blade—10 Gb (5gb Per Side)

Available Bandwidth Per Blade—20 Gb

(10 Gb Per Side)

Available Bandwidth Per Blade—up to 160 Gb

(80 Gb Per Side)

Fabric Interconnect

Fabric Interconnect

Fabric Interconnect

F E X

F E X

•  Statically pinned to Individual fabric links

•  Deterministic Path

•  Statically pinned to Individual fabric links

•  Deterministic Path •  No oversubscription, each

blade gets 20 Gb

•  Statically pinned to Port-channel

•  Shared bandwidth, better bandwidth utilization.

UCS 2.0 Hardware Introduction UCS VIC1280


UCS VIC 1280

  Dual 4x 10 GE port-channels to a single server slot   Host connectivity PCIe Gen2 x16   HW Capable of 256 PCIe devices

•  OS restriction apply   PCIe virtualization OS independent (same as M81KR)   Single OS driver image for both M81KR and 1280 VIC   Fabric Failover supported •  128 VIFs (116 user VIFs) when paired with UCS 6148 FI and 2208 I/OM •  Port-channel configuration done automatically by system


UCS VIC 1280

UCS Blade Server

VIC 1280

2208 IOM 2208 IOM

Fabric Interconnects

Adaptor Links PC

(user configurable)

(system configured)


UCS VIC 1280

Pinned to PC

IOM 2208 VIC 1280 adaptor with DCE links in Port Channel

Regular adaptor with single 10G link

•  No slot based pinning •  Port-Channel to Port-Channel pinning •  No Invalid Link Count


New Hardware Compatibility Fabric Interconnect

IOM Adapter Connectivity benefits per side

6100 2104 UCS M81 KR Fabric and host connectivity same as Balboa 1.4(1)

6100 2208 UCS M81 KR All 8 IOM ports in discrete mode 10 Gb to the host

6100 2104 UCS1280 VIC 4 IOM ports in discrete mode 5 Gb to the host

6100 2208 UCS1280 VIC All 8 IOM ports in discrete mode 40 Gb to the host

6200 2104 UCS M81 KR 4 IOM ports in discrete mode 5 Gb to the host

6200 2208 UCS M81 KR All 8 IOM ports can be leveraged in Port-channel mode 10 Gb to the host

6200 2104 UCS1280 VIC All 8 IOM ports in discrete mode 10 Gb to the host

6200 2208 UCS1280 VIC All 8 IOM ports can be leveraged in Port-channel mode 40 Gb to the host


Maximum VIFs supported by hardware

Generation 1 hw Generation 2 hw

Fabric Interconnect

61xx (Gatos) 512 max vifs

128 max vifs (per switch port)

62xx (Carmel) 4096 max-vifs

Flexible allocation across switch ports

IO Module 2104 (Redwood) Pre-defined vntag offsets

for 8 slots

2208 (Woodside) Dynamic global vntag

translation Adaptor

Card M81KR (Palo)

128 vntag namespace VIC 1280 (Sereno)

256 vntag namespace


VIF Allocation Scheme

Max no. of user vifs (vnics+vhbas) available to a slot (adaptor) = (15*n) – 2 n = no. of acknowledged iom uplinks

Max no. of user vifs (vnics+vhbas) available to a slot (adaptor) = (63*n) – 2 n = no. of acknowledged iom uplinks

•  Generation 1

•  Generation 2


Polling Question 2

a)  Yes, UCS 6248 FI

b)   Yes, UCS 2208XP I/OM

c)  Yes, UCS 6248 FI and UCS 2208XP I/OM

d)   Not yet

Have you deployed new hardware in your environment?


Submit Your Questions Now Use the Q&A panel to submit your questions. Experts will start responding those

UCS 2.0 Software Introduction iSCSI Boot Support


iSCSI Boot Support   Allows stateless model with no FC or FCoE in the environment   Limited adapter support

•  Cisco VIC and VIC-1280 (iBFT only, no iSCSI offloads) •  Broadcom 57711 M51KR (full offload, iSCSI HBA)

  Operating System support •  ESXi 4.1U1 •  Windows 2008 R2 and R2SP1 •  RHEL 5.6, 6.0 and 6.1

  iSCSI Hardware Offload is not a requirement to support booting, only supporting iSCSI Boot Firmware Table (iBFT) in the option ROM

  This is the first UCS release represents an iSCSI device in the model as well as the GUI/CLI/API

  New pools and policies to support iSCSI vNIC attributes in the LAN tab


iSCSI Service Profiles

  A service profile for iSCSI boot will have subtle changes from a normal Service Profile. There is now an iSCSI vnic that needs to be created. Creating a Service Profile iSCSI boot must be done from the Expert Service Profile creation tool.

  There is an iSCSI adapter policy that needs to be used with the Service Profile. We will go over the policy and variables and what they do.

  There is also a new IP pool that can be used to automatically assign IP addresses to the iSCSI vnics.

  One caveat is an option for a pool of iqn initiator names that can automatically be assigned to iSCSI intitiators. For now this needs to be filled in by hand by the user. An enhancement for this is in the works.


iSCSI Adapter Policy   Broadcom

•  For installs, Boot to Target must be disabled. This forces the blade to bypass a bootable disk image and gain access to vMedia/DVD/PXE for installation

•  After installation, Boot to Target must be enabled •  You can create an install policy and boot policy and change the SP as needed •  Or set the boot policy and during the initial install, break into Broadcom option

ROM and disable Boot to Target (one time only) •  OS support for TCP off-load

  VIC •  No TCP off-load support •  Boot-to-Target is not used

UCS 2.0 Software Introduction Disjointed L2 Support


Disjointed L2 Support   Non-symmetric VLANs on uplink

  Multiple Layer 2 networks upstream

  Native support in End-Host mode allows simplification of configuration (no need for Switch mode)

  By default 1.4 behavior is preserved (all VLANs present in all uplinks)

  Hardware independent feature (supported in 6100 and 6200 FI)

  Max of 31 disjointed Layer 2 domains supported

  Overlapping VLAns are not supported

  Per VLAN broadcast/multicast listener


Disjointed L2 Support •  UCS pre-2.x

Network 1 VLAN 10,20,31,32

FI-A BIF 1 BIF 2 BIF 3

FI-B BIF 1 BIF 2 BIF 3



Network 1 VLAN 10,20

BIF 1 BIF 2 BIF 3



BIF 1 BIF 2 BIF 3

•  UCS 2.x

FI-A FI-B

UCS 2.0 Software Introduction VMDirectPath with vMotion


VMDirectPath with vMotion

  Data traffic from VM bypasses hypervisor

  Available in vSphere 5.0

  Other names it is known by •  VM-FEX High Performance Mode •  UPT •  VMDirectPath Gen2

  Co-exists with standard mode – Cisco VIC required

  Does not follow the vNetwork distributed switch model –PCI devices explicitly assigned

  UCS specifically called out in vSphere 5 Networking Guide


Modes of VM-FEX Standard Mode High Performance Mode (UPT)

Standard Mode  Each VM gets a dedicated

PCIe device  12%-15% CPU performance

improvement  Appears as distributed

vNetwork switch to hypervisor

 vMotion supported

High Performance Mode   Co-exists with Standard

mode

  Bypasses Hypervisor layer

  30% improvement in I/O performance

  Appears as distributed vNetwork switch to hypervisor

  vMotion supported


VMDirectPath Rules of Engagement

  A VM vNIC always connects to VM-FEX in standard mode. If it is marked for UPT, ESX will attempt to switch it to UPT after few seconds. ESX will always switch a VM vNIC to standard mode before disconnecting from VM-FEX. Ex. vMotion.

•  During mode switches, some packets are dropped. This is acceptable because the higher level protocols will retransmit.

•  The mode switches are invisible to the guest OS. He does not know if his vNIC is in standard or UPT mode

•  The mode switch between is not a link transition (up/down) on the network. As said before, it is invisible to the OS.

UCS 2.0 Software Introduction RedHat KVM with VM-Fex


RedHat KVM with VM-Fex   Adapter FEX extends port-extender (Pre-standard 802.1Qbh) architecture

to PCIe bus on hosts – to virtualize the network/cable/NIC construct

  VM-FEX further extends port-extender architecture to virtual machines

  Each VM gets its own PCIe device and a virtual port on the UCS Fabric Interconnect

  Architecture for VMware has been around since ESX4.0 update 1

  VM-FEX with KVM works with the VIC adapters for hardware-based switching to VM interfaces


RedHat KVM with VM-Fex

……

UCS Switch

Cis

co V

IC

……

Use

r K

erne

l Macvtap Interface

Netdev Interface

Netlink Socket

virtio-net

eth1

Macvtap 1

vhost-net

KVM Guest 2

virtio-net

Guest 1

eth2

Macvtap 2

vhost-net

eth0 ethn

Port Profile1: Qos1, vlan1




UCS 2.0 Software Introduction HDD Health Status


HDD Health Status   New HDD Fault Monitoring provides UCSM with blade-resident HDD

status

  Important because: Previous releases provided no HDD status to UCSM

  Feature only applies to B200 and B250 blade (LSI 1064E Raid Controller)

  No support for c-series

  Only error conditions that can be reported by the LSI1064E to the CIMC are covered

  For each HDD slot only one unidirectional fault signal from the LSI1064E to the CIMC


References   Cisco Support Community

https://supportforums.cisco.com/

  Unified Computing System Community https://supportforums.cisco.com/community/netpro/data-center/unified-computing

  Cisco.com pages http://www.cisco.com/en/US/products/ps10265/index.html

  Support pages: http://www.cisco.com/en/US/partner/products/ps11544/tsd_products_support_series_home.html

  Designing Secure Multi-Tenancy into Virtualized Data Centers http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/Virtualization/securecldg.html


Books   Cisco Unified Computing System (UCS)

http://www.ciscopress.com/bookstore/product.asp?isbn=1587141930

  I/O Consolidation in the Data Center http://www.ciscopress.com/bookstore/product.asp?isbn=158705888X

  Network Virtualization http://www.ciscopress.com/bookstore/product.asp?isbn=1587052482

  Data Center Fundamentals http://www.ciscopress.com/bookstore/product.asp?isbn=1587050234

unified computing system (ucs) 2.0 - new hardware...

Documents